Family Practice Vol. 17, No. 2, 173-179
© Oxford University Press 2000
Significant event audit in practice: a preliminary study
East Street Surgery, South Molton, Exeter EX36 3BU,
a 4 Parkfield Way, Topsham, Exeter EX3 0DP and
b Wyndham House Surgery, Silverton, Exeter EX5 4HZ, UK.
Westcott R, Sweeney G and Stead J. Significant event audit in practice: a preliminary study. Family Practice 2000; 17: 173–179.
Received 29 March 1999; Revised 15 October 1999; Accepted 26 October 1999.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResults and discussionConclusionsReferences |
|---|
Background. While well described and promoted as a useful activity, there remains a paucity of evidence on the process and experience of significant event audit (SEA) in primary care. To date, the most comprehensive evaluation of the process has been produced by comparing SEA with conventional audit. The current study intends to contribute to the debate by examining the attitudes and perceptions of a range of primary care staff who have been involved in the process.
Objectives. The aim of this study was to identify participants' perceptions of the benefits and problems associated with SEA in the context of primary care, and to derive suggestions which might improve the process of SEA.
Methods. Semi-structured interviews of 12 participants from a variety of primary care disciplines were conducted, using grounded theory to analyse the results.
Results. A set of six perceptions and seven recommendations for the facilitation of SEA were produced.
Conclusions. SEA constitutes a powerful tool, which can contribute to team building, enhanced communication and improved patient care, and represents a vital contributor to the development of clinical governance in primary care. However, its implementation and sustenance require sensitive handling for optimal benefit and to minimize difficulties. Our research has enabled us to propose suggestions to facilitate these processes.
Keywords. Clinical governance, leadership, quality improvement, significant event audit, team building.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResults and discussionConclusionsReferences |
|---|
Significant event audit (SEA) has been defined as occurring when "... individual cases in which there has been a significant occurrence (not necessarily involving an undesirable outcome for the patient) are analysed in a systematic and detailed way to ascertain what can be learnt about the overall quality of care and to indicate changes that might lead to future improvements".1
The technique of SEA has been well described.1–3 However, while a small number of research-based explorations of SEA in primary care have been undertaken,4,5 and there is evidence of the technique producing follow through into needs assessment and commissioning,6 the actual group process and the individual experience remain poorly described. Whilst there is some empirical evidence to support the particular model of SEA that has been advocated and widely adopted, few efforts have been made to ascertain or evaluate:
- the benefits to the participants involved in SEA
- the extent to which it meets (or fails to meet) the needs of participants
- the personal and professional risks that SEA might hold for individuals
- possible strategies and techniques to facilitate the process of SEA
- the contribution of SEA to team building, individual professional development and quality assurance.
In short, despite optimism about SEA, we have identified relatively little evidence describing the benefits, problems and other experiences of this process in primary care.
In this preliminary study, we set about exploring the use of SEA within primary care teams. Specifically, we aimed to: (i) describe and document individual team members' perceptions of SEA and (ii) on the basis of these perceptions, develop suggestions to improve the process of SEA.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResults and discussionConclusionsReferences |
|---|
The present study is placed firmly within the naturalistic paradigm, making use of the research interview as the strategy for collecting data, and of grounded theory as the method of data analysis. Grounded theory has two meanings.7 First it involves the notion of grounding theory in experiences, accounts and local contexts. The aim of the approach is seen as the generation of a ‘meaningful account' which accurately represents the complex nature of the participants' world. Second, the term ‘grounded theory' is used to describe a particular method which involves specific analytical strategies formulated for handling and making sense of ill-structured qualitative data. Strategies within grounded theory are presented to the researcher as ‘aids to analysis' rather than as methodological prescriptions. Glaser and Strauss8 describe the systematic development of an open-ended indexing system, in which the analyst works rigorously through the raw data to generate codes that refer to both low-level concepts and more abstract categories.
Data collection
Data were collected through a series of 12 one-to-one interviews, each of which lasted between 30 and 45 min. Absolute confidentiality was assured. Pools of potential participants were drawn from established SEA groups. Participants were recruited by approaching the practice manager in each of three geographically convenient primary care sites, who then provided a list of staff who were willing to participate. As is common in qualitative research, participants for this study were recruited on the basis of ‘theoretical sampling,7,8 in so far as sampling was data based and driven purely by theoretical concerns. As such, subjects were selected to provide a representative from each of the occupational groups within the primary care group, and to represent the wider community team. With the participant's consent, each interview was tape-recorded. The interview schedule was designed to elicit descriptive and explanatory information representing the interviewee's interpretation of the experience of SEA. Following an initial structured section (personal and situational variables and details about the practicalities of SEA), the interview was designed to be relatively unstructured.
Participant profile
In view of the theoretical sampling procedures employed, study participants were divided into two main groups; practice-employed staff (two practice managers, two GPs, two practice nurses and two receptionists) and community staff who played an active role in practice life (one district nurse, one health visitor, one community psychiatric nurse and one community physiotherapist). The aim of this sampling procedure was to provide a voice to individuals within the practice and to those who sometimes were viewed as outsiders. Whilst we were interested in what individuals had to say, we were equally interested in looking for themes and patterns within groups of data. Ages ranged from 37 to 58 years (mean 46 years). Eight subjects were female and four were male. Only three participants worked full time, the remainder working between 12 and 30 h per week. Their length of time in their post ranged from 2 to 18 years (mean 8.6 years).
Analysis of data
All qualitative research methods acknowledge the pivotal role of the researcher in shaping the study. The role is not hidden, passive or impersonal; the researcher and the researched are interdependent. This type of work also recognizes and acknowledges the biases that the researcher may bring to the study. In the current study, all data collection and data analyses were conducted by one of the authors (GS), who comes from a clinical (but non-medical) and academic background, and who entered the study with a high degree of scepticism about the process of SEA. The remaining two authors (RW and JS), hailing from medical backgrounds, were enthusiastic exponents of this form of audit, but recognized the need for more evidence to confirm earlier work by Pringle and others.
Initial indexing (‘coding’) proceeded by means of a tentative labelling of the phenomena which the analyst perceived in a specified piece of text and which was considered to be of potential relevance. The method employed to construct an indexing system was to examine each unit of text for analysis in turn, and ask "what categories, concepts, or labels do I need in order to account for the phenomena of importance in this paragraph?"9 It should be noted that within grounded theory, the aim is not to record all the reoccurrences of a phenomenon, but to collect a set of indicators describing the multiple facets of the concept. Initial coding resulted in the generation of 25 separate concept cards. At this stage in the analytical process, it became apparent that many of the concepts were in fact describing different aspects of the same phenomena, and core analysis commenced. Separate concept cards were clustered under 13 new headings, highlighting the most salient issues identified by the participants. These 13 ‘higher order' categories were then examined to explore further the underlying characteristics of each category. Saturation of categories occurred when any further analysis of this data failed to produce any additional examples or counter-examples of the interviewee's experience of SEA. Whilst all analysis was conducted by one individual, verification of emerging categories and theories was performed at monthly meetings amongst the three authors. ‘Disconfirming evidence' (arising from the introduction of ‘negative cases' to the study) has been amalgamated with the overall findings, as the aim of this type of analysis has been to generate a rich and dense theory, and to challenge initial assumptions.
|
Results and discussion |
|---|
|
TopAbstractIntroductionMethodsResults and discussionConclusionsReferences |
|---|
Table 1
presents the distilled themes as described above. We now address each category in turn.
|
Perceptions of SEA by participants
Advantages.. Participants welcomed the opportunity, indeed permission, that SEA gave them to come forward with both problems and difficulties as well as suggestions and advice. As in the study by Pringle et al.,2 these individuals could relate such discussions to team building: the creation of trust, mutual understanding and appreciation of other members' contributions. As a result, the work environment was enhanced and a better quality service could be offered, which itself further improved morale. They enjoyed the multidisciplinary format, offering opportunities to learn about others' experiences and opinions, to work together to problem solve and to resolve conflict, often without personalization. Members could see how SEA encouraged individual development both within and alongside the team.
"I think they [SEA meetings] could benefit the running of the practice, initiate improvements in quality and care of patients, and also perhaps improve relationships between staff, increase awareness of each others' roles. I think they present a challenge, make people stop and think rather than just jogging along and also perhaps make people feel more part of the whole team, so everyone's got a chance to voice an opinion". (Practice Nurse)
Disadvantages.. Pringle and his colleagues1 suggested that SEA may have a number of potential drawbacks, including its superficial, threatening and emotionally demanding nature. In addition, they posited the view there may be difficulties in structuring the meetings, that some issues may be difficult to resolve and that there may be a need for extra training of staff. In the current study, some members, particularly those employed by the doctors and lower in the hierarchy, felt vulnerable in speaking out, especially if their contribution might be seen as critical of those perceived to be of higher status. SEA represented a new and often uncomfortable experience for most of the team (in general least so for the GPs), who found the critical process generally disconcerting and could be embarrassed by revelations of other members' shortcomings. There was a Pandora's box fear: that lifting the lid might release uncontainable pressures with unexpected consequences.
Finding the time for the meeting, especially when part-time staff had to make special arrangements to attend, posed practical difficulties. Even when this could be resolved, these members often found they had to carry straight on at work, or to go home, without adequate opportunity to talk things over.
"... It's quite a short period of time for some sensitive issues. Especially when we've all got surgery and things to do afterwards". (GP)
Concerns.. Members were uncertain about boundaries and anxious not to overstep margins. While the traditional demarcated roles within primary care might (and many could understand ought) to be abandoned temporarily during the SEA meeting, it might be difficult to return to the ancien régime immediately afterwards—until next time.
Employed staff in particular worried about who was to lead SEA: without sufficiently strong and sympathetic chairing they felt vulnerable.
All staff feared the hornets' nest: it may be better to let some things be. Even when some difficult issues can be tackled, they may not be resolved adequately: "we still have to work together". As a result, SEA sometimes confined itself to safe areas, which were trivial.
Non-doctors were concerned that GPs' topics might dominate the agenda.
"... if we're going to start playing around with things that are quite emotional then I think you've got to have somebody who's skilled within that group to recognise it. So ... you've got to have somebody who's used to team work, knows how to start and end things and to intervene to keep everybody safe, because it's all very well us going round and doing these things, but if you're going to cause damage to the team because of it then that's not going to solve any problems, it's just going to create more". (Practice Nurse)
Conflicts.. Employed staff felt a conflict in roles: they found it difficult to behave as equals during the meeting and then to return to ‘employee status’. If the GP is your employer, it can be hard to speak out honestly. Other, externally employed, staff were differently stressed, with loyalties to the GPs and their own management structure sometimes in conflict. Some members (particularly nurses) felt their first loyalty was to their own staff group or discipline, which could interfere with SEA.
There could be a conflict between personal and professional matters for various participants.
"I know that we've been told how the meetings should work, that everyone should be completely frank, and it's confidential and whatever happens in the room is only for that particular time, and I think that, in all honesty, most of us feel that if we were to be picky with somebody in there, it would actually be very difficult to actually come out and work with them ... because of that, we have probably not been freely discussing the things that we should be". (Receptionist)
Motivation.. The selection of topics affected motivation. Because the leaders were more inclined to choose events that involved them, clinical GP topics could dominate SEA, particularly at first. This could alienate non-clinical staff.
Certainly some members felt more motivated to contribute than others. GPs' motivation was increased by PGEA approval for attending (with no similar reward for other groups) and a social benefit. Others with less need for the social contacts felt less motivated. These might be receptionists, who were sometimes able to talk about their work problems as they worked, and so had less need for SEA.
Because the initiative for SEA often came from the GPs, their motivation to attend, contribute and make the meetings successful was greater than that of other members, who occasionally saw the exercise as imposed. There was a danger of a vicious circle developing, with those attending dominating the proceedings and thus making SEA even less attractive for the non-attenders. It was particularly important for ‘external’ attached workers (community psychiatric nurses for example) to see relevant topics to motivate them to attend.
However, it was appreciated that the discussion of an event by those involved was a motivating experience and this encouraged people to attend the next time.
"The last couple I didn't bother [attending], because I was so busy and I didn't really, I haven't anything to give and yes, what was being said by other people didn't relate to me.... I only do 18 hours, and [I'm] quite pushed, so in that hour really I could have treated three patients". (Physiotherapist)
Solutions and resolutions.. The ability of SEA to solve problems and resolve awkward issues was generally appreciated. Guidelines for managing a variety of different situations could be discussed and recommended. People could be reassured they were ‘on track’ and an agreed direction of travel established, with the opportunity for everyone present to contribute. However, as recognized by Pringle et al.,1 our participants acknowledged that the short time available could mean that quick, easy or superficial solutions might be adopted. Members also recognized that it was sometimes difficult to be sufficiently honest.
"I'm not saying that people are shut up, you know, but all you can do is register that you're upset about it and what could be done to stop that problem occurring again. I don't see it as a psychologically supportive arena other than on a fairly superficial level really ... I think most of them are handled reasonably well, I think perhaps some people might think that they didn't have quite enough time". (Health Visitor)
Facilitating SEA
Pringle and his colleagues1 highlighted a number of important points in relation to the facilitation of the process of SEA. They state, "that successful audit requires a practice leader on audit (not necessarily a doctor); a supportive practice team; an enthusiastic practice manager; trust within the team; re-audit at intervals; explicit agreed standards; protocol development; feedback; and appropriate resources (specifically time, money and staff) ... the need to implement change as a consequence of the audit process".13 We would agree with these points and have identified some additional issues.
Rules and guidelines.. Participants need a clear set of rules, particularly at the outset. Specifically, managing emotional issues and those affecting absent team members requires agreed ground rules. With an initiative as new and as powerful as SEA, workers feel the need for a defined structure which can provide limits for protection, and safety.
As SEA evolves and participants become more comfortable, the structure can change appropriately, with, if necessary, new rules. Our experience has shown that in any event rules need to be revisited and reiterated regularly. Some participants are prepared to take a more pragmatic approach and develop rules with the activity: these are a minority and must be sensitive to the needs of the (possibly less outspoken) majority.
"it was a forum for disclosure really ... It was crucial that people felt that they could say ... not only how good everything was, but ... ‘it's ok to criticise.’" (GP)
Ownership and commitment.. Commitment can only be assumed if the participants' need for safety (which the ground rules will have identified) has been addressed. The team has to take responsibility for making it practicable for its members to attend. Full-time workers may find it easier to commit themselves than part-time, job-sharing and lower paid workers who can be disadvantaged in this respect. This uneven playing field has to be recognized. If the team can address the particular needs of various individuals, providing appropriate cover or in lieu off-duty arrangements, loyalty to SEA meetings can be won.
The evolution of Practice Professional Development Plans (PPDPs)10 can contribute to formal acknowledgement of SEA for all participants, which will help build corporate ownership.
Commitment also requires judicious selection of topics to ensure continued interest, especially in the early stages (see below).
"I would (come) if it was relevant ... every other Friday is my day off, we get a lot of other meetings, I've got one tomorrow on my day off, and I've got to come into that. So I don't make a habit of coming in unless I feel it's really important ...". (Practice Nurse)
Selection of topics.. The needs of all team members must be borne in mind when the SEA agenda is drawn up. If one or more individuals can see little relevance in the topics, it is to be expected that their support will wane. In addition, a sensitive choice of topics is crucial in order to ensure generalizability: the risk of issues becoming personalized rather than universalized has to be remembered. Topics which can stimulate the group but which have the potential for resolution which can be dealt with safely are essential.
Other problems with topic selection include the need for awareness of a hot topic becoming cold, items suggested only because no one else has thought of any (‘scraping the barrel’) and hidden agendas by one or more individuals. The important issue of individuals' rights to remove proposed topics is addressed separately (see below).
Successful agenda construction (and appropriate modification as the meeting proceeds) is a vital determinant of SEA success and dependent on quality leadership (see below).
"I think it's got to be handled very carefully, and if you've got to raise issues that might make people feel uncomfortable, I think they need to be aware of that before they get involved ... I think you've got to have a lot of preparation, rather than just lurching into it without agreement". (Practice Nurse)
Leadership.. Skilful leadership is essential to give confidence and facilitate all these items. Less assertive team members depend upon effective leadership to overcome their hesitancy. The qualities expected from the SEA leadership are extensive, ranging from support and encouragement (before, during and after the actual meetings) (see Management of the process below) through chairing to challenging, summarizing, planning and debriefing. This set of qualities is probably not available in any one individual in a practice team, and certainly could seem to rule out rotating the leadership between several people, desirable as this may be.
However, the requirements for leadership are clear and have to be recognized. Once identified, they may be met by the team exploiting its own skill mix, a team building activity which is both contributory to and helped by the process of SEA.
Ensuring a corporate responsibility for leadership is also needed in order to counteract the perennial problem of inequality, both perceived and actual, in the primary care team. Some suggested that the best placed individual, regardless of personal attributes, to co-ordinate if not provide most of this input is the practice manager. Certainly we were made aware of the difficulties flowing from the often automatic assumption of doctor leadership. Pringle et al.1 also identified leadership as a potential problem in the process of SEA, by stating, "leadership is a difficult issue for a general practice team which is said to be non-hierarchical and yet is often led, or apparently led, by doctors or even an individual doctor."
We encountered gender issues here: participants suggested that female approaches to the challenge of leadership might in general represent a more appropriate model, particularly with regard to ongoing care (see Debriefing below).
"You need positive leadership to put things back in boxes, so people aren't walking away feeling fragile or unresolved or bruised". (Practice Nurse)
Debriefing.. There is an under-recognized need to offer time and support to individuals or groups after SEA meetings. While the ideal is to address and complete all outstanding issues (particularly emotional) within the meeting, it has to be acknowledged that participants are left with unresolved matters which may be carried on into their next clinical session which might follow immediately.
Just as individual learning requires a period of reflection, so too do groups: SEA meetings need to devote attention to this need. Whether individual debriefing should be the responsibility of the leader, the practice manager, a defined group or the whole team, and when and how it is addressed, is less important than the recognition of its need.
"There is a danger and there is no mechanism for checking out with people at the end of it that they're not really upset or pissed off or whatever ... Maybe if just a few of us got together afterwards, you know, people who were involved, specifically involved. But we're so bad at dealing with that sort of stuff. Nobody is used to it really". (GP)
Censoring and vetting.. We would agree with previous assertions1 that some topics are inappropriate for SEA. We have learnt that individual poor performance needs very sensitive and intimate handling which should not be addressed in an SEA forum. Less professional but no less personal matters (e.g. personal hygiene) are also obviously better dealt with elsewhere. Important as they may be, and indeed needful of attention, there will be confidential (staff health for example), contractual, sex discrimination and other issues which, if the individuals involved do not have the right to strike them from the agenda, can cause severe personal stress and disruption to SEA, if not the practice itself.
"I would talk to those people involved away from this setting and if I needed to take the issue to the partners, then I'd deal with it in a more sensitive manner, a private manner..." (Practice Manager)
Management of the process.. The whole process of SEA then requires ongoing supervision. The meetings themselves are but part of a continuing developmental spiral, characterized by discussions and preparation leading to SEA which produces actions, review, reflection and reinforcement with perhaps new approaches. All these require active management—management of the practice itself, management of people and management of change. A delicate balance has to be struck between the provision of safety and support, and the stimulation of challenge to improve and ensure quality in all its aspects —a dynamic task, which itself will evolve. Therefore, there has to be adequate opportunity for the team to reflect upon the pace of change, the methods chosen and the progress achieved, and especially the failures—where SEA may be used to audit the process of SEA.
Those implementing SEA must present practices with the chance to review, as part of the process. This could be addressed through dedicating a meeting at regular intervals to stocktaking, or to join with one or more other practices to share experiences, or to contribute to a workshop at Authority level. Teams may or may not want to use their own resources with or without imported experts. Possibly the evolving role of clinical governance might facilitate such developments, with its emphasis on joint working at inter- and supra-practice level.11
"I think the chairmanship of a particularly negative incident has to be handled very carefully, very carefully, because if they're going to be cosy chats over a cup of tea, you might as well just pack it in. If they are going to be significant then you might as well make them flipping significant really, and that, I think, takes some skill in the chairmanship. You've got to listen very hard as to how stressed individuals are and you have to be able to allocate responsibility without allocating blame in a personal sense. It sounds very simple, but I think that's a little elusive". (GP)
|
Conclusions |
|---|
|
TopAbstractIntroductionMethodsResults and discussionConclusionsReferences |
|---|
This study builds on the earlier work by Pringle and others. By using a different methodology and a different sample, we successfully have confirmed findings of previous research, and extended our knowledge of the subject. Many of the benefits of SEA, such as its ability to stimulate clinical audit and needs assessment, to inform commissioning and improve quality, have been well documented.1,2,6 In terms of the process, it can be seen that SEA represents a powerful team building experience for primary care team members, capable of involving all members in a multidisciplinary approach and creating better morale. All members can appreciate the individual benefits along with the better communication, improved mutual understanding, happier work environment and the resulting enhanced patient care. These are important contributions to the well functioning primary care team demanded by today's National Health Service.10,11
However, there are substantial difficulties, not previously documented, which not only can prevent the successful implementation of SEA but which can alienate individuals and cause damage to teams. Members fear exposure, find it difficult to step out of role, worry about causing offence (especially to GPs who may be their employers) and need sensitive encouragement based upon an awareness of these various anxieties. However, by establishing clear rules, ensuring general ownership, carefully selecting the right topics and using good leadership skills, allowing for proper support and protecting individuals, SEA may be implemented successfully.
This qualitative study has shown that teams and their members have derived many personal, professional and corporate benefits from SEA. The new emphasis upon clinical governance11 requires structures and processes which can create and sustain a framework for individual professionals and teams to respond to the new agenda for quality improvement. Our research leaves us in no doubt that SEA represents a crucially important tool for primary care teams. As with all new tools, both training and support are needed to ensure optimal use and indeed prevent damage. However, by following our suggestions for facilitating the process, we believe teams will be able to introduce and maintain SEA as a powerful opportunity to strengthen teamwork, enhance quality and improve patient care, its benefits far outweighing its disadvantages.
|
Acknowledgments |
|---|
We are grateful to our colleague Liz Cosford and to Professor Mike Pringle for their helpful comments on earlier drafts of this manuscript. We record with thanks funding from the Somerset and North Devon Education Purchasing Consortium, and acknowledge the support of Wyndham House Surgery, Silverton, an NHS Research and Development-funded practice.
|
References |
|---|
|
TopAbstractIntroductionMethodsResults and discussionConclusionsReferences |
|---|
1 Pringle M, Bradley CP, Carmichael CM, Wallis H, Moore A. Significant Event Auditing. Occasional Paper 70. Exeter: Royal College of General Practitioners, 1995.
2 Pringle M, Bradley CP. Significant event auditing: a user's guide. Audit Trends 1994; 2: 70–73.
3
Robinson LA, Stacy R, Spencer JA, Bhopal RS. Use of facilitated case discussions for significant event auditing. Br Med J 1995; 311: 315–318.
4 Benett IJ, Danczak AF. Terminal care: improving teamwork in primary care using significant event analysis. Eur J Cancer Care 1994; 3: 54–57.
5
Berlin A, Spencer JA, Bhopal RS, Zwanenberg van TD. Audits of deaths in general practice: pilot study of the critical incident technique. Qual Health Care 1992; 1: 231–235.
6
Pringle M. Preventing ischaemic heart disease in one general practice: from one patient, through clinical audit, needs assessment, and commissioning into quality improvement. Br Med J 1998; 317: 1120–1124.
7 Strauss AL, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage Publications Ltd, 1990.
8 Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine, 1967.
9 Turner BA. Some practical aspects of qualitative data analysis: one way of organising the cognitive processes associated with the generation of grounded theory. Qual Quant 1981; 15: 225–247.[ISI]
10 Chief Medical Officer. A Review of Continuing Professional Development in General Practice. London: Department of Health, 1998.
11 Department of Health. A First Class Service. London: Department of Health, 1998.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||