Family Practice Vol. 18, No. 2, 123-130
© Oxford University Press 2001
Supporting GPs whose performance gives cause for concern: the North Trent experience
Institute of General Practice and Primary Care, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK.
Dr Helen Joesbury, Institute of General Practice and Primary Care, Community Sciences Centre, Northern General Hospital, Herries Road, Sheffield S5 7AU, UK.
Joesbury H, Mathers N and Lane P. Supporting GPs whose performance gives cause for concern: the North Trent experience. Family Practice 2001; 18: 123–130.
Received 2 February 2000; Revised 15 August 2000; Accepted 30 October 2000.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background. The North Trent scheme to address the problem of GPs whose performance gives cause for concern was implemented in 1997. This paper describes the structure and process of the scheme and evaluates the main outcomes.
Methods. We used non-participant observation and semi-structured interviews with representatives of the seven Health Authorities (HAs) of North Trent including medical and prescribing advisors and senior primary care managers. Twenty-one GPs who were members of the Performance Review Quartets (PRQs) were also interviewed. Qualitative data were analysed using a constant comparative method to identify emergent themes.
Results. Performance indicators were agreed between HAs and the profession in the seven North Trent localities. The scheme identified 18 GPs whose performance gave cause for concern, of whom 15 GPs in six practices received a formal visit. Educational plans were agreed and implemented with three GPs. The remaining 12 received administrative and clinical support. Three of the 18 GPs initially refused to co-operate with the scheme. Two of these have since agreed a practice visit following a visit by a senior local medical committee representative. The performance indicators used in the scheme have not been specific to individual GPs except those in single-handed practices. Some indicators used by PRQs related to cost effectiveness rather than quality of care for individual patients. Current resources were adequate for the small number of underperforming GPs identified by the scheme.
Conclusions. The North Trent scheme has identified a number of underperforming GPs, 83% of whom have been willing to participate in a supportive intervention. The scheme will need some modification with the advent of primary care trusts and the proposed assessment and support centres.
Keywords. Clinical governance, continuing professional development, GPs, health authorities, underperformance.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The authors have taken great care in the following paper to ensure the confidentiality of the respondents and avoid the identification of the Health Authorities, Performance Review Quartet and individual GPs who have taken part in this study.
The recent consultation document Supporting Doctors, Protecting Patients published by the Department of Health is concerned with the prevention, recognition and management of poor performance by NHS doctors.1 It states that the present procedures to deal with poor clinical performance need to be modernized and calls for pilot schemes aimed at detecting the poor clinical performance of doctors to be evaluated. Although guidance on ‘Measures to assist GPs whose performance gives cause for concern’ were published in August 1997,2 many health authorities (HAs) have not had a well-defined structure or process in place for supporting GPs at the ‘trailing edge’ of acceptable performance.
This paper describes the structure, process and outcomes of the North Trent pilot scheme developed jointly by members of the profession and the HAs to provide proactive support to these underperforming doctors during 1997–1999.
It provides important recommendations for primary care groups who now have responsibility for clinical governance and has implications for the proposed Assessment and Support Centres.
Evaluation data for the pilot scheme are presented in the form of quotes from various stakeholders in the process and the authors' observations of the process of implementation.
Principles of the North Trent scheme
The components of the North Trent scheme were derived originally from the guidance developed for the Department of Health.2 In summary, they were:
- A framework of principles for the support of GPs whose performance gives cause for concern was established between local professional representatives and the local HA.
- GP underperformance was defined as an overall pattern of underperformance which encompassed the clinical, organizational and management components of the care of patients.
- Underperforming GPs were identified using agreed performance indicators based on minimal acceptable standards of performance (Table 1
).
- The structure and process of the scheme were chosen to be, as far as possible, transparent, accountable and fair within the constraints of confidentiality.
|
The scheme was based on a group of four practising GPs representing the local medical committee (LMC), continuing medical education (CME), the medical audit advisory group (MAAG) and the medical advisor to the HA who provided the link between the HA and the profession locally. The group was referred to as the Performance Review Quartet (PRQ) and its role was to work closely with a senior manager of the HA to review information on the performance indicators for practices, discuss areas of concern and recommend appropriate action in a process which remained confidential to an individual GP and the PRQ.
Outline of the scheme (Fig. 1)
It was intended that under the scheme, the HA identified a doctor whose performance gave cause for concern using the performance indicators, and the PRQ was then asked to review the evidence. The PRQ could then dismiss the evidence or contact the doctor concerned for further investigation. In the latter case, a practice visit by a member of the PRQ was arranged at which a ‘friend’ of the GP could be present. A full assessment of the doctor's professional activity was to be made and an educational plan agreed with clear objectives to be achieved within a specified time scale. Reassessment was then to take place after an agreed interval.
|
The North Trent scheme was launched in September 1997 following the introduction of the Department of Health Guidance to Health Authorities and the distribution of the outline proposals for a local procedure from the Postgraduate Director of General Practice Education (PL).
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
All the primary care leads of the HAs in North Trent were contacted by the research team and asked, in the light of the guidance, what arrangements they had made for supporting underperforming GPs in their locality, and if they would be willing to discuss this with a member of the research team. A semi-structured interview subsequently was carried out with the primary care leads in each HA, who was usually the medical advisor or a senior HA manager. This interview asked for details of the local structure and process which each HA proposed in order to meet their statutory obligations in this area. In this way, contact was made with members of the PRQs in each locality.
PRQ members were asked if the meeting and discussions of the PRQs could be observed and recorded. Individual members were also interviewed as the North Trent scheme was being implemented. In total, 12 HA staff (including medical advisors) and 22 members of the PRQs were interviewed or observed. No GPs whose performance gave cause for concern were interviewed for this study.
Field notes were made during each interview/meeting and, immediately after data collection, were expanded into full transcriptions, copies of which were sent to the individual stakeholders for correction and comment prior to analysis. These agreed transcriptions were then analysed using a constant comparative method, and emergent themes were identified independently by two members of the research team (HJ and NM).
The evaluation took place over 18 months from October 1997 to March 1999.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Implementation of the scheme
The first task of each PRQ was to develop performance indicators, which were both valid and reliable.
PRQs spent considerable time discussing the relative merits of such indicators. One group only agreed a final list after almost a year of discussion, although three had provisional lists in place within 3 months of starting the scheme. All PRQs had difficulty in agreeing a threshold for intervention. However, by scoring practices across a range of indicators, a pattern emerged which suggested a general level of underperformance relative to other local practices. This is illustrated in Figure 2 where it can be seen that in this HA, three practices stand out as having difficulty in meeting more than six of the performance indicators.
|
The number of indicators finally agreed ranged from 21 to 34. It was accepted that all practices were likely to score poorly on a small number of indicators.
The indicators which were used reflected different areas of practice activity and included:
- Demographic data on the practice population, list size and morbidity data to reflect workload.
- PACT data, referral patterns and items of service claims to give some insight into clinical behaviour.
- Organizational effectiveness may be revealed by geographical scatter of patients and speed of return of records at the request of the HA.
- Educational priorities were deduced from audit activity, staff training and PGEA uptake.
- Patient satisfaction was reflected in patient removals without a change of address or numbers of complaints.
It became immediately apparent that the indicators used within the scheme, with the exception of PGEA claims and possibly complaints against an individual doctor, were entirely practice based and only reflected individual behaviour in the case of single-handed practitioners.
"A pattern in the jigsaw begins to emerge showing the same doctors." (CME tutor)
Emergent themes
The following themes of diagnosis, intervention, confidentiality, PRQ roles and obstacles to implementation of the scheme were all identified by the authors from analysis of the qualitative data. Each theme is supported by illustrative quotes from key participants in the process and also includes the results of non-participant observation by the authors.
Diagnosis was a key issue for all the participants in the study.
"Indicators are only indicators. Discussion with the practice is the final test." (PRQ member)
Agreeing on the structure of a practice visit was a controversial issue in the development of the scheme and generated the most uncertainty among the PRQ members. There was, however, considerable exchange of ideas across the region about the mode of conduct of the visit and the appropriateness of areas to be covered.
Although formal arrangements for contacting GPs were agreed by all the PRQs, approaches to half of the GPs were opportunistic and informal in the light of the sensitive nature of the issue. Most PRQ members felt that the objective of gaining access to the practice and beginning a dialogue with the GP was, in practice, more likely to be achieved in this way.
One PRQ decided to ask the GP to complete a self-assessment form prior to the visit, partly to gain information from it, but also to encourage awareness in the GP of the areas that would be investigated.
"The PRQ has no right of entry. Its brief is ‘professional risk management’ not policing." (Medical advisor)
Three GPs refused to recognize the authority of the PRQ and initially declined a practice visit.
Two PRQ members, but not the medical advisor, who represented the HA, normally conducted the practice visits which were intended to be modelled on visits to training practices. Two PRQs used ‘Good Medical Practice’ (GMC)3 as the basis for their enquiries. It soon became apparent that minimal standards were more difficult to define than high standards.
"What is the proportion of doctors lying between the very bad and those at the leading edge? What is the bottom line? Should it continually rise? How many of the middle range doctors should be offered help?" (CME tutor)
GPs were given the opportunity of have a professional friend (e.g. from the LMC) present at the practice visit, but this offer was accepted in only a minority of cases.
Initially, only the specific areas of concern were discussed, but invariably many other professional and personal issues that affect a GP's work became apparent, and visits scheduled for an hour regularly lasted much longer.
Information was gathered by many means including interviewing the doctor(s) concerned, touring the premises, talking to staff and reviewing records.
"The meeting will be friendly and supportive but it will be clear that the PRQ must have a responsibility to maintain standards." (CME tutor)
The GPs were expected to offer their own explanations of their difficulties and to provide feedback to the PRQ about the causes of and solutions to the problems identified.
The interview was concluded by offers of support, educational recommendations and practical assistance appropriate to the needs of the GP. A time scale for achieving objectives was recommended. One of the PRQ members (usually the CME tutor) would normally offer to become a mentor for the doctors of the practice.
A further major theme identified from the transcripts was disscussion by the PRQ on the nature and implementation of an intervention.
It was a principle of the scheme that interventions were to be relevant to the needs of the doctor concerned and appropriate to their preferred style of learning. The mentor was responsible for helping the doctors to meet their own educational objectives.
"We need to enable practices to recognize when they are losing control . . . and including staff retraining and patient education; a package to enable them to manage clinical practice." (Medical advisor)
In one HA, the use of two salaried doctors who had been recently trained but not yet ready to settle into partnership proved successful. On a 6-month secondment, they were able to introduce new ideas, both clinical and organizational, to the doctor without being a threat to his or her seniority. Their terms of employment needed to be clearly defined to avoid relieving the GP of his or her normal duties.
It became apparent that HAs were developing access to a pool of practice nurses and practice managers who were willing to be seconded to give practical and educational assistance in practices where needed.
"We have a pharmacist working in a practice under the HA support project, to help them manage their budget. He enquires ‘What can I do about their dodgy prescribing?’ The medical advisor is invited to meet them and give diplomatic feedback. They were leaving 20 prescriptions a day for unseen patients and did not realize that not all doctors, even with a heavy workload, behaved like that." (Medical advisor)
The development of portfolio learning packages in North Trent had already encouraged innovative styles of learning so, for example, there were few difficulties in arranging consultant attachments in out-patients.
Confidentiality was an important theme identified by all participants in the scheme. In the North Trent scheme, all of the information which had been passed to the PRQ from the HA was routinely available to HA staff within each section, but only the responsible senior manager had access to all the data. When evidence of an individual GP's performance was reviewed by the PRQ, most members were aware of the identity of the doctor under discussion. If further action was not taken, the PRQs decided not to inform an individual GP of their deliberations. The records pertaining to the discussion of an individual GP remained the property of the PRQ and have been kept in a secure facility, sometimes the LMC office. If a GP's performance is called into question again, they can be used as evidence that appropriate procedures have been followed.
"Should an official record be kept if there is no case to answer? Yes, but lodged with the LMC (who else can you trust?) not the Health Authority. If there is trouble later on, a confidential inquiry report is evidence that the doctor was found to be all right before. It may protect him later from the GMC." (Medical advisor)
Confidentiality on a practice visit was more difficult to maintain as it was sometimes necessary to involve practice staff in the investigation of the GP's performance.
The views of the individual PRQ members in their new roles was a new issue identified by the qualitative data analysis. None of the PRQ members had received formal training for their role but most had a background in CME and two had been on a mentoring course. All were well-established and experienced GPs in active practice.
However, all but two of the tutors described feeling unprepared to make the cultural shift to assessment and most were concerned that their new role would become an additional burden.
"There is a conflict of interest for the CME tutors. The mentor and assessor (roles) should be separate. You cannot be educationalist and policeman. I have had minimal training for underperforming doctors. I am used to empowering people who are there by choice." (CME tutor)
A number of PRQ members saw significant personal advantages in terms of their own professional development and self-esteem.
"There is inevitably an effect on your own practice. Sharing makes your job more interesting: you feel better about yourself. You can relate to your peers and compare standards. It reverses insularity. Groups support each other." (CME tutor)
Financial rewards were not an issue, and no doctor interviewed had claimed expenses for their PRQ work.
Obstacles to implementation of the scheme were raised by all the stakeholders in the process. Some HAs in the North Trent scheme were less willing than others to initiate the process to identify and support struggling GPs, and there were many reasons for this. For example, there was a sense amongst medical advisors of how difficult it can be to facilitate change in GPs.
"Odd personalities are difficult to change." (Medical advisor)
There was also a belief that there are no underperforming doctors within a particular district, which initially inhibited one PRQ who saw no need to look for any (sic); conversely, when a large number of underperforming GPs was predicted by one medical advisor, there was a reluctance to investigate in case taking action in these cases could cause the service ‘to collapse’.
HAs publicized the process either by writing to all the GPs in their areas or by holding local meetings. The reaction of GPs at meetings was variable. In the early stages, the GPs often were hostile, but as it has gradually become more accepted nationally that some doctors are underperforming, the provision of a locally supportive mechanism became increasingly acceptable.
"We were asked at the meeting ‘Who are you? What right do you have to do this?’" (CME tutor)
Single-handed doctors proved to be far more vulnerable to identification in the scheme because all the practice-based indicators refer to one individual. However, PRQs were mindful of this and were concerned to protect individuals from unfair criticism while maintaining a rigorous process for identification.
"It is biased against single-handed practices (who can be like chalk and cheese, some are into everything and are well in touch and others are isolated). Underperformance is a practice issue." (Medical advisor)
Outcomes of the scheme
Performance indicators were agreed between HAs and the profession in all seven North Trent localities, but some indicators used by PRQs related to cost effectiveness rather than quality of care for patients.
During the course of the study, 18 GPs were identified whose performance gave cause for concern and 15 GPs in six practices were visited. Three GPs declined assistance and were at risk of GMC referral. Two have been visited by senior LMC representatives who are not PRQ members, but have worked in nearby practices. They have persuaded the GPs to agree to a practice visit. The third has publicly denounced the scheme and has received further written information from the PRQ emphasizing the supportive, not punitive, nature of the scheme.
Practice visits to partnerships have included all the doctors in the assessment because the indicators, except PGEA and possibly complaints, reflect practice rather than individual performance.
Remedial plans with clearly defined educational objectives have been agreed and implemented with three GPs. Formal educational plans have not been drawn up for the remaining practices but they have been provided with administrative and clinical support. Three practices were reported during the study as being unaware that their performance was deficient until approached by the PRQ. It has not been possible within the duration of the project to reassess the indicators relating to a specific practice following an intervention.
Each HA made minor alterations to the outline scheme to suit local needs, but current resources were adequate for the small number of underperforming GPs identified.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
After 18 months, all seven HAs in North Trent had procedures in place for identifying and supporting underperforming GPs. Concerns that the numbers of doctors identified could lead to difficulties maintaining general medical services in some areas have proved unfounded. A survey of HAs in 19974 suggested that poorly performing GPs represent between 1 and 10% of GPs. The number of doctors in the HAs investigated in this study who deviated significantly from the mean supports the lower figure (i.e. 1–2%). Questioning HA staff about ‘soft’ information on practices appears to correlate well with the practices that scored badly on the chosen indicators, although further systematic evaluation is required. Asking local consultants to rate practices also showed some correlation. Conversely, on one occasion during this study, the indicators suggested underperformance in a practice where it was not anticipated.
The implementation of the scheme by each HA was smoother where the GPs who made up the PRQ had experience of working together and where relations between the HA and the LMC had previously been co-operative. In spite of having a representative on the PRQ, LMCs showed different levels of concern about the scheme, probably related to their previous relationship with the HA. When it was demonstrated that the emphasis of the scheme was not on ‘policing’ GPs as much as raising standards of patient care through education and support for ‘struggling’ GPs, the process was implemented with fewer obstacles. Similarly, the GPs in the areas where the scheme was publicized earlier had more concerns than those who were informed later, when the profession generally had become more accustomed to the importance of the scheme.
The process of developing indicators was time consuming and the PRQs exchanged information regularly with others throughout the region. However, they still preferred to develop their own rather than utilize those already published. The PRQs attempted to test indicators in different ways, the majority scoring all local practices to identify patterns, and others applying them to practices that were suspected of underperformance and comparing them with practices thought to be performing well. Scoring practices across a range of indicators enabled patterns of underperformance to emerge relative to other local practices. Poor performance in more than one area was usually found to be associated with poor performance in others.
The reliability and validity of the indicators may be questioned but in the North Trent scheme they were used solely as a screening tool. Diagnoses of underperformance were only made after formal practice visits. One HA asked local consultants to rank practices that had scored either very poorly or very highly on their range of indicators, and were satisfied with the degree of agreement which resulted. Another HA asked staff informally from different departments in the HA to rank practices, producing a graph very similar to that derived from the indicators, the correlation being greater where practices had frequent contact with the HA for any reason.
The performance indicators which had been developed and used in the North Trent scheme have the considerable advantage of being jointly agreed by the HAs and the profession locally.5 These are based mainly on information already held by HAs and have been used by the PQRs as a screening rather than a diagnostic tool. However, they clearly require more development in order to make them sensitive to the performance of individual practitioners within group practices and to ensure that single-handed practitioners are not identified unfairly. However, all of the practices identified with poor performance indicators were found to be in need of improvement, and single-handed practitioners, although more vulnerable to identification, have the same duty of high quality care as group practices. The false-positive rate of the scheme is low, but further investigation would be required to determine the false-negative rate. The most important limitation of performance indicators is that they measure only certain aspects of performance and tell us little about what most GPs would consider to be their most important role: the clinical care of individual patients.6 In addition, since the indicators have been constructed on data routinely collected, there may be errors in these data, although in the North Trent scheme some of the barriers in HAs to improving quality in general practice identified by Marshall have been overcome.7
The PRQs varied in their approach to the practice visit. Some felt that the ‘dignity’ of the GP was preserved by a formal visit by two PRQ members; others took a less formal line of enquiry allowing one PRQ member who knew the doctor best to make contact. There was no consensus on what form the practice visit should take and clearly the personal style of the visitor may influence the outcome. Two mentors felt more comfortable visiting a partnership than a single-handed practice, where misunderstanding and confrontation were thought to be more likely.
It is not clear how the scheme will develop with the introduction of primary care trusts (PCTs). Up to now, primary care groups (PCGs) have had responsibility for clinical governance with the HAs. The HAs in North Trent have all responded differently to recent changes, although in most the system so far has remained in place as a resource for PCGs. Elsewhere, responsibility has been transferred to the clinical governance leads of the PCGs who have yet to decide on their own arrangements. There is some concern that PCTs initially will lack the expertise to support underperforming GPs, and an inevitable preoccupation with organizational and financial matters may result in a more punitive response towards GPs whose performance indicators deviate from the norm.
PCT clinical governance leads will need clear guidance on how to deal with underperformance and where accountability lies. The expertise which already exists within HA and PCTs should be utilized by PCGs and the Assessment and Support Centres.
Practice visits by the PCT lead should take place and not be confined to discussion of the performance indicators but used as the basis for more detailed investigation of underperformance. A doctor with a clearly defined problem should be referred rapidly with his or her agreement to one of the proposed Assessment and Support Centres for impartial assessment. The new centres should also be willing to take on the PRQ role of self-referral whereby doctors could voluntarily ask for advice, help or support of their clinical performance.
If the outcome of referral was that the doctor could continue to practise but be monitored according to specified criteria, then the expertise of the GP tutors should be utilized for mentoring. Initially, the HA should retain overall responsibility for managing the support required, although as PCTs become established, this responsibility should be devolved to the Trusts.
Support for underperforming GPs in the form of additional resources should be flexible and tailored to individual practices. Confidentiality was a key issue in the implementation of the North Trent scheme, and details of individual GP performance should be kept on a ‘need to know’ basis with records stored in a secure place. It is important to emphasize that within any local scheme no stigma should be attached to referral and that support is available as appropriate for GPs who are struggling at the trailing edge of performance due to insufficient resources.
Jewell has called for a change of climate to enable existing systems (of professional self-regulation) to work properly.8 The implementation of the North Trent scheme has changed the climate amongst GPs and, despite the need for further development, is a model of professional self-regulation which could be used as a basis for clinical governance and the management of GP underperformance by PCTs.
"I can only see it as being a good thing; a very positive thing for the profession. Medicine has never been seen to put its house in order like other professions do. GPs particularly can say, ‘I can do almost anything.’ There has been no public accountability unless things get so bad they go before the GMC."
|
Acknowledgments |
|---|
The research team would like to thank the members of the Performance Review Quartets of Barnsley, Doncaster, North Derbyshire, North Lincolnshire, North Nottinghamshire, Rotherham and Sheffield for making themselves available for interview and for providing regular updates on their progress, and to Mr Chris Locke, Secretary of Nottingham LMC. Thanks also to Deborah Watkin and Sharon Hart for secretarial support. This study was funded by North Trent MADEL monies.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1 Department of Health. Supporting Doctors, Protecting Patients. London: Department of Health, 1999.
2 University of Sheffield. Measures to Assist GPs Whose Performance Gives Cause for Concern. Sheffield: University of Sheffield, 1997.
3 General Medical Council. Duties of a Doctor: Guidance from the General Medical Council. London: General Medical Council, 1997.
4
Taylor G. Underperforming doctors: a postal survey of the Northern Deanery. Br Med J 1998; 316: 1705–1708.
5 Old P, Voss S, Davidge M. Performing arts. Health Service J 1994; 104: 28–29.
6
Majeed FA, Voss S. Performing indicators for general practice. Br Med J 1995; 311: 209–210.
7
Marshall M. Improving quality in general practice: qualitative care study of barriers faced by health authorities. Br Med J 1999; 319: 164–167.
8 Jewell D. Supporting doctors or the beginning of the end for ‘self regulation’. Br J Gen Pract 2000; 50: 4–5.[Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. MacLeod The challenge of providing mentorship in primary care Postgrad. Med. J., May 1, 2007; 83(979): 317 - 319. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||