Family Practice Vol. 20, No. 2, 199-206
© Oxford University Press 2003
Education in Primary Care |
Do clinical practice education groups result in sustained change in GP prescribing?
Department of Public Health and General Practice, Christchurch School of Medicine, University of Otago, PO Box 4345, Christchurch, New Zealand.
Correspondence to Dr Dee Richards; E-mail: derelie.richards{at}chmeds.ac.nz
Richards D, Toop L and Graham P. Do clinical practice education groups result in sustained change in GP prescribing? Family Practice 2003; 20: 199–206.
Received 19 June 2002; Revised 7 October 2002; Accepted 4 November 2002.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background. Concern has been expressed at the poor uptake of evidence into clinical practice. This is despite the fact that continuing education is an embedded feature of quality assurance in general practice. There are a variety of clinical practice education methods available for dissemination of new evidence. Recent systematic reviews indicate that the effectiveness of these different strategies is extremely variable.
Objective. Our aim was to determine whether a peer-led small group education pilot programme used to promote rational GP prescribing is an effective tool in changing practice when added to prescribing audit and feedback, academic detailing and educational bulletins, and to determine whether any effect seen decays over time.
Methods. A retrospective analysis of a controlled trial of a small group education strategy with 24 month follow-up was carried out. The setting was an independent GPs association (IPA) of 230 GPs in the Christchurch New Zealand urban area. All intervention and control group GPs were already receiving prescribing audit and feedback, academic detailing and educational bulletins. The intervention group were the first 52 GPs to respond to an invitation to pilot the project. Two control groups were used, one group who joined the pilot later and a second group which included all other GPs in the IPA. The main outcome measures were targeted prescribing data for 12 months before and 24 months after each of four education sessions.
Results. An effect in the expected direction was seen in six of the eight key messages studied. This effect was statistically significant for five of the eight messages studied. The effect size varied between 7 and 40%. Where a positive effect was seen, the effect decayed with time but persisted to a significant level for 6–24 months of observation.
Conclusion. The results support a positive effect of the education strategy on prescribing behaviour in the intervention group for most outcomes measured. The effect seen is statistically significant, sustained and is in addition to any effect of the other pharmaceutical educational initiatives already undertaken by the IPA.
Keywords. Behaviour change, education, general practice, prescribing.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Concern has been expressed at the poor uptake of evidence into clinical practice.1 This is despite the fact that continuing education is an embedded feature of quality assurance in general practice. There are a variety of clinical practice education methods available for dissemination of new evidence.2 Recent systematic reviews indicate that the effectiveness of these different strategies is extremely variable.2–8 There are only a few studies of continuing medical education (CME) methods for GPs that evaluate the extent and duration of changes in clinical practice.2–4,6 An opportunity arose to study a pilot of a peer-led small group education programme introduced by a New Zealand Independent General Practitioners Association (the Pegasus IPA). Educational strategies already in place within the IPA included academic detailing (individual prescribing education visits to all GPs by a pharmaceutical facilitator) and audit and feedback—strategies both shown individually to be moderately effective in changing behaviour.2–5 Similar combination strategies have been found to be effective: audit and feedback combined with expert-panel discussion was effective in reducing unnecessary antibiotic use in a controlled trial in Sweden,9 while an Australian study with general practice trainees showed audit and feedback combined with printed guidelines and a targeted high prescriber outreach visit were successful in reducing antibiotic prescribing for upper respiratory tract infections (URTIs).10
The small group education programme was set up with the aim of promoting more rational prescribing. Areas of prescribing were identified where areas of wide variation existed and there was a significant gap between actual and ideal prescribing. There was an assumption that more rational prescribing would also be more cost-effective prescribing, although cost saving was a secondary aim of the education programme. In this pilot study, our first aim was to assess whether the small group education programme resulted in a change in clinical practice independent of these other educational activities. Our second aim was to assess how any change in clinical practice persisted or was extinguished over time.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The key aspect of the education strategy is clinical practice education groups (CPEGs) (Box 1
). These are groups of 
10 GPs who meet monthly to discuss topics related to clinical practice. The 52 GPs were divided into four groups in an ad hoc manner. Group membership was constant and members of the same practice were grouped together where possible. The intervention group was 19% female, 77% New Zealand trained, had been in practice for an average of 22.5 years since registering in New Zealand as a medical practitioner, and 24.0 years since qualification. Each group is led by an experienced GP. Discussion is based on evidence-based topic notes prepared for each leader as well as individual prescribing and laboratory data related to the topic that is provided to each GP. Although the education groups cover all aspects of clinical practice, messages relating to prescribing were chosen as key indicators, as prescribing is a measurable outcome indicator for which data are readily available.
|
The education strategy was piloted initially with four small groups. This pilot stage has provided a ‘natural experiment’—an opportunity to evaluate the effect of this education strategy with a naturally occurring control group. This first pilot group of 52 volunteer GPs began the education programme in February 1995. Other GPs who responded after the cut-off point were put on a waiting list to join. These 52 GPs made up the intervention group for this study. Those GPs who did not attend a particular meeting were excluded from the analysis for that topic, and those attending another groups’ meeting were included with that group for the data analysis over time for that particular topic. This pilot study covers the education sessions during the time when the groups were first established in 1995.
The key messages of the first four topics covered are shown in Table 1.
|
Prescribing rates in the groups receiving the education were compared with two control groups. Two comparison groups have been used in order to investigate the ‘volunteer’ effect. GPs were excluded from the control groups if they were in the same practice as a pilot member (to prevent any bias due to ‘diffusion’ effects), if they were not practising for the complete study period or if they were education pilot steering committee members.
Control group 1 comprised the next GPs who were on the waiting list for the programme (n = 25 after exclusion criteria applied). Control group 2 comprised all other GPs in the IPA (n = 83 after exclusion criteria applied). It was felt that other GPs from control group 1 were likely to be similar in characteristics to the intervention group if there were a volunteer effect. The control groups were compared to see if they behaved differently as control groups. Although all IPA GPs give permission for their prescription data to be analysed, GPs were not aware at the time of the intervention that their prescribing would be analysed for that particular period. This provided allocation concealment. The unit of analysis was at individual GP level.
Data on prescribing were obtained from Health Benefits Limited (NZ) claims data which records all prescriptions dispensed where a government subsidy is payable. HBL prescribing data relating to the identified outcomes were extracted for a period of 12 months before the intervention and then for continuous 3-month intervals after the education session for a period of 2 years. Since the pilot groups did not cover the same topics concurrently, control group data were extracted four times for each topic (in 3-month time periods) in order to replicate the follow-up period for each pilot group.
Statistical methods
In order to investigate the, possibly time-varying, effect of the intervention on prescribing behaviour, we employed logistic regression models in which the proportions of scripts written by a GP for a preferred drug within a class of drugs was modelled as a function of intervention status (intervention group, control group 1 or control group 2), quarter of follow-up, indicators for the actual time period, prescribing pattern in the year prior to the intervention and an index of the socio-economic characteristics of the area in which the practitioner’s practice was located.11
The impact of the intervention was allowed to vary with the quarter of follow-up by including quarter of follow-up by intervention group interaction terms in the model.
Initially, quarter-specific intervention effect parameters were included in the model, but we found that representing the time-varying intervention effects via restricted cubic spline functions12 gave rise to more plausible intervention effect time trends. Whereas the quarter-specific models required eight parameters to model the time trend in intervention effects, the restricted cubic spline models required only three parameters for the intervention effect time trend. Consequently, the results reported below are based on the cubic spline representation of the time-varying intervention effect. However, for all analyses, we found that results from the model with quarter-specific effect parameters fell within the 95% confidence limits for the effect estimates derived from the cubic spline model.
The aim of our modelling approach was to elucidate those features of intervention effect time trends that are likely to be reproducible, and to avoid over-interpretation of short-term effects likely to be due to extraneous effects associated with the particular study location and time period. Figure 1 illustrates the modelling approach for the case of message 3. With the exception of the sixth quarter, the estimates from the cubic spline model are close to those from the model incorporating separate parameters for the intervention effect in each quarter. However, a decline in effect in the sixth quarter followed by a ‘rebound’ in the seventh quarter, as suggested by the quarter-specific model, seems more likely to be attributable to extraneous factors rather than the intervention per se. The best-fitting cubic spline model focuses attention on what would appear to be the main message of the analysis; the effectiveness of the intervention declines over the first four quarters and becomes negligible by the fifth quarter. Nevertheless, in absolute terms, the difference between the estimates from the two models is small: in the sixth quarter, the point estimate from the model with quarter-specific intervention effect parameters is 0.93 compared with 1.03 from the restricted cubic spline models.
|
It should be noted that cubic spline models are very flexible and can fit a wide variety of time trends. In particular, as illustrated by Figure 1
, cubic spline models do not impose an assumption of monotonicity on time trends and, although sharp turning points or discontinuities cannot be modelled with cubic spline models, they can nevertheless capture reasonably rapid variation in effects over time (for example see Durrleman and Simon11). Consequently, the relatively slowly varying quadratic appearance of the cubic spline-based estimates in Figure 1 is not a necessary feature of the cubic spline model, but rather a reflection that, for the example of message 3, a slowly varying quadratic shape provides the best fit to the data among the wide range of possible trends which can be accommodated by cubic spline models. The repeated quarterly assessments of prescribing behaviour for each individual cannot be modelled as statistically independent quantities. Consequently, standard model-based variance estimates and confidence limits for logistic regression parameters are inappropriate in this setting. Therefore, we employed model robust standard error estimates,13 which do not assume that repeated measures within individuals are independent. It should be noted that this strategy also accommodates the within-individual correlation induced by repeated extraction of the control data to cover the different intervention periods.
In order to provide more readily interpretable effect measures than logistic regression model parameters, we used predicted values from the fitted models to compute standardized prescribing ratios (SPRs) which represent the ratio of the predicted proportion of scripts written for a particular drug by a large future cohort of GPs exposed to the education programme, compared with the proportion of scripts that would be written by an identical cohort if not exposed to the intervention programme. SPRs were computed for each quarter as well as for the full follow-up period. An example of the SPR calculations is given in Table 2.
|
Statistical inference for the standardized prescribing ratios employed a Monte-Carlo procedure which accounted for uncertainty due to estimation of the regression parameters and the distribution of pre-intervention prescribing indices and deprivation scores for the hypothetical future cohort of GPs.14,15
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Complete information was available for 48 (92%) of the 52 intervention group GPs (four GPs had retired or resigned). Considering the first study period as a whole, the education programme produced an effect in the expected direction in six of the eight key messages studied. This effect was statistically significant for five of the eight messages studied. The effect size, expressed in terms of SPRs, ranged from 7 to 40%. The change was greatest for key messages relating to short-term scripts such as antibiotics. The mean effect size was 1.20 (20%) for all the key messages and 1.27 (27%) for the messages where an effect was seen.
The duration of effect was variable (Table 3). There was an overall tendency to decay of the positive effects with time. The effects remained significant for between 6 and 24 months. The mean duration of significant effect was 14.5 months. Messages that related to medication dose (no. 3 and 4) had the shortest duration of effect at 6 months.
|
In response to message 5, the magnitude of change was small. This is the only message where the sole aim was to reduce costs. The cost impact of this small change must be examined more closely to determine its relevance. The proportion of metered dose inhaler scripts in the intervention group increased by 7% compared with the control groups. These results are statistically significant, with a small decline over this time period to 3% in the final 3-month period. Over the 24-month study period, 43 792 scripts were written by the intervention group GPs. Using the mean effect size for the full follow-up period, 2190 of these were written for metered dose inhalers which would previously have been written for breath-activated devices. Using the average price of medium dose metered dose and breath-activated inhaler devices at that time, this represents a cost saving of
NZ$22 000 for the 52 intervention group GPs for the 24-month study period.
Following the education session delivering message 2, where GPs were encouraged to prescribe amoxycillin rather than amoxy-clav as a first-line antibiotic, the proportion of amoxy-clav scripts in the intervention group decreased by 18% compared with the control groups. This effect was still significant at 2 years, with little difference in the magnitude of effect over this time and little difference between tablet and liquid (paediatric) preparations. This effect may have an interaction with the effect of message 2, which discussed the appropriate dose of amoxy-clav. There were three messages for which no statistically significant effect was detected.
Assessment of volunteer effect
The two control groups were compared to see whether they behaved differently as control groups (Table 5). The odds ratios shown contrast long run prescribing proportions or rates for the two control groups. The odds ratios are all close to 1, the confidence intervals all cross 1, and the P-values are all very high, indicating that the control group of early volunteers and the main control group did not behave in a significantly different manner. The control group data are therefore combined in all the results tables.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The general aim of continuing education in the general practice setting is to maintain and improve the quality of practice. While it is difficult to define quality practice explicitly and therefore measure it, appropriate or ‘rational’ prescribing is recognized as one of the key indicators of quality of general practice.16,17 GPs are under increasing pressure in their attempts to keep abreast of current prescribing literature in the face of drug company advertising, and direct marketing to consumers by the pharmaceutical industry. There is significant interest in using CME to promote rational prescribing amongst GPs. Two influences appear to drive this: (i) wide variation in prescribing practice; and (ii) a desire to minimize unnecessary expenditure of finite resources. It is important that CME strategies for GPs are of proven effectiveness given the large resource and time commitment required from the funders, organizers and attending GPs.
It is clear from our results that peer-led small group education can result in changes in prescribing practice. It is also clear that these changes, while being sustained for at least 9–24 months, decay with time. These changes are significant clinically—the number of scripts per annum in the intervention group relating to key messages 1–5 was 45 050. The change was also significant economically for the one message (5) which was related to cost saving alone—$NZ22 000 was saved for an estimated outlay of $NZ100 per GP in direct session costs. There were two key areas where no change was seen. It is possible that the limitations around the data reduced the ability to detect any effect in the ratio of ß2 agonist:steroid inhalers. In New Zealand, when inhaled asthma medication is prescribed, only the number of inhalers dispensed (rather than dose or duration) is recorded in the database, with no patient linkage. The message recommending restriction of use of selective seratonin reuptake inhibitors (SSRIs) as a first-line antidepressant was particularly unsuccessful in the face of strong local specialist preference, reflecting the international trend to increasing use of SSRIs as the first-line antidepressant of choice.
The effect of the intervention on long-term scripts written for chronic diseases showed a less marked rise than for short-term scripts. This may be explained by the ongoing nature of treatment for chronic disease: in this case, the GP must wait for patients to present before modifying ongoing scripts, and patients must be willing to change from a familiar/effective regime.
The study design had a number of obvious strengths and weaknesses. The use of control groups allowed negation of a number of other potential sources of bias and confounding, such as seasonal variation, disease outbreaks, changes in medication pricing, script dispensing regulations, pharmaceutical company promotions and prescriptions driven by other sources. These potential effects make ‘before and after’ comparisons difficult to interpret. The similarity of deprivation profile of the three groups seen in Table 6 offers extra reassurance that disease outbreaks in more deprived areas could not be responsible for differences seen.
|
The main weakness of this study is that it was a retrospective analysis of a pilot. The GPs in the intervention and control groups were not selected randomly. The GPs who responded first to the invitation to the study were included as the intervention group and may be different in their responses to education strategies from those in the control groups. Even though efforts have been made to account for any ‘volunteer’ effect by using the next 25 GPs who responded to the invitation to join the pilot as an extra control group, there is now a need for a prospective randomized controlled trial of this education strategy.
A major strength of the study is the ability to track the effect for some years after the education intervention, which allows a proper assessment of time to decay of the effect of the message. These results demonstrate the need to look at repeating/reinforcing messages at 12–24 month intervals. It may be possible to test a variety of reinforcing strategies as part of a prospective randomized controlled trial.
One of the distinguishing features of this programme is that it is peer led—unlike many postgraduate continuing education programmes in general practice. The knowledge gaps to be targeted with ‘key messages’ were identified by GPs, and the education intervention was also run by GPs. This peer-embedded programme is a departure from the usual ‘specialist/expert-led’ format of GP continuing education programmes. The importance of a needs-identification process and the involvement of the programme user group in this process have been identified as crucial factors in the success of any effective learning programme.18
This study shows that a peer-led small group education process is a useful strategy for effecting changes in prescribing in general practice, and that these changes are sustained over time against a background of ongoing monthly meetings on new topics. A key aspect of this study of the CPEG programme effectiveness is its use in an environment where other effective education strategies were already in place. During the study period, all Pegasus GPs were included in other educational initiatives related to pharmaceutical budget holding, each of which is effective in its own right: audit and feedback of prescribing compared with average figures for all GPs in the IPA, and academic detailing by pharmaceutical facilitators. In addition, all GPs received mailed educational bulletins. These activities covered the same topics as the small group sessions, and the combination is similar to that used effectively by Zwar in the study on prescribing for URTIs.10 This indicates that the effect of this small group education strategy is achieved in addition to any effects of these other strategies on the intervention group GPs.
This education strategy should be generalizable. However, as improvements in prescribing are made, the identifiable gaps between reality and best practice should narrow, with a reduction in potential beneficial effects of education strategies.
|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1 Petersen S. Time for evidence based medical education. Br Med J 1999; 318: 1223–1224.
2 Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J 1995; 153: 1423–1431.[Abstract]
3 Thomson MA, Davis DA, Haynes RB, Freemantle N, Harvey EL. Audit and feedback to improve professional practice and health care outcomes. The Cochrane Library 1998; Issue 4.
4 Thomson MA, Davis DA, Haynes RB, Freemantle N, Harvey EL. Outreach visits to improve professional practice and health care outcomes. The Cochrane Library 1999; Issue 1.
5 Freemantle N, Harvey EL, Wolf F, Grimshaw JM, Grilli R, LA B. Printed educational materials to improve the behaviour of health professionals and patient outcomes. The Cochrane Library 1999; Issue 1: CD-ROM.
6 Haynes RB, Davis DA, McKibbon A, Tugwell P. A critical appraisal of the efficacy of continuing medical education. J Am Med Assoc 1984; 251: 61–64.
7 Davis D, O’Brien M, Freemantle N, Wolf F, Mazmanian P, Tayloy-Vaisey A. Impact of formal continuing medical education. Do conferences, workshops, rounds and other traditional continuing education activities change physician behaviour or health care outcomes? J Am Med Assoc 1999; 282: 867–874.
8 Davis DA, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. Can Med Assoc J 1997; 157: 408–416.[Abstract]
9 Melander E, Bjorgell A, Bjorgell P, Ovhed I, Molstad S. Medical audit changes physicians’ prescribing of antibiotics for respiratory tract infections. Scand J Primary Health Care 1999; 17: 180–184.[CrossRef][Web of Science][Medline]
10 Zwar NA, Gordon JJ, Sanson-Fisher RW. Evaluation of an educational program in rational prescribing for GP trainees. Aust Family Physician 1995; 24: 833–838.[Medline]
11 Crampton P, Salmond C, Sutton F. NZDep 91 Index of Deprivation. Wellington: Health Services Research Centre, Victoria University, 1997.
12 Durrleman S, Simon R. Flexible regression models with cubic splines. Stats Med 1989; 8: 551–561.[Web of Science][Medline]
13 Royall R. Model robust confidence intervals using maximum likelihood estimates. Int Stats Rev 1986; 54: 221–226.
14 Gelman A, Carlin B, Stern H, Rubin DB. Bayesian Data Analysis. London: Chapman and Hall, 1995.
15 Rubin DB. The Bayesian bootstrap. Ann Stats 1988; 16: 1684–1695.
16 Moussa A, Bridges-Webb C. Quality of care in general practice. A delphi study of indicators and methods. Aust Family Physician 1994; 23: 465–468, 472–473.[Medline]
17 Heaney DJ, Walker JJ, Howie JG et al. The development of a routine NHS data based index of performance in general practice (NHSPPI). Fam Pract 2001; 18:
18 Stein LS. The effectiveness of continuing medical education: eight research reports. J Med Educ 1981; 56: 103–110.[Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  F May, D Simpson, L Hart, D Rowett, and D Perrier Experience with academic detailing services for quality improvement in primary care practice Qual. Saf. Health Care, June 1, 2009; 18(3): 225 - 231. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Figueiras, M. T. Herdeiro, J. Polonia, and J. J. Gestal-Otero An educational intervention to improve physician reporting of adverse drug reactions: a cluster-randomized controlled trial. JAMA, September 6, 2006; 296(9): 1086 - 1093. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. P Herbert, J. M Wright, M. Maclure, J. Wakefield, C. Dormuth, P. Brett-MacLean, J. Legare, and J. Premi Better Prescribing Project: a randomized controlled trial of the impact of case-based educational modules and personal prescribing feedback on prescribing for hypertension in primary care Fam. Pract., October 1, 2004; 21(5): 575 - 581. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||