Skip Navigation


Family Practice Advance Access originally published online on October 18, 2006
Family Practice 2007 24(1):26-33; doi:10.1093/fampra/cml051
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
24/1/26    most recent
cml051v1
Right arrow E-letters: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Siriwardena, A.
Right arrow Articles by Dewey, M
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Siriwardena, A.
Right arrow Articles by Dewey, M
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author (2006). Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oxfordjournals.org

Investigation of the effect of a countywide protected learning time scheme on prescribing rates of ramipril: interrupted time series study

AN Siriwardenaa, P Fairchildb, S Gibsonc, T Sachd and M Deweye

a School of Health and Social Care, University of Lincoln
b The Surgery, 11 Church Street Hibaldstow, Brigg DN20 9ED
c Lincolnshire Prescribing Support Team, West Lincolnshire Primary Care Trust, Cross O'Cliff Court Bracebridge Heath, Lincoln LN4 2HN
d University of Nottingham, School of Community Health Sciences, 13th Floor Tower Building, University Park Nottingham NG7 2RD
e School of Community Health Sciences, B41 Medical School, Queen's Medical Centre Nottingham NG7 2UH

Correspondence to Professor A N Siriwardena School of Health and Social Care, University of Lincoln, Court 11, Apartment 1, Room 2, Campus Way, Lincoln LN6 7BG, UK; Email: nsiriwardena{at}lincoln.ac.uk

Received 7 February 2005; Revised 2 August 2006; Accepted 13 September 2006.


   Abstract
Top
 Abstract
Introduction
 Method
 Results
 Discussion
 Funding body
 Ethics committee
 Contributors
 Competing interests
 References
 
Background. Protected learning time (PLT) schemes have been set up in primary care across the UK. There is little published evidence of their effectiveness.

Objective. To investigate the effect of a PLT intervention for general practice to increase prescribing of ramipril for prevention of cardiovascular outcomes.

Design. Quasi-experimental, interrupted time series.

Setting. Lincolnshire, UK.

Methods. Prescribing data were analysed one year before and after the education for change in rate of increase of prescribing of ramipril, whether change in prescribing was related to postulated explanatory variables and to determine intervention costs.

Main outcome. The primary outcome was the rate of change of ramipril (10 mg) prescription items 12 months after compared with before the educational intervention. Secondary outcomes included cost.

Results. Ramipril prescribing at therapeutic dosage increased significantly (odds ratio 1.50, 95% CI 1.07–1.93) following education by 52 345 items (31 132 items at 10 mg) at a cost of £292k to £460k depending on formulation. This occurred despite a background of secular change. Most practices were represented by GPs, nurses or both during the education. Single-handed GPs were less likely to attend. Practices showed considerable variation in response to the educational intervention. The only predictor of whether practices increased in prescribing rate after the education was whether a practice nurse had undertaken specific diabetes training. Total list size, dispensing, training or single-handed status and GP attendance did not predict a change in prescribing.

Conclusion. PLT schemes can contribute to beneficial changes in prescribing across a large geographical area.

Keywords. Education, family practice, diabetes, prescribing, angiotensin-converting-enzyme inhibitor.


   Introduction
Top
 Abstract
 Introduction
Method
 Results
 Discussion
 Funding body
 Ethics committee
 Contributors
 Competing interests
 References
 
Protected learning time (PLT) schemes for primary care have proliferated across the UK, but there has been little published evidence of their effectiveness. Lincolnshire TARGET (Time for Audit, Review, Guidelines, Education and Training) was set up in 2001 as an innovative PLT scheme across a large rural county involving all general practices and primary care teams. Following the original scheme in Doncaster (Yorkshire, UK) the Lincolnshire model adhered to national recommendations for continuing professional development (CPD) focusing on patient outcomes, practitioner needs and evidence-based educational methods promoting multidisciplinary learning.1,2

The aim of Lincolnshire TARGET was to provide needs-based learning for GPs and other primary care staff during working hours (protected time) by using local arrangements to provide primary care services during educational sessions. Each session was organized and delivered by a team of educators, led by a clinical director with administrative support. The sessions aimed to focus on an important topic based on both local and practitioner need, and national priorities using a variety of methods ranging from lectures to interdisciplinary facilitated small group work.

It has been argued that more studies of educational interventions should be conducted in primary care and, to increase generalizability, should evaluate change in a geographical area (rather than a single practice) targeting an identifiable need associated with a patient outcome.3 We are not aware that PLT schemes in the UK or other similar initiatives abroad have been rigorously evaluated. This scheme provided an opportunity for such an evaluation.

Because of the benefit of angiotensin-converting-enzyme (ACE) inhibitors4 in improving clinical outcomes for patients with diabetes at high risk, we used prescribing rates for ramipril, a specific ACE inhibitor, as an outcome. The HOPE study showed evidence that patients with coexisting diabetes and hypertension or other cardiovascular risk factors should be treated with an ACE inhibitor at a therapeutic dose (specifically ramipril 10 mg)5 to reduce cardiovascular morbidity and mortality.

The aim of this study was to investigate whether education for primary health care workers improved preventive care for patients with diabetes through prescribing of ramipril.


   Method
Top
 Abstract
 Introduction
 Method
Results
 Discussion
 Funding body
 Ethics committee
 Contributors
 Competing interests
 References
 
An interrupted time series design was used to analyse prescribing data for 1 year in all (101) practices before and after the educational session for change in the rate of prescribing of ramipril.

This design was chosen to take into account secular trends, because of the inappropriateness of randomizing practices (since all were invited to the educational session) and the difficulty in providing an unbiased control group owing to potential contamination.6 It sought to detect whether the intervention had a significantly greater effect than the underlying secular trend. The data collection was retrospective to reduce the likelihood of a ‘Hawthorne effect’ (non-specific beneficial effect of participation in the research) biasing the results. The design did not preclude effects other than the chosen intervention affecting the particular outcome.

The intervention was an educational session relating to a national priority, ACE inhibitor prescribing for prevention of heart disease in high-risk diabetic patients.5 In total six training sessions were run in 1 month (November 2001) across the county. The session lasted 2.5 hours and consisted of an outline of objectives and opening talk setting the scene of diabetes care in general practice followed by two streams of talks. The first for clinicians were workgroups looking at the HOPE trial in which a local GP who had successfully implemented the findings into his own practice was able to describe practical steps leading to improvements in prescribing. For other staff the patient perspective was provided by Diabetes UK, a talk on the diabetic clinic given by a specialist diabetic nurse and a workgroup looking at computer data, risk assessment and routine reception procedures took place. Finally practice teams had the opportunity to discuss barriers to implementation of HOPE and an action plan of how they might overcome these. Information from practice discussions was then fed back to the whole audience with opportunity for further plenary before closing the session. The TARGET budget included educational sponsorship from a number of pharmaceutical companies and promotional stands were also a feature of all the sessions. Lunch was provided before the educational meeting began but no other incentives were offered to encourage attendance.

Data on number of prescriptions for 1.25, 2.5, 5 and 10 mg doses of ramipril and net ingredient cost (NIC) for the same doses was collected across three primary care trusts (PCTs), East Lincolnshire, Lincolnshire South West and West Lincolnshire, and by individual practice using Prescribing Analysis and Cost (PACT). Ramipril was chosen, rather than ACE inhibitors more generally, because this was the drug used in the HOPE study and also because this drug was supported for use by the countywide prescribing team and formulary. Data points were taken at monthly intervals in 2001 and 2002, approximately 12 months before and 12 months after the intervention to remove seasonal effects and to ensure reliability. Although the intervention took place in November 2001 it was considered unrealistic to have expected the change to take immediate effect as some time would have been needed for implementation of change and because the Christmas period would have hindered this. January 2002 was decided a priori as the starting period to observe the effect of the intervention.

The data file also included practice list size, training, dispensing or single-handed status, or whether the practice nurse had undertaken specific diabetes training. Qualitative data from semi-structured interviews of key personnel around facilitators and barriers to change to investigate changes in organizational behaviour in individual practices were also collected and are presented in a separate paper.

The statistical analysis fulfilled methodological criteria based on current guidelines,7,8 evaluated the intervention alone, independent of other changes using an objective outcome variable (PACT data) not directly affected by the intervention. Trend over time and changes in trend at about the time of the intervention were tested for. Covariates that might have predicted the extent of change were also examined. Descriptive statistics for the sample as a whole (on an intention to educate principle) were presented but the analysis of the intervention included whether the practice attended as a covariate (similar to an on-treatment analysis).

Mixed effects models were fitted using the NLME package10 and generalized mixed effects models with glmmPQL from the MASS library.11 Results are presented with 95% confidence intervals.

Brown et al. concluded that there was an urgent need to undertake robust economic evaluations of CPD, in order to assess the economic value of CPD, and to inform resource allocation decision making.12 For a number of reasons, outlined in the discussion, neither a full economic evaluation13 nor a comprehensive cost analysis was undertaken. Instead, the study reports the costs of the intervention from a health service perspective. These costs (UK£2004) included the educational session, primary care cover (through GP cooperatives, etc.) and organization of training (including pharmaceutical sponsorship). These were collected at 2001 prices and inflated to 2004 prices using the Hospital and Community Health Services inflation index.14 In addition, change in prescribing costs as a result of the intervention were assessed using the NIC in UK£2004 per item based on preparation strength and presented both as if all had capsules or all received tablets. NIC is the cost of ramipril before discounts, without dispensing costs applied, and without adjusting for any income received from prescription charges.15 Although the cost of ramipril fell during the period 2001 and 2004, the most recently published NIC data is presented to make the analysis as relevant and up-to-date as possible. Data for costs of the educational intervention were routinely collected by Lincolnshire TARGET and changes in number of prescriptions of ramipril were collected using PACT data. Since the aim of the educational intervention was to increase the number of prescriptions of ramipril (10 mg), the incremental cost of the educational intervention per item, per mg and per 10 mg increase is shown.


   Results
Top
 Abstract
 Introduction
 Method
 Results
Discussion
 Funding body
 Ethics committee
 Contributors
 Competing interests
 References
 
The original dataset included 105 practices. Two practices were newly established during the study and, therefore, without prescribing data were excluded. Another two practices disappeared through resignation and retirement of doctors during the study period. The final total of 101 practices comprised 38 from East Lincolnshire, 25 from Lincolnshire South and 38 from West Lincolnshire.

Of these, 15 were training practices, 56 Personal Medical Services (PMS), 18 single-handed practices and 59 dispensing. There were 23 practices, which had at least one nurse trained in diabetes care (the Warwick University Certificate in Diabetes Care).

At least one GP attended the intervention session from 68 practices, and from 64 at least one practice nurse or health visitor attended. No clinical professional attended from 25 practices. From 12 practices only one or more GPs attended and no nurses, from 8 practices one or more practice nurses or health visitors but no GPs and from 56 at least one GP and at least one practice nurse or health visitor attended.

Correlates of attendance
Results from logistic regression predicting GP attendance from other practice characteristics is shown (Table 1). Single-handed practices were less likely to have a GP attend. There was a weak suggestion that training practices were more likely to attend, but the wide confidence interval revealed that there were relatively few such practices. A model predicting whether anyone from the practice attended, GP or practice nurse, or health visitor, showed a similar pattern although the confidence interval for single-handed practices included unity.


View this table:
[in this window]
[in a new window]

  		TABLE 1 Logistic regression predicting GP attendance

 
Patterns of response from raw data
Raw data for 10 mg prescriptions of ramipril at each practice in East Lincolnshire PCT are shown (Fig. 1). Similar patterns emerged for all three PCTs. Each panel corresponded to a practice and within each figure they were ordered by the maximum number of 10 mg prescriptions for ramipril.


Figure 1
View larger version (42K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 1 Graphs showing changes in prescribing of ramipril 10 mg over time by individual practice in East Lincolnshire Primary Care Trust

 
There were a number of patterns. Some practices did not change prescribing behaviour, whereas others showed various changes. Examining Fig. 1 and for convenience of reference numbering the panels from 1 to 8 along the bottom row, 9 to 16 the second and so on patterns observed included:
(a) Practices in which prescribing rates are increasing, an increase that rises after the mid-point either a little (number 10) or a lot (29).
(b) Practices that change sharply at about the time of the intervention (22 or 24).
(c) Late adopters (16).
(d) Practices that were always rising (38).
(e) Various other patterns (34).

Random effects model
Table 2 shows the confidence intervals for a model including all the effects shown in the table. The model includes a random term for time and a random term for time since intervention (abbreviated as ‘timepost’ in the tables) as both were needed for a good fit. In order to account for the serial correlation between observations the correlation structure was estimated using an AR1 model.


View this table:
[in this window]
[in a new window]

  		TABLE 2 Model for number of prescriptions of 10 mg of ramipril including predictors

 
Prescribing increased over time and the rate of increase went up after the intervention. There were also differences between the three PCTs. Practices where at least one nurse had attended the Warwick diabetes course had a higher rate of prescribing. Because the variance increased with the mean and a log transformation produced variance decreasing with the mean the square root transformation was used, which yielded more nearly normally distributed residuals that did not vary with the mean. However a model using this transformation gave conclusions that were essentially the same as for the untransformed analysis, and in view of the difficulty in interpreting effects on the square root scale the untransformed results are shown.

It was perhaps surprising that total list size was not important in the model, but the estimate was negatively correlated with the estimate of the effect of the presence of a Warwick trained nurse. If that variable was removed total list size did have an effect. Fitting a model replacing whether a GP attended with whether either a nurse or GP attended did not lead to any substantial change.

Testing the effect of attendance
If attendance had affected prescribing an interaction between GP attendance and time after intervention would be expected. The rise over time was greater when the GP had attended the intervention, but there was no difference in the rate of increase after the intervention (Table 3). Fitting a model replacing whether a GP attended with whether either a nurse or GP attended did not lead to any substantial change.


View this table:
[in this window]
[in a new window]

  		TABLE 3 Model including interaction between whether the GP attended and time, and with time after the intervention

 
Proportion of prescriptions for ramipril
The main secondary outcome was the proportion of prescriptions for ramipril, which were of the evidence-based 10 mg dose. Plots (not shown) suggested that there was evidence of a rise over time but not of any substantial increase after the intervention.

A logistic model was fitted, again allowing for time and timepost to be random effects. The rate of key variables in the model increased with time with no further increase after the intervention (Table 4). There was a tendency for higher rates if at least one GP attended but this had no influence either on overall rate or on the rate after intervention. Although models with the other predictor variables were fitted they did not lead to an improvement in the fit.


View this table:
[in this window]
[in a new window]

  		TABLE 4 Modelling the proportion of prescriptions which were for 10 mg of ramipril

 
Cost analysis
The total cost to the health service of providing this intervention was estimated to be £42 854.38. The biggest cost was administration, which included staff costs of the TARGET office team (apportioned approximately to cover the educational session on diabetes care only, see Table 5). The NIC of ramipril at 2004 prices for 2002 (post-educational session), compared with 2001 (pre-educational session), increased across all three Lincolnshire PCTs to varying degrees (Table 6). The incremental cost, therefore, of increasing the number of Ramipril prescription items via the protected learning scheme was £0.82 per item or £81.87 per 100 items prescribed across the whole of Lincolnshire. However, as each item prescribed can vary between 1.25 mg and 10 mg a more appropriate estimate, given the aim is not just to increase prescriptions of ramipril but to increase them to a specified level, is the cost of per milligram of ramipril prescription. The incremental cost of the protected time learning scheme was £0.11 per mg or £1.08 per 10 mg increase of ramipril in the 12 months post-educational intervention (10 mg was the recommended prescription during the scheme).


View this table:
[in this window]
[in a new window]

  		TABLE 5 Costs of providing protected learning time for diabetes care (UK£2004)

 


View this table:
[in this window]
[in a new window]

  		TABLE 6 Change in Net Ingredient Cost (NIC) and number of items of ramipril in 2002 compared with 2001 (UK£2004)

 

   Discussion
Top
 Abstract
 Introduction
 Method
 Results
 Discussion
Funding body
 Ethics committee
 Contributors
 Competing interests
 References
 
This study showed significant change in ramipril prescribing at therapeutic dosage following the educational intervention. The change occurred despite a background of secular change and was partly related to increased ramipril prescribing for diabetes but probably also for heart failure and hypertension. Most practices were represented at the educational session. Single-handed GPs were less likely to attend perhaps because of actual or felt additional work pressures. Individual general practices varied in their response to the educational intervention. The only predictor of increase in practice prescribing rate was whether a practice nurse had undertaken formal education in diabetes care. Total list size, dispensing, training or single-handed status and GP attendance did not predict a change in prescribing.

The strengths of this study included its quasi-experimental design, use of retrospective routinely collected prescribing data minimizing measurement effects and inclusion of all practices over a large geographical area16 The control period before the intervention and sufficient data points before and after the intervention accounted for secular changes. A weakness was that an indirect measure was used rather than directly measuring ramipril or ACE inhibitor prescribing in diabetic patients. Although a ‘control county’ for comparison would have provided more robust evidence that other external factors produced the change, this was precluded by resource limitations.

Changes in prescribing rates may be partly explained by factors that we were unable to control for, for example the effect of other changes occurring around the same time as the training session such as either national or local policy changes. Such factors may have included previous published research, a countywide prescribing formulary and pharmaceutical company activity. In addition, the observed expansion of ramipril prescribing may, in part, reflect substitution away from other brands of ACE inhibitors, and since in this study all brands were not monitored it was impossible to state how much of the expansion occurred as a result of substitution rather than new prescriptions. Despite not being able to rule out the influences of these effects the largest increase across all three PCTs was at the 10 mg level as recommended at the training session. As a consequence the NIC cost increase would also have been largely driven by this increase in prescribing at the 10 mg level.

Other confounding interventions, occurring at the same time as the educational session, were investigated using semi-structured interviews of key practice personnel. This was done by asking doctors, nurses or managers involved in delivering or organizing diabetes care in a stratified sample of practices what changes occurred in diabetes care in their practice and how it was affected by the educational session or other factors. Participants cited examples of change introduced directly as a result of the educational session. The results of this qualitative study are presented in a separate paper. The most important changes following the educational intervention reported included switching patients to glitazones; putting patients on insulin either directly or by referral to secondary care; using ACE inhibitors as a first line for patients with diabetes who develop hypertension; and increased use of aspirin. Other reported facilitators of change included financially driven performance targets, research evidence and national guidance.

Prescribing costs also increased above secular trends. Together with the cost of providing the educational intervention, this shows that such schemes do incur expenditure in the short term. However, due to the study design it was not possible to determine whether this additional cost was offset by cost savings or whether the intervention was cost-effective. Potential cost savings could occur, for example, if ramipril prescriptions increased because GPs substituted away from more expensive medications. However, prescription rates of other ACE inhibitors may also have increased concurrently, in part as an indirect result of the education, although this was not monitored. Cost savings could also have been achieved as a result of the intervention if increased prescription rates of ramipril led to reduced morbidity and healthcare utilization rates. Any such effect, were it to occur, would be observed over the longer term and was, therefore, beyond the scope of this study. The educational intervention could also increase costs if more patients taking ramipril led to increased attendances for follow-up checks with the doctor or practice nurse.

This study did not present a full economic evaluation or conclusively determine the value for money afforded by the educational intervention, since this would have required data, such as patient identifiable utilization data pre-intervention or post-intervention, that were not feasible within the chosen design. Therefore, the change in total incremental costs likely to result from the educational intervention could not be estimated. However, providing costs captured appropriate follow-on costs then an incremental cost per milligram or per prescription item could be used in resource allocation decision making where a decision has to be made about how best to increase prescription rates (technical efficiency) given money had already been allocated to education programmes that aimed to increase prescription rates. Such information could not be used to decide how to allocate resources between different types of interventions or service (allocative efficiency), since the outcome is unlikely to be comparable or meaningful for all interventions or services. Cardiovascular outcomes often require lengthy follow-up periods in order to capture them in terms of both the effect on quality and length of life using a generic measure such as a Quality Adjusted Life Year (QALY), but without the direct involvement of patients in this study we were unable to collect data relating to utility and, therefore, unable to estimate the change in QALYs associated with the intervention.

This study provides limited evidence for the effectiveness of a PLT scheme in which a tailored educational intervention17 using opinion leaders as change agents18 for multidisciplinary team-based education19,20 was used. The intervention using a theoretical framework based on multiple techniques, including social modelling by opinion leaders, marketing and outreach, active learning, overcoming barriers and strengthening facilitators underpinned by management support, was shown to be effective in producing change in practice and outcomes.21

Although PLT schemes can be hampered by trying and failing to meet educational needs of different professional groups,22 the educational process in this study was fundamentally a catalyst for individual and collective learning. This enabled different professionals providing care for a single patient group to learn together and consider how they could improve care through teamwork as a critical driver for change.23

Other reasons for effectiveness of the intervention was that the outcome of interest was unambiguous, the evidence for it was strong, likely to have been believed by practitioners and change relatively easy to implement.21,24 In this respect the PLT scheme was an example of true interdisciplinary learning25 providing evidence that participating practices and PCTs were true learning organizations.26

This study provides a model for further research on the effects of educational interventions on outcomes. It also invites investigation of the relative effectiveness and cost-effectiveness of PLT schemes in comparison with other methods of innovation diffusion.


   Funding body
Top
 Abstract
 Introduction
 Method
 Results
 Discussion
 Funding body
Ethics committee
 Contributors
 Competing interests
 References
 
RCGP Scientific Foundation Board provided funding for this study. Pilot work was funded by Trent RDSU. Members of the research team were also supported by West Lincolnshire PCT (ANS, SG) and Trent Research and Development Support Unit (ANS) through a Designated Research Team Award.


   Ethics committee
Top
 Abstract
 Introduction
 Method
 Results
 Discussion
 Funding body
 Ethics committee
Contributors
 Competing interests
 References
 
Lincolnshire Research Ethics Committee (study number: 02/1/680). The study was approved for research management and governance by West Lincolnshire PCT.


   Contributors
Top
 Abstract
 Introduction
 Method
 Results
 Discussion
 Funding body
 Ethics committee
 Contributors
Competing interests
 References
 
ANS had the initial idea, contributed to the design and led the study. MD contributed to the design and undertook statistical and data analysis. PF organized the educational intervention and made available cost and attendance data. SG contributed to the design and methodology of the study and provided prescribing data. All authors contributed to the drafting of the article and approved the final version of the paper. ANS is the guarantor for the paper.


   Competing interests
Top
 Abstract
 Introduction
 Method
 Results
 Discussion
 Funding body
 Ethics committee
 Contributors
 Competing interests
References
 
PF was Medical Director of Lincolnshire TARGET at the time of the study.


   Acknowledgments
 
We thank the Lincolnshire practices and PCTs who participated in this study. Thanks also to Rick Ansell (RA), Prescribing Analyst Lincolnshire, and Jane Dyas, Research Facilitator Trent Research and Development Support Unit (RDSU) for their help with the study.


   Notes
 
Siriwardena AN, Fairchild P, Gibson S, Sach T and Dewey M. Investigation of the effect of a countywide protected learning time scheme on prescribing rates of ramipril: interrupted time series study. Family Practice 2007; 24: 26–33.


   References
Top
 Abstract
 Introduction
 Method
 Results
 Discussion
 Funding body
 Ethics committee
 Contributors
 Competing interests
 References
 
1 Department of Health. (1998) A Review of Continuing Professional Development in General Practice: A Report by the Chief Medical Officer(Department of Health, London).

2 Jones RV. (1986) Working together--learning together. J R Coll Gen Pract Occas Pap 33:1–26.

3 Freudenstein U and Howe A. (1999) Recommendations for future studies: a systematic review of educational interventions in primary care settings. Br J Gen Pract 49:995–1001.[Web of Science][Medline]

4 Tatti P, Pahor M, Byington RP, et al. (1998) Outcome results of the Fosinopril Versus Amlodipine Cardiovascular Events Randomized Trial (FACET) in patients with hypertension and NIDDM. Diabetes Care 21:597–603.[Abstract]

5 Heart Outcomes Prevention Evaluation Study Investigators. (2000) Effects of ramipril on cardiovascular and microvascular outcomes in people with diabetes mellitus: results of the HOPE study and MICRO-HOPE substudy. Lancet 355:253–259.[CrossRef][Web of Science][Medline]

6 Grimshaw J, Campbell M, Eccles M, Steen N. (2000) Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract 17:Suppl 1, S11–S16.[Free Full Text]

7 The Cochrane Collaboration on Effective Professional Practice. (1997) Methods used in reviews. 3-12. York, The Cochrane Collaboration on Effective Professional Practice.

8 Abbasi K and Smith R. (1999) Education group for guidelines on evaluations. Guidelines for evaluating papers on educational interventions. BMJ 318:1265–1267.[Free Full Text]

9 Ihaka R. (1996) Gentleman R. R: a language for data analysis and graphics. J Comput Graph Stat 5:299–314.

10 Pinheiro JC and Bates DM. (2000) Mixed-Effects Models in S and S-PLUS(Springer-Verlag, New York).

11 Venables WN and Ripley BD. (2002) Modern Applied Statistics with S(Springer-Verlag, New York).

12 Brown CA, Belfield CR, Field SJ. (2002) Cost effectiveness of continuing professional development in health care: a critical review of the evidence. BMJ 324:652–655.[Free Full Text]

13 Drummond MF, Sculpher MJ, Torrance GW, et al. (2005) Methods for the Economic Evaluation of Health Care Programmes 3rd edn. Oxford University Press.

14 Curtis L and Netten A. Unit Costs of Health and Social Care 2005. Availabale at http://www.pssru.ac.uk/pdf/uc/uc2005/uc2005_inflation.pdf (accessed 12 July 2006).

15 The information centre for health and social care. (2005) Prescription Cost Analysis: England. Available at http://www.ic.nhs.uk/pubs/prescostanalysis2005 (accessed 12 July 2006).

16 Ukoumunne OC, Gulliford MC, Chinn S, Sterne JAC, Burney PGJ. (1999) Methods for evaluating area-wide and organisation-based interventions in health and healthcare: a systematic review. Health Technol Assess 3:iii-92.

17 Baker R, Reddish S, Robertson N, Hearnshaw H, Jones B. (2001) Randomised controlled trial of tailored strategies to implement guidelines for the management of patients with depression in general practice. Br J Gen Pract 51:737–741.[Web of Science][Medline]

18 Thomson O'Brien MA, Oxman AD, Haynes RB, Davis DA, Freemantle N, Harvey EL. (2000) Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev CD000125.

19 Hulscher ME, Wensing M, van Der WT, Grol R. (2001) Interventions to implement prevention in primary care. Cochrane Database Syst Rev CD000362.

20 Siriwardena AN, Rashid A, Johnson MRD, Dewey ME. (2002) Cluster randomised controlled trial of an educational outreach visit to improve influenza and pneumococcal immunisation rates in primary care. Br J Gen Pract 52:735–740.[Web of Science][Medline]

21 Siriwardena AN. (2003) The impact of educational interventions on influenza and pneumococcal vaccination rates in primary care. 1-350. (De Montfort University, Leicester).

22 Brooks N and Barr J. (2004) Evaluation of protected learning time in a primary care trust. Qual Prim Care 12:29–35.

23 Stone EG, Morton SC, Hulscher ME, et al. (2002) Interventions that increase use of adult immunization and cancer screening services: a meta-analysis. Ann Intern Med 136:641–651.[Abstract/Free Full Text]

24 Rogers EM. (1995) Diffusion of Innovations(Free Press, New York; London).

25 Freeth D, Hammick M, Koppel I, Reeves S, Barr H. (2002) A critical review of evaluations of interprofessional education. Occasional Paper No. 2. (Learning and Teaching Support Network, Centre for Health Sciences and Practice, King's College, London).

26 Senge PM. (1990) The Fifth Discipline: The Art and Practice of the Learning Organization(Doubleday/Currency, New York).


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
24/1/26    most recent
cml051v1
Right arrow E-letters: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Siriwardena, A.
Right arrow Articles by Dewey, M
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Siriwardena, A.
Right arrow Articles by Dewey, M
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?