Family Practice Vol. 20, No. 6, 646-654
© Oxford University Press 2003, all rights reserved
Article |
Cluster randomized trial of a guideline-based open access urological investigation service
a Health Services Research Unit, c Department of Public Health and g Department of General Practice and Primary Care, University of Aberdeen, d Department of Urology and f X-ray Department, Grampian University Hospitals Trust, h Elmbank Medical Group, Grampian Primary Care NHS Trust, Aberdeen and e Health Economics Research Centre, Institute of Health Sciences, University of Oxford, Oxford, UK
b Present address: Clinical Epidemiology Programme, Ottawa Health Research Institute and Centre for Best Practice, Institute of Population Health, University of Ottawa, Ottawa, Canada
Correspondence to Ruth Thomas, Research Fellow, Health Services Research Unit, University of Aberdeen, Foresterhill, Aberdeen AB25 2ZD, UK; Email: r.e.thomas{at}abdn.ac.uk
Received 7 October 2002; Revised 19 June 2003; Accepted 14 July 2003.
Thomas RE, Grimshaw JM, Mollison J, McClinton S, McIntosh E, Deans H and Repper J. Cluster randomized trial of a guideline-based open access urological investigation service. Family Practice 2003; 20: 646–654.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionSupplementary dataReferences |
|---|
Background. Out-patient services are trying to achieve effective and efficient health care in overcrowded, busy clinic settings. ‘One stop’ and ‘open access’ clinics have been advocated as a way of improving out-patient services.
Objectives. Our aim was to evaluate the effectiveness and efficiency of a guideline-based open access urological investigation service.
Methods. General practices were randomized to receive either referral guidelines and access to the investigation service for lower urinary tract symptoms (LUTS) or referral guidelines and access to the investigation service for microscopic haematuria (MH). The study population comprised 66 general practices in the Grampian region of Scotland referring 959 patients. The outcome measures were compliance with guidelines (number of recommended investigations completed), number of general practice consultations, the number and case mix of referrals, waiting time to initial hospital appointment, and the number of patients with a management decision reached at initial appointment and discharged by 12 months after referral.
Results. GPs' compliance with referral guidelines increased (difference in means 0.5; 95% confidence interval 0.2–0.8, P < 0.001). Approximately 50% of eligible patients were referred through the new system. The number and case mix of referrals were similar. The intervention reduced the waiting time from referral to initial out-patient appointment (ratio of means 0.7; 0.5–0.9, patients with LUTS only) and increased the number of patients who had a management decision reached at initial appointment (odds ratio 5.8; 2.9–11.5, P < 0.00001, both conditions). Patients were more likely to be discharged within 12 months (odds ratio 1.7; 0.9–3.3, P = 0.11). There were no significant changes detected in patient outcomes. Overall the new service was probably cost saving to the NHS.
Conclusions. The guideline-based open access investigation service streamlined the process of out-patient referral, resulting in a more efficient service with reduced out-patient waiting times, fewer out-patient and investigation appointments and release of specialist and clinic time.
Keywords. Cluster randomized controlled trial, hospital, out-patient clinics, practice guidelines, referral and consultation.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSupplementary dataReferences |
|---|
In the UK, urology out-patient clinics are struggling to deal with increasing workload and demand for services, and hence to meet performance indicators1,2 and waiting time targets.2,3 Long out-patient waiting times are common. Similar to other out-patient services,4 they are trying to achieve effective and efficient health care in overcrowded, busy clinic settings. Other health care systems are facing similar problems.5–7 ‘One stop’ and ‘open access’ clinics have been cautiously advocated8,9 as a way of improving out-patient services. The premise is that they will provide a more efficient, patient-focused service by reducing the number of out-patient and investigation appointments for patients, releasing specialist and clinic time, and reducing out-patient waiting times. On this basis, the urology department at Grampian University Hospitals NHS Trust (GUHT) instigated guideline-based open access investigation services for two common conditions: lower urinary tract symptoms (LUTS) and microscopic haematuria (MH). It was recognized that the evidence to support this was weak and inconclusive; this study evaluated the effectiveness and efficiency of the new service.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSupplementary dataReferences |
|---|
Intervention
A guideline-based open access investigation service for each of two common urological conditions, LUTS and MH, was introduced in GUHT in August 1995.
Under the existing system, patients usually attended an initial out-patient appointment (either at GUHT or at one of three peripheral out-patient clinics) and at least one further appointment for routine day case investigations at GUHT, prior to a management decision. The new system allowed GPs to refer patients directly from primary care for the day case investigations (open access investigation service) at GUHT using referral guidelines. Patients attending the open access investigation service would have all their routine investigations and a management plan determined at this single consultation.
Two local multidisciplinary groups developed the guidelines for management and referral to the new services for the study conditions (one group per condition).
Participating GPs were offered a 2-h educational meeting and were mailed a guideline package, including a guideline booklet, quick reference flowchart and structured referral checklists.
Study design
The study was a cluster randomized 2 x 2 balanced incomplete block design.10 All general practices in the Grampian region of Scotland were invited to take part. Those agreeing were randomized to receive the intervention for one condition (the meeting, guideline package and access directly for day case investigations) and to provide control data (did not receive the package, meeting nor were given access to the service) for the other condition (Fig. 1). They were randomized by a statistician independent of the research team using computer-generated numbers (stratified by location and fundholding status). The study was approved by the Grampian Research Ethics Committee.
|
Patient identification and data collection
Referral letters were screened weekly in the urology out-patient department, using keywords, to identify patients referred for LUTS or MH. A pre-intervention cohort of patients was identified from February to July 1995 and a post-intervention cohort from August 1995 to May 1996.
Data on the investigations done and the number of consultations in general practice, before and after referral, were collected from the referring GP, by computer-assisted telephone interviews,11 shortly after patient identification, and by postal questionnaire 12 months after referral.
Data were abstracted from hospital medical records 12 months after referral to determine waiting time to initial appointment, and the date of management decision and discharge from hospital care. Routine data on the waiting times for all new referrals to urology were obtained from GUHT.
Case mix and patient outcome were assessed using the SF36 Mental (MCS) and Physical Component Summary Scales (PCS),12,13 the anxiety component of the Hospital Anxiety and Depression (HADS) Scale14 and the American Urological Association symptom score (AUA)15,16 (LUTS patients only). These data were collected by postal questionnaire following identification and 12 months after referral. Details of patient time and travel to appointments were also collected in the 12 month questionnaire.
All the postal surveys used a single reminder. During data collection and entry, the researchers were blind to the intervention status of the general practices.
Statistical methods and analysis
All outcome measures except the number of referrals, costs and waiting time for all urology referrals were analysed using the patient as the unit of analysis and multilevel modelling using MLWiN version 1.017 to account for the clustering of patients within practices. Exact model specifications are detailed in footnotes to tables.
In all multilevel models, a random effect for the between-cluster variation was fitted. This ensures that any clustering of observations, within practice, would be accounted for in the analysis.
Analysis was conducted on the combined data set from the two conditions. The effect of the intervention is the trend over time (pre- to post-), between intervention and control. To test for a differential effect of the intervention, across the two study conditions, a three-way interaction term was fitted. Where a statistically significant intervention effect was detected, between the two conditions (P < 0.05), estimates of the intervention effect are provided for each condition separately. When no statistically significant effect of the intervention was detected across the two conditions, the intervention effect for the combined data has been presented.
The effects of the intervention on the number of referrals (practice level data) and cost data were analysed at general practice level with suitable summary statistics (difference in number of referrals from pre- to post-intervention period and arithmetic mean cost, respectively18). In these analyses, cluster means were weighted by cluster size,19 and confidence intervals for cost data were constructed using a non-parametric bootstrap implemented in STATA version 6.0.20,21
Waiting times for all urology referrals were adjusted for the number of new appointments and clinic sessions (to account for a waiting list initiative during the period of the study) and pre- and post-intervention were compared using an unpaired t-test.
The original sample size (405 patients with LUTS and 270 patients with MH) was calculated at a 5% significance level to give 80% power to detect a difference between groups of greater than a third of a standard deviation of continuous variables (time and costs), a change of less than 5 points of the HADS scale and 5–10 points on all other scales of the SF36. This sample size was not inflated to account for cluster randomization by general practice. Therefore, for each outcome considered, the effect of the intervention, 95% confidence intervals (CIs) and the intracluster correlation coefficients (ICCs) have been presented. Given the pragmatic nature of the study, analyses were undertaken on an ‘intention to treat’ basis.
Economic analysis
The resource implications of the intervention (developing and disseminating referral guidelines and any extra costs of set up and running the open access service) were based upon valuation of staff time, travel costs, and printing and stationery costs (at 1997 prices).
Cost differences in NHS direct costs (general practice and hospital management), and patient travel due to the intervention were calculated from the study data sets using unit costs from available literature2225 and the Finance Department of GUHT. The intervention costs reflect the study costs based on a 3-year life span and applying a 6% discount rate for annuitization. Sensitivity analyses based on the cost estimates are also reported.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSupplementary dataReferences |
|---|
Recruitment and characteristics of participating practices
Seventy-six (84%) of 90 practices in the Grampian Region agreed to participate in the study (Fig. 1) and were randomized. During the study, we were not confident that all referrals to two of the peripheral clinics were identified. Data from 10 practices that usually referred to those clinics were excluded from the analysis to avoid bias occurring if we identified only their open access referrals. The practice characteristics of the intervention groups are shown in Table 1. GPs from 11 out of 30 practices for the LUTS intervention and 14 out of 36 practices for the MH intervention attended the educational meetings.
|
Identification of patients
Fifty-five of the 66 included practices made 959 referrals during the study period. The completion rates for data collection and the characteristics of patients referred from intervention and control groups are shown in Supplementary tables 1 and 2, available at Family Practice Online.
General practice management and referral
Forty-eight per cent (143/297) of referrals from the intervention group in the post-intervention cohort were through the guideline-based open access service (Supplementary table 1); this was the equivalent of 
350 out-patient appointment slots being vacated over a 12 month period. No statistically significant changes in the mean difference in number of referrals were identified (Supplementary table 3).
A ‘compliance’ score for each referral was calculated based on the number of guideline-recommended investigations done before referral. The score ranged from 0 to 5 (one per recommended investigation out of a possible five). Following the intervention, the mean compliance score was significantly improved in the intervention group with a difference of 0.5 investigations (95% CI 0.2–0.8). There were no significant changes detected in the number of general practice consultations per patient before or after referral. The case mix of referred patients was similar across the intervention and control group (Table 2).
|
Hospital management
The intervention reduced waiting times from referral until initial hospital appointment by 30% (95% CI 11–45%) for LUTS patients. In MH patients, no effect of the intervention was identified. For both conditions, intervention group patients were 5.8 times (95% CI 2.9–11.5) more likely than control patients to have a management decision at their initial appointment. Twelve months following referral, intervention group patients were almost twice as likely to have been discharged from consultant care, although this did not reach statistical significance [odds ratio (OR) = 1.7, 95% CI 0.9–3.3] (Table 3).
|
Patient outcome
There were no significant changes detected in patient outcomes (Supplementary table 4).
Waiting times for all urology referrals
Before the intervention, the average waiting time was 24.3 weeks compared with 13.3 weeks after the intervention, a difference of 11 weeks (95% CI 7.1–15.0 weeks) (adjusted for clinic sessions and new referrals) (Fig. 2).
|
Economic evaluation
The cost of the intervention over the study period was £3782 for LUTS and £4181 for MH, equivalent to £22.25 and £32.92 per patient, respectively. This is likely to be an overestimate of the true cost of the intervention, as the most extreme monetary values were used in the valuation of guideline development time, i.e. actual earnings and leisure equivalent (43% of earnings).26 Table 4 shows there were savings in total NHS direct costs of £69.30 for LUTS and £33.20 for MH patients (per patient), which, although potentially important, were not statistically significant (Table 4 and Supplementary table 5). However, the hospital costs for MH gave rise to significant cost savings when considered as a separate variable (£44.79, 95% CI £16.76–£70.14). Removing the per patient cost of the intervention from the total direct cost gives rise to net cost savings of £47.05 in the LUTS group and £0.28 in the MH group (per patient). Thus, under this scenario, the guideline-based open access service gave rise to NHS direct cost savings during the study period of £16 000. Hence, with intervention costs of £7963, the net cost savings during the study period were £8037. A sensitivity analysis using the upper and lower bounds of the total NHS direct cost suggests that the intervention could result in net cost savings as great as £33 421 or net costs incurred of £21 327 (further sensitivity analyses are available from the authors). Patients in both intervention groups incurred less travel costs and less time spent travelling to appointments, 34 minutes less and 20 minutes less in LUTS and MH, respectively, although these did not reach statistical significance (Table 4).
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSupplementary dataReferences |
|---|
We hypothesized that the guideline-based open access service would streamline the process of hospital care by increasing the proportion of patients referred with all relevant general practice investigations completed (due to increased compliance with referral guidelines). This was expected to reduce out-patient waiting times, provide a management decision earlier, complete hospital care sooner and reduce hospital management costs. However, we were concerned that the service might unacceptably increase general practice workload. We did not anticipate any major changes in patient outcomes as patients received the same elements of care even if these were packaged differently.
In general, the new service achieved these benefits with no evidence of negative consequences. The new system appeared acceptable to GPs who increased their compliance with referral guidelines. It was associated with reduced out-patient waiting times for the study condition referrals and all urological referrals. Patients referred through the new system were more likely to receive a management decision at the first hospital appointment and be discharged from consultant care 12 months following referral. Whilst the study demonstrated no differences in clinical outcome measures, previous studies suggest that patients value improvements in service delivery such as reductions in waiting times.27
Overall, the new service was probably cost saving to the NHS largely due to reductions in hospital management costs. Such cost savings alongside the resulting ‘process’ benefits suggest that the intervention was cost-effective. Data analysis assumed that missing data were missing completely at random, hence the most conservative approach of complete case analysis was used in this study. Further analysis of the cost data will explore alternative approaches to handling missing data. In addition to this, the costs of developing and disseminating the guidelines were valued using the highest values. As a consequence, the development and dissemination costs and resulting direct cost savings are likely to have been over- and underestimated, respectively, and therefore it is likely that the estimated cost savings are highly conservative.
This study is, to our knowledge, the first rigorous evaluation of an open access urological investigation service. Our study design was a 2 x 2 balanced incomplete block design using cluster randomization at the level of the general practice to minimize the risk of contamination.28 The majority of general practices within a single region participated. The study design has a number of advantages for studies of professional behaviour change. First, all practices receive similar intensity of interventions and data collection; as a result, any Hawthorne effect (the non-specific benefit of participating in research) is equalized across the two groups and accounted for within the analysis.29 Secondly, the effects of the intervention are tested across two study conditions, increasing the generalizability of the study findings.29
However, control patients also experienced a reduction in waiting time. This was partly because of the increase in the available number of new out-patient slots as intervention group patients referred to the guideline-based open access service bypassed the initial out-patient appointment. This dilutes the effect of the intervention. Thus the effects found in this study are likely to be underestimates of the true effect of the intervention. The only design that would avoid this problem would be a randomized trial where individual hospital clinics are randomized to open access or traditional services; however, the logistical and costs considerations of such a trial would be substantial.
We excluded from the analysis practices that usually refer patients to two peripheral clinics as we were not certain we were identifying all their referrals. We were concerned that this could introduce bias, as we would be identifying all the referrals to the new service at GUHT but failing to identify their referrals to the peripheral clinics, thus biasing the study in favour of the intervention. Although the removal of these practices minimized bias, it may reduce the power of the study.
There is increasing interest in the concept of seamless health care across the primary–secondary care interface9 and interventions to reduce patient waiting times. One stop and open access clinics have been promoted as one approach to achieve this. This is one of the first rigorous evaluations of an open access clinic. The results of this study suggest that, within a single clinical service for carefully chosen conditions, open access clinics with management and referral guidelines can result in benefits for patients referred to the clinic. They can also result in additional benefits to patients with other conditions referred to traditional out-patient services. This service model could be introduced at modest cost in other centres. It may also provide a model for other types of referral where GPs are referring for specific investigations (e.g. endoscopy). In summary, this study demonstrated that the introduction of a guideline-based open access service streamlined the referral process and was probably cost saving.
|
Supplementary data |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSupplementary dataReferences |
|---|
Supplementary data are available at Family Practice Online.
|
Acknowledgments |
|---|
We are grateful to all the participating general practices and hospital staff. We would like to acknowledge the contributions of David Torgerson to the design of the study, Robyn Webber for clinical input into the design of data collection tools, Alison MacDonald and Ian Ronald for data collection, and Julie Barber for providing advice and the statistical program for bootstrapping. The project, the Health Services Research Unit and the Health Economics Research Unit are funded by the Chief Scientist Office of the Scottish Executive Department of Health. The views expressed are those of the authors and not necessarily the funding body.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionSupplementary dataReferences |
|---|
1 NHS Executive, Department of Health. NHS Performance Assessment Framework. London: Department of Health, 1999.
2 Department of Health. The Patients Charter and You: A Charter for England. Leeds: Department of Health, 1995.
3 Department of Health. Your Guide to the NHS. Leeds: Department of Health, 2001.
4 Sellu D. Have we reached crisis management in outpatient clinics. Br Med J 1998; 316: 635–636.
5 Sheldon T. Dutch coalition tackles waiting lists. Br Med J 1998; 316: 955.
6 Sheldon T. Dutch waiting lists increase despite £36m campaign. Br Med J 2000; 321: 530.
7 Dorozynski A. French health care costs rising rapidly. Br Med J 2000; 321: 528.
8 Scottish Executive, Health Department. Our National Health, a Plan for Action, a Plan for Change. Edinburgh: Stationary Office, 2000.
9 Scottish Office, Department of Health. Acute Services Review Report. Edinburgh: Stationary Office, 1998.
10 Cochran WG, Cox GM. Experimental Designs, 2nd edn. New York: Wiley, 1992.
11 Harris DM, Grimshaw JM, Lemon J, Russell IT, Taylor RJ. The use of a computer-assisted telephone interview technique in a general practice research study. Fam Pract 1993; 10: 454–458.
12 Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 Health Survey Manual and Interpretation Guide. Boston (MA): The Health Institute, New England Medical Centre, 1993.
13 Ware JE, Kosinski M, Keller SD. SF-36 Physical and Mental Health Summary Scales: A User's Manual. Boston (MA): The Health Institute, New England Medical Centre, 1994.
14 Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand 1983; 67: 361–370.[ISI][Medline]
15 Fowlers FJ, Barry MD. User's Manual: Prostatism TyPESM Specification. Maine: Health Outcomes Institute, 1993.
16 Barry MJ, Fowler FJ, O'Leary MP et al. The American Urological Association symptom index for benign prostatic hyperplasia. J Urol 1992; 148: 1549–1557.[ISI][Medline]
17 Goldstein H, Rasbash J, Plewis I et al. A User's Guide to MlwiN, Version 1.0, Jan 1998. Multilevel Models Project. London: University of London, Institute of Education, 1998.
18 Barber JA, Thomson SG. Analysis and interpretation of cost data in randomised controlled trials: a review of published studies. Br Med J 1998; 317: 1195–1200.
19 Bland JM, Kerry SM. Weighted comparison of means. Br Med J 1998; 316: 129.
20 Stata Corp. Stata Statistical Software: Release 6.0. College Station (TX): Stata Corporation, 1999.
21 Barber JA, Thomson SG. Analysis of cost data in randomised trials: an application of the non-parametric bootstrap. Stat Med 2000; 19: 3219–3236.[CrossRef][ISI][Medline]
22 British Medical Association, Royal Pharmaceutical Society of Great Britain. British National Formulary, Number 32, September 1996. London: BMA and Pharmaceutical Press, 1996.
23 Information and Statistics Division. Scottish Health Service Costs 1995/96. Edinburgh: Information and Statistics Division, 1996.
24 Netten A, Dennet J. Unit Costs of Health and Social Care. Kent: Personal Social Services Research Unit, University of Kent, 1996.
25 The Automobile Association. AA Technical Information, Motoring Costs 1998. Basingstoke (UK): The Automobile Association, 1998.
26 Department of Transport. Value of Time and Vehicle Operating Costs for 1989. COBA 9 Manual, Annex II, Section 8.2, 8.3–8.10. London: Department of Transport, 1989.
27 Propper C. The disutility of time spent on the United Kingdom's National Health Service waiting lists. J Hum Resources 1995; 30: 677–700.[CrossRef]
28 Grimshaw JM, Campbell MK, Eccles MP, Steen IN. Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract 2000; 17: S11–S18.
29 Cook TD, Campbell DT. Quasi Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally, 1979.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Julian, N. J Naftalin, M. Clark, A. Szczepura, A. Rashid, R. Baker, N. Taub, and M. Habiba An integrated care pathway for menorrhagia across the primary-secondary interface: patients' experience, clinical outcomes, and service utilisation Qual. Saf. Health Care, April 1, 2007; 16(2): 110 - 115. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. J Wilson, N. Torrance, J. Mollison, M S. Watson, A. Douglas, Z. Miedzybrodzka, R. Gordon, S. Wordsworth, M. Campbell, N. Haites, et al. Cluster randomized trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk Fam. Pract., October 1, 2006; 23(5): 537 - 544. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||