Family Practice Vol. 21, No. 2, 213-218
Family Practice Vol. 21, No. 2 © Oxford University Press 2004, all rights reserved.
Article |
An alternative trial design to overcome validity and recruitment problems in primary care research
a Department of Epidemiology and c Department of General Practice, Maastricht University, b Department of Medical Psychology, UMC Nijmegen, d Department of Physiotherapy, Hogeschool Zuyd and e Department of General Practice, Institute for Research in Extramural Medicine, VUmc The Netherlands
Correspondence to MJH Huibers, Department of Medical, Clinical and Experimental Psychology, Maastricht University, PO Box 616, 6200 MD Maastricht, The Netherlands; E-mail: m.huibers{at}dmkep.unimaas.nl
Received 24 February 2003; Revised 2 September 2003; Accepted 3 November 2003.
Huibers MJH, Bleijenberg G, Beurskens AJHM, Kant IJ, Knottnerus JA, van der Windt DAWM, Bazelmans E and van Schayck CP. An alternative trial design to overcome validity and recruitment problems in primary care research. Family Practice 2004; 21: 213–218.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background. Although the randomized controlled trial is widely accepted as the best design to investigate new interventions, conducting a trial in primary care may present researchers with many methodological problems.
Objective. Our aim was to present an alternative trial design to overcome internal validity and recruitment problems.
Methods. In a randomized controlled trial, fatigued employees absent from work were selected among the population of an occupational health service in the South of The Netherlands. Patients randomly assigned to the experimental condition received cognitive behavioural therapy by a research GP near their home address, whereas patients in the control group received no intervention. We describe our considerations for building an alternative design. Research GPs and patients were recruited separately for the study. The pre-randomization design was applied.
Results. Nine research GPs performed all the interventions. Seventy-six experimental patients and 75 control patients were selected for study participation. Of these, only six patients in the experimental group and seven patients in the control group withdrew from the study at some point during follow-up.
Conclusion. Results suggest that recruitment and randomization procedures in the alternative design served their purpose well. The alternative design proposed here might have several advantages compared with conventional trial procedures. However, our design is not widely applicable and there are ethical aspects involved that should be considered. Researchers should address their creativity when trying to minimize the problems they may encounter in designing a study.
Keywords. Informed consent, patient recruitment, primary health care, randomized controlled trial, research design.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The randomized controlled trial is widely accepted as the best design to investigate new interventions. In order to run a randomized trial successfully, it is vital to use effective and practical research strategies.1 However, researchers may face many methodological problems when a trial is conducted in a general practice setting.1,2 Here, we will focus on a few problems that relate to recruitment and randomization procedures.
The recruitment of GPs and patients is a general concern in primary care research. Obtaining the voluntary participation of GPs in research projects has become a major challenge.2–7 Lack of available time is the most cited reason for the reluctance among GPs to become involved in research activities. Once a sufficient number of GPs has agreed to become active in the research project, a phenomenon referred to as Lasagna's Law8 almost inevitably emerges. This means that researchers and clinicians invariably overestimate the number of patients available for study participation so that recruitment of eligible patients falls short of the expectations.2 Factors that might explain this phenomenon are several: eligibility criteria are often overly strict and do not reflect the prevalence of the disorder on which sample size calculations are based; recruiting physicians are not always motivated or able to enrol eligible patients;2,9,10 and eligible patients are not always willing to participate. Ultimately, studies might be delayed, suspended or even prematurely terminated due to unsuccessful recruitment.10
Recruitment of patients becomes virtually impossible when interventions are complex and, as a consequence, only a small number of GPs can be trained and supervised in performing the intervention. If patients are then to be recruited by their own GPs, as is customary in primary care research, only a small number of patients can be approached for study participation. These recruitment problems become even worse if the disorder under investigation is rare and/or if the information needed to identify eligible patients is not directly available to the GP.
In some instances, the randomization procedure is likely to cause internal validity problems. In most primary care intervention studies, patients are recruited by their own GPs and randomly allocated to one of the treatment conditions after informed consent. As widely used as it is, this conventional approach has problematic features if ‘usual care’ or ‘no treatment’ is the control condition.11
With ‘usual care’ or ‘no treatment’ as the control condition, it can become very difficult to find patients who are willing to be randomized, especially if the experimental condition is an appealing option to patients.11 More importantly, disappointment because of control group allocation among those who do agree with randomization might lead to (selective) withdrawal from the study, non-compliance11 or contamination12 if control patients, inspired by their research participation, start to look for treatment elsewhere.
Researchers foreseeing recruitment, internal validity or other methodological problems should try and minimize these problems. The question here is how. Obviously, there is no single answer to this question. Instead, we wish to share our particular experiences by presenting an alternative study design that enabled us to overcome some of the difficulties mentioned here. We conducted a trial in primary care in which we assessed the efficacy of cognitive behaviour therapy by GPs for persistent fatigue. In this paper, our trial will serve as case illustration. Following the description of the design, the results of the recruitment and randomization procedures will be presented and discussed.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Starting point for building the design
Several considerations were taken into account when we started designing the study. First, we wished to conduct a randomized trial with an experimental and a control condition and a follow-up period of 12 months. Since there is no standard treatment for persistent fatigue and our intervention is new and unfamiliar to GPs, we decided to assess the efficacy (can it work under ideal circumstances?) of the intervention rather than the effectiveness (does it work in ordinary settings?).13 We therefore chose to create optimal conditions for the experimental intervention in order to assess its potential effect on the one hand, and merely to follow-up the natural course in the control group on the other hand.
Choosing efficacy over effectiveness presented us with two problems. Firstly, the rather complex experimental intervention consists of five to seven 30 min sessions by a GP. In order to administer the intervention under the best circumstances, GPs should be extensively trained and supervised throughout the trial. Our time and resources enabled us to train and supervise up to 10 GPs. If these GPs had to recruit all patients in their own practice, only a small number of patients could be enrolled in the study. Also, we feared that classical randomization of selected patients after informed consent might lead to selective withdrawal and contamination of the trial conditions, which could pose a serious threat to the internal validity of the study. Consequently, we were challenged to design alternative recruitment and randomization procedures.
Recruitment of GPs and patients
We decided to recruit patients and GPs separately. GPs were recruited among the population of GPs in the Southeast of The Netherlands. We aimed to deploy up to 10 research GPs whose practices were equally distributed across the recruitment area. We decided that research GPs would only become active if a sufficient number of patients from their area could be enrolled in the study. However, instead of these research GPs recruiting their own patients for the trial, patients were recruited in collaboration with a local occupational health service (OHS) that monitors a working population of 80 000 employees who live in the recruitment area. We needed 150 patients in total (75 patients per group) to detect a clinically relevant difference on the main outcome measure, the Checklist Individual Strength (CIS).14–16 Eligibility criteria for the trial were severe fatigue (measured with the CIS) for >4 months as one of the main health problems and complete absenteeism from work for >5 weeks.
On a monthly basis, employees monitored by the OHS who were on sick leave for >2 weeks were sent study information by the OHS, irrespective of the reason for sick leave, followed by a ‘blind’ reminder 2 weeks later. The recruitment letter included a screening list that could be sent back to the research team. This recruitment route secured the privacy of selected employees and enabled us to identify eligible patients in an early stage of absenteeism. Based on the screening lists sent back to us, we contacted potential candidates who were willing to participate. Eligible patients were invited to visit the university research centre.
Pre-randomization design
As stated earlier, we foresaw internal validity problems such as contamination and selective withdrawal if patients were to be randomized after informed consent. Therefore, we used the pre-randomization design,12 also referred to as randomized consent or Zelen design.17 In the original Zelen design, informed consent is obtained after randomization and only from patients in the experimental group (see Figure 1A) in order to prevent selective withdrawal and contamination during the trial. Although the use of this design has raised many objections among researchers,18 others have suggested it as an alternative for classical randomization if the experimental intervention is attractive to all participants and the control group will receive standard (or no) treatment.11
|
For the purpose of our study, we used the adapted pre-randomization design (see Figure 1B).12 Informed consent was obtained from participants in both groups. However, randomization took place before detailed information about the study was provided, and participants allocated to one group were kept blind to the randomization procedure and thus, the existence of the other group. In the initial recruitment letter sent by the OHS, we invited fatigued employees absent from work to participate in a ‘health-related’ study, without mentioning the experimental intervention. At their visit to the research centre, patients who were selected for study participation and randomized were handed an envelope containing written information about the allocated condition only: patients in both groups learned that they would be followed-up over the next 12 months, while patients in the experimental group in addition were offered the experimental intervention. Experimental patients who entered the study were assigned to a research GP near their home addresses and a first appointment was made by the researchers. Regular GPs of patients in both groups were contacted, informed about the study participation of their patients and asked to register the medical consumption over the following 4 months. The medical ethics committee of Maastricht University approved the study protocol.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Recruitment of GPs
Figure 2 presents the results of the recruitment of GPs that started in March 1999. Of the 1030 GPs addressed by letter, 90 (8.7%) responded. Based on their geographical position, we invited 30 GPs to attend our training sessions. Twenty-two GPs attended the first training session and 15 GPs also attended the second training session, which was a requirement to become active as a research GP. Of these 15 trained GPs, we eventually deployed nine research GPs—based on the local intake of patients—who performed all the interventions.
|
Recruitment of patients
Figure 3 presents the results of the recruitment of patients that started in November 1999. A total of 8736 employees who were absent from work for >2 weeks were addressed within the course of 20 months (November 1999–June 2001). A total of 4242 employees (48.5%) responded to the initial recruitment letter. Of the 2454 employees absent from work for >2 weeks who were willing to participate in the study, only 195 (7.9%) met our strict eligibility criteria (i.e. severe fatigue, complete absenteeism for >5 weeks) after screening and were invited to visit the research centre. Eventually, 151 employees entered the study.
|
Refusal of treatment after randomization
As a result of the pre-randomization procedure, a phenomenon we refer to as ‘refusal of treatment after randomization’ emerged: since patients were not informed beforehand that an experimental intervention could be offered to them, five patients refused the intervention once they were informed about the experimental condition (see Figure 3). Since they were randomized into the study, these patients were encouraged to take part in the follow-up assessments and were included in intention-to-treat analyses.
Withdrawal rates
Of the 151 patients who entered the study, 13 (8.6%) withdrew from the study at some point during the 12-month follow-up period (see Figure 3). Data were lost for two experimental patients (1.3%) and three control patients (2%) at the 4-month assessment, for three experimental patients (2%) and six control patients (4%) at the 8-month assessment, and for six experimental patients (4%) and seven control patients (4.6%) at the 12-month assessment. Of the five experimental patients who refused the intervention directly after randomization, two patients did not complete data collection.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Results from the recruitment and randomization procedures applied in our study suggest that the developed research design was successful. By approaching a considerable number of GPs for study participation, we were able to select a small group of highly motivated research GPs who were willing to undergo the extensive training and supervision and perform complex interventions. It also seems likely that we would have selected fewer patients in the same period of time if conventional recruitment procedures had been used, although Lasagna's Law8 dictates that no reliable estimate of the number of patients potentially recruited in general practice can be made. The non-selective and relatively low withdrawal rates throughout the trial suggest that the use of pre-randomization had the desired effect. However, in the absence of similar studies using classical randomization, we can merely speculate what would have happened if our randomization procedures had been different. Regular GPs did not voice any complaints regarding the study participation of their patients.
Since we only by-passed recruitment by a physician, the influence of Lasagna's Law was not eliminated in our trial. According to the Maastricht Cohort Study,19 an affiliated large-scale prospective study of fatigue among 12 000 employees, the point prevalence of persons in the working population meeting all our eligibility criteria is 0.8%. Over a period of 20 months, however, we were only able to select 151 prevalent and incident cases (0.2%) among the 80 000 employees of our source population. The intake of patients varied between four and 16 patients per month, another illustration of the fact that recruitment also remained a challenge in our design.
The trial design proposed here has several advantages compared with conventional trial procedures: researchers are encouraged to open up new recruitment markets, e.g. through collaboration with the OHS in our example, through secondary care services or newspaper advertisements; researchers maintain control over all recruitment procedures, which contributes to creating optimal and efficient study conditions; recruitment is facilitated of patients who are otherwise more difficult to identify due to strict eligibility criteria, low prevalence of the disorder and/or the fact that information needed for detection is not directly available to the GP; and internal validity problems likely to occur due to patient preference (i.e. selective withdrawal, selective non-compliance and contamination) are reduced.
However, in weighing these advantages against the use of conventional trial procedures, one should keep in mind that we tailored the design to our particular research needs. Not all of the problems that we encountered are universal problems in primary care research, and the solutions we have advocated here are only applicable in certain circumstances. The proposed design is of no use if recruitment in the general practice is essential to the aim of the study or when the deployment of a research GP instead of the regular GP is considered to be a threat to the external validity. More importantly, the decision to use a pre-randomization design should not be made light heartedly.
The pre-randomization design has been the topic of an unresolved debate concerning the ethical aspects of informed consent procedures.20 As stated in the recently published Research Governance Framework, informed consent of patients is at the heart of ethical research and should be taken as agreed with the ethics committee.21 In 1999, the Health Council in The Netherlands reported on the ethical aspects of informed consent.20 It was concluded that application of a pre-randomization design is only ethically justified if all of the following conditions are met: the study has a clinically relevant objective; it is likely that the study will lead to important new insights; an internally valid evaluation of the effects is impossible using classical randomization and informed consent procedures; and use of the consent procedures bears no potential harm to participants. Knottnerus added to these criteria the necessity to offer at least usual care to patients in the control group and the approval of an independent medical ethics committee.12 An important issue that deserves special attention is the perspective of participants: how do patients feel about participating in a pre-randomized trial?
We decided to apply a pre-randomization design because we feared the lack of blinding would pose a serious threat to the internal validity of our study, i.e. would lead to contamination of the research groups and selective withdrawal from the study. We made sure that all of the conditions for using the design were met, including approval from a medical ethics committee. Until now, we have not been able to assess how participants felt about our randomization procedure because we are still awaiting a final follow-up round.
However, it is important to realize that use of the pre-randomization design is by no means justified when the objective is merely to speed up the recruitment of patients. We used a separate strategy to tackle the recruitment problems we foresaw, something that should not be confused with our choice to use a pre-randomization design.
Despite these limitations, our trial design may present an interesting starting point for researchers who are challenged to be creative in dealing with the methodological and practical problems they encounter. In that sense, our design may serve as a basis for some efficacy studies and is less applicable when the effectiveness of an intervention is evaluated.
|
Acknowledgments |
|---|
The authors wish to thank Professor Paul Knipschild for his useful comments on the manuscript. This research project was funded by the Health Research and Development Council (ZorgOnderzoek Nederland), The Netherlands. The recruitment of patients was made possible by the donation of services by ‘Arbo Unie Limburg’ (Occupational Health Service Limburg).
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1 Ward E, King M, Lloyd M, Bower P, Friedli K. Conducting randomized trials in general practice: methodological and practical issues. Br J Gen Pract 1999; 49: 919–922.[Web of Science][Medline]
2 Van der Windt DAWM, Koes BW, Van Aarst M, Heemskerk MAMB, Bouter LM. Practical aspects of conducting a pragmatic randomised trial in primary care: patient recruitment and outcome assessment. Br J Gen Pract 2000; 50: 371–374.[Web of Science][Medline]
3 Huibers MJH, Van der Windt DAWM, Boeke AJP. De deelname van huisartsen aan wetenschappelijk onderzoek: problemen en oplossingen [Participation of general practitioners in primary care research: problems and solutions]. Huisarts en Wetenschap 2002; 45: 454–458.
4 Borgiel AE, Dunn EV, Lamont CT et al. Recruiting family physicians as participants in research. Fam Pract 1989; 6: 168–172.
5 Levinson W, Dull VT, Roter DL, Chaumenton N, Frankel RM. Recruiting physicians for office-based research. Med Care 1998; 36: 934–937.[CrossRef][Web of Science][Medline]
6 Gray RW, Woodward NJ, Carter YH. Barriers to the development of collaborative research in general practice: a qualitative study. Br J Gen Pract 2001; 51: 221–222.[Web of Science][Medline]
7 Heywood A, Mudge P, Ring I, Sanson-Fisher R. Reducing systematic bias in studies of general practitioners: the use of a medical peer in the recruitment of general practitioners in research. Fam Pract 1995; 12: 227–231.
8 Harris EL, Fitzgerald JD (eds). The Principles and Practices of Clinical Trials. Edinburgh: E & S Livingstone; 1970.
9 Peto V, Coulter A, Bond A. Factors affecting general practitioners’ recruitment of patients into a prospective study. Fam Pract 1993; 10: 207–211.
10 Tognoni G, Alli C, Avanzini F et al. Randomised clinical trials in general practice: lessons from a failure. Br Med J 1991; 303: 969–971.
11 Schellings R, Kessels AGH, Ter Riet G, Sturmans F. The Zelen design may be the best choice for a heroin-provision experiment. J Clin Epidemiol 1999; 52: 503–507.[CrossRef][Web of Science][Medline]
12 Knottnerus JA. Gezondheidszorgonderzoek in extramurale settings [Health care research in extramural settings]. In: Ten Have HAMJ, Blijham G, Engberts DP (eds) Ethiek en Recht in de Gezondheidszorg [Ethics and law in health care]. Deventer: Kluwer; 1997; pp. 151–198.
13 Fletcher RH, Fletcher SW, Wagner EH. Clinical Epidemiology: The Essentials. Baltimore (MD): Williams & Wilkins; 1996.
14 Beurskens AJHM, Bültmann U, Kant IJ, Vercoulen JHHM, Bleijenberg G, Swaen GMH. Fatigue among working people: validity of a questionnaire measure. Occup Environ Med 2000; 57: 353–357.
15 Vercoulen JHMM, Alberts M, Bleijenberg G. De Checklist Individual Strength (CIS). Gedragstherapie 1999; 32: 31–36.
16 Vercoulen JHMM, Swanink CM, Fennis JF, Galama JM, Van der Meer JW, Bleijenberg G. Dimensional assessment of chronic fatigue syndrome. J Psychosom Res 1994; 38: 383–392.[CrossRef][Web of Science][Medline]
17 Zelen M. Randomised consent designs for clinical trials: an update. Stat Med 1990; 9: 645–656.[Web of Science][Medline]
18 Ellenberg SS. Randomization designs in comparative clinical trials. N Engl J Med 1984; 310: 1404–1408.[Web of Science][Medline]
19 Bültmann U, Kant IJ, Kasl SV, Beurskens AJHM, Van den Brandt PA. Fatigue and psychological distress in the working population: psychometrics, prevalence and correlates. J Psychosom Res 2002; 52: 445–452.[CrossRef][Web of Science][Medline]
20 Gezondheidsraad (Netherlands Health Council). Informed consent en prerandomisatie [Informed consent and prerandomisation]. The Hague: Netherlands Health Council (Gezondheidsraad); 1999.
21 Department of Health. Research Governance Framework for Health and Social Care, 2nd edn. London: Department of Health; 2003.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  L Sharp, S C Cotton, L Alexander, E Williams, N M Gray, and J M Reid Reasons for participation and non-participation in a randomized controlled trial: postal questionnaire surveys of women eligible for TOMBOLA (Trial Of Management of Borderline and Other Low-grade Abnormal smears) Clinical Trials, October 1, 2006; 3(5): 431 - 442. [Abstract] [PDF]
|
|
|
|
 |
|
 S. C Dean, C. E Harper, F. P Cappuccio, E. Rink, C. Dirckx, J. Arnout, F. Zito, L. Iacoviello, and on behalf of the European Collaborative Group of t The challenges of cross-national research in primary health care across Europe Fam. Pract., June 1, 2005; 22(3): 341 - 346. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||