Family Practice Vol. 20, No. 6, 675-681
© Oxford University Press 2003, all rights reserved
Article |
A qualitative evaluation of implementing a randomized controlled trial in general practice
Department of Epidemiology, Statistics and Public Health, University of Wales College of Medicine, Heath Park, Cardiff CF14 4XN, a Department of General Practice, University of Wales College of Medicine, Llanedeyrn Health Centre, Cardiff CF23 9PN, UK and b Centre for the Evaluation of Medicines, St Joseph's Healthcare, 105 Main Street East, Level P1, Hamilton, Ontario L9N 1G6, Canada
Correspondence to Hayley Prout; E-mail: prouth{at}cardiff.ac.uk
Received 21 October 2002; Revised 9 June 2003; Accepted 14 July 2003.
Prout H, Butler C, Kinnersley P, Robling M, Hood K and Tudor-Jones R. A qualitative evaluation of implementing a randomized controlled trial in general practice. Family Practice 2003; 20: 675–681.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodResultsDiscussionReferences |
|---|
Background. For findings of randomized controlled trials in primary care to be applicable, both the sample of clinicians implementing the trial and the recruited patients should be as representative as possible. The processes of conducting trials should be made ‘user-friendly’ to clinician investigators in order to maximize their participation in research. Formal evaluations of trial implementation are unusual. This study reports clinicians' perspectives on acting as a clinician investigator in a randomized controlled trial (the SAVIT study) in general practice.
Objective. Our purpose was to explore clinicians' accounts of taking part in a randomized controlled trial in which subjects were recruited opportunistically during general practice consultations.
Method. Individual semi-structured interviews were conducted with nine GPs and one practice nurse practising in the Bro Taf area of South Wales who recruited children into the SAVIT study. A structured interview guide was used and data were analysed using the qualitative method of pattern coding.
Results. Major emerging themes included recruitment difficulties and concerns about the safety of the study medication. Participants also outlined positive aspects of the study (clarity and simplicity of the study, potential benefits to clinicians and patients and study team follow-up of recruited patients). Recommendations for possible improvements in study implementation included the simplification and reduction of patient reading materials and improved presentation of study materials.
Conclusions. Difficulty in recruiting patients was the most frequently mentioned problem by clinician investigators. Insufficient time in the consultation was perceived as the main barrier. Ingredients of successful trial implementation include good organization, simple documentation and study procedures, and the ability to allay concerns about patient safety. Findings from this evaluation may assist researchers in the design and implementation of future community-based randomized controlled trials.
Keywords. Evaluation, general practice, qualitative methods, randomized controlled trials, trial implementation.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
As general practice generates more evidence from real world settings about important common conditions, there is increasing pressure on busy clinicians to act as investigators and recruit patients into research studies. Patient samples are unlikely to be representative if representative samples of clinicians cannot be recruited to act as clinician investigators in community-based research studies. Researchers therefore need to ensure that implementing research involves the minimum burden to clinician investigators if studies are to provide unbiased answers to questions, and practices are to remain committed to participating in research. However, few studies share information on lessons learnt about the process of research implementation in primary care. Indeed, the overview of Prescott and colleagues1 of factors that limit the quality, number and progress of randomized controlled trials (RCTs) highlights the small number of trials that have been carried out in primary care.
Recruitment bias appears to be a particular concern. Little and colleagues2 report that high recruiting doctors in their trial of management of acute otitis media in children were more likely to recruit younger children and children with ear drum perforations. Moreover, Zermansky and colleagues3 express concern that participants in their trial of a medication review of older people were not typical of the practices' eligible elderly populations.
GPs' attitude to the research is also a concern. Tognoni and colleagues4 discuss lessons learnt from a failed RCT regarding treatment of isolated systolic hypertension in elderly people. Failure was due mainly to GPs' unsatisfactory attitude to the study. Murphy and colleagues5 highlight the importance of creating a sense of collective ownership between researchers and participants when negotiating participation in research in primary care settings.
Lastly, Asch and colleagues6 discuss time pressures on physicians and their staff as a particular reason for declining participation in research, whilst Foy and colleagues7 highlight the lack of evidence-based use of recruitment strategies to RCTs in primary care.
The SAVIT (Suspected Acute Viral Infection Treatment) study8 was a pragmatic double-blinded randomized placebo-controlled trial of an established drug for a new indication. It aimed to assess whether intranasal disodium cromoglycate (DSCG) improved symptoms of upper respiratory tract infection (URTI) in children more than a normal saline control. Toogood9 states that DSCG is a safe drug and, while local irritation may occur, serious side effects are exceedingly rare. A total of 290 children aged between 1 and 12 years were recruited by 54 GPs and one practice nurse based in the South Wales area. Clinicians were reimbursed £30 for recruiting each child. Figure 1 outlines the procedure for recruiting children. A variety of methods to enhance recruitment and interest in the study were used (Box 1).
|
| BOX 1 Methods for enhancing clinician interest in the SAVIT study Frequent nurse and team visits to participating clinicians. Monthly SAVIT study newsletters. Individualized postcards to each GP acknowledging each patient recruited. Waiting room posters informing patients that a study is in progress. Clinician desktop reminder cards, for example computer stickers. Competitions.
|
Previous evaluations of trial implementation have focused on quantitative assessments of processes.1,6,7 We felt that exploring the perspectives of the clinician investigators themselves was of prime importance. We therefore set out to formally evaluate their accounts of implementing the SAVIT study using qualitative research methods.
|
Method |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
We wished to obtain data from clinicians with various level of engagement in the study. Therefore, we assigned them to one of five groups according to the number of children they had recruited (Table 1). Clinicians within each group were then approached for an interview in random order until at least two from each of these groups had been interviewed. Interviews lasted
20 min and were conducted using a semi-structured interview guide (Box 2). This guide allowed for an in-depth exploration of clinicians' views on participating in RCTs in general as well as their attitudes to the study design and implementation of the SAVIT study. Each interview was tape recorded and transcribed. Analysis was carried out using ‘pattern coding’,10 a method of coding segments of data which are developed inductively into a smaller number of themes and constructs. The themes that emerged were similar to the topics covered by the interview guide. The study nurse and an academic GP, who was not previously associated with the SAVIT study, carried out the analysis. Areas of disagreement were discussed and categories and themes identified and agreed by consensus or after discussion with a third researcher.
|
| BOX 2 Interview guide What do you think of the design of the SAVIT study and how it worked for you? Why did you decide to take part in this study? How did you feel about participating as an investigator in a randomized controlled trial? This study was about children, did that raise any particular issues for you? What do you feel worked well in the study? Do you have any suggestions on what could have been improved upon?
|
|
Results |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
Clinician perceptions of the study in general
Overall, clinicians were complementary of the study design, emphasizing its clarity and simplicity.
"It was a super little study, good ideas, very nice paperwork, presented nicely, easy to present to the patient, very little hassle or difficulty afterwards." (Clinician Group 4)
However, several clinicians reported difficulty in identifying eligible patients. Many clinicians reported lack of time for recruitment due to the pressures of a busy surgery, for example:
"I think in any trial ... there's never a good time to pick out patients in the middle of a busy surgery ... and sometimes that can put a bit of pressure on you." (Clinician Group 4)
Furthermore, seasonal influences on the frequency of presentation of children with URTI were identified as a reason for slow recruitment.
Attitude shift and behaviour changes in society, possibly induced by media messages not to expect antibiotic treatment from GPs, were also used to explain slow recruitment, for example:
"I get the impression that there are less kids with colds basically coming to surgery than there used to be. Whether that's our success in educating them ... or whether that's just changes in society, I don't know." (Clinician Group 2)
Moreover, several clinicians proposed that including children in a trial of treatment for URTI was medicalizing the condition, which might encourage inappropriate future consultation. This ambivalence about the main purpose of the study (endeavouring to find an effective treatment for suspected acute viral URTI in children) was apparent in the comments of a number of the clinicians.
Reasons for participating in this study
Clinicians' reasons for participation were for the most part based upon three main sub-themes.
Benefits to clinicians and the patients
Clinicians highlighted the concern they had regarding the lack of effective treatment for common viral illness:
"Because there's no active treatment for viral infection and I mean children get very distressed and miserable don't they and they're crying all the time and it's just something to try and see if it will improve their symptoms over and above Calpol basically." (Clinician Group 5)
Scientific curiosity
Several clinicians were happy to carry out the study simply because of the perceived intrinsic value of participating in research:
"I'm quite keen to know what's going on. I think it's important, so I've got no problems with that." (Clinician Group 2)"I thought it was a scientifically very interesting idea. Super idea. I'm very keen on taking advantage of people's ideas when they suddenly notice something and to be able to build it up and help it come to fruition." (Clinician Group 4)
Feasibility of participating in the trial
Ease of participation was also an appealing aspect, with one clinician stating that previous studies in which he had participated were simply not feasible in the context of time-pressured consultations:
"well, I mean, you've only got so much spare time and you know, if it's taking half an hour a patient, ... but if you've got two in a morning I mean, it makes life just miserable." (Clinician Group 5)
Feelings about participating as an investigator in an RCT
The issue of safety was most commonly highlighted in response to this question. Clinicians were more concerned about whether the use of a placebo was justified compared with the safety profile of the experimental drug. However, many spontaneously pointed out that URTI is a self-limiting illness and therefore it made no difference to them whether the child got the active drug or the placebo.
Feelings about participating as an investigator in a treatment trial involving children
Safety was again of great importance for several clinicians. One clinician pointed out:
"there's always going to be a concern at the back of your mind ... what happens if they went and got worse ... how would the parents respond to that." (Clinician Group 4)
However, it was generally agreed that the majority of parents were happy to take part and there was no cause for concern. Other individual worries included the legitimacy of family relatives or childminders in providing consent for a child to participate in the study as well as the motivation of parents to fill in symptom diaries adequately.
Perceptions of what worked well in the study
Three main opinions were given. The study nurse contacting patients on the day of recruitment and nurse follow-up was most frequently mentioned:
"I think it helps reinforce what you say [because] people will say yes here, and then walking up the road they might have second thoughts about it, ... it gives them a feeling that somebody's looking after them as well I suspect." (Clinician Group 5)
Moreover, initial training sessions were well liked since it allowed the clinicians an opportunity to discuss the trial prior to agreeing to participate.
Regular study newsletters, providing information and news about the study, also proved popular:
"It's interesting to know how it's going generally isn't it, you don't feel you're in isolation." (Clinician Group 5)
A commitment to trying to publish results following completion of the study was valued:
"when I've been approached in the past, one of the things that they've never been able to give a guarantee on is that they would endeavour to publish the results, which you lot did and that's always struck me as very important. I mean, I'd never have been advising patients to volunteer for a trial if the organization running it could just bin the results if they didn't like them." (Clinician Group 4)
Study improvement suggestions
The simplification and reduction of patient reading materials was proposed:
"you could see the patients sort of wanting to sit down and read it [the patient information] properly before they signed the dotted line except that they were conscious like I was of the time factor." (Clinician Group 4)
Written feedback to clinicians on patient well-being following nose-spray use was also proposed. Improved practice organization was also identified as a way of improving recruitment, for example receptionists preferentially directing potential recruits to a participating clinician, scheduling fewer patients on certain days (allowing more time to recruit during surgery), and the dispersal of study materials into all the rooms used by potential recruiters as opposed to having study materials centralized in only one room.
The SAVIT study's influence on clinician's decisions to take part in future studies
Clinicians were generally positive about taking part in future studies. However, several pointed to the commitment needed if agreeing to participate in a study:
"when you agree to be involved in a study, you know how seriously you've got to take it ... I think that makes you appreciate it more and it's not something that should be taken lightly." (Clinician Group 2)
Other individual comments included the need for a more realistic perception by the clinician of the work load involved in participating in a study, better clinician organization and prior discussions with clinician colleagues on who would participate:
"I think that's the lesson that three of us have learned, that three of us are keen and the other two or three are not so keen and so in future I think it would be the keenies that got involved and the keenies who got the payment." (Clinician Group 4)
|
Discussion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
Qualitative investigations of the perspectives of clinicians implementing an RCT in general practice are not common. The main advantage of using qualitative methods is that the voices of the clinician investigators are heard. Understanding their concerns and hearing their suggestions is crucial to the future successful implementation of RCTs in general practice.
Method considerations
A potential weakness of this evaluation is that the trial nurse also acted as study evaluator. Clinicians may have been reluctant to express criticism because of her role in the study and her membership of the study team. Moreover, only those clinicians who had strong feelings about the study, either positive or negative, may have accepted our requests for an interview.
The issue of ‘extras’
The fact that clinicians were often too busy to explain the study during a consultation could be due to many children with URTI arriving as ‘extras’ (urgent, same-day appointments). These consultations are usually particularly time-pressured. If, as one clinician pointed out, it took 10 min to recruit a child to the SAVIT study, this may often be unacceptably long for an ‘extras’ consultation. It should be noted, however, that clinicians were reimbursed for any additional tasks adding to consultation length.
Conflicting interests and ‘buy-in’
Several clinicians were concerned that treating URTI might ‘medicalize’ the condition, encouraging patients to consult for coughs and colds in the future. This might undermine the positive effects of other messages to discourage patients from consulting with these conditions. Interestingly, there appeared to be a difference in initial perceptions of recruiting children with URTI and subsequent experiences in actual recruitment. Clinicians generally predicted that it would be relatively easy to recruit children with URTI because of the large number of children seen with the condition. Many clinicians stated that they would easily recruit 15 children in 1 week, thus requesting extra study materials at the outset. Some clinicians reported that they saw fewer eligible patients than anticipated, while others blamed time pressure for slower than anticipated recruitment. Initial enthusiasm should not therefore be used as a predictor of clinician behaviour during trials. Ideally, clinicians should not be ambivalent about the underlying value of a study.
Financial incentive or philanthropic deed?
It is accepted practice that research payments should cover practice expenses associated with participating in the research. It is difficult to assess the effect of payment on clinicians participating in trials because of the delicate nature of the topic. Clinicians may feel reluctant to identify financial gain as a motivator to take part in a study that aimed to improve the health of children. Indeed, all but one clinician agreed that money was not, or should not be, an incentive for recruitment of patients into clinical trials. Only one clinician reported that the financial reimbursement was far too low and inadequate compensation for the burden associated with study participation.
Informed consent in children
Whilst obtaining consent from all parents/carers was mandatory, the study team advised clinicians during training to obtain written consent on a specially designed form from older children, but this was ultimately discretionary. Eighteen children with an average age of 9 years actually signed consent forms. The subject of children themselves providing consent for the SAVIT study raised a number of issues.
One of the main practical implications of promoting child consent is the extra time needed to explain the study not only to the child, but also to the carer. This is especially relevant in the context of time-pressured consultations. However, time constraints should ideally not deter clinicians from obtaining child consent where appropriate, even if this option is detrimental to recruitment. Sharing recruitment responsibilities between clinicians may provide more effective solutions.
A second implication of child consent was the issue of conflicting attitudes among care givers to a child being recruited onto a study. The person bringing the child to the clinician was sometimes a care giver other than a parent, usually a friend or relative. They often expressed concern about making the decision to enter the child into a study without the parent's or guardian's consent. However, they were given the opportunity to take home the study materials and discuss the study with the parent or guardian before the child actually took study medication. Furthermore, the study nurse would telephone the child's home usually during the evening of the day of recruitment, offering additional support and advice. This also provided the parents with the opportunity to discuss the study with the nurse in more detail and to withdraw the child from the study if they wished. Moreover, it became apparent that parents sometimes disagreed with each other as to whether the child should be entered into the study once the second parent had returned from work and learnt about their child's recruitment into a clinical trial. The opportunity for the study nurse to discuss the trial with the second parent in the evening was seen as helpful and prevented possible withdrawal of several children.
Conclusions
Study design.
A well-designed study that involves recruiting help-seeking subjects during routine consultations should above all ensure simplicity of study documentation and procedures for recruiting both clinicians and patients. Raising the subject of participating in a trial, explaining the trial, and completing documentation including taking consent have to be feasible in routine consultations for trials relying on opportunistic recruitment. If not feasible, potential subjects should be referred to another person with more time. There should be minimum inconvenience to patient and practice. Attention should also be paid to adequate study budgeting as well as potential time slippage relating to recruitment delays or other problematic issues that may surface during the study.
Clinician recruitment and retention. The research team should discuss safety aspects fully with clinician investigators. Clinician ‘ownership’ of the study may also be promoted by emphasizing the usefulness and importance of the trial and the potential benefits to both clinicians and patients. This should be a major focus during initial meetings between the study team and clinicians, since ‘buying into’ the importance of a trial appears to be a prerequisite for successful recruitment. Regularly informing clinicians of the study's progress is usefully carried out by newsletters. These may state an individual clinician's recruitment position in relation to other participating practices in an anonymized way.
Informed consent. Where appropriate, children participating in research should be given the opportunity to consent to their taking part in a study. Additional documentation should include an information sheet and consent form specially designed for children. Additional support to parents/carers by a study nurse is advantageous and gives the carer the opportunity to defer their informed consent. Discussions with the study nurse later in the day will allow the carer to reflect on the implications of the child taking part. A dedicated telephone line to the study nurse may add to their feelings of being supported.
Process evaluation. Process evaluation of trials within the community will help to improve the design and implementation of future research projects. This will lead to better quality primary care trials, improvements in efficiency, increasing clinician satisfaction and lowering the burden on patients who agree to participate in trials. Good experiences of research participation are likely to make easier to recruit clinicians to act as investigators in future studies. This will enhance the representative nature of the clinicians who recruit patients into the study, and thus add to the generalizability of findings. Enriching the evidence base of general practice therefore depends on careful attention to processes of study implementation.
|
Acknowledgments |
|---|
We thank the care givers, children and clinicians for taking part in the trial and acknowledge the extra contribution made by the clinicians who were interviewed for this study evaluation. The UK MRC funded the study (grant number MRC G9900236). CB was supported by a fellowship from NHS Wales Research and Development for Health and Social Care.
|
References |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
1 Prescott RJ, Counsell CE, Gillespie WJ et al. Factors that limit the quality, number and progress of randomised controlled trials. Health Technol Assess 1999; 3:
2 Little P, Gould C, Williamson I, Moore M, Warner G, Dunleavy J. Pragmatic randomised controlled trial of two prescribing strategies for childhood acute otitis media. Br Med J 2001; 322: 336–342.
3 Zermansky AG, Petty DR, Raynor DK, Freemantle N, Vail A, Lowe CJ. Randomised controlled trial of clinical medication review by a pharmacist of elderly patients receiving repeat prescriptions in general practice. Br Med J 2001; 323: 1–5.
4 Tognoni G, Alli C, Avanzini F et al. Randomised clinical trials in general practice: lessons from a failure. Br Med J 1991; 303: 969–971.[ISI][Medline]
5 Murphy E, Spiegal N, Kinomnth A. ‘Will you help me with my research?’ Gaining access to primary care settings and subjects. Br J Gen Pract 1992; 42: 162–165.[ISI][Medline]
6 Asch S, Connor SE, Hamilton EG, Fox SA. Problems in recruiting community-based physicians for health services research. J Gen Intern Med 2000; 15: 591–599.[CrossRef][ISI][Medline]
7 Foy R, Parry J, Duggan A et al. How evidence based are recruitment strategies to randomized controlled trials in primary care? Fam Pract 2003; 20: 83–92.
8 Butler CC, Robling M, Prout H et al. The management of suspected acute viral upper respiratory tract infection in children: a community based randomised controlled trial of treatment with intranasal sodium cromoglycate. Lancet 2002; 359: 2153–2158.[Medline]
9 Toogood J. Multi-centre surveillance of long term safety of sodium cromoglycate. Acta Allergol 1977; 13: 44–54.[Medline]
10 Miles MB, Huberman AM. Qualitative Data Analysis. A Source Book of New Methods. London: Sage, 1984: 67.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. Mosis, A. E. Vlug, M. Mosseveld, J. P. Dieleman, B. C. Stricker, J. van der Lei, and M. C.J.M. Sturkenboom A Technical Infrastructure to Conduct Randomized Database Studies Facilitated by a General Practice Research Database J. Am. Med. Inform. Assoc., November 1, 2005; 12(6): 602 - 607. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||