Family Practice Advance Access originally published online on February 27, 2008
Family Practice 2008 25(2):119-126; doi:10.1093/fampra/cmn010
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
General practice research training: impact of the Australian Registrar Research Workshop on research skills, confidence, interest and involvement of participants, 2002–2006
a Discipline of General Practice, The University of Adelaide, South Australia
b Department of General Practice, University of Western Australia, Western Australia
c Graduate School of Medicine, University of Wollongong, New South Wales, Australia
Email: karin.ried{at}adelaide.edu.au
Received 3 July 2007; Revised 18 January 2008; Accepted 24 January 2008.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Background. An intensive 3-day training programme, the ‘Registrar Research Workshop’ (RRW), has aimed to build research capacity among Australian general practice registrars since 1994.
Objectives. To investigate the impact of the RRW on participants' skills, confidence, interest in research and research activity.
Methods. Cross-sectional postal survey in 2006 of five groups of registrars who participated in the annual workshop in 2002–2006 (response rate: 64%; 77 of 121). Outcome measures included research experience and skills prior to and after the workshop; impact of the workshop on capacity, confidence, attitude and interest in research; and research involvement as measured by publications and grant funding.
Results. Self-reported research skills increased over time for the whole group (two-way analysis of variance: P = 0.047), most significantly for registrars with little or no research experience (P < 0.001) and research project participants (P = 0.003). The impact of the workshop on capacity, confidence and interest in research was rated highly (mean 3.5–4.0 ± 0.1 on a five-point scale). Two-thirds of the survey respondents had been research active, 34% presented their findings at conferences, 25% published in peer-reviewed journals and 31% received research funding. Eighty-four per cent of respondents indicated a high interest in undertaking research in the future. All survey respondents recommended the workshop to other registrars.
Conclusions. The RRW provides a useful model for effective research training for interested general practice trainees. Such training has the potential to increase knowledge of research methods, which might augment future research activity in general practice.
Keywords. Academic training, family practice, medical education, research activity, training programme.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
It is widely acknowledged that research in general practice is vital for the improvement of patient health care outcomes.1,2 Compared to other clinical disciplines, however, general practice has produced significantly less published research.3 The development of capacity for primary care research is therefore crucial. Recently, there has been considerable investment in research capacity building in Australia.4–6
Incorporating research in the process of vocational training is one way of developing research capacity in a clinical discipline. The Royal Australian College of General Practitioners (RACGP) acknowledges the need for research in its training programme curriculum with a ‘critical thinking and research’ statement,7 and now some regional training providers include compulsory research projects as part of general practice vocational training in Australia.8
The Registrar Research Workshop (RRW) is a national initiative to build research capacity among general practice trainees. This event is an intensive, annual, 3-day workshop which involves approximately 25 GP registrars, who are chosen via a competitive process. Applicants must demonstrate a special interest in learning about research, but existing knowledge or experience of research is neither a prerequisite nor an exclusion criterion. The workshop was initiated by the RACGP in 1994 and has been organized by General Practice Education and Training (GPET) since 2002. The event relies on experienced members of the GP research community, who offer presentations about the research process. Interweaved with these presentations are small group sessions in which registrar teams are guided to develop a research proposal. Beginning with ideas from clinical practice, registrars are led to develop a research question, identify research methods, plan a budget, address ethical concerns and develop a presentation on their project to share with their peers.9
Although participants' immediate evaluations of the workshops had been positive, no long-term data existed on the participants' impressions of the workshop or their subsequent research behaviour. In this study, we aimed to measure the impact of the workshop on participants' skills, confidence and interest in research and their subsequent research involvement.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
We undertook a cross-sectional survey in mid-2006 of 5-year groups of workshop participants (n = 121, 2002–2006), for which contact details were available from GPET. Our questionnaire collected data on participants' demographics, qualifications, research experience prior to the workshop and current research expertise and involvement. We also gathered participants' opinions on the workshop programme, the impact of the workshop on personal research capacity and their anticipated future participation in research. Our questionnaire was modified from an existing questionnaire used in previous studies.5,6 Participants categorized their research involvement level using a previously described four-tiered model.10 The validated ‘research spider’11 was used to measure experience in 10 core areas of research skills. A mixture of categorical data, Likert scales and free text responses were used for other items.
Contact information was retrieved by the GPET member of the research team from GPET databases, supplemented occasionally by information from regional training providers and state and territory medical boards. Participants were informed about the study via email and posted an introductory letter, the questionnaire and reply-paid envelope.
Confidentiality of responses was ensured, as de-identified coded questionnaires were sent directly to the University of Adelaide research team for data entry and analysis. Received questionnaire codes were reported to the GPET researcher for follow-up of non-responders. Non-responders were sent a reminder email and a second copy of the questionnaire a month after the first mail-out. A third and final email reminder was sent to non-responders.
Quantitative data were analysed in SPSS version 13.0 and SAS version 9.1. Data of all five cohorts were combined to increase sample size and therefore power and generalizability. We determined statistical differences in response rates between cohorts using chi-square analysis and used log binominal regression modelling to analyse trends in publication and grant application. Two-way analyses of variance (ANOVAs) were fitted to investigate self-reported changes over time in research skills using the research spider model.11 Bowker's test of symmetry was applied to compare changes over time in self-reported research involvement. Thematic analysis was performed on qualitative data but not all qualitative data are reported in this article.
Information about non-responders was of poor quality, particularly for early cohorts, limiting comparisons between responders and non-responders. However, we compared gender distribution between the two groups and performed a sensitivity analysis to examine the potential effect of non-response on publication and funding outcomes.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
A total of 77 of 121 past workshop participants returned valid questionnaires (64% response rate). Response rates differed significantly (chi-square statistic: P = 0.025) between cohorts: higher response rates were observed in the 2006 (79%, 19 of 24), 2005 (74%, 17 of 23) and 2003 (67%, 16 of 24) cohorts, while lower response rates were achieved in the 2004 (52%, 13 of 25) and 2002 (48%, 12 of 25) cohorts. Response rates did not differ significantly by gender between responders and non-responders (chi-square statistic: P = 0.577).
Table 1 gives an overview of respondents' demographics, postgraduate qualifications and training at the time of our survey.
|
Research experience and skills
We assessed participants' level of involvement in research using a previously described four-tiered model10: non-participants (Category 1, not participating in research activities), participants (Category 2, participating in research as member of a team), clinician researcher (Category 3, managing own project) and academic (Category 4, experienced researcher).
Our survey took snapshots of registrars' research involvement at two time points. Table 2 gives an overview of the research categories which survey participants considered themselves to belong to ‘prior to’ the workshop (time point 1) and at time of the survey ‘after’ the workshop (time point 2). Time point 1 varied between cohorts (2002–2006), while time point 2 gave a cross-sectional view at time of the survey.
|
Table 2 indicates that two-thirds (66%, 51 of 77) of survey participants had undertaken research (categories 2–4) at some time, 42% (32 of 77) prior to the workshop and/or 51% (39 of 77) at the time of the survey. Forty-two per cent (19 of 45) of non-researchers (Category 1) at time point 1 increased their research involvement. Among all participants, a third (32%, 25 of 77) increased, 44% (34 of 77) sustained the same level and about a quarter (23%, 18 of 77) reduced or stopped their research involvement by time of the survey. However, 13 of the latter 18 indicated their plans to pursue research in the next 5 years. Confounding life events included, for example, maternity leave. While a number of individuals moved up or down the research category scale during the investigated time interval, Bowker's test of symmetry revealed no statistically significant change in research categories for the group as a whole (P = 0.082).
Figure 1 illustrates respondents' self-reported mean research experience levels in 10 core skill areas of research and compares skill levels of respondents prior to the workshop with skill levels at the time of our survey. These range from low levels of experience for ‘publishing research’ [mean = 1.6 ± 0.1 standard error (SE)] and ‘applying for research funding’ (mean = 1.7 ± 0.1) to ‘some experience’ for ‘finding literature’ (mean = 3.2 ± 0.1) and ‘critically reviewing literature’ (mean = 3.0 ± 0.1).
|
Two-way ANOVA tests were conducted to analyse skill levels over time for the whole group and by research category. Research skill levels increased over time in all 10 research skill areas for the whole group (P = 0.047) and in particular for non-participants (Category 1: mean = 1.6 ± 0.1 SE prior to workshop and mean = 2.7 ± 0.1 SE after workshop, P < 0.001) and participants (Category 2: mean = 2.2 ± 0.2 SE prior to workshop and mean = 3.1 ± 0.2 SE after workshop, P = 0.003).
While there were differences in mean skill levels between cohorts, no significant correlation between perceived skill level and time since workshop participation (2002–2006) was observed.
Importance and impact of the RRW
The majority (over 80%) of past workshop participants rated the general structural aspects of the workshop, such as presentations by experts to the whole group, the small group work and networking opportunities, ‘important’ or ‘very important’ with nearly half of the respondents (46%) stating that ‘networking with experts and facilitators’ had been very important (Box 1).
| BOX 1 Examples of participants' comments on the workshop programme.
The best aspect of the workshop was having access to senior GP academics—as role models—and meeting early career researchers—as reassurance. Role modelling from academics and research-minded registrars was influential. It was good to discover that academics were not scary! The general enthusiasm amongst researchers was infectious on us registrars! The small group work brought alive what we learnt in the presentations and was a vital part of the workshop.
|
We assessed the impact of the RRW on participants' self-reported capacity (ability/knowledge), confidence and interest in research using a five-point Likert scale. Survey respondents felt that the workshop had ‘moderate impact’ (means = 3.5–4.0 ± 0.1 SE) on their capacity, confidence and interest in undertaking research in the future, but felt that the workshop had slightly less impact (mean = 3.3 ± 0.1) on their confidence and capacity for ‘applying evidence-based research in clinical practice’ (Fig. 2).
|
The majority of surveyed workshop participants (75%) reflected positively on the workshop's influence on their perception of research; stating a better understanding of the research process; raised awareness of its relevance, scope and importance in general practice; and realization that research is ‘doable’(details of thematic analysis available from authors).
Registrars' research involvement and outcomes (publications and grants)
Table 3 provides an overview of all survey respondents' research involvement and outcomes, such as publications and grants, as well as details of those respondents who were not involved in research prior to the workshop Category 1 (non-participants). A quarter of all respondents had published their research in a peer-reviewed journal and a third had been successful in receiving grant funding (Table 3, columns 2 + 3). Similarly, almost a quarter of respondents who were not research active prior to the workshop had published or submitted a paper for publication at time of the survey, and 22% had received grant funding (Table 3, columns 4 + 5).
|
Because non-responders to our questionnaire might be less research active than responders, we undertook a simple sensitivity analysis to counter for non-response. Even if all our non-responders had been completely inactive in research, a total of 21% of all workshop participants would have presented research at conferences, 16% would have published in a peer-reviewed journal and 20% would have successfully received grant funding.
Since time frames for applying for research funding, conducting and disseminating research differed between cohorts, we analysed research outcomes by cohort (year of participation in the RRW) using a log binominal regression model. The analysis revealed a significant increase over time available in the number of research grant applications made (P < 0.009) and successful receipt of grant funding (P < 0.008); 1-year difference was associated with a 27% increase in receipt of grant funding. Time trend analysis on publication rates was borderline significant (P < 0.06), with some suggestion that a greater percentage of articles had been published by earlier cohorts (2002: 50% and 2003: 31%) than more recent cohorts (2004: 23%, 2005: 12% and 2006: 21%).
Participation in future research
Registrars and GPs who had participated in the RRW and taken part in our survey were generally very enthusiastic about pursuing research in the future. About 80% indicated being highly interested in conducting further research, publishing research and attending further research training. Enthusiasm was a little lower for preparing grant applications (57%). Encouragingly, two-thirds of past workshop participants planned to integrate research into their career in general practice, mainly as clinician researcher (57%).
Perceived barriers to future research involvement matched previously identified themes12: lack of time and having other priorities were mentioned by the majority of respondents (69%), followed by lack of funding (53%), appropriate supervision and support (13%), research opportunities (8%), rural isolation (8%) and lack of collaborators (7%).
The future of the workshop
All 77 respondents of the surveyed five workshop cohorts agreed that they would recommend the RRW to other registrars. Consistent themes emerging from responses included the workshop's uniqueness as a model of research training in general practice and the non-threatening and supportive nature of guidance offered by senior researchers at the workshop. Furthermore, some registrars emphasized the importance of continuation of the workshop for future registrar cohorts and suggested that the workshop be made available to more registrars (Box 2).
| BOX 2 Examples of registrars' comments on importance of workshop to general practice research and possible future directions.
The RRW is an important foundation stone that will underpin future clinical research that can be used to provide better health outcomes in the Australian community. The workshop is a much needed gateway to furthering interest and capacity in GP research and academic career options. The concept of the workshop should continue. GP training needs to have research input like most other [medical] training programs. Even if GPs won't be doing research, they need to be able to critically evaluate the literature. This is an important workshop to continue offering to foster the future of research in general practice. Make the workshop more widely available.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Our study demonstrates that the RRW had a positive impact on participants' capacity, confidence and interest in research. Self-reported research experience and involvement, publication and grant application rates increased over time in the surveyed group.
Our study has several limitations. First, because the participants in the workshop are a self-selected group of registrars interested in research, the benefits suggested in our study may not be generalizable to less motivated groups. Some degree of responder bias is likely to be present despite high response rates, and our limited knowledge of the demographics of non-responders prevented us from assessing whether responders were typical of all potential participants. It is possible that more research-motivated participants were more likely to have completed the questionnaire. Due to our retrospective study design and our questionnaire's reliance on self-reporting, recall bias might have influenced our study results. Also, several of the authors were known to participants from their involvement in past RRWs; this might have introduced a social desirability bias. Finally, changes in skill levels cannot be causally attributed to the workshop, due to the lack of a control group.
Despite these limitations, our study is of importance. Since the inception of this annual workshop in 1994, this is the first systematic evaluation of the views of multiple cohorts of participants on the effectiveness and impact of this educational model in general practice research training. We included multiple cohorts in our study to increase sample size and minimize bias in relation to individual workshop characteristics such as group dynamics (between participating registrars, presenters and small group facilitators), with the purpose of increasing generalizability. Furthermore, the literature on research activities of registrars and early career GPs is sparse, and our study addresses this gap.
Our study established that the majority of surveyed clinicians were pursuing or planning to pursue their interest in research in general practice. Our 77 respondents reported a total of 31 peer-reviewed published articles. While our data do not allow exact calculation of yearly publication rates, this level of publication in our early career population seems to compare favourably to the wider general practice publication rate in the 1990s of one article per 1000 GPs per year.3 In addition, research grant funding was received by a third of surveyed workshop participants.
Our results concur with previous findings regarding perceived barriers to research involvement such as lack of funding, supervision, opportunities and collaboration.12 Some of these barriers might be addressed by opening research channels between academics and medical students or registrars, such as finding suitable academic mentors, involving students in established research projects and building in feedback loops on grant applications.
Our study is timely given the current Australian Government interest in development and evaluation of research capacity-building programmes in general practice and primary health care. This interest is exemplified by the Primary Health Care Research Evaluation Development Strategy, funded by the Australian Government Department of Health and Ageing since 2000,13 which aims to nurture primary health care research.
Our findings imply that research-training initiatives such as this workshop can positively influence GP registrars' interest in and knowledge of research. At the same time, such initiatives provide a useful means of introducing registrars to a network of academics and GP clinician researchers. There is scope to harness these groups of research-active junior doctors to advance academic general practice.
Clinical researchers in general practice are much needed. How best to educate such clinicians and enable them to become active in research is a worthy subject for further research. Such research could include long-term follow-up of a group of clinicians, a suitable control group and ideally a randomized experimental design. While such evidence is awaited, the workshop we describe may serve as a useful model of an educational intervention for GPs or other health professionals.
|
Declaration |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Funding: Primary Health Care Research Evaluation and Development programme; Australian Government Department of Health and Ageing.
Ethical approval: The study was approved by the University of Adelaide Human Research Ethics Committee.
Conflicts of interest: BDM was an employee of GPET at the time that the study was planned and data were collected. KR and EAF have been invited presenters/facilitators at the RRW. Funding sources had no influence on interpretation and analysis of data, preparation of the manuscript or decision to submit for publication.
|
Acknowledgments |
|---|
We are grateful to GPET for providing invaluable assistance in this project. We also would like to thank all survey participants and acknowledge Thomas Sullivan (University of Adelaide) for statistical advice. Contributions: all authors conceptualized the study. BDM was responsible for contacting participants and mailing questionnaires. KR conducted data analysis and prepared the manuscript with contributions by BDM, NPS and EAF. All authors approved the final version.
|
Notes |
|---|
Ried K, Montgomery BD, Stocks NP and Farmer EA. General practice research training: impact of the Australian Registrar Research Workshop on research skills, confidence, interest and involvement of participants, 2002–2006. Family Practice 2008; 25: 119–126.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
1 Mant D, Del Mar CB, Glasziou P, Knottnerus A, Wallace P, van Weel C. The state of primary-care research. Lancet (2004) 364:1004–1006.[CrossRef][Web of Science][Medline]
2 Askew D, Del Mar C, McAvoy B, Lyle D. General practice research. In: General practice in Australia: 2004 (2005) Canberra, Australia: Australian Government Department of Health and Ageing. 214–266.
3 Glasziou PP, Del Mar CB. Research output of Australian general practice: a comparison with medicine, surgery and public health. Med J Aust (2001) 175:77–80.[Web of Science][Medline]
4 Shah N, Pond D, Heaney S. Research capacity building in general practice: the new Australian scene. Aust Fam Physician (2002) 31:201–204.[Medline]
5 Ried K, Farmer EA, Weston KM. Setting directions for capacity building in primary health care: a survey of a research network. BMC Fam Pract (2006) 7:8.[CrossRef][Medline]
6 Ried K, Farmer EA, Weston KM. Bursaries, writing grants and fellowships: strategy to develop research capacity in primary health care. BMC Fam Pract (2007) 8:19.[CrossRef][Medline]
7 The Royal Australian College of General Practitioners. The RACGP Curriculum for Australian General Practice: Critical Thinking and Research (2007) South Melbourne: The RACGP. http://www.racgp.org.au/curriculum (accessed 18 January 2008).
8 Gartlan J, Cooling N, Nelson M. General practice research and research skill needs—attitudes of GP supervisors. Aust Fam Physician (2006) 35:1003–1005.[Medline]
9 Montgomery B, McMeniman E, Cameron S, Duncan T, Prosser S, Moore R. Sex, pain and cranberries: ideas from the 2006 Registrar Research Workshop. Aust Fam Physician (2007) 36:93–94.[Medline]
10 Farmer E, Weston K. A conceptual model for capacity building in Australian primary health care research. Aust Fam Physician (2002) 31:1139–1142.[Medline]
11 Smith H, Wright D, Morgan S, Dunleavey J. The ‘Research Spider’: a simple method of assessing research experience. Prim Health Care Res Dev (2002) 3:139–140.[CrossRef]
12 Thomsen JL, Jarbol D, Sondergaard J. Excessive workload, uncertain career opportunities and lack of funding are important barriers to recruiting and retaining primary care medical researchers: a qualitative interview study. Fam Pract (2006) 23:545–549.
13 The PHCRED strategy. Australian Government Department of Health and Ageing. http://www.phcris.org.au/phcred/index.php. (accessed on 18 January 2008).
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. Graffy Engaging family practitioners in research: are we getting it right? Fam. Pract., April 1, 2008; 25(2): 69 - 70. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||