Skip Navigation


Family Practice Advance Access originally published online on April 6, 2005
Family Practice 2005 22(4):458-464; doi:10.1093/fampra/cmi022
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
22/4/458    most recent
cmi022v1
Right arrow E-letters: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (10)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Wetzel, D.
Right arrow Articles by Scheidt-Nave, C.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Wetzel, D.
Right arrow Articles by Scheidt-Nave, C.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author (2005). Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oupjournals.org

Participation in a quality of care study and consequences for generalizability of general practice research

Dirk Wetzela, Wolfgang Himmela, Regine Heidenreicha, Eva Hummers-Pradiera, Michael M Kochena, Anja Rogauscha, Joerg Siglea, Harro Boeckmannb, Steffen Kuehnelc, Wilhelm Nieblingd and Christa Scheidt-Navea,e

a Department of General Practice, University of Goettingen, b QP-Quality Practices GmbH Bad Krozingen, c Centre for Social Science Methodology, University of Goettingen, d Teaching Unit for General Practice, University of Freiburg and e Robert Koch Institute Berlin, Germany

Correspondence to Dirk Wetzel, Department of General Practice and Family Medicine, University of Goettingen, Humboldtallee 38, 37073 Goettingen, Germany; Email: dwetzel{at}gwdg.de

Received 30 October 2004; Accepted 30 December 2004.

Wetzel D, Himmel W, Heidenreich R, Hummers-Pradier E, Kochen MM, Rogausch A, Sigle J, Boeckmann H, Kuehnel S, Niebling W and Scheidt-Nave C. Participation in a quality of care study and consequences for generalizability of general practice research. Family Practice 2005; 22: 458–464.


   Abstract
Top
 Abstract
Introduction
 Methods
 Results
 Discussion
 Declaration
 Appendix 1 Questionnaire...
 References
 
Background. Non-participation of General Practitioners (GPs) hampers primary care research. Using existing network structures can improve participation, but may introduce sampling effects.

Objectives. To examine the role of network-based recruitment and other determinants of participation, and to estimate effects of sampling and non-participation on generalizability.

Methods. For a study of quality of care assessment, we recruited GPs from a regionally defined population and a GP network. Effects of sampling and non-participation were analysed by comparing characteristics between participants, target samples, and reference data for all German GPs. Factors influencing study participation were assessed in multiple logistic regression.

Results. Compared to the regional sample, network GPs were more likely to complete survey questionnaires (92% versus 69%) and to participate in the study (66% versus 23%). Compared to national reference data, study participants from both populations were younger, had a higher level of professional training, and included more men. These differences were already present in the network target sample, but were largely attributable to selective participation in the regional sample. Network membership remained the strongest determinant of participation in multiple logistic regression (odds ratio 5.01; 95% confidence interval 2.53–9.91). A younger age, higher professional training, and membership in the German Society of General Practice were also predictive of participation.

Conclusions. Although network-based recruitment of GPs increases participation rates, sample effects are similar in size and direction as effects of non-participation in the regionally defined population. Careful analysis of participants based on publicly available data is therefore crucial for the assessment of generalizability.

Keywords. Generalizability, general practice, non-participation, research design.


   Introduction
Top
 Abstract
 Introduction
Methods
 Results
 Discussion
 Declaration
 Appendix 1 Questionnaire...
 References
 
Research in primary care settings yields a valuable source of data for general practice research and health care planning. One of the major obstacles and potential threats to the validity of research involving GPs and other community-based doctors is non-participation.1,2 This is particular true for quality of care studies.3 Recruitment based on existing physician networks can be a powerful tool to increase participation rates,2 but may lead to sampling effects. Recent general practice research has drawn attention to the fact that improving participation rates does not necessarily improve the generalizability of the study results. Several authors recommended to study the effects of potential bias in greater detail.1,2,4

MedViP [Medizinische Versorgung in der Praxis (Medical Care in General Practice)] is a primary health care research project funded by the German Ministry of Education and Research. The MedViP study is coordinated by the Department of General Practice, University of Goettingen Medical School. The feasibility and validity of computerized medical records for quality of care assessment in German general practices will be investigated. Doctors were recruited from a network of GPs and from a regionally defined but otherwise unselected target population of GPs. The aims of the present analysis were: (1) to examine the effects of sampling and non-participation on the representativeness of the study population; and (2) to assess the relative impact of network-based recruitment and other factors on study participation.


   Methods
Top
 Abstract
 Introduction
 Methods
Results
 Discussion
 Declaration
 Appendix 1 Questionnaire...
 References
 
Study design
The design and aims of the main study have been described in detail elsewhere.5,6 In brief, MedViP is a cluster study, where the individual practice represents the unit of analysis. Using medical data routinely entered into the computer, process measures will be compared to current clinical guideline recommendations, especially for patients with asthma or chronic obstructive pulmonary disease (COPD), heart failure, or urinary tract infections. Participating practices receive a one-time compensation of 500 Euro and Post Graduate Education Allowance Points. Ethical approval was granted by the Ethics Committees of the University of Goettingen and the University of Freiburg Medical Schools.

Target population
In January 2002, a total of 460 primary care doctors (board certified GPs or general internists; practitioners without board certification in any particular specialty) were registered in 363 general practices in the administrative district of Goettingen, Lower Saxony. Their names and addresses were obtained from the KV Goettingen (Kassenaerztliche Vereinigung, the regional Association of Statutory Health Insurance Physicians). Excluding 13 doctors from 10 practices (currently not or no longer in practice), the net Goettingen target sample consisted of 447 GPs in 353 practices. The GP quality assurance network in the vicinity of Freiburg, Baden-Wuerttemberg listed 83 doctors in 66 practices (www.qualitaetspraxen.de). The majority of these doctors have previously contributed to research projects. Two authors are network members (HB; WN) and have been involved in the design of the MedViP study.

Recruitment
A two-stage protocol was used to recruit the study population:

  1. In September 2002, the coordinating center contacted eligible doctors by mail (doctors in group practices were addressed individually) and informed about the aims and contents of the MedViP study. Doctors were asked to return an enclosed standardized questionnaire on practice- and doctor-related characteristics, irrespective of their decision to participate in the research project. The standardized questionnaire (Appendix 1) consisted of questions concerning main characteristics of the study population, computer equipment and access to the internet, interest to join the MedViP study (yes; need further information; no), and, if applicable the main reason for decline (no time, not interested, other). Two reminders were mailed at intervals of 6 to 8 weeks. Doctors who completed the questionnaire were counted as survey responders.
  2. GPs who were interested in joining the research project received further information illustrating the study protocol and time schedule. Between December 2002 and January 2004, technical staff were sent out to these practices to assist the computer export of electronic medical data via a standard interface (Behandlungs-Daten-Traeger, BDT) as previously described.7 Doctors who successfully completed the data export were counted as study participants.

Publicly available data
The KBV (Kassenaerztliche Bundesvereinigung, the national Association of Statutory Health Insurance Physicians) provided aggregate reference data about age, gender, time in office practice, vocational training status and practice type among German GPs. Respective information for the MedViP target samples was obtained from the KV Goettingen and the office of the Freiburg GP network.

GPs involved in teaching medical students were identified from faculty listings at the Universities of Goettingen and Freiburg Medical Schools. Members of the German Society of General Practice and Family Medicine (DEGAM) were identified from membership directories.

Statistical analysis
In order to estimate the effects of sampling and non-participation, main characteristics of the German GP reference population from KBV statistics were compared to the MedViP target samples, survey responders and study participants in Goettingen and Freiburg, respectively. Relative frequencies, means and 95% confidence intervals were calculated and compared between groups for overlap.

Among survey responders, descriptive data obtained from postal questionnaires as well as information regarding teaching and DEGAM membership were compared by target group (Freiburg versus Goettingen), and between study participants and non-participants within target groups.

The Statistical Analysis Software (SAS) package, Version 8.2 was used for data analysis. A P-level of <0.05 was considered statistically significant based on two-sided tests.

Chi-Square or Fisher's exact test (dichotomous data), and Wilcoxon test (continuous data) were used to test for group differences as appropriate. Correlation between variables was assessed by Spearman's rho. A multiple logistic regression model was fitted to examine the independent effects of target group and covariates on study participation (yes versus no). Only variables that showed significance at the 5% level (P < 0.05) in univariate analyses were entered into the model. Dummy variables were computed for variables with more than two response levels.


   Results
Top
 Abstract
 Introduction
 Methods
 Results
Discussion
 Declaration
 Appendix 1 Questionnaire...
 References
 
Response and participation rates
Recruitment results are summarized in Figures 1a and 1b and Table 1. Overall, 92% (76/83) of Freiburg network members and 69% (308/447) of eligible GPs in the Goettingen area responded to the initial postal survey (survey responders). The proportion of doctors who initially agreed to participate in the MedViP project was about twice as high (67% versus 32%) in Freiburg (56/83) as in Goettingen (144/447) (P < 0.001).



View larger version (42K):
[in this window]
[in a new window]
  		FIGURE 1 (a) Steps of study participation in Freiburg. (b) Steps of study participation in Goettingen

 

View this table:
[in this window]
[in a new window]
  		TABLE 1 Distribution of main characteristics among MedViP target and study populations compared to national reference data for all German GPs

 
Twenty Goettingen doctors and one doctor from Freiburg had later withdrawn their initial consent to participate in the study for of a variety of reasons (mainly ‘work overload’). Data from another 20 doctors in Goettingen could not be obtained due to technical reasons (software or hardware did not allow data export).

Eventually, 104 doctors (23%) from the Goettingen sample and 55 doctors from the QP network (66%) successfully completed the baseline data export from their practice computers (study participants, Figures 1a and 1b, Table 1).

Reasons for non-participation
Among survey responders who declined to participate in the study (n = 144 in Goettingen, n = 21 in Freiburg), 117 doctors in Goettingen and 19 in Freiburg had checked their reason for non-participation on the survey questionnaire. In both groups, ‘no time’ was by far the predominant reason (60% in Goettingen, 68% in Freiburg, data not shown). Nine percent in Goettingen had checked ‘not interested’. ‘Other reasons’ was checked by 32% in Goettingen (n = 37) and in Freiburg (n = 6), respectively.

Where ‘other reasons’ were specified, in Goettingen they included advanced age or bad health (n = 10) as well as technical or practice related obstacles (n = 13), such as lack of computerization or specialization on psychotherapy or homeopathy. One doctor wrote that he did not understand the goal of the study, and two wrote that they were too frustrated with budget restraints or guideline dictations. Another two doctors expressed concerns with data protection. In Freiburg technical and practice related reasons prevailed (n = 3), and one doctor indicated that he was already involved in a different research project.

Effects of sampling and non-participation
Study participants in Goettingen significantly differed from the German reference population of GPs with respect to nearly all characteristics available from publicly available data (Table 1). They were significantly younger, more likely to be male, to be board certified GPs and to be DEGAM-members. Most differences were entirely attributable to non-participation effects, with some exceptions: sampling effects apparently accounted for the overrepresentation of doctors in solo practices in the target sample and contributed to the overrepresentation of board-certified GPs and the underrepresentation of general internists working in primary care. To check whether data from the 20 Goettingen doctors who could not participate due to technical reasons alone would have changed results, we recalculated these analyses counting them as study participants. The results did not materially change and the estimates were generally intermediate between survey responders and ‘real’ study participants (data not shown, available upon request).

Study participants from the Freiburg network significantly differed from the reference population with respect to a lower proportion of GPs without board certification and a higher proportion of board certified internists. This difference was already evident in the target sample (Table 1). To a lesser degree (slightly overlapping 95%CI) this was also true for a lower proportion of female doctors and a longer mean time in office practice among study participants compared to all German GPs.

Determinants of participation
Among survey responders, study participants in Goettingen were significantly less likely than non-participants to be female or to be GPs without board certification, but more likely to teach medical students, to be a DEGAM member, to use a practice computer system and to run a vocational training practice (Table 2). They were also, on average, 3 years younger, a difference that was also reflected by less years in office practice.


View this table:
[in this window]
[in a new window]
  		TABLE 2 Characteristics of MedViP study participants and non-participants by target population

 
Study participants in the Freiburg network differed from non-participants in only one aspect: they were more likely to have access to the internet at the office (P < 0.001).

In multiple logistic regression (analysis), belonging to the Freiburg network remained the strongest independent determinant of study participation (odds ratio 5.01; 95% confidence interval 2.53–9.91). A younger age, board certification and DEGAM-membership were also significant predictors (Table 3).


View this table:
[in this window]
[in a new window]
  		TABLE 3 Determinants of participation—results of full multiple logistic regression modela

 

   Discussion
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
Declaration
 Appendix 1 Questionnaire...
 References
 
In line with studies from the UK, the USA and Canada,2 the use of existing network structures in our study proved to be very effective with respect to increasing the participation rate. Recruitment of GPs from an established network nearly tripled the participation rate in this quality of primary care study (66% versus 23%). The difference in participation rates could not be explained by differences in basic characteristics of the target populations.

As recently suggested by Barclay and colleagues,4 we used publicly available data to assess the size, direction and source of potential selection effects in our study populations compared to the population of German GPs at large. Study participants from both target groups differed from the German national GP population in a similar direction albeit for different reasons. Effects of non-participation predominated in the Goettingen regional sample as opposed to sampling effects in the Freiburg network. This means that higher participation rates do not guarantee a better representativeness of the study population, because selection effects due to non-participation may be traded for selection effects due to sampling. Non-participation among Goettingen GPs as well as preselection of the Freiburg GP network resulted in overrepresentation of male doctors, GPs with board certification and doctors holding a DEGAM membership, although the differences in the network sample were not always statistically significant, probably due to small numbers. Goettingen but not Freiburg study participants were significantly younger than the average of German GPs, an effect that was entirely attributable to selective participation as previously described by others.8,9 Sampling effects accounted for the fact that the two target samples significantly differed in the proportion of GPs trained as internists. They were overrepresented among network GPs but underrepresented in the Goettingen regional sample.

Results from multiple logistic regression confirmed results of previous studies, as a younger age8,9 and a higher level of professional training8 were also significantly associated with participation, in addition to the strong influence of network membership.

Participation rates in general practice research strongly depend on the topic of the study10 as well as on the incentives offered to participants.11 While there was no difference in monetary incentives (500 Euro per participating practice), network GPs can be expected to have been particularly interested in the study topic, i.e. the quality of care for general practice patients. This was reflected by a number of differences between the Freiburg and Goettingen study populations: study participants from the Freiburg network had a higher vocational training level, a longer experience in general practice, and were more likely to have internet access at the office and to currently participate in quality circles. Nevertheless, adjustment for these variables did not explain the difference in participation.

The interpretation of our study-results is limited in several ways. First, our comparison between participants and non-participants may be biased by non-response. Although overall response rates to the postal questionnaire were reasonably high,4 31% of eligible GPs in Goettingen and 8% in Freiburg did not answer the postal questionnaire. However, comparison between survey responders and the target sample for demographic variables and other publicly available characteristics revealed no significant differences; hence a severe bias at this point is rather unlikely.

Secondly, with respect to the observed effect of ‘network recruiting’ on participation rates, we cannot separate the effects of local peer support from other network issues. Therefore, ‘network membership’ may represent multiple characteristics. Further research needs to separate these different components.

Third, by using a standardized questionnaire, we inevitably missed important factors influencing participation. As previously reported by others,2,12,13 ‘no time’ was the reason for non-participation most frequently reported in the present study. Although time restraints are an important issue in general practice, this argument may also be used as a socially desirable excuse for not getting involved in research activities. Our results support the view that female GPs, GPs who were older and less well trained obviously felt less able to cope with, or were less willing to accept additional workload resulting from study participation. Qualitative interviews will be performed in a subset of non-participants to get deeper insight into the ‘true’ reasons for non-participation.

In summary, a considerably higher participation rate among network-based GPs was largely traded for sampling effects on the study population. This is likely to bias results of a study dealing with quality of care largely in the same direction and to a comparable degree as effects due to non-participation in the regional sample. As for our study, this could result in an overestimation of the ability to construct quality of care indices from electronic patient records in general practices.14

For the sake of generalizability of primary care research, our study results are a plea to identify and, if possible, to eliminate factors that impede higher participation rates in unselected populations of GPs. Network-based recruitment nevertheless assures higher participation rates, but sampling effects should be considered. Careful analysis of the study population based on publicly available data is therefore essential to the interpretation of study results and to assess the generalizability of general practice research.


   Declaration
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Declaration
Appendix 1 Questionnaire...
 References
 
Funding: the MedViP study is supported by a grant of the German Ministry of Education and Research (01GK0201) and an unconditional grant by Ratiopharm GmbH, Ulm.

Ethical approval: granted by the Ethics Committees of the University of Goettingen and the University of Freiburg Medical Schools.

Conflicts of interests: none.


   Appendix 1 Questionnaire concerning characteristics of GPs
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Declaration
 Appendix 1 Questionnaire...
References
 

  1. Years in practice?
  2. Years of your vocational training?
  3. Age?
  4. Practice size?
    <800 listed patients per quarter
    800–1200 listed patients per quarter
    >1200 listed patients per quarter

  5. Do you take part in a quality circle? yes no
  6. Is your practice involved in vocational training of junior doctors? yes no
  7. Do you use a computer in your practice?
  8. Do you have access to the internet? yes, in practice yes, at home no
  9. Would you like to participate in the study?
    Yes, I would like to participate in the study
    I am interested, but I want more information, please contact me
    I do not want to participate in the study

  10. If you do not want to participate, what is your main reason?
    Not interested
    No time
    Other reason


   References
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Declaration
 Appendix 1 Questionnaire...
 References
 
1 Templeton L, Deehan A, Taylor C, Drummond C, Strang J. Surveying general practitioners: does a low response rate matter? Br J Gen Pract 1997; 47: 91–94.[Web of Science][Medline]

2 Asch S, Connor SE, Hamilton EG, Fox SA. Problems in recruiting community-based physicians for health services research. J Gen Intern Med 2000; 15: 591–599.[CrossRef][Web of Science][Medline]

3 Borgiel AM, Dunn EV, Lamont CT et al. Recruiting family physicians as participants in research. Fam Pract 1989; 6: 168–172.[Abstract/Free Full Text]

4 Barclay S, Todd C, Finlay I, Grande G, Wyatt P. Not another questionnaire! Maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of GPs. Fam Pract 2002; 19: 105–111.[Abstract/Free Full Text]

5 MedVip website: www.medvip.uni-goettingen.de (accessed 01/07/2004).

6 Wetzel D, Scheidt-Nave C, Rogausch A et al. Quality of medical care in general practice: an assessment model for the improvement of health services research in general practice (in German). Z Allgemeinmed 2003; 79: 394–398.[CrossRef]

7 Hummers-Pradier E, Simmenroth-Nayda A, Scheidt-Nave C et al. Medical care research based on family doctor routine data—are interface-communicated treatment data feasible? (in German). Gesundheitswesen 2003; 65: 109–114.[CrossRef][Web of Science][Medline]

8 Armstrong D, Ashworth M. When questionnaire response rates do matter: a survey of general practitioners and their views of NHS changes. Br J Gen Pract 2000; 50: 479–480.[Web of Science][Medline]

9 Shelton BJ, Wofford JL, Gosselink CA et al. Recruitment and retention of physicians for primary care research. J Comm Health 2002; 2: 79–89.[CrossRef]

10 Ward J. General practitioners' experience of research. Fam Pract 1994; 11: 418–423.[Abstract/Free Full Text]

11 Deehan A, Templeton L, Taylor C, Drummond C, Strang J. The effect of cash and other financial inducements on the response rate of general practitioners in a national postal study. Br J Gen Pract 1997; 47: 87–90.[Web of Science][Medline]

12 Kaner EFS, Haighton CA, McAvoy BR. ‘So much post, so busy with practice—so, no time!’: A telephone survey of general practitioners' reasons for not participating in postal questionnaire surveys. Br J Gen Pract 1998; 48: 1067–1069.[Web of Science][Medline]

13 Key C, Layton D, Shakir SAW. Results of postal of the reasons for non-response by doctors in a prescription event monitoring study of drug safety. Pharmacoepidemiol Drug Safety 2002; 11: 143–148.[CrossRef][Web of Science][Medline]

14 Heidenreich R, Himmel W, Boeckmann H, Hummers-Pradier E, Kochen MM, Niebling W, Rogausch A, Sigle J, Wetzel D, Scheidt-Nave C. Recording electronic medical data in general practice: a telephone survey (in German), Z Arztl Fortbild Qualitatssich 2005; 99: in press.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 Ann Fam MedHome page
J. M. Galliher, A. J. Bonham, L. M. Dickinson, E. W. Staton, and W. D. Pace
Representativeness of PBRN Physician Practice Patterns and Related Beliefs: The Case of the AAFP National Research Network
Ann. Fam. Med, November 1, 2009; 7(6): 547 - 554.
[Abstract] [Full Text] [PDF]

Home page
 Fam PractHome page
E. Hummers-Pradier, C. Scheidt-Nave, H. Martin, S. Heinemann, M. M Kochen, and W. Himmel
Simply no time? Barriers to GPs' participation in primary health care research
Fam. Pract., April 15, 2008; (2008) cmn015v1.
[Abstract] [Full Text] [PDF]

Home page
 Health Educ ResHome page
E. L. James, C. Fraser, K. Anderson, and F. Judd
Use of research by the Australian health promotion workforce
Health Educ. Res., August 1, 2007; 22(4): 576 - 587.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
22/4/458    most recent
cmi022v1
Right arrow E-letters: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when E-letters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (10)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Wetzel, D.
Right arrow Articles by Scheidt-Nave, C.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Wetzel, D.
Right arrow Articles by Scheidt-Nave, C.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?