Family Practice Advance Access originally published online on July 25, 2006
Family Practice 2006 23(5):529-536; doi:10.1093/fampra/cml040
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© 2006 The Author(s)
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Data quality improvement in general practice
Division of Clinical Information and Public Health, Department of General Practice, Academic Medical Center, University of Amsterdam Amsterdam, The Netherlands
Correspondence to HJ Brouwer, Division of Clinical Information and Public Health, Department of General Practice, Academic Medical Center, University of Amsterdam, Meibergdreef 15, 1105 AZ Amsterdam, The Netherlands; E-mail: h.j.brouwer{at}amc.uva.nl
Received 18 July 2006; Revised 19 May 2006; Accepted 28 June 2004.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsConclusionsREFERENCES |
|---|
Background. The importance of routine data generated by GPs has grown extensively in the last decade. These data have found many applications other than patient care. More attention has therefore been given to the issue of data quality. Several systematic reviews have detected ample space for improvement of data quality. A new review was conducted in order to find out which methods of improvement are effective.
Method. The Medline database was searched using an iteratively composed set of terms and MeSH (Medical Subject Headings) headings. Only papers that focused on explicit attempts at improving data quality of medical records in general practice were included.
Results. Twelve studies met the inclusion criteria. No study used patient-based comparison of records with external sources as the method to assess data quality improvement. Ten studies used internal indicators or markers of data quality instead. Attempts at data quality improvement often involve some sort of individualized feedback, and nearly all attempts seem to have some positive effect. Only one of the included studies fulfilled the basic methodological requirements of an intervention study. The most recent studies used a simple before–after design.
Conclusion. No intervention to improve data quality has been put to a rigorous enough test. We still lack empirical knowledge as to how improvement can be brought about.
Keywords. Family practice, general practice, primary health care, research.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsConclusionsREFERENCES |
|---|
Groups of GPs recording medical data for research purposes have a long tradition. The aims and scopes vary, so do the number of practices involved and the methods used.1–5 The number of these networks has exploded in the last decade.2,3 Three factors have contributed towards this development: first, a growing awareness of the potential of primary health care data for public health policy and post-marketing surveillance6–8; second, the evolution of basic tools, such as widely accepted primary health care classification and coding systems9,10; and finally, the emergence of the computerized patient record (CPR) that has made it possible to record data more easily and continuously with little or no disturbance of daily practice.11–13
Data quality in general practice records has been studied regularly. Two recent systematic reviews summarize data quality in general practice computerized medical records.14,15 Many included studies are assessment studies and are of a snapshot nature; they present a momentary picture of data quality. From these reviews, it becomes clear that data quality of medical records assessed in terms of completeness and correctness varies greatly, depending on the item of focus, the diagnosis studied and the reference standard used for comparison. There seems to be ample space for improvement, although no criteria have been suggested or developed as to what are acceptable levels of data quality, and when is ‘good’ or ‘high’ ‘good enough’ or ‘high enough’.
Furthermore, within the framework of a group of GPs dedicated to continuously recording data for research purposes, assessment is only one (although necessary) element within a continuous cycle of quality assurance. Depicting the factors that influence data quality and designing quality improvement programs seem at least of equal importance. Hence, we asked ourselves the following questions. Within the context of general practice,
- which methods of data quality improvement have been used?
- which methods have been proven effective?
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsConclusionsREFERENCES |
|---|
Search strategy
Our aim was to identify and review the literature on the improvement of data quality in general practice. As there is no MeSH (Medical Subject Headings) heading for ‘data quality’, a search strategy was developed iteratively using keywords and MeSH headings of key papers on the subject. The result was a search strategy that could include all of the key papers. The following terms were used to search the Medline database until publication date 31 December 2005: (MH & TW: general pract* OR family pract* OR primary care) AND (MH & TW: medical records) AND (TW: improve* OR control* OR assur*). We did not confine our search to studies of CPRs because we were curious about the similarities or differences in comparison with quality assurance studies of paper patient records. References of collected papers were searched as well. The papers identified through the search were downloaded from Medline into a Reference Manager database.
Results of search strategy and selection of articles
A total of 1541 papers were identified as potentially relevant. Titles and abstracts were scanned by the first author in Reference Manager using four criteria: (i) empirical study in the general practice setting (ii) that made use of the medical record and (iii) in which an intervention was attempted to improve data quality of the medical record and (iv) where the results of at least two measurements of data quality in time were presented. On the basis of title and abstract 1464 papers could be excluded. The remaining 77 papers were retrieved and read by the first author to decide on inclusion. When in doubt, the second author was consulted and a joint decision was made. Excluded were non-empirical discussion papers, quality of care improvement studies using medical records, single assessment studies without intervention and studies on the quality of information and communication systems (hardware and software) in general practice (see Fig. 1). Thus, in total 12 studies were included in the review. These studies are described in Table 1.
|
|
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsConclusionsREFERENCES |
|---|
Methodology of included studies
Only one of the 12 included studies fulfilled the basic methodological requirements of an intervention study: control group, randomized allocation of intervention.19 Two studies used a control group without randomizing the intervention.16,17 All three are among the earliest of the 12 studies. Two of the three controlled studies dealt with the improvement of paper records.16,19
Seven studies used a before–after design without a control group.18,21–23,25–27 Two studies appeared not to have aimed at an intervention from the outset. They merely seem to have analysed medical record data at several moments in time and named certain activities as interventions retrospectively.20,24 One paper did not reveal whether paper or computerized records were involved.23
Interventions
In only two studies17,19 the interventions were described with enough detail to replicate, to compare across studies or to apply to a different setting. Nearly all studies (on paper as well as on computerized record improvement) seemed to have used some sort of individual feedback to improve data recording, either with17,18,23 or without19,20,22,24–27 a comparison with group performance. Group meetings of recording GPs and training sessions were reported as well.17–19,22,25,27 Another study attempted to measure the effect of an update of a GP information system on data quality.21 One study used a self-audit system to improve paper recording.16
Assessment of data quality
None of the studies complied with the criterion Hogan and Wagner28 formulated for the assessment of completeness and correctness of (computerized) medical records: an external reference standard for comparing the medical records with. Instead, in 10 studies internal indicators or markers of data quality were formulated as items of the medical record considered most essential (i.e. % of diagnoses coded, % of allergies recorded), and then a change in that indicator was used as an outcome measure.16–19,21–24,26,27 Two studies used even less direct measures of data quality: a before–after change in prevalence rates and a comparison of prevalence rates with criterion prevalence rates.20,25
All studies were directed at improving the completeness of medical records. In two studies the correctness of medical records was targeted as well.21,25
When completeness was measured, the items concerned various aspects of patient-oriented information, but often included the presence of medical diagnosis or assessment16,17,19,22–24 and/or prescriptions.21,24,27
Effect of interventions
All studies did report improvement on at least some outcome measures. In the non-controlled studies the effect of interventions could, of course, not be disentangled from a possible Hawthorn effect. In the three controlled studies, interventions did improve the (completeness of) reporting of, amongst other items, diagnoses or assessments, while the control group did not show an improvement in time.16,17,19
Two of these used audit16,19 to improve the quality of paper records; one combined several different types of interventions17 to improve the quality of computerized records. The latter study showed that an informal meeting without feedback improved the completeness of diagnosis recording better than meetings with feedback. However, as the interventions were cumulative, it could simply have been a case of the first intervention cutting the deepest.
There is some evidence that the extent to which diagnoses are coded can be sustained and improved over rather long periods of time: 2–9 years.24,26 In these studies the interventions, although insufficiently described, seemed to consist of a continuous cycle of assessment, feedback and training
While the studies did report effects, no conclusion went beyond the reporting of statistical significance; none of them commented on the absolute level achieved, and whether the improvement was large enough.
|
Conclusions |
|---|
|
TopAbstractIntroductionMethodsResultsConclusionsREFERENCES |
|---|
Our review showed that data quality improvement studies in general practice are few and very often not up to the standard of intervention study methodology. Most attempted interventions are described insufficiently and aimed at improving adherence to ‘norms of good record keeping’. As far as could be deciphered it seems that some sort of confrontation with one's own recording effort has been the most prevailing intervention. This feedback is sometimes followed by instruction/training sessions. However, this type of intervention has not been put to a sound methodological test.
Striving for completeness seems the first broad step in improving the quality of general practice records. Only when the quality of recording for specific diseases or diagnoses is studied is correctness targeted as well. Assessment of both completeness and correctness has only been indirect, and not patient-based.
Keeping these methodological shortcomings in mind, it seems certain that record keeping is a skill that can be influenced, though it is not clear whether it is time, being part of a registration effort or an intervention that brings it about.
The background of the practices studied were diverse: explicit registration and research networks,20,21,24,26 separate group practices, health centres and teaching practices16–18,22,23,25,27 and participants in a national morbidity study.19 The 12 studies had their origin in 7 different countries, the 4 most recent ones were from the United Kingdom. All this shows how the importance of the subject is considered irrespective of the various uses of the medical record in general practice. The implications of this issue seem nevertheless most urgent for networks of practices where continuously data are recorded for research purposes. Here, the consequences of over- and under-recording of diagnoses, for instance, can be calculated respectively in terms of extra efforts to include cases for studies and the bias due to selection of more serious or outstanding cases. The benefit of quality improvement can then be clearly weighted.
Interestingly enough, the interventions used to improve computerized medical records have not been drastically different than those used with paper records. In just one instance was a change in software studied as a tool for data quality improvement.21
In our final search strategy we did not use the terms ‘data’ and ‘quality’, nor the MeSH headings ‘quality assurance’, ‘quality control’ or ‘quality indicators’. In our iterative approach we found that inclusion of these terms and headings sharply increased the number of articles to be screened, but still missed some key studies. The combination of terms ‘data’ and ‘quality’ draws large numbers of studies where medical record data are used to study aspects of quality of care.
It has been found that our method of using primarily one reviewer to decide on inclusion of studies may lead to 8% of relevant literature being missed.29 In our search, this would mean that ultimately one other relevant study has been missed.
Given all the attention for data quality in general practice and for the space and need for improvement14,15 it seems surprising that so few studies have been dedicated specifically to the topic of quality improvement. More rigorous testing is clearly needed, with adherence to the following standards:
- a sufficiently detailed description of the interventions
- a before–after control design,
- outcomes measures in terms of both completeness and correctness
- assessment of completeness and correctness using independent patient-based reference measures.
Funding to pay the Open Access publication charges for this article was provided by Acadamic Medical Center, University of Amsderdam.
|
Notes |
|---|
Brouwer HJ, Bindels PJE and Van Weert HC. Data quality improvement in general practice. Family Practice 2006; 23: 529–536
|
REFERENCES |
|---|
|
TopAbstractIntroductionMethodsResultsConclusionsREFERENCES |
|---|
1 Thomas P, Griffiths F, Kai J, O'Dwyer A. (2001) Networks for research in primary health care. Br Med J 322:588–590.
2 Clement S, Pickering A, Rowlands G, Thiru K, Candy B, De Lusignan S. (2000) Towards a conceptual framework for evaluating primary care research networks. Br J Gen Pract 50:651–652.[ISI][Medline]
3 Hart HE, Van der Wouden JC, Hoppener P, van Schendel GJ, Knottnerus JA. (1999) General practice registration projects in The Netherlands: a brief report. J Am Med Inform Assoc 6:173–175.
4 Lawson DH, Sherman V, Hollowell J. (1998) The General Practice Research Database. Q J Med 91:445–452.
5 Green LA, Hames CG, Nutting PA. (1994) Potential of practice-based research networks: experiences from ASPN. J Fam Pract 38:400–406.[ISI][Medline]
6 Wood M, Mayo F, Marsland D. (1986) Practice-based recording as an epidemiological tool. Annu Rev Public Health 7:357–389.[CrossRef][ISI][Medline]
7 Schouten L, Hoppener P, Brandt Pvd, Knottnerus JA, Jager J. (1993) Completeness of cancer registration in Limburg, The Netherlands. Int J Epidemiol 22:369–376.
8 Scobie S, Basnett I, McCartney P. (1995) Can general practice data be used for needs assessments and health care planning in an inner-London district? J Public Health Med 17:475–483.
9 Lamberts H, Wood M, Hofmans-Okkes IM. (1993) International primary care classifications: the effect of fifteen years of evolution. In Lamberts H, Wood M, Hofmans-Okkes IM (Eds.). The International Classification of Primary Care in the European Community (Oxford University Press, Oxford) pp. 9–17.
10 Cimino J. (1996) Formal descriptions and adaptive mechanisms for changes in controlled medical vocabularies. Methods Inf Med 35:202–210.[ISI][Medline]
11 Lei Jvd, Duisterhout J, Westerhof H, et al. (1993) The Introduction of computer-based patient records in The Netherlands. Ann Intern Med 119:1036–1041.
12 Knottnerus JA. (1999) Role of the electronic patient record in the development of general practice in The Netherlands. Methods Inf Med 38:350–354.[ISI][Medline]
13 De Lusignan S and van Weel C. (2005) The use of routinely collected computer data for research in primary care: opportunities and challenges. Fam Pract 23:253–263.
14 Thiru K, Hassey A, Sullivan F. (2003) Systematic review of scope and quality of electronic patient record data in primary care. Br Med J 326:1070–1074.
15 Jordan K, Porcheret M, Croft P. (2004) Quality of morbidity coding in general practice computerized medical records: a systematic review. Fam Pract 21:396–412.
16 Dickie GL and Bass MJ. (1980) Improving problem oriented medical records through self-audit. J Fam Pract 10:487–490.[ISI][Medline]
17 Gilliland A, Mills KA, Steel K. (1992) General practitioner records on computer—handle with care. Fam Pract 9:441–450.
18 Del Degan S, Clavet D, Baillargeon L. (1996) Recording of drug prescriptions in medical records. How to improve quality? Can Fam Physician 42:2193–2197.[ISI][Medline]
19 Del Mar CB, Lowe JB, Adkins P, Arnold E, Baade P. (1998) Improving general practitioner clinical records with a quality assurance minimal intervention. Br J Gen Pract 48:1307–1311.[ISI][Medline]
20 Thiru K, de Lusignan S, Hague N. (1999) Have the completeness and accuracy of computer medical records in general practice improved in the last five years? The report of a two-practice pilot study. Health Inform J 5:224–232.
21 Hiddema-van de Wal A, Smith RJA, van der Werf GT, Meyboom-De Jong B. (2002) Towards improvement of the accuracy and completeness of medication registration with the use of an electronic medical record (EMR). Fam Pract 18:288–291.
22 Lopez-Picazo Ferrer JJ, Agullo Roca F, Villaescusa Pedemonte y M, Cerezo Corbalan JM. (2002) Clinical data that are essential for the primary care clinical records: an experience of evaluation and improvement. Aten Primaria 30:92–98.[Medline]
23 Brami J and Doumenc M. (2002) Improving general practitioners records in France by a two-round medical audit. J Eval Clin Pract 8:175–181.[CrossRef][ISI][Medline]
24 De Lusignan S, Stephens PN, Adal N, Majeed A. (2002) Does feedback improve the quality of computerized medical records in primary care? J Am Med Inform Assoc 9:395–401.
25 Brown PJB and Warmington V. (2003) Info-tsunami: surviving the storm with data quality probes. Inform Prim Care 11:229–237.[Medline]
26 Porcheret M, Hughes R, Evans D, et al. (2004) Data quality of general practice electronic health data: the impact of a program of assessments, feedback and training. J Am Med Inform Assoc 11:78–86.
27 De Lusignan S, Hague N, Brown A, Majeed A. (2004) An educational intervention to improve data recording in the management of ischaemic heart disease in primary care. J Public Health 26:34–37.
28 Hogan WR and Wagner MM. (1997) Accuracy of data in computer-based patient records. J Am Med Inform Assoc 4:342–355.
29 Edwards Ph, Clarke M, DiGiuseppi C, et al. (2002) Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med 21:1635–1640.[CrossRef][ISI][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. Gormley, D. Connolly, D. Catney, L. Freeman, L. J Murray, and A. Gavin Reporting of research data by GPs: a cautionary tale for primary care researchers Fam. Pract., June 1, 2008; 25(3): 209 - 212. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||