Family Practice Vol. 19, No. 5, 442-447
© Oxford University Press 2002
Feasibility of AsthmaCritic, a decision-support system for asthma and COPD which generates patient-specific feedback on routinely recorded data in general practice
a Department of Medical Informatics,
b Family Practice,
c Pediatrics and
d Pulmonology, Erasmus Medical Center, Rotterdam, The Netherlands.
MM Kuilboer, Erasmus Medical Center Rotterdam, Ee2100, PO Box 1738, 3000 DR Rotterdam, The Netherlands; E-mail: kuilboer{at}mi.fgg.eur.nl
Kuilboer MM, van Wijk MAM, Mosseveld M, van der Does E, Ponsioen BP, de Jongste JC, Overbeek SE and van der Lei J. Feasibility of AsthmaCritic, a decision-support system for asthma and COPD which generates patient-specific feedback on routinely recorded data in general practice. Family Practice 2002; 19: 442–447.
Received 6 September 2001; Revised 22 January 2002; Accepted 13 May 2002.
 |
Abstract |
|---|
 Top Abstract IntroductionAsthmaCriticMethodsResultsDiscussionReferences |
|---|
Background. Introducing decision-support systems as a tool to stimulate the dissemination of clinical guidelines in daily practice has been disappointing. Researchers have argued that integration of such systems with clinical practice is a prerequisite for acceptance. The big question concerns the feasibility of a true integration—if only routinely recorded data are used for such a system, can patient-specific feedback be produced?
Objective. The aim of this study was to assess the feasibility of generating patient-specific feedback based on routinely recorded data in general practice by AsthmaCritic, a decision-support system for asthma and chronic obstructive pulmonary disease (COPD).
Methods. We built the decision-support system AsthmaCritic and assessed its ability to detect asthma and COPD patient records and generate patient-specific feedback by retrospective analysis of routinely recorded data in 103 713 electronic patient records from primary care practices. We grouped feedback into categories of comments by age group (<12 years and 
12 years). The main outcome measures were the number and percentage of ‘triggered’ (selected) asthma and COPD patient records, and the number and percentage of records on which AsthmaCritic produced at least one feedback comment during the 1-year study period, by category of comments.
Results. AsthmaCritic detected 8784 (8.5%) asthma and COPD patient records. During the study period, AsthmaCritic generated 255 664 feedback comments (mean 3.4 per patient visit). The most frequently generated category of comments in the case of patients aged 12 years was ‘non-compliant prescription’ (23.7%), whereas the most frequent category in the case of patients <12 years was ‘non-compliant route’ (31.1%).
Conclusions. This study shows that, using routinely recorded data only, AsthmaCritic is able to detect asthma and COPD patient records for further analysis and to produce patient-specific feedback.
Keywords. Asthma, COPD, decision-support systems, family practice, guideline adherence.
|
Introduction |
|---|
|
TopAbstractIntroductionAsthmaCriticMethodsResultsDiscussionReferences |
|---|
Asthma and chronic obstructive pulmonary disease (COPD) are chronic diseases with a high prevalence, accounting for significant health care expenditure.1 In recent years, the treatment of asthma and COPD has changed considerably. The consecutive guidelines for asthma and COPD issued by the Dutch College of General Practitioners, for example, illustrate the development of new treatment regimens.2–8 Physicians face the challenge of coping with the changing and ever-increasing amount of medical knowledge.9–11 In view of the current emphasis on evidence-based medicine, clinical practice guidelines12 are considered to be an important tool for disseminating new medical knowledge.13–15 Nevertheless, their use in daily practice has been disappointingly low.16–20 Computer-based decision-support systems may facilitate the implementation of guidelines in daily practice.21,22 However, to be successful, many investigators argue that these systems need to be integrated with computer-based patient records.23–26 In the absence of such integration, physicians have to record data already available in the electronic medical record a second time.
In The Netherlands, most GPs have replaced their paper-based patient records with computer-based records; the practitioners themselves record patient data into the computer during patient encounters.25 To code patient data, they use the International Classification of Primary Care (ICPC) for symptoms, procedures and diagnosis.27 Prescriptions are coded according to the Anatomical, Therapeutic and Chemical (ATC) coding system of the World Health Organization.28 The GP may also record data as free text.
As the first, essential step to demonstrate the feasibility of integrated support, we developed AsthmaCritic, a computer-based decision-support system for asthma and COPD, and let it analyse routinely recorded data in electronic patient records of GPs. In this paper, we first describe the system, followed by a description and discussion of our feasibility study.
|
AsthmaCritic |
|---|
|
TopAbstractIntroductionAsthmaCriticMethodsResultsDiscussionReferences |
|---|
The objective of the decision-support system AsthmaCritic is to review the physician’s treatment in the light of the most recently published guidelines. The system generates patient-specific feedback in the form of critiquing comments. These comments review the physician’s diagnostic and therapeutic interventions, thus enabling physicians to reflect on their decisions, while being focused on the patient at hand. AsthmaCritic generates these comments based on data routinely recorded by the GP in an electronic patient record.
The knowledge base of AsthmaCritic is derived predominantly from the asthma and COPD guidelines of the Dutch College of General Practitioners.2–4 Building the knowledge base has been a 3-year iterative process under guidance of a medical content board consisting of four local experts [two GPs (BP and ED), a pulmonologist (SO) and a paediatric pulmonologist (JJ)] and seven national experts. Members of the medical content board reviewed each new version of the knowledge base.
At the end of each patient contact, the electronic patient record activates AsthmaCritic. AsthmaCritic first searches the medical record for clues, triggers, that indicate the possibility of asthma or COPD: ICPC codes for asthma (R96), chronic bronchitis (R91), emphysema (R95) and other chronic pulmonary diseases (R83.4), and the ATC code for prescriptions used in the treatment of asthma or COPD (R03). When AsthmaCritic encounters a trigger, the record is selected for a full analysis. AsthmaCritic subsequently reviews different aspects of the physician’s treatment and may generate feedback. The system does not question the correctness of the data recorded by the physician. For example, if the physician records a diagnosis asthma, AsthmaCritic does not judge the physician’s opinion.
AsthmaCritic presents feedback to the GP as a list of brief comments. The system is able to provide for each comment one or more of the following kinds of additional information: elaborated advice, a further explanation, the applied patient data or the underlying medical knowledge. By selecting a comment (‘clicking the comment’), the GP can access the additional information. If, for example, the system detects a decrease in peak flow, a short comment ‘Alarming situation: decreasing peak flow’ is included in the list; by selecting that comment, the GP can inspect the elaborated advice, the patient data, the interpretation of the measurement and the relevant sections of the guidelines.
|
Methods |
|---|
|
TopAbstractIntroductionAsthmaCriticMethodsResultsDiscussionReferences |
|---|
Study design
To assess the feasibility of our approach, we analysed electronic patient records of >100 000 patients in 28 general practices. This analysis consisted of two stages. AsthmaCritic first examined all complete records to detect triggers, i.e. the identification of data pointing to asthma or COPD; records containing a trigger, the so-called triggered records, were marked for further analysis. Of the triggered records, the system subsequently reviewed each patient contact within the study period (January to December 1996). Reflecting different aspects of treatment, we divided AsthmaCritic’s comments into 12 categories. Table 1
shows a short description and a brief example for each category. The category ‘alarming situations’, for example, comprises those comments that detect a deterioration of the patient’s condition. Adhering to the Guidelines of the Dutch College of General Practitioners, we divided the population into two age groups; one including patients <12 years old, and one including patients 12 years old.2,3
|
Setting
The Department of Medical Informatics of the Erasmus Medical Center Rotterdam collaborates with general practices located in different parts of the country that make their data available for research in primary care;29 in 1996, the study period, the number of collaborating practices was 28. From these practices, we retrieved the electronic patient records of all patients enrolled in 1996; these records subsequently were analysed by AsthmaCritic.
Measurements
We counted the number and calculated the percentage of AsthmaCritic’s triggered records. For the triggered records, we counted the number of comments and the number of contacts, and calculated the average number of comments per contact. For the different categories of comments, we calculated the percentage of triggered records in which at least one comment from that category was made during the study period (counting each instance of a generated comment would yield unrealistic frequencies because of the retrospective nature of the study; physicians could not change their behaviour in response to generated comments, therefore, once a comment was generated and the circumstances did not change, a comment was generated at each contact).
|
Results |
|---|
|
TopAbstractIntroductionAsthmaCriticMethodsResultsDiscussionReferences |
|---|
During the study period, 103 713 patients were enrolled in the 28 practices. Of the 103 713 records, 8784 (8.5%) were selected by AsthmaCritic for further analysis: 53.6% were triggered by diagnosis and 46.4% by medication. Of the 8784 patients with a trigger in their record, 8412 had at least one encounter with the GP during the study period. Of the 8412 patients with at least one encounter, 6190 (73.6 %) were
12 years old (3352 female) and 2222 (26.4 %) were <12 years old (1005 girls). An overview of the results is presented in Table 2.
|
Of the 8412 patients who had at least one encounter with their GP in 1996, AsthmaCritic performed an analysis of all encounters during 1996, taking all information preceding each encounter into account. The 8412 patients who had seen their GP in 1996 had a total of 74 709 encounters with their GP; an average of nine contacts per patient (patients aged
12 years had an average of 10 contacts, mode: 5, SD: 9; and patients aged <12 years had an average of 5.5 contacts, mode: 2, SD: 4). AsthmaCritic reviewed all 74 709 encounters in 1996 and generated in total 255 664 comments, an average of 3.4 comments per encounter. For the different categories of comments, we calculated the percentage of the triggered records in which at least one comment from that category was made during the study period. The results for patients aged 12 years and older are shown in Table 3 and for patients younger than 12 years in Table 4. The most frequently generated category of comments in patients aged 12 years and older was ‘non-compliant prescription’ [of the 6190 triggered records, 1467 (23.7%) at least once in the study period] whereas the most frequent category in patients younger than 12 years was ‘non-compliant route’ of administration (31.1%).
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionAsthmaCriticMethodsResultsDiscussionReferences |
|---|
Integrating decision-support systems with electronic patient records is an important factor in the applicability of such systems in daily practice.23,24 In a previous study, we showed that electronic patient records contain sufficient information for experts to review the treatment of asthma and COPD.30 Based on this study, we built AsthmaCritic, a system that generates critiquing comments using data routinely recorded by GPs in their electronic patient records. In this study, AsthmaCritic selected 8.5% of >100 000 records as belonging to patients with asthma or COPD, which matches with the 5–10% prevalence rate known from Dutch registration networks.5–8,31–33 Of the selected records, AsthmaCritic analysed the medical record for each of the 74 709 encounters, and generated a total of 255 664 comments, an average of 3.4 per encounter.
For patients older than 12 years, the most frequent comment of AsthmaCritic was the category ‘non-compliant prescriptions’ (23.7%). Although comments in this category are also frequent in patients younger than 12 years (13.7 %), the most frequent comment in these younger patients was the category ‘non-compliant route’ (31.1%). Compared with the guidelines for patients aged 12 years, determining the optimal route of administration is difficult in patients younger than 12 years; the route depends on age and the patient’s clinical condition.34 It is, therefore, not surprising that comments in the category ‘non-compliant route’ are much more frequent in patients younger than 12 years old than in those aged 12 years old.
Because decision-support systems regard data with a limited scope, physician interpretation of comments will be needed to determine AsthmaCritic’s clinical relevance. The extent to which physician judgement is required depends on a comments category. For example, in 22.3% of the patients aged over 12 years, AsthmaCritic pointed out the presence of contraindications. Many of these contraindications, however, are relative. AsthmaCritic will point out that asthma is a contraindication for the prescription of cyclo-oxygenase inhibitors. The physician, however, may accept that risk. Another example that underscores the importance of physician interpretation is comments dealing with the frequent use of antibiotics. AsthmaCritic will generate comments when the patient receives four or more courses of antibiotics over a period of 12 months. In 8.6% of the patients aged 12 years and older and 6.1% of those younger than 12 years, AsthmaCritic pointed out that the fourth course of antibiotics in 12 months had been prescribed, and recommended the use of anti-inflammatory medication. However, although the Dutch guidelines recommend anti-inflammatory medication instead of repeated use of antibiotics, the physician may have good reasons to prescribe antibiotics. Other comments alert to clear deviations from the guidelines. For example, 11.0% of the patients aged 12 years and older, and 13.0% of the patients younger than 12 years old received medication with a dose outside the recommended range; frequently, the physician had prescribed too low a dose.
Although most comments of AsthmaCritic are associated with specific recommendations (e.g. the recommendation to start long-acting bronchodilators), comments in the category ‘alarming situations’ (14.7% of the patients aged 12 years and 10.6% of the patients <12 years old) and ‘inconsistent route’ (9.7 % in patients aged 12 years and 3.6% in patients <12 years old) only point out that the patient requires evaluation. AsthmaCritic, for example, detects decreasing peak flow measurements or increased consumption of bronchodilators and draws the attention of the physician to these trends; the clinical response is left to the physician.
Our study with actual electronic patient records from primary care practices shows that AsthmaCritic is both able to select asthma or COPD patient records and to generate patient-specific comments. The number of comments (on average, 3.4 per encounter) is considerable for daily practice. The acceptance of a decision-support system, however, depends not only on the number of comments, but also on the kind of comments generated and the way feedback is presented. If physicians’ behaviour were to be influenced, it is not clear whether the number of comments will increase or decrease in response to that changed behaviour. On the one hand, one can expect the number of generated comments to decrease because the physician may decide to follow the guidelines (e.g. change his dosing schemas or start prescribing non-antibiotic anti-inflammatory medication in the appropriate cases). On the other hand, the system may stimulate a more complete recording of medical data, thereby increasing the system’s ability to generate (more specific) comments. The acceptance of these comments may differ from comments made before the change in behaviour. Field studies will be needed to assess these effects.
AsthmaCritic is developed to be part of a physicians’ working environment. Integration with daily practice is the key. In addition to being able to deliver patient-specific feedback, integration implies leaving the physician in control and—if available—using routinely recorded data. As we have argued, leaving the physician in control is required from a medical point of view. In addition, if a decision-support system has to fit daily practice, the physician should be able to control the system to match his or her available time and needs at any moment. AsthmaCritic, therefore, has to provide the physician with tools enabling him to execute such control. Using routinely recorded data prevents the physician from having to record data twice and prevents workflow interruptions. In a previous study, we observed that routinely recorded data are sufficient for human reviewers to generate patient-specific feedback.30 This study shows that a computer-based decision-support system can generate patient-specific feedback based on routinely recorded data, thereby enabling the physician to reflect on the treatment for an individual patient based on current guidelines. Additional studies will have to assess the validity and usability of the feedback produced and whether AsthmaCritic is able to change physicians’ behaviour with respect to diagnosis and treatment of asthma and COPD.
|
Acknowledgments |
|---|
We wish to thank the members of our national medical content board for their share in the development of the AsthmaCritic knowledge base: Dr B Bottema, Dr PNR Dekhuyzen, Dr EJ Duiverman, Dr EEM van Essen, Dr ThB Voorn, Dr MHJ Vaessen and Dr A van der Kuy.
|
References |
|---|
|
TopAbstractIntroductionAsthmaCriticMethodsResultsDiscussionReferences |
|---|
1 Rutten-van Molken MP, Postma MJ, Joore MA, Van Genugten ML, Leidl R, Jager JC. Current and future medical costs of asthma and chronic obstructive pulmonary disease in The Netherlands. Respir Med 1999; 93: 779–787.[Web of Science][Medline]
2 Bottema BJAM, Fabels EJ, Van Grunsven PM et al. NHG Standaard CARA bij volwassenen: diagnostiek [Guidelines of the Dutch College of General Practitioners: chronic respiratory diseases in adults: diagnostics]. Huisarts Wet 1992; 35: 430–436.
3 Dirksen WJ, Geyer RMM, De Haan M et al. NHG Standaard Astma bij kinderen [Guidelines of the Dutch College of General Practitioners: asthma in children]. Huisarts Wet 1992; 35: 355–362.
4 van der Waart MAC, Dekker FW, Nijhoff S et al. NHG Standaard CARA bij volwassenen: behandeling. [Guidelines of the Dutch College of General Practitioners: chronic respiratory diseases in adults: therapy]. Huisarts Wet 1992; 35: 437–443.
5 Geijer RMM, Thiadens HA, Smeele IJM et al. NHG-Standaard COPD en astma bij volwassenen: diagnostiek [Guidelines of the Dutch College of General Practitioners: chronic obstructive respiratory diseases and asthma in adults: diagnostics]. Huisarts Wet 1997; 40: 415–428.
6 Geijer RMM, van Hensbergen W, Bottema BJAM et al. NHG-Standaard astma bij volwassenen: behandeling [Guidelines of the Dutch College of General Practitioners: asthma in adults: therapy]. Huisarts Wet 1997; 40: 443–454.
7 Geijer RMM, van Schayck CP, van Weel C et al. NHG-Standaard COPD: Behandeling [Guidelines of the Dutch College of General Practitioners: chronic obstructive respiratory diseases: therapy]. Huisarts Wet 1997; 40: 430–442.
8 Dirksen WJ, Geijer RMM, De Haan M, De Koning G, Flikweert S, Kolnaar BGM. NHG-standaard astma bij kinderen [Guidelines of the Dutch College of General Practitioners: asthma in children]. Huisarts Wet 1998; 41: 130–143.
9 Wyatt J. Uses and sources of medical knowledge. Lancet 1991; 338: 1368–1372.[Web of Science][Medline]
10 Haines A, Jones R. Implementing findings of research. Br Med J 1994; 308: 1488–1492.
11 Freemantle N, Grilli R, Grimshaw J, Oxman A. Implementing findings of medical research: the Cochrane Collaboration on Effective Professional Practice. Qual Health Care 1995; 4: 45–47.
12 Field MJ, Lohr KN. Clinical Practice Guidelines: Directions for a New Program. Institute of Medicine, 1990.
13 Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. Br Med J 1996; 312: 71–72.
14 McColl A, Smith H, White P, Field J. General practitioners’ perceptions of the route to evidence-based medicine: a questionnaire survey. Br Med J 1998; 316: 361–365.
15 Moulding N, Fahy N, Foong LH, Yeoh J, Silagy C, Weller D. A Systematic Review of the Current Status of Evidence-based Medicine and its Potential Application to Australian General Practice. Flinders University of South Australia: Department of General Practice, 1997.
16 Woolf SH, Grol R, Eccles M, Grimshaw J. Potential benefits, limitations, and harms of clinical guidelines. Br Med J 1999; 318: 527–530.
17 Grimshaw J, Freemantle N, Wallace S et al. Developing and implementing clinical practice guidelines. Qual Health Care 1995; 4: 55–64.
18 Lomas J, Anderson GM, Domnick-Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med 1989; 321: 1306–1311.[Abstract]
19 Kassirer JP. A report card on computer-assisted diagnosis—the grade: C. N Engl J Med 1994; 330: 1824–1825.
20 Elson RB, Connelly DP. Computerized decision-support systems in primary care. Prim Care 1995; 22: 365–384.[Web of Science][Medline]
21 Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. J Am Med Assoc 1998; 280: 1339–1346.
22 McDonald CJ, Wilson GA, McCabe GJJ. Physician response to computer reminders. J Am Med Assoc 1980; 244: 1579–1581.
23 Linnarsson R. Decision support for drug prescription integrated with computer-based patient records in primary care. Med Inf 1993; 18: 131–142.
24 Miller RA. Medical diagnostic decision support systems—past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc 1994; 1: 8–27.
25 van der Lei J, Duisterhout JS, Westerhof HP et al. The introduction of computer-based patient records in The Netherlands. Ann Intern Med 1993; 119: 1036–1041.
26 Mitchell E, Sullivan F. A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980–97. Br Med J 2001; 322: 279–282.
27 Boersma JJ. ICPC: International Classification of Primary Care: Short Titles and Dutch Subtitles. Utrecht: Nederlands Huisartsen Genootschap, 1995.
28 WHO International working group for drug statistics methodology. Anatomical Therapeutic Chemical (ATC) Classification and Defined Daily Dose (DDD) for Pharmaceuticals and Vitamins. Oslo: WHO Collaborating Centre for Drug Statistics Methodology, 1996.
29 Vlug AE, van der Lei J, Mosseveld BMT et al. Postmarketing surveillance based on electronic patient records: the IPCI project. Methods Inf Med 1999; 38: 339–344.[Web of Science][Medline]
30 Kuilboer MM, van der Lei J, de Jongste J, Overbeek S, Ponsioen B, van Bemmel JH. Exploring the role of an integrated critiquing system: a simulation. J Am Med Inform Assoc 1998; 5: 194–202.
31 Gijsen R, Verkleij H, Dijksterhuis PH, van de Lisdonk EH, Metsemakers JFM, van de Velden J. Ziektespecifieke vergelijking van de geregistreerde morbiditeit in vier huisartsenregistraties: een analyse ten behoeve van VTV-1997 [Disease-specific comparison of recorded morbidity in four general practitioner practices: an analysis on behalf of the Public Health Status and Forecast-1997]. Report No. 431501017. Bilthoven: Rijksinstituut voor Volksgezondheid en Milieu, 1997.
32 Registratienet Huisartsenpraktijken [Data network general practitioner practices]. [Gezondheidsproblemen en Diagnosen in de huisartsenpraktijk [Health care problems and diagnosis in general practitioner practices]. Report No. 1. Maastricht: Rijksuniversiteit Limburg, 1993.
33 de Waal M. RNUH-LEO Basisrapport VI. Report No. 6. Leiden: LUMC, 1998.
34 Barry PW, Fouroux B, Pedersen S, O’Callaghan C. Nebulizers in childhood. Eur Respir J 2000; 10: 527–535.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  D. L. Sanders and D. Aronsky Biomedical Informatics Applications for Asthma Care: A Systematic Review J. Am. Med. Inform. Assoc., July 1, 2006; 13(4): 418 - 427. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||