Family Practice Vol. 16, No. 1, 90-93
© Oxford University Press 1999
Critical appraisal using the READER method: a workshop-based controlled trial
Division of Epidemiology, The Queen's University of Belfast, Mulhouse Building, Royal Victoria Hospital, Belfast BT12 6BJ, UK.
MacAuley D and McCrum E. Critical appraisal using the READER method: a workshop-based controlled trial. Family Practice 1999; 16: 90–93.
Received 19 May 1998; Accepted 7 October 1998.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodResultsDiscussionReferences |
|---|
Background. Critical reading is an important skill for those trying to practice evidence-based medicine. There are a number of recognized structures for critical reading, including the READER model. These methods should be subjected to high-quality studies.
Objectives. We aimed to evaluate the READER method in a practical teaching setting using the highest quality research methodology.
Methods. We carried out a modified randomized controlled trial. Two groups of GP trainers were invited to appraise critically the two articles using either the READER acronym or a semi-structured free appraisal.
Results. Of 99 participants in the workshop, 92 completed the study. One-third of participants (33.7%) read more than five articles per month and those who had been in practice the longest read fewer articles (P < 0.05). Both groups attributed the lowest score to paper 2. The median total scores were higher using the READER method, although only significant for paper 2 (P < 0.05). The median score attributed to the methodology was lower using the READER method than the free appraisal, although this difference was only significant for paper 1 (P < 0.05). Overall, 51% (70% of the READER group) believed that taking part in the exercise would encourage them to be more critical of published articles in the future.
Conclusion. Those using the READER method attributed a higher total score, but were more critical of the methodology than those using a free appraisal. Participants found the study useful and believed that it would be of help in future critical appraisal. The study raises interesting questions about the relative importance to GPs of methodological rigour compared with other factors when evaluating research papers.
Keywords. Controlled trial, critical reading, family practice..
|
Introduction |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
Critical reading is recognized as an important skill for general practice, endorsed by academic bodies in the United Kingdom,1 Canada2,3 and Australia.4 Many doctors cite reading as their main method of keeping upto-date,5 so it is essential that GPs be able to read and appraise research findings appropriately in order to uphold the principles of evidence-based practice. There are a number of published texts6–8 and guides9–11 available, but these are rather complex, and there is a need for a simple, easy to apply guide for GPs.
The READER method,12 first described in 1994, has been used by doctors in a number of different environments.13–15 A recent randomized controlled trial demonstrated how this model may be used by GPs reading research articles in their own time.16 The aim of this study was to evaluate the READER method in a practical teaching setting using the highest quality research methodology. The objectives were to compare the scores using both methods, to examine factors which influence the scores achieved using these methods, and to enter into dialogue with participants in order to improve the method. The null hypothesis was that the READER method was no better than a free appraisal. This paper compares assessments using the READER method with a free appraisal in a controlled trial. The particular strengths are in the methodology, which is based on the randomized controlled trial, and in the large number of GP participants.
|
Method |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
The study population consisted of those GPs attending the Devon and Cornwall Annual Residential Trainers' Workshop who attended the critical reading component. Four groups were chosen for the study using a two-stage random selection. The first stage was selection by seniority. Participants were then randomly distributed into four groups so that each group contained a similar number of those who had been trainers for 5 years or more. Of these four groups (A, B, C and D), two groups (A and B) acted as intervention groups and two groups (C and D) as controls. Groups A and B, the intervention groups, attended a 30 minute teaching session on the READER method of critical appraisal and then performed a critical appraisal of one of two papers using a free appraisal. Validity studies have shown the READER method to be both accurate and repeatable.16 The control groups, C and D, each appraised one of the same two papers using a free appraisal. They had had no prior training or guidance on critical reading and their instructions were simply to rate the papers under two categories: (1) the scientific quality (on a scale of one to 10) and (2) their overall importance to everyday practice (on a scale of one to 25). We also requested some personal and practice details from all participants which we anticipated could possibly be confounding factors. Participants had no prior knowledge of the study before taking part. We chose the papers (for details, see footnote to Table 3
) used in this study from a previous appraisal of the December 1995 edition of the British Journal of General Practice (unpublished). We undertook a pilot study using the proposed methodology among 10 GP registrars attending a half-day release course (unpublished).
|
The chi-square test was used to compare the personal characteristics between groups, and differences in the scores between the two methods were examined using the Mann–Whitney U test.
|
Results |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
Of 104 participants in the residential workshop, 99 attended the critical reading component, of whom 92 completed the study. There were 79 males and 13 females (Table 1
). We compared the groups in order to identify possible confounding factors, but there were no significant differences. The four groups were similar in respect of length of time in practice and practice type, and, although one of the four groups reported that they read more articles from academic journals per month than the others, there was no significant difference overall between the intervention (READER) and control groups. One-third of participants (33.7%) reported that they read more than five articles per month (Table 2), but there was no relationship between the number of articles read per month and either gender or practice type, although those who had been in practice the longest read fewer articles (P < 0.05). The groups ranked the two articles in the same order, based on all components of the two scoring methods (Tables 3 and 4
). Both groups attributed the lowest scores to paper 2.
|
|
|
Both papers attracted a higher median total score using the READER method than using the free appraisal, which was statistically significant (P < 0.05) for paper 2 but not for paper 1.
Comparing the Discrimination component of the READER model with the Scientific quality in the free appraisal, both papers attracted a lower score using the READER method, although this difference was only significant for paper 1 (P < 0.05).
Most participants (75%) stated that they enjoyed taking part. In the READER group, 70% believed that they would be more critical in their future reading, compared with 31% of the free appraisal group (P < 0.001) .
|
Discussion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
There are a number of published methods of critical reading, but, although widely accepted as valid, few have been subjected to objective evaluation. This study used a high-quality methodology to assess the READER method in a practical setting. It included a large number of participants with a high response rate (93%). All participants were GP trainers, but it is possible that this group of trainers may not be representative of all trainers or indeed all GPs.
There was a marked difference in the scores attributed to the two papers, and paper 1 scored higher with both groups than did paper 2. Although the median total score was higher for both papers using the READER method, the difference was not statistically significant for paper 1, but was significant for paper 2 (P < 0.05). Similarly, although both papers attracted a lower median score for their methodology using the READER method, the difference was significant only for paper 1. Thus, the READER method attributed a higher total, but enabled participants to be more critical of the methodology than a free appraisal. The null hypothesis was void.
These findings do, however, raise an interesting point about the READER method and the importance that GPs attribute to methodological rigour. While the READER method discriminated between the methodologies, the overall score was not discriminatory, and indeed the overall score was higher for the paper that scored lowest on methodology. This raises the issue of the relative importance in general practice of the methodological score versus overall score. For GPs it may be that factors other than the methodology have greater significance. Methodological purists would, however, disagree, as they emphasize the importance of the hierarchy of study design. This highlights one of the controversies in critical reading of the literature. Clearly, GPs consider other factors to be as important, if not more so, than methodological rigour.
Paper 1, exploring the use of telephone consultations, was considered to be highly relevant. Neither paper scored highly on Education, which according to the READER model, means that they would not encourage participants to change what they do. The papers did not score highly under the heading ‘Discrimination', but as neither of the papers used high quality methodology, this limits their potential score using the READER scoring grid.
Unlike most other studies, the participants had an opportunity to discuss the papers and their appraisal in a plenary session following the exercise. This revealed an interesting source of pre-test bias at the lower end of the range of scores. For papers that score exceedingly low, the READER method cannot score less than a total of 4. Those using the free appraisal were able to allocate 4 or less. This limits the mathematical comparison of papers that score very low, but is purely a theoretical problem, as papers of this quality are unlikely to make any practical impact.
The original paper on the READER method suggests four ways of reacting to a paper so that a score of 24+ indicates that it is a classic paper that should make an immediate impact on practice, a score of 20–23 indicates that it is a paper of value which may be retained for access, a score of 15–19 means that while not fulfilling all criteria the paper may be of interest, while a paper scoring less than 15 should be ignored. In the discussion, participants felt that there should be only two categories and that one should take note of a paper only if it scores in excess of 20 overall.
In conclusion, the objectives of the study were achieved and the null hypothesis was void. Both the READER method and the free appraisal discriminated between the papers, but there were differences in the scores attributed using the two different methods. Those using the READER method attributed a higher total score but were more critical of the methodology than those using a free appraisal. This may reflect the difference weight attributed by GPs when assessing a paper to factors other than methodology. Participants found the study useful and believed that it would be of help in future critical appraisal.
|
Acknowledgments |
|---|
This research was only possible with the considerable help and enthusiasm of the Devon and Cornwall trainers.
|
References |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionReferences |
|---|
1 RCGP Critical reading question (CRQ). Occasional paper. R Coll Gen Pract 1990; 46: 25–26.
2 Herbert CP. Critical thinking and navel gazing. Can Fam Physician 1988; 34: 777.
3 Perkin RL. Organised curiosity. Can Fam Physician 1988; 34: 779.
4 Holmwood C. Critical thinking within the RACGP Training program and the college examination. Austr Fam Physician 1995; 24: 1673–1675, 1678–1679.[Medline]
5
Smith R. What clinical information to doctors need? Br Med J 1996; 313: 1062–1068.
6 Sackett DL, Haynes RB, Tugwell P. Clinical Epidemiology, A Basic Science for Clinical Medicine. Boston: Little Brown, 1985.
7 Jones R, Kinmonth A-L. Critical Reading for Primary Care. Oxford General Practice Series. Oxford: Oxford University Press, 1995.
8 Crombie IK. The Pocket Guide to Critical Appraisal. London: BMJ Publishing Group, 1996.
9 Haynes RB, McKibbon KA, Fitzgerald D, Guyatt GH, Walker CJ, Sackett DS. How to keep up with the medical literature: 1. Why try to keep up and how to get started. Ann Int Med 1986; 105: 149–153.
10 Fowkes FG, Fulton PM. Critical appraisal of published research: introductory guidelines. Br Med J 1991; 302: 1136–1140.
11
Haynes RB, Sackett DL, Tugwell P. Problems in the handling of clinical and research evidence by medical practitioners. Arch Int Med 1983; 143: 1971–1975.
12 MacAuley D. READER: An acronym to aid critical reading in general practice. Br Gen Pract 1994; 44: 83–85.
13 MacAuley D, Sweeney KS. A new model for critical reading. Forum (Journal of the Irish College of General Practitioners) 1994; 10: 52–54.
14
MacAuley D. Critical reading using the READER acronym at an international workshop. Fam Pract 1996; 13: 104–105.
15 MacAuley D, Sweeney K. Critical reading using the READER acronym by general practitioner registrars in the Republic of Ireland, and Northern Ireland, and experienced general practitioners. Irish J Med Sci 1997; 166: 21–23.
16
MacAuley D, McCrum E, Brown C. ‘Randomised controlled trial of the READER method of critical appraisal in general practice.' Br Med J 1998; 316: 1134–1137.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  C Bleakley and D MacAuley The quality of research in sports journals Br. J. Sports Med., April 1, 2002; 36(2): 124 - 125. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||