Family Practice Vol. 20, No. 2, 207-212
© Oxford University Press 2003
Education in Primary Care |
Effect of extension of postgraduate training in general practice on the acquisition of knowledge of trainees
a National Centre for Evaluation of Postgraduate Training in General Practice (SVUH), Utrecht,
b Centre for Quality of Care Research, Universities of Nijmegen and Maastricht and
c Department of Educational Development and Research,
University of Maastricht, The Netherlands.
Correspondence to AWM Kramer, Mauritsstraat 92, 3583 HV Utrecht, The Netherlands; E-mail: Carol_ann{at}tiro.nl
Kramer AWM, Düsman H, Tan LHC, Jansen KJM, Grol RPTM and van der Vleuten CPM. Effect of extension of postgraduate training in general practice on the acquisition of knowledge of trainees. Family Practice 2003; 20: 207–212.
Received 5 March 2002; Revised 6 August 2002; Accepted 4 November 2002.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background. Postgraduate training in general practice aims to develop clinical competence. However, little is known about its effect on trainees’ development of competence.
Objective. Our aim was to investigate whether 3 years of postgraduate training in general practice leads to a higher level of knowledge than 2 years training while maintaining the same structure and educational objectives.
Methods. This retrospective study had a mixed longitudinal design. Trainees of the Dutch postgraduate training in general practice participated. Knowledge was assessed by written progress testing of knowledge relevant to general practice embedded in real life situations. Test results were collected from 1992 to 1999. The results of trainees who received the 2-year and 3-year curriculum were compared.
Results. Both curricula started with the same entrance level and showed the largest acquisition of knowledge at the start and towards the end of training. Both curricula showed stagnation in growth at the end of the training period in which trainees rotate through hospitals, nursing homes and mental health institutions. The level of knowledge at the end of training was significantly higher for the 3-year curriculum.
Conclusion. This study shows that postgraduate training contributes to an increase in knowledge and that a 3-year programme leads to a higher level of knowledge than a 2-year programme. The stagnation in growth found at the end of rotations through hospitals, nursing homes and mental health institutions questions the impact of these rotations on the development of competence and the efficacy of the training as a whole. Further study is needed to draw more firm conclusions.
Keywords. Clinical competence, curriculum, education, educational measurement, family practice, graduate.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Postgraduate training in general practice shows considerable variation in duration and content, even between countries with comparable health care systems.1–7 Although it has been demonstrated that GPs with postgraduate training provide high quality and cost-effective care3 and, moreover, perform better during their career than those who did not receive such training,8 the effect of postgraduate training on development of competence of trainees has only been evaluated rarely.9,10 One study into postgraduate training for psychiatrists demonstrated incremental growth in competence during their 3-year programme.10 Yet, insight into the change in trainees’ competence over time is valuable to evaluate postgraduate training. This is the focus of the study presented here. We investigated the effect of the extension of a postgraduate training programme in general practice on the acquisition of knowledge.
The content of postgraduate training in The Netherlands is based on the basic job description for the GP.11 The curriculum has a clearly outlined structure, being composed of three blocks of equal length, each with specified learning objectives (see Box 1
).12
|
The first and third block are reserved for practical training in a general practice, whereas the second block is dedicated to rotations through hospitals, nursing homes and mental health institutions. This block is placed deliberately after the first to ensure that this training is perceived as a learning environment for a GP trainee rather than a specialist (hospital) training. The learning objectives of the first block are to gain knowledge, skills and attitudes in the field of general practice with emphasis on common complaints, disorders and problems. The goal of the second block is the acquisition of additional knowledge and skills relevant to general practice that cannot be learned satisfactorily in general practice itself. Finally, in the third block, trainees have to integrate their new knowledge and skills into their activities as a GP with emphasis on management of complex situations such as preventive and continuous care. They also have to prepare themselves for independent practice. In addition to the practical training, trainees meet 1 day per week for education in theory and for reflection, and a half-day per week is reserved for self-directed learning.
Since the introduction in 1974 of compulsory postgraduate training in general practice in The Netherlands, there has been discussion about the content, structure and duration of its curriculum.12 The curriculum was extended from 2 to 3 years in 1994. The consequence of this extension was a change in the duration of the three blocks. In the 2-year curriculum, each block took 8 months. In the current 3-year curriculum, each block takes 1 year. An argument for the extension of the curriculum was the assumption that the educational objectives of the training could not be achieved adequately in 2 years. A 3-year curriculum should lead to a higher level of competence. This hypothesis is tested in our study, at least for knowledge. Research has shown that knowledge is core to any kind of clinical competence and that written test performance is predictive of performance in daily practice.9,13–19 We compared the growth in knowledge of trainees involved in the 2-year and 3-year curricula and we tested whether there was a difference in level of knowledge between the curricula at completion of training.
For this comparison, we have used longitudinally assembled assessment data of the 2-year and the 3-year training. Our assessment instrument was a written test that measures overall general practice knowledge relevant to daily practice.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Subjects
All trainees who received the regular 2-year curriculum in the years 1992–1996 and the regular 3-year curriculum in the years 1994–1999 were included in the study.
Instrument
The test of knowledge in our study has a progress-testing format, so that the test reflects the cognitive final objectives of the curriculum.20,21 These tests are administered periodically to all Dutch trainees in general practice at fixed intervals during their training to monitor progress. Naturally, a different test is used on every occasion. Each individual trainee will therefore be assessed a number of times and will demonstrate growth of knowledge as a function of training time. Since 1992, equivalent tests are constructed according to a blueprint, so that the content of each test contains a representative sample of the domain of general practice knowledge.9 The (multidimensional) blueprint is based on the basic job description of GPs.11 Items are classified according to the International Classification of Primary Care (ICPC)22 and four additional dimensions (age of patient, aspects of consultation, urgency of the problem and course of illness). Each test consists of 160 items. All items are embedded in vignettes representing general practice situations to increase their relevance to daily practice and assessment of crucial knowledge instead of knowledge of trivial facts. The response format is of the true–false–don’t know type. An example is shown in Box 2. A national test committee critically reviews the content and wording of all items. To discourage guessing, a score is calculated as the sum of the correct minus incorrect answers (formula scoring) and is expressed as a percentage of the maximum score. The reliability (Cronbach’s alpha) of the knowledge tests used in this study varied from 0.61 to 0.76. The tests did not become more reliable over the years. These are satisfactory reliabilities, particularly for the interpretation of group results such as is used in this study.
|
Procedure and analysis
A retrospective design was adopted. From 1992 to 1996, the test was administered to all Dutch trainees three times per year at fixed times. For practical reasons, the frequency has changed from three times to twice a year since 1996. Test results were used for formative purposes. For both curricula, we collected all available individual test scores and used these to calculate the average test score per training period. This mixed longitudinal design is considered an optimal design for the measurement of change over time.23 As scores were sampled from different training cohorts, each average test score was composed of scores of different tests. In this way, potential differences in difficulty between tests were averaged. This was particularly important because no further actions were taken to standardize the test difficulties over the years. Variable starting times of the training programme and variable number of trainees per starting time resulted in a considerable variation in the number of trainees per training period and in the duration of training time per training period. To allow comparison between training periods, the total training time was divided into equal periods of 4 months. Thus, training period 1 comprised test results of training months 1–4, training period 2 comprised test results of training months 5–8, etc. Average test scores per training period were determined by assembling individual test scores within each training period. A range of 4 months was chosen for practical reasons. In the 2-year curriculum, the average scores of six training periods were calculated (six training periods x 4 months = 24 months). In the 3-year curriculum, the test was not administered in the ninth training period (nine training periods x 4 months = 36 months). Therefore, the average scores of eight training periods were calculated. As a consequence, the level of knowledge at the very end of the 3-year curriculum was not measured.
For the 2-year and 3-year curricula, the mean test scores and 95% confidence intervals (CIs) were calculated per training period. Statistical difference was inferred from non-overlapping 95% CIs.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
In total we collected 11 952 test scores, obtained from 3682 different trainees (1563 of the 2-year curriculum and 2119 of the 3-year curriculum). The number of test scores per training period varied between 64 and 1849, and the number of tests used to calculate the average score per training period varied between six and 13. The mean test scores with 95% CIs of the 2- and 3-year curricula are presented for each training period in Table 1
and Figure 1.
|
|
For both curricula, the entrance level of knowledge was similar, and the greatest acquisition of knowledge occurred in the first 8 months of training. This acquisition was significantly greater for the 3-year curriculum. Thereafter, the level of knowledge in the 2-year curriculum increased less rapidly but continuously, with a spurt towards the end of the curriculum. The growth increment was significant after each training period, except for periods 4 and 5. The fourth training period covered the last 4 months of the second block, and in the fifth period trainees started their third block. In the 3-year curriculum, the number of trainees in the third and sixth training period was rather small in comparison with the other training periods. Following a rapid increase, the acquisition of knowledge levelled off after training period 2 and increased rather slowly up to and including training period 7 to show again a rapid growth in training period 8. While the level of knowledge increased significantly between training periods 2 and 5, in training period 5 and between training periods 5 and 8, this increase was rather small, especially in training periods 6 and 7. Training period 6 covered the last 4 months of the second block, and in the seventh training period trainees started their third block.
After 1 year of training (training period 3) and after 2 years of training (training period 6), both curricula showed comparable levels of knowledge. Finally, the level at the last measurement of the 3-year curriculum (30 months training) was significantly higher than at the end of the 2-year curriculum (24 months training): 55.8% versus 52.8%.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
This study tested the assumption that a 3-year postgraduate training programme in general practice would lead to a higher competence level than a 2-year programme. Competence was assessed by written progress testing of general practice knowledge embedded in real life situations. Our results show that the overall level of knowledge with the 3-year curriculum is higher after 2.5 years training than with the 2-year curriculum after 2 years training. The difference (3%) is statistically significant and in our view meaningful when compared with a total increase in knowledge of 15% in the 2-year curriculum and when compared with the average standard deviation of scores (10%). A 3% gain is about one-third of the standard deviation of scores. The difference would probably have been larger if a real end measurement of the 3-year curriculum had been available.
The study was conducted to provide insight not only into the level of knowledge at the end point of both curricula but also into the acquisition of knowledge during training. The results with the 2-year curriculum show a continuous growth of knowledge with a spurt at the beginning and at the end of the training programme and a levelling off at the end of the second block and the beginning of the third block. The pattern for the 3-year curriculum is comparable with that of the 2-year curriculum with respect to the greatest acquisition of knowledge at the beginning and towards the end of the training programme, and the rather continuous, but less rapid, increase during the rest of the training time. However, it is different in the duration of the period of smaller increase (20 months in the 3-year curriculum versus 12 months in the 2-year curriculum), and in the indication of a decline in knowledge in training period 6. However, due to the low number of trainees in this training period, this finding has to be interpreted carefully.
In both curricula, most stagnation in growth occurs at the end of the second block, in which trainees rotate through hospitals, nursing homes and mental health institutions, and at the start of the third block, where they return to training in general practice. The stagnation at the end of the second block can be explained by (i) a lack of acquisition of new general practice knowledge; (ii) a lack of assimilation of this new knowledge due to a learning environment in which it is not needed; and (iii) a loss of acquired general practice knowledge because there is no need for this knowledge to be used. All three explanations seem possible. It does not necessarily indicate that trainees do not acquire new knowledge and skills. Perhaps other knowledge, not measured by the test of this study, is acquired in this period or the acquisition of skills is more pronounced. Because the only insight we have into the contribution of the second block to the development of competence of trainees is the results of this study, further research into the second block is necessary. This research should aim to determine the nature and extent of its contribution to the development of competence of trainees and to the efficacy of the training as a whole. The stagnation in growth at the start of the third block is harder to explain. Perhaps trainees need time to become accustomed to their new learning environment or they make up for what they have missed.
When we compare the results of this study with those of a similar study into the growth of knowledge in medical school, we see that the level of knowledge at the start of postgraduate training approximates the level at the end of medical school (38% versus 40%).24 At the end of postgraduate training, the level of knowledge has increased by 15% in the 2-year curriculum and by 18% in the 3-year curriculum. In our view,15 this a considerable increase in comparison with the increase of 36% found in undergraduate medical training across 6 years.
To our knowledge, this is the first study into the growth of knowledge during postgraduate training in general practice. Although a controlled research design would give the best evidence,25,26 such a design is impossible when complete curricula are compared. In our study, we have aimed at the best available evidence by collecting over the years a large number of test scores for both curricula. Even with this design, some methodological shortcomings remain. Firstly, the two curricula were in place at different times. Although educational objectives and the structure of the curricula remained the same, changes in content of the training programme seem inevitable due to new developments in the perception of good general practice and education. This drawback has been solved to some extent by the fact that each knowledge test is composed of recent knowledge supported by recent literature. Another time order effect could be due to changes in quality across trainee cohorts starting postgraduate training (e.g. changes in high school or in gender representation). This may influence learning styles and development of competence. As shown in our study, it did not influence the level of knowledge at the start of postgraduate training. Secondly, the two curricula would have been more adequately compared when the assessment of the trainees of the 2-year curriculum was continued in their first year after completing their postgraduate training. By doing so, the effect of expiration of time without structured education on the development of knowledge could have been examined. Despite these shortcomings, this study has provided some insight into the effect of postgraduate training in general practice on the development of knowledge over time.
In conclusion, the extension of the 2-year postgraduate training by 1 year, while maintaining the same structure and educational objectives, leads to a higher level of general practice knowledge. In the 2-year and in the 3-year curriculum, most acquisition of knowledge occurs in the first 8 months of training and towards the end of training. Both curricula show the largest stagnation in growth at the end of the second block. This finding questions the impact of training in hospitals, nursing homes and mental health institutions on the development of competence of trainees in general practice and on the efficacy of the training as a whole. Further research into these aspects is needed before more firm conclusions can be drawn.
|
Acknowledgments |
|---|
We would like to thank all trainees who took part in the study. We are grateful for the support from the staff of the eight Dutch institutes for postgraduate training in general practice. The members of the national committee of the knowledge test are thanked for their contribution to these knowledge tests. IJsbrand Kramer improved the English. This study was initiated by the Registration Committee of Postgraduate Training in General Practice (HVRC), financially supported by the Foundation of Postgraduate Training in General Practice (SBOH) and executed by the National Centre for Evaluation of Postgraduate Training in General Practice (SVUH).
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1 Boerma WGW, De Jong FAJM, Mulder PH. Health Care and General Practice Across Europe. Utrecht: NIVEL, 1993.
2 Grol R, Wensing M, Jacobs A, Baker R. Quality Assurance in General Practice. The State of the Art in Europe. Utrecht: NHG, 1993.
3 Haq C, Ventres W, Hunt V et al. Where there is no family doctor: the development of family practice around the world. Acad Med 1995; 70: 370–380.[Web of Science][Medline]
4 Pabst R. Medical education and reform initiatives in Germany. Acad Med 1995; 70: 1006–1011.[Web of Science][Medline]
5 Campbell LM, Murray TS. The effects of the introduction of a system of mandatory formative assessment for general practice trainees. Med Educ 1996; 30: 60–64.[Web of Science][Medline]
6 Zweifler J, Ringel M, Maudlin RK, Blossom HJ. Extended educational sessions at three family residency programs. Acad Med 1996; 71: 1059–1063.[Web of Science][Medline]
7 Cassie JM, Armbruster JS, Bowmer MI, Leach DC. Accreditation of postgraduate medical education in the United States and Canada: a comparison of two systems. Med Educ 1999; 33: 493–498.[CrossRef][Web of Science][Medline]
8 Leigh TM, Young PR, Haley JV. Performances of family practice diplomats on successive mandatory recertification examinations. Acad Med 1993; 68: 912–919.[Web of Science][Medline]
9 Van Leeuwen YD. Growth in knowledge of trainees in general practice. Dissertation. Maastricht: Universitaire Pers Maastricht, 1995.
10 Yudkowsky R. Can resident evaluations demonstrate increase in residents’ skills over time? Acad Med 1999; 74 (10 Suppl): S108–S110.[Medline]
11 Springer MP (ed.) Basic Job Description for the General Practitioner. Utrecht: Dutch National Association of General Practitioners (LHV), 1983.
12 Dubois V, Everwijn S, Van Geldorp G et al. The Construction of a New Curriculum of Vocational Training for General Practice in the Netherlands. Utrecht: Royal Dutch Medical Association (KNMG), 1987.
13 Schmidt H, Norman G, Boshuizen H. A cognitive perspective on medical expertise: theory and implications. Acad Med 1990; 65: 611–621.[Web of Science][Medline]
14 Bordage G. Elaborated knowledge: a key to successful diagnostic thinking. Acad Med 1994; 69: 883–885.[Web of Science][Medline]
15 Regehr G, Norman GR. Issues in cognitive psychology: implications for professional education. Acad Med 1996; 7: 988–1001.
16 Van der Vleuten CPM. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ 1996; 1: 41–67.[CrossRef]
17 Boshuizen HPA, Van der Vleuten CPM, Schmidt HG, Machiels-Bongaerts M. Measuring knowledge and clinical reasoning skills in a problem-based curriculum. Med Educ 1997; 31: 115–121.[Web of Science][Medline]
18 Ramsey PG. Does clinical performance correlate with physicians’ scores on written examinations? In Scherpbier AJJA, Van der Vleuten CPM, Rethans JJ, Van de Steegh AFW (eds). Advances in Medical Education. Dordrecht: Kluwer Academic Publishers, 1997.
19 Ram P, Van der Vleuten C, Rethans J, Schouten B, Hobma S, Grol R. Assessment in general practice: the predictive value of written knowledge tests and a multiple station examination for actual medical performance in daily practice. Med Educ 1999; 33: 197–203.[CrossRef][Web of Science][Medline]
20 Kramer AWM, Pollemans MC. Nationwide progress tests assessing knowledge in vocational training for general practice. In Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra PR (eds). Teaching and Assessing Clinical Competence. Groningen: BoekWerk Publishers, 1990.
21 Van der Vleuten CPM, Verwijnen GM, Wijnen WHFW. Fifteen years of experience with progress testing in a problem-based learning curriculum. Med Teach 1996; 18: 103–109.
22 Lamberts H, Wood M (eds). ICPC. International Classification of Primary Care. Oxford: Oxford University Press, 1987.
23 Van ‘t Hof MA, Roede MJ, Kowalski CJ. A mixed longitudinal data analysis model. Hum Biol 1977; 49: 165–179.[Web of Science][Medline]
24 Verhoeven BH, Verwijnen GM, Scherpbier AJJA, Van der Vleuten CPM. Growth of medical knowledge. Med Educ 2002; 36: 711–717.[CrossRef][Web of Science][Medline]
25 Harden RM, Grant J, Buckley G, Hart IR. Guide No. 1: best evidence medical education. Med Teach 1999; 21: 553–562.[CrossRef]
26 Bligh J, Parsell G. Research in medical education: finding its place. Med Educ 1999; 33: 162–163.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||