Family Practice Advance Access originally published online on November 26, 2007
Family Practice 2007 24(6):594-600; doi:10.1093/fampra/cmm066
The use of mixed methodology in evaluating complex interventions: identifying patient factors that moderate the effects of a decision aid
a National Primary Care Research and Development Centre, University of Manchester, Williamson Building, Manchester M13 9PL
b School of Community Based Medicine, University of Manchester, Walmer Street, Rusholme, Manchester M14 5NP, UK
Correspondence to: Joanne Protheroe; Email: j.protheroe{at}manchester.ac.uk.
Received 6 February 2007; Accepted 11 October 2007.
 |
Abstract |
|---|
 Top Abstract BackgroundMethodsResultsDiscussionDeclarationReferences |
|---|
Background. Mixed method research, the combined use of qualitative and quantitative methods, is increasingly popular in health services research, especially as a way to examine ‘complex interventions’. This paper seeks to provide a case example of the use of mixed methods in the analysis of a complex intervention (a computerized interactive decision aid) to test whether their use affords insights into potential moderators of the intervention (i.e. patient factors that were associated with the impact of the intervention).
Methods. We conducted a pragmatic randomized controlled trial (n = 149) of a decision aid in women attending their primary care practitioners with menorrhagia. Alongside the trial, a qualitative study was conducted with interviews with a sample of women who had received the intervention (n = 18). Hypotheses generated by the qualitative study were used to inform subsequent quantitative subgroup analyses.
Results. The results from both studies showed that the decision aid was broadly beneficial. The qualitative study found that women with less formal education reported greater levels of benefit from the intervention. However, quantitative analyses of formal education as a moderator of treatment effect found that the intervention provided the greatest benefit in women with greater formal education. The findings of the qualitative and quantitative studies thus demonstrated a significant inter-method discrepancy. Possible reasons for this discrepancy are discussed.
Conclusions. Mixed methods can illuminate different aspects of an intervention or provide greater insight into particular issues. Health service researchers need to be aware that the qualitative and quantitative results may be discrepant, and methods need to be developed to deal with such discrepancies. However, exploring seemingly discrepant results can lead to greater insight and the development of new hypotheses and avenues of research.
Keywords. Complex interventions, mixed methodology, patient decision aids.
|
Background |
|---|
|
TopAbstractBackgroundMethodsResultsDiscussionDeclarationReferences |
|---|
Mixed method research, the combined use of qualitative and quantitative methods, is increasingly popular in health services research,1–4 especially as a way to examine ‘complex interventions’. Complex interventions are defined as those ‘built up from a number of components, which may act both independently and interdependently’.5,6
Analysis of a complex intervention may involve an analysis of moderators and mediators. A ‘moderator’ specifies for whom or under what conditions the treatment or intervention works and a ‘mediator’ identifies possible mechanisms through which a treatment or intervention might achieve its effects.7,8
Moderators, in specifying for whom or under what conditions the intervention is most successful, help decision making about subsequent trials and routine service delivery, as they suggest to health professionals which patients may benefit most from the intervention. Quantitatively, these are often explored through so called subgroup analyses.
The Medical Research Council has developed a framework for the evaluation of complex interventions, which they suggest requires the use of both qualitative and quantitative methods.5 They describe a sequential, phased approach to evaluation, while others have considered that the evaluation may be much more flexible or iterative in nature.6,9 Regardless of the approach used, it is common to find that the results of mixed method evaluations are published separately.1,10,11
Combining quantitative and qualitative data in the evaluation of a complex intervention has two potential benefits. First, the two methods may complement one another, such that qualitative methods illuminate some aspects of the intervention, while quantitative methods provide insight into others. Second, the use of mixed methods may be multiplicative, in that the combination affords greater insight than either method alone.12 It is the integration of data or findings from the different components of the study that is the key characteristic and unique contribution of mixed method research.12
Decision aids as an exemplar of complex interventions
Decision aids are technologies designed to support the process of decision making, and can range from simple audiotapes and booklets, to complex interactive computer programmes. In the last decade, there has been an increasing number of randomized controlled trials (RCTs), evaluating the use of patient decision aids in enhancing decision making in health care, which show that decision aids consistently improve knowledge, enhance realistic expectations of benefits and harms, reduce decisional conflict and stimulate active participation in decision making.13 However, little is known about how decision aids achieve their effects and what ‘ingredients’ are critical to an intervention.
A patient decision aid is a good exemplar of a complex intervention. In addition to providing information about a condition or treatment, they focus on the benefits and risks of the alternatives, specifically tailored to the individual patient. They provide detailed descriptions of benefits and risks in functional terms including explicit probabilities. Decision aids should then include a values clarifying exercise, such as ranking all the different treatments in order of personal preference.14 Because of this range of potential ‘active ingredients’, the effect of decision aids may be moderated by patient characteristics. For example, certain types of patients may prioritize choice and shared decision making, and thus may benefit more from such interventions than patients who prefer professional-led decision making.
This paper reports on a study to examine the impact of a computerized decision aid on treatment decision making in women with menorrhagia (heavy menstrual bleeding). Menorrhagia is a condition for which there are many equally effective treatments, with different side effect profiles. Patient involvement in choosing between these different treatments is advocated in guidelines from the Royal College of Obstetricians and Gynaecologists,15 making this an ideal condition in which to study the effect of a decision aid.
This paper seeks to provide a case example of the use of quantitative and qualitative methods in the analysis of a complex intervention to test whether the use of both methods achieves the two potential advantages outlined above: the two methods may complement one another, and second, the use of mixed methods may be multiplicative, in that the combination affords insights which the application of either method alone would fail to provide.
|
Methods |
|---|
|
TopAbstractBackgroundMethodsResultsDiscussionDeclarationReferences |
|---|
We conducted a pragmatic RCT of a decision aid in women attending their primary care practitioners with menorrhagia. The aim of the trial was to evaluate whether the addition of a computerized decision aid improved decision making in these women compared to written information alone. Alongside the trial, a qualitative study was conducted with interviews with a sample of women who had received the intervention. The aim of this qualitative study was to examine possible moderators of the decision aid with a view to informing subsequent quantitative subgroup analyses.
Intervention
The computerized decision aid was the Clinical Guidance Tree.16 This is an interactive computer programme designed to provide individualized treatment information, including specific risk information, elicit values and preferences and perform a decision analysis which suggests a preferred treatment option, based on individual preferences (see Box 1). In addition, all women received a commercially available printed information leaflet.
BOX 1 The decision aid intervention.
The Clinical Guidance Tree takes the patient through three distinct stages:
|
Quantitative study
In the RCT, individuals were randomly allocated to either the intervention group (information leaflet and an appointment to use the Clinical Guidance Tree) or the control group (information leaflet only). This trial is reported in detail and published separately.17,18
The primary outcome was the decisional conflict score 2 weeks after the intervention.19 This scale measures the degree of uncertainty about which course of action to take and has been used as an outcome measure in many trials of decision aids.20–24 Secondary outcomes included anxiety, menorrhagia quality of life and knowledge (2 weeks and 6 months after the intervention). Process measures, collected at 6 months, included treatment preferences, hospital outpatient appointments and treatments actually received.
Qualitative study
Women who had received the Clinical Guidance Tree intervention were purposively sampled and invited to take part in an interview. The sampling was iterative: initially women were invited to participate in the interviews 4–6 weeks after they had received the intervention. The baseline demographic data from the quantitative study seemed to suggest the recruitment to the trial of a higher proportion of women from lower socio-economic groups (as defined by employment status and home ownership) and with lower levels of formal education than usually seen in trials of this nature.25–27 Further, the theoretical underpinnings of the intervention, the Decision Support Framework,14 included client characteristics such as socio-economic status and education as two of many suggested factors that may be important in the usefulness of the decision aid. Women were therefore sampled purposively to include both higher and lower levels of formal education and women from higher and lower socio-economic groups.
In this way, the sampling was linked to the theoretical and conceptual framework underlying the intervention, the preliminary quantitative data and the developing qualitative themes. Sampling was continued until no new themes were emerging from the data. JP or a research assistant specifically recruited to the study carried out interviews. Analysis was carried out by all three authors, with agreement of themes and actively seeking out disconfirmatory evidence. The interview schedule was modified iteratively in the light of emerging themes. The schedule covered topics such as symptoms, knowledge about treatment options before the intervention, involvement in decision making, the use of the intervention, the effect of this on consultations since the intervention and the effect on decision making.
|
Results |
|---|
|
TopAbstractBackgroundMethodsResultsDiscussionDeclarationReferences |
|---|
The main results of the RCT are reported in a separate publication,17 for the purposes of this paper a brief overview of the results is described.
Quantitative study
Fifteen of the 19 general practices taking part in the study had positive Jarman scores denoting low socio-economic status. One hundred and forty-nine women consented to participate in the study. There was no difference between consenting and non-consenting women in terms of age.
The Clinical Guidance Tree significantly reduced decisional conflict over and above the effect of the information leaflet [difference in means –16.6, 95% confidence interval (CI) –21.5 to –11.7; P < 0.001]. Neither the computerized decision aid nor the information leaflet had any effect on patient anxiety scores at the 2-week or 6-month follow-up.
At the 6-month follow-up, the use of the decision aid led to a significant increase in the women's knowledge of menorrhagia and its treatment options (difference in means 9.3, 95% CI 1.9–16.6; P < 0.01). More interestingly, without apparently affecting either anxiety or treatments undertaken, the use of the decision aid had a beneficial effect on Menorrhagia Specific Utility Scale scores28 (difference in means 10.9, 95% CI 0.9–21.0; P < 0.05). Process measures demonstrated that women in the intervention group felt more able to express a treatment preference, but this did not seem to have been carried through to being more likely to have received their preferred treatment, or indeed any treatment.
Qualitative study
Eighteen women who received the Clinical Guidance Tree computerized decision aid were interviewed. These women were all white and ranged in age from 31 to 47 years. This range of attributes was similar to the main sample (age range 30–52, 98% white).
Eleven women owned their own homes and were in employment. Five had achieved educational qualifications of A-Levels or above. This range of attributes was again similar to those in the RCT.
The broad themes to emerge from the data were summarized into six main areas: information seeking, legitimacy of condition, effect on decision making, practicalities of intervention use, information from decision aid and empowerment. It is the final theme which is the focus of the present paper.
Overall, the decision aid had a greater effect on decision making than the information leaflet as it was seen as interactive, personalized and containing specific risk information. By presenting women with detailed risk information, eliciting their treatment preferences and giving them a printout of the result of the decision analysis, they reported greater confidence to participate in decisions about future management. This was labelled empowerment, defined as the feeling that an individual has the power or authority to do something.29
However, this reported empowerment was most noticeable in those women with lower formal education (Table 1). Women with less formal education reported feeling that their future consulting behaviour had been, or will be, directly influenced by having used the decision aid. They reported increased feelings of confidence to participate actively in discussions with health professionals about their future management. Data from interviews with women with greater levels of formal education demonstrate that the intervention was reported as useful in terms of a source of information, but not in terms of how future decision-making behaviour will change as a result of it (see Table 1). This latter group of women had often already expressed a feeling of ownership of their treatment decisions ‘I decided what I decided’ (C35016) and ‘I can make choices’ (118030) or ‘we are trying this’ (B87012) etc. They saw the intervention as beneficial in terms of another supplementary resource for information and possibly as legitimizing their treatment decision.
|
These results from the qualitative study generated the hypothesis that the intervention was more effective in women who had less formal education. This hypothesis was then examined in quantitative subgroup analysis.
Quantitative subgroup analyses
The qualitative work described above generated the hypotheses that level of formal education may be an important moderator of the Clinical Guidance Tree intervention. Analyses were conducted to see if the change in the primary outcome [Decisional Conflict Scale (DCS)] was moderated (i.e. exaggerated or diminished) according to the educational status. The participants were considered to have higher formal education if they had achieved educational qualifications of A-Levels or above (n = 32, 23%).
The mean DCS scores in the control and intervention groups at baseline and primary follow-up in each education group are shown in Figures 1 and 2.
|
|
In Figure 1 (the control group), it is apparent that the participants with a higher level of education have greater decisional conflict at baseline. This is reduced by a similar amount at the 2-week follow-up in both the higher and lower educated group. The gradients of the lines on Figure 1 are similar.
At baseline, participants in the intervention group with a higher level of education also had higher levels of decisional conflict. However, Figure 2 shows the intervention successfully reduced decisional conflict in both the higher education and lower education groups, but a greater reduction from baseline decisional conflict has been achieved in the higher education group (signified by the steeper gradient of the line in Fig. 2).
An analysis of covariance was conducted, including a test for interaction between the intervention and educational status which showed that the effect of the intervention is moderated by educational status, but appeared to reduce decisional conflict more successfully in participants with a higher level of education (difference in means –12.9; 95% CI –24.8 to –0.88; P = 0.036).
|
Discussion |
|---|
|
TopAbstractBackgroundMethodsResultsDiscussionDeclarationReferences |
|---|
The results showed that the decision aid was broadly beneficial compared to information leaflets alone. The women interviewed were positive about the use of the decision aid and specifically the impact of its use on decision making. An apparent difference in empowerment reported by women with different levels of formal education suggested a hypothesis that this variable would moderate the effect of the intervention.
However, contrary to the hypothesis generated by the qualitative study, quantitative subgroup analyses suggest that, although the intervention successfully reduces decisional conflict in both groups of differing educational status, it reduces DCS to a significantly greater extent in those of higher educational level. Therefore, the findings of the qualitative and quantitative analyses seem to demonstrate a significant inter-method discrepancy.30 The following discussion will consider four possible explanations, concerning limited power in the quantitative study, biased sampling in the qualitative study, differences in interpretation of decisional conflict in the women with different levels of formal education and differences in the meaning of the concepts of empowerment and decisional conflict.
Previous studies have reported discrepancies where qualitative findings have not been statistically significant in the quantitative analysis.1,2 One obvious explanation is that the lack of significant effect reflects insufficient power. In other words, with a greater sample size and power, similar results may have been demonstrated in the quantitative study. As this study demonstrated a significant effect in the quantitative study (albeit opposite to the predicted direction), this is unlikely to be the explanation in this case. To our knowledge studies reporting apparently directly opposing and significant results such as this have not been reported.
Another explanation is that the findings of the qualitative sample are based on a biased subsample of patients, and the levels of empowerment reported did not generalize to the larger quantitative sample. The number of participants in the qualitative subset was small (n = 18), compared to the whole study (n = 149), although the sampling frame was confined to the 72 women who received the intervention. The women were sampled purposively, linked to both the underlying theoretical framework and the developing account as described in the Methods. The use of a theoretically driven sampling frame greatly increases the transferability of the results over and above a convenience sample,31 and we would argue that the methods applied are in line with current qualitative conventions. However, there remains the possibility that the analysis of the qualitative work is merely over-emphasizing a minor difference in a particular subgroup of participants, which may not generalize to the whole sample.
A third explanation is that the marked difference in baseline decisional conflict between the groups is due to the difference between perceived rather than actual decisional conflict in groups of different educational background. This phenomenon has been described before in a previous study in maternity care, where women with lower educational attainment and manual occupational backgrounds were more likely to report exercising informed choice.32 The authors suggested that patients may perceive levels of informed choice in very different ways, based on the meaning they ascribe to informed choice, expectations concerning its availability and their preferences for exercising it.
In the current study, patients who had a lower level of formal education had lower decisional conflict at baseline. This may reflect the fact that these women were less likely to have been aware that participating in decisions about their treatment was an option, resulting in low levels of perceived conflict. For example, one of the items on the DCS was as follows: ‘I am aware of the choices I have to treat my heavy menstrual bleeding’. If the patient felt that their only options were to continue with symptoms or to have a hysterectomy, they might answer ‘strongly agree’, scoring low decisional conflict. Conversely, patients with high levels of education had higher baseline levels of decision conflict, possibly because they were more aware of the availability of possible treatment choices, but felt uninformed about them and unsupported in terms of deciding between them. The inter-method discrepancy may partly reflect the fact that the DCS is interpreted differently by women with different backgrounds.
If the ‘artificial’ difference in DCS at baseline (driven by these different interpretations) is removed, then the benefits of the decision aid would be similar in the two education groups. The inter-method discrepancy would remain in part, because qualitatively, women with lower educational attainment still reported a greater impact, whereas quantitatively there would be no difference. In this case, the more pronounced qualitative impact of the decision aid in women with lower educational attainment may reflect the fact that their interpretation of the meaning of decisional conflict changes before and after the intervention. That is, these women report a larger impact partly because their expectation of choice was initially so much more limited.
Finally, although there is clearly some overlap between the concepts of empowerment identified in the qualitative work and decisional conflict, it is possible that empowerment may not be fully encompassed by the DCS and therefore the quantitative and qualitative studies were measuring different outcomes. The more pronounced benefit in women with lower levels of education, described by the qualitative study, was an increase in perceived empowerment. Empowerment describes the feeling that an individual has the power or authority to do something, whereas decisional conflict is the uncertainty over which course of action to take. These women were describing an effect that appears greater than simple information provision (see Table 1). They describe that not only were they ‘informed a bit more’ (ID774032) but also they were now in a position to discuss treatment options with the health professional and, if desired, challenge their opinion.
We can postulate why the qualitative study found that women with less formal education appeared to be more ‘empowered’ to participate in decision making following using the decision aid. This could simply be because they had the greatest capacity to benefit; women with more formal education may already have been participating in decision making. Empowerment itself may be either an important mediator of the effect of complex interventions such as decision aids or an important outcome in itself, which requires further study and investigations.
Mixed methods can illuminate different aspects of an intervention or provide greater insight into particular issues. Health service researchers need to be aware that the qualitative and quantitative results may be discrepant, and methods need to be developed to deal with such discrepancies. However, even when discrepant results occur, exploring seemingly discrepant findings performs a useful scientific function leading both to further insights into the issue under study (including possible limitations of outcome measures used) and in suggesting new hypotheses and avenues of research.12
|
Declaration |
|---|
|
TopAbstractBackgroundMethodsResultsDiscussionDeclarationReferences |
|---|
Funding: The Medical Research Council provided funding for the study and support for JP with a Training Fellowship in Health Services Research.
Ethics approval: Ethical approval was obtained from Manchester, Stockport, Salford and Trafford, Tameside and Glossop, South Cheshire, Bolton, Bury and Rochdale, Sheffield and Wirral Local Research Ethic Committees.
Conflicts of interest: None.
|
Acknowledgments |
|---|
The authors would like to express their thanks to the practices and patients involved in this study; Tom Fahey and Tim Peters for their comments on earlier drafts of this paper and Angela Swallow for her help with patient interviews.
|
Notes |
|---|
Protheroe J, Bower P and Chew-Graham C. The use of mixed methodology in evaluating complex interventions: identifying patient factors that moderate the effects of a decision aid. Family Practice 2007; 24: 594–600.
|
References |
|---|
|
TopAbstractBackgroundMethodsResultsDiscussionDeclarationReferences |
|---|
1 Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and qualitative data in health services research—what happens when mixed method findings conflict? [ISRCTN61522618]. BMC Health Serv Res (2006) 6:28.[CrossRef][Medline]
2 Cox K. Assessing the quality of life of patients in phase I and II anti-cancer drug trials: interviews versus questionnaires. Soc Sci Med (2003) 56:921–934.[CrossRef][Web of Science][Medline]
3 Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ (2006) 332:413–416.
4 Stacey D, Graham ID, O'Connor AM, Pomey MP. Barriers and facilitators influencing call center nurses’ decision support for callers facing values-sensitive decisions: a mixed methods study. Worldviews Evid Based Nurs (2005) 2:184–195.[CrossRef][Medline]
5 Medical Research Council. A Framework for Development and Evaluation of RCTs for Complex Interventions to Improve Health (2000) London: Medical Research Council.
6 Campbell M, Fitzpatrick R, Haines A, et al. Framework for design and evaluation of complex interventions to improve health. BMJ (2000) 321:694–696.
7 Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol (1986) 51:1173–1182.[CrossRef][Web of Science][Medline]
8 Kraemer HC, Wilson GT, Fairburn CG, Agras WS. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry (2002) 59:877–883.
9 Campbell NC, Murray E, Darbyshire J, et al. Designing and evaluating complex interventions to improve health care. BMJ (2007) 334:455–459.
10 Montgomery AA, Fahey T, Peters TJ. A factorial randomised controlled trial of decision analysis and an information video plus leaflet for newly diagnosed hypertensive patients. Br J Gen Pract (2003) 53:446–453.[Web of Science][Medline]
11 Weiss MC, Montgomery AA, Fahey T, Peters TJ. Decision analysis for newly diagnosed hypertensive patients: a qualitative investigation. Patient Educ Couns (2004) 53:197–203.[CrossRef][Web of Science][Medline]
12 O'Cathain A, Murphy E, Nicholl J. Integration and publications as indicators of "yield" from mixed methods studies. J Mixed Methods Res (2007) 1:147–163.[CrossRef]
13 O'Connor AM, Stacey D, Entwistle V, et al. Decision Aids for People Facing Health Treatment or Screening Decisions, [A Cochrane Review]. In: The Cochrane Library (Issue 2) (2003) Oxford: Update Software.
14 O'Connor AM, Tugwell P, Wells GA, et al. A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns (1998) 33:267–279.[CrossRef][Web of Science][Medline]
15 Royal College of Obstetricians and Gynaecologists. The Initial Management of Menorrhagia (1998) London: RCOG Press.
16 Dowding D, Swanson V, Bland R, et al. The development and preliminary evaluation of a decision aid based on decision analysis for two treatment conditions: benign prostatic hyperplasia and hypertension. Patient Educ Couns (2004) 52:209–215.[CrossRef][Web of Science][Medline]
17 Protheroe J, Bower P, Chew-Graham C, Peters T, Fahey T. Effectiveness of a computerized decision aid in primary care on decision making and quality of life in menorrhagia: resutls of the MENTIP randomized controlled trial. Med Decis Making (2007) 27:575–584.
18 Protheroe J. Enhancing decision making in menorrhagia in primary care using computerised decision analysis: randomised trial and qualitative evaluation (2005) PhD Thesis, University of Manchester, Manchester, UK.
19 O'Connor A. Validation of a decisional conflict scale. Med Decis Making (1995) 15:25–30.
20 O'Connor A, Tugwell P, Wells GA, et al. Randomized trial of a portable, self-administered decision aid for postmenopausal women considering long-term preventative hormone therapy. Med Decis Making (1998) 18:295–303.
21 Davison BJ, Kirk P, Degner LF, Hassard TH. Information and patient participation in screening for prostate cancer. Patient Educ Couns (1999) 37:255–263.[CrossRef][Web of Science][Medline]
22 Man-Son-Hing M, Laupacis A, O'Connor AM, et al. A patient decision aid regarding antithrombotic therapy for stroke prevention in atrial fibrillation—a randomised controlled trial. JAMA (1999) 282:737–778.
23 Goel V, Sawka CA, Thiel EC, Gort EH, O'Connor AM. Randomized trial of a patient decision aid for choice of surgical treatment for breast cancer. Med Decis Making (2001) 21:1–6.
24 Murray E, Davis H, See Tai S, Coulter A, Gray A, Haines A. Randomised controlled trial of an interactive multimedia decision aid on benign prostatic hypertrophy in primary care. BMJ (2001) 323:493–496.
25 Green MJ, Biesecker BB, McInerney AM, Mauger D, Fost N. An interactive computer program can effectively educate patients about genetic testing for breast cancer susceptibility. Am J Med Genet (2001) 103:16–23.[CrossRef][Web of Science][Medline]
26 Dolan JG, Frisina S. Randomized controlled trial of a patient decision aid for colorectal cancer screening. Med Decis Making (2002) 22:125–139.
27 Rostom A, O'Connor A, Tugwell P, Wells G. A randomized trial of a computerized versus an audio-booklet decision aid for women considering post-menopausal hormone replacement therapy. Patient Educ Couns (2002) 46:67–74.[CrossRef][Web of Science][Medline]
28 Shaw RW, Brickley M, Evans L, Edwards M. Perceptions of women of the impact of menorrhoea on their health using multi-attribute utility assessment. Br J Obstet Gynaecol (1998) 105:1155–1159.[Web of Science][Medline]
29 Christensen M, Hewitt-Taylor J. Empowerment in nursing: paternalism or maternalism? Br J Nurs (2006) 15:695–699.[Medline]
30 Fielding NG, Fielding JL. Linking Data (1986) London: Sage.
31 Greenhalgh T. How to read a paper: papers that go beyond numbers (qualitative research). BMJ (1997) 315:740–743.
32 O'Cathain A, Thomas K, Walters SJ, Nicholl J, Kirkham M. Women's perceptions of informed choice in maternity care. Midwifery (2002) 18:136–144.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||