Family Practice Advance Access originally published online on May 2, 2007
Family Practice 2007 24(3):276-282; doi:10.1093/fampra/cmm007
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Making sense of patient priorities: applying discrete choice methods in primary care using ‘think aloud’ technique
National Primary Care Research and Development Centre, University of Manchester, Manchester M13 9PL, UK. Correspondence to: Peter Bower; Email: peter.bower@manchester.ac.uk
Received 19 September 2006; Revised 6 February 2007; Accepted 26 February 2007.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Background. Delivering effective health care within limited budgets requires an understanding of patient priorities. Discrete choice experiments (DCEs) provide patients with choices, where each choice differs in terms of certain attributes (such as waiting times, quality of care). Although this technique has significant potential in examining priorities, its use raises practical and conceptual issues. This paper describes the development of a DCE evaluating patient priorities in primary care.
Methods. Twenty patients completed a DCE using a ‘think aloud’ protocol, where they verbalized their thinking while making choices. The analysis examined their decision-making processes.
Results. There was evidence that patients reinterpreted some attributes, and related some to others outside the task. The cost attribute was interpreted in a variety of ways, dominating some patients’ decision-making, being seen as irrelevant by others and being interpreted appropriately by some. The deree to which patients exhibited trading in line with theoretical assumptions also varied. Some choices in the hypothetical task were restricted by their previous experience, but more frequently patients tested the boundaries of the task in ways which directly reflected the primary care context.
Conclusion. Patient interpretation of the discrete choice task was varied and some went beyond the formal boundaries of the task to make their choices. This highlights the importance of piloting attributes, providing clear instructions about the task and developing models of patient decision-making so that responses can be interpreted correctly.
Keywords. Behavioural sciences, health economics, qualititative research, patient involvement, decision science.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Planning publicly funded health services requires understanding patient priorities. Discrete choice experiments (DCEs) can assist in this process. In DCEs, services are described by a range of characteristics (attributes), and the value of a service depends upon these attributes. Individuals are presented with choices, each consisting of a number of attributes with different levels (Fig. 1). DCEs can examine priorities by estimating the relative importance of different attributes and determine whether patients are willing to trade between them.1 The technique is becoming popular in primary care,2–4 but there are significant challenges in its use.
|
Developing attributes in DCEs
DCEs need attributes and levels which are plausible and capable of being traded (i.e. patients are willing to accept a decreased value of one attribute for an increase in another). However, creating appropriate wording for attributes can be challenging, especially with complex concepts such as patient-centredness.5–7
DCEs also require a ‘payment vehicle’ for the interpretation of trade-offs between attributes. The conventional payment vehicle is cost, from which willingness to pay can be derived. However, this is potentially problematic in the context of health care systems which are free at the point of contact.
Making sense of patient responses to DCE
DCEs assume ‘rationality’ (i.e. respondents who behave in line with the axioms of economic theory).8,9 Respondents’ preferences are assumed to exist prior to the task and to be based on thorough reasoning and reflection. They should also exhibit ‘compensatory decision-making’ (i.e. respondents are willing to trade decreases in one attribute for increases in another).
However, there is evidence that individuals use simplifying heuristics when faced with complex decisions,10,11 and that preferences may be constructed at the time rather than existing previous to the task.12 Individuals may also be unwilling to trade, making choices based on a single high priority attribute.13 It is not clear whether behaviour which does not correspond to the assumptions underlying economic theory reflects limitations in the decision maker8 or task-related demand characteristics.14
Exploring DCE methods with think aloud techniques
There has been recent interest in qualitative methods within economics.15 One relevant method is ‘think aloud’. Unlike traditional qualitative interviewing, think aloud involves less of a dialogue between interviewer and respondent. Rather, the focus is on respondents verbalizing their ongoing thoughts during a task, as a way of accessing their decision-making processes.16,17
This paper describes the development of a DCE questionnaire to examine patient preferences in primary care. The questionnaire was being designed for a large survey, but this part of the study used think aloud techniques to both assist in the development of plausible and tradeable attributes, and to examine the process of patient decision-making and its relationship to assumptions underlying DCE technique.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Recruitment
Twenty respondents were recruited through mass e-mailing to university staff, face-to-face recruitment and a recruitment agency. Age and economic status are key predictors of primary care preferences,18 and it was hypothesized that rates of service utilization may also be important. Quota sampling was conducted around these three characteristics. The mass e-mailing to staff provided more respondents than required, and the other methods were used to recruit particular types of respondent to meet the quota.
Procedures
Interviews were conducted on university premises, where informed consent was obtained. Respondents were initially asked to recall personal experiences of good and poor primary care, to orient them to the task. As think aloud is an unfamiliar process, warm-up exercises were employed, including basic arithmetic to familiarize respondents with thinking aloud and a practice DCE using more familiar content (choosing a holiday destination).16
Respondents were then asked to complete two written DCE questionnaires in the presence of two researchers, one (SCS) who had experience of DCE and led on the task, and one (either RM or NM) who had greater experience of interviewing and health services research. Initial DCE questionnaires were developed, based on a previous review.19 One had a broad range of ‘generic’ primary care attributes and one focussed on ‘patient-centred care’ (Fig. 2). Each questionnaire involved eight choices, and each choice involved two primary care consultations with five to six attributes (an example questionnaire is shown in Fig. 1). The exact attributes varied throughout the development process, but Figure 2 shows the final list.
|
The tasks lasted approximately 90 minutes, and were tape-recorded and transcribed verbatim. Respondents were given a small honorarium. If respondents were silent for any length of time, the researchers prompted them (‘if you could keep thinking aloud ...’). It is recommended that disruptions to the think loud process are minimal,16 and the process was designed to encourage thinking aloud rather than a dialogue between respondents and researchers. Therefore, the main input from the researchers were the prompts and responding to occasional requests for clarification. All three researchers were non-clinical health services researchers, only one had significant experience of DCEs, and the entire research team exhibited a range of opinions concerning the utility of DCEs.
Analysis
The initial analysis was informed by the existing literature, 8,9,11,20–23 from which an initial coding frame was developed. Respondent statements were assigned to the frame. As coding progressed, other issues emerged, which were incorporated into the coding frame and applied to new transcripts. When no new issues emerged, the finalized coding frame was applied to all the transcripts. The analysis was iterative, in line with conventional procedures.24 No formal reliability testing was conducted, but initial transcripts were coded by the entire team, and extensive discussions took place to ensure that coding was consistent. Quotes are identified by interviewee number, sex, age, economic status and service utilization. ‘A’ and ‘B’ represent the two choices. Italics have been added to some quotes for emphasis.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Sample
The sample included 11 females (55%), eight high users of primary care (40%), nine patients from manual occupations (45%) and six patients aged 18–35 years (30%), 11 aged 36–59 years (55%) and three aged 60 years or over (15%).
Making sense of attributes
Reinterpretation of attributes..
In some cases, respondents reinterpreted attributes in a way that was subtly different from that presented. For example, the original levels for the interpersonal attribute were ‘formal and businesslike’ and ‘warm and friendly’, but one patient responded as follows:
I think probably I'd go for A in that situation, ... I might feel nervous about talking about personal matters to someone who's formal and distant. (01, male, 58, non-manual, low user)
The next quote shows how another reinterpretation seemed to justify the respondent's choice:
The doctor's formal and businesslike, well you want that sometimes anyway to be professional. You don't want somebody laid back and rocking as you go in I suppose. (03, female, 42, non-manual, low user)
In addition to these reinterpretations, other responses indicated that attributes and their combinations could lead to the assumption of additional attributes outside the formal confines of the task:
A explains things thoroughly. Looks at my social and emotional well-being as well as my symptoms. Doesn't care what I think. Just trying to picture him, he's a nice guy. Go for A. (01, male, 58, non-manual, low user)
The actual pattern of attribute levels in the DCE is based on statistical considerations, and assumes independence of the attributes, but they were not always perceived as such, with respondents perceiving causal relationships, which may impact on the importance attached to attributes:
now you've got the positive thing of it costing a lot more and he's more interested in you because it's costing more. (13, male, 56, manual, high user)This doctor's only doing a thorough exam because he doesn't know you. (09, male, 50, manual, low user)
Expectations about the relationships between attributes could conflict with the patterns actually presented in the DCE, which could cause difficulties:
I suppose I'm, I'm falling into a stereotype view of someone who's formal and businesslike is more likely to be like that than like this. (04, female, 41, non-manual, low user)So he's formal and businesslike but he knows me well ... that's a difficult one that isn't it? Because it kind of conflicts slightly. (15, male, 32, non-manual, high user)
Cost as a payment vehicle.. The pilot was designed to test the feasibility of including a cost attribute, and the questionnaire introduction asked patients to imagine that they were in a country like the US where they would have to pay for health care. However, respondents were explicitly informed that there was no plan to introduce charges on the basis of the results.
Reactions to the cost attribute varied. Some made statements dismissing its relevance, which reflected both task instructions and attitudes towards payment for medical care:
I dispute paying for appointments to see doctors, I'm not happy about that, I pay enough already. (18, female, 39, non-manual, high user)So now I've got three and three [preferred attributes associated with each choice], so I need to think really which three would be more important. So I mean the cost is arbitrary anyway. (04, female, 41, non-manual, low user)
In contrast, other respondents demonstrated that issues of cost were dominant rather than irrelevant:
It's this cost thing I, I'm sorry I know it's not really being very helpful and looking at the other options but they are not really a huge problem. (18, female, 39, non-manual, high user)I would definitely choose A, because it's cheaper. (16, female, 49, non-manual, low user)
However, many responses seemed to indicate that patients were interpreting the cost attribute in appropriate ways. This sometimes concerned comparing the attribute with other goods and services:
One thing I have not mentioned. The difference between the £28 or £18. I don't know if it's the twenty pound barrier and I think well it's only a night out or it's only, you're comparing it to something. (11, female, 32, non-manual, high user)Cos those £18 I could probably buy me daughter about 6 Miffy books for that, which is, she's obsessed with that. That's how I tend to rate these things. (15, male, 32, non-manual, high user)
Only very rarely were comments made which related the cost attribute to ability to pay, and it was more frequently related to the perceived severity or urgency:
If you found a lump or something then it wouldn't really matter what the cost is you would pay to see somebody. (10, female, 53, manual, high user)
Making sense of the DCE task
Do respondents trade?.
DCEs depend on a willingness to trade, and there was clear evidence of trading in many cases. However, there was also clear evidence of non-trading, reflected both by the rapidity of the respondent's decision and by the respondent only naming one attribute during the think aloud sequence concerning a particular choice:
I think this is going to be doctor B again for the simple reason that he is asking you about your social and emotional well-being as well the physical symptoms. (10, female, 53, manual, high user)
Some strategies involved a combination with a focus on some attributes and the possibility of trading restricted to these:
I am conscious of the fact that I am not necessarily giving consideration about the examination and I am concentrating on time and choice and money, not necessarily in that order but the part whether he knows me or not doesn't bother me. (11, female, 32, non-manual, high user)
The effects of previous experience.. Consulting in primary care is a familiar task for patients, and some constrained their decisions in the hypothetical DCE task by their experience of current services:
Most of the time you don't get a decision about your treatment anyway, so I think that would be option A. (10, female, 53, manual, high user)You are offered only one appointment time, which I am anyway. (08, male, 62, manual, high user)
Testing the boundaries of the DCE task.. Patients demonstrated a number of methods which they used to test the boundaries of the task, refusing to keep within the boundaries set by the researchers and drawing in features from their own interpretations or their perceptions of the wider context or time frame. The function of such boundary testing was often to deal with decision-making dilemmas and ensure consistency in their responses.
Primary care is characterized by open access and continuity, and some participants made decisions on the basis of assumptions about future consultations:
I think on balance I'll take a gamble on B without any great hope that it's going to solve the problem and go back to A if necessary. (01, male, 58, non-manual, low user)I think if I had seen somebody on this £8 level and if it hadn't worked then I would be prepared to move up to a different option. (11, female, 32, non-manual, high user)
Although primary care still acts as a gatekeeper to specialist services, patients can access other forms of help and choices were influenced by presumptions about the wider health care system:
Well I'll choose option A but if I'm not happy then I'm going to the A&E department anyway. (17, female, 42, non-manual, low user)But for itchy skin, I think that you can probably get something from the chemist anyway before you see your doctor. (08, male, 62, manual, high user)
One significant limitation of DCEs is that they cannot represent the fact that patients and professionals are ‘responsive’ to each others’ behaviour. In some cases, this could be used by patients to assist their decision making by overcoming the perceived limitations of a particular attribute:
Although he doesn't know your medical history, he's warm and friendly so you can talk to him about it. (03, female, 42, non-manual, low user)
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Limitations of the study
The participants were all volunteers and would not be representative of a primary care population. In qualitative research such as this, generalization cannot be made on the grounds of representativeness, but depends on the broad concepts identified proving relevant in other settings.25
Interpretation of the results
The findings build on issues that have been identified in previous studies, as well as identifying new issues. For example, issues such as willingness to trade 21 and the impact of the ‘halo effect’26 have received attention in the literature. However, the available literature examining the issue of the cost attribute in health and primary care is very limited. New issues arising out of the study relate most to the interpretation of attributes and the particular effects of the primary care context (e.g. the effect of knowing that other primary care services exist, and other ways of ‘testing the boundaries’).
Some respondents struggled to engage with the hypothetical task, and their choices seem to be determined by their previous experience.23 However, it seemed that respondents struggled more frequently with the formal constraints of the task. Both types of responses are in line with judgement processes as active and constructive,27 summed up by Bruner: ‘the most characteristic thing about mental life, over and beyond the fact that one apprehends the events of the world around one, is that one constantly goes beyond the information given’.28
A halo effect is a cognitive bias in which the assessment of an individual quality serves to influence the judgement of other qualities,29 and this seemed to be present in a number of cases. Breaking ‘primary care’ into a series of discrete attributes allows understanding of their individual importance, and a previous study comparing DCE with conventional attitude surveys suggested that this process did go some way towards avoiding halo effects.26 The present analysis suggested that the DCE could not entirely avoid halo effects, as patients ‘imposed’ relationships on the discrete attributes, linking them causally or extending them to infer other attributes outside the task. These processes may have important effects on judgement. For example, in an earlier example, two discrete attributes of care were interpreted in terms of a broader judgement about GP personality (i.e. a ‘nice guy’). Such a GP may be seen as less culpable for failures in health care.30
Equally, the boundaries set by the DCE task were not viewed as fixed. This sometimes reflected an avoidance of dilemmas, but cannot be considered simply an artefact of the task, because the methods of testing the boundaries were consistent with patients’ experience of service use. These methods may account for responses which might be considered ‘irrational’ in conventional terms.8
Instructions with the DCE task may need to be expanded so as to highlight its hypothetical nature, and ask patients not to take into account issues ‘outside’ the task in their decision-making. For example, patients should be made to feel that they are not ‘constrained’ by their previous experience, and should feel able to choose options even if those options do not seem to be available currently. However, it may prove difficult to control patient's decision-making in this way, and may be impossible in the context of large-scale postal administration, which is the main method used in DCE.31
The use of a cost attribute is controversial. The present results showed a range of responses, some of which may be likened to ‘protest’ responses where cost is essentially ignored, and others where it may be dominant. Many of the responses showed that the use of the cost attribute was relatively unproblematic, and seemed to be interpreted in appropriate ways. However, the interview context may encourage respondents to be co-operative, and their responses may differ in a postal survey.
The data would suggest that patients are active participants in the DCE task, routinely ‘going beyond the information given’ in their interpretation of attributes, relationships between attributes and the use of other information in decision-making. This might suggest that the technique has limited utility. However, it should be noted that similar issues in interpretation arise when qualitative techniques are utilized to analyse patients’ responses to standardized questionnaires such as the SF36.32 It may be that issues of interpretation in DCE are magnified when dealing with more complex attributes such as interpersonal care, compared with more concrete issues such as waiting times and location of care.33
In practical terms, we would highlight the following issues in the development of effective DCE questionnaires.
- (i) Attributes must be clearly linked to the core concepts under test.
- (ii) Iterative piloting of attributes using qualitative methods is useful to ensure that attributes are plausible and tradeable, and that patients are trading attributes as intended.
- (iii) Respondents should be given clear directions about the hypothetical nature of the task and the boundaries.
- (ii) Iterative piloting of attributes using qualitative methods is useful to ensure that attributes are plausible and tradeable, and that patients are trading attributes as intended.
|
Declaration |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
Funding: National Primary Care Research and Development Centre is funded by the Department of Health.
Ethical approval: None.
Conflicts of interest: None.
|
Acknowledgments |
|---|
The views expressed in this paper are those of the authors alone. The authors would like to thank Dr Arne Rise Hole (University of York) for helpful comments on an earlier draft and the patients who took part in the study.
|
Notes |
|---|
Cheraghi-Sohi S, Bower P, Mead N, McDonald R, Whalley D and Roland M. Making sense of patient priorities: applying discrete choice methods in primary care using ‘think aloud’ technique. Family Practice 2007; 24: 276–282.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionDeclarationReferences |
|---|
1 Ryan M, Bate A, Eastmond C, Ludbrook A. Use of discrete choice experiments to elicit preferences. Qual Health Care (2001) 10:i55–i60.
2 Scott A, Vick S. Patients, doctors and contracts: an application of principal-agent theory to the doctor-patient relationship. Scott J Polit Econ (1999) 46:111–134.[CrossRef][Web of Science]
3 Vick S, Scott A. Agency in health care. Examining patients’ preferences for attributes of the doctor-patient relationship. J Health Econ (1998) 17:587–605.[CrossRef][Web of Science][Medline]
4 Morgan A, Shackley P, Pickin M, Brazier J. Quantifying patient preferences for out of hours primary care. J Health Serv Res Policy (2000) 5:214–218.[Medline]
5 Bower P, Roland M, Campbell J, Mead N. Setting standards based on patients’ views on access and continuity: secondary analysis of data from the general practice assessment survey. Br Med J (2003) 326:258.
6 Mead N, Bower P. Patient-centredness: a conceptual framework and review of the empirical literature. Soc Sci Med (2000) 51:1087–1110.[CrossRef][Web of Science][Medline]
7 Haggerty J, Reid R, Freeman G, Starfield B, Adair C, McKendry R. Continuity of care: a multidisciplinary perspective. Br Med J (2003) 327:1219–1221.
8 Lancsar E, Louviere J. Deleting ‘irrational’ responses from discrete choice experiments: a case of investigating or imposing preferences? Health Econ (2006) 15:797–811.[CrossRef][Web of Science][Medline]
9 Baker R, Robinson A. Responses to standard gambles: are preferences ‘well constructed'? Health Econ (2004) 13:37–48.[CrossRef][Web of Science][Medline]
10 Tversky A, Kahneman D. Judgement under uncertainty: heuristics and biases. Science (1974) 185:1124–1131.
11 Cairns J, van der Pol M, Lloyd A. Decision making heuristics and the elicitation of preferences: being fast and frugal about the future. Health Econ (2004) 11:655–658.[CrossRef]
12 Payne J, Bettman J, Schkade D. Measuring constructed preferences: towards a building code. J Risk Uncertain (1999) 19:243–270.[CrossRef][Web of Science]
13 Kenny P, Hall J, Viney R, Haas M. Do participants understand a stated preference health survey? A qualitative approach to assessing validity. Int J Technol Assess Health Care (2003) 19:664–681.[Web of Science][Medline]
14 Orne M. On the social psychology of the psychological experiment: with particular reference to demand characteristics and their implications. Am Psychol (1962) 17:776–783.[CrossRef]
15 Coast J. The appropriate uses of qualitative methods in health economics. Health Econ (1999) 8:345–353.[CrossRef][Web of Science][Medline]
16 Ericsson K, Simon H. Protocol Analysis: Verbal Reports as Data (1984) Cambridge: MIT Press.
17 Boren M, Ramey J. Thinking aloud: reconciling theory and practice. IEEE Trans Prof Commun (2005) 43:261–278.
18 Jung H, Baerveldt C, Olesen F, Grol R, Wensing M. Patient characteristics as predictors of primary health care preferences: a systematic literature analysis. Health Expect (2003) 6:160–181.[CrossRef][Medline]
19 Cheraghi-Sohi S, Bower P, Mead N, McDonald R, Whalley D, Roland M. What are the key attributes of primary care for patients? Building a conceptual ‘map’ of patient preferences. Health Expect (2006) 9:275–284.[CrossRef][Web of Science][Medline]
20 Ryan M, San Miguel F. Revisiting the axiom of completeness in health care. Health Econ (2004) 12:295–307.[CrossRef][Web of Science]
21 Ryan M, Bate A. Testing the assumptions of rationality, continuity and symmetry when applying discrete choice experiments in health care. Appl Econ Lett (2001) 8:59–63.[CrossRef][Web of Science]
22 Salkeld G, Ryan M, Short L. The veil of experience: do consumers prefer what they know best? Health Econ (2000) 9:267–270.[CrossRef][Web of Science][Medline]
23 San Miguel F, Ryan M, Amaya-Amaya M. ‘Irrational’ stated preferences: a quantitative and qualitative investigation. Health Econ (2005) 14:307–322.[CrossRef][Web of Science][Medline]
24 Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P. Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assess (1998) 2.
25 Green J. Commentary: generalisability and validity in qualitative research. Br Med J (1999) 319:421.[Web of Science]
26 Phillips K, Johnson F, Maddala T. Measuring what people value: a comparison of ‘attitude’ and ‘preference’ surveys. Health Serv Res (2002) 37:1659–1679.[CrossRef][Web of Science][Medline]
27 Forgas J, Williams K, von Hippel W. Responding to the social world: explicit and implicit processes in social judgement. In: Social Judgments: Implicit and Explicit Processes—Forgas J, Williams K, Hippel W, eds. (2006) Cambridge: Cambridge University Press. 1–22.
28 Bruner J. Beyond the information given. In: Studies in the Psychology of Knowing—Angler J, ed. (1957) London: George Allen Unwin. 218–238.
29 Murphy K, Jako R, Anhalt R. Nature and consequences of halo error: a critical analysis. J Appl Psychol (1993) 78:218–225.[CrossRef][Web of Science]
30 Williams B, Coyle J, Healy D. The meaning of patient satisfaction: an explanation of high reported levels. Soc Sci Med (1998) 47:1351–1359.[CrossRef][Web of Science][Medline]
31 Bryan S, Dolan P. Discrete choice experiments in health economics: for better or for worse? Eur J Health Econ (2004) 5:199–202.[CrossRef][Medline]
32 Mallinson S. Listening to respondents: a qualitative assessment of the Short Form 36 Health Status Questionnaire. Soc Sci Med (2002) 54:11–21.[CrossRef][Web of Science][Medline]
33 Gerard K, Lattimer V. Preferences of patients for emergency services available during usual GP surgery hours: a discrete choice experiment. Fam Pract (2005) 22:28–36.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Cheraghi-Sohi, A. R. Hole, N. Mead, R. McDonald, D. Whalley, P. Bower, and M. Roland What Patients Want From Primary Care Consultations: A Discrete Choice Experiment to Identify Patients' Priorities Ann. Fam. Med, March 1, 2008; 6(2): 107 - 115. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||