Family Practice Advance Access originally published online on August 28, 2007
Family Practice 2007 24(4):293-294; doi:10.1093/fampra/cmm054
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Editorial |
Economic analysis and complex randomized controlled trials
Editor, Family Practice, Professor of Primary Care, The University of Birmingham, Primary Care Clinical Sciences Building, Edgbaston, Birmingham B15 2TT, UK
Email: b.c.delaney{at}bham.ac.uk
Historically many health care interventions outside therapeutics have been applied without formal evaluation, as they ‘seemed appropriate’. This has been partly as the political imperatives of healthcare reforms have promoted change before evaluation, examples being numerous in the UK National Health Service. However, the evaluation of the complex intervention that inevitably ensues when we change who by or how treatments are carried out is challenging indeed. About 10 years ago the UK Medical Research Council published a framework for the evaluation of complex interventions.1 The framework emphasized the necessary theoretical groundwork to develop an intervention adequately before testing it, yet funders are often reluctant to fund this type of work and researchers may rightly feel that a proposal that has not yet fully developed its interventions may be deemed ‘premature’. Emphasis was also placed on mixed-methods, embedding qualitative and observational work to explain ‘why’ as well as ‘what’. The framework was less detailed in its prescription for economic analysis. Traditionally health economics has been grafted on as a late addition to an existing clinically based randomized controlled trails (RCT), usually underpowered and struggling with inappropriate protocol issues. In order to address this problem, we have seen the rise of the ‘pragmatic RCT’.2 ‘Pragmatic’ would not have been my choice of term, as it implies a compromise of theoretical robustness in the face of practical reality, not what is meant at all. I prefer the term ‘primary cost-effectiveness trial’, as the trial protocol is designed to measure ‘effectiveness’ rather than ‘efficacy’ and to include an economic analysis from the outset. This has fundamental effects on the trial protocol compared with an exploratory trial. We require open rather than blinded designs, as we want to include the placebo-effect in outcomes, and protocol-driven follow-up is kept to a minimum so as not to distort patterns of care. The aim is to measure real-world effects of the intervention in comparison to ‘usual’ or existing practice.
In an individually randomized cost-effectiveness RCT, the effects of treatments on symptoms or health-related quality of life as well as health resource consumption are collected over a period of time and unit costs applied to the resource use to create a total cost for each subject. The analysis of this data has undergone considerable methodological advance in the past 10 years. For decision making we need to know the mean costs and effects, and the statistical uncertainty around these results. Unfortunately, not only is ‘cost-effectiveness’ as defined by the difference in costs divided by the difference in effects between interventions, a ratio, but it is not normally distributed. Costs and effects are also usually highly correlated at subject level, as symptomatic patients consume more resource. ‘Cost-effective’ means that difference in cost/difference in effect is less than the maximum that could be afforded for a unit benefit (known as maximum willingness to pay or the ‘ceiling ratio’). Early attempts to deal with this employed a non-parametric approach based on the proportion of subjects satisfying the decision rule that in order to be cost-effective a subject had to lie below and to the right of a gradient line representing maximum willingness to pay on a plot of cost (y axis) against effect (x axis). A plot of this proportion varying against the maximum willingness to pay is known as a cost-effectiveness acceptability curve (CEAC).3 The interpretation of a CEAC is essentially ‘Bayesian’ in that the proportion can be interpreted as the likelihood of an intervention being cost-effective at a given willingness to pay. Later, parametric evaluation of cost-effectiveness was enabled by rearranging the formula to define net monetary benefit. Incremental net benefit (INB) is calculated as the net monetary gain, weighting QALYs gained (
QALY) by the maximum willingness to pay (
) for a QALY, and subtracting the cost difference (C), INB = QALY – C. Stochastic cost effectiveness analysis can be conducted using standard statistical methods as Incremental Net Benefit is normally distributed.4 Uncertainty around the result is usually displayed using a plot of net monetary benefit against maximum willingness to pay.
Vu et al.5 have faced the challenging task of embedding a cost-effectiveness analysis in a complex cluster-randomized trial of the introduction of a community pharmacist to a nursing home wound care team. This is additionally complex as not only did the trial require the multi-level modelling of effects at nursing home, patient and ulcer levels, but also adjustment for the imbalance in case-mix that can sometimes arise in open and particularly cluster-randomized RCTs. An additional factor here was censoring for ulcers that remained unhealed during follow-up. The use of Incremental Net Benefit for the economic analysis allows these methods to be used, and Figure 3 of their article shows the effect of the maximum willingness to pay on incremental net benefit. All values of willingness to pay give positive values of INB, with a quite gradual slope, indicating that the cost-effectiveness is driven mainly by the cost-savings rather than a dramatic difference in effect. There are some difficulties to be borne in mind when reflecting how portable this result would be on an international scale. Firstly one could just convert the total costs by exchange rates, but health care resources may not ‘convert’ at currency exchange rates, in fact the clearly do not. An alternative would be to reanalyse the data using local resource costs, an approach taken in a recent individual patient data meta-analysis of dyspepsia RCTs from four countries.6 For this to be acceptable we have to assume that healthcare providers are relatively insensitive to differential costs and do not substitute cheaper resources for costly ones. There are some remaining uncertainties with the study by Vu: how much effect their imperfect randomization procedure had, whether the results are applicable on an international scale, and what the ‘essential’ component of the intervention was. However, the study shows that much can be achieved by using appropriate statistical techniques in economic analysis, and is a good example of new directions in health services research.
Notes
Delaney B. Economic analysis and complex randomized controlled trials. Family Practice 2007; 24: 293–294.
References
1 Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, Guthrie B, Lester H, Wilson P, Kinmonth AL. Designing and evaluating complex interventions to improve health care. BMJ (2007) 334:455–459.
2 Roland M, Torgerson DJ. Understanding controlled trials: What are pragmatic trials? BMJ (1998) 316:285.
3 Briggs A, Gray A. Handling uncertainty when performing economic evaluation of healthcare interventions. Health Technol Assess (1999) 3(2).
4 Dinh P, Zhou XH. Nonparametric statistical methods for cost-effectiveness analyses. Biometrics (2006) 62(2):576–588.[CrossRef][Web of Science][Medline]
5 Vu T, Harris A, Duncan G, Sussman G. Cost effectiveness of multidisciplinary wound care in nursing homes: a pseudo-randomised pragmatic cluster trial. Fam Prac (2007) 24:372–379.
6 Ford AC, Qume M, Moayyedi P, Arents NL, Lassen AT, Logan RF, McColl KE, Myres P, Delaney BC. Helicobacter pylori "test and treat" or endoscopy for managing dyspepsia: an individual patient data meta-analysis. Gastroenterology (2005) 128:1838–1844.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||