Read Core Topics in General & Emergency Surgery: Companion to Specialist Surgical Practice Online
Authors: Simon Paterson-Brown MBBS MPhil MS FRCS
Core Topics in General & Emergency Surgery: Companion to Specialist Surgical Practice (12 page)
Whilst cost-effectiveness analysis and cost–utility analysis tell us whether a programme or intervention has better outcomes at additional costs or gains more QALYs, they cannot tell us whether the use of resources to achieve those outcomes is justified. Cost–benefit analysis is a type of evaluation that places a single value, usually in monetary terms, upon the benefits and outcomes from differing programmes of healthcare, i.e. it determines the absolute benefit of both quality and quantity, which is vital in resource allocation. In order to do this the health outcomes from treatment need to be measured in the same units as cost. This can be carried out as an extension to cost–utility analysis where the costs and benefits are converted to the same units. In the UK this is usually done with reference to the WTP threshold set by NICE. For example, if one were to consider a treatment that produces one additional QALY then at a WTP threshold of £20 000 this may be considered equivalent to a £20 000 benefit. If the cost of providing that treatment were £10 000 the treatment would result in a net benefit of £10 000. However, at a cost of £30 000 the net benefit would be − £10 000, implying that there would be a net loss from providing the treatment (as, for example, if it were to displace a more cost-effective use of the available resources). An alternative way of presenting such analysis is in terms of net health benefit rather than economic benefit, so that the result is presented in terms of QALY rather than monetary terms (0.5 or − 0.5 QALY in the above example).
63
The appropriate method of economic evaluation depends upon which choices need to be made and the context within which those choices need to be reached (for example, refer to
Table 2.4
). If outcomes are expected to be the same then the choice is quite straightforward: cost-minimisation analysis may be used. The limitations of cost-effectiveness with disease-specific outcomes should be borne in mind. Cost–utility analysis has increased in popularity in an attempt to standardise and allow comparisons across different conditions and healthcare programmes. Cost–benefit analysis may offer decision-makers an alternative way of viewing such analysis but is dependent upon a predetermined WTP threshold.
Table 2.4
An example of how to choose a type of economic evaluation based on the question
Evaluations will always be subject to elements of uncertainty, be it in terms of resource use, costs or effectiveness. Sensitivity analysis is essential in such circumstances as it allows us to assess how sensitive the study results are to variations in key parameters or assumptions that have been used in the analysis. This allows us to assess whether changes in key parameters will result in savings or costs.
It is possible to undertake sensitivity analysis using as few or as many variables as desired. Commonly, variables such as production variables or discount rates will be used, or if statistical analysis of the variables has been undertaken one can carry out sensitivity analysis around known confidence intervals. Although sensitivity analysis is advocated for evaluations, a review by Briggs and Sculpher
52
found that only 39% of articles reviewed had taken at least an adequate account of uncertainty, while only 14% were judged to have provided a good account of uncertainty. In addition, 24% had failed to consider uncertainty at all. There are differing methods of sensitivity analysis, which are discussed below.
Simple sensitivity analysis, in which one or more parameters contained within the evaluation are varied across a plausible range, is widely practised. With one-way analysis, each uncertain component of the evaluation is varied individually in order to assess the separate impact that each component will have upon the results of the analysis. Multi-way sensitivity analysis involves varying two or more of the components of the evaluation at the same time and assessing the impact upon the results. It should be noted that multi-way sensitivity analysis becomes more difficult to interpret as progressively more variables are varied in the analysis.
52
Threshold analysis involves the identification of the critical value of a parameter above or below which the conclusion of a study will change from one conclusion to another.
64
Threshold analysis is of greatest use when a particular parameter in the evaluation is indeterminate, for example a new drug with a price that has not yet been determined. A major limitation of threshold analysis is that it deals only with uncertainty in continuous variables, meaning that it is normally only useful for addressing uncertainty in analyses with data inputs.
52
In analysis of extremes, a base-case analysis is undertaken that incorporates the best estimates of the inputs and then further analyses consider extreme estimates of the relevant variables. For example, if two alternative treatment strategies are being compared, then both the high and low costs can be considered for both therapies and costs can be assessed for each of the options based upon combinations of these. Analysis of extremes can be particularly effective in situations where a base-case value is known together with a plausible range, but the actual distribution between the outer limits is unknown. However, a problem with this approach is that it does not consider how likely it is that the various scenarios will arise.
52
A final approach to dealing with uncertainty is through the use of probabilistic sensitivity analysis (PSA). This method allows ranges and distributions to be assigned to variables about which we are uncertain, thus allowing for combinations of items that are more likely to take place. For example, it is unlikely that all of the pessimistic factors regarding costs will occur in the evaluation. Techniques such as Monte Carlo simulations allow for the random simultaneous selection of items at designated values and undertake analysis based upon hypothetical patient cohorts. This approach allows the proportion of patients to be estimated for whom one of the options under evaluation is preferred; generally, proportions approaching 100% suggest that the intervention is nearly always preferable under a range of conditions. PSA is generally considered to be the most rigorous form of sensitivity analysis and is gaining widespread use.
65
Value of information analysis is a recent development that is an extension of PSA. The method uses the results of PSA to consider the effect of reducing the uncertainty. Whilst PSA can provide a measure of the uncertainty around a prediction of cost-effectiveness, expected value of perfect information (EVPI) gives a measure that also incorporates the importance of such uncertainty.
66
Further developments of this may help to guide priorities for future research
67
or help to design studies and estimate required sample size.
68
Any formal method for determining the costs and benefits of different treatments that may be used to allocate resources is likely to raise complex ethical issues. In particular, certain methods may create apparent discrimination against certain groups, such as the elderly or disabled, due to reduced capacity to gain from a particular treatment. Such methods may also fail to take into account other issues that are seen by society as being important in allocating resources, such as preferences relating to the process of care and issues such as equity.
69
It is important that such economic methods should not be used without considering these wider implications of the decisions which stem from such analyses.
Most economic evaluations in healthcare use the above-mentioned methods looking at monetary value for new treatment options. There are, however, a number of complex issues in economic evaluation that remain controversial. These include whether to use patient or societal preferences, weighting of QALY to consider severity of disease, carer benefits and the incorporation of a value for innovation. Over the last decade, multi-criteria decision analysis (MCDA) has been suggested as a way to incorporate these complex and often conflicting values in economic evaluation. In MCDA, ‘criteria’ refers to the value taken into consideration. The process involves consideration of multiple criteria, each of which is given a weight in coming to an ‘objective’ decision.
70
Currently, NICE health technology appraisals predominantly use ICER provided by cost–utility analysis. This is considered, using informal methods for incorporating other issues that are not thought to be incorporated in the costs or QALY measures, often by adjusting the WTP threshold that is considered acceptable.
71
Another major change in NICE economic evaluations evolved in appraisals for interventions involving ‘end of life’. As mentioned in earlier sections, NICE considers interventions to be cost-effective if the cost per QALY gained is less than £20 000–30 000. However, in 2009, NICE issued guidance wherein some ‘end-of-life’ interventions or therapeutics that cost more than £30 000 per QALY gained may be given consideration if the treatment is indicated for conditions with a life expectancy of less than 24 months and if there is sufficient evidence that the new intervention improves life expectance by at least 3 months compared to the available NHS treatment and if the treatment if licensed for small population groups.
72
Whether making individual or policy decisions regarding healthcare provision, it is becoming increasingly important for clinicians to take into account evidence about both the effectiveness and the cost-effectiveness of the treatment options. This requires that they examine the available evidence with particular attention to the appropriateness of the outcome measures used and of any techniques for economic analysis. In particular, there is a need for both clinicians and researchers to focus upon outcomes that are relevant to patients and truly represent their views about the relative values of the health states and events that they may encounter. Outcome research and economic evaluation are relatively new areas of healthcare research but they are progressing rapidly. An understanding of the methods used is a prerequisite for an adequate interpretation of the conclusions drawn from such work.
Key points
The choice of outcome measure is important in assessing the results of surgical treatment and needs to be carefully considered.
The measure used should be clinically relevant and preferably have been validated by previous research.
Possible measures relevant to surgery include mortality, condition-specific measures, standard pain questionnaires and generic measures of health-related quality of life.
Quality-adjusted life-years are a commonly used measure of outcome and there are several different ways to produce the weights (utilities) that are required to calculate these.
The estimation of the cost of treatments should include a detailed analysis of the resources used and their valuation, and may require consideration of the timing of incurring various costs.
There are several different methods of economic evaluation, including cost-minimisation, cost-effectiveness, cost–utility and cost–benefit analysis.
The use of cost-effectiveness analysis may allow comparison of health benefits to be gained by expenditure on different treatments but is not without both technical and ethical problems in its application.
1.
Fitzpatrick, R., Davey, C., Buxton, M.J., et al. Evaluating patient-based outcome measures for use in clinical trials.
Health Technol Assess
. 1998;2(14):i–iv. [1–74].
2.
Collin, C., Wade, D.T., Davies, S., et al, The Barthel Index: a reliability study.
Int Disabil Stud
. 1988;10(2):61–63.
3403500
3.
Aissaoui, Y., Zeggwagh, A.A., Zekraoui, A., et al, Validation of a behavioral pain scale in critically ill, sedated, and mechanically ventilated patients.
Anesth Analg
. 2005;101(5):1470–1476.
16244013
4.
Campbell, D.T., Fiske, D.W., Convergent and discriminant validation by the multitrait–multimethod matrix.
Psychol Bull
. 1959;56(2):81–105.
13634291
5.
Perkins, J.M., Collin, J., Creasy, T.S., et al, Exercise training versus angioplasty for stable claudication. Long and medium term results of a prospective, randomised trial.
Eur J Vasc Endovasc Surg
. 1996;11(4):409–413.
8846172
6.
Stockton, D., Davies, T., Day, N., et al. Retrospective study of reasons for improved survival in patients with breast cancer in east Anglia: earlier diagnosis or better treatment.
Br Med J
. 1997;314(7079):472–475.
7.
Sowden, A.J., Sheldon, T.A., Does volume really affect outcome? Lessons from the evidence.
J Health Serv Res Policy
. 1998;3(3):187–190.
10185378
8.
Jones, J., Rowan, K., Is there a relationship between the volume of work carried out in intensive care and its outcome?
Int J Technol Assess Health Care
. 1995;11(4):762–769.
8567208
9.
Brazier, J., Dixon, S., The use of condition specific outcome measures in economic appraisal.
Health Econ
. 1995;4(4):255–264.
8528428
10.
Spilker, B., Molinek, F.R., Jr., Johnston, K.A., et al. Quality of life bibliography and indexes.
Med Care
. 1990;28(12, Suppl):DS1–D77.
11.
Meenan, R.F., Mason, J.H., Anderson, J.J., et al. AIMS2. The content and properties of a revised and expanded Arthritis Impact Measurement Scales Health Status Questionnaire.
Arth Rheum
. 1992;35(1):1–10.
12.
Goldman, L., Hashimoto, B., Cook, E.F., et al, Comparative reproducibility and validity of systems for assessing cardiovascular functional class: advantages of a new specific activity scale.
Circulation
. 1981;64(6):1227–1234.
7296795
13.
Melzack, R., The McGill Pain Questionnaire: major properties and scoring methods.
Pain
. 1975;1(3):277–299.
1235985
14.
Garratt, A.M., Macdonald, L.M., Ruta, D.A., et al. Towards measurement of outcome for patients with varicose veins.
Qual Health Care
. 1993;2(1):5–10.
15.
Guyatt, G.H., Berman, L.B., Townsend, M., et al, A measure of quality of life for clinical trials in chronic lung disease.
Thorax
. 1987;42(10):773–778.
3321537
16.
Carroll, D., Rose, K., Treatment leads to significant improvement. Effect of conservative treatment on pain in lymphoedema.
Prof Nurse
. 1992;8(1):32–33. 35–6.
1480641
17.
Payen, J.F., Bru, O., Bosson, J.L., et al, Assessing pain in critically ill sedated patients by using a behavioral pain scale.
Crit Care Med
. 2001;29(12):2258–2263.
11801819
18.
Young, J., Siffleet, J., Nikoletti, S., et al, Use of a Behavioural Pain Scale to assess pain in ventilated, unconscious and/or sedated patients.
Intensive Crit Care Nurs
. 2006;22(1):32–39.
16198570
19.
Brazier, J., Deverill, M., Green, C., et al. A review of the use of health status measures in economic evaluation.
Health Technol Assess
. 1999;3(9):i–iv. [1–164].
20.
Group, T.E. EuroQol – a new facility for the measurement of health-related quality of life.
The EuroQol Group. Health Policy
. 1990;16(3):199–208.
21.
Rabin, R., de Charro, F., EQ-5D: a measure of health status from the EuroQol Group.
Ann Med
. 2001;33(5):337–343.
11491192
22.
Brazier, J.E., Harper, R., Jones, N.M., et al. Validating the SF-36 health survey questionnaire: new outcome measure for primary care.
Br Med J
. 1992;305(6846):160–164.
23.
Brazier, J.E., Roberts, J., The estimation of a preference-based measure of health from the SF-12.
Med Care
. 2004;42(9):851–859.
15319610
24.
Hunt, S.M., McKenna, S.P., McEwen, J.
Measuring health status
. London: Croom-Helm; 1986.
25.
Torrance, G.W., Measurement of health state utilities for economic appraisal.
J Health Econ
. 1986;5(1):1–30.
10311607
26.
Drummond, M.F., Sculpher, M.J., Torrance, G.W., et al.
Methods for the economic evaluation of healthcare programmes
, 3rd ed. Oxford: Oxford University Press; 2005.
27.
Williams, A. Economics of coronary artery bypass grafting.
Br Med J (Clin Res Ed)
. 1985;291(6491):326–329.
28.
Birch, S., Gafni, A., Cost-effectiveness ratios: in a league of their own.
Health Policy
. 1994;28(2):133–141.
10136058
29.
Drummond, M., Torrance, G., Mason, J., Cost-effectiveness league tables: more harm than good?
Soc Sci Med
. 1993;37(1):33–40.
8332922
30.
Mason, J., Drummond, M., Torrance, G. Some guidelines on the use of cost effectiveness league tables.
Br Med J
. 1993;306(6877):570–572.
31.
Drummond, M., Mason, J., Torrance, G., Cost-effectiveness league tables: think of the fans.
Health Policy
. 1995;31(3):231–238.
10142619
32.
Mehrez, A., Gafni, A., Healthy-years equivalents versus quality-adjusted life years: in pursuit of progress.
Med Decis Making
. 1993;13(4):287–292.
8246700
33.
Buckingham, K., A note on HYE (healthy years equivalent).
J Health Econ
. 1993;12(3):301–309.
10145202
34.
Johannesson, M., Jonsson, B., Karlsson, G., Outcome measurement in economic evaluation.
Health Econ
. 1996;5(4):279–296.
8880165
35.
Brazier, J., Roberts, J., Deverill, M., The estimation of a preference-based measure of health from the SF-36.
J Health Econ
. 2002;21(2):271–292.
11939242
36.
Johannesson, M., O'Conor, R.M., Cost–utility analysis from a societal perspective.
Health Policy
. 1997;39(3):241–253.
10165464
37.
Bleichrodt, H., Johannesson, M., Standard gamble, time trade-off and rating scale: experimental results on the ranking properties of QALYs.
J Health Econ
. 1997;16(2):155–175.
10169092
38.
von Neumann, J., Morgenstern, O.
Theory of games and economic behaviour
. New York: Wiley; 1967.
39.
Torrance, G.W., Thomas, W.H., Sackett, D.L., A utility maximization model for evaluation of health care programs.
Health Serv Res
. 1972;7(2):118–133.
5044699
40.
Hollingworth, W., Deyo, R.A., Sullivan, S.D., et al, The practicality and validity of directly elicited and SF-36 derived health state preferences in patients with low back pain.
Health Econ
. 2002;11(1):71–85.
11788983
41.
Stein, K., Fry, A., Round, A., et al, What value health? A review of health state values used in early technology assessments for NICE.
Appl Health Econ Health Policy
. 2005;4(4):219–228.
16466273
42.
Rasanen, P., Roine, E., Sintonen, H., et al, Use of quality-adjusted life years for the estimation of effectiveness of health care: a systematic literature review.
Int J Technol Assess Health Care
. 2006;22(2):235–241.
16571199
43.
Dolan, P., Modeling valuations for EuroQol health states.
Med Care
. 1997;35(11):1095–1108.
9366889
44.
Brazier, J., Usherwood, T., Harper, R., et al, Deriving a preference-based single index from the UK SF-36 Health Survey.
J Clin Epidemiol
. 1998;51(11):1115–1128.
9817129
45.
Brazier, J., Roberts, J., Tsuchiya, A., et al, A comparison of the EQ-5D and SF-6D across seven patient groups.
Health Econ
. 2004;13(9):873–884.
15362179
46.
Bansback, N., Marra, C., Tsuchiya, A., et al. Using the health assessment questionnaire to estimate preference-based single indices in patients with rheumatoid arthritis.
Arth Rheum
. 2007;57(6):963–971.
47.
Wilson, J., Yao, G.L., Raftery, J., et al. A systematic review and economic evaluation of epoetin alpha, epoetin beta and darbepoetin alpha in anaemia associated with cancer, especially that attributable to cancer treatment.
Health Technol Assess
. 2007;11(13):iii–iiv. [1–202].
48.
Sculpher, M.J., Price, M., Measuring costs and consequences in economic evaluation in asthma.
Respir Med
. 2003;97(5):508–520.
12735668
49.
Stolk, E.A., Busschbach, J.J., Validity and feasibility of the use of condition-specific outcome measures in economic evaluation.
Qual Life Res
. 2003;12(4):363–371.
12797709
50.
Gerard, K., Determining the contribution of residential respite care to the quality of life of children with severe learning difficulties.
Child Care Health Dev
. 1990;16(3):177–188.
2350870
51.
Auld, C., Donaldson, C., Mitton, C., et al. Economic evaluation. In: Detel R., et al, eds.
Oxford textbook of public health, Vol. 2:The methods of public health
. Oxford: Oxford University Press, 2002.
52.
Briggs, A., Sculpher, M., Sensitivity analysis in economic evaluation: a review of published studies.
Health Econ
. 1995;4(5):355–371.
8563834
53.
Cairns, J., Discounting and health benefits: another perspective.
Health Econ
. 1992;1(1):76–79.
1342634
54.
Drummond, M., Maynard, A.
Purchasing and providing cost-effective health care
. Edinburgh: Churchill Livingstone; 1993.
55.
Hartwell, D., Colquitt, J., Loveman, E., et al. Clinical effectiveness and cost-effectiveness of immediate angioplasty for acute myocardial infarction: systematic review and economic evaluation.
Health Technol Assess
. 2005;9(17):iii–iiv. [1–99].
56.
Baboolal, K., McEwan, P., Sondhi, S., et al, The cost of renal dialysis in a UK setting – a multicentre study.
Nephrol Dial Transplant
. 2008;23(6):1982–1989.
18174268
57.
Mason, J., Drummond, M., Reporting guidelines for economic studies.
Health Econ
. 1995;4(2):85–94.
7613600
58.
Raftery, J. Should NICE's threshold range for cost per QALY be raised?
No. Br Med J
. 2009;338:b185.
59.
Towse, A. Should NICE's threshold range for cost per QALY be raised?
Yes. Br Med J
. 2009;338:b181.
60.
Rawlins, M.D., Culyer, A.J. National Institute for Clinical Excellence and its value judgments.
Br Med J
. 2004;329(7459):224–227.
61.
Appleby, J., Devlin, N., Parkin, D. NICE's cost effectiveness threshold.
Br Med J
. 2007;335(7616):358–359.
62.
Martin, S., Rice, N., Smith, P., The link between health care spending and health outcomes: evidence from English programme budgeting data. Centre for Health Economics Research Paper 24. University of York; 2007.
63.
Stinnett, A.A., Mullahy, J. Net health benefits: a new framework for the analysis of uncertainty in cost- effectiveness analysis.
Med Decis Making
. 1998;18(2, Suppl):S68–S80.
64.
Pauker, S.G., Kassirer, J.P., The threshold approach to clinical decision making.
N Engl J Med
. 1980;302(20):1109–1117.
7366635
65.
Claxton, K., Sculpher, M., McCabe, C., et al, Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra.
Health Econ
. 2005;14(4):339–347.
15736142
66.
Felli, J.C., Hazen, G.B., Sensitivity analysis and the expected value of perfect information.
Med Decis Making
. 1998;18(1):95–109.
9456214
67.
Claxton, K.P., Sculpher, M.J., Using value of information analysis to prioritise health research: some lessons from recent UK experience.
Pharmacoeconomics
. 2006;24(11):1055–1068.
17067191
68.
Ades, A.E., Lu, G., Claxton, K., Expected value of sample information calculations in medical decision modeling.
Med Decis Making
. 2004;24(2):207–227.
15090106
69.
Ubel, P.A., DeKay, M.L., Baron, J., et al, Cost-effectiveness analysis in a setting of budget constraints – is it equitable?
N Engl J Med
. 1996;334(18):1174–1177.
8602185
70.
Belton, V., Stewart, T.J.
Multi criteria decision analysis: an integrated approach
. Dordrecht: Kluwer Academic; 2002.
71.
Thokala, P.
Multiple criteria decision analysis for health technology assessment
. Decision Support Unit, NICE; 2011.
72.
(NICE) NIoCE. Appraising life-extending, end of life treatments. Available at
http://www.nice.org.uk/media/E4A/79/SupplementaryAdviceTACEoL.pdf
, 2009. [[accessed 05.01.12]].
