Economic Evaluations in Shoulder Pathologies: a systematic review of the literature

Abstract

Background

Economic evaluations provide decision makers with a tool for reducing healthcare costs because they assess both the costs and consequences of healthcare interventions. The purpose of this study was to review the quality of published economic evaluations for shoulder pathologies.

Method

A MEDLINE search from 1980–2010 was conducted to identify articles that contained either “cost” or “economic” in combination with terms for several shoulder disorders and treatments. We selected studies that fit the definition of one of the four routinely performed economic evaluations: cost-minimization, cost-effectiveness, cost-utility and cost-benefit analyses. Study quality was determined by measuring adherence to six established health economic principles as described in the literature.

Results

The search retrieved 942 studies. Of these, 32 studies were determined to be economic evaluations. Fifty three percent of the economic evaluations were published from 2005–2010. Only 25% (n=8) of the 32 studies adhered to all six health economic principles. Publication in a non-surgical journal (p<0.05) or in more recent years (p<0.01) was significantly associated with higher quality.

Conclusion

It is likely that future healthcare resource allocation will be based on the economic feasibility of treatments. Although the number and quality of economic evaluations on shoulder disorders have risen in recent years, the current state of the literature is poor. Given that availability of such data may factor in private and public reimbursement decisions, there is a clear demand for more rigorous economic evaluations.

Level of evidence

Level IV, Systematic Reviews and Meta-Analyses

Introduction

A necessary feature in future healthcare policy will be reducing the rising cost of care in the United States of America. No high-income country spends as much on healthcare as does the United States. In 2009, the United States of America spent 17.3% of its Gross Domestic Product (GDP) on healthcare – more than twice the average spent by the next ten richest countries ^1,46 But despite spending significantly more than other high-income countries, studies have shown that Americans do not enjoy better health outcomes.¹⁹

A significant contributor to rising healthcare spending is the burden of musculoskeletal disorders. In 2004, 30% of the population had a musculoskeletal condition that required medical care, and the annual direct cost for musculoskeletal healthcare was estimated to be $510 billion for the years 2002–2004.⁴⁸ Shoulder disorders occupy a significant and growing proportion of musculoskeletal care. In 2005, chronic shoulder pain was reported by 18.3 million persons aged 18 and older, and it was the third most common musculoskeletal complaint after spine and knee pain.⁴⁸

Economic evaluations play an important role in reducing healthcare costs because they allow decision makers to compare the cost and consequences of two or more healthcare interventions and thus they expose inefficient procedures. ^40,45 Many textbooks and articles offer detailed introductions to economic evaluations and offer various ways of classifying them. ^{5,17,23,27,41} In this paper, we classify economic evaluations into four categories, based upon the definitions published by the British Medical Journal.^27,41 The first is cost minimization. This evaluation is appropriate when the interventions have the same or similar outcomes; thus outcomes are not valued. Rather, the goal is to discover the intervention, which is the least costly. Some economists do not consider cost-minimization evaluations to be full economic evaluations because they only consider costs and not consequences.¹⁷ The second is a cost-effectiveness analysis where outcomes are valued in common natural healthcare units such as change in blood pressure. The third is a cost-utility analysis. This type of evaluation allows the comparison of interventions that result in different outcomes. Outcomes are standardized into health utility measures, which combine both morbidity and mortality in a form that represents the preferences of affected individuals or the society for various healthcare states. The most commonly used health utility measure is the quality adjusted life year (QALY). This measure is generated by multiplying each year lived after the treatment with a utility weight for the health-related quality of life of each year and then adding the products. The weights are generated using methods such as the standard gamble, which elicit individual or societal preferences for various health states. More preferred health states receive greater weights, and the weight scale is anchored at 0(death) and 1(perfect health). The fourth economic evaluation is a cost-benefit analysis. In this type of evaluation, outcomes and costs are valued in monetary terms. The outcomes are converted to monetary units by determining the subject’s willingness to pay for the consequences of the treatment or the subject’s increased productivity.

Economic evaluations of low quality provide limited insights into the efficiency of treatments, and thus it is important that such studies are conducted properly. The objective of our study was to review the quality of economic evaluations in shoulder pathologies based upon established principles in health economics. We were also interested in determining the characteristics that were associated with higher quality studies.

Methods

A MEDLINE search was conducted of studies published between January 1980 and November 2010. The search retrieved English-language studies that contained the words “cost” or “economic” in combination with a range of shoulder pathologies and shoulder specific treatments (See Appendix 1). From this list, studies were included based on the following criteria: First, if they attempted to compare the costs of two or more treatments since a comparison is required for economic evaluations. ¹⁷ Second, if the primary pathology was related to the shoulder. Third, if they reported currency amounts. Studies that claimed that a treatment was cost-effective but did not perform an economic evaluation using currency amounts were excluded.

From each study, we extracted the name of the authors, the title, the intervention and its alternative, the year of publication and the type of economic evaluation conducted. We also noted if the study was published in a surgical journal or not. The classification of journals into surgical versus non-surgical categories is listed in Appendix 2.

The criteria used to judge the quality of the studies was based on six principles developed by Udvarhelyi and his colleagues.⁴⁷ The six criteria are widely accepted in the literature and have been used by previous authors to evaluate the quality of economic evaluations. ^3,6,8,54 The principles also concur with the recommendations proposed by the Panel on Cost-Effectiveness in Health and Medicine.⁴³ The first principle states that an explicit statement of the study’s perspective should be provided. Possible perspectives include: society, patient, healthcare providers. The second principle states that a description of the benefit of the intervention should be provided. The third principle states that the source of costs that were used or considered should be specified. The fourth principle states that if costs or consequences accrue during different periods, then discounting should be used to adjust for the differential timing. A sensitivity analysis to test important assumptions comprises the fifth principle. The sixth principle states that a summary measurement of efficiency, such as a cost-benefit or cost-effectiveness ratio, should be calculated and preferably expressed in marginal or incremental terms unless one alternative or strategy is dominant. Studies were noted as adhering to this principle if they reported a summary measurement either average or incremental. Cost minimization studies were counted as including a summary ratio if they simply stated the costs of the studies. The principles are listed in Table 1.

Table 1.

Economic Evaluations Topics

Topics	# of Studies	%
Rotator Cuff Repair	8	25%
Shoulder Arthroplasty	6	19%
Chronic Shoulder Pain	5	16%
Frozen Shoulder	3	9%
Anesthesia	3	9%
Instability	3	9%
Impingement	3	9%
Diagnosis	1	3%
	32

Each study was given a point for adhering to each of the six principles, and a quality index was generated by summing up the points achieved on the six principles. Thus, a maximum score of 6 and a minimum score of 0 could be obtained. An ordinary least squares regression was performed to assess the characteristics associated with a higher score on the quality index. Statistical analysis was conducted with use of Intercooled Stata for Windows (version 11.1; Stata, College Station, Texas)

Results

The MEDLINE search produced 942 studies. Of these, 32 studies were determined to be economic evaluations that focused on shoulder pathologies.^{2,4,9–15,18,20–22,24–26,28,29,31,33–39,42,49–53} The studies covered eight different possible topics (Table 2). The three most popular topics were rotator cuff disease (25%), shoulder arthroplasty (19%) and operative and non-operative strategies used to treat chronic shoulder pain (16%). Sixty nine percent of the studies were published from 2000 to 2010, while only 3% of the studies were published from 1980 to 1990 (Table 3). Sixty three percent of the studies were published in surgical journals. The studies were conducted in seven different countries (Table 4). The United States of America was the most represented country with 56% of the studies. The second most represented country was the United Kingdom where 19% of the studies were conducted. The majority of the studies were solely cost effectiveness analyses (63%) or cost-utility analyses (22%) (Table 5). Six percent of the studies performed both a cost-utility and cost-effectiveness analyses. Cost-minimization studies made up 9% of the studies. None of the articles constituted a cost-benefit analysis.

Table 2.

Economic Evaluation Years of Publication

Years	# of Studies	%
1980–1985	0	0%
1986–1990	1	3%
1991–1995	2	6%
1996–2000	7	22%
2001–2005	5	16%
2005–2010	17	53%
	32

Table 3.

Publication in a Surgery Specific Journal

Published in a surgery journal?	# of Studies	%
Yes	20	63%
No	12	38%
	32

Table 4.

Country of Study

Country	# of Studies	%
America	18	56%
United Kingdom	6	19%
Netherlands	3	9%
Australia	2	6%
Finland	1	3%
Turkey	1	3%
Norway	1	3%
	32

Table 5.

Type of Study

Type of Study	# of Studies	%
Cost minimization Analysis	3	9%
Cost Effectiveness Analysis	20	63%
Cost Utility Analysis	7	22%
Cost Benefit Analysis	0	0%
Cost Effectiveness and Cost	2	6%
Utility Analysis
	32

The studies retrieved did not consistently follow all six principles. Fifty six percent of the studies adhered to three or less of the six, and only twenty five percent of the studies satisfied all six (Table 6). Table 7 provides a detailed summary of the eight studies that met all of the six principles. Our regression analysis indicates that studies published in a surgery specific journal were significantly associated with a 1.8 point (p<0.05) decrease in their quality indices (Table 8). Studies published in more recent years had better quality indices (regression coefficient=0.156; p<0.01). Conducting a study in an American institution had no impact on its quality.

Table 6.

Quality Index

Quality Index	# of Studies	%
1	2	6%
2	8	25%
3	8	25%
4	4	13%
5	2	6%
6	8	25%
	32	100%

Table 7.

Summary of Studies that met the six minimum health economic standards

	Mather et al. 2010	Vitale et al. 2010	Van Til et al. 2006	Van den Hout et al. et al. 2005	Gereats et al. 2006	Buchbinder et al., 2007	Bruijn et al., 2007	McKenna et al. 2009
Topic	Shoulder Arthroplasty	Rotator Cuff Repair	Chronic Shoulder Pain	Frozen Shoulder	Chronic Shoulder Pain	Frozen Shoulder	Chronic Shoulder Pain	Chronic Shoulder Pain
Country	America	America	The Netherlands	The Netherlands	The Netherlands	Australia	The Netherlands	United Kingdom
Analysis Type	Cost−Utility	Cost−Utility	Cost−Utility	Cost−Utility	Cost−Utility	Cost Utility and Cost Effectiveness	Cost−Effectiveness	Cost−Utility
Intervention and Comparators	Total Shoulder Arthroplasty (TSA) Hemiarthroplasty (HA)	Rotator Cuff Repair Doing Nothing	Percutaneous neuromuscular electrical simulation (P−NMES) Slings Anti−inflammatory injections	High−grade mobilization techniques Low−grade mobilization techniques	Behavioral graded exercise therapy (GET) Usual Care	Manual therapy and directed exercise Placebo	Education and Activation Program (EAP) Usual Care	1st Study. Trained general practitioners Untrained general practitioners 2nd Study Local anesthetic injection (Lignocaine) Steroidal Injection (Cortisone)
Population	64−year−old patient	Patients between 40–80 years Rotator cuff tear verified on arthroscopic evaluation 12 or more month of failed non−surgical treatments	Stroke victims with Hemiplegic pain	Patients who had suffered from adhesive capsulitis for at least a month	Patients with chronic shoulder complaints in the Netherlands	Patients over 18 years old who had pain and stiffness for greater than three months and restriction of passive motion greater than 30 degrees in more than two planes of movement.	Patients older than 18 years old suffering from shoulder complaints that lasted up to thee months	Patients with shoulder pain
Time Frame	Patient’s Lifetime	52 weeks	24 weeks	52 weeks	12 weeks, 52 weeks	6, 12 and 26 weeks	26 weeks	52 weeks
Perspective	Societal	Societal	National Health Insurance Board	Societal	Societal	Societal	Societal	Societal
Costs Included	Direct Medical Costs	Direct Medical Costs	Direct medical costs	Direct medical costs, Direct non−medical costs and Indirect Costs	Direct medical costs, Direct non−medical costs and Indirect Costs	Direct medical costs, Direct non− medical costs and Indirect Costs	Direct medical costs, Direct non− medical costs and Indirect Costs	Direct medical costs and Indirect Costs
Source of Costs	National average Medicare reimbursement rates for the procedures in 2008 dollars.	Charges collected from patients medical records and then converted to costs using a cost to charge ratio.	Natural units obtained from patients and valued using pricing data from the Advisory Board for Healthcare Pricing the manufacturer prices and medical compass 2003	Natural units gathered from patients with quarterly cost questionnaires and valued using Dutch reimbursement rates.	Cost diaries filled by patients and physiotherapist in natural units. Valued using the guidelines of the Dutch Health Care Insurance Counsel.	Patients filled out monthly cost diaries in dollar amounts.	Patients filled out a cost diary every six weeks in natural units. Valued based upon stated assumptions and standard reimbursement rates.	Patients filled out a questionnaire of natural units. Valued using national average unit cost estimate for 2005–2006.
Type of Sensitivity Analysis	One, 2− and 3−way sensitivity analyses performed on all variables	1−way analyses varying costs, QALYs and discount rates	Monte Carlo Simulation	Non−parametric bootstrapping	Alternative analyses using imputation of group mean for outliers	Repetition of analyses with calculation of bootstrap standard errors	Bootstrap Estimation	A sensitivity analysis was performed by removing productivity costs.
Results	TSA resulted in a higher number of average QALYs at a lower cost than HA. Authors concluded it was efficient.	The Rotator Cuff repair yielded a ratio of $3,091.90/QALY by use of the EuroQol. Authors concluded it was efficient.	P−NMES had an incremental cost− utility of €32,821/QALY over injections and €27,085 when compared to slings	Low−grade mobilization was significantly more expensive, and did not significantly affect the change in QALY.	GET had an incremental cost effectiveness ratio of €5,278 per unit of EQ−5D in a year follow− up. Authors concluded it was efficient.	No significant difference in cost, pain, function or quality of life.	EAP was not cost−effective due to the high costs.	Training GPS resulted in an incremental cost of effectiveness ratio of (ICER) £2,813 using the EQ−5D a year later. Authors concluded that training GPs was efficient. Lignocaine had an ICER of £122,000.

Table 8.

Ordinary Least Squares Regression of the Determinants of Higher Quality

	Coefficient (Standard error)
Study published in a Surgery Specific Journal? (1 – yes, 0 – no)	−1.787** (0.678)
Study conducted in an American institution? (1 – yes, 0 – no)	0.185 (0.704)
Year of Publication	0.156*** (0.0429)

^***p<0.01,

^**p<0.05,

^*p<0.1

When we examined each principle individually, we found that only two principles had more than 50% adherence among the studies: listing the source of cost data and stating the benefit of the intervention (Table 9). Ninety seven percent of the studies listed the benefits of the intervention they were examining, and eighty one percent of the studies listed the source of the costs data they used. The remaining four principles had less than 50% adherence. Thirty four percent of the studies examined explicitly stated the perspective from which the study was conducted. Discounting, sensitivity analyses and summary ratios were rarely provided. Thirty eight percent of the studies had cost or consequences which accrued in different time periods, but of these studies only 33% attempted to discount costs or benefits. Only 34% of the studies used sensitivity analyses to test the validity of their assumptions and the robustness of their conclusions, and only 41% of the studies included a summary ratio.

Table 9.

Evaluation of Study Quality

	%
1. Perspective explicitly stated	34%
Private or Public Third Party Payer	36%
Hospital	9%
Patient	0%
Physician	0%
Society	64%
2. Benefits explicitly stated	97%
3. Costs
Source of Costs Stated	81%
Cost Data in Natural Units	41%
Direct Medical Costs	100%
Direct Non−Medical Costs	16%
Indirect Costs	16%
4. Timing
Costs and Benefits have differential timing	38%
Discounting used if costs and benefits have differential timing or rationale provided	33%
5. Sensitivity analysis used	34%
6. Cost−benefit or cost−effectiveness ratio	41%
Average only reported	8%
Incremental and average	23%
Incremental Only	62%
Average, and dominant alternative	15%

Discussion

Our observations reveal a dearth of economic analysis in the area of shoulder pathologies. In this study, we assessed the quality of economic evaluations of shoulder pathologies. We found that while the number of economic evaluations on shoulder disorders has risen with over half the studies conducted in the last five years, the total amount (n=32) remains small. Further, studies displayed low adherence to established health economic principles. As shoulder pathologies are a growing contributor to healthcare costs, it is important that more rigorous economic evaluations be conducted.

Our results correlate with those of previous authors who have also noted the low quality of economic evaluations in the field of orthopedics. Bozic and his colleagues examined the quality of economic evaluations in total hip arthroplasty from January 1966 to July 2002. ⁶ They concluded that though the number of publications had risen, the methodological quality remained inadequate. Brauer and his colleagues examined cost-utility analyses in orthopedic surgery from 1976 to 2001. ⁷ They concluded that cost-utility analyses in orthopedics were of lower quality than in other areas of medicine.

Although few studies included the perspective, it is important to state the perspective from which the economic evaluation is performed because the results will vary based upon the perspective chosen. This difference occurs because the perspective chosen, such as: society, patient or healthcare providers, influences the costs that are included in the study and the outcomes that are measured.¹⁶

Among the studies examined, discounting was used sparingly. Costs and consequences should be discounted when they accrue in different time periods. ^16,27 Costs should be discounted due to inflation, the investment opportunity for money and risk, which means that the value of a dollar next year will always be less than its value today. ⁴⁴ While the decision to discount health outcomes can be controversial, it has become standard practice to discount outcomes as well in order to avoid irrational or absurd conclusions. For example, if outcomes are not discounted then projects could become more economically attractive simply by postponing their implementation into the future. ³⁰ Future studies should ensure that discounting occurs otherwise the results of economic evaluations will be biased.

Much like many of the other principles, sensitivity analyses were rarely used. Economic evaluations are subject to uncertainty because cost and benefit data are often subject to assumptions or come from sources of varying reliability. Therefore, it is important to test the robustness of conclusions to variations in assumptions through sensitivity analyses. ³² If data is collected from individual patients, then statistical analyses should be used to calculate confidence intervals around costs, outcomes and estimated incremental cost effectiveness. ¹⁶ However, if aspects of the data are derived from a single source or based upon expert opinion, then one-way or multi-way analyses should be performed to see how the conclusions changes when the assumptions are changed. ⁴⁴

Decision indices such as cost-effectiveness ratios aid decision makers in choosing between two or more treatments that are competing for their limited resources by effectively summarizing the costs and consequences of adopting a certain treatment. ¹⁷ Economic decisions are often made at the margin; individuals are considering whether to provide a certain intervention as opposed to another. Therefore, marginal analyses in the form of incremental ratios are often more helpful than average ratios. ^16,27 For example, if a new treatment is both more effective and more costly, incremental ratios allow decision makers to determine if the increase in effectiveness is worth the cost of adopting the new treatment. Incremental ratios do not need to be reported if it is clear that one intervention dominates by providing greater benefits at a lower costs. As stated above, less than half of studies included any sort of ratio.

Studies in this review generally provided information on source of their cost data. There are three main costs to be quantified: direct medical costs, direct non-medical and indirect costs. ^5,44 Gold and her colleagues define direct costs as “the value of all goods, services and other resources consumed in providing a healthcare intervention or dealing with the side effects”. ²³ Direct medical costs include: the cost of medications, tests etc; direct non-medical costs include: the cost of transportation, home aids etc.; indirect costs include the cost of work absenteeism. All of the studies included in our review attempted to value direct medical costs from their chosen perspective, although some studies used gross costing estimates such as medicare reimbursement rates as a proxy for direct medical costs in the societal perspective. Although, gross costing may provide satisfactory estimates in certain circumstances, it is important to note that they are not true measures of direct medical costs in the societal perspective, which are best observed through micro-costing techniques such as cost diaries that observe resource use directly.²³ Only 16% of the studies included direct non-medical costs and indirect costs. Although, it is also important to note here that the type of costs included will vary based on the perspective.

Although economic evaluations were seldom conducted properly, when they were, they provide interesting insights into the treatment of shoulder pathologies. For example, in a study that met all six principles, Mather and his colleagues performed a cost-utility analysis from a societal perspective that compared total shoulder arthroplasty to hemiarthroplasty in patients over 64 years old who had osteoarthritis.³³ They found that total shoulder arthroplasty resulted in a greater utility for the patients at a lower cost. In another study that met all six principles, Vitale and his colleagues performed a cost-utility analysis from a societal perspective that examined the cost-utility of rotator cuff repair. ⁵¹ Although, the alternative was not explicitly stated, it appeared that the authors were comparing rotator cuff repairs to an alternative of no treatment. They found that rotator cuff repair resulted in an incremental ratio of $3,091.84/Quality Adjusted Life year (QALY). This ratio is below the threshold of $50,000/QALY that is commonly used, and the authors concluded that rotator cuff repairs were an efficient use of resources.

The results from the studies above indicate that economic evaluations have the potential to identify inefficient treatments. However, the present state of the literature suggests that shoulder specialists will require either increased collaborations with skilled health economists or better training in order to conduct these studies properly. The lower quality index associated with surgical journals calls for a more rigorous review process that should be instituted in these journals.

Our review methods have some limitations. Although we examined an exhaustive list of shoulder keywords, it is possible that some keywords were not included. Furthermore, despite our examination for adherence to certain principles, we did not study the degree to which these principles were followed. For example, we did not attempt to judge if the right parameters were chosen for sensitivity analyses or if discounting was done appropriately. Using these more stringent criteria would probably have decreased our measure of the quality of the publications. We also did not use other search engines such as EMBASE, so it is possible that a few studies not indexed in MEDLINE may have been missed.

Conclusion

Our study reports on the inadequate quality and quantity of publications on the economics of shoulder care. We reviewed the literature on economic evaluations of shoulder pathologies from 1980 – 2010, and assessed their use of six established health economics principles. We found that few economic evaluations have been conducted and the qualities of these publications were poor. Publication in a non-surgical journal and in more recent years was significantly associated with higher quality. If the rising costs associated with shoulder related treatments are to be justified, rigorous economic evaluations are needed in the future.