Cost-effectiveness studies in upper extremity orthopaedic surgery: A systematic review of published literature

Abstract

The goal of this study was to assess the quality and scope of the current cost-effectiveness analysis (CEA) literature in the field of hand and upper extremity orthopaedic surgery. We conducted a systematic review of MEDLINE and the CEA Registry to identify CEAs that were conducted on or after Jan 1, 1997; studied a procedure pertaining to the field of hand and upper extremity surgery; were clinical studies; and reported outcomes in terms of quality-adjusted life years. We identified a total of 33 studies that met our inclusion criteria. The quality of these studies was assessed using the Quality of Health Economic Analysis (QHES) scale. The average total QHES score for all 33 studies was 82 (high-quality). However, over time, a greater proportion of these studies have demonstrated poorer QHES quality (scores <75). Lower scoring studies demonstrated several deficits including failures in identifying reference perspectives; incorporating comparators and sensitivity analyses; discounting costs and utilities; and disclosing funding. It will be important to monitor the ongoing quality of CEA studies in orthopaedics.

Introduction

The cost-effectiveness analysis (CEA) is one of the most commonly utilized tools in economic evaluation of medical care and assesses the value of an intervention relative to a comparator by assessing differences in costs and subsequent quality of life¹.

In the field of orthopaedics, prior systematic reviews have assessed the quality of CEAs across sports² and trauma³, finding the overall quality of studies to be good in these orthopaedic subspecialties. However, there has been no systematic examination of CEAs in upper extremity surgery. Brauer and colleagues conducted two separate studies evaluating the quality of CEAs across all orthopaedic subspecialties from 1976 to 2001 and included a total of only five hand and upper extremity studies^4,5. No subgroup analyses were conducted on these studies.

Cost-effectiveness plays an important role in the field of upper extremity surgery, where a patient’s level of function can have direct and indirect effects on quality of life⁶. There is very little known about the scope and quality of the CEA literature in hand and upper extremity surgery. Our goal was to conduct a systematic review to (1) assess the quality and scope of the current collection of hand and upper extremity CEAs, (2) identify areas for further economic evaluation in the field of hand and upper extremity surgery, and (3) identify opportunities for quality improvement.

Materials and Methods

Overview

This review focuses on cost-utility analyses, which measure health outcomes in quality-adjusted life-years (QALYs) that incorporate subjective valuations of health states by patients^2,5,7. Cost-utility analyses utilize an incremental cost-effectiveness ratio (ICER) which represents the difference in costs divided by the difference in QALYs between two interventions¹. ICERs are evaluated against a willingness-to-pay (WTP) threshold, which represents the maximum threshold cost that society is willing to spend for an additional QALY, commonly $50,000 or $100,000 per QALY⁸. If a procedure’s ICER falls below the WTP threshold, it can be considered a cost-effective alternative to its comparator.

Search and Inclusion Criteria

The systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist⁹. We employed a systematic search of MEDLINE as well as the Cost-Effectiveness Analysis (CEA) Registry by the Center for the Evaluation of Value and Risk in Health at Tufts University¹⁰.

This CEA registry is a well-established and high-quality repository of CEAs^11,12. The registry possesses over 5600 cost-utility analyses and incorporates strict inclusion criteria for papers that are published in English, are CEAs, and measure health benefits in QALYs. The registry excludes reviews, editorials, and methodological articles. We used a supplementary MEDLINE search to capture any studies that met the above inclusion criteria but were not yet included within the CEA registry¹³.

We included studies from the CEA Registry with a keyword pertaining to upper extremity anatomy. Our MEDLINE search incorporated any study 1) pertaining to upper extremity anatomy; 2) conducted on or after Jan 1, 1997; and 3) utilizing the terms quality-adjusted life-years (QALYs) and costs. Following our preliminary searches of the CEA registry and MEDLINE, we included any study in this review that cleared the following exclusion criteria: 1) was conducted on or after Jan 1, 1997; 2) studied a procedure consistent with the field of orthopaedic upper extremity surgery; 3) was a clinical study; and 4) utilized methodology consistent with a cost-utility analysis.

Our rationale for including a time limit in our search was based on the recommendations of the First Panel on Cost-Effectiveness in Health and Medicine, published on October 16, 1996, which outlined the first consensus-based guidelines for the conduct of CEAs¹. We restricted our search from the start of January 1997 to December 2016 to allow adequate time for studies to reflect the guidelines. We restricted our review to clinical studies, excluding any reviews, methodological studies, or studies focusing on non-operative interventions without including a surgical/procedural comparator. We excluded CEAs not incorporating QALYs, cost-benefit analyses, or cost-minimization analyses¹⁴. We restricted our search to studies conducted in humans and in English-only for ease of interpretation. Although a language restriction can possess bias, there is no evidence that it has any effects on resultant data in systematic reviews and meta-analyses¹⁵.

CEA Registry Search

The CEA Registry screening process was searched by “[Anatomic location].” Anatomic location terms included the broad term “upper extremity” and more specific anatomic regions pertinent to upper extremity clinical anatomy: shoulder, glenohumeral, labrum, humerus, elbow, wrist, radius, ulna, scaphoid, carpal, hand, thumb, finger, metacarpal, carpometacarpal, metacarpophalangeal, phalanx, phalanges, intermetacarpal, and interphalangeal. Additionally, we included terms specific to anatomic injuries: “rotator cuff” and “carpal tunnel syndrome.” The CEA Registry yielded a total of 134 studies.

MEDLINE search

We utilized the PubMed interface to search the MEDLINE database¹³. Search terms were grouped into three broad categories: [Anatomic location], [Procedure], and [Cost Study] (Appendix 1). The MEDLINE query resulted in 61 studies, of which 35 were not captured by the CEA Registry search.

These 169 total studies independently underwent a (1) combined title and abstract screening followed by a (2) full text screening separately by two authors (PVR and RAQ) to ensure full adherence to the inclusion criteria. This resulted in a final total of 33 studies to be included in this review (Figure 1).

Figure 1.

Overview of screening process used for systematic review.

Quality Scoring and Data Extraction

We assigned a quality score to each study using the Quality of Health Economic Analysis (QHES) scale¹⁶. The QHES scale ranges from 0 to 100; although there is no standard for the QHES score, a score above 75 is considered high-quality¹⁶. Each of the 33 studies underwent comprehensive paper review by the primary author (PVR), followed by independent review (RAQ) of a random sampling of 20% of these studies to ensure at least 90% inter-rater agreement.

We also recorded the following study characteristics: study design (CEA conducted alongside a randomized-controlled trial or a prospective cohort study, decision tree, Markov cohort, or microsimulation); lowest level of evidence used to derive health state transition probabilities (“level I” for randomized-controlled trials, “level II” for prospective cohort studies, “level III” for retrospective studies, or “level IV” for case series); whether the study reported an ICER; perspective (“health care payer” or “societal,” where societal perspective incorporates unpaid and informal health care costs); time horizon lengths; sensitivity analyses conducted (“deterministic” for parameter variations over a range or “probabilistic” for variations conducted over a distribution); and utility assessment (“direct” if the study elicited health-state preferences using tools such as time trade-off, standard gamble, or visual analog scales, or “indirect” if the study used tools such as EuroQol (EQ-5D)^17,18 or SF-6D^19,20).

Statistical Analyses

QHES scores extracted from each study were averaged across all studies, and by anatomical region and time frame. We used frequencies to describe the distributions of study characteristics, and we calculated averages for study characteristics expressed by continuous factors (e.g. time horizon). We conducted the analysis for the overall sample and stratified by anatomic region, time period, and geographic region to elucidate any trends. We extracted ICERs from studies, when available, and adjusted to 2016 US Dollars using the Consumer Price Index²¹ and 2016 United Kingdom Pounds using Bank of England inflation values²².

Results

Overview

The CEA Registry yielded a total of 134 studies. The MEDLINE query resulted in 61 studies, of which 35 were not captured by the CEA Registry search. Of these 169 studies, 33 studied met our inclusion criteria and were included in this review (Figure 1). Table I provides an overview of the 33 CEAs screened into this review.

Study characteristics

Table 2 provides the general study characteristics for the 33 studies. All anatomic categories have a collection of model-based and randomized controlled trial (RCT)-based study designs. All RCT-based studies have been conducted in the United Kingdom, Australia, or Europe whereas most (95%) decision tree, Markov cohort, and microsimulation designs have been conducted in the United States or Canada.

Table 2.

General study characteristics of N = 33 cost-effectiveness studies of upper extremity orthopaedic surgery. ^*

Study characteristic	n (%)
Study design/model
RCT	9 (27%)
Prospective cohort	5 (15%)
Decision tree	14 (42%)
Markov model, cohort	4 (12%)
Microsimulation	1 (3%)
Level of evidence **
Level I	12 (36%)
Level II	3 (9%)
Level III	4 (12%)
evel IV	14 (42%)
Perspective
Health care payer	10 (30%)
Societal	16 (48%)
Both	2 (6%)
None	5 (15%)
Time Horizon
Lifetime	8 (24%)
20 years	5 (15%)
20 years	20 (61%)
Country of origin
United States	17 (52%)
United Kingdom	9 (27%)
Canada	3 (9%)
Others‡	4 (12%)
Utility Assessment
Direct	11 (33%)
Indirect	22 (67%)
Sensitivity analysis
Deterministic only	17 (52%)
Probabilistic only	1 (3%)
Both	11 (33%)
None	4 (12%)
ICER reported?
Yes	26 (79%)
No	7 (21%)
Cost methodology explained?
Yes	29 (88%)
No	4 (12%)
Statement of funding?
Yes	28 (85%)
No	5 (15%)

^*RCT = cost-effectiveness analysis conducted alongside a randomized controlled trial; Prospective cohort = cost-effectiveness analysis conducted alongside a cohort or series of patients followed prospectively for data on outcomes; ICER = incremental cost-effectiveness ratio.

^**Refers to lowest level of evidence the study used to derive health state transition probabilities

^‡Other countries include Australia, Norway, Netherlands, and Italy.

Studies from the United States predominantly use a societal perspective (70.6%) whereas studies from the United Kingdom predominantly use a health care payer perspective (55.6%). When stratified by time period, there is a trend towards shorter time horizons in recent years; for example, CEAs from 2015-2016 demonstrate a range of only 1 to 2 years. 47% and 100% of the studies in United States and Canada, respectively, use direct utility measurements such as time trade-off, standard gamble, or visual analog scales. However, all studies from the United Kingdom, Australia, and Europe utilize indirect utility measurement tools such as EQ-5D, SF-36, or health utility index (HUI).

77% of studies from the United States use deterministic analyses only, whereas 67% of studies from the United Kingdom utilize both deterministic and probabilistic sensitivity analyses. Of the 28 studies with a statement of funding, 12 are funded by public funds, 5 by private funds, 2 by both, and 9 did not specify. We found studies conducted in the United States use a wide range of WTP thresholds ($4,836 per QALY to $100,000 per QALY) while studies conducted in the United Kingdom tend to use standard WTP thresholds of £20,000-£30,000 per QALY.

Study quality

The average total QHES score for all 33 studies is 82 (high-quality). When stratified according to anatomic group, the QHES scores are: 77 shoulder; 87 arm; 78 elbow; 85 forearm; 86 hand and wrist; and 91 general upper limb. Figure 2 demonstrates an increase in the number of upper-extremity CEAs published since 1997. Over time, a greater proportion of these studies have demonstrated poorer QHES quality (scores <75), with scores as low as 41.

Figure 2.

Bar graph illustrating the number of cost-effectiveness analyses published over time, coupled with the proportion of cost-effectiveness analyses for each time grouping that demonstrated quality QHES scores <75.

Discussion

Since the First Panel Recommendations were published in 1997, the overall quality of cost-effectiveness literature within the field of upper extremity orthopaedic surgery has been high. When stratified by each anatomic region of study, the average QHES score is still >75. Over time, however, there has been a greater proportion of studies demonstrating lower quality scores in recent years. Lower scores <75 tend to demonstrate several deficits including failures in identifying reference case perspectives; incorporating comparators, ICERs, and sensitivity analyses; discounting costs and utilities; and including a statement of funding or support.

Reviews within the subspecialties of sports medicine² and trauma³ have found the quality to be strong, with QHES averages of 81.8 and 79.25, respectively. Brauer and colleagues have noted more orthopaedic studies complying with the recommendations of the First Panel after 1998⁵. Our study, however, seems to indicate a divergence in the quality of the upper extremity orthopaedic cost-effectiveness literature since then. Given this increasing variability, the new recommendations from the Second Panel on Cost-Effectiveness in Health and Medicine published in October 2016 are timely to re-establish standard methodological practices²³.

Our review identified the shoulder and wrist to be the anatomic areas of considerable cost-effectiveness research, with emphasis on rotator cuff tears and carpal tunnel syndrome given their procedural prevalence. Studies in this review have supported single-row, arthroscopic rotator cuff repairs to be more cost-effective alternatives to nonoperative, open, or double-row procedures^24-26, and endoscopic carpal tunnel release to be a more cost-effective option than open release when both techniques are performed in outpatient settings^27,28. There is, however, a paucity of cost-utility analyses of the elbow and of pediatric upper extremity pathologies.

Studies conducted in the United States tend to utilize decision tree or Markov models with societal perspectives. All analyses conducted alongside an RCT came from the United Kingdom, Australia, or Europe; these studies tend to use health care payer perspectives. Studies conducted in the United States were observed to utilize a range of WTP thresholds. These differences reflect the experience of CEAs in each country. Decision-making bodies in Europe, Australia, and Canada have more formally integrated CEAs into coverage decisions, opting to use third-party payer perspectives as opposed to the United States-based First Panel’s recommendation for societal perspectives^23,29. Likewise, health technology assessment bodies in Europe have emphasized funding for RCT-based research to assess clinical evidence alongside cost-effectiveness^30,31. The United States experience has been far more limited, with political resistance to incorporation of CEAs in federal coverage decisions³². The Second Panel’s efforts represent an effort to standardize CEAs across countries for ease of comparability.

This review should be viewed in light of its limitations. Given the stringent inclusion criteria of the CEA Registry, we may have biased our results towards a higher quality score. However, similar reviews have observed equally high qualities when primarily using databases such as MEDLINE^2,3. The QHES tool values the design and reporting of cost-effectiveness studies; therefore, it is possible that a study lost points for not being explicit in its reporting of study design and results. The quality scores are certainly susceptible to the biases of the individual grader; however, we ensured at least a 90% inter-rater agreement across two reviewers.

Cost-effectiveness is an area of increasing interest in upper extremity orthopaedic surgery, especially in the shoulder and the wrist. We observe a high quality of CEA studies in upper extremity surgery since 1997, but a growing proportion of lower quality studies in recent years. It will be important to monitor the ongoing quality of CEA studies in orthopaedics and ensure proper peer-review of future studies based on the Second Panel recommendations to ensure standards of reporting and comparability.

Table I.

Main findings and study designs of cost-effectiveness studies included in this review, organized by anatomic region.^*

ANATOMY	AUTHOR	YEAR	COUNTRY	STUDY DESIGN	ICER (2016 values)	WTP	MAIN FINDINGS
Shoulder	Butt33	2015	UK	Prospective cohort	NA	NA	Arthroscopic decompression for subacromial impingement provides 0.23 QALYs gained for £5,683
	Jowett34	2013	UK	RCT	Dominant	£20,000/QALY	Corticosteroid injection and exercise therapy may be cheaper and more effective (i.e. dominant) to exercise-alone in moderate to severe subacromial impingement syndrome
	Carr24	2015	UK	RCT	£32,510/QALY	£20,000/QALY	Open rotator cuff repair was not a cost-effective alternative to arthroscopic rotator cuff repair in the base case, intention-to-treat analysis
	Crall25	2012	USA	Microsimulation	Multiple; all <WTP or dominant	$25,000/QALY	Primary arthroscopic stabilization was a cost-effective alternative to nonoperative treatment for first-time anterior glenohumeral dislocation with ICERs <$25,000/QALY across majority of age groups
	Coe35	2012	USA	Markov cohort	$103,668/QALY**	$100,000/QALY	Reverse shoulder arthroplasty was a cost-effective alternative to humeral head replacement for rotator cuff tear arthropathy
	Genuario26	2012	USA	Decision tree	Tears <3 cm: $638,601/QALY; Tears ≥3 cm: $514,233/QALY	$100,000/QALY	Double-row rotator cuff repair is not a cost-effective alternative for any size rotator cuff tears
	Mather36	2013	USA	Markov cohort	Dominant	$50,000/QALY	Surgical rotator cuff repair is a cost-effective alternative for the U.S. populations compared to nonoperative treatment with lifetime savings of $13,771 and 0.62 QALY improvement
	Renfree37	2013	USA	Prospective cohort	NA	NA	Reverse shoulder arthroplasty for rotator cuff arthropathy provides 2-year gain of 1.02 (SF-36) and 1.97 (EQ-5D) QALYs for a cost of $21,536
	Vavken38	2015	USA	Decision tree	$132,009/QALY	$100,000/QALY	Platelet-rich plasma after arthroscopic rotator cuff repair is not a cost-effective alternative to repair without platelet-rich plasma, assuming a 5% revision rate
	Vitale39	2007	USA	Prospective cohort	NA	NA	Surgical rotator cuff repair provides a mean lifetime gain of 0.81 (HUI) and 3.43 (EQ-5D) QALYs for total cost of $10,605
	Dattani40	2013	UK	Prospective cohort	NA	NA	Arthroscopic capsular release for contracture of the shoulder provides 0.50 QALYs gained for £2204
	Mather41	2010	USA	Markov cohort	Dominant	$50,000/QALY	Total shoulder arthroplasty is a cost-effective alternative to hemiarthroplasty for glenohumeral osteoarthritis with $1,970 less costs and 0.77 more QALYs
	Pearson42	2010	USA	Decision tree	Base case: $80,546/QALY; >9 years: $34,883/QALY	$50,000/QALY	ORIF of displaced, midshaft clavicular fractures can be a cost-effective alternative to nonoperative treatment if the incremental QALYs gained (0.014) persists beyond 9 years. At the base case, ORIF is not a cost-effective alternative
Arm	Corbacho43	2016	UK	RCT	Dominated	£20,000/QALY	Surgical treatment for displaced, proximal humerus fractures in adults is not a cost-effective alternative to non-operative treatment with greater total costs and lower total QALYs (i.e. dominated)
	Fjalestad44	2010	Norway	RCT	£315,922/QALY	NA	Surgical treatment for displaced, proximal humerus fractures in adults did not produce statistically significant different costs or QALYs from nonoperative treatment.
Elbow	Coombes45	2016	Australia	RCT	$30,287/QALY	$50,000/QALY	Physiotherapy-alone was a cost-effective alternative to corticosteroid injection with or without physiotherapy for chronic lateral epicondylalgia
	Giannicola46	2013	Italy	Prospective cohort	NA	NA	Open surgical treatment of elbow stiffness produces 0.1539 annual increases in QALYs with an average cost of £3565
	Song47	2012	USA	Decision tree	$2,265/QALY	$100,000/QALY	Simple decompression was the most cost-effective initial procedure for ulnar neuropathy of the elbow when compared to anterior subcutaneous and submuscular transpositions and medial epicondylectomy
Forearm	Karantana48	2015	UK	RCT	£44,814/QALY	£10,000-£50,000/QALY	Volar locking plating was not a cost-effective alternative to percutaneous wire fixation for distal radius fractures
	Rockwell49	2004	Canada	Markov cohort	Dominated	NA	Prophylactic plating of the donor radius after harvest of radial osteocutaneous flap is not a cost-effective alternative to treatment of fractures when they occur, producing higher cost ($2071 vs. $140) with lower QALYS (8.55 vs. 9.92)
	Shauver50	2011	USA	Decision tree	$17,130/QALY	$50,000/QALY	ORIF dominated wire fixation and external fixation and was a cost-effective alternative to casting for distal radius fractures in the elderly
	Tubeuf51	2015	UK	RCT	£96,793/QALY	£30,000/QALY	Volar locking plating was not a cost-effective alternative to percutaneous Kirschner wire fixation for dorsally displaced distal radius fractures
Hand & Wrist	Baltzer52	2013	Canada	Decision tree	Collagenase: $303,654/QALY; Fasciectomy: Dominated	$50,000-$100,000/QALY	Collagenase was not a cost-effective alternative to percutaneous needle aponeurotomy for Dupuytren’s contracture; partial fasciectomy was dominated by aponeurotomy
	Chen53	2011	USA	Decision tree	Fasciectomy: $916,405/QALY; Collagenase: $55,865/QALY**; Aponeurotomy: $55,458/QALY**	$50,000/QALY	All compared to no treatment for Dupuytren’s contracture, open partial fasciectomy is not a cost-effective alternative; collagenase can be a cost-effective alternative if priced <$945; aponeurotomy can be cost-effective if success rate is 100%
	Chung27	1998	USA	Decision tree	25-year-old: $293/QALY; 65-year-old: $1,042/QALY	$4836/QALY to $13,508/QALY	Endoscopic carpal tunnel release is a cost-effective alternative to open release for carpal tunnel syndrome
	Korthals-de Bos54	2006	Netherlands	RCT	£469/QALY	£2,500/QALY	Surgery is a cost-effective alternative to splinting for carpal tunnel syndrome
	Thoma28	2006	Canada	Decision tree	Main OR: $147,665/QALY; Day unit: dominant	$100,000/QALY	Endoscopic carpal tunnel release is not a cost-effective alternative to open release for carpal tunnel syndrome when endoscopic release is performed in the main operating room and when open release is performed in the day surgery unit. When both are performed in the day surgery unit, endoscopic release dominates open release.
	Cavaliere55	2010	USA	Decision tree	TWA vs. nonsurgical: $2,512/QALY; TWA vs. arthrodesis: $2,564/QALY	$50,000/QALY	Total wrist arthroplasty (TWA) was a cost-effective alternative to both nonsurgical management and to total wrist arthrodesis for the rheumatoid wrist
	Chung56	2010	USA	Decision tree	Prosthesis vs. transplant: dominant; Double-hand transplant vs. prosthesis: $426,808/QALY	$50,000-$100,000/QALY	Prosthetic use dominated hand transplantation for unilateral hand amputation. Double hand transplantation was preferred over prostheses (26.73 vs. 25.20 QALYs) for double hand amputation, but was not cost-effective.
	Davis57	2006	USA	Decision tree	$7,135/QALY	$20,000/QALY	ORIF is a cost-effective treatment alternative to cast immobilization for acute nondisplaced mid-waist scaphoid fractures
	Sears58	2014	USA	Decision tree	Single-digit: $145,643/QALY; 3-digit: $28,963/QALY; 4-digit: $25,413/QALY	$100,000/QALY	Replantation had greater costs and QALYs compared with revision amputation in all injury scenarios. Replantation of single-digit injuries was not a cost-effective alternative to revision amputation, whereas replantation of 3- or 4-digit amputations were cost-effective alternatives
General	Doan59	2013	UK	Markov cohort	£11,130/QALY	£30,000/QALY	OnabotulinumtoxinA along with usual care was a cost-effective alternative to usual care for upper-limb post-stroke spasticity
	Shaw60	2010	UK	RCT	£119,049/QALY	£20,000/QALY	Addition of botulinum toxin A to a therapy program for upper-limb post-stroke spasticity was not cost-effective

^*RCT = cost-effectiveness analysis conducted alongside a randomized controlled trial; Prospective cohort = cost-effectiveness analysis conducted alongside a cohort or series of patients followed prospectively for data on outcomes; ICER = incremental cost-effectiveness ratio; QALY = quality-adjusted life-year; WTP = willingness-to-pay threshold; HUI = health utility index; ORIF = open reduction internal fixation.

^**Note: adjusting ICER to 2016 values raised the ratio above the corresponding willingness-to-pay threshold