Policy-Relevant Research: When Does It Matter?

Abstract

Summary: Evidence-based medicine is most meaningful to policy makers when research questions are clearly informed by strategic health policy questions. In Washington State workers’ compensation, key structural characteristics allow for the conduct of effective policy-relevant research. These include clear authority and a stable funding stream, a formal relationship between a policy agency and a University, development of appropriate research capacity, development of research questions related to strategic goals, and a robust data source. The research conducted relies on computerized medical bills and work disability records, medical records, structured telephone surveys to collect data on pain, functional status, quality of life, and computerized data on employment status. The types of policy-relevant research include identification of factors leading to preventable disability, outcomes research of specific procedures, technology assessment, and “real-time” research that addresses rapidly emerging questions. Health policy changes implemented from research have been substantial in Washington State workers’ compensation, including: 1) noncoverage or partial coverage decisions for emerging technologies not proven to be of value to injured workers, 2) formal treatment guidelines and utilization review criteria for invasive, expensive, or marginally effective procedures, 3) disability prevention efforts, and 4) relatively rapid changes in policy as emerging patterns suggest harmful outcomes from existing treatments (e.g., schedule II opioids). Key structural characteristics must be in place to conduct policy-relevant research effectively. The workers’ compensation system in Washington State is a single-payer system with other unique properties that have allowed the emergence of these structural characteristics and the conduct of research linked to the strategic goals of policy makers.

INTRODUCTION

Over the past 15 years, we have conducted policy-relevant research in the Washington State workers’ compensation system. The successful development of this program has been dependent on the convergence of specific structural characteristics that, while uncommon, could be duplicated in other state or national health policy venues. The first part of this paper will discuss the minimum characteristics, without which a substantial policy-relevant research effort would not likely be successful. The second part of this paper will discuss specific types of research and policy questions, and their relationship to each other. In addition, the influence of political and financial factors on these relationships will be discussed.

STRUCTURAL CHARACTERISTICS CRITICAL TO A SUCCESSFUL POLICY-RELEVANT RESEARCH PROGRAM

Executive sponsorship and dedicated funding stream

The policy makers who wish to use research to assist in policy development must establish, in statute or regulation, the authority and stable means to conduct such research. For our health care research program, the authority is in State Statute (RCW 28B.20.450), and the funding must be approved biannually. The level of funding is not large (approximately $1,000,000/biennium) compared to the annual workers’ compensation bill (about $1.5 billion annually). In other words, the investment is tiny for a potentially large policy payback.

Policy makers who establish such authority and dedicated funding streams may waver in their support if research findings result in adverse political fallout. The Agency for Health Care Policy and Research (AHCPR) created a stir with a low back pain treatment guideline that was not favorable to spine surgery. The subsequent lobbying effort by a few influential spine surgeons nearly spelled the demise of the agency.¹ While the agency’s funding has remained stable, its emphasis has shifted from policy (AHCPR) to quality, as reflected by the decision to rename the agency the Agency for Healthcare Research and Quality (AHRQ).

A formal university/government link

The statutory authority and funding stream helped establish a formal relationship between the University of Washington (UW) and the Washington State Department of Labor and Industries (DLI) (FIG. 1). The relationship operates under a formal Memorandum of Understanding (MOU) reviewed every 3–4 years. A key component of the MOU is the authority to transfer data from the state agency to the UW, a critically important step in obtaining Institutional Review Board approval for workers’ compensation research projects.

FIG. 1.

Policy-relevant research: unique alignment of resources and authority.

Research capacity

One clear benefit of a formal relationship between an executive agency and a university is access to a higher level of research capacity. This is a necessary but not sufficient characteristic. G.M. Franklin holds a policy position at DLI (Medical Director) and a research position at UW (Research Professor). This dual relationship assures that the research conducted is informed by policy questions, and that the findings of the research are adequately translated and implemented in policy (FIG. 1).

The types of research capacity required to best inform health care delivery policy would include an understanding of the best use of population-based data (epidemiology), an ability to determine patterns of health care use (health services research), a capacity to assess outcomes of delivered health services (outcomes research), and the impact on costs to the system (health economics). Our ability to collaborate with senior faculty in these areas has allowed the most efficient use of statutory funds, and more leverage in obtaining additional federal and private foundation funding.

Alignment of research with policy makers’ strategic goals

The key policy issue in workers’ compensation is the tremendous human and financial cost of work-related disability. Five to 10 percent of workers’ compensation cases account for 80–85% of the disability and costs to the system.² Most of this productive life lost³ is likely preventable, but empirically based knowledge concerning identification of individuals at a high risk for chronic disability and effective early intervention strategies is lacking.⁴ Health care research aimed at improving disability prevention needs to focus on the following questions: What are the risk factors for disability (these may be worker, health services, administrative, and job-related factors) and what early interventions are effective in preventing disability? Given its central importance, therefore, our research emphasizes disability status as a primary outcome variable. Policy makers and key stakeholders understand the importance of this outcome; the approach is completely consistent with their strategic goals.

In addition to alignment of strategic goals, demonstrating the value of the research to DLI operations is also crucial. An immediately translatable type of research related to operations is program evaluation research.⁵ While this type of research may often be left up to internal program analysts, the complex methods required to adequately assess program outcomes are often lacking.

Stakeholder advice and consent

A crucial structural characteristic relates to the method for obtaining stakeholder advice and consent regarding the relevance of research to policy. Both the workers’ compensation system (DLI) and the research enterprise (UW) have formal business-labor advisory committees. Because employers pay the majority of costs of the workers’ compensation system, and the employees are the beneficiaries of the system, their advice and consent on the policy relevance of research is critical (FIG. 1). These “owners” of the system have the greatest stake in policy changes that may be engendered by research. Important secondary stakeholders affected by health care research include physicians and other providers caring for injured workers. Policy regarding health care coverage informed by research has greater traction in the medical community when their professional organizations participate in the policy development. In Washington State, committees of both the state medical and chiropractic associations are established, by regulation, as advisory to DLI health care policy functions. No important health care policy decision is made without such advice and, in most cases, consent.

Population-based data sources

The following are rich, population-based data sources available for research in the Washington State workers’ compensation system: 1) computerized claims and medical bill payment data, 2) medical records, and 3) structured telephone surveys (computer-assisted telephone interviews). These rich data sources are available because Washington has a “two-way” workers’ compensation system. There is no private workers’ compensation insurance. As such, detailed claims information, including medical and administrative data, are available for two-thirds of the nonfederal employees in the state. Encounter level data on all claims, including detailed pharmacy data, is captured by provider, including the amounts paid. In addition, all claim-related information, including detailed lost time (disability payment) data, is also captured and can be readily linked to the medical data. Detailed medical records and employment data from a sister state agency are readily available, and it is also possible to conduct structured telephone surveys of injured workers (e.g., to capture pain, function, and quality-of-life information).

POLICY-RELEVANT RESEARCH IN WASHINGTON STATE WORKERS’ COMPENSATION

Disability prevention research

Figure 2 (adapted from Cheadle et al.⁶) is a survival curve of 28,000 claims with work disability using cumulative time loss days as the outcome variable. The disability experience of workers with carpal tunnel syndrome (CTS) is substantially worse than that of workers with back/neck sprains, fractures, or all other work-related injuries as a group.

FIG. 2.

Disability prevention is the key health policy issue. Adapted from Cheadle et al.⁶ Copyright © 1994, APHA. All rights reserved.

We have begun to identify predictors of disability. Table 1 summarizes the proportions of workers by gender and age groups with work-related disability who had at least 6 months of work disability. Older male workers with CTS (≥45 years of age) had a 33% chance of incurring long-term disability, whereas older female workers had a 40% chance. This type of information has been used internally to target more resources to workers most likely to have chronic disability, because there is little other information that would provide a more comprehensive screening tool⁴ for practitioners. We are prospectively following 3000 workers with low back and CTS claims to develop more comprehensive models of disability prediction. Identification of modifiable risk factors may lead to early interventions to prevent disability. The potential dividends in terms of improved quality of life and decreased costs from early interventions, if effective, would be great.

TABLE 1.

Chronic Disability by Age, Gender, and Injury Type among Workers with Work Disability, 1987–1989

Sex	Age Category	Fracture	Sprain: Back	Sprain: Other	Carpal Tunnel Syndrome	All Other	Total
Male	<30 years	7.8	17.2	9.9	23.3	6.9	10.9
	30–44 years	10.1	27.4	17.2	21.3	11.9	18.6
	>45 years	12.6	29.0	22.2	33.3	16.4	21.4
Female	<30 years	2.1	18.0	14.9	31.8	12.4	15.2
	30–44 years	13.2	33.2	28.3	28.6	22.3	28.6
	>45 years	15.0	24.4	38.8	40.0	24.4	27.3
Total		18.1	24.8	18.1	26.8	13.6	19.0

Percentage of workers with at least 180 days of time loss.

Outcomes research, treatment guidelines, and utilization review

We have conducted a series of outcomes studies related to medical treatments that are common (CTS surgery),⁷ invasive, and of unclear efficacy (lumbar fusion,⁸ thoracic outlet syndrome surgery⁹), or costly (multidisciplinary pain treatment programs¹⁰). Although these studies are observational and retrospective, they have several strengths: they are population-based, they have long follow-up, and the outcome (time loss status) is objective and available for 100% of cases (Table 2). Thus, we believe that the outcomes found in these studies represent the outcomes one might expect from these treatments in injured workers on a community basis.

TABLE 2.

Outcome Study Methods

• Population-based: identify all cases in study period

• Retrospective cohort of incident cases

• 3- to 5-year mean follow-up

• Time loss outcome, e.g., on/off time loss at 1 or 2 years after procedure or intervention

• Multivariate analysis for baseline predictors of principal outcome

• Secondary outcomes (reoperation, QOL)

QOL = quality of life.

The outcome of CTS surgery, although somewhat worse than in non-workers’ compensation patients, was quite good; 83% of workers who received surgery returned to work within 3 years.⁷ This is consistent with a recent randomized trial that found that 80% of those who received CTS surgery were completely recovered or much improved.¹¹ Based on these outcome studies, the DLI developed a policy that 1) encourages timely, early diagnosis including electrodiagnostic testing to corroborate presence of CTS, and 2) approves timely CTS surgery if conservative treatment measures fail to allow return to work. We believe that the CTS disability problem discussed above may, in part, be a result of delayed diagnosis of CTS and failure to receive surgery in a timely manner.

Lumbar fusion (arthrodesis) is an extraordinarily invasive and costly procedure that is applied to treat chronic low back pain, despite the lack of scientifically proven efficacy. Our data (Table 3) revealed that older age, longer time since injury, time on disability before fusion, number of previous back surgeries and number of levels fused predicted worse work disability 2 years after fusion.⁸ None of the stated indications for fusion (presence of instability, stenosis, radiculopathy, spondylolisthesis, or pseudoarthroses), nor implantation of instrumentation were associated with disability outcomes. Even surgical volume of the surgeon (<5 vs ≥5 fusions/year) did not predict outcome.

TABLE 3.

Lumbar Fusion Outcome

Variable	RR	95% CI	p Value
Baseline markers of severity
Age at injury (10-year increase)	1.37	1.10–1.71	0.006
Time from injury to index lumbar fusion (years)	1.11	1.04–1.20	0.003
Time on work disability during 6 months before index lumbar fusion (20% increase)	1.48	1.30–1.67	<0.001
Number of previous low back surgeries (0, 1, 2, 3+)	1.37	1.12–1.19	0.003

Prediction of work disability 2 years after index lumbar fusion, August 1, 1986 to July 31, 1987.

N = 338 cases.

RR = risk ratio; CI = confidence interval.

Overall, the results of the lumbar fusion study were not consistent with the generally positive outcomes reported from case series. We found that two-thirds of workers were still totally disabled 2 years after fusion.⁸ This adopted policy was associated with a significant decline in the rate of lumbar fusions performed in Washington State (FIG. 3). ¹³

FIG. 3.

Lumbar fusion policy translation. Proportion of low back operations paid by DLI involving fusions. Reproduced with permission from Elam et al. Impact of a worker’s compensation practice guideline on lumbar spine fusion in Washington State. Med Care 35:417–424. Copyright © 1997, Lippincott Williams and Wilkins. All rights reserved.

Rapid diffusion of new technology, e.g., introduction of interbody fusion cages, has been associated with a doubling of lumbar fusion requests between 1995 and 2002.¹⁴ This is an example of new technology driving an expensive and invasive procedure of unproven efficacy. Recent high-quality studies have demonstrated no substantial benefit of lumbar fusion using somewhat older fixation devices.¹⁵ In addition, the diagnostic test used to select patients for fusion (discography) is of questionable validity.¹⁶ We are repeating the lumbar fusion outcomes study, examining outcomes of patients who received recently developed instrumentation devices, and will continue the research/policy effort in this area.

Thoracic outlet syndrome (TOS) has long been a disputed issue in the neurologic community.¹⁷ We were able to compare disability outcomes and costs in workers with a diagnosis of TOS who did versus workers who did not receive TOS surgery (Table 4). ⁹ Even after adjusting for important covariates, workers who received TOS surgery were 4–5 times more likely to be disabled 2 years after the procedure. A new guideline, crafted with the Washington State Medical Association after presentation of these results, included a specific requirement for abnormal electrodiagnostic findings demonstrating definite pathology in the brachial plexus. Since most requests for TOS surgery include only nonspecific provocative or Doppler test results, TOS surgery approvals are now much less frequent in Washington State workers’ compensation.

TABLE 4.

Time Loss and Medical Costs for Cases with and without TOS Surgery Diagnosed 1987–1989

Time Loss, Costs	Unadjusted		Adjusted*
	TOS Surgery	No Surgery	TOS Surgery	No Surgery
Mean medical costs, $	25,614	14,063	22,576	15,652
Percent on time loss at 1 year	56.8	21.1	52.7	15.8
Percent on time loss at 2 year	40.5	12.6	37.2	7.8

^*Adjusted for age, cervical spine diagnosis, number of years from injury to diagnosis, sex, time loss in 6 months before diagnosis, previous injury, and previous surgery. All p values were <0.01.

Adapted from Franklin et al. Outcome of surgery for thoracic syndrome in Washington state workers’ compensation. Neurology 54:1252–1257, 2000. Copyright © 2000, Lippincott Williams and Wilkins. All rights reserved.

Guidelines development with the Washington State Medical Association is an ongoing effort. Table 5 lists some of the guidelines that have been developed.¹² For the most part (an exception is opioid pain medications), these guidelines are not placed in regulation; we wish to remain flexible as new information emerges.

TABLE 5.

Clinical Practice Guidelines and Health Technology Assessments Conducted by the Washington State Department of Labor and Industries

Clinical practice guidelines*

Lumbar spine MRI (1994)

Thoracic Outlet Syndrome (1995)

Porphyria (1995)

Occupational Carpal Tunnel Syndrome (1995)

Psychiatric/psychological evaluation (1995)

Chronic Regional Pain Syndrome (1997)

Hospitalization for low back pain (1998)

Fibromyalgia (1998)

Opioids (2000)

Lumbar fusion (2001)

Shoulder surgery (2002)

Off-label Neurontin use (2003)

Health technology assessment21

Acupuncture (1998)

Aquamed (1998)

Hyaluronic acid (1998)

Vertebral axial decompression (1998)

Intradiscal heating (2000)

Autologous chondrocyte implantation (2002)

Cervical traction devices (2002)

Dynatron STS device (2002)

Meniscal allograft transplantation (2002)

Microprocessor controlled prosthetic knee (2002)

Extracorporeal shockwave therapy (2003)

^*Developed with the Washington State Medical Association.¹² Year of publication is indicated in parentheses.

Health technology assessment

Table 6 provides information concerning the regulation of drugs, devices, and procedures. Although the Food and Drug Administration (FDA)-approved drugs require at least one well designed, randomized controlled trial (RCT) demonstrating efficacy, devices are most frequently approved based only on equivalence to a device that existed before 1976.¹⁸ Even the most rigorously assessed medical devices are usually approved based on trials less rigorous than those required for drug approval; clinical effectiveness is not usually well addressed.

TABLE 6.

Regulation of Drugs, Devices, and Procedures

Drugs
On-label	FDA approved based on at least one RCT
Off-label	Frequently based on marketing; often limited RCT information
Devices	FDA regulates, but approval not usually based on clinical effectiveness or RCT
Procedures	Not regulated at all; limited RCT data available

RCT = randomized controlled trial.

We also conduct formal technology assessments of rapidly emerging devices that have not been proven effective in RCTs (Table 5; list of technology assessments also available at http://www.lni.wa.gov/ClaimsInsurance/Providers/TreatmentGuidelines/TechAssess/default.asp). Frequently, until such scientific data are available, we make a noncoverage or partial coverage policy decision on such rapidly emerging technologies.

Policy decisions on technologies that already have been diffused are more difficult. Our experience with spinal cord stimulation (SCS) is an important example. SCS was approved for marketing in the mid-1970s, but the first RCT for its use (in reflex sympathetic dystrophy) was only published in 2000.¹⁹ The technology is expensive, costing $50,000–80,000 over a 5-year period. The one RCT that assessed pain and functional outcomes found a modest effect on pain and no clear effect on function at 6 months in a group of patients with reflex sympathetic dystrophy.¹⁹ DLI, early in the 1990s, thus concluded that SCS, without evidence that this technology could improve function, should not be covered. A recent systematic review of the literature drew similar scientific conclusions, and documented substantial adverse event rates.²⁰ Lobbying by a leading manufacturer of SCS equipment has led to recently increased legislative pressure to fund SCS for workers’ compensation claimants, and we are now revisiting the noncoverage policy with the Medical Association.

A key research and policy question on effectiveness of devices and other treatments relates to the relationship between pain and function. Most workers who become disabled do so because of chronic pain. Theoretically, substantial improvement in pain ought to be associated with clear improvement in function, and an increased probability of returning to productive work. We are currently investigating, in a prospective study, how much improvement in pain is required, in general, to obtain important improvement in function.

Real-time outcomes: the case of opioid medications and chronic noncancer pain

Sometimes one has to make a policy decision before the final answer is known, based upon scientific evidence. In the late 1990s, most states changed their regulations regarding opiate use for pain, based on weak scientific data. Whereas opiate use for chronic noncancer pain generally was previously prohibited, the new regulations removed most barriers to use. Since 1998 (Table 7), we have seen a dramatic shift toward schedule II opioids and the average daily dose (morphine equivalents) of schedule II opiates paid for by DLI has increased by more than 50% (from ∼80 mg/day to >120 mg/day). In addition, we have seen a disturbing pattern of increased mortality related to accidental overdose of prescription opioids for chronic pain. This observation is consistent with findings of a Drug Enforcement Agency (DEA) medical examiner study of oxycodone-related deaths.²¹

TABLE 7.

Opiates for Chronic, Noncancer Pain

• Dramatic shift in public policy 1998, e.g., based on methodologically weak studies, advocacy groups, and drug lobby

• 4-year follow-up: dramatic shift toward long-acting, schedule II opiates (e.g., oxycodone)

• 50% increase in average daily (morphine equivalent) doses (80–120 mg morphine equivalents/day)

• 20–30 deaths in 3 years, some related to accidental overdose

• Department of Justice/Drug Enforcement Agency Medical Examiner report of 469 deaths likely attributable to oxycodone21

A recent review of opioid therapy for chronic pain²² concluded “… very large doses of opioids are prescribed for patients with chronic pain that is not associated with terminal disease, often in the absence of any real improvement in the patient’s pain or level of functioning. Whereas it was previously thought that unlimited dose escalation was at least safe, evidence now suggests that prolonged, high-dose opioid therapy may be neither safe nor effective.” We will send a warning letter to all providers shortly, and will soon revisit with the Washington State Medical Association the overall policy regarding coverage of opiates for chronic, noncancer pain.

CONCLUSIONS

Policy-relevant research in the Washington State workers’ compensation system has allowed that system to move to the forefront of state-level policy research sponsored by a public payer of health care.²³ We have described some of the policy studies conducted to date on the health care delivered through the system. Other important studies involving evaluation of a major managed care demonstration initiative,^24–27 outcome assessment of a multi-year drug-free workplace initiative,²⁸ evaluation of an ongoing quality improvement initiative,^24,29 and patient satisfaction surveys^30,31 have been conducted and reported elsewhere. Most importantly, we believe that the structural capacity built here has had a long-term beneficial effect on both quality of care and cost. The Institute of Medicine has called for a national effort in using evidence-based decision making as our health care system evolves.³² Based partly on the type of work presented here, the Washington State legislature (ESHB 1299) has recently mandated that all of the health care purchasing agencies in Washington move toward formal (evidence-based) assessment of the health services that are purchased.