Skip to main content
The Iowa Orthopaedic Journal logoLink to The Iowa Orthopaedic Journal
. 2021;41(1):11–12.

Resist the Rise of Irrelevant Outcome Measures in Clinical Research

Benjamin J Miller 1,✉
PMCID: PMC8259178  PMID: 34552397

Length of stay is not an outcome. Consistently, I return to this thought while writing, reviewing, and reading manuscripts, revealing a trend that has become clear: the orthopaedic community is not talking about pathologic basis of disease anymore. The manuscripts that draw our attention today discuss issues of efficiency, cost, pain management, discharge location, and patient experience. It is a remarkable change from a generation ago, where reports were consistently centered on describing diagnostic techniques, new methods of treatment, and objective measures of outcome directly related to an underlying disease process.

The optimistic explanation, which is also logical and generally accurate, is that many of the most important historical problems in orthopaedics have been adequately addressed. Joint arthroplasty for osteoarthritis is the most accessible example, where indoctrinated procedures and reliable implants now provide improved quality-of-life and decades of durable outcome for hundreds of thousands of patients. There simply is not a need for novel reports describing the histologic appearance, radiographic characteristics, physical exam findings, or natural history of osteoarthritis. The paucity of residual big and unanswered questions leaves a vacuum being filled by motivated investigators with goals to contribute positively to our shared knowledge.

The most familiar historical outcomes are tangible and anchored – death, infection, revision. These are end results with purpose and power, but their prior relevance has dissipated as essentially all modern treatments are satisfactorily safe and effective viewed by traditional metrics. In short, proving that a treatment extends life, prevents infection, or lasts for an extended period of time is no longer adequate.

The essential challenge of modern clinical research is to define and report optimal outcome measures. This is an issue of critical importance and should motivate investigators to consider and advocate for measures most relevant to patient care, or risk having outcomes assigned by non-clinical entities and not accurately reflective of appropriate treatment goals. The manner by which to complete this task is not easy or obvious, but needs to be motivated by the intent to record measures that quantify the success of an intervention in terms of function and quality-of-life, and presented in a way that is meaningful to the patients receiving treatment. Many of these types of assessments, in particular PROs (patient-reported outcomes), have become part of the healthcare lexicon to the extent that any reasonable investigation of clinical outcome will include some measure of direct patient response. This should be applauded, and these measures will become increasingly important to improve the quality of care, engage patients in medical research, and modify our healthcare system. However, vigilance is required to avoid having outcome measures of importance be supplanted by the less relevant data points of patient satisfaction, cost, and length of stay.

Patient satisfaction has unquestionably arrived and is not going anywhere. The various surveys and comparisons are ubiquitous to practitioners and administrators. While seemingly noble in intent, there is much to criticize. Queries intending to assess compassion, empathy, respect, communication, patient understanding, and involvement in the decision-making process are important reflections on the virtues of healthcare providers and can be meaningful in the correct context. However, there are substantial limitations in how these surveys should be used, and attempts to judge the effectiveness of an individual or healthcare system to eradicate disease by reporting “patient satisfaction” should provoke skepticism. Any facility-related issue (e.g. parking, food service, aesthetics) cannot be convincingly argued to correlate with the most important issues in medical care, such as accurate diagnoses, appropriate treatments, minimal complications, and optimal function. There also is a problematic perverse incentive to focus more intently on the perception of the patient’s experience rather than the medical goals of treatment. For example, if a patient is referred for consideration of a surgical procedure, insofar as their expectation is that they will receive it, and the consulting provider does not deem that intervention indicated, the patient may leave the visit dissatisfied and complete a survey as such. Outwardly the appearance insinuates a negligent or incompetent provider, and theoretically manifests as an unwelcome suggestion that patient desires should be held in higher regard than medical appropriateness.

Cost of care is of critical importance in the modern US healthcare system, and should not be ignored. Healthcare is a limited resource, and the current level of spending, along with the consistent trends of increasing financial liability, are unsustainable and worrisome. The principal problem with exploring cost in clinical research is that there is no agreement on what it is, how to measure it, or where to find it. The first consideration is simply one of agreement in definitions. “Cost” should be understood to indicate the amount of currency required to provide a service or intervention; this is rarely available. As a surrogate, many investigations report “charges” (the bill an institution sends a patient or payer) or “reimbursement” (what is actually paid by insurance and patients). Neither charges nor reimbursement represents the true cost of care and are therefore of limited utility. The second issue is that neither of the two parties involved in medical decision-making (patient and provider) has a clear idea of the true or relative cost of an intervention. It’s quite embarrassing to me that I would have no idea how to respond if a patient asked me how much a procedure would cost – I could easily be off by a factor of 10. Almost as concerning, there is no clear repository in which to find this information. This leaves a gap impeding any use of cost consciousness in medical decisions. Patients are not able to make financial-based decisions as there is not a reliable mechanism to compare hospitals or interventions in any meaningful way. This problem has to be sorted out by increasing transparency in pricing, making the information easily accessible, and creating measures to determine the true cost of medical care, not charges or reimbursement.

Finally, length of stay is often reported and implied to be representative of treatment outcomes. There are arguments to be made for length of stay as a surrogate measure to represent superiority in hospital efficiency, such as preoperative discharge planning, perioperative pain control, and appropriate setting of expectations in elective procedures. Less time in the hospital could lead to fewer hospital-related complications and reason supports that most patients would prefer to spend as little time in the hospital as possible. But we must resist the inertia pushing length of stay to become synonymous with high quality care. In its essence it has nothing to do with the important aspects of medical treatment or recovery. There is no connection to accurate diagnosis, intelligent surgical planning, masterful execution of treatment, or effective rehabilitation protocols. At best, this is a general assessment of hospital discharge capability. At worst, it is a false idol that will incentivize surgeons and institutions to work toward a meaningless objective that has almost nothing to do with restoring health or alleviating suffering.

The priorities of clinical research are continuing to evolve and it is best not to resist these changes. Questions of quality and efficiency are important in modern healthcare, and for most practitioners and patients making an accurate diagnosis and effectively delivering interventions are more important than knowledge of the fundamental pathologic basis of disease. Clinicians and researchers must recognize that current and future outcomes will be more nuanced than in the past. If we do not continually advocate for outcome measures that truly represent success after medical treatment, with a focus on function and quality-of-life, we risk having less important measures mandated for use for reasons of administrative utility at the expense of clinical relevance.


Articles from The Iowa Orthopaedic Journal are provided here courtesy of The University of Iowa

RESOURCES