Patient-reported outcome measures following revision knee replacement: a review of PROM instrument utilisation and measurement properties using the COSMIN checklist

Abstract

Objectives

To identify: (1) patient-reported outcome measures (PROMs) used to evaluate symptoms, health status or quality of life following discretionary revision (or re-revision) knee joint replacement, and (2) validated joint-specific PROMs, their measurement properties and quality of evidence.

Design

(1) Scoping review; (2) systematic review following the COnsensus-based Standards for selection of health status Measurement INstruments (COSMIN) checklist.

Data sources

MEDLINE, Embase, AMED and PsycINFO were searched from inception to 1 July 2020 using the Oxford PROM filter unlimited by publication date or language.

Eligibility criteria for selecting studies

Studies reporting on the development, validation or outcome of a joint-specific PROM for revision knee joint replacement were included.

Results

51 studies reported PROM outcomes using eight joint-specific PROMs. 27 out of 51 studies (52.9%) were published within the last 5 years. PROM development was rated ‘inadequate’ for each of the eight PROMs studied. Validation studies were available for only three joint-specific PROMs: Knee Injury and Osteoarthritis Outcome Score (KOOS), Lower Extremity Activity Scale (LEAS) and Western Ontario and McMaster Universities Arthritis Index (WOMAC). 25 out of 27 (92.6%) measurement properties were rated insufficient, indeterminate or not assessed. The quality of supporting evidence was mostly low or very low. Each of the validated PROMs was rated ‘B’ (potential for recommendation but require further evaluation).

Conclusion

Joint-specific PROMs are increasingly used to report outcomes following revision knee joint replacement, but these instruments have insufficient evidence for their validity. Future research should be directed toward understanding the measurement properties of these instruments in order to inform clinical trials and observational studies evaluating the outcomes from joint-specific PROMs.

Strengths and limitations of this study.

• This is the first study to apply the Consensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist to report the quality of patient-reported outcome measure (PROM) development and validation studies for discretionary revision knee joint replacement.

• Our search strategy was based on the Oxford PROM filter, which has been shown to be a sensitive tool for identifying relevant studies.

• PROM instruments that were not patient completed were excluded, which maintained a patient-focus, but limited the number of eligible instruments for evaluation.

• While our study has critically summarised PROM measurement properties, qualitative studies may be needed in the future to provide deeper insights into the outcomes from revision knee replacement that are most important to patients.

Introduction

Primary knee replacement is a successful procedure that improves quality of life for the majority of patients by reducing pain and improving joint function.¹ However, not all patients achieve a good outcome. For example, approximately 13% of patients are dissatisfied with their outcome following knee replacement,² with higher rates in younger patients³ and those with partial thickness cartilage loss.⁴ Many of these patients are managed with supportive treatment.⁵ However, at 10 years following primary knee replacement, 3.5% of patients will have undergone revision surgery. In total, 6500 revision knee replacement procedures are performed each year in the UK.⁶ The majority of these procedures (~85%) are for discretionary indications, where the goal of surgery is to improve joint function and quality of life.⁶ These contrast to non-discretionary procedures (such as for infection or fracture), which are necessary to prevent catastrophic joint failure or new comorbidity. To measure the success or otherwise of the outcome from discretionary revision knee replacement, one important aspect is the ability to measure pain and joint function from the perspective of the patient.

Patient-reported outcome measures (PROMs) are widely used for this purpose in lower limb surgery. Many PROMs aim to report quality of life and functional outcomes, while others assess sporting performance, activities of daily living or psychological health. However, not all have optimal measurement properties.^{7 8} For primary knee replacement, many PROMs have good quality evidence for their validity.^{9 10} This has facilitated utilisation of PROMs to support patient choice and manage healthcare providers,^{2 11 12} with many schemes also including revision procedures. A prominent example is the NHS PROMs programme,² which has collected data from more than 10 000 patients who have undergone revision knee replacement.¹³ However, interpretation of this data has been critically limited by a lack of PROM validation.

Revision knee replacement is one of the most expensive procedures in modern healthcare¹⁴ and high-quality PROM data is important to evaluate cost-effectiveness.¹⁵ While generic PROMs can be used to compare patients with different conditions, they may miss important items in specific populations.¹⁶ The COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) initiative provides tools to aid systematic reviews and selection of measurement instruments.¹⁷ The ideal PROM is developed or subsequently validated in the population of interest, has good measurement properties (GMP) and is supported by high-quality evidence. PROM instruments meeting these criteria can be selected for a core outcome set in order to standardise outcome measurement. If there are no suitable PROMs, then further validation studies may be required or the development of a new PROM. For discretionary revision knee replacement, no systematic review has evaluated PROMs in current use, their measurement properties or the quality of this evidence. This limits meta-analysis of previous research and design of future trials.

The aims of this review were: (1) to scope the literature to identify PROMS in current use for evaluation of symptoms, health status or quality of life following discretionary revision (or re-revision) knee replacement, and (2) to identify validated joint-specific PROMs, their measurement properties and quality of evidence.

Methods

This section is structured to follow the COSMIN Handbook and a figure to illustrate our methods is provided in an online supplemental appendix 1.¹⁷

Patient and Public Involvement

Patients and the public were involved in the design, or conduct, or reporting, or dissemination plans of our research. This article was motivated by the James Lind Alliance Priority Setting Partnership for revision knee replacement,¹⁸ particularly the question: ‘How should we measure the outcomes following revision knee surgery in a way that is meaningful to patients?

Part A: aim and literature search

Step 1: Aims

Described above.

Step 2: Study eligibility criteria

Randomised and non-randomised studies were eligible for inclusion. Revision knee replacement was defined as any procedure where an arthroplasty component was removed, modified or added. This included isolated liner exchange, secondary patellar resurfacing and re-revision procedures. Studies where the majority of procedures were performed for non-discretionary indications (such as infection or malignancy) were excluded, as well as amputations and arthrodesis procedures. Since 85% of revisions are for discretionary indications, studies where the indication was not specified were deemed eligible for inclusion. PROMs were required to address one of the following domains:

• Pain (eg, Western Ontario and McMaster Universities Arthritis Index (WOMAC) pain subscale¹⁹),

• Function (eg, WOMAC functional limitation subscale),

• Combined pain and function (eg, Oxford Knee Score²⁰),

• Joint-related health status (eg, Knee Injury and Osteoarthritis Outcome Score (KOOS) quality of life (QOL)²¹), or

• Patient activity (eg, Lower Extremity Activity Scale (LEAS²²).

Collectively, we have termed these ‘joint-specific’ PROMs. The focus of this study was not to examine generic health-related quality of life instruments (eg, EQ-5D²³). However, we did report the use of these instruments in conjunction with a joint-specific PROM. Outcome scores not considered to be patient-centred were excluded; for example, surgeon-completed scores such as the Bristol Knee Score (BKS) and the Knee Society Score (KSS). Studies with less than 50 patients were excluded as their sample size would be considered inadequate when applying COSMIN rules for rating of measurement properties and evidence quality.¹⁰

Step 3: search strategy

This is provided in online supplemental appendix 2. MEDLINE, Embase, AMED and PsycINFO were searched on 1 July 2020 using the Oxford PROM filter.²⁴ Searches were translated for each database. There were no limitations on language or publication date. The citations of included studies were searched to identify additional articles.

Step 4: study selection

Two authors (SAS and EAH) independently reviewed title and abstract for all records returned by the search against eligibility criteria. Disagreement was resolved through discussion of the full text publication. Data were extracted using a calibrated form on name and type of PROM, geography, journal, year of publication and number of patients. Data were summarised using counts with percentage frequency for each of the data items collected.

Part B: evaluation of measurement properties of the included PROMs

Steps 5, 6 and 7: content validity, internal structure, reliability and responsiveness

Descriptions of terminology for measurement properties are provided in online supplemental appendix 3. Each measurement property was evaluated in three separate sub-steps:

Substep 1: evaluation of methodological quality

Two authors (SAS and SGFA) independently evaluated the measurement properties in each article against the COSMIN Risk of Bias checklist. A priori hypotheses for construct validity and responsiveness were set (online supplemental appendix 4, table 1). Study quality was assessed separately for each measurement property using a four-point rating system (very good, adequate, doubtful or inadequate). The ‘worst score counts’ principle was used, where the overall rating for each measurement property is given by the lowest rating of any standard in the box.²⁵

Substep 2: application of criteria for GMP

Two authors (SAS and SGFA) independently extracted data on: PROM characteristics (intended construct for measurement, measurement properties, method of administration), study sample (number of patients, patient demographics, diagnosis) and study details (setting, country, language). The few disagreements were resolved through discussion. The results from each study on a measurement property were assigned a quality rating as: sufficient (+), insufficient (−) or indeterminate (?).

Substep 3: summary and grading of quality of evidence

This section refers to rating the quality of the PROM as a whole. PROMs were qualitatively summarised and assigned a four-point quality rating. A modified Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach (omitting publication bias) was used to assign evidence quality as high, moderate, low or very low.²⁶

Part C: selecting a PROM

Step 8: description of interpretability and feasibility

Interpretability and feasibility were analysed descriptively as per COSMIN guidance.¹⁷

Step 9: formulation of recommendations

PROMs were categorised into three categories: (A) Sufficient content validity and at least low-quality evidence for internal consistency; (B) Between ‘A’ and ‘C’; and (C) High-quality evidence for an insufficient measurement property. PROMs rated ‘A’ can be recommended for use. PROMs rated ‘B’ have potential for recommendation but require further evaluation. PROMs rated ‘C’ should not be recommended.

Step 10: reporting of the systematic review

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram is provided in figure 1.

Figure 1.

PRISMA flow diagram. The full search strategy is provided in online supplemental appendix 2. PROM, patient-reported outcome measure.

Results

Part A

Study selection

One thousand two hundred and five unique articles were identified for screening. Sixty-six full text articles were assessed for eligibility. Fifty-one studies were included in the scoping review, reporting on eight joint-specific PROMs. Four studies met inclusion criteria for PROM validation, describing measurement properties for three PROMs (figure 1).

Characteristics of studies reporting PROM outcomes for revision knee replacement

Fifty-one studies reported on PROM outcomes (tables 1 and 2) recruiting a median of 104 (range 51–1391) patients. Study designs included 1 (2.0%) randomised controlled trial, 14 (27.5%) prospective cohort studies, 29 (56.9%) retrospective cohort studies, 3 (5.9%) reports from national joint registries, 3 (5.9%) cross-sectional surveys and 1 (2.0%) data analysis of routinely collected secondary care data. Twenty-five studies (49.0%) were from Europe, 19 (37.3%) from North America, 6 (11.8%) from Asia and 1 (2.0%) from Australasia. The joint-specific PROMs reported were the WOMAC Index (25 studies, 49.0%), Oxford Knee Score (OKS) (19 studies, 37.3%), KOOS (8 studies, 15.7%), LEAS (4 studies, 7.8%), University of California at Los Angeles Activity Score (UCLA, 4 studies, 7.8%), Kujala score (2 studies, 3.9%), Lower Extremity Functional Scale (LEFS, 2 studies, 3.9%) and the Lysholm score (1 study, 2.0%). The majority of studies were published within the past 5 years (27/51 (52.9%) studies) (online supplemental appendix 4, figure 1).

Table 1.

Characteristics of studies reporting PROMs for revision knee replacement

Authors	Year	Country	Journal	Study design	No of revision knees	Validation study?	PROM(s) used
Hartley et al40	2002	UK	BJJ	Prospective cohort	60		SF12, WOMAC
Meek et al41	2003	Canada	BJJ	Prospective cohort	107		SF12, WOMAC
Meek et al42	2004	Canada	JOA	Cross-section	67		OKS, SF12, WOMAC
Saleh et al29	2005	USA	JBJS(Am)	Prospective cohort	297	Yes	LEAS, WOMAC
Masri et al43	2006	Canada	JOA	Retrospective cohort	126		OKS, SF12, WOMAC
Dahm et al44	2007	USA	JOA	Cross-section	335		UCLA
Ghanem et al45	2007	USA	CORR	Prospective cohort	80		SF36, WOMAC
Mulhall et al46	2007	USA	J Knee Surg	Prospective cohort	291		LEAS, SF36, WOMAC
de Groot et al32	2008	Netherlands	Health Qual Life	Prospective cohort	54	Yes	KOOS, SF36
Ghomrawi et al22	2009	USA	JBJS(Am)	Prospective cohort	308	Yes	LEAS, SF36, WOMAC
Kim and Kim47	2009	South Korea	JBJS(Am)	Retrospective cohort	157		WOMAC
Ghanem et al33	2010	USA	JBJS(Am)	Retrospective cohort	152	Yes	SF36, WOMAC
Greidanus et al48	2011	USA	JOA	Retrospective cohort	60		OKS, SF12, WOMAC
Hanna et al49	2011	UK	CORR	Retrospective cohort	56		OKS
Lavernia et al50	2011	USA	CORR	Retrospective cohort	132		SF36, WOMAC
Richards et al51	2011	Canada	JOA	Cross-section	72		SF12, UCLA, WOMAC
Baker et al36	2012	UK	CORR	Joint Registry	797		EQ-5D, OKS
Malviya et al52	2012	UK	KSSTA	Prospective cohort	175		SF36, WOMAC
Baier et al53	2013	Germany	J Orth Sci	Retrospective cohort	78		WOMAC
Huang et al54	2014	USA	Orthopaedics	Prospective cohort	96		SF36, WOMAC
Kasmire et al55	2014	USA	The Knee	Retrospective cohort	175		SF36, WOMAC
Luque et al56	2014	Spain	Int Orth	Retrospective cohort	125		OKS
Stambough et al57	2014	USA	BJJ	Retrospective cohort	81		UCLA
Weiss et al58	2014	Sweden	Acta Orthop	Retrospective cohort	65		EQ-5D, KOOS
Hitt et al59	2015	USA	J Knee Surg	Prospective cohort	95		KOOS, LEAS, SF36
Kim et al60	2015	South Korea	JOA	Retrospective cohort	228		WOMAC
Konrads et al61	2015	Germany	Int Orth	Retrospective cohort	62		Kujala, OKS, SF36
Lunebourg et al62	2015	France	JOA	Retrospective cohort	54		KOOS
Grayson et al63	2016	USA	JOA	Retrospective cohort	177		UCLA
Leta et al64	2016	Norway	JBJS(Am)	Joint Registry	1346		EQ-5D, KOOS
Leta et al65	2016	Norway	Int Orth	Joint Registry	308		EQ-5D, KOOS
Hamilton et al66	2017	UK	JOA	Prospective cohort	53		OKS
Lim et al67	2017	Singapore	BJJ	Retrospective cohort	75		OKS, SF36
Martin-Hernandez et al68	2017	Spain	KSSTA	Prospective cohort	134		SF12, WOMAC
Rajgopal et al69	2017	India	JOA	Retrospective cohort	98		WOMAC
Sandiford et al70	2017	Canada	CORR	Retrospective cohort	450		OKS, SF12, WOMAC
Zhamilov et al71	2017	Turkey	JOA	Retrospective cohort	92		LEFS
Agarwal et al72	2018	UK	The Knee	Prospective cohort	104		EQ-5D, OKS
Boelch et al73	2018	Germany	Int Orth	RCT	51		OKS, SF36
Eibich et al15	2018	UK	BMJ Open	Routine data	1391		EQ-5D, OKS
Gomez-Vallejo et al74	2018	Spain	J Orth Traum	Retrospective cohort	67		SF36, WOMAC
Weber et al75	2018	Germany	BioMed RI	Retrospective cohort	68		EQ-5D, WOMAC
Bin Abd Razak et al76	2019	Singapore	J Knee Surg	Retrospective cohort	163		OKS, SF36
Konrads et al77	2019	Germany	J Knee Surg	Retrospective cohort	135		Kujala, OKS, SF36
Kurmis et al78	2019	Australia	JOA	Retrospective cohort	321		OKS, WOMAC
Lim et al79	2019	Singapore	The Knee	Retrospective cohort	70		OKS, SF36
Scior et al80	2019	Germany	JOA	Prospective cohort	482		OKS
Stockwell et al81	2019	Canada	The Knee	Retrospective cohort	234		OKS
Klim et al82	2020	Austria	KSSTA	Retrospective cohort	93		SF36, WOMAC
Larsen et al83	2020	Denmark	BMC Sports Sci	Retrospective cohort	51		KOOS
Oliver et al84	2020	Spain	Orth Surg	Retrospective cohort	89		KOOS, Lysholm

KOOS, Knee Injury and Osteoarthritis Outcome Score; LEAS, Lower Extremity Activity Scale; LEFS, Lower Extremity Functional Scale; OKS, Oxford Knee Score; PROM, patient-reported outcome measure; SF, Short Form; UCLA, University of California at Los Angeles Activity Score; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Table 2.

Summary characteristics for studies reporting PROMs following revision knee replacement

	Number of studies (%)
No of patients	Median 104 (range 51–1391)
Continent
Europe	25 (49)
North America	19 (37.3)
Asia	6 (11.8)
Australasia	1 (2)
Type of study
Randomised controlled trial	1 (2)
Prospective cohort	14 (27.5)
Retrospective cohort	29 (56.9)
Joint Registry	3 (5.9)
Routine data analysis	1 (2)
Cross-sectional survey	3 (5.9)
Joint-specific PROMs
KOOS	8 (15.7)
Kujala	2 (3.9)
LEAS	4 (7.8)
LEFS	1 (2)
Lysholm	1 (2)
OKS	19 (37.3)
UCLA	4 (7.8)
WOMAC	25 (49)
Generic PROMs
EQ-5D	7 (13.7)
SF12	8 (15.7)
SF36	18 (35.3)

Number of studies reporting each measure (%)

KOOS, Knee Injury and Osteoarthritis Outcome Score; LEAS, Lower Extremity Activity Scale; LEFS, Lower Extremity Functional Scale; OKS, Oxford Knee Score; PROM, patient-reported outcome measure; SF, Short Form; UCLA, University of California at Los Angeles Activity Score; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Part B

Quality of PROM development studies

The quality of PROM development for the eight disease-specific PROMs identified in Part A is summarised in table 3. The construct to be measured was clear in two studies (25%), with the remainder rated ‘inadequate’. One example of a study rated ‘inadequate’ was the Kujala study.²⁷ This rating was made because, while the score was designed to measure anterior knee symptoms, the specific aspects of these symptoms to be measured were not described (such as pain intensity or pain interference). The Lysholm score²⁸ was rated ‘very good’ due to a specific description (defining ‘the lowest activity level needed during walking, running or jumping to produce giving way or pain and swelling’). The origin of the construct to be measured was clear in only two studies (25.0%). One example of a study rated ‘very good’ for this property was the LEFS study,²⁹ which referenced the WHO’s International Classification of Functioning, Disability and Health (ICF) conceptual framework.³⁰ The context of use was rated ‘very good’ for three studies (37.5%). These studies provided at least one clear description of the intended application of the instrument. For example, the OKS was designed to evaluate patients before and after knee replacement surgery.²⁰ All studies were rated as ‘very good’ for their description of a clear target population. While many studies provided a very broad description (eg, the LEFS described patients ‘with lower extremity orthopaedic conditions’³¹), the COSMIN guidance is permissive for rating this property. However, the PROM development sample was rated ‘inadequate’ for all studies either because the patient sample was not correspondingly broad or, taking a view on the patient sample of interest in this review, did not recruit a sample representative of discretionary revision knee replacement. While the LEAS study did recruit patients with revision knee replacements for some aspects of PROM development, a surgeon panel was used in lieu of patients for content validity, justifying an ‘inadequate’ rating.²⁹ In summary, the total PROM development was rated ‘inadequate’ for all studies based on the ‘worst score counts’ principle recommended by COSMIN. However, this does not reflect positive ratings for some aspects of PROM development as described above.

Table 3.

Quality of PROM development

PROM	PROM design							Cognitive interview (CI) study*				Total PROM development
	General design requirements					Concept elicitation†	Total PROM design	General design requirements	Comprehensibility	Comprehensiveness	Total CI study
	Clear construct	Clear origin of construct	Clear target population for which the PROM was developed	Clear context of use	PROM developed in sample representing the target population			CI study performed in sample representing the target population
Joint-specific
KOOS21	I	D	VG	D	I		I	Yes				I
Kujala27	I	D	VG	D	I		I	No				I
LEAS29	I	D	VG	VG	I		I	No				I
LEFS31	I	VG	VG	D	I		I	No				I
Lysholm28	VG	D	VG	D	I		I	No				I
Oxford Knee Score20	I	D	VG	VG	I		I	No				I
WOMAC19	VG	VG	VG	VG	I		I	Yes				I
UCLA85	I	D	VG	D	I		I	No				I
Generic
EQ-5D23	I	D	VG	VG	I		I					I
SF-3639	VG	VG	VG	VG	I		I					I
SF-1286	VG	VG	VG	VG	I		I					I

*Empty cells indicate that a CI study (or part of it) was not performed.

†Where the PROM was not developed in a sample representing the target population, the concept elicitation was not further rated.

A, adequate; D, doubtful; I, inadequate; KOOS, Knee Injury and Osteoarthritis Outcome Score; LEAS, Lower Extremity Activity Scale; PROM, patient-reported outcome measure; SF, Short Form; UCLA, University of California at Los Angeles Activity Score; VG, very good; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Characteristics of PROM validation studies

Four studies22 29 32 33 from the scoping review validated three joint-specific PROMs (KOOS, LEAS, WOMAC) (table 4). The mean age of patients in the included studies ranged from 67 to 77 years. Female patients accounted for 50% to 78% of the study populations. The primary objective of the included articles varied from validation of a PROM, validation of another instrument with the PROM as a comparator, development of a new instrument and reporting of clinical outcome after revision knee replacement. The characteristics of the PROMs included in the validation studies are described in table 5.

Table 4.

Characteristics of PROM validation studies

Study	Instrument(s)	Primary objective	Country (language)	Population (inclusion/exclusion criteria)	Enrolled (n)	LTFU (n)	Final (n)	Age (years)	Female (%)	FU (months)	Indications for revision
de Groot et al32	KOOS SF-36 VAS for pain	To validate the Dutch translation of KOOS	Netherlands (Dutch)	Inc: Revision TKR Exclusion: Unable to understand Dutch written language.	54	7	47	77 (36–89)	78	NR	NR
Saleh et al29	LEAS WOMAC	To develop and validate the Lower Extremity Activity Scale	USA (English)	Inc: First revision TKR capable of completing questionnaires in English and ≥18 years Exc: Re-revision, failed UKR, poly. exchange only, bone tumour, reflex sympathetic dystrophy, unfit for revision TKR, neurological deficit of affected limb, referred pain from spine, declined to participate, concern about compliance, inability to consent, progressive muscular condition of quadriceps, infection delay, stiffness not requiring component revision.	297	12	285	68.6 (r 34–85)	55	6	Instability n=82 (28.8%) Tibial osteolysis n=78 (27.4%) Poly. wear n=70 (24.5%) Fem. osteolysis n=64 (22.5%) Tibial loosening n=63 (22.1%)
Ghomrawi et al22	LEAS SF-36 WOMAC	To characterise patterns of functional improvement after revision total knee arthroplasty over a 2-year period using Lower Extremity Activity Scale	USA (English)	As per Saleh et al29 (2005)	308	87	221	68.7 (r 34–85)	55	24	Instability 28.9%* Poly. wear 24.5% Failed poly. insert 18.1% Malalignment 9.4% Tibial loosening 22.2%	Fem. loosening 14.1% Infection 10.4% Tibial lysis 27.5% Fem. lysis 22.5% Patella lysis 9.4%
Ghanem et al33	WOMAC SF-36 KSS 4-point Likert	To determine validity and responsiveness of the Knee Society Rating System	USA (English)	Inc: Revision TKR Exc: Infection (n=85), Patella or poly. exchange only (n=35); Conversion of UKR or internal fixation (n=15), Non-prosthetic failure (n=4)	165	13	152	67 (r 36–89)	NR	24	Mechanical failure: Aseptic loosening 69.7% Knee instability 30.3%

*Data for this population are provided in a separate paper by Mulhall et al,⁸⁷ number of patients not provided only percentages.

Age, mean (SD) or (r=indicating range); KOOS, Knee Injury and Osteoarthritis Outcome Score; KSS, Knee Society Rating System; LEAS, Lower Extremity Activity Scale; Lig., ligamentous; LTFU, lost to follow-up; NR, not reported; Poly., polyethylene; PROM, patient-reported outcome measure; SF-36, Short Form 36; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Table 5.

Characteristics of the joint-specific PROMs evaluated in validation studies

Instrument	Year developed	Original language	Target population	Intended construct/domains	No of questions	Best/worst score
Symptoms and functional status
KOOS21	1998	English and Swedish	Younger and more active subjects at risk of knee osteoarthritis following knee injury	Pain Symptoms Activities of daily life function Sports and recreation function Knee-related quality of life	42 questions	100/0
WOMAC19	1982	English	Patients with OA of the hip or knee	Pain Stiffness Function and daily activities	24 questions	0/96
Activity level
LEAS29	2005	English	Patients awaiting or had undergone primary or revision lower limb joint replacement	Physical activity	1 question	18/1

KOOS, Knee Injury and Osteoarthritis Outcome Score; LEAS, Lower Extremity Activity Scale; PROM, patient-reported outcome measure; SF-36, Short Form 36; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Quality of studies on measurement properties

In total, 20 measurement properties for the KOOS, LEAS and WOMAC were evaluated (table 6). There were 40 additional opportunities to evaluate measurement properties that were not attempted. Two (10.0%) measurement properties were rated ‘very good’, 5 (25.0%) ‘adequate’, 3 (15.0%) ‘doubtful’ and 10 (50.0%) ‘inadequate’. For structural validity, de Groot’s evaluation for the KOOS was rated ‘inadequate’ due to an insufficient sample size for factor analysis (less than five times the number of participants). Three out of four (75.0%) studies that reported on responsiveness were rated ‘inadequate’ due to their construct approach. For example, Saleh et al²⁹ used an ‘inadequate’ comparator instrument for development of the LEAS—the measurement properties of the WOMAC are not well enough known for revision. Ghomrawi et al²² did not set hypotheses for construct validity, and their statistical methodology did not allow these to be evaluated at review. Two studies reported on reliability. These were rated ‘adequate’ as, while they chose an appropriate interval, they did not also ensure patients were stable.

Table 6.

Quality of studies on measurement properties

PROM	Study	Structural validity	Internal consistency	Cross-cultural validity	Reliability	Measurement error	Criterion validity	Construct validity		Responsiveness
								Convergent validity	Known groups validity	Comparison with gold standard	Comparison with other instruments	Comparison between subgroups	Comparison before/after intervention
KOOS	de Groot et al32	I	VG	I	A	A	N	D	N	N	N	N	N
LEAS	Saleh et al29	N	N	N	A	A	N	I	N	N	I	N	A
LEAS	Ghomrawi et al22	N	N	N	N	N	N	N	N	N	I	I	I
WOMAC	Ghanem et al33	N	N	N	N	N	N	D	N	N	D	N	VG
WOMAC	Ghomrawi et al22	N	N	N	N	N	N	N	N	N	I	I	I

A, adequate; D, doubtful; I, inadequate; KOOS, Knee Injury and Osteoarthritis Outcome Score; LEAS, Lower Extremity Activity Scale; N, not assessed; PROM, patient-reported outcome measure; VG, very good; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Quality of the evidence for measurement properties of the PROMs

The quality of the evidence for measurement properties of the included PROMs is provided in table 7. Twenty-five out of 27 (92.6%) measurement properties were rated insufficient, indeterminate or not assessed. The only measurement property to receive a ‘sufficient’ rating was reliability for both the KOOS and the LEAS, supported by ‘low’ and ‘moderate’ quality evidence, respectively.

Table 7.

Quality of the evidence for measurement properties of the PROMs

	KOOS		LEAS		WOMAC
	Overall rating	Quality of evidence	Overall rating	Quality of evidence	Overall rating	Quality of evidence
	+/−/?	High, moderate, low, very low	+/−/?	High, moderate, low, very low	+/−/?	High, moderate, low, very low
Structural validity	–	Very low	N	N	N	N
Internal consistency	?	Moderate	N	N	N	N
Cross-cultural validity	?	Very low	N	N	N	N
Measurement invariance	?	Very low	N	N	N	N
Reliability	+	Low	+	Moderate	N	N
Measurement error	?	Low	?	Very low	N	N
Criterion validity	N	N	N	N	N	N
Construct validity	–	Low	–	Very low	?	Very low
Responsiveness	N	N	?	Very low	?	Very low

+ = sufficient, – = insufficient, ? = indeterminate.

KOOS, Knee Injury and Osteoarthritis Outcome Score; LEAS, Lower Extremity Activity Scale; N, not assessed; PROM, patient-reported outcome measure; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Part C

Data on the interpretability of the studies is summarised in table 8. The mode of PROM administration was unclear for all studies except de Groot et al.³² Missing responses ranged from 25% to 60%. No study reported on missing items within a PROM instrument. Floor and ceiling effects were not reported, except by Saleh et al.²⁹ No PROM met criteria either to be recommended or not recommended for use. Each of the validated PROMs (ie, KOOS, LEAS and WOMAC) was therefore assigned recommendation ‘B’, indicating that further evidence is needed.

Table 8.

Interpretability including missing items, response rate and floor/ceiling effects

Instrument and study	Administration	Missing responses (%)	Missing items (%)	Overall % achieving lowest possible total score (floor)	Overall percentage achieving highest possible score (ceiling)	Items/domains with >15% responses with lowest score (floor)	Items/Domains with >15% responses with highest score (ceiling)	MIC
Symptoms and functional status
KOOS
de Groot et al32	Postal	25	NR	NR	NR	Sports/recreation	Nil	NR
WOMAC
Ghomrawi et al22	Unclear	30.5	NR	NR	NR	NR	NR	NR
Ghanem et al33	Unclear	NR	NR	NR	NR	NR	NR	NR
Saleh et al29	Unclear	NR	NR	NR	NR	NR	NR	NR
Health-related quality of life
SF-36
de Groot et al32	Postal	NR	NR	NR	NR	NR	NR	NR
Ghomrawi et al22	Unclear	30.5	NR
Ghanem et al33	Unclear	NR	NR	NR	NR	NR	NR	NR
Activity level
LEAS
Ghomrawi et al22	Unclear	30.5	NR	NR	NR	NR	NR	NR
Saleh et al29	Unclear	59.6*	NR	0	0	NR	NR	NR

*Reported response rate was 96%. However, histograms have 177 or 178 patients out of a possible 297 (59.6%).

KOOS, Knee Injury and Osteoarthritis Outcome Score; LEAS, Lower Extremity Activity Scale; MIC, minimal important change; NR, not reported; SF-36, Short Form 36; WOMAC, Western Ontario and McMaster Universities Arthritis Index.

Discussion

This review has demonstrated the increasing use of PROMs to evaluate symptoms and functional outcomes following discretionary revision knee replacement. The majority of studies were retrospective and observational, with only one randomised controlled trial. Eight different joint-specific PROMs were identified, with the WOMAC index (25 studies, 49.0%) and the OKS (19 studies, 37.3%) the most frequent. Only three joint-specific PROMs were supported by a validation study: KOOS, LEAS and WOMAC. Each of these validation studies had ‘low’ or ‘very low’ quality evidence and the majority of measurement properties were either not evaluated or rated ‘inadequate’ or ‘indeterminate’. As such, each of these PROMs requires more evidence in order to be recommended for use.

Secondary findings and relation to other studies

Musculoskeletal disorders account for one-third of all reviews on the COSMIN database.³⁴ At least three reviews have evaluated the measurement properties of PROMs following primary knee replacement.^{9 10 35} These studies found that many PROM instruments had limited evidence to support their measurement properties, justifying the need for further research. We are not aware of previous reviews that have examined the measurement properties of PROMs following discretionary revision knee replacement. While many of the goals from discretionary revision knee replacement are shared with primary knee replacement, there are important differences in the patient populations and disease processes being treated and the surgical interventions themselves. For example, while primary knee replacement treats predominantly osteoarthritis, discretionary revision knee replacement treats many varied disease processes.³⁶ The revision patient population is also more comorbid and may have different expectations from surgery.³⁷ As such, the evidence for PROMs developed in primary knee replacement cannot necessarily be assumed to be transferable across.

Strengths and weaknesses

This study has a number of important strengths, including the use of a broad search strategy based on the Oxford PROM filter²⁴ and the application of latest COSMIN guidelines. The use of a priori hypotheses by our review team to evaluate construct validity and responsiveness is novel and meant these properties could be considered even when not a focus of the original article. This study was motivated by the James Lind Alliance Priority Setting Partnership for revision knee replacement, which generated the question: ‘How should we measure the outcomes following revision knee surgery in a way that is meaningful to patients?’.³⁸ As such, outcome scores that were not patient completed were excluded. We acknowledge that this has restricted the number of eligible studies from North America, where use of the KSS is prevalent. In the future, qualitative studies to explore patients’ reasons for choosing surgery and to identify the outcomes that are most important to patients may be needed.

Implications for practice

We have not put forward a PROM for recommendation because the quality of the available evidence was low, and data were lacking for many of the measurement properties. However, we can make recommendations to direct future research and to move towards developing a core outcome set for discretionary revision knee replacement. First, we wish to highlight that standards for reporting of psychometric studies have changed considerably over the past 20 years.⁹ COSMIN tools are not limited to systematic reviews and may be used guide the scope and detail required to develop a new instrument or to evaluate an existing one. Second, this study has highlighted a number of common methodological flaws that result in high risk of bias. For example, when evaluating structural validity, none of the validation studies performed confirmatory factor analysis to understand whether the PROM scores reflected the dimensionality of the construct. For reliability, test conditions were not recorded with sufficient detail to ensure that not only the repeat interval was appropriate but also that the patient remained stable. For interpretability, none of the studies calculated a minimal important change nor comprehensively assessed floor and ceiling effects. Third, we recommend that future studies planning to use an existing joint-specific PROM to evaluate outcomes after revision surgery do so in conjunction with a validated generic health-related quality of life instrument (such as the Short Form-36 (SF36)³⁹ or EQ-5D²³). While neither the EQ-5D or SF36 were developed in patients undergoing revision knee replacement, their measurement properties have been studied extensively and allow generalisability between different conditions. This approach will provide valuable information on construct validity and responsiveness in the future.

Conclusion

In conclusion, joint-specific PROMs are increasingly used to report outcomes following revision knee replacement, but these instruments have insufficient evidence for validity. Future research is needed to target the deficiencies highlighted by this review in order to inform clinical trials and observational studies evaluating these outcomes.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.