Quality of patient-reported and proxy-reported outcomes for children with impairment of the lower extremity: systematic review protocol

Abstract

Introduction

As patient-reported outcome measures (PROMs) have become of significant importance in evaluation of care and clinical research, adequately selecting the appropriate instrument is an integral part of paediatric orthopaedic research and clinical practice. This systematic review aims to provide a comprehensive overview of PROMs targeted at children with impairment of the lower limb, and to critically appraise and summarise the quality of their measurement properties by applying the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology.

Method and analysis

A systematic search of the MEDLINE and EMBASE databases will be performed to identify relevant publications reporting on the development and/or validation of PROMs used for evaluating children with impairment of the lower extremity. Data extraction and quality assessment of the included studies will be undertaken by two reviewers independently and in accordance with COSMIN guidelines.

Ethics and dissemination

It is not necessary to obtain ethical approval for this systematic review. The results will be published in a peer-reviewed journal and will be presented at relevant conferences to enhance information dissemination.

PROSPERO registration number

CRD42021287323.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Using patient-reported outcome measures (PROMs), in clinic and in research, to evaluate the changes in a patients (mental) health status has become common practice.

• Good and easy-to-use PROMs for children with lower extremity problems are scarce, and a clear overview is currently unavailable.

• In general, the majority of existing paediatric PROMs are used without proper scientific justification and lack a prior high-quality validation study.

WHAT THIS STUDY ADDS

• This study will provide insight into the knowledge gap existent within the proper usage of PROMs for children with impairment of the lower extremity.

• A high-quality methodological assessment using the COSMIN Guidelines to report the psychometric properties from multiple validation studies.

• A categorised overview and comprehensive recommendation of all viable PROMS for children with impairment of the lower extremity.

Introduction

The patient experience and patient related outcomes (PROs) have become the conventional approach to describing the effects of their perceived healthcare services both in clinic and in trials.¹ PROs, measured using patient-reported outcome measures (PROM), provide a comprehensive assessment of the patients’ current health condition.^{2 3} Orthopaedic surgeons and researchers often use PROMs to evaluate the health status of a patient and track the change over time or after treatment.^{1 2 4} A recent publication by Arguelles et al ⁵ identified the major challenges clinicians and researchers face when using PROMs in paediatric orthopaedic research; for example, most PROMs are used without formal validation for both content and context. Thus, the clinical or researcher interpreting these results could potentially make a misleading recommendation or perform an unnecessary intervention.⁶

In orthopaedic, clinical practice most often PROMs are used to evaluate the physical and mental status of a patient before or after an intervention, and PROMs are used to objectively measure subjective physical and/or behavioural patterns in patients with a physical impairment due to a disease to the musculoskeletal system. The selection of a PROM is vital to its own effectiveness in providing valuable information to the physician, if the PROM has not been validated for the proposed use and the intended joint, it will provide false information to the physician.

Therefore, clinicians and researchers need to use an age-appropriate PROM specifically designed for the potential diagnosis or injury of the patient; the content validity should fit both the disease and the population. Even though a variety of studies on the quality of development and validation of PROMs are readily available for clinicians and researchers, it remains challenging to judge whether the results of these studies justify an appropriate use of PROM for the intended patient population.^{4 7} This may result in a PROM being used in either an unvalidated patient population with a validated disease or in a validated patient population with an unvalidated disease.⁵ Up to 18 years of age, children will develop most of their cognitive physical skills. If a child were to develop an impairment to the upper or lower extremity due to a disease of the musculoskeletal system, the orthopaedic surgeon would use a PROM to objectively measure the progression of the disease. The outcome of the PROMs in children needs to verify if their progression/regression is due to normal physiological growth or due to a disease.

When developing and using PROMs in paediatric research, clinicians and researchers must take developmental influences such as age-dependent disease-awareness and cognitive–linguistic ability, into careful consideration for paediatric qualitative research.⁸ These conditions impede the proper development of validated and easy-to-use paediatric PROMs, which results in a significant sparseness; consequently increasing the difficulty of proper PROM selection in the paediatric orthopaedic patient population.⁷ The improper selection is exacerbated by the wide variety of individual PROMs per joint.⁹

The COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) initiative has developed the COSMIN checklist and guideline for systematic reviews of outcome measurement instruments.^{10 11} This checklist was specifically developed to evaluate the quality of the PROMs development and the methodological quality of individual studies on psychometric properties of patient-related outcomes measurements in concordance with the GRADE approach, the PRISMA statement and the Cochrane handbook for systematic reviews of interventions and for diagnostic test accuracy.^{10 12 13}

Objectives

The primary aim of the review is to construct a comprehensive index of all paediatric orthopaedic PROMs validated for children with impairments of the lower extremity, and to critically appraise and summarise the quality of their measurement properties with use of the COSMIN checklist. The secondary goal of this review is to provide an evidence-based recommendation for PROM selection in paediatric orthopaedic research and clinical practice for patients with an impairment of the lower extremity.

Methods

Design

This protocol was developed using the Preferred Reporting for Items for Systematic Reviews and Meta-Analyses-Protocols (PRISMA-P) checklist¹⁴ and the 10-step procedure in the COSMIN Risk-of-Bias checklist for systematic reviews of PROMs.^{11 12 15} The conduct and reporting of this review will be in accordance with the PRISMA statement and in accordance with the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach.

Information sources and search strategy

The systematic review will involve a broad search according to the COSMIN guidelines. The electronic databases of PubMed and Embase will be systematically searched using a search designed with the help of a clinical librarian, with an emphasis on sensitivity, not specificity, to ensure identification of all relevant articles. The search will consist of four distinct elements: (1) search terms describing the population of interest with a validated paediatric study search filter by Leclercq et al,¹⁶ (2) the comprehensive PROM filter developed by the PROM Group of the University of Oxford,¹⁷ and two validated filters by Terwee et al ¹⁸: (3) a highly sensitive measurement property filter and (4) an exclusion filter. The search will be restricted to English articles only by using language filters (a preliminary search for studies in the native languages spoken by the authors (Dutch) resulted in no viable studies to be included), and eligible articles have to be published after 1 January 2000 to identify contemporary outcome measures. The reference lists of included articles will be manually screened to identify additional relevant studies or citations. The search strings used for PubMed and Embase can be found in online supplemental material 1.

Eligibility criteria, inclusion and exclusion criteria

A study will be included if a full-text original version of the article is available. The article must report on an original study describing the development and/or the evaluation of one or more measurement properties of a disease-specific patient-reported and/or proxy-reported questionnaire of any language. The study population must consist of children (0–18 years old) with an orthopaedic impairment in the lower extremity region.

The exclusion criteria are: (1) any study design in which the patient-reported and/or parent/proxy-reported questionnaire was only used as an outcome measurement instrument (eg, randomised controlled trials, longitudinal studies, systematic reviews or meta-analysis); (2) any study in which the questionnaires evaluated the use of prosthetic limbs.

Patient and public involvement

No patients will be involved during this systematic review of the current worldwide available literature on PROMs for paediatric patients with an impairment of the lower extremity.

Study selection

The literature search will be uploaded to Endnote (X9, Clarivate Analytics, London, UK) and all duplicate records will be removed from the results. A full-text selection will be made during a two-phase selection procedure. During the first part, the two reviewers (TFFS and RK) will independently identify all eligible articles from title and abstract for full-text review based on the predefined eligibility criteria using the Rayyan web application,¹⁹ which facilitates this blinded selection procedure. During the second part, the two reviewers (TFFS and RK) will independently screen all full-text articles based on the eligibility criteria and conclude the final inclusion. Disagreements during both phases will be resolved by a third reviewer (IS or CvB).

Quality appraisal

The methodological quality of studies on measurement properties included in this review will be assessed according to the extensive and recently improved COSMIN methodology for qualitatively evaluating studies on PROMs.¹⁶ Detailed information on the COSMIN taxonomy, the stepwise approach of the COSMIN methodology and the COSMIN checklists applied in this review, can be found in the corresponding publications by Mokkink et al.^{12 13} Prinsen et al,¹⁵ and Terwee et al.¹¹

The modular tool, providing risk-of-bias scores for the psychometric properties, uses a four-point rating scale: ‘very good’, ‘adequate’, ‘doubtful’ or ‘inadequate’.¹² ‘The worst score counts’ principle will be applied to come to an overall methodological quality rating for each individual study on a measurement property.¹⁵ To evaluate reports on content validity and/or PROM development, a separate COSMIN guideline will be used: the COSMIN methodology for evaluating content validity.¹¹

During selection, quality appraisal and data extraction, the inter-rater agreement of the reviewers (TFFS and RK) will be considered appropriate when reviewers reach >80% agreement.²⁰ Per recommendation (Mokkink et al ²⁰), if the inter-rate agreement falls below 80%, both reviewers will review and redefine the inclusion and selection criteria during selection, quality appraisal and data-extraction, and repeat the entire process until>80% agreement is reached.

Data extraction

Two reviewers (TFFS and RK) will independently extract data from the included studies. The COSMIN guidelines have provided a guidance document¹⁵ with tables and figures for data extraction, which will be used to standardise the information that will be gathered. All information gathered will be divided into three categories with multiple subcategories (box 1).

Box 1. Visual overview of the standardised parameters gathered (three main categories and multiple subcategories) from each individual study on patient-reported outcome measures (PROMs).

Characteristics of the included PROMs

• Testable construct

• Target population

• Mode of administration

• Recall period

• Subscales (number of items in questionnaire)

• Response options

• Range of scores/scoring

• Original language

• Available translations

Characteristics of the included study populations

• Number of patients

• Age (mean and dispersion) in years

• Gender (% of females)

• Disease characteristics

  • Disease, duration in years, severity

• Instrument administration

  • Setting, country, language

• Response rate

Evidence of the psychometric measurement properties of the studied PROMs

Data synthesis

A qualitative synthesis of the evidence per measurement property per PROM will be constructed to come to an overall conclusion of PROM quality. If consistent (ie, ≥75% of the results are either rated ‘sufficient’ or ‘insufficient’), the results of the individual studies on measurement properties will be qualitatively summarised and rated against the criteria for good measurement properties. If inconsistent, an explanation for this inconsistency will be sought. If the inconsistency remains unexplained, the overall result will be rated as ‘inconsistent’ (±). An ‘indeterminate’ (?) rating will be given if the individual results are all rated as ‘indeterminate’.

After qualitatively synthesising and rating the overall results per measurement property, per PROM, the quality of this evidence will be graded according to the modified GRADE approach.^{21 22} The summarised results will be graded as ‘high’, ‘moderate’, ‘low’ or ‘very low’, based on three factors: risk of bias (based on methodological quality), inconsistency and imprecision (ie, sample size). The fourth factor ‘indirectness’ will not be taken into consideration in evaluating evidence quality; this review will only include studies with a predefined and fixed patient population. If the quality of the summarised result is rated ‘inconsistent’ or ‘indeterminate’, the quality of the evidence cannot be graded.

Patient and public involvement

No patients are involved in this study.

Discussion

The use of questionnaires/PROMs in orthopaedic medical care will become the futures’ main focus to determine patient-related clinical outcomes in both daily practice and in research.^{4 23} The research done by Arguelles et al ⁵ has shown that the majority of PROMs are used without supportive evidence for content validity and construct validity.

With this study on validated PROMs use, we hope to highlight an area of expertise, which will benefit from a research focus on validating a PROM for a specific patient population, a specific disease or a combination of disease and population. Increasing awareness about the improper PROMs usage will encourage the future research to focus more on proper development and validation studies on PROMs.

Data availability statement

No data are available.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

There is no need to obtain ethical approval for this systematic review protocol. The results will be submitted for publication in a peer-reviewed scientific journal for easy distribution among clinicians and researchers. The results will be submitted to relevant conferences to spread more awareness on adequate patient-reported outcome measure selection.