Tools to assess the measurement properties of quality of life instruments: a meta-review protocol

Abstract

Introduction

Using specific tools to assess the measurement properties of health status instruments is recommended both to standardise the review process and to improve the methodological quality of systematic reviews. However, depending on the measurement standards on which these tools are developed, the approach to appraise the measurement properties of instruments may vary. For this reason, the present meta-review aims to: (1) identify systematic reviews assessing the measurement properties of instruments evaluating health-related quality of life (HRQoL); (2) identify the tools applied to assess the measurement properties of HRQoL instruments; (3) describe the characteristics of the tools applied to assess the measurement properties of HRQoL instruments; (4) identify the measurement standards on which these tools were developed or conform to and (5) compare the similarities and differences among the identified measurement standards.

Methods and analysis

A systematic review will be conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols Guidelines. Electronic search will be carried out on bibliographic databases, including PubMed, Cumulative Index to Nursing and Allied Health Literature, Psychological Information, SCOPUS, Web of Science, COSMIN database and ProQuest Dissertations & Theses Global, being limited by time (2008–2018) and language (English). Descriptive analyses of different aspects of tools applied to evaluate the measurement properties of HRQoL instruments will be presented; the different measurement standards will be described and some recommendations about the methodological and research applications will be made.

Ethics and dissemination

Ethical approval is not necessary for systematic review protocols. The results will be disseminated by its publication in a peer-reviewed journal and presented at a relevant conference.

PROSPERO registration number

CRD42017065232

Strengths and limitations of this study.

• The search strategy has been designed to be comprehensive, following the Peer Review of Electronic Search Strategies guidelines and including filters for finding studies on measurement properties of measurement instruments.

• The systematic review protocol is developed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Protocols guidelines.

• Inclusion of studies published in English only may lead to language bias.

Introduction

Systematic reviews of measurement properties critically appraise and compare the content and measurement properties of all instruments measuring a certain construct of interest in a specific study population.¹ High quality systematic reviews can provide a comprehensive overview of the measurement properties of patient-reported outcome measures and support evidence-based recommendations in the selection of the most suitable health status instrument for a given purpose (ie, research or clinical practice).² To be confident that the design, conduct, analysis and interpretation of the review results and conclusions are adequate, the methodological quality of systematic reviews should be appraised.¹

Because of this, different authors evaluate systematic reviews assessing the measurement properties of health status assessment instruments, as Mokkink et al¹ or Terwee et al.³ In both cases, authors examine the search strategy, data extraction (two or more reviewers), data synthesis and whether the measurement properties of health status instruments were assessed using specific tools that are recommended both to standardise the review process and to improve the methodological quality of systematic reviews.³ However, depending on the measurement standards on these tools were developed, the approach to analyse the measurement properties of instruments may vary. Given this, the present meta-review aims to discuss the methodological, research and practical applications of these tools in systematic reviews that assess the measurement properties of instruments evaluating the quality of life within the context of health and disease, that is, health-related quality of life (HRQoL) instruments.⁴

Methods

Objectives

To identify systematic reviews assessing the measurement properties of HRQoL instruments.

To identify the main tools applied to assess the measurement properties of HRQoL instruments.

To describe the most relevant characteristics of the tools applied to assess the measurement properties of HRQoL instruments (validity, reliability, feasibility, etc).

To identify the measurement standards on which these tools were developed or conform to.

To compare the similarities and differences among the identified measurement standards.

Study design

Where applicable, the present meta-review will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols guidelines.⁵

Search strategy

A systematic review will be performed in PubMed, US National Library of Medicine, by National Center for Biotechnology Information; Cumulative Index to Nursing and Allied Health Literature by EBSCOhost; Psychological Information by APA PsycNET; SCOPUS by Elsevier; Web of Science CORE by Thomson Reuters and COSMIN database by COSMIN Initiative (http://www.cosmin.nl/). In addition, ProQuest Dissertations & Theses Global will be used for searching grey literature, and search alerts in all databases will be set. The search strategy will follow the Peer Review of Electronic Search Strategies guidelines recommendations^{6 7} and will consist of 3 filters composed of search terms for the following: (1) systematic review methodology; (2) HRQoL instruments and (3) measurement properties. The latter filter was developed for the VU University Medical Center for finding studies on measurement properties of measurement instruments.⁸ All filters will be adapted for all databases. The systematic search will be performed in July 2018, limited by time and language (English) (table 1 shows the string of terms in PubMed).

Table 1.

Search strings for PubMed

1	(“Quality of Life”[Mesh] OR HRQL[tiab] OR HRQoL[tiab] OR QoL[tiab] OR “quality of life”[tiab]
2	(instrument[tiab] OR instruments[tiab] OR questionnaire[tiab] OR questionnaires[tiab] OR scale[tiab] OR scales[tiab] OR tool[tiab] OR tools[tiab]
3	(Validation Studies[pt] OR “reproducibility of results”[MeSH Terms] OR reproducib*[tiab] OR “psychometrics” [MeSH] OR psychometr*[tiab] OR clinimetr*[tiab] OR clinometr*[tiab] OR “observer variation”[MeSH] OR observer variation[tiab] OR “discriminant analysis”[MeSH] OR reliab*[tiab] OR valid*[tiab] OR coefficient[tiab] OR “internal consistency”[tiab] OR (cronbach*[tiab] AND (alpha[tiab] OR alphas[tiab])) OR “item correlation”[tiab] OR “item correlations”[tiab] OR “item selection”[tiab] OR “item selections”[tiab] OR “item reduction”[tiab] OR “item reductions”[tiab] OR agreement[tw] OR precision[tw] OR imprecision[tw] OR “precise values”[tw] OR test-retest [tiab] OR (test[tiab] AND retest[tiab]) OR (reliab*[tiab] AND (test[tiab] OR retest[tiab])) OR stability[tiab] OR interrater[tiab] OR inter-rater[tiab] OR intrarater[tiab] OR intra-rater[tiab] OR intertester[tiab] OR inter-tester[tiab] OR intratester[tiab] OR intra-tester[tiab] OR interobserver[tiab] OR inter-observer[tiab] OR intraobserver[tiab] OR intra-observer[tiab] OR intertechnician[tiab] OR inter-technician[tiab] OR intratechnician[tiab] OR intra-technician[tiab] OR interexaminer[tiab] OR inter-examiner[tiab] OR intraexaminer[tiab] OR intra-examiner[tiab] OR interassay[tiab] OR inter-assay[tiab] OR intraassay[tiab] OR intra-assay[tiab] OR interindividual[tiab] OR inter-individual[tiab] OR intraindividual[tiab] OR intra-individual[tiab] OR interparticipant[tiab] OR inter-participant[tiab] OR intraparticipant[tiab] OR intra-participant[tiab] OR kappa[tiab] OR “kappa’s”[tiab] OR kappas[tiab] OR “coefficient of variation” [tiab] OR repeatab*[tw] OR ((replicab*[tw] OR repeated[tw]) AND (measure[tw] OR measures[tw] OR findings[tw] OR result[tw] OR results[tw] OR test[tw] OR tests[tw])) OR generaliza*[tiab] OR generalisa*[tiab] OR concordance[tiab] OR (intraclass[tiab] AND correlation*[tiab]) OR discriminative[tiab] OR “known group”[tiab] OR “factor analysis”[tiab] OR “factor analyses”[tiab] OR “factor structure”[tiab] OR “factor structure”[tiab] OR dimensionality[tiab] OR subscale*[tiab] OR “multitrait scaling analysis”[tiab] OR “multitrait scaling analyses”[tiab] OR “item discriminant”[tiab] OR “interscale correlation”[tiab] OR “interscale correlations”[tiab] OR ((error[tiab] OR errors[tiab]) AND (measure*[tiab] OR correlat*[tiab] OR evaluat*[tiab] OR accuracy[tiab] OR accurate[tiab] OR precision[tiab] OR mean[tiab])) OR “individual variability” [tiab] OR “interval variability”[tiab] OR “rate variability”[tiab] OR “variability analysis”[tiab] OR (uncertainty[tiab] AND (measurement[tiab] OR measuring[tiab])) OR “standard error of measurement”[tiab] OR sensitiv*[tiab] OR responsive*[tiab] OR (limit[tiab] AND detection[tiab]) OR “minimal detectable concentration”[tiab] OR interpretab*[tiab] OR (small*[tiab] AND (real[tiab] OR detectable[tiab]) AND (change[tiab] OR difference[tiab])) OR “meaningful change’[tiab] OR “minimal important change’[tiab] OR “minimal important difference’[tiab] OR “minimally important change’[tiab] OR “minimally important difference”[tiab] OR “minimal detectable change”[tiab] OR “minimal detectable difference”[tiab] OR “minimally detectable change”[tiab] OR “minimally detectable difference”[tiab] OR “minimal real change”[tiab] OR “minimal real difference”[tiab] OR “minimally real change”[tiab] OR “minimally real difference’”[tiab] OR “ceiling effect”[tiab] OR “floor effect”[tiab] OR “Item response model”[tiab] OR IRT[tiab] OR Rasch[tiab] OR “Differential item functioning”[tiab] OR DIF [tiab] OR “computer adaptive testing”[tiab] OR “item bank”[tiab] OR “cross-cultural equivalence”[tiab]
4	#1 AND #2 AND #3
5	(“protocol”[ti] OR “addresses”[Publication Type] OR “biography”[Publication Type] OR “case reports”[Publication Type] OR “comment”[Publication Type] OR “directory” [Publication Type] OR “editorial”[Publication Type] OR “festschrift”[Publication Type] OR “interview”[Publication Type] OR “lectures”[Publication Type] OR “legal cases”[Publication Type] OR “legislation”[Publication Type] OR “letter”[Publication Type] OR “news”[Publication Type] OR “newspaper article”[Publication Type] OR “patient education handout”[Publication Type] OR “popular works”[Publication Type] OR “congresses”[Publication Type] OR “consensus development conference”[Publication Type] OR “consensus development conference”[Publication Type] OR “practice guideline”[Publication Type])
6	#4 NOT #5
7	FILTER: Article Type (Review or Systematic Review)
8	FILTER: Subject (Systematic Review)
9	FILTER: Language (English)
10	FILTER: Period (2008–2018)

Inclusion criteria

Time frame

We will limit our search to studies published between 2008 and 2018.

Study design

Systematic reviews aiming to report or to assess the measurement properties of instruments evaluating the quality of life within the context of health and disease, namely HRQoL instruments,⁴ including all studies examining at least two or more measurement properties of a HRQoL instrument. Systematic reviews were required to include the full results report and detailed information about the instruments used to assess the measurement properties.

Setting and participants

We will include the whole range of ages (new borns, toddlers, children, teenagers, young adults, middle age adults and elderly people), in any healthcare setting.

Condition or domain being studied

The quality of health status and the quality of life instruments are essential to obtain accurate diagnoses and to assess the efficacy or effectiveness of a specific intervention in healthcare. Evaluating and improving the quality of life, as well, is considered a public health priority,⁴ and because of this the present meta-review is focused on systematic reviews that appraised the measurement properties of HRQoL instruments.

Context

To study the characteristics of tools assessing the measurement properties of HRQoL instruments in systematic reviews and to compare the measurement standards on which these tools were developed or conform to, with examples found in Viladrich and Doval⁹: attributes and criteria to assess health status and quality of life instruments,^{10 11} the standards for educational and psychological measurement^{12 13} or the health status measures in economic evaluation.^{14 15}

Primary outcomes

Identification of the main specific tools applied to assess the measurement properties of HRQoL instruments and comparison of their most relevant characteristics. Identification and comparison of the measurement standards on which these tools were developed. Appraisal of how authors of the systematic reviews include the assessment of the quality of the HRQoL instruments in their results and how they use this evaluation to come to an overall conclusion regarding the quality of each instrument.

Instruments

We will include tools aiming to assess the quality of measurement properties of HRQoL instruments.

Study screening

References identified by the search strategy will be entered into Mendeley bibliographic software, and duplicates will be removed. Titles and abstracts will be screened independently by two reviewers. When decisions are unable to be made from title and abstract alone, the full paper will be retrieved. Full text inclusion criteria will be checked independently by two reviewers. Discrepancies during the process will be resolved through discussion (with a third reviewer where necessary).

Data extraction

Extracted information of each selected systematic review and meta-analysis will include: general information (author, year, country of origin and papers, theoretical/conceptual framework); tools applied to assess the measurement properties of HRQoL instruments (title, purpose/use, number of items, response categories, criteria to assess the measurement properties on specific measurement standard, ease and usefulness of interpretation, level of expertise required for scoring and interpreting and time required to completion); reporting of the measurement properties assessed and use of the results from the evaluation of the measurement properties to come to an overall conclusion regarding the quality of each HRQoL instruments. Authors of eligible studies will be contacted to provide missing or additional data if necessary.

Strategy for data analysis

We will initially categorise the tools applied to assess the measurement properties of the HRQoL instruments according to the measurement standards on they were developed or conform to. Next, we will detail the most relevant characteristics of these tools according to their measurement standards and their conceptual frameworks.

Strategy for data synthesis

Descriptive analyses of different aspects of the identified tools applied to evaluate the measurement properties of HRQoL instruments. The extracted information related to these tools will be reported in a table to facilitate their comparison. Some recommendations about the methodological, practical and the research applications of each tool will be made.

Patient and public involvement

No patient or public involvement.

Ethics and dissemination

Ethical approval is not necessary for systematic review protocols. The results will be disseminated by its publication in a peer-reviewed journal and presented at a relevant conference.

Discussion

To date, there are not meta-reviews of tools assessing the measurement properties of HRQoL instruments and the different measurement standards on which these tools were developed. The findings of this work will be useful, first, to compare the minimum criteria and attributes recommended to assess the measurement properties of HRQoL; second, to establish the most relevant differences and similarities among both the measurement standards and the assessment tools of measurement properties and finally, to discuss the methodological, research and practical applications of these tools in systematic reviews. This information will facilitate and improve the work of researchers and clinicians that conduct systematic reviews of HRQoL instruments measurement properties.