Protocol for a systematic review and meta-analysis of minimal important differences for generic multiattribute utility instruments

Abstract

Introduction

Generic multiattribute utility instruments (MAUIs) are efficient tools for determining and enumerating health-related quality of life. MAUIs accomplish this by generating health state utilities (HSUs) via algorithms. Minimal important differences (MIDs) assist with the interpretation of HSUs by estimating minimum changes that are clinically significant. The overall goal of the proposed systematic review and meta-analysis is the development of comprehensive guidelines for MID estimation.

Methods and analysis

This protocol defines a systematic review and meta-analysis of MIDs for generic MAUIs. The proposed research will involve a comprehensive investigation of 10 databases (EconLit, IDEAs database, INAHTA database, Medline, PsycINFO, Embase, Emcare, JBIEBP and CINAHL) from 1 June 2022 to 7 June 2022, and will be performed and reported in accordance with several validated guidelines, principally the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The quality of papers, considered for inclusion in the review, will be appraised using the COnsensus-based Standards for the selection of health Measurement INstruments, inter alia.

Narrative analysis will involve identifying the characteristics of MIDs including methods of calculation, sources of heterogeneity, and validation. Meta-analysis will also be conducted. The descriptive element of meta-analysis will involve the generation of I² statistics and Galbraith plots of MID heterogeneity. Together with narrative analysis, this will allow sources of MID heterogeniety to be identified. A multilevel mixed model, estimated via restricted maximum likelihood estimation, will be constructed for the purposes of meta-regression. Meta-regression will attempt to enumerate the effects of sources of heterogeneity on MID estimates. Meta-analysis will be concluded with pooling of MIDs via a linear random-effects model.

Ethics and dissemination

Ethics approval is not required for this review, as it will aggregate data from published literature. Methods of dissemination will include publication in a peer-reviewed journal, as well as presentation at conferences and seminars.

PROSPERO registration number

CRD42021261821.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The systematic review will investigate ten databases (both biomedical and economic) and apply a broad range of search terms, both of which will minimise the risk of study omission.

• Restricted maximum likelihood estimation (REML) was chosen for meta-regression to allow for variability in fixed-effects estimates and degrees of freedom consumption.

• Use of REML will permit superior statistical inference compared with generic maximum likelihood estimation.

• A comprehensive suite of validated guidelines is to be adopted in the systematic review to ensure study quality and limit the potential for bias.

• Due to a lack of consistent terminology, relevant articles may be missed if they have paraphrased ‘minimal important difference’ in an unusual way which is not capture by the systematic review’s search strategy.

Introduction

The following is a protocol for a systematic review and meta-analysis of minimal important differences (MIDs) for generic multiattribute utility instruments.

Multiattribute utility instruments

Multiattribute utility instruments can be generic, and adopted for use with any study population or sample, or be disease or symptom-specific. Multiattribute utility instruments operate by eliciting health states, which are profiles of health-related quality of life measured across several dimensions of health. Multiattribute utility instrument health states are based on arrays of patient-reported outcomes, obtained through instrument-specific surveys.¹

Multiattribute utility instruments surveys function by posing questions about several physical and psychosocial dimensions of health.² These questions require respondents to rank their dimensional health.² Uniquely, the Assessment Quality of Life-8 Dimensions (AQoL-8D)³ generic multiattribute utility instrument coalesces dimensional scores into superdimensional scores, which provides measures of overall physical and mental health. Other common, generic multiattribute utility instruments include the European Quality of Life-5 Dimensions-5 Levels (EQ-5D-5L),⁴ Quality of Wellbeing,⁵ Short Form-6 Dimensions Version 1⁶ and Health Utilities Index Version 3,⁷ which all vary in size and the health dimensions they assess. See table 1 for a list of common, generic multiattribute utility instruments, the dimensions of health they analyse and the number of items (questions) in each.

Table 1.

Health dimensions assessed by eight multiattribute utility instruments, and the number of items in each

Instrument name	Health dimensions assessed		No of items
EQ-5D-5L4	Mobility Self-care Usual activities	Pain/discomfort Anxiety/depression	5
AQoL-8D3	Independent Living Senses Pain Mental health	Happiness Self-Worth Coping Relationships	35
HUI3 (self-administered) 7	Vision Hearing Speech Ambulation	Dexterity Emotion Cognition Pain	15
QWB9	Chronic symptoms Acute symptoms Mental health	Mobility Usual Activity Physical Activity	74
15-D 40	Breathing Mental function Speech (communication) Vision Mobility Usual activities Vitality Hearing	Eating Elimination Sleeping Distress Discomfort and symptoms Sexual Activity Depression	15
SF-6Dv1 41	Physical function Role limitation Social function	Bodily pain Mental health Vitality	6
EQ-5D-5L Psychosocial42	Mobility Self-care Usual activities Pain/discomfort Anxiety/depression	Vitality Sleep Social relationships Community connectedness	9
PROPr Scoring System for the PROMIS 43	Cognitive function Depression Fatigue Pain Interference	Physical function Sleep disturbance Social roles and activities	Variable

AQoL-8D, Assessment Quality of Life-8 Dimensions; 15-D, 15-Dimension; EQ-5D-5L, European Quality of Life-5 Dimensions-5 Levels; HUI3, Health Utilities Index Version 3; PROMIS, Patient-Reported Outcome Measurement Information System; PROPr, PROMIS Preference; QWB, Quality of Wellbeing; SF-6Dv1, Short Form-6 Dimensions Version 1.

Each health state, generatable by a multiattribute utility instrument via its survey, has an associated health state utility, which is a discrete, ordinal ranking of health-related quality of life.⁸ These health state utilities are assigned to health states using a variety of experimental economics techniques including standard gambles, visual analogue scales, discrete choice experiments, and time trade-offs.⁹ Health state utilities are best defined as representing the position of a person’s health state on a death (0) to full health (1) continuum, relative to the positions of all other possible health states. The representation of health state utilities as a pseudocontinuous measure is facilitated by the large number of health states identifiable by multiattribute utility instruments. For example, the AQoL-8D can generate 2.4×10²³ discrete health states.³ This attribute also allows the magnitude of difference between health state utilities to bear comparative significance, adding an element of cardinality to an otherwise ordinal measure. Health state utilities are frequently applied in cost-utility analyses (a type of comprehensive health economic analysis, used to evaluate medical interventions), clinical assessments, and evaluations of patient-reported outcomes.^{1 4} In figure 1, the function of the EQ-5D-5L is presented to exemplify the operation of a generic multiattribute utility instrument.

Figure 1.

This figure illustrates the function of the EQ-5D-5L multiattribute utility instrument. The first element of the process involves obtaining participant responses to the relevant multiattribute utility instrument survey. In the case of the EQ-5D-5L, participants are required to select one of five ranks for each of the five survey items. These responses are then collated and used to produce a profile of participant health, known as a health state. Finally, the health state utility associated with the participant’s health state is retrieved, usually via an algorithm. EQ-5D-5L, European Quality of Life-5 Dimensions-5 Levels.

Minimal important differences

Although variations in health-related quality of life can be measured using multiattribute utility instruments, these instruments provide no evaluation of what constitutes a clinically significant/meaningful change. Therefore, MIDs are required.¹⁰ These values are the smallest change in health state utility that is statistically significant and represents a meaningful adjustment to patient health-related quality of life.¹¹ MIDs can lack robustness across multiattribute utility instruments and populations.^12–14

MID calculation methods

Major methods of MID estimation are described as distribution-based and anchor-based.¹⁰ Distribution-based methods rely on statistical techniques to develop MIDs. An example of such a method is Cohen’s effect sizes.¹⁵ Cohen’s effect sizes are calculated as $ES=\left(M_2-M_1\right)/S_1$.¹⁵ In this equation, $M_1$ is the average baseline health state utility for a sample of participants. $M_2$ is a health state utility greater than the average baseline health state utility, which represents, comparatively, a superior health state. $S_1$ is the standard error (SE) of the mean, baseline health state utility. Using a classification scale, the output of the equation can be used to classify a change in health state utility as large (not an MID) or small (possibly an MID).¹⁶ Another distribution-based method involves using a fraction of the SE of the mean change in HSU as a MID.³

Anchor-based methods can be subdivided into external and internal anchors. External anchors can involve respondents being separately questioned, following multiattribute utility instrument implementation, regarding whether changes in their health state utility represent meaningful changes in their health.^10–17 They can also involve the use of clinical markers to validate the materiality of variations in health state utility. Contrastingly, internal anchors are instrument defined. They are derived as the difference in attributable health state utilities between two minimally different health states, which are thought to be clinically distinct.¹¹

Other methods of MID calculation include using legacy anchors, triangulation, and the Delphi method. Legacy anchors are MIDs sourced from previous work and are either reapplied to a new study or used to benchmark new MIDs.¹⁸ Triangulation involves the use of both distribution and anchor-based methods to generate a single MID.¹⁹ MID triangulation is intended to provide increased internal validity to MID estimates.¹⁹ Lastly, the Delphi method involves establishing MIDs by consensus.

Gaps in the literature

No study has been conducted which is a specific and systematic review and meta-analysis of MIDs for generic multiattribute utility instruments. Due to this evidence gap, there are also no guidelines regarding MID estimation for generic multiattribute utility instruments which are validated by a systematic review and meta-analysis. Existing literature has either reviewed MIDs for multiattribute utility instruments in conjunction with MIDs for disease or symptom-specific instruments^20–23 or focused on MIDs relevant to a particular intervention or disorder.^{24 25} Studies applicable to the former category have often been limited in scope, searching few databases.^{20 22} Other such studies had different aims than guideline construction, such as highlighting research gaps through systematic review²¹ or establishing an MID repository.²³

Research questions

The proposed systematic review and meta-analysis will address the following research questions regarding MIDs for generic multiattribute utility instruments:

1. How were MIDs calculated?

   • Which methods were applied?

   • Which methods were most commonly used?

   • Were some methods novel and if so in what way?

   • Did different calculation methods produce significantly different MIDs?

2. For what multiattribute utility instruments and diseases were MIDs calculated?

   • Were MIDs consistent across multiattribute utility instruments and diseases?

   • Is variation present in MIDs across iterations using the same, similar, and different study cohorts?

3. Are applied methods of MID estimation theoretically and empirically sound?

   • Were there any mathematical errors or controversial innovations?

   • Were the methods validated?

4. How were MIDs evaluated?

   • What, if any, guidelines were used to evaluate MIDs and were these guidelines validated?

   • What was the result of MID evaluations?

5. What variables, if any, contribute systematically to heterogeneity in MID estimates?

   • Can regression-based evidence be acquired to support relevant associations?

   • If influential variables are controlled for, do MID estimates converge?

   • What level of unexplained heterogeneity exists?

6. Can existing MIDs be applied to new research and under what circumstances?

Aim and rationale

The review aims to generate complete and nuanced guidelines for MIDs for generic multiattribute utility instruments, validated by a systematic review and meta-analysis. Specifically, these guidelines will inform researchers regarding appropriate methods of MID estimation, provide benchmarks against which MIDs may be compared, and expound on potential sources of heterogeneity. Regarding the latter, this will assist researchers in determining the applicability of existing MIDs to new studies and allow benchmark MIDs to have greater comparability to a wider range of MIDs.

Methods and analysis

Patient and public involvement

The was no public or patient involvement, due to the proposed study being a systematic review.

Validated guidelines: protocol and systematic review

This protocol has been developed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols guidelines (PRISMA-P).²⁶ The proposed systematic review will be performed and reported in accordance with the PRISMA guidelines.²⁷ The review will also adhere to the Consolidated Health Economic Evaluation Reporting Standards²⁸ checklist and the Professional Society for Health Economics and Outcomes Research good research practices task force report regarding health state utilities in clinical studies.²⁹

Validated guidelines: quality appraisal and risk of bias assessment for reviewed studies

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology for patient-reported outcome measures assessment checklist will be adapted and applied to evaluate the quality of papers considered for inclusion in the study, as well as their associated risk of bias.^30–33 Any studies found to be at high risk of bias will be weighted in meta-analysis, to reduce their impact on review results. In addition, references from included papers will be screened for relevant articles to identify potential omissions in the systematic review, thereby ensuring quality through completeness.

Validated guidelines: evidence appraisal and risk of bias assessment for the systematic review

To assess the overall risk of bias in the systematic review’s body of evidence, the Risk of BIas assessment tool for Systematic reviews (ROBIS) was selected.³⁴ The ROBIS tool has several domains under which bias may be judged: study eligibility criteria (did the study adhere to predefined eligibility criteria), identification and selection of studies (was every effort made to collect the maximum number of eligible papers), data collection and study appraisal (was potential bias in individual studies assessed and all pertinent data collected), and synthesis and findings (was all available data synthesised appropriately and any potential bias in results made transparent).³⁴ In addition, to evaluate the overall certainty and strength of the body of evidence generated by the systematic review, the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework will be implemented.³⁵

Search methodology

A prestudy, preliminary search for relevant papers was conducted using the PubMed database. This permitted collection of keywords appropriate for use in electronic database searches. A professional librarian was enlisted to assist with this task. Collected terms were grouped based on synonymity, as shown in figure 2.

Figure 2.

Synonymic groupings of search words associated with ‘minimal important difference’ are divided by which element of the phrase they are interchangeable with. From top to bottom, the words are associated with ‘minimal’, ‘important’ and ‘difference’. In addition, ‘(word)’ indicates that singular words from the same category should be added. For example, ‘clinically’ would become ‘clinically important’ and ‘clinically significant’. Words associated with ‘multiattribute utility instrument’ are divided into (top) instrument names (generic and specific) and (bottom) outcome measures associated with multiattribute utility instruments. AQoL-8D, Assessment Quality of Life-8 Dimensions; EQ-5D-5L, European Quality of Life-5 Dimensions-5 Levels; QWB, Quality of Wellbeing; SF-6D, Short Form-6 Dimensions.

The search strategy selected requires one word or phrase from each of the ‘MID’ divisions and a phrase or name from either ‘multiattribute utility instrument’ division to be present in an article’s title and/or abstract for that paper to be considered for inclusion. In addition, search terms will be applied as pluralised (hyphenated) and singular (non-hyphenated) variants. Relevant acronyms are to be applied in searches, as well as their respective expansions. Note that many phrases synonymous with the technical term (MID) are present in the search strategy due to the heterogeneity of their usage and the lack of a firmly established nomenclature.³⁶ See online supplemental appendix for the precise search strategy used in all database searches.

Both economic and biomedical electronic databases will be searched in this review, from 1 June 2022 to 7 June 2022. Economic databases to be investigated are the American Economic Association database (EconLit) via EBSCO, the IDEAs database by Research Papers in Economics (RePEc), and the International Health Technology Assessment Database (INAHTA). Biomedical databases that are to be examined include Medline, via PubMed and Ovid; PsycINFO, Embase, Emcare, and the Joanna Briggs Institute Evidence-Based Practice (JBIEBP) database via Ovid; and the Cumulative Index to Nursing and Allied Health Literature (CINAHL), via EBSCO. In addition, we will also search Health Business Elite via EBSCO, and google scholar will be utilised to maximise the completeness of the review.

Inclusion criteria

This review will include English language papers that incorporate MIDs for generic multiattribute utility instruments that generate health state utilities. Studies with various response rates, sample sizes and MID calculation techniques will be included, without qualification, to ensure comprehensiveness. No study conducted before 1989 will be considered, as MIDs were introduced into the literature in that year.³⁷ Furthermore, only original, published studies will be included; editorials, commentaries, protocols, reviews, unpublished works and meta-analyses are to be excluded. Case, in vitro, and animal studies will also be excluded.

Study screening

The first author (GJH) will collect all articles found using the search strategy. Duplicates will be eliminated, and abstracts sorted, using the Covidence program. GJH and JAC will independently screen accumulated papers through analysis of titles and abstracts, excluding those not meeting the inclusion criteria (detailed in section 2.1). The second round of screening (conducted by GJH and JAC) will examine the full text of the remaining articles, excluding articles that fail to satisfy the inclusion criteria, and determining which articles contain sufficient information to be included in meta-analyses. Where disagreements occur during screening, coauthors will be invited to mediate.

Data extraction

Completeness and quality of data extraction will be controlled using a data extraction form. Adherence to this form will be validated by JAC. Where data are not present in a paper, authors will be contacted. The following will be extracted from included studies:

1. Participant characteristics: age, socioeconomic status, sex, education, the urbanity of residences, health insurance coverage, number of participants, diseases and comorbidities, exposure to socialised medicine, countries of residence, response rate, attrition rate, and medication usage.

2. Publication attributes: first and last author, date, journal, country of origin, type of study, quality, risk of bias, and adherence to validated guidelines.

3. Mathematical features: instrument(s) involved, methods of MID calculation, and approach to MID evaluation.

4. Details of sample selection: exclusion criteria, inclusion criteria, and details of participant recruitment method.

5. Results: MID values, MID SEs, and MID robustness.

6. Key discussions: comparisons to the literature, strengths and limitations, and self and peer appraisals of study MID values.

Note that data will be extracted in a qualitative (as opposed to quantitative) form where necessary.

Data management

As noted in section 2.6, extracted abstracts will be sorted, and duplicates removed, using Covidence. After screening, accumulated data will be stored by the first author (GJH) in Excel spreadsheets and saved on both an institutional cloud and a personal hard drive. The senior author (JAC) will also maintain digital a copy to further ensure data is restorable.

Narrative analysis

Narrative analysis will comprise collation and review of extracted data. For example, methods of MID estimation, frequency of method usage, and context of application will be synthesised into guidelines informing MID application, during this phase. Similar undertakings will occur for other data which does not require further, mathematical analysis. Narrative analysis will also include quality and risk of bias appraisals for included papers.

Meta-analysis

Provided that sufficient data are extracted from studies meeting the inclusion criteria (specifics regarding what comprises sufficient data are currently unknown), meta-analyses and meta-regressions will be performed using Stata V.17 (StataCorp, 2022). Descriptive meta-analysis will consist of generating and analysing summary statistics pertaining to MID heterogeneity (including I² statistics and Galbraith plots) and undertaking subgroup analysis using stratification. Subgroups will consist of MIDs estimated for specific multiattribute utility instruments and diseases, as well as estimated using different techniques. This will facilitate preliminary identification of relationships between MID heterogeneity and study characteristics. Meta-regression will be informed using these results.

Meta-regression

A multilevel mixed model, estimated via restricted maximum likelihood estimation, will be used to evaluate sources of MID heterogeneity while controlling for confounding (sources of which are currently indeterminant) and unexplained heterogeneity. Clustering in the data is hypothesised to arise from methods of MID calculation, and the multiattribute utility instruments that MIDs are estimated for. This hypothesis arises from multiattribute utility instruments using different scales and possessing varying levels of sensitivity. The inclusion of the aforementioned levels in the meta-regression model is contingent on hypothesis confirmation. Further details of model specification will be decided after descriptive analysis and subsequent backward elimination of irrelevant variables.

Restricted maximum likelihood estimation is preferred over iterative maximum likelihood approaches which ignore variability in fixed effects and degrees of freedom consumption, during coefficient estimation.³⁸ Notably, a small sample is expected in the proposed meta-analysis due to the limited number of articles recovered during prestudy, ad hoc database searches. Consequently, disregarded degrees of freedom consumption would likely invalidate statistical inferences in the meta-regression. To maximise the accuracy of statistical inference, restricted maximum likelihood estimation will be paired with the Kenward-Roger small sample correction.³⁹

MID pooling

A linear random effects model will be applied to subsets of MIDs, such as those associated with specific multiattribute utility instruments or diseases. This will facilitate the pooling of MID estimates to create multiattribute utility instrument- and methodology-specific legacy MIDs (or legacy anchors). Combined with knowledge of contributors to MID heterogeneity, these legacy MIDs can be used as standards against which MID estimates may be compared.

Ethics and dissemination

Ethics approval is not required for this systematic review, as it intends to analyse existing works. The primary method of study dissemination will be publication in a peer-reviewed journal. Secondary methods of distribution will include presentations at conferences and seminars.

Ethics statements

Patient consent for publication

Not applicable.