Patient‐reported outcome instruments that evaluate adherence behaviours in adults with asthma: a systematic review of measurement properties

Abstract

Aim

The aim of the present study was to identify systematically the measurement properties of patient‐reported outcome instruments (PROs) that evaluate adherence to inhaled maintenance medication in adults with asthma.

Methods

We conducted a systematic review of six databases. Two reviewers independently included studies on the measurement properties of PROs that evaluated adherence in asthmatic participants aged ≥18 years. Based on the COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN), the reviewers: (i) extracted data on internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross‐cultural validity, criterion validity and responsiveness; (ii) assessed the methodological quality of the included studies; (iii) assessed the quality of the measurement properties (positive or negative); and (iv) summarized the level of evidence (limited, moderate or strong).

Results

We screened 6068 records and included 15 studies (14 PROs). No studies evaluated measurement error or responsiveness. Based on methodological and measurement property quality assessments, we found limited positive evidence of: (i) internal consistency of the Adherence Questionnaire, Refined Medication Adherence Reason (MAR) scale, Medication Adherence Report Scale for Asthma (MARS‐A) and Test of the Adherence to Inhalers (TAI); (ii) reliability of the TAI; and (iii) structural validity of the adherence questionnaire, MAR scale, MARS‐A and TAI. We also found limited negative evidence of: (i) hypotheses testing of the Adherence Questionnaire; (ii) reliability of the MARS‐A; and (iii) criterion validity of the MARS‐A and TAI.

Conclusions

Our results highlighted the need to conduct further high‐quality studies to evaluate the reliability, validity and responsiveness of the available PROs.

What is Already Known About this Subject

• Proper measurement of adherence to inhaledmaintenance asthma medication is crucial both in routine care and research settings.

• Patient‐reported outcome instruments (PROs) are simple, timely and inexpensive tools that healthcare professionals can use on a daily basis. They are also widely used in research.

• The COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) have been developed to help healthcare professionals and researchers select the best available PRO for a certain purpose, such asmeasuring adherence behaviours, or to identify PROs that need further validation.

What this Study Adds

• In this systematic review, we report the measurement properties of 14 PROs that have been validated to assess adherence to inhaled maintenance medications in adults with asthma.

• Our results suggested that, for each of the PROs, there was a combination of positive, negative and unknown evidence in regard to their reliability and validity, and none of the studies assessed the responsiveness of any of the available PROs.

• Our results highlighted the need to conduct further high‐quality studies that will aim to evaluate the reliability, validity, responsiveness and interpretability of the available PROs.

Introduction

Asthma is a chronic respiratory disease that affects 300 million people worldwide 1. It is associated with a high clinical and economic burden 2, 3, 4. To optimize symptom control and reduce the risk of future exacerbations, the Global Initiative for Asthma (GINA) report has recommended doctors to prescribe inhaled corticosteroids (ICS), either alone or in combination with long‐acting β₂‐agonists (LABAs), as the preferred maintenance therapy in adults with asthma 5. The use of inhaled maintenance medication is particularly beneficial to individuals diagnosed with asthma because it enhances patients' quality of life and lung function, and reduces symptoms and exacerbations, when compared with a placebo medication 6.

Despite clear recommendations to use ICS or ICS ‐LABA combinations, many patients still underuse these pharmacological treatments 7, 8, 9, 10. As a result, non‐adherence to asthma inhaled maintenance medication has been associated with poor asthma control 7, 9, and increased hospitalizations 7, 10 and healthcare costs 7, 9.

Based on a stepwise approach to optimize asthma control, the GINA report has suggested that doctors should first assess adherence to medication and then escalate pharmacotherapy in individuals with uncontrolled asthma who are adherent to their prescribed treatment 5. Hence, it is crucial that healthcare professionals (HCPs) adequately measure adherence to asthma maintenance medication in routine care. In this regard, objective measures of adherence, such as rates of prescription refills and electronic monitoring, have been promoted 11, although more simple, timely and inexpensive tools, such as patient‐reported outcome instruments (PROs), may be more suitable for daily use 12.

The COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) have been developed to help HCPs and researchers select the best available PRO for a certain purpose, such as measuring adherence behaviours or to identify PROs that need further validation 13. Based on a Delphi survey of 43 epidemiology, statistics, psychology and clinical medicine experts, the COSMIN group has reached international consensus on the definitions of seven measurement properties for PROs 14. These definitions are presented in Table 1 and fully explained in a book (see de Vet et al. 15). In addition, a checklist for assessment of the methodological quality of studies that aim to evaluate the measurement properties of any PRO has been published (see Mokkink et al. 16 and Mokkink et al. 17). Criteria for rating the quality of the measurement properties are also available (see Mokkink et al. 16 and Kotecha et al. 18).

Table 1.

Definitions

		Definition	COSMIN standardsa
Measurement property
Reliability domain
Internal consistency		‘The extent to which scores for patients who have not changed are the same for repeated measurement under several conditions, for example: using different sets of items from the same [PRO]’ 14	Cronbach's α
Reliability		‘The extent to which scores for patients who have not changed are the same for repeated measurement under several conditions, for example: […] over time, by different persons on the same occasion, or by the same persons (i.e. raters or responders) on different occasions’ 14	ICC
Measurement error		‘The systematic and random error of a patient's score that is not attributed to true changes in the construct to be measured’ 14	LoA
Validity domain
Content validity		‘The degree to which (the items of) a [PRO] indeed looks as though it is an adequate reflection of the construct to be measured’ 14	PCA
Construct validity
Structural validity		‘The degree to which the scores of a [PRO] are an adequate reflection of the dimensionality of the construct to be measured’ 14	EFA or CFA
Hypotheses testing		‘The degree to which the scores of a [PRO] are consistent with hypotheses (for instance with regard to internal relationships, relationships to scores of other instruments or differences between relevant groups) based on the assumption that the [PRO] validly measures the construct to be measured’ 14	Correlation coefficient
Cross‐cultural validity		‘The degree to which the performance of the items on a translated or culturally adapted [PRO] is an adequate reflection of the performance of the items of the original version of the HR‐PRO instrument’ 14.	CFA
Criterion validity		‘The degree to which the scores of a [PRO] are an adequate reflection of a ‘gold standard’ 14	Correlation coefficient
Responsiveness domain
Responsiveness		‘The ability of a [PRO] to detect change over time in the construct to be measured’ 14. Responsiveness is ‘an aspect of validity. In fact, the only difference between validity and responsiveness is that validity refers to the validity of a single score (estimated on the basis of one measurement), and responsiveness refers to the validity of a change score (estimated on the basis of two measurements)’ 15	Correlation coefficient
Other property of PROs
Generalizability		‘Description of the sample in which the measurement properties of the PRO were evaluated’ 17	Mean age, distribution of gender, disease characteristics, country, etc.
Interpretability		‘The degree to which one can assign qualitative meaning – that is, clinical or commonly understood connotations to an instrument's quantitative scores or change in scores. Interpretability is not considered a measurement property but an important characteristic of a measurement instrument’ 14	MCIC

The definitions are quoted from Mokkink et al. 14, de Vet et al. 15, and Mokkink et al. 17.

CFA, confirmatory factor analysis; COSMIN, COnsensus‐based Standards for the selection of health Measurement INstruments; EFA, exploratory factor analysis; ICC, intraclass correlation coefficient; LoA, limits of agreement; MCIC, minimal clinically important change; PCA, principal component analysis; PRO, patient‐reported outcome

^aHere, we only reprint examples of the COSMIN standards that stand for continuous outcomes

To the best of our knowledge, no systematic review has yet been conducted to find out the measurement properties of PROs designed to evaluate adherence behaviours in adults with asthma. Given that poor health outcomes have been associated with patient non‐adherence to treatment 7, 9, 10 and that HCPs are required to determine whether uncontrolled asthma is due to non‐adherence to inhaled maintenance medication or difficult‐to‐control asthma 5, we aimed, in the present study, to identify systematically the measurement properties of PROs used for assessing adherence to inhaled maintenance medication in adults with asthma.

Methods

Design

We conducted a systematic review of measurement properties in accordance with the COSMIN methodology 14, 15, 17, 19.

Search methods to identify studies

We searched for any study that aimed to evaluate the measurement properties of any PRO that was reported but not restricted to be used for measurement of adherence to inhaled maintenance medication in adults with asthma. By ‘adherence’, we referred to the following adherence behaviours: initiation, implementation and persistence, according the Ascertaining Barriers to Compliance Taxonomy for Medication Adherence 20, 21.

We searched the following six databases: CINAHL, Embase, Medline, PsycINFO, the Cochrane Library and Web of Science.

With the assistance of an experienced librarian (Scientific Library, Laval University, Quebec City, QC, Canada), we designed two electronic search strategies that denoted: (i) adherence and (ii) asthma. Additionally, we used two published search strategies designed to denote: (i) PROs and (ii) measurement properties (see Terwee et al. 22 and Mokkink et al. 16). These four strategies were based on text and thesaurus words. They were enriched using synonyms, related terms, variant spellings and truncated words. For each database, a specific syntax was used. Detailed search strategies are listed in Table S1 (see online for supporting information).

We searched the six databases from inception up until March 2017. We ran the first two search strategies in December 2013, as part of a scoping review that aimed to identify all available PROs used to measure initiation of, implementation of and persistence with inhaled maintenance medication in adults with asthma 23. To include up‐to‐date studies, we launched these four search strategies in March 2017.

Selection of studies

Search results were exported to Thomson Reuters® ENDNOTE®, and one reviewer (M.G.) was responsible for identifying and eliminating duplicates. Two reviewers (M.G., N.P.) independently read titles and abstracts for identification of potentially relevant studies. The reviewers then identified the eligible reports, based on the full‐text articles.

We included articles that fulfilled the following criteria:

1. Target condition (construct of interest): adherence (initiation, implementation and persistence) to asthma inhaled maintenance medication.

2. Target population (population of interest): adults with asthma.

3. Index test (type of instrument): any PRO.

4. Study design: studies that reported the evaluation of at least one measurement property, as defined by the COSMIN group 14.

5. Type of publication: original study.

Articles that were written in English, French or Spanish were included because at least two members of our team could read each of those languages. Reviewers' disagreements were resolved by consensus.

Data extraction and management

Based on the COSMIN definitions 14, 15, the reviewers extracted data on seven measurement properties:

1. Internal consistency.

2. Reliability.

3. Measurement error.

4. Content validity.

5. Construct validity, including: (a) structural validity, (b) hypotheses testing, and (c) cross‐cultural validity.

6. Criterion validity.

7. Responsiveness – that is: comparison of change in scores in the PRO under study with change in scores in a gold standard instrument or in a comparator instrument.

Data were also gathered on generalizability and interpretability, including: (i) total score distribution; (ii) minimal (clinically) important change; and (iii) mean ± standard deviation (SD) for different normative groups or subgroups.

Reviewers' disagreements were resolved either by consensus or discussion with a third author (J.M.).

Data analysis and synthesis

Using the COSMIN checklist 17, 19, the two reviewers assessed the methodological quality of each of the included studies, which was rated either as excellent, good, fair or poor. The quality of the measurement properties was rated as positive, negative or undetermined, according to the criteria used by Kotecha et al. 18. Reviewers' disagreements were resolved either by consensus or discussion with a third author (J.M.).

We used the COSMIN criteria that were reported by Kotecha et al. 18 to synthesize, for each PRO, the results of methodological and outcome quality assessments and to determine the level of evidence as strong, moderate, limited or unknown.

Results

Study selection

Figure 1 illustrates the flow of information through the different phases of the systematic review. Overall, our electronic searches resulted in 6073 records. Of these, 659 were retrieved for full‐text screening. At the end, 15 studies met our inclusion criteria. These studies were published between 1994 and 2017. Fourteen PROs were identified: (i) the Adult Asthma Adherence Questionnaire (AAAQ) 24; (ii) the Adherence Questionnaire 25; (iii) the Asthma Diary 26; (iv) the 20‐item Adherence Starts with Knowledge (ASK‐20) 27; derived from the Self‐Reported Medication‐Taking Scale 28 (SRMTS, also referred to as the Morisky ‐Green): (v) the Inhaler Adherence Scale (IAS) 29, 30; (vi) the 15‐item Medication Adherence Reasons (MAR) Scale) 31; (vii) the 20‐item or refined MAR scale 32; (viii) the Medication Adherence Report Scale for Asthma (MARS‐A) 33, 34; also derived from the SRMTS 28: (ix) the Medication Adherence Scale (MAS) 30; (x) the Medication Intake Survey–Asthma (MIS‐A) 35; (xi) the Patterns of Asthma Medication Use Questionnaire (PAMUQ) 36; (xii) the Questions of Interest (QIs) 37; (xiii) an unnamed PRO 38; and (xiv) the Test of the Adherence to Inhalers (TAI) 39.

Figure 1.

Flow of information throughout the review, based on the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) statement 40. (A). Initial search (from inception to 2013). (B) Up‐to‐date search (2013–2017)

Measurement properties

cThe measurement properties that were assessed in each study are summarized in Table 2. The methodological quality of each of the included studies and the quality of each of the measurement properties are presented in Table 3. The levels of evidence for each measurement properties are shown in Table 4.

Table 2.

Measurement properties assessed, according to the COSMIN definitions 14, 15

	Reliability domain			Validity domain
	Internal consistency	Reliability	Measurement error	Content validity	Construct validity			Criterion validity	Responsiveness	Interpretability
PRO Reference					Structural validity	Hypotheses testing	Cross‐cultural validity			Total score distribution	MCIC	Mean ± SD for normative groups
AAAQ 24	×				×	×		×
Adherence Questionnaire 25	×				×	×				×
Asthma Diary 26								×
ASK‐20 27	×					×
IAS 29	×					×				×
IAS 30	×					×				×
MAR scale (15 items) 31	×				×			×		×
Refined MAR scale (20 items) 32	×			×	×	×		×		×
MARS‐A 33					×
MARS‐A 34	×	×			×	×	×	×		×
MAS 30	×					×				×
MIS‐A 35						×				×
PAMUQ 36				×				×		×
QIs 37				×				×		×
Unnamed PRO 38								×		×
TAI (patient domain) 39	×	×			×			×		×

×: Measurement property assessed, according to the COSMIN definitions 14, 15. AAAQ, Adult Asthma Adherence Questionnaire; ASK‐20, 20‐item Adherence Starts with Knowledge; COSMIN, COnsensus‐based Standards for the selection of health Measurement INstruments; IAS, Inhaler Adherence Scale; MAR, Medication Adherence Reasons; MARS‐A, Medication Adherence Report Scale for Asthma; MAS, Medication Adherence Scale; MCIC, minimal (clinically) important change; MIS‐A, Medication Intake Survey–Asthma; PAMUQ, Patterns of Asthma Medication Use Questionnaire; PRO, patient‐reported outcome; QIs, Questions of Interest; SD, standard deviation; TAI, Test of the Adherence to Inhalers

Table 3.

Measurement properties and methodological quality study assessments

	Reliability domain		Validity domain
	Internal consistency	Reliability	Content validity	Construct validity
PRO Reference				Structural validity	Hypotheses testing	Cross‐cultural validity	Criterion validity
AAAQ 24	O: N/Ac			O: Undetermined	O: Negative		O: Undetermined
							S: Fair
	S: Fair
				S: Fair	S: Poor
Adherence Questionnaire 25	O: Positive			O: Positive	O: Negative
	S: Fair			S: Fair	S: Fair
Asthma Diary 26							O: Negative
							S: Fair
ASK‐20 27	O: Undetermined				O: Positive
	S: Poor				S: Poor
IAS 29	O: Undetermined				O: Negative
	S: Poor				S: Poor
IAS 30	O: Undetermined				O: Uncleard
	S: Poor				S: Poor
MAR scale (15 items) 31	O: Negative			O: Positive			O: Undetermined
							S: Poor
	S: Fair			S: Fair
Refined MAR scale (20 items) 32	O: Positive		O: Undetermined	O: Positive	O: Undetermined		O: Undetermined
							S: Poor
	S: Fair		S: Uncleard	S: Poor	S: Fair
MARS‐A 33				O: Undetermined
				S: Good
MARS‐A 34	O: Positive	O: Negative		O: Positive	O: Positive	O: Undetermined	O: Negative
				S: Fair	S: Poor	S: Poor	S: Fair
	S: Fair
		S: Fair
MAS 30	O: Undetermined				O: Uncleard
	S: Poor				S: Poor
MIS‐A 35					O: Negative
					S: Fair
PAMUQ 36			O: Undetermined				O: Undetermined
							S: Poor
			S: Poor
QIs 37			O: Undetermined				O: Undetermined
							S: Fair
			S: Poor
Unnamed PRO 38							O: Uncleard
							S: Fair
TAI (patient domain) 39	O: Positive	O: Positive		O: Positive			Negative
	S: Fair			S: Fair			Fair
		S: Fair

The quality of each measurement property that was assessed was rated as positive, negative or undetermined.a The methodological quality of the included studies were rated as excellent, good, fair or poor.b AAAQ, Adult Asthma Adherence Questionnaire; ASK‐20, 20‐item Adherence Starts with Knowledge; IAS, Inhaler Adherence Scale; MAR, Medication Adherence Reasons; MARS‐A, Medication Adherence Report Scale for Asthma; MAS, Medication Adherence Scale; MIS‐A, Medication Intake Survey–Asthma; N/A, not applicable; O, Quality of the outcome – that is, the measurement property; PAMUQ, Patterns of Asthma Medication Use Questionnaire; PRO, patient‐reported outcome; Qis, Questions of Interest; S, methodological quality of the study; TAI, Test of the Adherence to Inhalers

^aWe used the outcome quality criteria reported by Kotecha et al. 18

^bWe used the COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) methodological quality criteria 17, 19

^cExploratory factor analysis was performed but authors subsequently decided that the five items of the AAAQ were not part of a single factor, as hypothesized 24

^dUnclear: not enough data were available

Table 4.

Level of evidence for each measurement property

	Reliability domain		Validity domain
	Internal consistency	Reliability	Content validity	Construct validity
PRO Reference				Structural validity	Hypotheses testing	Cross‐cultural validity	Criterion validity
AAAQ 24	N/Aa			N/R	?		N/R
Adherence Questionnaire 25	+			+	−
Asthma Diary 26							−
ASK‐20 27	N/R				?
IAS 39	N/R				?
IAS 30	N/R				N/R
MAR scale (15 items) 31	−			+			N/R
Refined MAR scale (20 items) 32	+		N/R	?	N/R		N/R
MARS‐A 33				N/R
MARS‐A 34	+	−		+	?	N/R	−
MAS 30	N/R				N/R
MIS‐A 35					−
PAMUQ 36			N/R				N/R
QIs 37			N/R				N/R
Unnamed PRO 38							N/R
TAI (patient domain) 39	+	+		+			−

‘+ + +’ or ‘– – –’, strong positive or negative evidence; ‘+ +’ or ‘– –’, moderate positive or negative evidence; ‘ +’ or ‘ –’, limited positive or negative evidence; ‘ +/−’, conflicting findings; ‘ ?’, unknown, due to poor methodological quality; N/R, not rated owing to insufficient data on outcome quality.

AAAQ, Adult Asthma Adherence Questionnaire; ASK‐20, 20‐item Adherence Starts with Knowledge; IAS, Inhaler Adherence Scale; MAR, Medication Adherence Reasons; MARS‐A, Medication Adherence Report Scale for Asthma; MAS, Medication Adherence Scale; MIS‐A: Medication Intake Survey ‐Asthma; N/A, not applicable; PAMUQ: Patterns of Asthma Medication Use Questionnaire; PRO, patient‐reported outcome; QIs, Questions of Interest; TAI, Test of the Adherence to Inhalers

We used the COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) criteria for data synthesis that were reported by Kotecha et al. 18

^aExploratory factor analysis was performed but authors subsequently decided that the five items of the AAAQ were not part of a single factor, as hypothesized 24

Internal consistency

The internal consistency was assessed, using Cronbach's alphas, in nine studies of either fair or poor methodological quality 24, 25, 27, 29, 30, 31, 32, 34, 39. Methodological quality was lowered because there was no description of how missing items were handled or because no factor analysis was used to check the unidimensionality of the scales. In four studies of fair methodological quality 25, 32, 34, 39, Cronbach's alphas were ≥0.70, which denoted a positive rating. These results suggested a limited level of evidence of the internal consistency of the Adherence Questionnaire 25, the refined MAR scale 32, the MARS‐A 34 and the TAI 39.

Reliability

The reliability of the MARS‐A and the TAI was measured in two studies of fair methodological quality 34, 39. The intraclass correlation coefficient calculated by Plaza et al. 39 was ≥0.70, which reflected a positive rating. However, the methodological quality of the study suggested only a limited level of evidence of the reliability of the TAI, owing to the lack of description of how missing items were handled.

Measurement error

No study evaluated the measurement error of any of the included PROs.

Content validity

Three studies reported assessment of content validity 32, 36, 37. Given that there was not enough information available on what was relevant for adherence measurement or on whether the PROs were comprehensive, neither the methodological quality of these studies nor the quality of this measurement property could be rated.

Construct validity: (i) structural validity, (ii) hypotheses testing and (iii) cross‐cultural validity

The structural validity of the AAAQ 24, the Adherence Questionnaire 25, the MAR scale 31, the refined MAR scale 32, the MARS‐A 33, 34 and the TAI 39 was reported. In four studies of fair methodological quality 25, 31, 34, 39, factor(s) explained at least 50% of the variance, which suggested a limited level of evidence regarding the structural validity of the Adherence Questionnaire 25, the MAR scale 31, the MARS‐A 34 and the TAI 39.

Eight studies 24, 25, 27, 29, 30, 32, 34, 35 relied on hypotheses testing to assess the construct validity of their PRO. The methodological quality of five 24, 27, 29, 30, 34 of these eight studies was rated as poor, mainly because the hypotheses tested were not formulated a priori.

The cross‐cultural validity of the MARS‐A, translated from English into Spanish, was assessed by Cohen et al. 34. The methodological quality of this study was rated as poor, given that no confirmatory factor analysis was performed.

Criterion validity

Nine studies 24, 26, 31, 32, 34, 36, 37, 38, 39 reported assessing the criterion validity of their PRO. Three studies of fair methodological quality 26, 34, 39 relied on electronic monitoring as gold standards. In these three studies, correlations with electronic monitoring were <0.70, which reflected a negative rating.

Responsiveness

None of the 15 included studies assessed the responsiveness of any of the 14 PROs, as defined by the COSMIN group; none of the included studies compared change in scores in their PRO with change in scores in a gold standard instrument or in a comparator instrument.

Other properties of PROs

Generalizability and interpretability

The characteristics of study populations are presented in Table 5. Most of the included studies were conducted in North America (n = 9) or Europe (n = 4).

Table 5.

Characteristics of the study populations

PRO Reference	Year of publication	Country	N a	Age, yearsb	Gender, women b	Asthma severityb	Duration of asthma, yearsb
AAAQ 30	2013	USA	420	42 ± 9	280 (66.7%)	–	–
Adherence Questionnaire 27	2016	Sweden	104	49 ± 14	65 (62%)	–	–
Asthma Diary 31	1998	USA	55	50 ± 16	36 (66%)	Moderate: 41 (74%)	20 ± 17
						Severe: 14 (26%)
ASK‐20 32	2017	Japan	290	58 ± 16	181 (62%)	Mild: 114 (39%)	–
						Moderate: 106 (37%)
						Severe: 70 (24%)
IAS 33	2001	USA	100	44 ± 14	65 (65%)	Mild: 1 (1%)	22 ± 17
						Moderate: 69 (69%)
						Severe: 30 (30%)
IAS/MAS 34	1994	USA	495	Unclear	Unclear	Unclear	–
MAR scale, 15 items 35	2015	USA	399	49	245 (61%)	–	–
Refined MAR scale, 20 items 28	2014	USA	80	53 ± 16	45(56%)	–	–
MARS‐A 36	2011	USA	294	48 [20–87]	241 (82%)	–	–
MARS‐A 26	2009	USA	318	48 ± 13	264 (83%)	–	Age at onset ≤20 years: 162 (51%)
MIS‐A 37	2017	UK, France	683	Unclear	426 (47.2%)	–	–
PAMUQ 38	2005	UK	185	42 ± 11	134 (66%)	–	–
QIs 39	2004	Canada	70	50 ± 16	39 (56%)	–	15 ± 13
Unnamed PRO 33	2013	New Zealand	111	47 ± 12	61 (55%)	–	–
TAI (patient domain) 24	2016	Spain	599	50 ± 17	397 (66%)	–	16 ± 12

‘–’, not reported. AAAQ, Adult Asthma Adherence Questionnaire; ASK‐20, 20‐item Adherence Starts with Knowledge; IAS, Inhaler Adherence Scale; MAR, Medication Adherence Reasons; MARS‐A, Medication Adherence Report Scale for Asthma; MAS, Medication Adherence Scale; MIS‐A, Medication Intake Survey–Asthma; PAMUQ, Patterns of Asthma Medication Use Questionnaire; PRO, patient‐reported outcome; Qis, Questions of Interest; TAI, Test of the Adherence to Inhalers

^aCorresponds to the number of adults with asthma

^bData are reported as mean, mean ± standard deviation, mean [range], or n (%)

Even though data on the distribution of adherence scores was reported as mean ± SD or n (%) in 12 out of the 15 included studies, no information on minimal (clinically) important change, or means ± SDs for different normative groups or subgroups was found in any of the included studies.

Discussion

Summary of findings

In routine care, doctors may have to escalate pharmacotherapy in patients with uncontrolled asthma who are found to be adherent to their treatment 5. To measure adherence, PROs have been widely used because they are simple, timely and inexpensive tools 12. In order to help HCPs or researchers to select the best available PRO to assess adherence behaviours, in the present systematic review we synthesized the measurement properties of 14 PROs. Our results suggested that, for each of the PROs, there was a combination of positive, negative and unknown evidence in regard to their reliability and validity, and that none of the studies assessed the responsiveness of any of the available PROs, as the COSMIN group defines it. Accordingly, our results highlighted the need to conduct further validation studies. These results led us to make five observations.

First, more than half of the studies included in the present systematic review assessed the internal consistency of their PRO, along with construct or criterion validity. Although there was a limited positive evidence of the internal consistency for some PROs, positive evidence of construct and criterion validity of each of the included PROs remains to be demonstrated. In this regard, future studies should formulate a priori hypotheses and/or use objective measures of adherence as comparator instruments. Further studies should also focus on demonstrating positive evidence of measurement error, especially in view of the fact that this psychometric property has not been evaluated in any of the included studies.

Secondly, in the present systematic review, no study assessed the responsiveness of any of the included PROs, based on the COSMIN definition 14, 15. Accordingly, there is no evidence that any of these PROs can detect a change in adherence scores over time. Interestingly, in the most recent Cochrane review of randomized controlled trials of interventions to improve adherence with prescribed medication (including but not restricted to asthma), Nieuwlaat et al. 41 reported that even the most efficient interventions did not result in substantial improvements in adherence. They also suggested that many studies might have failed to show between‐group differences owing to inaccurate (self‐reported) adherence measurements 41. When evaluating the impact of any intervention, it is crucial that HCPs and researchers alike rely on an instrument that has been shown to be responsive to change 14, 15. Although many definitions of responsiveness have been proposed, the COSMIN group has stated that the responsiveness of any PRO should be evaluated in a longitudinal study that would aim to compare change in scores in the PRO under study with change in scores in a gold standard instrument or in a comparator instrument 15. Based on the COSMIN definition, responsiveness is an aspect of validity 15. Accordingly, the estimation of a statistically significant change in a PRO scores after the implementation of an intervention could not serve to demonstrate its responsiveness 15. Therefore, we suggest that the COSMIN group improves its communication and dissemination activities, in order to help researchers to implement the methodology that it promotes.

Thirdly, given that studies were conducted in North America, Europe, Japan or New Zealand, the measurement properties of the included PROs can only be generalized to individuals from high‐income countries. Thus, additional studies are needed to assess the reliability, validity and responsiveness of PROs used to measure adherence to inhaled maintenance medication in low‐ and middle‐income countries.

Fourthly, our results suggest that no studies provided HCPs and researchers with commonly understood connotations of the PRO scores. Further high‐quality validation studies should evaluate the minimal clinically important change in score of PROs and describe the means ± SDs for different normative groups or subgroups.

Finally, our results suggested that there was limited positive as well as limited negative evidence of the reliability of the MARS‐A 34, and also conflicting evidence of the validity of the Adherence Questionnaire 25 and the TAI 39. In addition, even though our results showed limited positive evidence of the internal consistency of the refined MAR scale 31, the quality of the content, construct and criterion validity of this PRO could not be determined in the present systematic review, owing to the lack of information available, and, in turn, we were not able to synthesize the evidence in regard to these three measurement properties. Consequently, our results highlighted the need to conduct further high‐quality studies in accordance with the COSMIN methodology 14, 17, 19.

Strengths and limitations

Our systematic review delivered some important messages, as it synthesized the measurement properties of 14 PROs found to measure adherence to inhaled maintenance medication in adults with asthma. Based on a study‐specific checklist used to rate the quality of systematic reviews of measurement properties 22, we acknowledged three study strengths. First, our search strategies were comprehensive as we used a published filter for measurement properties and also because we searched six databases. Secondly, two reviewers performed data extraction and quality assessments independently. Thirdly, we synthesized data on measurement properties.

However, despite these strengths, our systematic review had some limitations. First, as in any systematic review, some studies might have been missed, even though we believe, as advocated above, that our search strategies were comprehensive. Secondly, we did not include PROs when they were used solely to elicit barriers to, or beliefs associated with, adherence. Assessment of such constructs could shed light on the reasons why patients do or do not adhere to their prescribed medication and thus be useful for HCPs in routine care. However, these constructs were beyond the target of the present systematic review.

Conclusions

Our results suggested that there was a combination of positive, negative and unknown evidence in regard to the reliability and validity of the available PROs, and that there was no evidence of the responsiveness of any available instrument. At this point, no recommendation regarding the use of a particular PRO in routine care or in research settings can be provided.

Patient adherence has been shown to be suboptimal in adults with asthma and is associated with high socioeconomic burden 7, 9, 10. To assist HCPs in assessing patient adherence, we recommend that researchers conduct further high‐quality studies to evaluate the reliability, validity, responsiveness and interpretability of their PROs, in accordance with the COSMIN 14, 17, 19. These studies should also be conducted in high‐, middle‐ and low‐income countries.

Findings from an updated version of the present systematic review could inform HCPs and researchers on the most appropriate PRO for measurement of adherence behaviours or advocate for the development and validation of a new instrument. As a result, proper assessment of adherence could help HCPs to understand patients' behaviours better, and researchers better to assess the impact of interventions designed to optimize adherence to beneficial treatment, such as the use of inhaled maintenance medication in adults with asthma.

Competing Interests

Potential conflicts of interest to disclose are: (1) the Knowledge Translation, Education and Prevention Chair in Respiratory and Cardiovascular Health is supported by unrestricted grants from AstraZeneca; (2) the Chair on Adherence to Treatments was supported by unrestricted grants from AstraZeneca, Merck Canada, Sanofi Canada, Pfizer Canada and the Prends soin de toi program. M.G., N.P. and J.M. have no conflict of interest to declare. L.P.B. considers having no conflict of interest, but wishes to declare what can be perceived as potential conflicts of interest. Advisory Boards: GlaxoSmithKline, Novartis. Conferences (honoraria): AstraZeneca, GlaxoSmithKline, Merck, Novartis. Sponsorship for investigator‐generated research: AstraZeneca, GlaxoSmithKline, Merck Frosst, Schering. Sponsorship for research funding for participating in multicenter studies: AllerGen, Altair, Amgen, Asmacure, AstraZeneca, Boehringer‐Ingelheim, Genentech, GlaxoSmithKline, Novartis, Ono Pharma, Pharmaxis, Schering, Wyeth. Support for the production of educational materials: AstraZeneca, GlaxoSmithKline, Merck Frosst, Boehringer‐Ingelheim, Novartis. Organisational: Chair of the Global Initiative for Asthma (GINA) Guidelines Dissemination and Implementation Committee, Knowledge Translation, Education and Prevention Chair in Respiratory and Cardiovascular Health, Member of the Executive Committee of Interasma (Global Asthma Organisation). The authors alone are responsible for the content and writing of this paper.

The work was supported by the Canadian Institutes of Health Research through a Knowledge Translation Canada research project stipend, by the Laval University Knowledge Translation, Education and Prevention Chair in Respiratory and Cardiovascular Health, and the Laval University Chair on Adherence to Treatments. The authors acknowledge Frédéric Bergeron MSI for assistance in designing our electronic searches. They are also grateful to Françoise Proust PhD for commenting on the study protocol, along with Marie‐Ève Boulay MSc and Julie Turmel PhD for revising the manuscript. We thank Erica Pridoehl for editing the English manuscript.

Supporting information

Table S1 Electronic search strategies

Gagné, M. , Boulet, L.‐P. , Pérez, N. , and Moisan, J. (2018) Patient‐reported outcome instruments that evaluate adherence behaviours in adults with asthma: a systematic review of measurement properties. Br J Clin Pharmacol, 84: 1928–1940. 10.1111/bcp.13623.