Patient-Reported Outcome Measures for Health-Related Quality of Life in Patients With Acne Vulgaris

Key Points

Question

What are the measurement properties of existing patient-reported outcome measures that assess health-related quality of life in patients with acne?

Findings

In this systematic review of 21 patient-reported outcome measures, 2 measures met standards to be recommended for use in acne clinical studies—the Acne-Q and CompAQ. All measures were lacking data on content validity or some measurement properties.

Meaning

Patient-reported outcome measures used for research or in clinical settings should have sufficient content validity and other measurement properties; these results suggest that while 2 measures can be currently recommended, further understanding of these and other measures will be critical for patient-reported outcome measure use among patients with acne.

This systematic review evaluates the standards of evidence used in patient-reported outcome measures assessing health-related quality of life for patients with acne.

Abstract

Importance

Multiple patient-reported outcome measures (PROMs) for health-related quality of life (HRQoL) exist for patients with acne. However, little is known about the content validity and other measurement properties of these PROMs.

Objective

To systematically review PROMs for HRQoL in adults or adolescents with acne.

Data Sources

Eligible studies were extracted from PubMed and Embase (OVID).

Study Selection

Full-text articles published in English or Spanish on development, pilot, or validation studies for acne-specific, dermatology-specific, or generic HRQoL PROMs were included. Development studies included original development studies, even if not studied in acne patients per Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) recommendations. If a study included several diagnoses, the majority (ie, over 50%) of patients must have acne or acne-specific subgroup analyses must be available. Abstract and full-text screening was performed by 2 independent reviewers.

Data Extraction and Synthesis

Two independent reviewers assessed study quality applying the COSMIN checklist and extracted and analyzed the data. For each distinctive PROM, quality of evidence was graded by measurement property.

Main Outcomes and Measures

PROM properties (target population, domains, recall period, development language), PROM development and pilot studies, content validity (relevance, comprehensiveness, comprehensibility), and remaining measurement properties (structural validity, internal consistency, cross-cultural validity, reliability, measurement error, criterion validity, construct validity, and responsiveness). Quality of evidence was assigned for each measurement property of included PROMs. An overall recommendation level was assigned based on content validity and quality of the evidence of measurement properties.

Results

We identified 54 acne PROM development or validation studies for 10 acne-specific PROMs, 6 dermatology-specific PROMs, and 5 generic PROMs. Few PROMs had studies for responsiveness. The only acne-specific PROMs with sufficient evidence for content validity were the CompAQ and Acne-Q. Based on available evidence, the Acne-Q and CompAQ can be recommended for use in acne clinical studies.

Conclusions and Relevance

Two PROMs can currently be recommended for use in acne clinical studies: the Acne-Q and CompAQ. Evidence on content validity and other measurement properties were lacking for all PROMs; further research investigating the quality of remaining acne-specific, dermatology-specific, and generic HRQoL PROMs is required to recommend their use.

Introduction

Acne is a common inflammatory skin condition with a profound impact on quality of life.¹ Patient-reported outcome measures (PROMs) are a valuable tool to assess health-related quality of life (HRQoL), and their use can complement clinical assessments, such as lesion counts, and global assessments.^2,3 Guidelines from the European Academy of Dermatology and Venerology recommend including PROMs assessing HRQoL in both clinical trials and routine clinical practice.^4,5

As the breadth of available acne-specific, dermatology-specific, and generic HRQoL PROMs continues to grow, heterogeneity in outcome reporting in clinical research may result.^1,6 This heterogeneity can stifle efforts to synthesize research across studies by limiting the ability to conduct systematic reviews and meta-analyses. One way to combat this issue is to establish core outcome sets for clinical studies, including trials.⁷ In 2017, international stakeholders identified key domains for acne to move toward standardizing outcomes in acne research.⁸ These domains included satisfaction with appearance, extent of dark marks and scars, long-term acne control, signs and symptoms, satisfaction with treatment, and HRQoL.

To determine which HRQoL PROMs are most appropriate for acne research and clinical practice, it is important to evaluate the content validity and other measurement properties of these measures.^9,10 The purpose of this review was to rigorously examine HRQoL PROMs for use in adolescents and adults with acne, including: (1) identifying all PROM development and validation studies; (2) evaluating the methodological quality of these studies and the quality of the measurement properties of PROMs using the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) criteria; (3) evaluating the quality of evidence for the summarized measurement property ratings according to the adapted Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) framework; and (4) providing recommendations for PROM use in patients with acne based on synthesized evidence.

Methods

Protocol and Registration

The review protocol was registered on PROSPERO (CRD42021234975). This study did not require ethics approval.

Eligibility Criteria

We included any full-text article published in English or Spanish that investigated development, pilot studies, or evaluation of 1 or more measurement properties for a PROM assessing HRQoL in patients with acne. In studies investigating multiple dermatologic conditions, acne patients had to comprise 50% or more of the patients or subgroup analyses on acne-specific data had to be available. Study population could include children, adolescents, or adults. Studies that only used the PROM as an outcome measurement or where PROMs were included to validate a new or other PROM were excluded. General HRQoL instruments were considered those that were developed for use in patients with any variety of medical conditions and were not specific to any one condition or organ system. Dermatology-specific HRQoL instruments are those that were developed in patients with dermatologic conditions to measure cutaneous disease-mediated HRQoL. Lastly, acne-specific instruments were defined as those developed for use only among patients with acne and measuring acne-mediated HRQoL status.

Data Collection and Data Items

This review followed the COSMIN reporting guideline for systematic reviews of PROMs.¹¹ Bibliographic databases used for identifying articles included PubMed and Embase (OVID) (eAppendix 1 in the Supplement). Search results were uploaded into Covidence to facilitate screening of abstracts and full-text articles for final data extraction. All identified abstracts were screened by 2 independent reviewers. In cases of disagreement or if an abstract was deemed relevant by only 1 reviewer, the full-text article was retrieved and screened. Full-text screening and data extraction was performed by 2 independent reviewers. In cases of disagreement, the reviewers discussed the case, and if needed, a third reviewer was queried.

Extracted information for each study included study characteristics (author, year, country of origin, language, and study design), characteristics of the PROM (construct[s] being measured, target population, number of items, and response options), and measurement properties of the instruments. In accordance with COSMIN guidelines, we assessed the following properties for each PROM development or validation study: content validity, internal consistency, structural validity, construct validity, cross-cultural validity, reliability, measurement error, and responsiveness.¹² The criteria for evaluating results of studies using hypothesis testing to assess construct validity were developed by the study team prior to conducting the review (eAppendix 2 in the Supplement). Spanish translation, if required, was performed by 2 coauthors (J.S.B. or L.M.P.C.).

Risk of Bias

The COSMIN Risk of Bias checklist was used to evaluate the methodological quality of the included studies.^12,13 Each study could be rated as very good, adequate, doubtful, or inadequate. Disagreements were discussed by the group until consensus was reached. For structural validity and internal consistency, the instruments’ measurement model (reflective vs formative) was considered. Reflective scales “reflect” the latent construct, ie, changes in HRQoL caused changes in the item scores measured. Formative (sometimes called “causal”) models measure items that directly cause changes in HRQoL.¹⁴ We characterized each instrument’s original description as reflective or formative. However, when a description was not available, this determination was made by the authors using guidance such as the “thought test.”¹⁵ Structural validity and internal consistency were not evaluated for formative instruments.^15,16,17 If the instrument contained a mix of reflective and formative items and structural validity and internal consistency were reported, the instrument was assumed to be based on a reflective model and such measurement properties were evaluated.¹²

Evaluation of Measurement Properties

The result of each study on a measurement property were extracted and evaluated using the Criteria for Good Measurement Properties proposed by COSMIN guidelines. Accordingly, each result was rated as sufficient, insufficient, or indeterminate.¹¹ Results from individual studies were then qualitatively summarized per measurement property per PROM. The summarized result was also compared against the same criteria and rated as sufficient, insufficient, indeterminate, or inconsistent.

Grading the Quality of Evidence

For each of the included PROMs, evidence quality for the resulting summary scores was estimated. The quality of the evidence was rated as high, moderate, low, or very low according to modified GRADE guidelines.^11,12 These ratings are based on 4 factors: risk of bias (ie, quality of the studies), consistency of results from studies, directness (different populations, interventions or outcomes than those of interest to the review), and precision (width of confidence intervals).¹¹

Recommendations for Use of PROMs in Acne

Each reviewed PROM was assigned to 1 of the 3 standardized COSMIN recommendation for use categories.¹¹ These include can be recommended for use (signified by an A rating), has the potential to be recommended for use but requires further validation (B rating), and should not be recommended for use (C rating). An A-level recommendation for use is defined as the PROM having evidence for sufficient content validity (any level of evidence quality) and at least low-quality evidence for sufficient internal consistency. A C-level recommendation is defined as the PROM having high-quality evidence demonstrating insufficient measurement criteria, and a B-level recommendation is defined as any PROM not meeting criteria for an A-level or C-level recommendation.

Results

Study Selection

A systematic literature search was performed on February 11, 2021. We identified 1200 abstracts; after removing duplicates, 983 potential abstracts remained. After screening, 47 reports met criteria for inclusion into the review, and an additional 7 reports on PROM development were included based on searching cited references of the included studies (Figure).^{18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69}

Figure. Study Flow Diagram.

HRQoL indicates health-related quality of life; PROM, patient-reported outcome measures.

Study Characteristics

Ten different acne-specific PROMs were identified: Acne Disability Index (ADI), Acne Impact on Adult Daily Life (AI-ADL), Acne Severity and Impact Scale (ASIS), Acne-Q, Acne Quality of Life Scale (AQOL), Acne-QoL, Acne Quality of Life Index (Acne-QOLI), Assessment of the Psychological and Social Effects of Acne (APSEA), Cardiff Acne Disability Index (CADI), and CompAQ (Table 1; eTable 1 in the Supplement). All PROMs were multi-item instruments and included adolescents and adults in their target populations. Six instruments were targeted for patients with facial and/or truncal acne, and 2 (Acne-QoL and ASIS) were targeted for facial acne alone. The Acne-QOLI and APSEA did not specify the type of acne they were designed for, only using the general term acne. Acne-related quality of life was largely made up of emotional functioning, physical functioning, and social functioning domains. Two measures were developed using Rasch Measurement Theory, the Acne-Q and the ASIS, and the remainder used classical test theory.

Table 1. Characteristics of the Included PROMs.

PROM	Construct	Target population	Recall period	Subscale, No. of items	Response options	Range of scores/scoring
Acne specific
Acne Disability Index	Acne disability	Adolescents and adults with facial and/or torso acne	Past month	10	Adjectival scale	0-100
Acne Impact on Adult Daily Life	Acne impact on daily life	Adolescents and adults with facial and/or torso acne	Past 4 weeks	14	6-Point adjectival scale, not applicable	0-70 (1-20, mild; 21-30, moderate; >30, severe)
Acne Severity and Impact Scale	Acne signs and impact	Adolescents and adults with facial acne	Signs: right now; impact: past 7 d	Signs (7), impacts (10)	5-Point adjectival scale	Signs (0-28), impacts (0-40)
Acne-Q	Acne impacts, symptoms, and scarring	Adolescents and adults with facial and/or torso acne	Past week	Facial acne (15), chest/back acne (10), facial skin (12), appearance-related distress (10), symptoms (6), scars (10)	4-Point adjectival scale	0-100
Acne Quality of Life Scale	Acne-related quality of life	Adolescents and adults with facial and/or torso acne	Not specified	Social (9), vocational (3)	4-Point adjectival scale	Social (0-9), vocational (0-3)
Acne-QoL	Acne-related quality of life	Adolescents and adults with facial acne	Past week	Self-perception (5), role-emotional (5), role-social (4), symptoms (5)	7-Point adjectival scale	Self-perception (0-30), role-social (0-24), role-emotional (0-30), acne symptoms (0-30)
Acne Quality of Life Index	Acne-related quality of life	Adolescents and adults with acne	Past week	Social functioning (7), psychological functioning (7), emotional functioning (7)	7-Point adjectival scale	21-147; Social functioning (7-49), psychological functioning (7-49), emotional functioning (7-49)
Assessment of the Psychological and Social Effects of Acne	Acne-related quality of life	Adolescents and adults with acne	Past week	Psychological well-being, personal constructs and activities	5-Point adjectival scale	0-138
Cardiff Acne Disability Index	Acne disability	Adolescents and adults with facial and/or torso acne	Past month	4	4-Point adjectival scale	0-15
CompAQ	Acne-related quality of life	Adolescents and adults with facial and/or torso acne	Past 30 d	Psychological/emotional (4), social-judgement (4), social-interactions (4), treatment concerns (4), symptoms (4)	9-Point adjectival scale	Psychological/emotional (0-32), social-judgement (0-32), social-interactions (0-32), treatment concerns (0-32), symptoms (0-32)
Dermatology specific
Dermatology Life Quality Index	Quality of life	Patients 16 y or older with skin disease	Past week	10	4-Point adjectival scale	0-30
Childrens’ Dermatology Life Quality Index	Quality of life	Children ages 4-16 y with skin disease	Past week	10	4-Point adjectival scale	0-30
Dermatology-Specific Quality of Life	Quality of life	Adolescents and adults with skin disease	Past month	Physical symptoms (9), daily activity (5), social function (10), work/school (7), self-perception (5), SF-36 mental (5), SF-36 vitality (4)	Measure-specific subscales: 4-point adjectival scale; global items: 10-point adjectival scale	Subscales (0-4), total score (0-20)
Oily Skin Self-assessment Scale	Oily facial skin severity	Adults	Past week	Visual (3), tactile (4), sensory (5), blotting (1)	10-Point adjectival scale	0-140
Oily Skin Impact Scale	Emotional impact of oily facial skin	Adults	Past week	Annoyance (3), self-image/concept (3)	10-Point adjectival scale	0-84
Skindex-29	Quality of life	Adolescents and adults with skin disease	Past 4 weeks	Symptoms (10), emotions (7), functioning (12)	5-Point adjectival scale	0-100
General
UK Sickness Profile	Quality of life	Adults	None	136	Yes or no	0-100
EuroQol 5-Dimension	Quality of life	Adolescents and adults with skin disease	Today	Mobility (1), self-care (1), activities (1), pain (1), anxiety/depression (1)	5-Point adjectival scale	1-5 for each subscale
SF-36	Quality of life	Adolescents and adults with skin disease	Past 4 weeks	Physical functioning (10), role limitations-physical (4), role limitations-emotional (3), energy (4), emotional well-being (5), social functioning (2), pain (2), general health (5)	Variable adjectival scales	0-100
Patient Benefit Index (PBI)	Therapeutic benefits	Adolescents and adults with skin disease	None	23	5-Point adjectival scale	0-4
Patient-Reported Outcomes Measurement System–Anxiety	Anxiety	Adolescents and adults with skin disease	Past 7 d	29	5-Point adjectival scale	0-100

Abbreviations: PROM, patient-reported outcome measure; SF-36, Short Form-36.

We identified 6 dermatology-specific PROMs that have been assessed in acne: the Dermatology Life Quality Index (DLQI), the Children’s Dermatology Life Quality Index (CDLQI), The Dermatology-Specific Quality of Life (DSQL), Oily Skin Self-assessment Scale (OSSAS), Oily Skin Impact Scale (OSIS), and the Skindex-29. Five generic PROMs that have been studied in acne were identified (UK Sickness Profile, EuroQol 5-Dimension, Short Form-36 [SF-36], Patient Benefit Index [PBI], and Patient-Reported Outcomes Measurement Information System–Anxiety).

Risk of Bias, Evaluation of Measurement Properties, and Quality of the Evidence

For dermatology-specific or generic HRQoL PROMs, development studies were evaluated despite having few or no acne patients, in accordance with COSMIN guidance (Table 2).¹¹ Among acne-specific PROMs overall development was rated adequate for the Acne-Q. The AI-ADL, AQOL, Acne-QOLI, ADI, and CADI were rated as inadequate. The remaining acne-specific PROMs were rated doubtful. Overall content validity was rated as sufficient for the Acne-Q (evidence quality was judged to be moderate), and CompAQ (moderate). The ADI, Acne-QOLS, and CADI were rated insufficient (very low, low, and very low, respectively). Evidence quality for content validity tended to be low, with no study having higher than a moderate level of evidence for any component of content validity.

Table 2. Summary of PROM Development and Content Validity.

Instrument	PROM development			Content validity
	Design	Pilot study	Overall	Relevance		Comprehensiveness		Comprehensibility		Overall
				Rating	GRADE	Rating	GRADE	Rating	GRADE	Rating	GRADE
Acne specific
Acne Disability Index	Inadequate	NA	Inadequate	Insufficient	Very low	Insufficient	Very low	Insufficient	Very low	Insufficient	Very low
Acne Impact on Adult Daily Life	Adequate	Inadequate	Inadequate	Sufficient	Moderate	Insufficient	Moderate	Inconsistent	Moderate	Inconsistent	Moderate
Acne Severity and Impact Scale	Adequate	Doubtful	Doubtful	Inconsistent	Low	Inconsistent	Low	Sufficient	Low	Inconsistent	Low
Acne-Q	Very good	Adequate	Adequate	Sufficient	Moderate	Sufficient	Moderate	Sufficient	Moderate	Sufficient	Moderate
Acne Quality of Life Scale	Doubtful	Inadequate	Inadequate	Insufficient	Low	Insufficient	Low	Insufficient	Low	Insufficient	Low
Acne-QoL	Doubtful	Doubtful	Doubtful	Inconsistent	Moderate	Inconsistent	Moderate	Sufficient	Moderate	Inconsistent	Moderate
Acne Quality of Life Index	Doubtful	Inadequate	Inadequate	Inconsistent	Very low	Inconsistent	Very low	Inconsistent	Very low	Inconsistent	Very low
Assessment of the Psychological and Social Effects of Acne	Inadequate	NA	Inadequate	Inconsistent	Very low	Insufficient	Very low	Inconsistent	Very low	Inconsistent	Very low
Cardiff Acne Disability Index	Inadequate	NA	Inadequate	Insufficient	Very low	Insufficient	Very low	Insufficient	Very low	Insufficient	Very low
CompAQ	Doubtful	Doubtful	Doubtful	Sufficient	Moderate	Sufficient	Moderate	Sufficient	Moderate	Sufficient	Moderate
Dermatology specific
Dermatology Life Quality Index	Doubtful	Doubtful	Doubtful	Inconsistent	Very low	Insufficient	Very low	Sufficient	Very low	Inconsistent	Very low
Childrens’ Dermatology Life Quality Index	Doubtful	Doubtful	Doubtful	Inconsistent	Very low	Insufficient	Very low	Sufficient	Very low	Inconsistent	Very low
Dermatology-Specific Quality of Life	Doubtful	Inadequate	Inadequate	Inconsistent	Very low	Insufficient	Very low	Inconsistent	Very low	Inconsistent	Very low
Oily Skin Self-assessment Scale	Doubtful	Doubtful	Doubtful	Insufficient	Very low	Insufficient	Very low	Inconsistent	Very low	Insufficient	Very low
Oily Skin Impact Scale	Doubtful	Doubtful	Doubtful	Inconsistent	Very low	Insufficient	Very low	Sufficient	Very low	Inconsistent	Very low
Skindex-29	Doubtful	Doubtful	Doubtful	Sufficient	Very low	Sufficient	Very low	Sufficient	Very low	Sufficient	Very low
General
UK Sickness Profile	Doubtful	Inadequate	Inadequate	Insufficient	Very low	Insufficient	Very low	Sufficient	Very low	Insufficient	Very low
EuroQol 5-Dimension	Inadequate	NA	Inadequate	Insufficient	Very low	Insufficient	Very low	Sufficient	Very low	Insufficient	Very low
SF-36	Inadequate	NA	Inadequate	Insufficient	Very low	Insufficient	Very low	Sufficient	Very low	Insufficient	Very low
Patient Benefit Index	Doubtful	Inadequate	Inadequate	Sufficient	Very low	Insufficient	Very low	Sufficient	Very low	Inconsistent	Very low
Patient-Reported Outcomes Measurement Information System–Anxiety	Doubtful	Doubtful	Doubtful	Sufficient	Very low	Insufficient	Very low	Sufficient	Very low	Inconsistent	Very low

Abbreviations: NA, not available; PROM, patient-reported outcome measure; SF-36, Short Form-36.

No studies evaluated measurement error, criterion validity, or cross-cultural validity or measurement invariance, with the exception of the ASIS, which had a single study finding no differential item functioning between White patients and patients with other races and ethnicities (cross-cultural validity; rated as inadequate) (Table 3; eTable 2 in the Supplement). No study explicitly specified whether the PROM under evaluation was based on reflective or formative models. We judged that most instruments were based on reflective models. However, instruments that had symptoms scores or questions about symptomatology (ASIS, Acne-Q, Acne-QoL, and CompAQ) were considered mixed, as some questions behaved as effect indicators of HRQoL (ie, reflective model) and others behaved as causal indicators of HRQoL (ie, formative model).^15,17 Since these instruments had mixed scales, structural validity and internal consistency were assessed for this review, although their results may be affected by this mixed structure. Structural validity was rated as sufficient for 7 of the 10 acne-specific PROMs and was not possible to be rated in the remaining 3 (ADI, APSEA, and CADI). GRADE quality of evidence for these measures was high for 2 PROMs (AI-ADL and CompAQ). Internal consistency was rated as sufficient for 9 of 10 acne-specific PROMs with a GRADE score of high for 2 of 10 PROMs. Construct validity was graded as sufficient for all 10 PROMs, but GRADE quality ranged from low (Acne-Q, AQOL, APSEA) to high (Acne-QoL, CADI, CompAQ). Responsiveness was assessed for 3 PROMs (Acne-QoL, APSEA, and CADI), but the quality of these data ranged from very low for the CADI to moderate for Acne-QoL. Findings were similar for dermatology-specific PROMs, but quality of evidence tended to be lower. These outcomes were less often evaluated for generic PROMs and quality of evidence was low or very low in most cases.

Table 3. Summary of PROM Measurement Properties.

Instrument	Structural validity		Internal consistency		Reliability		Construct validity		Responsiveness
	Rating	Grade	Rating	Grade	Rating	Grade	Rating	Grade	Rating	Grade
Acne specific
Acne Disability Index	Not assessed	NA	Sufficient	NAa	Sufficient	Low	Sufficient	Moderate	Not assessed	NA
Acne Impact on Adult Daily Life	Sufficient	High	Sufficient	High	Sufficient	Low	Sufficient	Moderate	Not assessed	NA
Acne Severity and Impact Scale	Sufficient	Moderate	Sufficient	Moderatea	Sufficient	Moderate	Sufficient	Moderate	Not assessed	NA
Acne-Q	Sufficient	Moderate	Sufficient	Moderatea	Sufficient	Low	Sufficient	Low	Not assessed	NA
Acne Quality of Life Scale	Sufficient	Very low	Sufficient	Very lowa	Sufficient	Very low	Sufficient	Low	Not assessed	NA
Acne-QoL	Sufficient	Moderate	Sufficient	Moderatea	Sufficient	High	Sufficient	High	Sufficient	Moderate
Acne Quality of Life Index	Sufficient	Low	Sufficient	Lowa	Sufficient	Very low	Sufficient	Moderate	Not assessed	NA
Assessment of the Psychological and Social Effects of Acne	Not assessed	NA	Not assessed	NA	Not assessed	NA	Sufficient	Low	Sufficient	Low
Cardiff Acne Disability Index	Not assessed	NA	Sufficient	NAa	Sufficient	Moderate	Sufficient	High	Sufficient	Very low
CompAQ	Sufficient	High	Sufficient	High	Not assessed	NA	Sufficient	High	Not assessed	NA
Dermatology specific
Dermatology Life Quality Index	Sufficient	Moderate	Sufficient	Moderatea	Sufficient	Low	Not assessed	NA	Sufficient	Very low
Children’s Dermatology Life Quality Index	Sufficient	Moderate	Sufficient	Moderatea	Not assessed	NA	Sufficient	Moderate	Not assessed	NA
Dermatology-Specific Quality of Life	Insufficient	Moderate	Sufficient	Lowa	Sufficient	Moderate	Sufficient	High	Sufficient	Moderate
Oily Skin Self-assessment Scale	Sufficient	Moderate	Sufficient	Moderatea	Inconsistent	High	Inconsistent	Moderate	Not assessed	NA
Oily Skin Impact Scale	Sufficient	Moderate	Sufficient	Moderatea	Inconsistent	High	Sufficient	Moderate	Not assessed	NA
Skindex-29	Not assessed	NA	Not assessed	NAa	Sufficient	Moderate	Not assessed	NA	Sufficient	Low
General
UK Sickness Profile	Not assessed	NA	Sufficient	NAa	Sufficient	Low	Sufficient	High	Not assessed	NA
EuroQol 5-Dimension	Not assessed	NA	Not assessed	NA	Not assessed	NA	Not assessed	NA	Sufficient	Very low
SF-36	Not assessed	NA	Not assessed	NA	Not assessed	NA	Not assessed	NA	Sufficient	Very low
Patient Benefit Index	Not assessed	NA	Sufficient	NAa	Insufficient	Very low	Sufficient	Very low	Sufficient	Very low
Patient-Reported Outcomes Measurement Information System–Anxiety	Not assessed	NA	Not assessed	NA	Not assessed	NA	Sufficient	Very low	Sufficient	Very low

Abbreviations: NA, not applicable; PROM, patient-reported outcome measure; QoL, quality of life; SF-36, Short Form-36.

^a Of note, since assessment of internal consistency depends on available evidence for structural validity (unidimensionality is a prerequisite for interpretation of internal consistency analyses), per the COSMIN guidelines, the quality of evidence for structural validity was considered the starting point to determine the quality of evidence for internal consistency.

Recommendations for Use of PROMs in Acne

Among acne-specific instruments, 2 instruments (Acne-Q and CompAQ) received a recommendation grading of A, or recommended for use. No instruments were graded as C, which requires high quality evidence of insufficient measurement criteria. The remaining instruments were graded as B for not having high-quality evidence for insufficient measurement properties or enough evidence for recommendation. All acne-specific instruments had high enough evidence of internal consistency except the APSEA, where this evidence was only assessed in a conference abstract that did not meet inclusion criteria for this review.⁷⁰ However, evidence for sufficient content validity was infrequent, with only the Acne-Q and CompAQ meeting the COSMIN criteria for sufficient content validity.

Discussion

We found that 2 instruments, the Acne-Q and the CompAQ, had sufficient evidence for content validity and internal structure (ie, structural validity and internal consistency) to receive an A recommendation for use. These instruments were also appropriate for both facial and truncal acne in both adults and adolescents. This is important because truncal acne is common and has a significant impact on quality of life.⁷¹ Despite the A ratings, there are missing areas of study for these 2 measures. The Acne-Q has not been assessed for responsiveness, and evidence for reliability and construct validity remains low. The CompAQ similarly has not been assessed for responsiveness or reliability. Interpretability studies have not been conducted for either PROM.

The Acne-Q has a total of 63 items, but these are split into separately graded scales for facial acne (15 items), chest and back acne (10 items), facial skin (12 items), appearance-related distress (10 items), symptoms (6 items), and scars (10 items), so scales can be used or omitted depending on patient situation. The Acne-Q was also developed using Rasch Measurement Theory, which may have some advantages over measures developed using classical test theory.⁷²

The CompAQ has 5 domains with a total of 20 items (psychological/emotional [4 items], social-judgment [4 items], social-interactions [4 items], treatment concerns [4 items], and symptoms [4 items]). The CompAQ also has an available short form for use in busier clinical settings; however, content and measurement properties have not been assessed for this shortened version. Additionally, classical test theory rather than more modern item response theory and Rasch Measurement Theory were used for item selection and scale generation, which may affect construct theory validity and scale interpretation.⁷³

In this review, no short-form HRQoL instruments for acne met our criteria for recommendation. This is an important shortcoming. Instruments that can be rapidly deployed in a clinical setting and which are less likely to induce survey fatigue in patients over multiple visits are critical for clinical implementation.⁷⁴

While concepts like informativity and discrimination (ie, floor and/or ceiling effects) are not evaluated within the COSMIN framework, these may be affected by differences in response options. Items with more available options or long scales may improve discrimination, reliability, validity, and informativity.^75,76 Together, while both the Acne-Q and the CompAQ are well-validated instruments, they assess different aspects of acne-related quality of life and have different instrument characteristics, which may be important when deciding on which is best for clinical or research applications. For instance, the available short forms for the CompAQ may improve feasibility of use in a routine clinical practice environment, while the detailed subscales of the Acne-Q may be valuable in a clinical trial setting.

The remaining measures were found to have a B rating, suggesting that while there was not enough evidence for sufficient content validity and internal structure for an A rating, we did not find high-quality evidence for insufficient measurement properties (which would result in a C score). Of note, the Skindex-29 was the only dermatology-specific instrument with sufficient overall content validity (with very low evidence quality). However, acne-specific data on the internal structure was not available for evaluation, hampering its consideration for an A grade.

Content validity is considered a key component of instrument validation.¹² Although the Acne-QoL is commonly used and has some of the strongest data to support sufficient measurement properties, it had inconsistent ratings with respect to content validity, which precluded it from being given an A rating. Similar issues were identified for the ASIS. Further studies are needed to examine the content validity of these measures. Although the ADI and CADI are frequently used to assess HRQoL in acne studies, these were found to have inadequate PROM development. Likewise, these instruments received overall scores of insufficient for content validity. The APSEA was also found to have inadequate PROM development. Additionally, the AQOL, while receiving a grade of doubtful for development design, did not have a pilot study testing items and received an overall content validity score of insufficient with low evidence. While these instruments did not meet the rather high standard for a C grading, based on these data we believe the evidence for the ADI, CADI, AQOL, and the APSEA are more limited and should not be used without further investigation.

One 2021 systematic review⁷⁷ evaluating PROMs for patient treatment satisfaction in acne found only 1 study with category B evidence (ie, not yet to be recommended). Although we identified 2 acne-specific instruments that can be recommended for use, more data are needed to establish core outcome sets for acne clinical research. Importantly, even for instruments categorized as A, evidence on key measurement properties were missing. In particular, few studies evaluated responsiveness, which is a critical measurement property to assess whether patients have improved with treatment in the setting of a clinical trial or routine clinical care. In addition, measurement error, criterion validity, and cross-cultural validity or measurement invariance were almost never evaluated. Measures for use in clinical research should meet the criteria designated by the OMERACT group, including truth, discrimination and feasibility.⁷⁸ There is a need for future validation studies to address these evidence gaps.

Limitations

This study had several limitations. Only aspects of studies that were reported could be assessed. Limits on publication space may not have allowed prior studies to report on all items in the COSMIN risk of bias checklists. Likewise, prior to the COSMIN initiative in 2005, certain details of methodology may not have been reported, even if performed. Multiple aspects of the risk of bias checklists are relatively subjective (eg, assumedly performed vs doubtful). For example, multiple reports noted following COSMIN checklists or FDA recommendations for instrument development despite missing or not reporting items in the risk of bias checklist. These cases may fall somewhere between assumable (because of mention of the COSMIN checklists) to doubtful, even inadequate (because items were not reported). Furthermore, the FDA guidance documents focus on HRQoL instrument design from a clinical trials perspective, so important design considerations from a clinical perspective may not be included. We attempted to mitigate reviewer bias by discussing and formalizing assumptions for these situations, having multiple reviewers extract data, and conducting group discussions regarding difficult cases. Additionally, although multiple search terms with various interactions were used, it is possible that important development, pilot, or validation studies were missed. As instrument validation work is actively ongoing, future updates are important to guide decisions on instrument use and quality.

Since no studies reported on measurement error or cross-cultural validity, this was not included in overall assessments. However, these are important for interpretability. Likewise, related instrument characteristics such as smallest detectable change, limits of agreement, or minimal important change were beyond the scope of this review. Subjectively, in our review of the evidence, we did not find evidence of these being described for most measures besides the Acne-QoL.

The COSMIN framework has been criticized for its focus on classical test theory and relative lack of modern measurement theory benchmarks, subjectivity and dependency on reviewer expertise, lack of evidence surrounding risk of bias checklist items and grading procedures, and poor interreviewer reliability.^73,79,80 Despite these important limitations, the COSMIN guidelines do offer a formal, Delphi methods–based framework for systematically reviewing PROMs and assessing development and validation quality. Therefore, our recommendations should be considered conservative baselines and further research investigating construct theory, scale design, and instrument will be important for further validation.^73,79,81

Conclusions

In this systematic review, the Acne-Q and CompAQ were found to be validated to a sufficient standard to support recommendation for measuring acne-associated quality of life. The Acne-QoL and ASIS could also be considered if additional evaluation of content validity is performed. Additionally, important measurement properties have not been studied, or have been studied insufficiently, for all instruments. Further research is needed to better define content validity, other measurement properties such as responsiveness, and interpretability of PROMs used to assess HRQoL in patients with acne.

Supplement.

eTable 1. Characteristics of the Included Study Populations

eTable 2. Results of Studies on Measurement Properties

eAppendix 1. Search Strategy

eAppendix 2. Guidance for Evaluating Hypothesis Testing

eReferences.