Identifying and appraising patient‐reported outcome measures on treatment satisfaction in acne: a systematic review

Abstract

Background

After dermatitis, acne is the next skin disease to contribute most to the burden of skin diseases worldwide. Recently, seven core outcome domains have been identified, which together form an Acne Core Outcome Set (ACORN). One of these was satisfaction with acne treatment.

Objectives

To identify studies that described the development of patient‐reported outcome measures (PROMS), evaluated one or more measurement properties of a PROM, or evaluated the interpretability of a PROM in patients with acne regarding treatment satisfaction.

Methods

The COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) search strategy for identifying PROMS on acne treatment satisfaction was used. We searched PubMed, MEDLINE, Embase, LILACS, Web of Science, Cochrane Library, Emcare, PsycINFO and Academic Search premier (June 2020). Study selection, data extraction and assessment of methodological quality according to COSMIN guidance were carried out independently by two authors.

Results

Only one study could be included, describing the development of a treatment satisfaction measure in patients with acne. The development was assessed as inadequate and data on measurement properties were lacking. Additionally, we found 188 studies reporting treatment satisfaction solely as an outcome, using a wide variety of methods, none of them standardized or validated.

Conclusions

We could not find a PROM on treatment satisfaction to recommend for a core outcome set in acne. There is an unmet need for a PROM on treatment satisfaction in acne that is robustly developed, designed and validated.

Short abstract

What is already known about this topic?

• Core outcome sets are consensus‐based minimum outcome measures that should be reported in clinical trials of a specific disease or target condition.

• The Acne Core Outcomes Research Network identified the following domains important for acne: satisfaction with appearance; extent of dark marks and scars; long‐term acne control; signs and symptoms; satisfaction with treatment; health‐related quality of life; and adverse events.

What does this study add?

• We could not find a PROM on treatment satisfaction that can be recommended for a core outcome set in acne.

• Many studies reported treatment satisfaction as an outcome, with a wide variety of methods, none of them standardized or validated.

• There is an unmet need for a PROM measuring treatment satisfaction in acne that is robustly developed and validated according to COSMIN standards.

Linked Comment: J. Kottner and J. Schmitt. Br J Dermatol 2021; 185:3–4.

Core outcome sets are consensus‐based minimum outcome measures that should be reported in clinical trials of a specific disease or target condition.¹ Outcomes can be conceived as constructs or domains, credibly established by patients and other relevant stakeholders, reflecting what is to be measured in trial participants to evaluate the effect of an intervention. Outcome measures are instruments or tools to measure the quality or quantity of the intervention on each of the domains. These measures can range from clinical examinations to patient responses to questionnaires to laboratory findings and to imaging studies.² Outcome measures to be included in a core outcome set are based on a systematic search, assessment of quality and recommendations to a voting panel for inclusion of only one outcome measurement for each domain where possible. In the absence of an adequate measure, a need for development would be established.³

In clinical trials investigating acne treatments, multiple outcome measures have been used with no established standards.⁴, ⁵ This multiplicity, heterogeneity and lack of quality have been problematic in data synthesis by impeding comparative outcomes research and contributing to resource wastage. An international consensus on a core outcome set for clinical trials in acne could address this unmet need by standardizing and harmonizing existing outcome measures, and identifying those that might be lacking.

Development of an acne core outcome set for acne clinical trials was initiated with the Acne Core Outcomes Research Network (ACORN), initially funded by a US National Institutes of Health/National Institute of Arthritis and Musculoskeletal and Skin Diseases grant (1U01AR065109‐01). In a landmark study involving stakeholders worldwide, including 307 patients or their parents, 218 healthcare professionals, 45 nonclinical researchers, 17 industry employees and nine journal editors, the most important domains for an acne core outcome set were identified.⁶ These included satisfaction with appearance, extent of dark marks and scars, long‐term acne control, signs and symptoms, satisfaction with treatment, health‐related quality of life and adverse events (AEs).⁶

A hierarchy of treatment satisfaction can extend from procedures, therapies, activity limitations and dietary restrictions, along with included medications. A more circumscribed concept of satisfaction with intervention or medication impacting on disease signs/symptoms and potential side‐effects is relevant to clinical trials.⁷

In this study, we focused on addressing treatment satisfaction in acne clinical trials, more specifically on satisfaction with any intervention or with the (general) care received. Our aims were to identify patient‐reported outcome measures (PROMS) of satisfaction with acne treatments and to evaluate their quality with COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) methodology.³

Materials and methods

This systematic review conformed to the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) statement.⁸ A prespecified protocol was submitted on 29 October 2019 to the International Prospective Register of Systematic Reviews (PROSPERO) and was registered on 18 May 2020 (CRD42020156473). Owing to the delay with PROSPERO, the protocol was also published at the Open Science Framework (www.osf.io) on 2 February 2020. We followed the COSMIN guideline for systematic reviews of PROMS.⁹

Eligible studies

Any study of people with acne vulgaris that described the development of a PROM, evaluated one or more measurement properties of a PROM, or evaluated the interpretability of a PROM regarding treatment satisfaction was eligible. Treatment satisfaction was interpreted as patients’ satisfaction with any intervention or with the (general) care received. Studies that solely used a PROM on treatment satisfaction as part of, for example, assessing the outcomes of an intervention would be excluded.⁹ However, the latter were tabulated separately to create an overview of instruments used to measure treatment satisfaction. Studies on acne conglobata, rosacea and hidradenitis suppurativa were excluded.

Literature search

A systematic literature search was conducted on 1 November 2019. The following bibliographic databases were searched from their inception for reports of outcome measures assessing satisfaction with treatment of acne: PubMed, MEDLINE (OVID), Embase (OVID), LILACS, Web of Science, COCHRANE Library, Emcare (OVID), PsycINFO (EbscoHOST) and Academic Search Premier (EbscoHOST). There were no language restrictions. The COSMIN search strategy for identifying all PROMs on satisfaction with acne treatment was used (Appendix S1; see Supporting Information), and contained the following: (i) construct – comprehensive search terms regarding treatment satisfaction; and (ii) population – comprehensive search terms regarding people with acne; and (iii) type of instrument – COSMIN PROM filter NOT COSMIN exclusion filter, as described by Terwee et al.¹⁰

A rerun of the search was conducted on 10 June 2020. Search results were uploaded into RAYYAN to facilitate selection of potentially eligible studies (http://rayyan.qcri.org/). Two authors (E.J.v.Z. and B.W.M.A.) independently assessed the eligibility of the studies based on title, abstract and keywords. Records were only excluded when there was sufficient information to support exclusion. From all the other records, also those lacking data, full‐text copies were obtained. These full‐text papers were independently assessed for eligibility by two authors, and the references of eligible studies were independently checked for additional studies. Any disagreement was resolved by discussion and consensus. As recommended by Prinsen et al.,⁹ studies that only used the PROM as an outcome measurement instrument were excluded based on the full text. These were as prespecified in the protocol, tabulated by collecting study details and outcomes using a predetermined form designed for this purpose and serving as supportive information for the ACORN project.

Data extraction and methodological quality of included studies

For data extraction of the included studies on PROM development, validation and evaluation, predefined COSMIN forms were used, as per the methodology.³ The authors only included data if there was an independently attained consensus. The methodological quality of the included studies on PROM development and validation was assessed using the COSMIN Risk of Bias checklist.¹¹ The COSMIN Risk of Bias checklist consists of 10 boxes, each with multiple items that can be scored as very good, adequate, doubtful, inadequate or not applicable.⁹ The lowest rating of any standard determines the overall quality of the PROM. Box 1 addresses PROM development, while the other nine address measurement properties.⁹, ¹¹ The overall assessment of included PROMs was performed according to the COSMIN guideline for systematic reviews of PROMs.³, ⁹

Results

Search results

The search identified 705 records (after removal of duplicates) for which abstracts were screened. A total of 451 references were excluded. Of the remaining 254, the full texts were obtained. Screening of the reference lists of these papers resulted in one additional study, making the total 255. Of these 254 were excluded.¹², ¹³, ¹⁴, ¹⁵, ¹⁶, ¹⁷, ¹⁸, ¹⁹, ²⁰, ²¹, ²², ²³, ²⁴, ²⁵, ²⁶, ²⁷, ²⁸, ²⁹, ³⁰, ³¹, ³², ³³, ³⁴, ³⁵, ³⁶, ³⁷, ³⁸, ³⁹, ⁴⁰, ⁴¹, ⁴², ⁴³, ⁴⁴, ⁴⁵, ⁴⁶, ⁴⁷, ⁴⁸, ⁴⁹, ⁵⁰, ⁵¹, ⁵², ⁵³, ⁵⁴, ⁵⁵, ⁵⁶, ⁵⁷, ⁵⁸, ⁵⁹, ⁶⁰, ⁶¹, ⁶², ⁶³, ⁶⁴, ⁶⁵, ⁶⁶, ⁶⁷, ⁶⁸, ⁶⁹, ⁷⁰, ⁷¹, ⁷², ⁷³, ⁷⁴, ⁷⁵, ⁷⁶, ⁷⁷, ⁷⁸, ⁷⁹, ⁸⁰, ⁸¹, ⁸², ⁸³, ⁸⁴, ⁸⁵, ⁸⁶, ⁸⁷, ⁸⁸, ⁸⁹, ⁹⁰, ⁹¹, ⁹², ⁹³, ⁹⁴, ⁹⁵, ⁹⁶, ⁹⁷, ⁹⁸, ⁹⁹, ¹⁰⁰, ¹⁰¹, ¹⁰², ¹⁰³, ¹⁰⁴, ¹⁰⁵, ¹⁰⁶, ¹⁰⁷, ¹⁰⁸, ¹⁰⁹, ¹¹⁰, ¹¹¹, ¹¹², ¹¹³, ¹¹⁴, ¹¹⁵, ¹¹⁶, ¹¹⁷, ¹¹⁸, ¹¹⁹, ¹²⁰, ¹²¹, ¹²², ¹²³, ¹²⁴, ¹²⁵, ¹²⁶, ¹²⁷, ¹²⁸, ¹²⁹, ¹³⁰, ¹³¹, ¹³², ¹³³, ¹³⁴, ¹³⁵, ¹³⁶, ¹³⁷, ¹³⁸, ¹³⁹, ¹⁴⁰, ¹⁴¹, ¹⁴², ¹⁴³, ¹⁴⁴, ¹⁴⁵, ¹⁴⁶, ¹⁴⁷, ¹⁴⁸, ¹⁴⁹, ¹⁵⁰, ¹⁵¹, ¹⁵², ¹⁵³, ¹⁵⁴, ¹⁵⁵, ¹⁵⁶, ¹⁵⁷, ¹⁵⁸, ¹⁵⁹, ¹⁶⁰, ¹⁶¹, ¹⁶², ¹⁶³, ¹⁶⁴, ¹⁶⁵, ¹⁶⁶, ¹⁶⁷, ¹⁶⁸, ¹⁶⁹, ¹⁷⁰, ¹⁷¹, ¹⁷², ¹⁷³, ¹⁷⁴, ¹⁷⁵, ¹⁷⁶, ¹⁷⁷, ¹⁷⁸, ¹⁷⁹, ¹⁸⁰, ¹⁸¹, ¹⁸², ¹⁸³, ¹⁸⁴, ¹⁸⁵, ¹⁸⁶, ¹⁸⁷, ¹⁸⁸, ¹⁸⁹, ¹⁹⁰, ¹⁹¹, ¹⁹², ¹⁹³, ¹⁹⁴, ¹⁹⁵, ¹⁹⁶, ¹⁹⁷, ¹⁹⁸, ¹⁹⁹, ²⁰⁰, ²⁰¹, ²⁰², ²⁰³, ²⁰⁴, ²⁰⁵, ²⁰⁶, ²⁰⁷, ²⁰⁸, ²⁰⁹, ²¹⁰, ²¹¹, ²¹², ²¹³, ²¹⁴, ²¹⁵, ²¹⁶, ²¹⁷, ²¹⁸, ²¹⁹, ²²⁰, ²²¹, ²²², ²²³, ²²⁴, ²²⁵, ²²⁶, ²²⁷, ²²⁸, ²²⁹, ²³⁰, ²³¹, ²³², ²³³, ²³⁴, ²³⁵, ²³⁶, ²³⁷, ²³⁸, ²³⁹, ²⁴⁰, ²⁴¹, ²⁴², ²⁴³, ²⁴⁴, ²⁴⁵, ²⁴⁶, ²⁴⁷, ²⁴⁸, ²⁴⁹, ²⁵⁰, ²⁵¹, ²⁵², ²⁵³, ²⁵⁴, ²⁵⁵, ²⁵⁶, ²⁵⁷, ²⁵⁸, ²⁵⁹, ²⁶⁰, ²⁶¹, ²⁶², ²⁶³, ²⁶⁴, ²⁶⁵ Reasons for exclusion were that treatment satisfaction was not, in fact, an outcome (n = 56),¹², ¹³, ¹⁴, ¹⁵, ¹⁶, ¹⁷, ¹⁸, ¹⁹, ²⁰, ²¹, ²², ²³, ²⁴, ²⁵, ²⁶, ²⁷, ²⁸, ²⁹, ³⁰, ³¹, ³², ³³, ³⁴, ³⁵, ³⁶, ³⁷, ³⁸, ³⁹, ⁴⁰, ⁴¹, ⁴², ⁴³, ⁴⁴, ⁴⁵, ⁴⁶, ⁴⁷, ⁴⁸, ⁴⁹, ⁵⁰, ⁵¹, ⁵², ⁵³, ⁵⁴, ⁵⁵, ⁵⁶, ⁵⁷, ⁵⁸, ⁵⁹, ⁶⁰, ⁶¹, ⁶², ⁶³, ⁶⁴, ⁶⁵, ⁶⁶, ⁶⁷ only using a treatment satisfaction instrument for acne treatments (n = 91)⁶⁸, ⁶⁹, ⁷⁰, ⁷¹, ⁷², ⁷³, ⁷⁴, ⁷⁵, ⁷⁶, ⁷⁷, ⁷⁸, ⁷⁹, ⁸⁰, ⁸¹, ⁸², ⁸³, ⁸⁴, ⁸⁵, ⁸⁶, ⁸⁷, ⁸⁸, ⁸⁹, ⁹⁰, ⁹¹, ⁹², ⁹³, ⁹⁴, ⁹⁵, ⁹⁶, ⁹⁷, ⁹⁸, ⁹⁹, ¹⁰⁰, ¹⁰¹, ¹⁰², ¹⁰³, ¹⁰⁴, ¹⁰⁵, ¹⁰⁶, ¹⁰⁷, ¹⁰⁸, ¹⁰⁹, ¹¹⁰, ¹¹¹, ¹¹², ¹¹³, ¹¹⁴, ¹¹⁵, ¹¹⁶, ¹¹⁷, ¹¹⁸, ¹¹⁹, ¹²⁰, ¹²¹, ¹²², ¹²³, ¹²⁴, ¹²⁵, ¹²⁶, ¹²⁷, ¹²⁸, ¹²⁹, ¹³⁰, ¹³¹, ¹³², ¹³³, ¹³⁴, ¹³⁵, ¹³⁶, ¹³⁷, ¹³⁸, ¹³⁹, ¹⁴⁰, ¹⁴¹, ¹⁴², ¹⁴³, ¹⁴⁴, ¹⁴⁵, ¹⁴⁶, ¹⁴⁷, ¹⁴⁸, ¹⁴⁹, ¹⁵⁰, ¹⁵¹, ¹⁵², ¹⁵³, ¹⁵⁴, ¹⁵⁵, ¹⁵⁶, ¹⁵⁷, ¹⁵⁸ or for acne scar treatment (n = 97),¹⁵⁹, ¹⁶⁰, ¹⁶¹, ¹⁶², ¹⁶³, ¹⁶⁴, ¹⁶⁵, ¹⁶⁶, ¹⁶⁷, ¹⁶⁸, ¹⁶⁹, ¹⁷⁰, ¹⁷¹, ¹⁷², ¹⁷³, ¹⁷⁴, ¹⁷⁵, ¹⁷⁶, ¹⁷⁷, ¹⁷⁸, ¹⁷⁹, ¹⁸⁰, ¹⁸¹, ¹⁸², ¹⁸³, ¹⁸⁴, ¹⁸⁵, ¹⁸⁶, ¹⁸⁷, ¹⁸⁸, ¹⁸⁹, ¹⁹⁰, ¹⁹¹, ¹⁹², ¹⁹³, ¹⁹⁴, ¹⁹⁵, ¹⁹⁶, ¹⁹⁷, ¹⁹⁸, ¹⁹⁹, ²⁰⁰, ²⁰¹, ²⁰², ²⁰³, ²⁰⁴, ²⁰⁵, ²⁰⁶, ²⁰⁷, ²⁰⁸, ²⁰⁹, ²¹⁰, ²¹¹, ²¹², ²¹³, ²¹⁴, ²¹⁵, ²¹⁶, ²¹⁷, ²¹⁸, ²¹⁹, ²²⁰, ²²¹, ²²², ²²³, ²²⁴, ²²⁵, ²²⁶, ²²⁷, ²²⁸, ²²⁹, ²³⁰, ²³¹, ²³², ²³³, ²³⁴, ²³⁵, ²³⁶, ²³⁷, ²³⁸, ²³⁹, ²⁴⁰, ²⁴¹, ²⁴², ²⁴³, ²⁴⁴, ²⁴⁵, ²⁴⁶, ²⁴⁷, ²⁴⁸, ²⁴⁹, ²⁵⁰, ²⁵¹, ²⁵², ²⁵³, ²⁵⁴, ²⁵⁵ or a report on the same study group (n = 10).²⁵⁶, ²⁵⁷, ²⁵⁸, ²⁵⁹, ²⁶⁰, ²⁶¹, ²⁶², ²⁶³, ²⁶⁴, ²⁶⁵ Therefore, just one study met the inclusion criteria (see Figure 1).²⁶⁶

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) flow diagram. n.a., not available; PROM, patient‐reported outcome measure.

Description of the included study

The study of Alomar et al. was published in 2004 and described the development of a treatment satisfaction questionnaire in patients with acne.²⁶⁶ The authors initiated the process by reviewing the literature in MEDLINE (1980–2002), with the objective of deriving a preliminary list of aspects considered important regarding patient satisfaction with treatment in general and specifically in dermatology. Subsequently, an exploratory method with a moderator‐led focus group was used to discuss aspects from the literature (semi‐structured script), such as degree of knowledge of acne and acne treatments; psychological impact of acne; effects on physical appearance; relationships; information and expectations of treatments; comfort and ease of application or intake; associated AEs; and overall satisfaction with treatment. The focus group consisted of just six women, of whom four were middle‐aged and two were adolescents; all were using isotretinoin or had used isotretinoin. After the 2‐h focus group session, all gathered information was used for the development of an 11‐item questionnaire. More details on this study are provided in Tables S1 and S2 (see Supporting Information). The items referred to satisfaction of various aspects related to treatment such as improvement of symptoms, satisfaction with provided information, satisfaction with treatment, mood, social life, AEs, route of administration, daily activities and the treatment in general. Responses were presented as 4‐ or 5‐point Likert scales. The overall scoring system was not clearly explained. The study authors concluded by stating that a validation study to evaluate the measurement properties still had to be completed. However, the authors confirmed that such a study had not been completed nor published. Furthermore, we found no evidence that this questionnaire was validated or used in other studies.

The methodological quality of this study was assessed with the COSMIN Risk of Bias checklist,¹¹ and independently conducted by two authors (E.J.v.Z. and B.W.M.A.). As the study of Alomar et al. only described development of the questionnaire we were only able to complete the first part of box 1 of that checklist,²⁶⁶ which addresses general design requirements and concept elicitation. The second part assesses the comprehensiveness and comprehensibility of the questionnaire; however, this was not reported in the study (see Table 1).

Table 1.

Quality of the patient‐reported outcome measure (PROM) development

PROM

PROM design

CI studya

Total PROM development

Treatment satisfaction in acne

General design requirements

Concept elicitationb

Total PROM design

General design requirements

Comprehensibility

Comprehensiveness

Total CI study

Clear construct

Clear origin of construct

Clear target population for which the PROM was developed

Clear context of use

PROM developed in sample representing the target population

CI study performed in sample representing the target population

Alomar et al. (2004)266

A

D

A

A

I

I

I

CI, cognitive interview; A, adequate; D, doubtful; I, inadequate. ^aEmpty cells indicate that a CI study (or part of it) was not performed. ^bWhen the PROM was not developed in a sample representing the target population, the concept elicitation was not further rated.

A key criterion is whether a PROM is developed in a sample of patients representing the target population for which the PROM is intended.²⁶⁷ The intention of Alomar et al. was to develop a treatment satisfaction questionnaire for patients with acne, which implies patients of both sexes, all ages, all treatment modalities and encompassing those with a spectrum of acne severity.²⁶⁶ However, only six women were included, with apparently more severe acne because they used or had used isotretinoin. Therefore, this sample was not representative of the target population of people with acne. According to COSMIN standards, this PROM development was considered to be inadequate (see Table 1). As the study of Alomar et al. contained no data on any further validation, we were unable to complete boxes 2–10 of the COSMIN Risk of Bias checklist.²⁶⁶ This shortcoming could not be supplemented with data from other validation studies regarding this PROM.

Evaluation of the PROM

Following COSMIN guidance, we reviewed this PROM against the 10 criteria for good content validity and applied the modified GRADE (Grading of Recommendations, Assessment, Development and Evaluation) approach regarding the quality of the supporting evidence (high, moderate, low or very low).⁹ Based on that, we assessed the content validity of this PROM as insufficient [very low quality evidence; Table S3 (see Supporting Information) and Table 1]. For the other measurement properties, we would have provided summary of findings tables per measurement property if more studies were available. As this was not the case, we opted to summarize all the ratings and the quality of the available evidence in one table, based on the spreadsheet provided by COSMIN (see Table 2). Owing to the lack of studies and data we were unable to describe the interpretability and feasibility of the PROM.

Table 2.

Quality of the evidence for measurement properties of the patient‐reported outcome measure (treatment satisfaction in acne)

PROM is not named	Overall rating	Quality of evidence
	+/–/ ?	High, moderate, low, very low
Content validity	–	Very low
Relevance	–	Very low
Comprehensiveness	–	Very low
Comprehensibility	–	Very low
Structural validity
Internal consistency
Cross‐cultural validity
Measurement invariance
Reliability
Measurement error
Criterion validity
Construct validity
Responsiveness

Empty cells indicate that there was no evidence available to assess. (+), sufficient; (?), Indeterminate; (–), insufficient.

The overall results of our findings were that, per the COSMIN definitions, this PROM was categorized as ‘category B’: there is no sufficient evidence for content validity, but high‐quality evidence for an insufficient measurement property is not available either. This means that this PROM may have a potential to be recommended, but further validation studies are needed.

Studies that only reported treatment satisfaction as an outcome

Studies that only used the PROM as an outcome measurement instrument were excluded, but study details were tabulated, per protocol. We identified 188 studies that reported treatment satisfaction as an outcome: 91 on treatment of acne⁶⁸, ⁶⁹, ⁷⁰, ⁷¹, ⁷², ⁷³, ⁷⁴, ⁷⁵, ⁷⁶, ⁷⁷, ⁷⁸, ⁷⁹, ⁸⁰, ⁸¹, ⁸², ⁸³, ⁸⁴, ⁸⁵, ⁸⁶, ⁸⁷, ⁸⁸, ⁸⁹, ⁹⁰, ⁹¹, ⁹², ⁹³, ⁹⁴, ⁹⁵, ⁹⁶, ⁹⁷, ⁹⁸, ⁹⁹, ¹⁰⁰, ¹⁰¹, ¹⁰², ¹⁰³, ¹⁰⁴, ¹⁰⁵, ¹⁰⁶, ¹⁰⁷, ¹⁰⁸, ¹⁰⁹, ¹¹⁰, ¹¹¹, ¹¹², ¹¹³, ¹¹⁴, ¹¹⁵, ¹¹⁶, ¹¹⁷, ¹¹⁸, ¹¹⁹, ¹²⁰, ¹²¹, ¹²², ¹²³, ¹²⁴, ¹²⁵, ¹²⁶, ¹²⁷, ¹²⁸, ¹²⁹, ¹³⁰, ¹³¹, ¹³², ¹³³, ¹³⁴, ¹³⁵, ¹³⁶, ¹³⁷, ¹³⁸, ¹³⁹, ¹⁴⁰, ¹⁴¹, ¹⁴², ¹⁴³, ¹⁴⁴, ¹⁴⁵, ¹⁴⁶, ¹⁴⁷, ¹⁴⁸, ¹⁴⁹, ¹⁵⁰, ¹⁵¹, ¹⁵², ¹⁵³, ¹⁵⁴, ¹⁵⁵, ¹⁵⁶, ¹⁵⁷, ¹⁵⁸ and 97 on treating acne scars (a possible sequela of acne).¹⁵⁹, ¹⁶⁰, ¹⁶¹, ¹⁶², ¹⁶³, ¹⁶⁴, ¹⁶⁵, ¹⁶⁶, ¹⁶⁷, ¹⁶⁸, ¹⁶⁹, ¹⁷⁰, ¹⁷¹, ¹⁷², ¹⁷³, ¹⁷⁴, ¹⁷⁵, ¹⁷⁶, ¹⁷⁷, ¹⁷⁸, ¹⁷⁹, ¹⁸⁰, ¹⁸¹, ¹⁸², ¹⁸³, ¹⁸⁴, ¹⁸⁵, ¹⁸⁶, ¹⁸⁷, ¹⁸⁸, ¹⁸⁹, ¹⁹⁰, ¹⁹¹, ¹⁹², ¹⁹³, ¹⁹⁴, ¹⁹⁵, ¹⁹⁶, ¹⁹⁷, ¹⁹⁸, ¹⁹⁹, ²⁰⁰, ²⁰¹, ²⁰², ²⁰³, ²⁰⁴, ²⁰⁵, ²⁰⁶, ²⁰⁷, ²⁰⁸, ²⁰⁹, ²¹⁰, ²¹¹, ²¹², ²¹³, ²¹⁴, ²¹⁵, ²¹⁶, ²¹⁷, ²¹⁸, ²¹⁹, ²²⁰, ²²¹, ²²², ²²³, ²²⁴, ²²⁵, ²²⁶, ²²⁷, ²²⁸, ²²⁹, ²³⁰, ²³¹, ²³², ²³³, ²³⁴, ²³⁵, ²³⁶, ²³⁷, ²³⁸, ²³⁹, ²⁴⁰, ²⁴¹, ²⁴², ²⁴³, ²⁴⁴, ²⁴⁵, ²⁴⁶, ²⁴⁷, ²⁴⁸, ²⁴⁹, ²⁵⁰, ²⁵¹, ²⁵², ²⁵³, ²⁵⁴, ²⁵⁵ These numbers show that clinicians and researchers do consider treatment satisfaction as an important outcome. However, the methods used clearly demonstrate the current diversity (Figures 2 and 3). A summary of the studies is supplied in Table 3 and characteristics of the studies in Tables S4 and S5 (see Supporting Information).

Figure 2.

Types of tools to measure satisfaction with acne treatment. NRS, numerical rating scale; VAS, visual analogue scale.

Figure 3.

Types of tools to measure satisfaction with acne scar treatment. NRS, numerical rating scale; VAS, visual analogue scale.

Table 3.

Characteristics of studies measuring patient‐reported treatment satisfaction in acne

	Treatment of acne (n = 91)	Treatment of acne scars (n = 97)	Total (n = 188)
Participants
Total	32 811	3791	36 602
Men	10 917 (33·3)	903 (23·8)	11 820 (32·3)
Women	15 664 (47·7)	1774 (46·8)	17 438 (47·6)
Sex unknown/not reported	6230 (19·0)	1114 (29·4)	7344 (20·1)
Mean age (years)	22·7	33·0
Minimum in a study (n)	10	3
Maximum in a study (n)	5131	352
Mean per study (n)	364·6	39·1
Median per study (n)	62	25
Location
Asia	20 (22)	27 (28)	47 (25.0·)
Australia	–	2 (2)	2 (1·1)
Europe	25 (27)	15 (15)	40 (21·3)
Latin America	4 (4)	7 (7)	11 (5·9)
Middle East	12 (13)	17 (17)	29 (15·4)
Multiple countries	9 (10)	6 (6)	15 (8·0)
North America	20 (22)	23 (24)	43 (22·9)
South Africa	1 (1)	–	1 (0⋅5)
Study design (%)
Cross‐sectional	11 (12)	1 (1)	12 (6·4)
Open label	29 (32)	56 (58)	85 (45·2)
RCT	43 (47)	24 (25)	67 (35·6)
Other	8 (9)	16 (16)	24 (12·8)
Study duration (weeks)
Minimum (n)	2	4
Maximum (n)	156	208
Mean (n)	18·6	33·5
Median (n)	12	26
Measurement methods
One question	46 (51)	58 (60)	104 (55·3)
Questionnaire	35 (39)	27 (28)	62 (33·0)
Unclear/not reported	10 (11)	12 (12)	22 (11·7)
Answer options
NRS/VAS	19 (21)	18 (19)	37 (19·7)
3‐point Likert	1 (1)	5 (5)	6 (3·2)
4‐point Likert	16 (18)	22 (23)	38 (20·2)
5‐point Likert	21 (23)	27 (28)	48 (25·5)
6‐point Likert	2 (2)	1 (1)	3 (1·6)
7‐point Likert	1 (1)	–	1 (0·5)
Other	4 (4)	1 (1)	5 (2·7)
Unclear/not reported	27 (30)	23 (24)	50 (26·6)

Data are n (%) unless otherwise indicated. RCT, randomized controlled trial; NRS, numeric rating scale; VAS, visual analogue scale.

Discussion

In dermatology, various collaborative groups have been established to facilitate the development of core outcome sets, including the International Dermatology Outcome Measures group (IDEOM)²⁶⁸ and the Cochrane Skin – Core Outcome Set Initiative (CS‐COUSIN).²⁶⁹ Disease‐specific efforts have been initiated in atopic dermatitis, psoriasis and hidradenitis suppurativa.²⁷⁰, ²⁷¹, ²⁷² That such an elaborate worldwide undertaking has an effect was shown with the Harmonising Outcome Measures for Eczema (HOME) initiative for atopic dermatitis, as more trials adhered to its recommendations.²⁷³

For acne this has been the remit of ACORN.²⁷⁴ As part of that initiative, this systematic review was undertaken to identify and evaluate the quality of outcome measures relevant to one of the seven previously identified core domains⁶ – treatment satisfaction. The methodology we followed was established by COSMIN.³, ⁹, ¹¹ Briefly, the steps involved were a search for existing instruments (from systematic reviews, literature searches and other sources) and quality assessment of the instruments found. This would then lead to recommendations on selection and a consensus procedure for final agreement on an outcome measure – in this case treatment satisfaction.

Treatment satisfaction has previously been defined as a patient‐assessed domain addressing attributes of the process and outcome of the treatment experience.²⁷⁵ This would be inclusive of benefits in relief of signs and symptoms and risk of AEs, inconvenience and cost. In clinical trials, cost of treatment would be excluded as medications are typically provided to patients, therefore not reflecting real‐world experience. Furthermore, in preapproval trials, cost of treatment may not have yet been established by the manufacturer. Regulatory authorities are increasingly recognizing the importance of these patient‐reported measures on aspects of treatment and disease impact that are not readily accessible or evaluable by other methodologies.

From our literature search, numerous studies were found to have treatment satisfaction of acne and of acne scars as a patient‐reported outcome. These findings underline the importance of treatment satisfaction in research and support the previous consensus involving patients with acne and other relevant stakeholders in identifying it as an important core outcome domain.⁶ This was also reflected by the increase in studies evaluating treatment satisfaction over the years (Figure 4). However, while investigators of these studies considered treatment satisfaction important and reported data for this outcome, none used a standardized, validated instrument – underlining the need for harmonization. Furthermore, we found instances where the degree of improvement was interpreted as a degree of treatment satisfaction, despite these constructs not being identical.

Figure 4.

Number of studies reporting treatment satisfaction over the years.

We found only one study that addressed the development of a PROM for treatment satisfaction in acne, but it was assessed as not yet to be recommended by COSMIN standards (category B).²⁶⁶ This means that there is still an unmet need for a well‐developed and validated PROM for treatment satisfaction in acne. PROM development and validation is not a trivial endeavour. Guidance from academia and regulatory authorities are available to facilitate the development of such a PROM.²⁷⁶, ²⁷⁷, ²⁷⁸ Development and validation should meet COSMIN standards, especially as it is intended to be part of a core outcome set to be used in clinical trials. In its absence, we observed that researchers are left to develop ad hoc, nonstandardized, unvalidated measures for evaluation of a domain they consider important in their clinical trials. Although it reflects the researchers’ involvement and creativity, it was this diversity and variety of measures that impeded comparison of outcomes on treatment satisfaction in acne between studies or the ability to conduct a systematic review with quantitative meta‐analyses on this important outcome.

The limitations are that our search strategy would not have detected studies that do not contain the construct treatment satisfaction and associated terms in the meta‐data of the consulted databases. This could have underestimated the number of trials evaluating treatment satisfaction in acne. The strengths of our review were the comprehensive search in multiple databases for potentially eligible studies without language restriction, ensuring no risk of language bias. Furthermore, although only one study could be included (in Spanish), we compiled a comprehensive list of measures of patient treatment satisfaction in acne research. As such, this review could serve as a resource for developing a PROM on treatment satisfaction in acne.

This study was undertaken to identify and assess the quality of existing measures corresponding to a core domain in a core outcome set for acne: treatment satisfaction. However, in our search and subsequent assessments, we were unable to find a single measure that fulfilled the necessary criteria. We did find a plethora of ad hoc scales addressing treatment satisfaction in acne and acne scarring, none standardized or validated. This represents an unmet need for a PROM measuring treatment satisfaction in acne that is robustly developed and validated according to COSMIN standards.

Author Contribution

Esther J van Zuuren: Conceptualization (lead); Data curation (lead); Formal analysis (lead); Investigation (lead); Methodology (lead); Project administration (lead); Supervision (lead); Validation (lead); Visualization (lead); Writing‐original draft (lead); Writing‐review & editing (lead). Bernd WM Arents: Conceptualization (equal); Data curation (equal); Formal analysis (equal); Investigation (equal); Methodology (equal); Supervision (equal); Validation (equal); Visualization (lead); Writing‐original draft (equal); Writing‐review & editing (equal). Maegan Miklas: Writing‐original draft (supporting); Writing‐review & editing (supporting). Jan W Schoones: Data curation (equal); Methodology (equal). Jerry Tan: Conceptualization (equal); Writing‐original draft (equal); Writing‐review & editing (equal).

Supporting information

Appendix S1 Search strategy.

Table S1 Characteristics of the included patient‐reported outcome measure.

Table S2 Characteristics of the included study populations.

Table S3 Content validity.

Table S4 Study characteristics and results of studies evaluating satisfaction with acne treatment.

Table S5 Study characteristics and results of studies evaluating satisfaction with acne scars treatment.

Powerpoint S1 Journal Club Slide Set.

Video S1 Author video