Differences in Psychometric Properties of Clinician and Patient-reported Outcome Measures for Atopic Dermatitis By Race and Skin Tone: A Systematic Review

Abstract

Psychometric validity and reliability of widely used atopic dermatitis (AD) outcome measures across different race and ethnicity is unclear. We describe rates of reporting race, ethnicity and skin tone in studies testing psychometric properties of AD outcome measures and compare psychometric analyses across race, ethnicity, and skin tone. We systematically reviewed MEDLINE and EMBASE for studies reporting psychometric properties of clinician reported (ClinROM) or patient reported outcome measures (PROM) in AD (PROSPERO: CRD42021239614). Overall, 16,100 non-duplicate articles were screened; 165 met inclusion criteria. Race and/or ethnicity were reported in 55 (33.3%) studies; of those, race was assessed by self-report in 10 (6.1%) or was unspecified in 45 (27.3%). Sixteen studies (9.7%) evaluated psychometric property differences by race and only 5 (4.4%) of those that did not recognized it as a limitation. Properties assessed across race, ethnicity or skin tone were differential item functioning, convergent validity feasibility, inter-rater reliability, intra-rater reliability, test-retest reliability, and known-groups validity. Multiple instruments demonstrated performance differences across ethno-racial groups. This review highlights the paucity of race/ethnicity consideration for psychometric property testing in AD outcome measurement instruments. More AD outcomes instruments should be validated in diverse populations.

INTRODUCTION

Atopic dermatitis (AD) is characterized by heterogeneous lesional morphology, distribution, symptom complex and comorbidities.(Silverberg et al., 2020g) Certain AD features vary between ethnoracial groups, including lesional distribution, frequency and degree of follicular and lichenoid lesions, lichenification and prurigo nodules.(Brunner and Guttman-Yassky, 2019, Kaufman et al., 2018) These differences may introduce challenges for standardized diagnostic criteria as well as disease severity assessments in research and clinical settings.(Zhao et al., 2017) For example, the Hanifin-Rajka and abridged United Kingdom Working Party criteria perform inadequately in certain non-white races and ethnicities.(Thyssen et al., 2020) Understanding psychometric properties of relevant measurement domains and outcome instruments across different races, ethnicities, and skin tones is therefore essential.

For clinical trials, Harmonizing Outcome Measures for Eczema (HOME) selected by international consensus preferred outcome measurements for AD symptoms (Patient-Oriented Eczema Measure [POEM]), signs (Eczema Area and Severity Index [EASI]), and quality of life (QoL) (Dermatology Life Quality Index [DLQI]) in adults.(Silverberg et al., 2020a) For clinical practice, HOME recommended POEM and Patient-Oriented SCORing Atopic Dermatitis (PO-SCORAD) to assess patient-reported symptoms.

However, little is known about differences in psychometric properties of commonly used patient-reported and clinician-reported outcome measures (PROMs and ClinROMs) by race, ethnicity or skin tone. Some signs evaluated (e.g. erythema, lichenification, and xerosis) and symptoms (e.g. pruritus) may present differently or in different degrees in skin of color (SOC).(Kaufman et al., 2018, Zhao et al., 2017) Some instruments were adapted for darker skin tones (e.g. PO-SCORAD)(Faye et al., 2020a), though most have not. Concerns remain regarding the potential of overestimating(Zhao et al., 2017) or underestimating(Ben-Gashir and Hay, 2002, Kaufman et al., 2018) AD severity in SOC patients depending on the instrument used.

It is imperative to assess AD signs and symptoms with instruments that are valid and reliable across all races, ethnicities and skin types in clinical care and research. Tools that perform differently across these groups may lead to inadequate clinical management, systematic bias and potentially spurious trial results. In this systematic review, we examined reporting of race, ethnicity and skin tone and compared results across these groups from studies of psychometric properties for outcome measures in AD.

RESULTS

Literature search

Overall, 16100 non-duplicate records underwent title and abstract screening and 166 full text review; 79 studies met eligibility criteria (Figure 1). Seventy-nine additional references were identified by snowballing from the 2013 and 2019 HOME group systematic reviews, of which 72 were non-review articles and available in full-text. An additional 23 articles were identified by snowballing from references of other included articles. In total, 165 non-duplicate articles were included in the review (Appendix 4). There was inconsistent use of terminologies for race and ethnicity in included papers. Results are reported according to the authors’ definitions, not consensus opinion (Appendix 5). The 165 studies included 41,146 persons (white: n=7,433 [18.1%]; black: n=1,003 [2.4%]; Hispanic: n=731 [1.8%]; Asian: n=1,630 [4.0%]; American Indian/Alaskan Native [n=6; 0.02%]; multiracial/other: n=430 [1.1%]; unspecified: n=29,913 [72.7 %]). Additional study and cohort characteristics and risk of bias analyses are presented in Supplemental Results.

Figure 1.

PRISMA flow diagram.

Reporting of race and ethnicity

Ethnoracial demographics of study participants were reported in only 55 (33.3%) studies; race was assessed by self-report in 10 (6.1%) or unspecified methods (n=45, 27.3%). Four (2.4%) studies described participant race, 9 (5.5%) ethnicity, 25 (15.2%) race and ethnicity combined, 7 (4.2%) race and ethnicity separately, and 10 (6.1%) unspecified.

Only 16 (9.7%) studies evaluated differences of psychometric properties by race, ethnicity, or skin tone; 113 (68.5%) did not, of which only 5 acknowledged absence of stratification by race as a study limitation or direction for future research. Race or ethnicity were assessed by self-report in 2 (1.2%) or unspecified in 14 (8.5%). The median (minimum-maximum) proportions of participants were 61% (36-71.9%) white, 11.88% (0.6-13.8%) black, 7.1% (0-18.8%) Hispanic, 16.7% (0-48%) for Asian, 0.0% (0-1.5%) American Indian/Alaskan native, and 3.9% (0-17.2%) multiracial/other (Table 1). The 16 studies included 5,900 persons (white: n=3,732 [63.3%]; black: n=330 [5.6%]; Hispanic: n=254 [4.3%]; Asian: n=436 [7.4%]; American Indian/Alaskan Native [n=2; 0.03%]; multiracial/other: n=135 [2.13%]; unspecified: n=1,011 [17.1%]).

Table 1.

Number of studies that investigated psychometric properties of clinician-reported and patient-reported outcomes by skin tone, race, or ethnicity by AD Instrument.

Clinician-reported Outcome Measure	Convergent Validity			DIF			Interrater Reliability			Intrarater Reliability
	Study	N	Findings^	Study	N	Findings#	Study	N	Findings	Study	N	Findings
EASI	1. (Zhao et al., 2015) Prospective Cross-Sectional study*	18					1.(Zhao et al., 2015) Prospective Cross-Sectional study*	18	−	1.(Zhao et al., 2015) Prospective Cross-Sectional study*	18	−
	2. (Zhao et al., 2017) Prospective Cross-Sectional study*	25					2.(Zhao et al., 2017) Prospective Cross-Sectional study*	25	−	2.(Zhao et al., 2017) Prospective Cross-Sectional study*	25	−
	3. (Vakharia P. P. et al., 2018) Prospective Cross-Sectional study*	388	−
	4. (Leshem et al., 2015a) Retrospective Cross-Sectional study*	170	−
	(Zhao et al., 2015) Prospective Cross-Sectional study*	18					(Zhao et al., 2015) Prospective Cross-Sectional study*	18	−	(Zhao et al., 2015) Prospective Cross-Sectional study*	18	−
oSCORAD	(Zhao et al., 2017) Prospective Cross-Sectional study *	25					(Zhao et al., 2017) Prospective Cross-Sectional study*	25	−	(Zhao et al., 2017) Prospective Cross-Sectional study*	25	−
	(Vakharia P. P. et al., 2018) Prospective Cross-Sectional study *	388	−
IGA	(Zhao et al., 2017) Prospective Cross-Sectional study*	25					(Zhao et al., 2017) Prospective Cross-Sectional study*	25	−	(Zhao et al., 2017) Prospective Cross-Sectional study*	25	−
TIS	(Zhao et al., 2015) Prospective Cross-Sectional study*	18					(Zhao et al., 2015) Prospective Cross-Sectional study*	18	+	(Zhao et al., 2015) Prospective Cross-Sectional study*	18	−
Greyscale							(Zhao et al., 2017) Prospective Cross-Sectional study*	25	−	(Zhao et al., 2017) Prospective Cross-Sectional study*	25	+
SGAxBSA	5. (Topham et al., 2020) Retrospective Cross-Sectional study*	138
SASSAD	(Zhao et al., 2015) Prospective Cross-Sectional study*	18					(Zhao et al., 2015) Prospective Cross-Sectional study*	18	−	(Zhao et al., 2015) Prospective Cross-Sectional study*	18	−
	(Zhao et al., 2017) Prospective Cross-Sectional study*	25
Patient-Reported Outcome Measure
DLQI				1. (Patel et al., 2019) Prospective Cross-Sectional study	340	+
				2.(Silverberg et al., 2019b) Prospective Cross Sectional Study ~	602	+
POSCORAD	6. (Silverberg et al., 2020c) Prospective Cross-Sectional Study*	291	−	3.(Silverberg et al., 2020c) Prospective Cross-Sectional Study	291	−
				4.(Silverberg et al., 2020h) Prospective Cross Sectional Study	602	+
POEM	(Zhao et al., 2017) Prospective Cross-Sectional study*	25		(Silverberg et al., 2020c) Prospective Cross-Sectional Study	291	−
	(Silverberg et al., 2020c) Prospective Cross-Sectional Study*	291		(Silverberg et al., 2020h) Prospective Cross Sectional Study ~	602	+
NRS Itch				(Silverberg et al., 2020h) Prospective Cross Sectional Study ~	602	+
Self-reported Global AD Severity	7. (Silverberg et al., 2018b) Prospective Cross Sectional Study~	602	−	5.(Silverberg et al., 2018b) Prospective Cross Sectional Study ~	602	−
	(Vakharia P. P. et al., 2018) Prospective Cross-Sectional Study *	266	−
Itchy QoL				(Patel et al., 2019) Prospective Cross-Sectional Study*	340	+
5D Itch				(Patel et al., 2019) Prospective Cross-Sectional Study*	340	+
SF12				(Silverberg et al., 2019b) Prospective Cross Sectional Study ~	602	+
PROMIS PIQ				6. (Lei et al., 2020a) Prospective Cross-Sectional Study	611	−
PROMIS SD				(Lei et al, 2020a) Prospective Cross-Sectional Study*	611	−
				7.(Lei et al., 2020b) Prospective Cross-Sectional Study*	420	−
PROMIS SRI				(Lei et al., 2020a) Prospective Cross-Sectional Study*	611
				(Lei et al., 2020b) Prospective Cross-Sectional Study*	420	−
ESS				(Lei et al., 2020a) Prospective Cross-Sectional Study*	611	−
PIQ Itch Triggers
PIQ Short Form				8.(Silverberg et al., 2020b) Prospective Cross-Sectional Study*	239	+
HADS				9.(Silverberg et al., 2019a) Prospective Cross Sectional Study~	602	+
Total studies	7			9			2			2

Clinician-Reported Outcome Measure	Test-retest reliability			Known groups validity			Feasibility
	Study	N	Findings	Study	N	Findings	Study	N	Findings
EASI							1.(Leshem et al., 2015a) Retrospective Cross-Sectional Study*	170	−
oSCORAD
IGA
TIS
Greyscale
SGAxBSA
SASSAD
Patient-Reported Outcome Measure
DLQI
POSCORAD	1.(Silverberg et al., 2020c) Prospective Cross-Sectional Study*	291	+				2.(Silverberg et al., 2020c) Prospective Cross-Sectional Study*	291	−
POEM	(Silverberg et al., 2020c) Prospective Cross-Sectional Study*	291	−				(Silverberg et al., 2020c) Prospective Cross-Sectional Study*	291	−
NRS Itch
Self-reported Global AD Severity				1.(Vakharia P. P. et al., 2018) Prospective Cross-Sectional Study*	265
Itchy QoL
5D Itch
SF12
PROMIS PIQ
PROMIS SD				2.(Lei et al., 2020b) Prospective Cross-Sectional Study*	420
PROMIS SRI				(Lei et al., 2020b) Prospective Cross-Sectional Study*	420
ESS
PIQ Itch Triggers							3.(Silverberg et al., 2020g) Prospective Cross-Sectional Study*	587	−
PIQ Short Form							4.(Silverberg et al., 2020b) Prospective Cross-Sectional Study*	239	−
HADS
Total studies	1			2			4

N=sample size

+ Significant between-group differences found by race, ethnicity, or skin tone

− No significant between-group differences found upon assessment

Blank No assessment of between-group significance performed when reporting differences

^{^}Indication of significance for correlation coefficients of specific subgroups is coded in Table 2

^*Clinical Practice setting

^#Clinical Practice setting (Inpatient)

^~Analysis of web-based survey

Differences of psychometric properties by race, ethnicity or skin tone

Cross-cultural validity, reliability and construct validity were examined by race, ethnicity or skin tone: differential item functioning (DIF) (n=9, 56.3%), convergent validity (n=7, 50%), feasibility (n=4, 25%), inter-rater reliability (n=2, 12.5%), intra-rater reliability (n=2, 12.5%), test-retest reliability (n=1, 6.2%), and known-groups validity (n=2, 12.5%) (Table 1). No studies examined differences of content validity, floor and ceiling effects, measurement error, criterion validity, responsiveness, internal consistency, criterion validity, content validity, dimensionality, interpretability, or meaningful change by race, ethnicity or skin tone.

Only 2 (11.8%) studies analyzed differences of psychometric properties by skin pigmentation or tone.(Zhao et al., 2017, Zhao et al., 2015) Skin tone was assessed via melanin index determined by machine readings of gluteal skin(Zhao et al., 2017) or clinician assessment of skin pigmentation (0 [no pigment] to 10 [darkest pigment]).(Zhao et al., 2015) One study validated AD instruments in an exclusively African population who were presumed to all be Fitzpatrick Group 6.(Faye et al., 2020a) Psychometric properties assessed by race and skin tone-based were summarized for each PROM and ClinROM (Appendix 5).

PROMS

Differential Item Functioning (DIF)

Nine studies investigated DIF by race or ethnicity.(Lei et al., 2020a, Lei et al., 2020b, Patel et al., 2019, Silverberg et al., 2018b, Silverberg et al., 2019a, Silverberg et al., 2019b, Silverberg et al., 2020b, Silverberg et al., 2020c, Silverberg et al., 2020h) A study of 291 adults found that no items from PO-SCORAD and POEM had significant DIF by race (white vs. non-white).(Silverberg et al., 2020c) In a study of 611adults, no items from the Patient-reported Outcome Measures Information System (PROMIS©) Itch Questionnaire (PIQ) Mood and Sleep, PROMIS Sleep Disturbance, PROMIS Sleep-Related Impairment, or ESS had significant uniform or nonuniform DIF by race/ethnicity (white vs. other).(Lei et al., 2020a) Three and four items from the Hospital Anxiety and Depression scales (HADS), respectively, had significant uniform and nonuniform DIF by race (white vs. black vs. Asian vs. other).(Silverberg et al., 2019a) A study of 239 adults with AD found that two items from PIQ short-forms met criteria for uniform or nonuniform DIF by race (white vs. non-white), but the magnitudes were low.(Silverberg et al., 2020b). A study of 340 adults with AD found 1 item from DLQI had uniform DIF and 1 item had nonuniform DIF by race; 4 items from ItchyQoL had uniform DIF and 5 items had nonuniform DIF; 1 item from 5-D itch had uniform DIF and 1 item had nonuniform DIF (white vs. non-white).(Patel et al., 2019) A study of 602 adults found no significant differences of self-reported global atopic dermatitis severity by race (white vs. non-white).(Silverberg et al., 2018b)

A study of 602 adults with AD found 1 and 0 items from SF-12 with uniform and nonuniform DIF; 5 and 2 items from DLQI displayed uniform and nonuniform DIF (Silverberg et al., 2019b); multiple items of PO-SCORAD (1 item with uniform; 2 items with nonuniform) and POEM (5 items with uniform DIF; 2 items with nonuniform DIF) by race (white vs black vs Asian vs other)(Silverberg et al., 2020h); Finally, no items in PROMIS SD or PROMIS SRI had significant uniform or nonuniform DIF by race (white vs. other).(Lei et al., 2020b)

DIF was assessed for DLQI, POSCORAD, POEM, PROMIS SD, PROMIS SRI in multiple studies. Two studies examined DIF using regression analysis. One showed that DLQI had one item with uniform DIF (Q10) and one with non-uniform DIF (Q2).(Patel et al., 2019) Another found 5 items (Q1, Q2, Q6, Q9, Q10) with uniform and 2 (Q2, Q6) with nonuniform DIF. Q10 and Q2 of DLQI showed uniform and nonuniform DIF, respectively, in both studies.(Patel et al., 2019, Silverberg et al., 2019b) One study used Item Response Theory (IRT) and also showed Q2 had uniform DIF.(Patel et al., 2019)

Two studies assessed DIF for PO-SCORAD and POEM and had contradictory findings. One found 4 items of PO-SCORAD with significant uniform (dry, NRS-itch, NRS-sleep) or nonuniform DIF (dry, red areas, NRS-itch).(Silverberg et al., 2020h) The same study found 5 items of POEM with significant uniform (sleep, bleeding, weeping, cracking, dry or rough) or nonuniform DIF (sleep, dry or rough). Another reported no significant DIF on any item for POSCORAD or POEM.(Silverberg et al., 2020c)

Convergent validity

Four studies identified correlations between PROMs by race, without significant inter-racial differences observed.(Silverberg et al., 2018b, Silverberg et al., 2020c, Vakharia P. P. et al., 2018, Zhao et al., 2017) Self-reported global AD severity showed significant associations with PO-SCORAD total and objective scores, NRS-itch, NRS-sleep, NRS-pain, POEM, and HADS in both white and non-white subgroups.(Silverberg et al., 2018b) Associations of self-reported global AD severity with objective-SCORAD, SCORAD, EASI, BSA, NRS-itch, POEM and DLQI showed similar effect-sizes in white vs. non-white patients.(Vakharia P. P. et al., 2018)

Correlational effect size data was available for eleven PROMs (POEM, POSCORAD, NRS worst itch, NRS average itch, DLQI, ItchyQoL, PHQ9, PROMIS SD, PROMIS SRI, INRS, PtGA) for both white/nonwhite patients and SOC/light skin patients.(Silverberg et al., 2020c, Zhao et al., 2017) The range of correlations for POEM in white patients was 0.41 (PROMIS SRI) to 0.65 (DLQI and ItchyQoL); in non-white patients this range was 0.48 (NRS worst) to 0.74 (ItchyQoL). The range of correlations for PO-SCORAD in white patients was 0.41 (PROMIS SD) to 0.68 (ItchyQoL); in non-white patients this range was from 0.17 (EASI) to 0.6 (ItchyQoL). All studies evaluating convergent validity of PROMs by race and skin tone using correlation coefficients are presented in Table 2.

Table 2.

Correlation between outcome measure in studies that stratified analyses by race, ethnicity or skin tone.

Outcome Measure	POEM	PO-SCORAD	EASI	oSCORAD	TIS	SGAxBSA
POEM		(Silverberg et al., 2020c) white (r=0.51) non-white (r=0.57)
PO-SCORAD
EASI	(Zhao et al., 2017) SOC (r=0.53), light skin (r=0.48)	(Silverberg et al., 2020c) white (r=0.43) non-white(r=0.17)
	(Silverberg et al., 2020c) white (p=0.48) non-white (r=0.54)
oSCORAD	(Zhao et al., 2017) SOC (r=0.42) light skin (r=0.31)		(Zhao et al., 2015) non-pigmented (r=1.00) mildly-pigmented (r=0.857) non-pigmented (r=0.867)
TIS			(Zhao et al., 2015) non-pigmented (r=0.919) mildly-pigmented (r=0.829) non-pigmented (r=0.645)	(Zhao et al., 2015) non-pigmented (r=0.919) mildly-pigmented (r=0.919)
NRS worst-itch	(Silverberg et al., 2020c) white (r=0.42), non-white(r=0.48)	(Silverberg et al., 2020c) white (r=0.57) non-white (r=0.34)
NRS average-itch	(Silverberg et al., 2020c) white (r=0.48) non-white(r=0.51)	(Silverberg et al., 2020c) white (r=0.58) non-white(r=0.35)				(Topham et al., 2020 white (r=0.74) non-white (r=0.62)
DLQI	(Silverberg et al., 2020c) white (r=0.65) non-white(r=0.72)	(Silverberg et al., 2020c) white (r=0.57) non-white(r=0.48)
ItchyQoL	(Silverberg et al., 2020c) white (r=0.65) non-white(r=0.74)	(Silverberg et al., 2020c) white (r=0.68) non-white(r=0.60)
PHQ9	(Silverberg et al., 2020c) white (r=0.44) non-white(r=0.68)	(Silverberg et al., 2020c) white (r=0.50) non-white(r=0.52)
Promis SD	(Silverberg et al., 2020c) white (r=0.46) non-white(r=0.50)	(Silverberg et al., 2020c) white (r=0.41) non-white(r=0.48)
Promis SRI	(Silverberg et al., 2020c) white (r=0.41), non-white(r=0.56)	(Silverberg et al., 2020c) white (r=0.52) non-white(r=0.49)
Patient global assessment of severity						(Topham et al., 2020) white (r=0.80) non-white (r=0.63)
IGA	(Zhao et al., 2017) SOC (r=0.64) light skin (r=0.30)
Static physician global assessment						(Topham et al., 2020) white (r=0.77) non-white (r=0.73)
SASSAD			(Zhao et al., 2015) non-pigmented (r=0.786) * highly pigmented (p=0.051)	(Zhao et al., 2015) non-pigmented (r=0.786)* non-pigmented (r=0.872)	(Zhao et al., 2015) non-pigmented (r=0.793)* non-pigmented (r=0.648)	**

^*nonsignificant, not reported for highly pigmented

^**nonsignificant, not reported for mildly pigmented

Italic=significant difference between r(POEM and instrument) and r(POSCORAD and same instrument)

Bolded=same study as split cell above, but ratings were performed overseas by a different dermatologist (original study Australian, overseas=South African)

Feasibility

Three studies examined feasibility of PROMs by race.(Silverberg et al., 2020b, Silverberg et al., 2020c, Silverberg et al., 2020g) The mean time required to complete PIQ Itch-Triggers was 0.8 minutes, without significant difference by race.(Silverberg et al., 2020g) Mean time required to complete PO-SCORAD and POEM also did not significantly differ by race.(Silverberg et al., 2020c) Mean time required to complete PIQ short forms (for 4 banks) was 1.8 minutes, without significant difference by race.(Silverberg et al., 2020b)

Test-retest Reliability

One study compared test-retest reliability in whites vs. non-whites.(Silverberg et al., 2020c) Significantly higher ICC was observed for the original PO-SCORAD in whites (0.86, 95% CI 0.78-0.92) than non-whites (0.58, 95% CI 0.34-0.75). Non-significant differences were identified in the ICC for POEM in white (0.90, 95% CI 0.83-0.95) versus non-white patients (0.79, 95% CI 0.63-0.89).

Known-groups Validity

Two studies analyzed known groups validity by race.(Lei et al., 2020b, Vakharia P. P. et al., 2018) One study found that both whites and non-whites had significant associations between patient-reported AD severity and severity strata for objective-SCORAD, SCORAD, EASI, NRS-itch, POEM, and DLQI.(Vakharia P. P. et al., 2018) A study found significant and stepwise differences in PROMIS SD and SRI scores across severity groups of sleep frequency, self-reported global AD severity, and VRS worst and average itch even after adjusting for race (white/black/non-white Hispanic/Asian) in models.(Lei et al., 2020b)

ClinROMs

Convergent validity

Six studies collected data regarding convergent validity in ClinROMs.(Leshem et al., 2015a, Silverberg et al., 2020c, Topham et al., 2020, Vakharia P. P. et al., 2018, Zhao et al., 2017, Zhao et al., 2015) In a study of 291 adults with AD, the strong and significant correlation of EASI with objective-SCORAD (r>0.90) did not significantly differ across race (white, black, Hispanic, multiracial/other).(Silverberg et al., 2020c) Similarly, in another study of 170 AD patients strong and significant correlation of EASI with investigator global assessment (IGA) (r>0.90) was consistent in whites, Asians, and other race, and both Hispanic and non-Hispanic ethnicity.(Leshem et al., 2015a) A study of 388 AD patients found no significant correlations of EASI with oSCORAD in white, black, Hispanic, or multiracial individuals (p>0.05).(Vakharia P. P. et al., 2018)

Correlational effect size data was available for seven ClinROMs (EASI, oSCORAD, SGAxBSA, TIS, IGA, SPGA, SASSAD) for both white/nonwhite patients and SOC/light skin patients.(Silverberg et al., 2020c, Topham et al., 2020, Zhao et al., 2017, Zhao et al., 2015) EASI correlated weakly in white patients with both POEM (0.48) and POSCORAD (0.43). In nonwhite patients, EASI correlated moderately and weakly with POEM (0.54) and POSCORAD (0.17), respectively. All studies evaluating convergent validity of ClinROMs by race and skin tone using correlation coefficients are presented in Table 2.

Feasibility

Mean time for investigators to perform EASI was 6.0 minutes, which did not significantly differ based on patient race or Hispanic ethnicity.(Leshem et al., 2015a)

Intrarater reliability

Two studies investigated intrarater reliability across different skin pigmentation.(Zhao et al., 2017, Zhao et al., 2015) In a study of photographic assessment of 3 patients, multiple instruments had differential performance as measured by intra-class correlation coefficients (ICC) in patients with different clinician-determined skin pigmentation levels.(Zhao et al., 2015) SASSAD showed best reliability in highly (0.787) and mildly (0.458) pigmented patients while TIS (−0.434) and oSCORAD (−0.537) had the poorest reliability in these patient groups, respectively. EASI and TIS had the highest (0.83) and lowest (−0.76) ICCs in nonpigmented patients, respectively. There were no significant between or within pigmentation group differences.(Zhao et al., 2015)

Another small study (n=6) analyzed EASI, objective-SCORAD, and IGA.(Zhao et al., 2017) Patients were categorized into light skinned or skin of color (SOC) based on a machine measuring melanin content of gluteal skin.(Zhao et al., 2017) SOC patients were further stratified into intermediate or very dark skinned for certain analyses. All instruments showed excellent reliability with the lowest correlation in SOC seen for IGA (0.89) and lowest correlation in lighter skin for objective-SCORAD (0.87). No significant between-group or within-group differences were observed by pigment level. This study also reported psychometric findings for an ad-hoc scale examining greyness in SOC patients. Replacing greyscale with erythema in EASI or adding greyscale to EASI maintained comparable reliability metrics to the unmodified EASI. When SOC was stratified into intermediate- and dark-skinned subgroups and greyscale was independently evaluated, it performed best with very dark patient (0.88) and poorly in intermediate-skinned patients (0.33).(Zhao et al., 2017)

Interrater reliability

The same two studies investigated interrater reliability across different skin pigmentation.(Zhao et al., 2017, Zhao et al., 2015) In a sample of 18 patients, multiple instruments had differential performance as measured by ICC in patients with different clinician-assessed skin pigmentation levels. In highly pigmented individuals, EASI showed best ICC (−0.054) and TIS showed poorest ICC (−0.21). In mildly pigmented patients, oSCORAD showed best ICC (0.588) and SASSAD showed worst ICC (0.341). In non-pigmented patients, SASSAD showed best ICC (0.667) and TIS showed worst ICC (0.403). TIS, SASSAD, EASI, and oSCORAD had poor ICC in highly pigmented fair ICC in mildly pigmented patients. In non-pigmented patients, TIS and oSCORAD had fair correlations; EASI and SASSAD had good ICC.(Zhao et al., 2015)

EASI, objective-SCORAD, IGA and Greyscale from 11 lighter skin and 14 SOC patients were evaluated for inter-rater reliability.(Zhao et al., 2017) SOC was further stratified into intermediate and very dark patients. In lighter skin patients, EASI (0.83) and objective-SCORAD (0.68) were the best and worst performing instruments, respectively. EASI was also the best performing instrument for all categories of SOC, displaying best performance in the darkest skinned patients (SOC overall: 0.77, intermediate: 0.67, very dark 0.84).

Greyscale alone was the poorest performing instrument for all SOC categories, but in SOC patients when it replaced erythema in EASI (0.78) or was added to EASI (0.79) it did not produce significantly better results compared to original EASI. Otherwise, IGA was the poorest performer in all categories of SOC, especially for intermediate patients (0.56).(Zhao et al., 2017)

Although each instrument performed differently for each pigmentation level, these cross-racial differences were not statistically significant, likely due to the very small study sample sizes. EASI items assessing components other than erythema contributed significantly greater variability (assessed by coefficient of variance) to EASI inter-rater reliability scores than erythema itself in lighter skin patients. In SOC, variability in EASI inter-rater reliability was not significantly influenced by presence of erythema nor greyness items of EASI. Variability in oSCORAD inter-rater reliability was not explained by erythema in any pigmentation group.(Zhao et al., 2017)

DISCUSSION

This systematic review reveals significant reporting and knowledge gaps with respect to psychometric properties of outcome measures by race, ethnicity or skin tone in AD. The included studies ranged in quality, and only 47% or 18% of measurement properties evaluated using COSMIN and NOS scales were rated as very good or good, respectively (Tables 3 and 4). Most validation studies did not even mention the distribution of race, ethnicity or skin tone in their studies. The few studies examining psychometrics by race, ethnicity or skin tone used disparate methodologies to assess race, ethnicity and skin tone. Some studies of ClinROMs suffered from very small samples. Additionally, many studies that assessed psychometrics of outcome measures in non-white populations were performed in homogeneous patient populations in different countries. However, little is known about how well these measures perform, particularly ClinROMs, in diverse patient populations. There is insufficient evidence to determine optimal PROM and ClinROMs for SOC. Since AD may manifest differently in SOC, potential shortcomings of outcome measures can impact clinical care and research.(Brunner and Guttman-Yassky, 2019)

Table 3:

COSMIN Risk of Bias Scores

Last name	Year published	Box 1	Box 2	Box 3	Box 4	Box 5	Box 6	Box 7	Box 8	Box 9	Box 10
Dauden(Daudén et al.)	2011	-	-	1	1	2	1	-	-	-	-
Giovannini(Giovannini et al., 2020)	2020	1	-	-	-	-	1	-	1	-	-
Zhao(Zhao et al., 2017)	2016	-	-	-	-	2	1	-	3	1	-
Whalley(Whalley et al.)	2004	2	-	1	1	-	2	-	-	1	1
Zhao(Zhao et al., 2015)	2015	-	-	-	-	-	1	-	-	0	-
Borzutsky(Borzutzky et al., 2014)	2014	-	-	-	-	-	3	-	-	-	-
Beattie(Beattie and Lewis-Jones)	2006	-	-	-	-	-	-	-	-	2	-
Coutanceau(Coutanceau and Stalder, 2014)	2014	-	-	-	-	-	-	-	-	1	-
Marinello(Marinello et al., 2016)	2016	-	2	-	-	-	2	-	-	2	-
Bozek(Bożek and Reich, 2017)	2017	-	-	-	-	-	2	-	-	-	-
Silverberg(Silverberg et al., 2020g)	2020	-	-	-	-	-	-	-	-	-	-
Taieb(Taïeb et al., 2015)	2015	2	3	-	3	2	2	-	-	2	-
Meni(Méni et al., 2013)	2013	2	3	-	2	2	2	-	3	3	-
Darrigade(Darrigade et al., 2020)	2018	-	-	-	-	-	-	-	1	1	-
Arents(Arents et al., 2019)	2019	2	-	1	1	-	1	-	-	1	-
Twiss(Twiss et al.)	2012	-	-	2	-	-	-	-	-	-	-
Mitchell(Mitchell et al., 2017)	2016	2	-	3	3	-	-	-	-	2	-
Chamlin(Chamlin et al.)	2007	-	-	-	-	-	2	-	1	-	3
Aziah(Aziah et al.)	2002	-	-	-	-	-	2	-	-	-	-
Kunz(Kunz et al., 1997)	1997	-	-	-	-	-	4	-	-	-	-
Silverberg(Silverberg et al., 2020c)	2020	-	-	-	-	-	2	-	2	2	2
Chernyshov(Chernyshov et al., 2015)	2015	-	-	-	-	-	-	-	-	3	-
Silverberg(Silverberg et al., 2018a)	2018	2	-	-	3	2	-	-	2	2	-
Chernyshov(Chernyshov et al.)	2018	-	-	-	-	-	-	-	-	-	-
Bruscky(Bruscky et al., 2017)	2017	-	-	-	4	-	2	-	-	1	-
Alzolibani(Alzolibani)	2013	-	-	-	3	-	2	-	-	1	-
Finlay(Finlay and Khan)	1994	3	-	-	4	-	2	-	-	-	-
Willemsen(Willemsen et al., 2009)	2009	-	-	-	-	-	1	-	-	2	-
Foley(Foley et al.)	2019	2	1	-	-	-	0	-	-	-	-
Heckman(Heckman C.J. et al., 2020)	2020	-	-	-	3	-	3	-	3	2	-
He(He et al.)	2014	3	-	2	2	-	2	-	2	2	-
Heckman(Heckman CJ et al., 2020)	2020	3	-	-	-	-	3	-	2	3	2
Pereyra-Rodriguez(Pereyra-Rodriguez et al., 2019)	2019	4	-	-	1	-	3	-	-	3	-
Cheng(Cheng et al., 2020)	2020	-	-	-	-	-	-	-	1	1	1
Yamaguchi(Yamaguchi et al.)	2016	4	-	4	1	-	3	-	1	-	-
Gabes(Gabes et al., 2020a)	2020	1	3	1	1	-	2	2	-	-	-
Chamlin(Chamlin et al.)	2005	1	3	1	1	-	-	-	1	-	-
Schram(Schram et al.)	2011	-	-	-	-	-	-	3	-	-	-
Tremp(Tremp et al., 2011)	2011	-	-	-	-	-	-	3	-	-	-
Gabes(Gabes et al., 2020b)	2019	-	-	-	-	-	-	-	-	-	-
Newton(Newton et al., 2019)	2019	-	1	1	-	-	-	3	-	-	-
Augustin(Augustin et al., 2004)	2004	-	-	1	-	-	-	3	-	-	-
Charman(Charman and Varigos, 1999)	1999	-	-	1	-	-	-	-	-	-	-
Chernyshov(Chernyshov, 2016)	2016	1	-	2	1	-	1	-	-	-	-
Howells(Howells et al., 2018)	2018	-	-	1	-	-	-	3	-	-	-
Gabes(Gabes and Apfelbacher, 2020)	2020	-	1	-	-	-	-	-	-	-	-
Chren(Chren et al.)	1997	-	1	2	1	-	2	1	2	2	2
Topham(Topham et al., 2020)	2020	1	-	2	-	2	-	-	-	2	2
McKenna(McKenna et al.)	2005	1	2	2	3	-	3	-	3	4	-
Neri(Neri et al.)	2012	-	-	2	3	-	-	1	-	-	-
Hon(Hon et al.)	2006	-	-	-	-	-	-	-	2	-	-
Holm(Holm et al.)	2006	-	-	-	-	-	-	3	-	-	-
Charman(Charman et al.)	1999	-	-	-	-	-	2	-	-	-	-
Herd(Herd et al.)	1997	-	-	-	-	-	-	-	3	-	-
Lei(Lei et al., 2020a)	2020	-	-	-	-	2	2	-	2	2	-
Silverberg(Silverberg et al., 2019a)	2019	-	-	-	2	2	-	-	-	3	-
Silverberg(Silverberg et al., 2020a)	2020	-	2	-	2	2	2	2	2	3	-
Silverberg(Silverberg et al., 2020b)	2020	-	2	-	2	2	-	-	2	3	-
Silverberg(Silverberg et al., 2020d)	2020	-	-	-	-	-	1	2	1	3	1
Silverberg(Silverberg et al., 2020e)	2020	-	-	-	-	3	1	2	1	2	2
Patel(Patel et al., 2019)	2019	-	2	-	2	1	1	-	-	2	1
Charman(Charman et al.)	2005	-	-	-	-	-	-	-	2	-	-
Badia(Badia et al.)	1999	-	3	-	3	-	2	-	-	-	1
Cheng(Cheng et al., 2019)	2019	-	-	-	3	-	2	-	-	3	-
Darsow(Darsow et al.)	2001	-	-	2	-	-	-	-	-	-	-
Kido-Nakahara(Kido-Nakahara et al., 2020)	2020	-	-	-	-	-	2	-	-	3	-
Stalder(Stalder et al.)	2011	-	-	-	-	4	-	-	-	1	-
Vourch'h-Jourdain(Vourc'h-Jourdain et al., 2009)	2009	3	2	-	-	-	-	-	-	1	-
Fishbein(Fishbein et al., 2020)	2020	-	-	-	-	-	4	-	-	1	-
Yosipovitch(Yosipovitch et al., 2019)	2019	-	1	-	-	-	1	1	-	1	1
Deon(Deon et al.)	2011	-	-	-	1	-	1	-	-	1	-
Suh(Suh et al., 2020)	2020	-	-	-	-	-	-	-	-	1	-
Van Valburg(van Valburg et al.)	2011	-	-	-	-	-	1	-	-	1	-
Bahmer(Bahmer et al.)	1991	-	-	-	-	-	-	-	-	-	-
Liu(Liu et al.)	2016	-	-	1	-	-	-	-	-	-	-
Chopra(Chopra et al.)	2017	-	-	-	-	-	-	-	-	-	-
Lee(Lee et al., 2018)	2018	-	-	-	3	-	3	4	1	-	1
Charman(Charman et al.)	2002	-	-	-	-	-	2	-	-	-	-
Mazzotti(Mazzotti et al., 2008)	2008	-	-	-	-	-	3	-	-	1	-
Simpson(Simpson et al., 2019)	2019	-	-	-	-	-	-	-	-	-	-
Pucci(Pucci et al.)	2005	-	-	-	3	-	-	-	-	-	-
Costa(Costa et al., 1989)	1989	-	-	-	-	-	4	-	-	2	-
Oranje(Oranje et al.)	1997	-	-	-	-	-	-	-	-	-	-
Wolkerstorfer(Wolkerstorfer et al.)	1999	-	-	-	-	-	1	-	-	1	-
Murao(Murao et al.)	2004	3	-	-	-	-	1	-	-	-	-
Sprikkelman(Sprikkelman et al.)	1997	-	-	-	-	-	-	1	-	-	-
Rullo(Rullo et al.)	2008	-	-	-	1	-	3	-	-	-	1
European Task Force on Atopic Dermatitis(1993)	1993	3	-	-	-	-	-	-	-	-	-
Chopra(Chopra et al., 2017)	2017	-	-	-	-	-	-	-	-	-	-
Vakharia(Vakharia PP et al., 2018b)	2018	-	-	-	-	-	-	-	-	-	-
Silverberg(Silverberg et al., 2018c)	2018	-	-	-	-	-	-	-	-	-	-
Berth-Jones(Berth-Jones)	1996	-	-	-	-	-	-	3	-	-	-
Baek(Baek et al., 2015)	2015	-	-	-	-	-	2	-	-	1	1
Martin(Martin et al., 2020)	2020	1	1	-	-	-	-	-	-	-	-
Carel(Carel et al.)	2008	-	-	-	-	-	-	-	-	1	-
Lewis-Jones(Lewis-Jones and Finlay)	1995	3	-	-	-	-	2	-	-	-	-
Charman(Charman et al., 1999)	1999	-	-	-	-	-	1	-	-	-	-
Hanifin(Hanifin et al.)	2001	-	-	-	-	-	1	-	-	-	-
Baars(Baars et al.)	2005	3	1	1	1	-	1	-	-	-	-
Lewis-Jones(Lewis-Jones et al.)	2001	3	-	-	-	-	1	-	-	1	1
Silny(Silny et al., 2005)	2005	4	-	-	-	-	-	-	-	-	-
Emerson(Emerson et al.)	2000	-	-	-	-	-	-	-	-	1	-
Angelova- Fischer(Angelova-Fischer et al.)	2005	-	-	-	-	-	-	-	-	-	-
Sugarman(Sugarman et al.)	2003	-	-	-	-	-	-	-	-	-	-
Gaunt(Gaunt et al., 2016)	2016	-	-	-	-	-	-	-	-	-	1
Charman(Charman et al.)	2004	1	1	-	1	-	1	1	-	1	-
Yew(Yew et al., 2020)	2020	-	-	-	-	-	-	-	-	-	-
Zhao(Zhao et al., 2015)	2015	-	-	-	-	1	1	-	-	1	-
Van Velsen(van Velsen et al.)	2010	-	-	-	-	-	-	-	-	1	-
Simpson(Simpson et al., 2020)	2020	-	1	-	-	-	1	-	-	-	-
Boleira(M et al., 2014)	2014	-	3	4	4	-	-	-	-	1	-
Housman(Housman et al.)	2002	-	-	-	-	-	-	-	-	-	-
Ganemo(Gånemo et al., 2016)	2016	-	-	-	1	-	-	-	-	1	1
Hon(Hon et al., 2019)	2019	-	-	-	-	-	-	-	-	1	-
Silverberg(Silverberg et al., 2019b)	2019	-	-	-	1	1	-	-	-	1	-
Ramirez-Anaya(Ramírez-Anaya et al.)	2010	-	4	-	1	-	2	-	1	-	-
Hon(Hon et al.)	2003	-	-	-	-	-	-	-	-	-	-
Silverberg(Silverberg et al., 2020h)	2020	-	-	-	1	1	-	-	1	1	-
Lei(Lei et al., 2020b)	2019	-	-	-	-	-	-	-	1	1	1
Vakharia(Vakharia PP et al., 2018a)	2018	-	-	-	-	-	-	-	-	1	1
Lei(Lei et al., 2020b)	2020	2	1	2	1	2	1	-	1	1	1
Udkoff(Udkoff and Silverberg, 2018)	2018	1	1	2	1	2	-	-	-	1	1
Chernyshov(Chernyshov et al., 2019)	2019	1	-	1	1	1	1	-	-	1	1
Barbier(Barbier et al.)	2003	-	-	-	1	-	1	3	1	1	1
Faye(Faye et al., 2020a)	2019	-	-	-	-	-	-	-	-	-	-
Mohammadi(Mohammadi et al., 2019)	2019	-	-	-	2	-	3	-	-	-	-
Silverberg(Silverberg et al., 2020f)	2020	-	-	-	-	-	-	-	-	-	-
Leshem(Leshem et al., 2015b)	2015	-	-	-	-	-	-	-	-	-	-
Charman(Charman et al., 2013)	2013	-	-	-	-	-	-	-	-	-	-
Hurault(Hurault et al., 2018)	2018	-	-	-	-	-	-	-	-	-	-

-, not assessed in paper

1, very good

2, adequate

3, doubtful

4, inadequate

Table 4:

Newcastle Ottawa Scale Scores

First Author	Year	Elements of Newcastle-Ottawa Scale
		Selection (0-5)	Comparability (0-1)	Outcome (0-3)	Total stars (0-9)
Cherynyshov(Chernyshov et al., 2013)	2013	2	1	3	6
Holm(Holm et al.)	2006	3	1	3	7
Hassab-el-Naby(Hassab-El-Naby et al.)	2009	2	0	2	4
Rea(Rea et al., 2018)	2018	2	0	3	5
El-Mongy(El-Mong et al.)	2006	2	1	3	6
Ring(Ring et al.)	2019	1	0	3	4
Hon(Hon et al., 2006)	2006	2	0	3	5
Cosickic(Cosickic et al., 2010)	2010	2	0	3	5
Al-buanain(Al-Buainain et al.)	2009	2	0	3	5
Hon(Hon et al.)	2008	2	0	3	5
Shim(Shim et al.)	2011	2	0	3	5
Yang(Yang et al.)	2010	2	0	3	5
Schafer(Schäfer et al.)	1997	2	1	3	6
Djurovic(Ražnatović Djurović et al., 2015)	2015	2	0	3	5
Maksimovic(Maksimović et al.)	2012	0	0	3	3
Andersen(Andersen et al., 2020)	2020	3	1	3	7
Campos(Campos et al., 2017)	2017	1	0	3	4
Alzolibani(Alzolibani, 2014)	2014	2	0	3	5
Weisshaar(Weisshaar et al., 2008)	2008	0	1	3	4
Vilsboll(Vilsbøll et al., 2020)	2020	5	0	3	8
Haeck(Haeck et al.)	2012	0	0	3	3
Ben-Gashir(Ben-Gashir et al.)	2004	3	1	3	7
Kim(Kim et al.)	2012	2	0	3	5
Lam(Lam)	2010	3	1	3	7
Mozaffari(Mozaffari et al.)	2007	0	1	3	4
Amaral(Amaral et al.)	2012	2	0	3	5
Pusitsek(Pustišek et al., 2016)	2016	2	1	3	6
Djurovic(Djurović et al., 2020)	2020	2	0	3	5
Ganemo(Gånemo et al., 2007)	2007	2	0	3	5
Monti(Monti et al.)	2011	2	0	3	5
Hachisuka(Hachisuka et al.)	2009	2	0	3	5
Verwimp(Verwimp et al.)	1995	0	0	1	1
Al Shobaili(Al Shobaili)	2010	4	0	3	7

7-9 stars=high quality, 4-6 stars=fair quality, 0-3 stars=poor quality.

Significant DIF by race was found for one or more item of PO-SCORAD, PIQ-Short Forms, POEM, DLQI, HADS, ItchyQoL, 5D-itch, SF-12, NRS-itch. For DLQI, in particular, item 2, “…how embarrassed or self-conscious have you been because of your skin?” and item 10, “…how much of a problem has the treatment for your skin been, for example by making your home messy, or by taking up time?” exhibited significant DIF by race in two studies.(Patel et al., 2019, Silverberg et al., 2019b) Nonuniform DIF may be corrected by removal of the item in question.(Pallant and Tennant, 2007) Alternatively, DLQI scores should be examined separately for patients in different racial/ethnic subgroups and should not be pooled together. This is concerning because DLQI was recently selected as the preferred outcome measure to assess QoL in AD. Though, it may not be optimal for use in diverse populations and non-white patients. PROMIS-SD and SRI were investigated in two studies; both showed no DIF for any items of either instrument.(Lei et al., 2020a, Lei et al., 2020b) These newer scales appear well suited to measure sleep specific impairment across races/ethnicities in AD.

Correlations of POEM with PHQ9 showed greatest differences between white and nonwhite patients (Δrho=0.24); correlations were moderate for non-whites and weak for whites.(Silverberg et al., 2020c) Subjective disease severity (POEM) may be more closely associated with the construct of depression (PHQ9) in nonwhite vs. white patients. Correlations of POEM with IGA differed the most between SOC and light skin (Δrho=0.34); the correlation was moderate for SOC and weak for light skinned patients.(Zhao et al., 2017) This suggest that IGA may better align with symptoms in SOC patients. Though, these observations should be interpreted with caution as they are from a single small-scale study.

Similar strengths of correlation (convergent validity) were observed for POEM with DLQI between white and nonwhite patients(Silverberg et al., 2020c) and POEM with EASI between white and nonwhite(Silverberg et al., 2020c) and light skin and SOC(Zhao et al., 2017) patients. These preliminary findings support the recommendations of the HOME to include POEM, EASI, and DLQI in the core outcome set of clinical trials. However, there is currently insufficient evidence to be confident about how well any PROMs or ClinROMs perform across different races, ethnicities, or skin tones in AD.

Differences of test-retest reliability by race were only tested for POEM and PO-SCORAD.(Silverberg et al., 2020c) The original PO-SCORAD showed significant test-retest reliability differences between white and nonwhite patients; an observation not seen for POEM.(Silverberg et al., 2020c) Recently, an updated version of PO-SCORAD was developed for black skin to address these limitations in SOC.(Faye et al., 2020b) There were no test-retest evaluations on the basis of race or skin tone for any other instruments.

EASI, SCORAD and other ClinROMs assess erythema using a spectrum of pinks and reds. In order to address difficulties with pigment masking of erythema in SOC, it is recommended by the HOME group to upcode erythema scores by one-level in SOC.(2017) It is unclear whether this guidance was used in the included studies of ClinROMs or whether such guidance is sufficient for properly assessing SOC. Erythema often appears in shades of purples and browns in AD patients with SOC, and can be difficult to assess reliably when patients have profound xerosis and lichenification. Results from a single small sample study suggest the erythema component of EASI can be supplemented with or replaced by an identically assessed clinical sign of greyness without compromising reliability; however, such a modification did not significantly improve EASI’s reliability,(Zhao et al., 2017) and has not undergone comprehensive psychometric testing. Guidance on how to score erythema and other AD signs should be expanded to address AD manifestations in SOC.

Race and ethnicity were inconsistently defined in included papers, presenting a barrier to standardizing categories for between-group analysis. Clinician assumptions about racial and ethnic identity may also be spurious. More importantly, different non-white races and ethnicities span a wide range from light to very dark skin tones.(Jothishankar and Stein, 2019) Thus, demonstration of ClinROM validity and reliability across different races does not mean that those tools perform well across all skin tones. It also does not obviate the need for studies specifically examining the psychometrics of ClinROMs across different skin types. Some validation studies included patients from one specific region or nation (e.g. Saudi Arabian children, Chinese adults). It is unclear whether these cohorts were diverse with respect to race, ethnicity or skin tone, and consequently whether the results are generalizable to other regions.

This study has several strengths, including comprehensive search strategy across multiple databases. Limitations include the relative dearth of studies specifically examining psychometrics across different races, ethnicities or skin types and the analysis of full-texts only accessible in English. Some studies of ClinROMs were underpowered to detect meaningful differences between patient subgroups. There were insufficient data to perform meta-regression of differences between patient subgroups. More studies are sorely needed to address these limitations.

METHODS

This study was exempt from institutional review board (IRB) approval because it only included published literature. This review was registered with PROSPERO (CRD42021239614). The review was carried out in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.(Liberati et al., 2009)

Literature Search

The following databases were searched through October 31, 2020: MEDLINE via PubMed (1996-present) and EMBASE (1947-present) for AD terms combined with terms regarding psychometric properties. Search strategies were modified from those previously used HOME group 2013 and 2019 systematic reviews of clinical sign instruments and quality-of-life instruments, respectively(Gabes et al., 2020c, Schmitt et al., 2013) (Appendix 1 and 2). The clinical signs search dates were (10-02-2012 to 10-30-2020) and the quality of life search dates were (1-23-2019 to 10-30-2020). Inclusion criteria are presented in Appendix 3.

Data Extraction

At least two authors (A.G., D.A., E.C., T.K, U.R) independently screened title/abstract and full-text articles, performed data extraction in duplicate, and performed quality or risk of bias assessment. Inconsistencies were resolved by consensus discussion. Additional references were identified by snowballing of all included articles, as well as excluded systematic reviews.

Data extracted included study design, validation or non-validation (such as correlational) study, demographic information, instruments analyzed, study participant breakdown by race, ethnicity, and skin type; and psychometric properties including content validity, construct validity, convergent validity, concurrent validity, inter-rater reliability, intra-rater reliability, responsiveness, and interpretability.

Quality and Risk of Bias Assessment

COSMIN Risk of Bias checklist was used to assess the methodological quality of validation studies. Newcastle Ottawa Scale (NOS) cross sectional adaptation was used to assess quality of non-validation non-randomized studies.(Mokkink et al., 2018),(Wells) We determined, a priori, that an NOS score of 7-9 stars would be considered high quality, 4-6 stars fair quality, and 0-3 stars poor quality.

Data availability:

All data were collected from the published literature. No datasets were generated during the current study.