Validation of a Patient Global Assessment for extent, severity and impact to define the severity strata for the Self Assessment Vitiligo Extent Score (SA‐VES)

Abstract

Background

The Self Assessment Vitiligo Extent Score (SA‐VES) is a validated, patient‐reported outcome measure to assess the body surface area affected with vitiligo. Information on how to translate the obtained score into extent, severity and impact strata (mild–moderate–severe) is still lacking. Stratification is helpful to define inclusion criteria for trials, enables comparison and pooling of trial results and can be used for epidemiological research.

Objectives

The aim was to develop extent, severity and impact strata for the SA‐VES based on validated anchor‐based questions.

Methods

In total, 315 patients with vitiligo (non‐segmental; age ≥ 16) recruited at the Ghent University Hospital (Belgium) completed a questionnaire that was conducted in cooperation with the Dutch Society for vitiligo patients to ensure content validity. First three anchor questions included in the questionnaire [Patient Global Assessment (PtGA) for vitiligo extent, severity and impact] were assessed for content validity, construct validity and intrarater reliability. Subsequently, the PtGAs were used to stratify the SA‐VES based on ROC analysis.

Results

For all PtGAs (PtGA extent, PtGA severity, PtGA impact), at least 75% of hypotheses evaluated for construct validity were confirmed. Intrarater reliability of all PtGAs was good to excellent (ICCs PtGA extent: 0.623; PtGA severity: 0.828; PtGA impact: 0.851). The optimal cut‐off values of the SA‐VES between the three global categories (mild/limited – moderate – severe/extensive) were 1.05% and 6.45% based on PtGA extent, 2.07% and 4.8% based on PtGA severity and 2% and 3.35% based on PtGA impact.

Conclusion

This study provides the first guide for the interpretation of the numerical output obtained by the SA‐VES (vitiligo extent) and enables the translation into a global vitiligo grading for extent, severity and impact. As patients’ interpretation of vitiligo extent, severity and impact may vary amongst patients worldwide, future international studies will be required.

Background

The Self Assessment Vitiligo Extent Score (SA‐VES) is a validated, patient‐reported outcome measure for monitoring vitiligo extent. It is based on an assessment of the affected body surface area (BSA) which is considered to be a relevant outcome in the assessment of vitiligo severity. It is a patient‐reported version of the Vitiligo Extent Score (VES) for physicians. ¹ Information on how to translate the obtained VES scores into disease severity strata from the physicians’ point of view has been described recently while this is still lacking for the patient‐reported version (SA‐VES). ² Stratification can be used to define inclusion criteria for clinical trials, epidemiological research and is important for correct comparison and pooling of trial results. The process to stratify the numeric output of an instrument into categories (mild–moderate–severe) is usually based on an ‘anchor question’ (e.g. Patient Global Assessment) that is preferably validated in advance. ³ , ⁴ A validated Patient Global Assessment for vitiligo is, however, lacking. A validated Global Assessment instrument could also be used as an anchor question for the interpretation and stratification of other outcome measure in vitiligo ([e.g. Dermatology Life Quality Index (DLQI), Vitiligo Impact Patient scale (VIPs)].). ⁵ , ⁶ , ⁷

The first aim of this study was to assess the content validity, construct validity and intrarater reliability of a Patient Global Assessment (PtGA) for disease extent, severity and impact. The second aim was to determine strata (mild–moderate and severe) for the SA‐VES based on the validated PtGA for extent, severity and impact.

Materials and methods

Study design and ethics

In this observational study, patients with vitiligo (non‐segmental) age ≥16 were consecutively recruited at the Ghent University Hospital (Belgium) (October 2017 beginning of October 2019) and were asked to complete a Dutch questionnaire including the SA‐VES as well as a 5‐point global assessment scale for extent (PtGA extent), severity (PtGA severity) and impact (PtGA impact) (Fig. 1).

Figure 1.

Translations Patient Global Assessments from original Dutch version. Remark related to Patient Global Assessment for severity: the wording “skin colour” was added at the last stage of the translation but was not included in the original Dutch version. Within the Dutch version we considered this to be clear as an additional question related to different skin types “(I‐VI)” Median SAVES scores for each PtGA forwas included in the same questionnaire in the majority of cases. In a minority of cases skin type was assessed by the physician for instance if the self assessment question related to skin type was not included in the questionnaire.”

In the preparative stage of this study, the PtGA instruments were evaluated and modified by several members of the Dutch Society for Vitiligo patients (vitiligo.nl) to ensure the content validity. The patients’ global assessment instruments were subsequently evaluated by asking the relevance, completeness and comprehensibility of the items included. The questions used to assess the Patients Reported Global Assessment were pilot tested at the department of dermatology in Ghent. The COSMIN checklist was used as a guidance for designing and reporting our study. ⁸ , ⁹ This study was approved at the local ethics committee (reference number: B670201421409), and written informed consent was obtained from all patients who completed the questionnaire. The cooperation with the Dutch Society for vitiligo patients (Vitiligo.nl) was reported at the Amsterdam Medical Center (W17_355#17.413).

Validity and reliability

Construct validity of the PtGA instruments was evaluated by testing at least four predefined hypotheses by Spearman correlations with other PROMs (e.g. SA‐VES, DLQI, impact score 0–10 and the PtGA scores included in this study) (Appendix S1). ¹⁰ As an impact score from 0 to 10 was also included to evaluate construct validity of the PtGA score, this impact score 0–10 was also validated for use (Appendix S1). To evaluate the intrarater (test–retest) reliability of the PtGA instruments and impact score 0–10, a subgroup of patients was asked to complete the questionnaire twice with an interval of 2 weeks. To increase the usability of the PtGA in an international setting, all related questions were translated (including 2 forward and 2 backward translations) in English (Fig. 1) by a professional translation agency (ElaN Languages, Heusden‐Zolder, Belgium) following the instructions for translations of measurement instruments as a guidance. ¹¹ All questions (PtGA and impact score 0–10; including translated version) used for this study are available on request at the corresponding author.

Stratification of affected self‐assessed body surface area (SA‐VES)

In the second part of this study, the validated PtGA for extent, severity and impact (Fig. 1) was used as anchor questions to stratify the SA‐VES. By this anchor‐based approach, the SA‐VES outcomes were compared (anchored) to the results of Patient Global Assessments collected at the same time. By defining the thresholds based on ROC analyses, the SA‐VES could be stratified into global strata (mild–moderate–severe).

Statistics/data analysis

Statistical analyses were performed using SPSS 25.0 (SPSS Science, Chicago, IL, USA). The intrarater agreement of the PtGA scores was calculated by ICC and reported as single measures [two way mixed, absolute agreement]. The following guidelines for the interpretation of the ICC were used as follows: below 0.4 considered as poor, between 0.4 and 0.59 as fair, between 0.6 and 0.74 as good, and ≥0.75 as excellent. A cut‐off point of a Spearman’s correlation of at least 0.4 was used in all hypotheses used to test construct validity (Appendix S1). Paired analysis was performed using Wilcoxon signed‐rank test. Global strata (cut‐off points) for the SA‐VES were based on Youden’s index assessed by ROC analyses using MedCalc v19.2.5. software (Medcalc, Mariakerke, Belgium). MedCalc was also used to assess the 95% CI for the median of the SAVES scores per severity strata for each PtGA (extent, severity and impact). To check the degree of agreement between the obtained severity strata (ranges of cut‐off points) and the PtGAs, the intraclass correlation coefficients (ICCs) were subsequently assessed. Missing values were excluded from the final analysis. For the DLQI score, if some questions were left unanswered this was scored 0. However, the questionnaire was not scored in case the questionnaire was considered not to be finalized by the patient (e.g. half of all questions left unanswered). In all cases, significance level was set at P < 0.05.

Results

Preparative stage (content validity PtGAs)

A draft version of the PtGA questions was thoroughly checked and modified multiple times according to the patients' preferences (7 members of Dutch society for vitiligo patients), based on completeness, comprehensibility and relevance. Based on the patient’s comments, the question that introduced the rating of the patients’ global assessment of severity (PtGA severity) was modified to a more detailed and comprehensible version for patients in order to rate their vitiligo severity without further explanation (Fig. 1). This detailed version illustrated more the different dimension of the PtGA severity compared to the PtGA for impact and extent. The question included in the global impact score was clarified by adding the word ‘influence’.

Reliability and construct validity of PtGA scores

In total, 315 patients (age ≥ 16) were included [male/female 43.5%/56.5%; mean age (±SD) at inclusion was 40 ± 14 years, range 16–73 years]; photo skin types: I (1.4%, 4/289), II (36%, 104/289), III (48.1%, 139/289), IV (12.1%, 35/289) and V (2.4%, 7/289); median (mean) BSA (SA‐VES) score was 1.65% (4.27%) (range 0.04–73.88%); and median DLQI score was 2 (range 0–21).

Supporting evidence for construct validity was provided for PtGA extent, PtGA severity, PtGA impact and impact score 0‐10 as at least 75% of hypotheses (Appendix S1) per instrument were confirmed (Table 1). All Spearman correlations included in the hypotheses were significant, although 2 did not reach the cut‐off level of 0.4. Highest correlation coefficients for the global assessment of severity (PtGAs) were observed with PtGA impact (r = 0.729). The different dimension of the PtGA severity compared to the PtGA impact was, however, confirmed by a paired analysis showing a highly significant difference (P < 0.001). In total, 111/287(38.7%) patients rated impact and severity differently with most patients reporting a higher severity compared to impact (75/111, 67.6%). Not surprisingly, the correlation between DLQI was strongest for PtGA impact (r = 0.598), followed by PtGA severity (r = 0.489) and PtGA extent (r = 0.411). For extent (PtGA extent), highest correlation coefficients were observed with the SA‐VES (r = 0.633) followed by PtGA severity (r = 0.586).

Table 1.

Correlations including those used for the construct validity testing of PtGAs and impact score 0–10

Spearman’s correlations (rho) Included in the construct validity test	PtGA extent	PtGA severity	PtGA impact	DLQI	SA‐VES	Impact score 0‐10
PtGA extent	NA	0.586*	0.479*	0.411*	0.633*	0.455
PtGA severity	0.586*	NA	0.729*	0.489*	0.442*	0.712
PtGA impact	0.479*	0.729*	NA	0.598*	0.359*	0.876
Impact score 0‐10	0.455*	0.712*	0.876*	0.601*	0.350*	NA

^*Significant at 0.01 level; boxes in light grey: correlations not included in the hypotheses used for construct validity testing. Cut‐off level for rho’s correlation of 0.4 not reached for two correlations. Number of patients included for the correlation test ranged between 199 and 291.

Fifty patients were included for the test–retest study. Test–retest results of the PtGA extent [n = 50; ICC = 0.623 (95% CI: 0.418–0.768)], PtGA severity [n = 50; ICC = 0.828 (95% CI: 0.716–0.898)], PtGA impact [n = 48; ICC = 0.851 (95% CI: 0.749–0.914)] and impact score 0‐10 [n = 34; ICC = 0.872 (95% CI: 0.760–0.934)] demonstrated all good to excellent test–retest intraclass correlation coefficients.

Stratification of SA‐VES

Median estimation of the PtGA severity was scored as ‘moderate’. Most patients (mode) reported ‘limited extent’ of their vitiligo (58.3%, 169/290). About 207 patients (72.1%, 207/287) reported a mild‐to‐moderate impact of vitiligo, 53 (18.5%, 53/287) reported severe‐to‐very severe influence of vitiligo on their quality of life. Table 2 shows the optimal cut‐off points and confidence interval of the SA‐VES per three categories based on the respective anchor questions (PtGA extent, PtGA severity, PtGA impact). The optimal cut‐off values of the SA‐VES between the three global categories (1. mild/limited, 2. moderate and 3. severe/extensive) were 1.05% and 6.45% based on PtGA extent, 2.07% and 4.8% based on PtGA severity and 2% and 3.35% based on PtGA impact. ICCs between obtained cut‐off point ranges and PtGAs for extent, severity and impact are included in Table 2.

Table 2.

Cut‐off points/severity strata for the SA‐VES into three categories

	Affected body surface area % (SA‐VES)
PtGA extent	No vitiligo any more– limited	Moderate		Extensive – very extensive
Range	≤1.05%	>1.05%–6.45%		>6.45%
Cut‐off ICC = 0.642		1.05% (0.91–2.78)		6.45% (3.13–10.83)
PtGA severity	Not severe at all– mild	Moderate		Severe – very severe
Range	≤2.07%	>2.07%–4.8%		>4.8%
Cut‐off ICC = 0.386		2.07%(1.32–4.47)		4.8 %(2.5–10.3)
PtGA impact	No impact – mild impact*	Moderate impact**		High impact‐ very high impact***
Range	≤2%	>2%–3.35%		>3.35%
Cut‐off ICC = 0.311		2% (0.61–3.99)		3.35% (0.25–6.15)

ICC, intraclass correlation between obtained ranges per strata and PtGAs; PtGA, Patients reported global assessment; SA‐VES, Self Assessment Vitiligo Extent Score. *In the questionnaire the wording was "no influence at all ‐ little influence". **In the questionnaire the wording was "moderate influence".***In the questionnaire the wording was "severe influence ‐ very severe influence".

Box plots for PtGA for extent, severity and impact representing three categories (mild – moderate – severe) based on SA‐VES are presented in Fig. 2a–c.

Figure 2.

(a) Box plots for SA‐VES representing three categories (limited – moderate – extensive) based on PtGA extent. (b) Box plots for SA‐VES representing three categories (mild – moderate – severe) based on PtGA severity. (c) Box plots for SA‐VES representing three categories (little – moderate – severe) based on PtGA impact. (d) Box plots for SA‐VES representing three categories (limited‐moderate‐extensive) based on PtGA extent on a different scale (Y axis on 40). The first category (1) in the boxplots of each PtGA also includes "no vitiligo any more/not at all severe/no influende at all" and the third (3) category also includes "very extensive/very severe/very severe influence".

The median, the IQR and 95% CI of the median, of the SA‐VES per category of the PtGA extent, PtGA severity and PtGA impact are presented in Table 3.

Table 3.

Median SAVES scores for each PtGA for 3 and 4 categories of extent, severity and impact.

PtGA categories	N	Median SAVES (%)	IQR	95% CI for the median †
PtGA extent
No vitiligo‐limited	163	0.78	0.315‐1.915	0.678–1.032
Limited	159	0.790	0.335–1.915	0.692–1.040
Moderate	72	2.740	1.3938–5.3838	2.033–3.580
Extensive	28	9.370	4.210–16.3225	5.185–13.631
Very extensive	9	20.235* (mean 33.745)	11.3475–58.250	10.911–63.771
Extensive & very extensive	37	11.935	5.465–19.6475	7.752–16.808
PtGA severity
Not severe‐ mild	113	0.790	0.315‐1.8875	0.587–1.141
Mild	85	0.985	0.3775–2.0325	0.700–1.305
Moderate	96	2.2325	0.821–4.740	1.518–3.128
Severe	52	4.675	1.581–10.7225	2.425–7.057
Very severe	11	1.580* (mean 14.8477)	0.655–15.00	0.641–21.402
Severe & very severe	63	4.475	1.220–10.780	2.140–6.454
PtGA impact (influence)
No impact‐little	143	1.035	0.370–2.425	0.780–1.486
Little	118	1.2875	0.385–2.486	0.892–1.657
Moderate	74	2.2425	0.9313–5.4163	1.430–3.349
Severe	41	5.060	1.86–10.5025	3.472– 7.707
Very severe	10	1.175* (mean 6.073)	0.530–4.5887	0.480–7.288
Severe & very severe	51	4.225	1.225–10.350	2.136–6.455

CI, confidence interval; IQR: interquartile range.

^*Median based on very low numbers of cases within this category.

^†95% confidence interval assessed by MedCalc.

Discussion

In this study, we validated a PtGA for extent, severity and impact using a simple scoring system based on a global assessment question. The PtGA is an intuitive and simple measure that is often used in clinical trials. The PtGAs were used to stratify the numeric score obtained by the SA‐VES. Stratification of outcome measures is crucial to interpreted the obtained scores and to perform research on homogenous study populations. Here, we confirm the intrarater reliability and construct validity of three PtGAs based on hypotheses testing. Using the validated PtGA‐anchor questions, the possible strata per category could be assessed for the SA‐VES score. Based on the cut‐off values, an affected body surface area (SA‐VES) of more than 1.05% and 6.45% was already considered as moderate extensive and extensive, respectively. The median SA‐VES score for moderate extensive and extensive was 2.740 and 9.370, respectively. Cut‐off values for PtGA severity were different compared to PtGA extent, emphasizing the importance of separating these two aspects in future trials.

Severity assessment requires an additional dimension from patient’s point of view including additional influencing factors such as location of the lesions or patients’ photo skin type especially for a pigmentary disorder like vitiligo. Insight into patients’ experiences is crucial to properly define severity in a way that is both useful to the physician and reflective of the patient’s status. A thorough investigation within a large vitiligo patient population may offer additional criteria to better define vitiligo severity.

Impact assessment involves a different dimension as it measures the individual experience of the influence of the disease on daily life. This may be a crucial factor in the therapeutic choice during a vitiligo consultation. Interestingly, >1/3 patients rated impact and severity differently indicating that patients recognize that the more objective concept of severity may result in a different subjective impact from one patient to another. Stratification based on PtGA for impact indicated that even a low‐affected BSA (SA‐VES) of 3.35% can have a high to very high impact on the quality of life. Conversely, a very high SA‐VES can have a moderate impact. This is also clearly illustrated by the distribution of outliers of BSA >32% in the box plots, in which most variation within the 3 categories is present for ‘impact’ (PtGA impact), followed by ‘severity’ (PtGA severity), while all highest outliers are included in the highest category for ‘extent’ (PtGA extent). This points again the importance to differentiate these three domains. An added value of the validated PtGA scores is that they may be used to stratify outcomes of other scores. In addition, they can be helpful in the clinic or trials to provide information ‘at a glance’. However, for studies investigating the impact of the disease more in‐depth, a detailed questionnaire is recommended [e.g. VIPs, DLQI, vitiligo‐specific quality‐of‐life instrument (VitiQoL)] to ensure the required information. ⁷ , ¹² To check the relevance of outliers, a sensitivity analysis was performed. Correlations were performed including and excluding the seven outliers which provided similar results.

A strength of this study was that patients were involved in the construction of the questions used to assess the PtGA for extent, severity and extent. However, for future studies, a diversity (e.g. different ethnic backgrounds) in patients’ population participating in the construction of the PtGA questions should be pursued to ensure generalization of its use.

Another important limitation of this study was the single‐centre setting including most often Caucasian photo skin types with rather limited BSA involvement. This study should therefore be repeated within different centres including patients of darker skin types (IV–VI) and more variation in extent. Future studies are required to confirm other measurement properties such as responsiveness and cross‐cultural validity. Moreover, it could be interesting to compare the physicians’ point of view with the patients' point of view related to the global assessments of extent and severity and related to the stratification into categories (by using VES) within the same population in future studies.

In conclusion, this study confirmed the content validity, construct validity and intrarater reliability of the PtGA for extent, severity and impact. These tools can be used for the interpretation and stratification of scores obtained with other patient‐reported outcome measures (PROMs) and can guide treatment decisions in vitiligo management.

Ethics

The study was approved in Ghent (reference number Ghent: B670201421409). The cooperation with the Dutch Society for vitiligo patients (Vitiligo.nl) was reported at the Amsterdam Medical Center (W17_355#17.413).

Supporting information

Appendix S1. For review