Reliability and validity of the modified 7-day Lee Chronic-versus-Host Disease Symptom Scale

Abstract

Chronic graft-versus-host disease (cGVHD) adversely affects patients’ quality of life, functional status and survival following allogenic hematopoietic cell transplantation. The Lee Symptom Scale is a 30 item scale that was developed to measure the symptoms of cGVHD. Although the original 30-item scale uses a one month recall period, we tested the reliability and validity of a 28 item scale (deleting two items based on supportive care needs rather than symptoms) with a 7-day recall period, a format that is more appropriate for use in clinical trials. Results show the modified 7-day scale is reliable and valid in the modern era and may be used to assess the symptom burden of cGVHD in clinical trials. Using the distribution method, a 5–6 point difference (half a standard deviation) is considered clinically meaningful.

Introduction

Chronic graft-versus-host disease (cGVHD) is a serious iatrogenic complication that affects survivors of allogeneic haematopoietic cell transplantation (HCT). Twenty to fifty percent of allogeneic transplant survivors develop cGVHD. The disease results in inflammation, scarring, and organ dysfunction. Chronic GVHD is the most common long-term complication of HCT and is associated with a decreased quality of life, impaired functional status, continued need for immunosuppressive medications, and increased non-relapse mortality.¹

The Lee cGVHD Symptom Scale (LSS) was developed to measure symptoms in adult outpatients with cGVHD. It was published in 2002.² The scale contains 30 items grouped in 7 subscales (skin, eye, mouth, lung, nutrition, energy, and psychological), and takes two minutes to complete. Patients report how “bothered” they feel about each symptom over the previous month using a five-point Likert scale from “not at all” to “extremely”. A one-month assessment period was chosen for the original scale to capture symptoms over a period of time because most patients with cGVHD are treated as outpatients, and cGVHD symptoms can wax and wane. Subscales range from 0 to 100, with a higher score indicating worse symptoms. Subscales may be scored if at least 50% of items are answered, and subscales are averaged to calculate the summary score. Readers are cautioned to use the correct scoring algorithm (Table 1) because the headers in the survey do not directly correspond to the subscales. In the original publication, a 6 to 7-point change in the summary score suggested a clinically meaningful difference in a patient’s symptomatology.

Table 1. Scoring Algorithm for the modified 7-day Lee Chronic GVHD Symptom Scale.

The modified 7 day Lee Chronic GVHD Symptom Scale (mLSS) is a 28 item instrument with 7 subscales (skin, eyes, mouth, lung, nutrition, energy and psych) containing 2–7 items which allow calculation of a summary score. Response options for “Please let us know if you have been bothered by any of the following problems in the past 7 days” range from 0–4 (Not at all, Slightly, Moderately, Quite a bit, Extremely). A clinically meaningful difference is considered 5–6 points on the summary score. Bolded items are scored under a different subscale than where they are located under the headers in the survey. Items p and k were deleted from the original 30 item scale.

Subscale Name	Number of items	Items
Skin	5	a. Abnormal skin color
		b. Rashes
		c. Thickened skin
		d. Sores on skin
		e. Itchy skin
Eye	3	f. Dry eyes
		g. Need to use eyedrops frequently
		h. Difficulty seeing clearly
Mouth	2	i. Need to avoid certain foods due to mouth pain
		j. Ulcers in mouth
Lung	4	l. Frequent cough
		m. Colored sputum
		o. Shortness of breath at rest
		aa. Fevers
Nutrition	4
		q. Difficulty swallowing solid foods
		r. Difficulty swallowing liquids
		s. Vomiting
		t. Weight loss
Energy	7	n. Shortness of breath with exercise
		u. Joint and muscle aches
		v. Limited joint movement
		w. Muscle cramps
		x. Weak muscles
		y. Loss of energy
		z. Need to sleep more/take naps
Psych	3	bb. Depression
		cc. Anxiety
		dd. Difficulty sleeping

Scoring rules:

^a.Note that the subscales do not conform exactly to the headers in the patient survey.

^b.Items p and k are deleted from the 7 day version.

^c.Subscales may be scored if 50% of more of the items in the subscale are completed.

^d.Scores are linearly transformed to a 0–100 scale where 0 means all answered items were a “0” and “100” means that all answered items were a “4”

^e.Missing items are not included in the scoring.

^f.The summary score is the average of the subscale scores, as long as 4 or more subscales are available.

^g.Higher scores indicate more severe symptoms.

In 2005 and 2014, the NIH Consensus Development Project on Criteria for Clinical Trials in Chronic GVHD proposed the LSS as one of the tools to determine the efficacy of cGVHD treatments.^3,4 The relevance of the scale to modern cGVHD patients was confirmed in a 2016 publication about the content validity of the scale.⁵ Even though the LSS is now commonly used to evaluate symptoms in cGVHD prevention^6,7 and therapy trials,^8–10 the Food and Drug Administration (FDA) and pharmaceutical sponsors prefer a shorter recall period of 7 days. Some have also questioned inclusion of two items: “Receiving nutrition from an intravenous line or feeding tube” and “Need to use oxygen” because they reflect use of supportive care measures rather than symptoms. In general, recall is better over shorter periods that are preferred for symptom assessment, but may be influenced by fluctuating symptomatology. A 7-day recall period matches the common quality of life instruments. The aim of the present study was to reassess the instrument’s reliability and validity in the modern era with a 7-day recall period to (a) establish internal consistency of items; (b) show that scores are stable if a patient’s condition does not change; and (c) demonstrate convergent and divergent validity. These are important features of any scale used to document effectiveness of treatments. We also evaluated the impact of deleting the two questions relating to supportive care on the performance of the scale.

Methods

Participants (N=68) in this study were (a) adults age 18 years or older, (b) able to communicate in English, (c) diagnosed with cGVHD per the 2014 NIH Consensus Criteria, and (d) symptomatic with active cGVHD with an NIH score greater than 1 in one or more organs associated with cGVHD. We anticipated enrolling up to 80 subjects to include at least 40 of those with unchanged cGVHD symptoms between the first and second administration of the LSS. Data collection continued until 40 participants indicated that there was no clinically significant change to their symptoms (“about the same”) between their baseline and follow-up surveys. The study was approved by the Fred Hutchinson Institutional Review Board, and was conducted between 2016–2017. The requirement for written documentation of consent was waived given the minimal risk nature of the study because participants were informed of all the components of informed consent including that they could skip over any topic they wished and that participation was voluntary and would not affect their care.

The LSS survey used in this study was identical to the published one month recall period version except that it used a recall period of “the past 7 days.” Participants were given a paper survey at enrollment which they completed in the clinic or mailed back. They then were mailed a second survey that they completed and returned by mail approximately one week after completion of the first LSS. The follow up survey asked whether their cGVHD symptoms had changed since enrollment, using a 7 point scale: very much worse, moderately worse, a little worse, about the same, a little better, moderately better, very much better. Patients also completed one page of socio-demographic questions with their enrollment survey. On both surveys, they answered questions about their overall cGVHD status from the NIH patient self-report cGVHD assessment which included reporting whether they considered their cGVHD “mild”, “moderate” or “severe”.

Descriptive statistics include patient, transplant and cGVHD characteristics. The survey was scored according to the published recommendations and excluding the two supportive care items in question;^2,5 nonresponse; was defined as inability to calculate a score because of missing data. The test-retest correlation was calculated for the summary score and the 7 subscales and reported as the Spearman correlation coefficient. The intraclass correlations are reported as Cronbach’s alphas. Generally, test-retest and Cronbach’s alpha values of >0.7¹¹ are considered acceptable.

Results

During the study 68 patients enrolled and 40 (59%) reported that their symptoms were “about the same” on the follow up survey. The other 28 patients either failed to complete a follow up survey (n=12, 18%) or indicated on the follow-up survey that their cGVHD symptoms had improved (n=16, 24%). No one reported that their symptoms had worsened. Psychometrics are reported based on the 68 enrollment surveys while the test-retest statistics are based on the 40 participants who reported that their cGVHD had not changed since they first completed the LSS. Sample characteristics are shown in Table 2. Ten participants (14.7%) identified themselves as non-white: Asian (n=3), Black (n=3), Hawaiian Native/Pacific Islander (n=1) and Other (n=3). Forty-three participants (64.2%) had a college or postgraduate degree, and 43 (64.2%) were married or living with a partner. Twenty-two participants (32.8%) were working full time; 6 (9.0%) part-time.

Table 2.

Cohort characteristics (N=68)

	N=68	N=40 used for test-retest
Age, median (IQR) (years)	57.5 (42.5–63.5)	58.5 (40.5–63)
Male sex, n (%)	41 (60.3%)	26 (65.0%)
Race
White	58 (85.3%)	37 (92.5%)
Asian	3 (4.4%)	1 (2.5%)
Black	3 (4.4%)	1 (2.5%)
Hawaiian Native/Pacific Islander	1 (1.5%)	0 (0%)
Other	3 (4.4%)1	1 (2.5%)2
Hispanic	0 (0%)	0 (0%)
Marital status
Married/living with partner	43 (64.2%)	27 (67.5%)
Single, never married	14 (20.9%)	7 (17.5%)
Divorced, separated	8 (11.9%)	6 (15.0%)
Widowed	1 (1.5%)	0 (0%)
Married/not living with partner	1 (1.5%)	0 (0%)
Missing	[N=1]
Education
Less than college	8 (11.9%)	5 (13%)
Some college	16 (23.9%)	9 (22.5%)
College graduate	24 (35.8%)	14 (35.0%)
Post graduate degree	19 (28.4%)	12 (30.0%)
Missing	[N=1]
Work/School status
Working or school full time	22 (32.8%)	15 (37.5%)
Working part time	6 (9.0%)	3 (7.5%)
Retired	16 (23.9%)	10 (25.0%)
Disabled, unable to work	12 (17.9%)	8 (20.0%)
Homemaker	7 (10.4%)	3 (7.5%)
On medical leave	2 (3.0%)	0 (0%)
Unemployed, looking foi work	2 (3.0%)	1 (2.5%)
Missing	[N=1]	[N=0]
NIH severity at enrollment (patient self-report)
Mild	36 (53.7%)	21 (52.5%)
Moderate	29 (43.3%)	17 (42.5%)
Severe	2 (3.0%)	2 (5.0%)
Missing	[N=1]
NIH severity at enrollment (per NIH criteria)
Mild	10 (14.7%)	5 (12.5%)
Moderate	30 (44.1%)	16 (40.0%)
Severe	28 (41.2%)	19 (47.5%)
Median time from transplant to chronic GVHD, months (IQR)	8.4 (5.4–11.8)	10.0 (5.8–12.3)
Median time from chronic GVHD diagnosis to enrollment, months (IQR)	34.9 (19.4–64)	34.9 (19.6 – 60.5)
Score 2–3 organ involvement
Skin	31 (45.6%)	19 (47.5%)
Eye	23 (33.8%)	15 (37.5%)
Mouth	5 (7.4%)	2 (5.0%)
Gastrointestinal	1 (1.5%)	1 (2.5%)
Liver (1 missing)	0 (0%)	0 (0%)
Lung (l missing)	7 (10.4%)	6 (15.4%)
Joint	12 (17.6%)	8 (20.0%)

¹Asian and Indian, Portuguese, and American

²American

The median time elapsed from HCT to the diagnosis of cGVHD was 8.4 months. The median time from HCT to enrollment was 34.9 months (interquartile range, 19.4–64), and the distribution of global severity of cGVHD using 2014 NIH consensus scoring was mild (n=10), moderate (n=30), and severe (n=28). NIH severity was higher than participant self-reported severity which was none (n=5), mild (n=31), moderate (n=29) and severe (n=2). The most common organs with scores of 2 or higher were skin in 31 participants (45.6%) and eye in 23 (33.8%).

Table 3 summarizes the psychometric properties of the survey for both the 30 item and the 28 item scales. Cronbach’s alpha was >0.7 for the energy, skin, eye, and mouth subscores and for the summary score but <0.62 for nutrition, lung and psychological scales. No participants endorsed the intravenous or feeding tube item, and two reported being “slightly” and “moderately” bothered by needing to use oxygen. Removing these items and recalculating the subscale scores minimally improved the Cronbach’s alpha of the nutrition and lung subscales to 0.61 and 0.43 respectively when compared to inclusion of all the items. Importantly, however, the Cronbach’s alpha of the summary score remained high and was 0.84 for the 30 item scale and 0.85 for the 28 item scale in the present study compared to 0.90 in the original description. Cronbah’s alpha remained 0.76–0.83 when evaluated in the 3 severity groups separately. The standard deviation was 10.5 in the 30 item scale and 10.7 for the 28 item scale; we estimate that a 5–6 point difference is clinically meaningful using the distribution method (half a standard deviation).^12,13

Table 3.

Reliability of the 7 day Lee Symptom Scale (N=68)

			Nutrition		Lung					Summary
	Energy	Skin	Original	Modified	Original	Modified	Psych	Eye	Mouth	Original	Modified
Items	7	5	5	4	5	4	3	3	2	30	28
Mean	28.0	15.0	5.0	6.3	3.4	3.9	16.5	44.2	19.9	18.9	19.1
STD	20.2	17.0	8.5	10.6	6.2	7.4	16.6	28.6	25.0	10.5	10.7
Median	25.0	10.0	0	0	0	0	12.5	41.7	12.5	17.6	18.0
Range	0–85.7	0–70	0–45	0–56.3	0–40	0–50	0–75	0–100	0–100	2.4–43.3	2.4–43.5
Cronbach’s α	0.85	0.74	0.57	0.61	0.40	0.43	0.5 7	0.83	0.71	0.84	0.85
Floor	5 (7.4%)	21 (30.9%)	40 (58.8%)	40 (58.8%)	41 (60.3%)	42 (61. 8%)	18 (26.5%)	7 (10.3%)	29 (42.6%)	1 (1.5%)	1 (1.5%)
Ceiling	1 (1.5%)	1 (1.5%)	1 (1.5%)	1 (1.5%)	1 (1.5%)	1 (1.5%)	1 (1.5%)	4 (5.9%)	1 (1.5%)	1 (1.5%)	1 (1.5%)
Nonresponse	0	0	0	0	0	0	0	0	0	0	0
Test-Retest (n=40)	0.89	0.88	0.76	0.76	0.71	0.70	0.82	0.85	0.79	0.79	0.81
Intercorrelation
Energy		0.381	0.502	0. 502	0.381	0.371	0.351	0.27	0.22	0.712	0.712
Skin			0.01	0.01	0.11	0.13	0.27	0.07	0.14	0.462	0.462
Nutrition					0.24		0.21	0.19	0.462	0.562
Nutrition-mod						0.28	0.21	0.19	0.462		0.572
Lung							0.01	0.10	−0.03	0.29
Lung-mod							0.05	0.05	0.01		0.29
Psychological								0.15	0.22	0.552	0.552
Eye									0.381	0.672	0.662
Mouth										0.622	0.622

¹p<0.01

²p<0.0001

All test-retest correlations were at least 0.7 and ranged from 0.70–0.89. Correlations were ≥0.80 for the energy, skin, psychological and eye subscales, and between 0.70–0.79 for the nutrition, lung and mouth subscales. The test-retest correlation for the summary score was 0.79 for the full scale and 0.81 for the 28-item version. Compared to the original description of the instrument, test-retest scores were higher or the same for energy, skin, lung, psychological, eye and summary scores and lower for nutrition and mouth but still adequate for all.

Interclass correlations showed that the energy subscale correlated with all subscales except the eye and mouth subscales. The mouth scores correlated with the nutrition and eye subscales but overall, the subscales were fairly independent. All except lung correlated with the summary score.

Table 4 shows that the LSS scores differed for each subscale (p<0.10) and for the summary score (p<0.001) between self-reported or NIH mild versus moderate/severe cGVHD except for two exceptions. The lung subscale was not correlated with self-reported cGVHD severity, and the psychological scale was not correlated with the NIH cGVHD severity. Removal of the “need to use oxygen” item did not improve the results of the lung subscale analysis.

Table 4.

Chronic GVHD symptoms by self-reported and NIH calculated chronic GVHD severity at enrollment

	Self-reported cGVHD severity
Symptoms	None1/Mild, Mean (SD) (n=36)	Moderate/Severe,2Mean (SD) (n=31)	p-value3
Energy	19.0 (13.1)	38.1 (22.0)	<.001
Skin	9.2 (11.9)	21.6 (19.5)	0.003
Nutrition	3.2 (8.0)	7.0 (8.6)	0.070
Nutrition - modified3	4.1 (10.1)	8.7 (10.8)	0.069
Lung	2.6 (4.4)	4.2 (7.7)	0.315
Lung - modified4	3.0 (5.1)	4.9 (9.4)	0.304
Psychological	12.3 (13.7)	21.4 (13.4)	0.023
Eye	32.9 (26.3)	57.0 (25.8)	<.001
Mouth	14.6 (17.0)	25.8 (30.9)	0.076
Summary	13.4 (7.9)	25.0 (9.6)	<.001
Summary - modified3	13.6 (8.0)	25.3 (9.9)	<.001
	cGVHD severity per NIH criteria
	Mild, Mean (SD) (n=10)	Moderate/Severe, Mean (SD) (n=58)	p-value3
Energy	11.1 (10.8)	30.9 (20.0)	0.003
Skin	3.0 (3.5)	17.1 (17.5)	<.001
Nutrition	1.5 (3.4)	5.6 (8.9)	0.014
Nutrition - modified4	1.9 (4.2)	7.0 (11.2)	0.014
Lung	0.0 (0.0)	4.0 (6.6)	<.001
Lung - modified4	0.0 (0.0)	4.5 (7.8)	<.001
Psychological	10.0 (10.2)	17.7 (17.3)	0.180
Eye	20.0 (16.8)	48.4 (28.2)	0.003
Mouth	8.8 (13.2)	21.8 (26.1)	0.024
Summary	7.8 (6.0)	20.8 (9.9)	<.001
Summary - modified4	7.8 (6.1)	21.0 (10.1)	<.001

¹5 patients indicated they had “none” and “0” or “1” severity on a 0–10 scale. They are included in the study because they had mild, moderate or severe cGVHD per NIH criteria.

²1 patient did not report chronic GVHD severity but was NIH severe so was grouped with the self-reported moderate/severe group

³Based on T-test

⁴Modified from the original by deletion of two items, see text

In two other prospective multicenter observational studies conducted from 2007–2012¹⁴ and 2013–2017,¹⁵ the rates of any endorsement of the “need to use oxygen” (3.2% and 6.1%) and “receiving nutrition from an IV or feeding tube” (1.2% and 2.2%) items were very low.

Discussion

When the LSS was first developed and validated in the late 1990’s, severe manifestations of cGVHD were more common. The survey also used a 1-month recall period since the intent was to use the instrument for clinical care and observational studies. This report shows that the items about need for oxygen or intravenous or tube feeding can be removed without adversely affecting test characteristics if a pure symptom scale is desired. The 7-day recall period may be used because the modified instrument retains its overall reliability and validity. Both of these changes result in a modified Lee Symptom Scale (mLSS) that is better suited to clinical trials.

Results from this study and from reanalysis of two earlier cohorts shows that endorsement of the oxygen and intravenous/feeding tube items was very infrequent. Although these questions were originally conceived to reflect bother due to the severity of cGVHD requiring need for such supportive care, they do not directly reflect cGVHD symptoms since even very symptomatic patients might refuse oxygen or feeding tubes. The low rate of endorsement seen in modern studies may also be due to better recognition and earlier/more effective treatment of cGVHD although these hypotheses are speculative. Regardless, this study shows that these two items may be removed from the scoring algorithm. Although absolute scores will be higher because we are removing items that are usually scored as zeros that bring down the average, as long as the enrollment and followup surveys are scored using the same formula, change scores are interpretable. Collection of the full 30-item version allows calculation of either the full or modified scale scores.

A previous study asked patients to compare how they would report their symptoms with a 7 day or one month timeframe, showing that some patients reported the time frame selected would have altered their answers. The primary reason given was that their cGVHD symptoms had changed for better or worse in the past month which is a legitimate reason for different answers, further justifying the change to a shorter recall period.

Intraclass correlations of three subscales (nutrition, lung and psychological symptoms) were <0.7 suggesting that the items are not measuring a single construct. Examination of the individual questions supports this conclusion, for example, the nutrition subscale includes difficulty swallowing, nausea and weight loss, all recognized symptoms and signs of gastrointestinal cGVHD which are not always found together.

Limitations of this study include the modest sample size and restriction to outpatients from one center. Participants were well-educated with 64% college graduates. Patients with self-reported severe cGVHD were underrepresented (3%), whereas there were 41% with severe cGVHD per the NIH criteria. There were very few patients with liver and gastrointestinal symptoms, and patients were only stable or improved (none worsened) between the two test and retest measurements, which might be explained because the retest survey was administered only one week after a clinic visit where symptoms may have been detected and treated.

In summary, our results document the reliability and validity of the 7-day mLSS for evaluating cGVHD symptoms and suggests a 5–6 point difference in the summary score is clinically meaningful. The 7-day mLSs may be used in modern clinical trials.

Highlights.

• The 28-item, modified Lee chronic GVHD symptom scale (mLSS) is valid and reliable.

• The scale takes 2 minutes to complete; a 5–6 point difference is clinically meaningful.