Use of the Patient-generated Index in Systemic Sclerosis to Assess Patient-centered Outcomes

Abstract

Objective

To evaluate the content and construct validity of an individualized patient-reported instrument, the Patient-generated Index (PGI), in patients with systemic sclerosis (SSc), and to compare its performance to that of other instruments and to the Patient-reported Outcomes Measurement Information System (PROMIS) framework.

Methods

Patients identified the 5 most important life areas affected by SSc, which we categorized into domains of the PROMIS framework (mental, physical, and social). Correlations were obtained between PGI and the Health Assessment Questionnaire (HAQ), the Medical Outcomes Study Short Form-36 (SF-36), and the Symptom Burden Index (SBI) scores.

Results

Sixty-two patients with SSc completed the PGI: 87% women, 69% white, mean age 53 years, mean disease duration 8 years, and 63% with diffuse disease. A total of 258 individual life area responses were recorded: 54% in social health (social function and relationship subcomponents); 28% in physical health (physical function, symptoms, general physical health); and 19% in mental health (consisting largely of the affect subcomponent). Patient PGI responses were categorized into 6 of the 7 subcomponents of the PROMIS framework; substance use/alcohol was not identified. Statistically significant correlations ranging in absolute value from 0.26 to 0.50 were observed between the PGI and the HAQ, SF-36 summary component scores, and the large majority of SF-36 subscales and SBI components.

Conclusion

The PGI is a personalized instrument that adequately assessed a wide range of health-related quality of life outcomes within the PROMIS framework. The PGI captured additional constructs not yet defined within the framework that are important for patients with SSc.

The course and severity of systemic sclerosis (SSc) vary greatly by patient; in addition to cutaneous involvement, multiple organs can be affected. Quality of life is a primary issue for patients, and each may be affected differently, highlighting the importance of conducting comprehensive, multifaceted patient evaluations^1,2,3,4,5,6. Health-related quality of life (HRQOL) has been measured in SSc with generic instruments such as the Medical Outcomes Study Short Form-36 (SF-36), or physical function measures such as the Health Assessment Questionnaire-Disability Index (HAQ-DI)^4,6,7,8. Both the SF-36 and the HAQ have been validated in this population, but are limited by the evaluation of each individual patient's experience of disease effect to the explicit content of the predetermined fixed item sets that are the same for all respondents. Patient-specific personalized instruments may provide better insight into the multifaceted aspects of patients' experiences, identifying unique areas of their lives or disease that are important to them⁹.

The Patient-generated Index (PGI) evaluates HRQOL on the basis of domains that individual patients identify as important to them. Patients rate the severity and importance of each affected area, thus providing quantitative information on the extent of their burden and value associated with it. The PGI has been used in patients with chronic diseases, but to our knowledge, this is the first study evaluating this personalized tool in patients with SSc^10,11. Another personalized instrument, the McMaster Toronto Arthritis Patient Preference Disability Questionnaire (MACTAR), uses open-ended questions and ranking to assess limitations in activities that the patient considers important^12,13. A study using the MACTAR in patients with SSc showed weak correlation with the HAQ, suggesting that personalized instruments add useful information¹³. Yet while the MACTAR assesses activities and symptoms, it may fail to bring to light psychosocial constructs. Further, the MACTAR was specifically developed and validated in patients with arthritis, compared to the PGI, which has been used in many diseases, and therefore allows for comparisons of disease burden across chronic conditions. The Canadian Occupational Performance Measure is also a personalized instrument but it is designed primarily to capture occupational performance, and thus may lack more general psychosocial constructs¹⁴.

The Patient-reported Outcomes Measurement Information System (PROMIS) framework has been proposed by the US National Institutes of Health to comprehensively evaluate patient-reported symptoms and outcomes using an item bank measuring latent constructs relevant to patients (www.nihpromis.org). PROMIS-29 is a PROMIS-derived instrument that measures physical, mental, and social health¹⁵. It has demonstrated construct validity in assessing important areas of a person's life affected by chronic diseases and is easily comparable among disease states. However, it may not capture commonly cited problems among patients with SSc, such as symptom burden on their social lives or emotional distress related to physical appearance^2,16,17. PROMIS-based items have been used in computer-adaptive tests (CAT)¹⁷. However, CAT are used for collecting pre-established constructs and may not capture new constructs of interest to patients.

The objectives of our study were to assess the content and construct validity of the PGI in patients with SSc, and to establish which personalized components of HRQOL were important to patients. To achieve these goals, we compared the PGI to other instruments used to measure HRQOL in SSc, under the hypothesis that moderate correlations would be observed (validity without redundancy). Moreover, to assess the comprehensiveness of the instrument in evaluating all important aspects of HRQOL, we mapped the individual areas of concern and burden identified by patients to the PROMIS framework. By mapping the PGI results to this framework we sought to establish content validity for the use of the PGI in the evaluation of HRQOL in patients with SSc. Additionally, we determined whether the PGI can elicit latent constructs relevant to patients with SSc not identified by other instruments or the current PROMIS framework.

Materials and Methods

Patients with SSc who participated in the study were recruited from an SSc clinic at the University of Texas Health Science Center. The majority were participants in the Genetics versus Environment in Scleroderma Outcome Study (GENISOS), a longitudinal cohort study based in Houston. The institutional review boards of Baylor College of Medicine and The University of Texas Health Science Center at Houston approved the study. Patients completed questionnaires in person or by telephone, giving clinical and demographic information, and using instruments outlined in the next section. Patients interviewed by telephone were first mailed questionnaires so they could follow along during the interview with a paper copy in front of them. Interviewers were trained to administer the PGI.

Measures

We used the generic version of the PGI developed by Ruta, et al, which provides general examples of areas of life that can be affected by disease¹⁰. The questionnaire is completed in 4 stages. In the first, patients select the 5 most important areas of their lives affected by SSc. Examples such as family, work, love, and friends are provided to guide patients and stimulate their thoughts. A sixth area is added for all patients: “all other areas of your life affected by your scleroderma.” In the second stage, patients score each of the 6 areas named in the first section using a scale ranging from 0 (“as bad as could possibly be”) to 6 (“as good as could possibly be”) to rate how each specific area of their lives was affected by SSc over the last month. In the third stage, patients are then asked to “spend 10 points to show which areas of your life you feel are most important to your overall quality of life.” Points do not have to be evenly distributed across areas of life, so that areas that are more important to a patient can have more points “spent” on them (Figure 1). The fourth stage is generation of an index value, done by multiplying the 6 scores for each area by the number of points spent, and then summing these products. The result is an index score that is transformed to a percentage ranging from 0 to 100, with lower scores indicating worse HRQOL.

Figure 1.

The Patient-generated Index Questionnaire.

SF-36

The SF-36 is a generic multi-item measure of health status including 8 dimensions: physical functioning, social functioning, role limitations due to physical problems, role limitations due to emotional problems, mental health, energy/vitality, pain, and general health perception, in the last 4 weeks. From the subscales, an overall physical (PCS) and mental (MCS) health component score can also be calculated. All SF-36 scores range from 0 to 100, with lower scores signifying worse health status^18,19.

HAQ

The HAQ measures a patient's level of functional ability over the past week across 8 categories of activities, with a total of 20 questions. Mean scores range from 0 to 3, with 3 indicating a maximal impairment of function. The HAQ is a reliable and valid measure for SSc-related functional status⁴.

Symptom Burden Index (SBI)

The SBI measures symptom burden and was previously validated by our group²⁰. It includes 8 domains: bowel function, calcinosis, eating, problems with hands, pain, shortness of breath, appearance, and sleep. Patients are asked 5 questions about each symptom they may experience: (1) how much of a problem was the symptom? (2) how often was the symptom a problem? (3) how much did the symptom interfere with daily activities? (4) how often did the symptom interfere with daily activities? and (5) how important a problem was the symptom? Each response is scored with a 0– to 10–point scale, 0 indicating no burden and 10, severe burden.

Mapping to the PROMIS framework

PROMIS health components include mental, physical, and social health, with 7 subcomponents (symptoms, function, affect, behavior, cognition, relationships, and function) and various domains and subdomains within each subcomponent. We categorized the areas patients identified with the PGI as most relevant using the PROMIS framework as a guide²¹. This was performed by 2 of the authors (SDA and MLO), who independently grouped the qualitative individual PGI items into the PROMIS domains and subdomains. Figure 2 shows an abbreviated version of the PROMIS framework that includes the domains and subdomains to which issues identified by the PGI were mapped.

Figure 2.

Patient-generated Index Questionnaire domains addressed by patients with systemic sclerosis, mapped to the Patient-reported Outcomes Measurement Information System (PROMIS) framework.

Statistical analysis

ANOVA and Pearson correlation coefficients were used to compare PGI scores with other patient-reported outcomes to assess construct validity and to rule out construct redundancy. As part of this analysis we examined mean PGI scores across tertiles of symptom burden (0–5, 6–7, and 8–9 symptoms reported) to evaluate whether, as expected, PGI scores decreased with increasing number of symptoms. We also compared mean scores of the PGI between subgroups of patients (e.g., sex or education) to evaluate the consistency of scores. Statistical analyses were performed using Stata version 9 (Stata Corp.).

Results

Sixty-two (82%) out of 76 patients who were approached agreed to participate. Table 1 displays the sociodemographic and clinical characteristics of the participants. No statistically significant differences were observed in PGI scores among demographic and clinical subgroups.

Table 1.

Patient-generated Index (PGI) scores across patient characteristics.

Characteristic		n (%)	PGI Score Mean (SD)	p*
Age	Range 21–79 yrs
	≤ 55 yrs	33 (54.1)	52.4 (28.3)	0.45
	> 55 yrs	28 (45.9)	57.9 (27.2)
Sex	Male	8 (12.9)	50.8 (25.6)	0.71
	Female	54 (87.1)	54.9 (28.4)
Ethnicity	White	43 (69.4)	52.8 (26.3)	0.52
	Nonwhite	19 (30.6)	58.0 (31.7)
Disease type	Limited	23 (37.1)	57.0 (26.3)	0.57
	Diffuse	39 (62.9)	52.8 (29.0)
Duration of disease	Range 2–36 yrs
	≤ 5 yrs	24 (40.0)	59.4 (30.0)	0.24
	> 5 yrs	36 (60.0)	50.6 (26.6)
Occupation	Employed or in school	33 (54.1)	57.9 (25.4)	0.24
	Unemployed, disabled, or retired	28 (45.9)	49.4 (30.3)
Marital status	Partnered	38 (61.3)	56.1 (25.9)	0.55
	Not partnered	24 (38.7)	51.7 (31.0)
Household members	None	10 (16.4)	46.7 (30.1)	0.28
	1 or more	51 (83.6)	56.9 (26.7)
Language	Spanish and English	8 (13.1)	61.0 (26.9)	0.56
	English	53 (86.9)	54.4 (27.5)
Education	Up to some college	36 (59.0)	56.1 (27.1)	0.76
	Bachelor's degree or above	25 (41.0)	53.9 (28.0)

^*Comparison of mean PGI scores among categorized patient characteristics by 2-tailed t test.

Statistically significant correlations were observed between the PGI and the other measures, with small to moderate coefficients (absolute value) ranging from 0.11 to 0.50, providing evidence of construct validity, while indicating lack of redundancy of the PGI compared to other measures (Table 2). The PGI was statistically significantly associated with the HAQ, with 7 of the 8 SF-36 subscales, and with both the PCS and MCS composite scores, in the expected directions. Symptom burden was ascertained with the SBI. Statistically significant negative correlations were observed between the PGI and SBI scores for appearance, bowel function, hands, pain, skin, and sleeping. A statistically significant negative relationship was noted between overall SBI symptom burden and the PGI score (more burden associated with worse HRQOL). Lastly, the PGI was compared across the burden count groups (distributional tertiles): participants reporting 0–5 burdens had higher PGI scores (mean 68.1 ± 29.2) versus those experiencing 6–7 burdens (mean 53.9 ± 24.2) or 8–9 burdens (mean 42.4 ± 26.8).

Table 2.

Correlations between Patient-generated Index (PGI) and other outcome measures.

	Mean (SD)	Range (minimum, maximum)	Correlation with PGI, r (p)†	(95% CI)
PGI	54.3 (27.8)	(0, 100)	—
HAQ	0.8 (0.6)	(0, 2.4)	−0.33 (0.01)	(−0.54, −0.09)
SF-36–Physical functioning	42.7 (29.3)	(0, 100)	0.33 (0.01)	(0.09, 0.55)
SF-36–Role physical	50.2 (29.6)	(0, 100)	0.42 (< 0.01)	(0.20, 0.65)
SF-36–Bodily pain	49.5 (25.4)	(0, 100)	0.39 (< 0.01)	(0.12, 0.64)
SF-36–General health	44.0 (25.4)	(0, 97)	0.35 (< 0.01)	(0.12, 0.56)
SF-36–Vitality	40.0 (22.4)	(0, 100)	0.20 (0.13)	(−0.13, 0.45)
SF-36–Social functioning	61.7 (30.7)	(0, 100)	0.46 (< 0.01)	(0.22, 0.68)
SF-36–Role emotional	74.3 (33.5)	(0, 100)	0.26 (0.04)	(−0.04, 0.52)
SF-36–Mental health	69.4 (21.6)	(0, 100)	0.28 (0.03)	(−0.02, 0.49)
SF-36–PCS	34.1 (11.5)	(13.8, 57.8)	0.39 (< 0.01)	(0.16, 0.60)
SF-36–MCS	47.4 (13.0)	(10.8, 64.7)	0.26 (0.04)	(–0.04, 0.52)
SBI–Appearance	2.6 (3.3)	(0, 10)	−0.36 (< 0.01)	(−0.57, −0.08)
SBI–Bowel	3.8 (3.3)	(0, 10)	−0.37 (< 0.01)	(−0.60, −0.10)
SBI–Calcinosis	1.4 (2.9)	(0, 10)	−0.38 (< 0.01)	(−0.57, −0.15)
SBI–Eating	3.2 (3.3)	(0, 10)	−0.13 (> 0.20)	(−0.42, 0.17)
SBI–Hand	5.0 (3.2)	(0, 10)	−0.33 (0.01)	(−0.56, −0.06)
SBI–Pain	5.1 (3.1)	(0, 10)	−0.50 (< 0.01)	(−0.72, −0.28)
SBI–Shortness of breath	3.6 (3.5)	(0, 10)	−0.11 (> 0.20)	(−0.38, 0.13)
SBI–Skin	4.4 (3.1)	(0, 10)	−0.35 (< 0.01)	(−0.59, −0.09)
SBI–Sleeping	4.0 (3.4)	(0, 10)	−0.35 (< 0.01)	(−0.61, −0.08)
SBI–Burden	6.2 (2.2)	(0, 9)	−0.46 (< 0.01)	(−0.65, −0.22)

^†Pearson correlation coefficients were used to compare PGI scores with patient-reported outcomes. SF-36: Medical Outcomes Study Short Form-36; SBI: Symptom Burden Index; PCS: physical component summary score; MCS: mental component summary score.

We categorized patient choices of affected life areas into the components and subcomponents of the PROMIS framework, grouped under related domains and subdomains (Figure 2). A total of 258 individual life area responses were reported (Table 3); some patients identified fewer than 5 areas. Overall, social health was the domain of concern identified most frequently (53.5%), followed by physical (27.5%) and mental (19.0%) health. Within the social component, 44.9% of responses were associated with relationships, such as “friends” or “family.” The remaining 55.1% of responses were associated with social function, e.g., “attending church.” In the physical health component, most responses (74.6%) related to physical function, e.g., “exercise” and “sports,” and activities of daily living such as “housework.” In the mental health component, the majority of responses were in the realm of affect (71.4%). Some areas identified by the PGI could not be clearly mapped to the PROMIS framework, including spirituality, faith, and finances, which were identified as affected by the disease in 11, 4, and 7 patients, respectively (in Table 2 added to well-being and social life). Other responses were complex and encompassed a multitude of domains, for example, “enjoying hobbies and interests,” “traveling,” and “appearance.” Patients clearly identified these areas as important, and as disrupted by their disease. Using PROMIS-based categories alone would have failed to capture these items. One of the PROMIS framework 7 subcomponents (substance use/alcohol) was not identified as an area of concern by any of our patients.

Table 3.

Frequency of individual responses identified by the PGI and mapped to PROMIS framework categories.

PROMIS General Comp.†	n (%)	PROMIS Subcomp.†	Count	Percentage of General Comp.†	Percentage of Overall Responses	PROMIS Domain or Subdomain†	Count	Percentage of Subcomp.*	Percentage of General Comp.	Percentage of Overall Responses*
Physical health	71 (27.5)	Physical function	53	74.6	20.5	Activities of daily living	18	34.0	25.4	7.0
						Physical activity	16	30.2	22.5	6.2
						Sexual function	6	11.3	8.5	2.3
						Upper extremity	6	11.3	8.5	2.3
						Mobility	5	9.4	7.0	1.9
						Sleep function	2	3.8	2.8	0.8
		Physical symptoms	17	24.0	6.6	Fatigue/energy	11	64.7	15.5	4.3
						GI symptoms	2	11.8	2.8	0.8
						Pain	2	11.8	2.8	0.8
						Symptoms	2	11.8	2.8	0.8
		Physical health	1	1.4	0.4	Physical health	1	100.0	1.4	0.4
Mental health	49 (19.0)	Positive affect	24	49.0	9.3	Subjective well-being	14	58.3	28.6	5.4
						Psychosocial illness effect (positive)	10	41.7	20.4	3.9
		Negative affect	11	22.4	4.3	Psychosocial illness effect (negative)	11	100.0	22.4	4.3
		Cognition	14	28.6	5.4	Self-efficacy	14	100.0	28.6	5.4
Social health	138 (53.5)	Social function	76	55.1	29.5	Ability to participate/satisfaction with participation	76	100.0	55.1	29.5
		Social relationships	62	44.9	24.0	Social relationships	63	100.0	44.9	24.0

^†Total n = 258 individual responses. Each response was categorized into one of the PROMIS subdomains.

^*Some cells total 100.1% because of rounding. PGI: Patient-generated Index; PROMIS: Patient-reported Outcomes Measurement Information System; comp.: component; subcomp.: subcomponent.

Discussion

Patients with SSc face a variety of unique burdens during the course of their disease, such as changes in appearance, and social issues that may not be well addressed by general assessment tools or by their physicians^2,22. Previous studies have found that patients' lives are affected not only by physical limitations but also by areas such as mental health, specifically depression, and limitations in social activ-ities^8,23,24,25. Further, patients with SSc may vary greatly in their burden of illness, and what might be a prominent issue for 1 patient could be irrelevant to another. Most of the measures frequently used in SSc limit the extent to which patients can accurately describe their individual experiences because of the fixed and thus limited item content, which restricts the addressing and evaluation of important issues that might have great relevance to a patient. The objective of our study was to evaluate the performance of an individualized patient-reported instrument, the PGI, in patients with SSc, and to compare its performance to other instruments and to the PROMIS framework. Our preliminary data show that the PGI measures specific aspects of HRQOL and correlates with other measures previously used in SSc such as the HAQ and SF-36, but does not provide redundant information because the correlations are moderate. These findings are consistent with previous research in other diseases and attest to the PGI's effectiveness in contributing unique patient-centered information^11,26.

The themes generated by the PGI showed that patients' concerns mapped comprehensibly to the PROMIS framework; however, the PGI was also capable of capturing items not present to date in the PROMIS domains. Over half the areas identified by the PGI concerned patients' social lives, an aspect that may not be well captured by current outcome tools. The most recent PROMIS framework does not have validated measurements of social activities such as church attendance, volunteering, and work, which were captured by the PGI. Future studies are needed to determine whether CAT approaches such as those used by PROMIS could benefit from a patient-centered instrument such as the PGI, which provides additional constructs using open-ended questions.

To complete the PGI, patients engage in a cognitively complex process that can help them identify and understand which areas are most important for them. This instrument can be useful for both clinical and research purposes. Within clinical settings it can help clarify (for both patients and their providers) relevant patient values²². The PGI could also assist in establishing goals the patient would like to achieve, and identify unrealistic expectations. In research, the PGI can be useful in identifying specific areas of concern that affect quality of life and may not be adequately ascertained by other tools, leading to a better understanding of which components of quality of life in SSc are most relevant, and how they relate to patient and clinical characteristics. This process can assist in the planning of interventions aimed to improve HRQOL. Because the PGI is a personalized generic instrument, it can also be useful to compare patient-selected relevant HRQOL domains across diseases. Before using the PGI in clinical trials and outcome studies, additional data are needed to establish its reliability, validity, and responsiveness to change in longitudinal studies. Studies using the MACTAR and HAQ indicate that patients' priorities change over time, and minimally important differences should be assessed^25,27. Future uses of the PGI should evaluate these changes.

Limitations of our study include the limited geographic setting from which the patients were recruited, the lack of heterogeneity in the demographic characteristics, and a small sample size. Previous studies have also highlighted the potential bias in a “prompt list.”²⁸ Our study did have a list of possible areas of the patients' lives that could be affected by SSc. Of the 258 patient responses obtained, about 35% were similar to examples from the predefined list provided to patients. This could have influenced respondents' perspectives of what areas of their life are important to them. Although the PGI allows participants to select nonprompted areas of HRQOL, providing examples may influence patients' responses. Finally, our study is limited in that it represents a cross-sectional view into patients' lives, and therefore we were not able to evaluate its responsiveness over time.

Our findings show that the PGI is a valid instrument to assess HRQOL in patients with SSc. Social health was identified by patients as the most important component in their lives affected by their disease, a domain that may not be accurately captured by other commonly used outcome measures. Additional research is needed to ascertain the responsiveness and potential role of personalized measures such as the PGI in longitudinal outcome studies and clinical trials in patients with SSc.