Psychometric Properties of the PROMIS Short Form Measures in a U.S. Cohort of 961 Patients with Chronic Hepatitis C Prescribed Direct Acting Antiviral Therapy

Structured Summary

Background

To better understand symptoms experienced by patients infected with chronic hepatitis C virus (HCV), valid and reliable patient-reported outcome (PRO) measures are needed.

Aim

To assess the reliability and validity of 10 Patient-Reported Outcomes Measurement Information System (PROMIS) measures and the Headache Impact Test-6 (HIT-6) in a large national sample of patients with HCV.

Methods

Pre-treatment data from 961 patients with HCV starting direct acting antiviral therapy at 11 U.S. liver centers were analyzed. Internal reliability was evaluated using Cronbach’s alpha coefficient; frequency distributions were examined for floor and ceiling effects; structural validity was investigated via item-response-theory models; convergent validity was evaluated using correlations with theoretically-similar items from the HCV-PRO and Memorial Symptom Assessment Scale (MSAS); and known-groups validity was investigated by observing PRO differences by liver disease status and number of comorbidities.

Results

The HIT-6 and the majority of the PROMIS measures yielded excellent reliability (alphas ≥ 0.87). Ceiling effects were infrequent (<4%), while 30–59% of patients reported no symptoms (floor effects). The data supported structural validity of the HIT-6 and most PROMIS measures. The PROMIS measures showed moderate to strong correlations with theoretically-similar items from the HCV-PRO and MSAS (0.39 – 0.77). Trends were observed between worse PRO scores and advanced cirrhosis and greater number of comorbidities, lending support for known-groups validity.

Conclusions

The psychometric properties of the HIT-6 and PROMIS measures performed satisfactorily in this large cohort of patients with HCV starting direct acting antiviral therapy. Opportunities exist for further refinement of these PROs. Evaluation of performance over time and in under-represented subgroups is needed.

Introduction

People living with chronic hepatitis C virus (HCV) infection often report a broad array of physical and mental symptoms^1–4 including but not limited to, fatigue, musculoskeletal pain, poor appetite, nausea, abdominal pain, cognitive impairment, depression, anxiety, irritability and sleep distubance^1,5. Numerous studies have investigated health related quality of life (HRQOL) in patients living with chronic HCV^6,7. However, a comprehensive quantitative analysis of patients’ experiences of specific symptoms that may be associated with HCV has not been conducted⁵.

The new direct acting antiviral (DAA) therapies to treat HCV are very well tolerated compared to previous treatments, nonetheless, they can still cause side effects, such as fatigue, nausea, and headache^8–11. Thus far, our understanding of patients’ experiences with DAA therapy has been limited to data derived from industry-sponsored DAA trials^12,13. Those studies provided very useful information regarding the effects of DAAs on patients’ HRQOL, work productivity, and fatigue¹⁴. However, no other symptom experiences have been evaluated from the patients’ perspective during DAA therapy.

The impact of disease, treatment and viral cure on peoples’ lives is nearly impossible to understand without direct input from patients themselves. Patients’ experiences are evaluated using patient-reported outcome (PRO) measures, defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else”¹⁵. PRO measures are the best way to capture patients’ experiences about how a disease, its treatment, or cure affects their lives in a meaningful way.

A recent literature review identified 22 key PRO concepts that were found to be important to people living with HCV in qualitative studies⁵. The majority of these concepts are specific symptoms such as depression, fatigue, anxiety, cognitive dysfunction, musculoskeletal pain, irritability, sleep problems, lack of appetite, and gastrointestinal symptoms. Many of these specific symptoms that concern patients have received inadequate attention in HCV clinical studies. Of the 18 PRO measures utilized in HCV clinical studies, only four have actually been developed or validated in the HCV population, three of which are broad HRQOL measures which are not designed to thoroughly measure specific symptoms or side effects^6,16,17.

In response to increasing recognition of the importance of evaluating patients’ experiences of illness and treatment, the National Institutes of Health Patient-Reported Outcomes Measurement Information System^® (PROMIS) initiative has developed a comprehensive well-evaluated set of self-report tools, some of which measure specific symptoms^18,19. PROMIS measures have been used in a few hepatology studies^20,21, one of which validated several instruments delivered via a computerized adaptive testing system in patients with cirrhosis²⁰. PROMIS measures have several advantages over other PRO measures: items were developed using qualitative and quantitative methods; surveys are designed to measure a single symptom and not overlap with others; surveys are brief (4–8 items); and scores can be compared across other health conditions and populations.

In the current study, we evaluated the reliability and validity of several PROMIS short form measures and the Headache Impact Test (HIT-6), a symptom not assessed by PROMIS²². The specific aims of the study were to evaluate the following psychometric properties: (1) internal reliability; (2) floor and ceiling effects; and (3) construct validity (structural validity, convergent validity, and known-groups validity).

Materials and Methods

Parent study

The psychometric properties of PROMIS measures were evaluated using baseline data from the Patient-Reported Outcomes Project of HCV-TARGET (“PROP UP”) (Clinical trial.gov: NCT02601820). A complete description of the PROP UP study is reported elsewhere²³. Briefly, PROP UP is a multi-site, longitudinal, observational, study designed to enroll 1,600 patients with chronic HCV infection at 11 U.S. liver centers to characterize patients’ experiences before, during and after DAA therapy. At five time points during the study_, patients completed multiple PRO measures in the clinic, via email link to a data capture system, or through phone interview with trained staff from a centralized call center. Completion of surveys at each time point averaged 25 minutes (range: 15–45 minutes). Recruitment began in January 2016. All sites received approval from their local Institutional Review Board prior to recruitment.

Study Design

Data from baseline assessments were used to conduct a cross-sectional study of the reliability and validity of the HIT-6 and 10 PROMIS measures. Patients who consented and completed baseline surveys were included in this analysis.

Research Participants

Inclusion/exclusion criteria have been previously reported²³. Briefly, patients were invited to participate in PROP UP if they were diagnosed with chronic HCV infection of any genotype; English-speaking; age 21 years or older; and prescribed one of five DAA regimens. For this analysis, we included data from a cohort of 961 patients who had completed baseline surveys at the time of data retrieval (11/28/2016).

Measures

PROMIS measures

The 10 PROMIS short forms included Fatigue-7a, Depression-8a, Anger-5a, Anxiety-4a, Pain Interference-8a, Sleep Disturbance-8a, Applied Cognition-General Concerns-8a, Belly Pain-6, Diarrhea-6, and Nausea/Vomiting-4 (available via www.HealthMeasures.net). The short forms were used to measure symptoms most common or salient to HCV or DAA therapy. Each PROMIS short form includes a subset of items from a larger item bank that were the best performing items in terms of content validity and reliability^18,19. PROMIS scores were scaled to standardized T-scores, with a mean of 50 and standard deviation of 10 in the U.S. general population. An exception is Sleep Disturbance, which was normed on a clinical and general population. Higher scores indicate worse symptoms. Studies in other medical populations suggest that the minimally important difference within or between groups generally ranges from 2–5 points^24–27.

Headache Impact Test-6 (HIT-6)

Headaches have been reported as an adverse event during industry-sponsored DAA trials^8–11, but no PROMIS instrument specifically measures this symptom. Therefore, we used the 6-item Headache Impact Test to measure headaches that could occur during treatment²². The six items are responded to on a 5-point Likert scale ranging from “Never” to “Always.” The final score is summed and can range from 36 to 78, with higher scores reflecting worse headaches.

Memorial Symptom Assessment Scale (MSAS)

The MSAS is a reliable and validated instrument used to evaluate 32 symptoms associated with common medical conditions treatment^28,29. Many of these symptoms may be experienced by patients with HCV or undergoing treatment. Participants first indicate the presence or absence of the 32 symptoms, and if present, then rate the symptom on three attributes (severity, frequency, interference with functioning) on a scale of 1 to 4. Each total item score ranges from 0 (if symptom is absent) to 4 (if present, the average of three attributes). Higher scores indicate worse symptoms. In this study, we used the MSAS as a comparison legacy measure (i.e., a previously evaluated measure), using several symptom item scores to evaluate convergent validity with specific PROMIS surveys (e.g., MSAS total pain score correlated with PROMIS Pain Interference score). Patient data were excluded from the analysis if more than 13% of items were missing, as recommended by the MSAS developers²⁸.

HCV-PRO

The HCV-PRO is a disease-specific survey that assesses well-being and HRQOL in people with chronic HCV^30,31. The measure was developed in accordance with the Food and Drug Administration guidelines and demonstrates good reliability and validity and was used as a second legacy measure. The survey includes 16 items with responses ranging from “1=all of the time” to “5=none of the time” with the sum transformed to a 0–100 scale. In contrast to the other PRO measures, a higher HCV-PRO score indicates higher HRQOL. In this study, we used several individual items from the HCV-PRO to evaluate convergent validity with specific PROMIS measures (e.g., HCV-PRO item “feeling tired” correlated with PROMIS Fatigue).

Sociodemographics

Patients self-reported their age, sex, race, ethnicity, marital status, education, and employment status.

Number of health comorbidities

Patients completed a 35-item medical history form to indicate the presence or absence of various common health conditions.

Presence or absence of cirrhosis

Classification of cirrhosis status (Yes/No) was based on one or more of the following source documents reviewed by trained study coordinators at each site: Transient Elastography (e.g., Fibroscan, Echosens, Waltham, Massachusetts, USA), blood serum markers (e.g., FibroSure, Laboratory Corporation of America, Burlington, North Carolina, USA)), and treating clinician’s impressions (based on cumulative medical history, liver biopsy, ultrasound, procedures, physical exam, laboratory tests). An expert hepatologist (M.W.F.) assisted with cross-checking cirrhosis status with other variables in the dataset to confirm classification, such as treatment duration and use of ribavirin, laboratory values, and the following blood serum markers: AST to platelet ratio index (APRI) > 2.0³², FIB-4 score > 3.25³³, or Model for End-Stage Liver Disease (MELD) score >6³⁴. APRI > 2.0 and MELD > 6 are suggestive of cirrhosis, FIB-4 > 3.25 is suggestive of advanced fibrosis, and MELD>12 is suggestive of advanced cirrhosis^32,33.

Statistical Analysis Strategy

The internal consistency reliability of each measure was evaluated using point- and interval-estimates of Cronbach’s alpha. We considered alpha > 0.70 to be an acceptable minimum criterion for establishing reliability in the cohort of patients studied^35,36. For generalization from our sample to a target population, we infer reliability in that population if the observed lower 95% confidence limit for alpha exceeds 0.70³⁷. Due to the substantial number of participants who did not experience the symptom, the estimates of Cronbach’s alpha for the PROMIS Belly Pain, Diarrhea, and Nausea/Vomiting were computed using only the data from participants who reported the presence of that symptom because many patients did not experience this symptom at all. In sensitivity analyses used to evaluate the impact of this approach, the alpha estimates were also computed based on all participants.

Ceiling and floor effects were evaluated in terms of the proportion of participants who had the maximum score (or minimum score)³⁸. Skewness and other properties of the frequency distributions of the scores were examined via graphical descriptive methods.

Factor-analytic assessment of structural validity (i.e., underlying unidimensionality) of each measure relied on unidimensional and multi-dimensional graded-response item-response-theory models³⁹. Overall model fit was evaluated based on the root mean square error of approximation (RMSEA)⁴⁰. An RMSEA value less than or equal to 0.06 was considered to indicate good fit, a value within 0.06 - 0.08 is a fairly good fit, and a value above 0.10 is a poor fit^41. The S chi-square test procedure was used to test the null hypothesis which states that the item-level fit is adequate⁴². Relative to longer questionnaires, shorter questionnaires provide less power and precision to detect and quantify item-level misfit. We assumed that item-level fit for the short-form measures is similar to the item-level fit for long-form measures. The standardized local dependence (LD) chi-square test procedure was used to detect items that are excessively related after controlling for the underlying domain: LD test statistic values larger than 10.0 identified items with substantial LD ⁴³. In sensitivity analyses, to evaluate the robustness of the main results to reasonable perturbation of the methods used, the LD tests were adjusted for multiple comparisons. Sensitivity analyses also included an additional check on the underlying dimensionality of the measure performed by fitting a bi-factor graded-response item-response-theory modelwith each identified LD pair or set of items as a second order factor^44.. Any LD violations were deemed negligible if the explained common variance (ECV) was at least 0.90^45,46. ECV represents the variance explained by the general factor in the bi-factor model.

To evaluate convergent validity, we used the absolute value of point- and interval-estimates of the Spearman correlations (ρ) between measures of theoretically-similar domains. For this purpose, for each of the PROMIS and HIT-6 measures, we identified item pairs which, logically, should be positively correlated. We anticipated that there should exist substantial positive correlations between those item pairs. We used Dancey and Reidy’s classification which specifies that 0.40 – 0.70 is the range of a “moderate” correlation⁴⁷.

Known-groups validity helps to support construct validity if it can be demonstrated that an instrument discriminates between groups that are known to differ on a given construct. The extant literature in HCV is extremely limited on definitive groups that would differ according to precise symptoms. However, a few studies, though not all, suggest that HRQOL and one symptom, fatigue, may differ among patients with and without cirrhosis^16,48–50. Based on this literature, the a priori analysis plan was to compare patients categorized as cirrhotic or noncirrhotic based on medical chart review (as described above). Secondarily, we used a three-category approach to better observe PRO scores in patients with and without cirrhosis, stratified by their baseline MELD scores: non-cirrhosis according to medical chart review; MELD score 6–12; and MELD >12. Thirdly, based on findings from Rothrock et al., we evaluated potential known-groups validity by exploring the association between the number of health comorbidities as a continuous variable with symptom scores, speculating that patients with more comorbid conditions should experience greater symptoms⁵¹.

Unless otherwise specified, all statistical computations were performed using SAS System software version 9.4 (SAS Institute, Cary, NC). Analysis of the graded-response item-response-theory models were performed using the SAS IRT procedure and using IRT-PRO software, version 4.1 (Scientific Software International, Lincolnwood, IL). PROMIS T-scores were computed using RStudio software, version 1.0.136 (RStudio Inc.).

Results

Patient characteristics

The characteristics of the 961 study participants at baseline are presented in Table 1. Mean age was 57 (SD= 10.7) years old. In addition to having chronic HCV and liver disease, participants reported an average of 4 other health comorbidities (range 0–15).

Table 1.

Patient Characteristics (N=961)

	Na	Mean (SD), Median	Range
Age (years)	955	57 (10.7), 58	23–82
Number of Comorbidities	961	4.1 (2.9), 4	0–15
	Na	Categories	N (%)
Sex	951	Male	523 (55.0)
		Female	428 (45.0)
Race	948	White	581 (61.3)
		Black or African American	311 (32.8)
		Other	56 (5.9)
Ethnicity	907	Not Hispanic or Latino	760 (83.8)
		Hispanic or Latino	37 (4.1)
		Other	110 (12.1)
Marital status	953	Married/in committed relationship	357 (37.5)
		Single	345 (36.2)
		Separated/Divorced/Widowed	251 (26.3)
Education	953	High school or equivalent diploma	527 (55.3)
		Vocational school or higher	426 (44.7)
Employment	937	Unemployed/Disabled/Applying	460 (49.1)
		Working full/part time	369 (39.4)
		Retired/Homemaker/Student	108 (11.5)
Cirrhosis status	912	Present	456 (50.0)
		Absent	456 (50.0)

Note:

^aN is the number of non-missing values. Missing data ranged from 0% to 5.6%; collectively only 1.5% of data values were missing. SD=Standard Deviation.

Missing Values

The frequency of missing data among the HIT-6 and PROMIS short form scores were less than 2.2%. The frequency of missing data for the HCV-PRO score was 7.7%. We judged the reasons for the incomplete data to satisfy the “missing completely at random “ criterion⁵².

Internal Consistency Reliability

Reliability of each of the symptom domains from the HIT-6, PROMIS short forms, and HCV-PRO are presented in Table 2. The HIT-6 and HCV-PRO measures had high reliability (all Cronbach’s alphas >0.90). All PROMIS short forms, except the PROMIS Nausea/Vomiting, yielded very good reliability estimates between 0.87 and 0.98.

Table 2.

Reliability of the HIT-6, PROMIS short forms, and HCV-PRO (N=961)

Measure	N	Items	Alphaa	95% C.I.b	N	Floor (%)c	Ceiling (%)c
HIT-6 scored	944	6	0.93	[0.92, 0.94]	944	30.9	0.2
PROMIS T-scoresd
Fatigue	941	7	0.87	[0.86, 0.88]	960	1.6	0.2
Pain Interference	944	8	0.98	[0.98, 0.98]	961	36.7	3.9
Sleep Disturbance	945	8	0.94	[0.93, 0.95]	960	6.8	2.5
Depression	942	8	0.96	[0.95, 0.96]	960	33.9	0.7
Cognitive Concerns	940	8	0.96	[0.96, 0.97]	961	30.1	1.4
Anger	958	5	0.91	[0.91, 0.92]	961	18.3	1.0
Anxiety	954	4	0.91	[0.90, 0.92]	954	40.0	0.6
Belly Paine	470	6	0.87	[0.85, 0.89]	960	49.7	0.1
Diarrheae	388	6	0.88	[0.87, 0.90]	948	58.5	0.3
Nausea/Vomitinge	452	4	0.63	[0.58, 0.69]	949	52.4	0.1
HCV-PROf	887	16	0.94	[0.94, 0.95]	887	0.1	5.9

Note:

^aCronbach’s alpha coefficient.

^b95% confidence interval (C.I.) for Cronbach’s alpha.

^cProportion of responses at the minimum (floor) or maximum (ceiling) of the scale.

^dHigher values of HIT-6 and PROMIS T-scores indicate more severe symptoms.

^eOnly patients who reported presence of the symptom were analyzed.

^fShown for reference in examining the properties of the HIT-6 and PROMIS scores. Higher values of HCV-PRO score indicate higher functional well-being.

Floor and Ceiling Effects

Ceiling effects were consistently low among all PRO measures (<4% of participants had scores at the upper limit), suggesting that all measures performed well in capturing severe symptoms at the upper limits of the scales (See Table 2). In contrast, many patients did not report symptoms and therefore notable floor effects (range 30% to 59%) were observed for the HIT-6 and a majority of the PROMIS measures. Floor effects were small for PROMIS Fatigue (1.6%) and Sleep Disturbance (6.8%) indicating that most patients reported at least some mild fatigue and sleep disturbance problems. The HCV-PRO had a negligible floor (0.10%) effect and a small ceiling effect (5.9%).

Structural Validity

Findings supported the unidimensionality of the HIT-6, PROMIS measures, and HCV-PRO (Table 3). Eight PRO measures had a very good fit (RMSEA ≤ 0.06). None of the measures had a poor fit (RMSEA > 0.10). The majority of the confirmatory factor analysis models fitted to the measures produced high factor loadings (>0.70). Exceptions were noted for the PROMIS Fatigue, Nausea/Vomiting and HCV-PRO, which showed multidimensionality. The PROMIS Fatigue measure had a small factor loading (0.30) for one item (“How often did you have enough energy to exercise strenuously”). This item is the only item on the survey that is reverse-scored and may explain why it did not hang together with the other items. Nonetheless, a good fit was suggested by the small RMSEA value of 0.04 and the absence of local dependence (LD). The PROMIS Nausea/Vomiting survey and HCV-PRO had items with factor loadings below 0.70 likely because these measures tap into multiple constructs (e.g., nausea and vomiting). Despite this, the overall model fit was still very good with RMSEA values < 0.06.

Table 3.

Structural Validity of the HIT-6, PROMIS short forms, and HCV-PRO (N=961)

Measure	Items	Factor Loadingsa	RMSEAb	ECVc
HIT-6 scored	6	0.86 – 0.97	0.06	---e
PROMISd
Fatigue	7	0.30f, 0.76 – 0.91	0.04	--- e
Pain Interference	8	0.95 – 0.98	0.09	0.96
Sleep Disturbance	8	0.84 – 0.92	0.08	0.86
Depression	8	0.90 – 0.96	0.05	0.96
Cognitive Concerns	8	0.90 – 0.96	0.07	0.98
Anger	5	0.88 – 0.92	0.06	0.96
Anxiety	4	0.89 – 0.93	0.05	0.96
Belly Pain	6	0.93 – 0.98	0.05	0.99
Diarrhea	6	0.83 – 0.98	0.07	0.87
Nausea/Vomiting	4	0.66g, 0.84 – 0.96	0.05	--- e
HCV-PROh	16	0.55i, 0.67j, 0.71 – 0.86	0.04	0.82

Note:

^aFactor loadings obtained from the unidimensional confirmatory factor analysis. Factor loadings >0.70 support unidimensionality.

^bRMSEA=Root Mean Square Error Approximation. RMSEA values ≤ 0.06 indicate good fit, values ≤ 0.08 are fair, and values above 0.10 generally reflect poor fit. Statistics were based on full marginal tables. Model-based weight matrix was used.

^cECV=Expected Common Variance. ECV represents the variance explained by the general factor in the bifactor model. ECV > 0.90 support unidimensionality.

^dHigher values of HIT-6 and PROMIS T-scores indicate more severe symptoms.

^eUnidimensionality was assessed using a one-factor confirmatory factor analysis.

^fFactor loading of the item: “In the past 7 days, how often did you have enough energy to exercise strenuously?”.

^gFactor loading of the item: “In the past 7 days, how often did you have a poor appetite?”.

^hShown for reference in examining the properties of the HIT-6 and PROMIS scores. Higher values of HCV-PRO score indicate higher functional well-being..

^IFactor loading of the item: “Having Hepatitis C affected my sex life”.

^jFactor loading of the item: “Having Hepatitis C was very stressful to me”.

Convergent Validity

As shown in Table 4, both the HIT-6 score and the PROMIS Belly Pain score were moderately correlated with the HCV-PRO item “I felt bothered by pain or physical discomfort” and with the MSAS pain score (range 0.37 to 0.48).

Table 4.

Correlation of the HIT-6 and PROMIS short forms with HCV-PRO and MSAS items (N=961)

Measure	Corresponding Measure	Na	Correlationb	95% C.I.c
HIT-6 scored	HCV-PRO Item: I felt bothered by pain or physical discomfort	935	0.45	[0.39, 0.50]
	MSASe pain score	936	0.37	[0.31, 0.43]
PROMIS T–scored
Fatigue	HCV-PRO Item: I felt too tired during the day to get done what I needed	958	0.74	[0.70, 0.77]
	MSAS lack of energy score	949	0.77	[0.74, 0.80]
Pain Interference	HCV-PRO Item: I felt bothered by pain or physical discomfort	951	0.74	[0.70, 0.77]
	MSAS pain score	952	0.76	[0.73, 0.79]
Sleep Disturbance	HCV-PRO Item: I had difficulty sleeping or slept too much	954	0.74	[0.71, 0.78]
	MSAS difficulty sleeping score	949	0.77	[0.75, 0.80]
Depression	HCV-PRO Item: I felt downhearted and sad	951	0.67	[0.63, 0.71]
	MSAS feeling sad score	949	0.64	[0.60, 0.68]
Cognitive Concerns	HCV-PRO Item: I was unable to think clearly or focus on my thoughts	957	0.78	[0.75, 0.81]
	MSAS difficulty concentrating score	951	0.67	[0.64, 0.71]
Anger	HCV-PRO Item: I felt restless or on edge	952	0.62	[0.58, 0.67]
	MSAS feeling irritable score	943	0.58	[0.54, 0.63]
Anxiety	HCV-PRO Item: Having Hepatitis C was very stressful to me	948	0.48	[0.43, 0.53]
	MSAS feeling nervous score	941	0.56	[0.51, 0.61]
	MSAS worrying score	939	0.61	[0.57, 0.66]
Belly Pain	HCV-PRO Item: I felt bothered by pain or physical discomfort	950	0.48	[0.42, 0.53]
	MSAS pain score	951	0.39	[0.34, 0.45]
Diarrhea	MSAS diarrhea score	937	0.51	[0.46, 0.57]
Nausea/Vomiting	MSAS nausea score	939	0.64	[0.60, 0.69]
	MSAS vomiting score	940	0.35	[0.29, 0.40]

Note:

^aThe number of participants with a non-missing value.

^bAbsolute value of Spearman’s rank correlation coefficient between each measure and a similar construct.

^c95% confidence interval (C.I.)for Spearman’s rank correlation coefficient.

^dHigher values on the HIT-6 and PROMIS scores indicate more severe symptoms.

^eMSAS = Memorial Symptom Assessment Scale. MSAS score = average of the three attributes: severity, frequency, and interference.

The PROMIS Pain Interference score was strongly correlated with the HCV-PRO item “I felt bothered by pain or physical discomfort” (0.74) and MSAS pain score (0.76). PROMIS Fatigue, Sleep Disturbance, and Cognitive Concerns were strongly correlated with HCV-PRO items and MSAS scores of similar constructs (range 0.67 to 0.78). PROMIS Depression, Anger, Anxiety, and Diarrhea were moderately correlated with specific HCV-PRO items and MSAS scores of similar constructs (range 0.48 to 0.67). PROMIS Nausea/Vomiting was moderately correlated with the MSAS vomiting and nausea items (0.35 and 0.64, respectively).

Known-Groups Validity

Differences in PRO scores between patients defined as noncirrhotic and cirrhotic via medical chart review were negligible (data not shown). In a subsequent analysis (Table 5) of three liver disease categories (i.e., noncirrhotic, MELD 6–12, MELD>12), a severity-response trend was observed such that worse symptoms were associated with advancing liver disease. In particular, patients with MELD>12 had higher (worse) scores on all symptoms compared to the other two groups, with mean differences ranging from 1.9 to 4.1. While the sample size of those with MELD>12 was small (n=37) and provided limited precision, the mean differences were comparable to established PROMIS minimally important differences reported in other medical populations^24–27. In the final known-groups validity analysis, we found positive correlations between a greater number of comorbid health conditions and worse symptom scores, consistent with prior studies⁵¹. Patients with more comorbid conditions had worse scores on PROMIS Fatigue, Sleep Disturbance, Pain Interference, and HRQOL on the HCV-PRO (moderate correlations ranging from 0.39 to 0.53). Smaller positive correlations were observed between the number of comorbid conditions and worse scores on PROMIS Depression, Cognitive Concerns, Anger, Anxiety, Belly Pain, Nausea, and HIT-6 (ranging from 0.29 to 0.34).

Table 5.

Known-Groups Validity: PROs scores by liver disease status

	Group A		Group B
Measure	No cirrhosis N=456a	Cirrhosis (MELD 6–11) N=334b	Cirrhosis (MELD≥12) N=37c	Group A – Group Bd
	Mean (SD)	Mean (SD)	Mean (SD)	Mean Difference (95% CI)
HIT-6 scoree	46 (11)	45 (10)	47 (11)	−1.5 [−4.9, 2.0]
PROMIS T-scorese
Fatigue	52 (11)	52 (11)	54 (10)	−1.9 [−5.6, 1.7]
Pain Interference	53 (12)	53 (11)	56 (10)	−3.0 [−6.7, 0.7]
Sleep Disturbance	52 (11)	52 (12)	54 (10)	−2.4 [−6.2, 1.4]
Depression	50 (11)	49 (11)	54 (9)	−4.1 [−7.6, −0.6]
Cognitive Concerns	33 (9)	33 (9)	35 (8)	−1.8 [−4.9, 1.2]
Anger	49 (12)	48 (11)	51 (10)	−2.9 [−6.7, 0.8]
Anxiety	51 (11)	49 (10)	54 (10)	−3.5 [−6.9, 0.0]
Belly Pain	37 (13)	37 (13)	40 (14)	−3.3 [−7.7, 1.1]
Diarrhea	36 (9)	35 (8)	39 (12)	−3.8 [−7.7, 0.2]
Nausea/Vomiting	42 (9)	42 (9)	46 (9)	−3.9 [−6.8, −1.0]
HCV-PROf	71 (23)	71 (23)	63 (23)	7.7 [0.0, 15.3]

Note: SD=Standard Deviation; CI=Confidence Interval

^aSample size ranges from 415–456.

^bSample size ranges from 312–334.

^cSample size ranges from 36–37.

^dGroup A = No cirrhosis or mild cirrhosis (MELD 6–12), Group B = Advanced Cirrhosis(MELD>12).

^eHigher values indicate more severe symptoms.

^fHigher values indicate higher functional well-being.

Discussion

The objectives of the current study were to establish the reliability and validity of several PRO measures, specifically 10 PROMIS measures and the HIT-6, to evaluate specific symptoms in patients with chronic HCV. Given the large national sample of patients recruited for the PROP UP study, it represented a reasonable platform in which to evaluate the psychometric properties of several new PRO measures that could be useful in future patient-centered outcomes research in the field of chronic HCV. The overall findings from this study suggest that the psychometric properties of these PROs are sufficiently satisfactory to be used in future HCV studies, with specific caveats noted below.

The internal consistency of the symptom scales was well above the acceptable limit of 0.70 (range: 0.87 – 0.98) indicating very good to excellent reliability. Only one exception was noted for the PROMIS Nausea/Vomiting measure which appears to be multidimensional. This measure had a lower reliability because its four items tap three constructs (nausea, poor appetite, vomiting), which while associated, are not highly correlated.

None of the PROs had notable ceiling effects, which is critically important to investigations that need to capture very severe symptoms, as might be the case with patients listed for liver transplantation. Thus, these PROs are suitable to evaluate patients’ experiences of severe and debilitating symptoms.

In contrast, floor effects were notable for most of the PROs, with the exception of the HCV-PRO and PROMIS Fatigue and Sleep Disturbance measures. Floor effects can be present when a measure does not represent the full range of possible human experiences, but we do not believe this to be the case with the PROMIS and HIT-6 measures. Floor effects can also be evident when the PRO score has both a binary component (presence or absence of the symptom) and a continuous component representing the frequency or severity of a symptom, when present. In the current sample of patients starting new DAA therapy, we would anticipate that approximately half of these patients would be asymptomatic for any one of the various symptom scales, thus producing minimal scores and demonstrating sizable floor effects. Floor effects occurred in this study because many patients did not have the symptom: 59% reported no diarrhea, 52% reported no nausea/vomiting, 50% reported no abdominal pain, and 37% reported no pain. These floor effects are not surprising given that half of the sample did not have cirrhosis and specific symptoms (nausea/vomiting, abdominal pain) may not be present in patients in the absence of advanced cirrhosis. Viewed from another perspective, these data indicate that 41–63% of patients actually do experience mild to severe symptoms, representing a substantial number of patients. One caveat for investigators who need to capture subtle differences in symptoms in patients with mild disease, these PROs may lack sufficient sensitivity. Future studies could evaluate if additional items from the PROMIS item banks would increase variability in scores of patients with mild disease. Additionally, a variety of statistical analysis methods designed for measures that have both a binary component (presence or absence) and continuous component may be useful; for example, mixture models such as a zero-inflated log-normal model can be applied for improved interpretability and precision.

The structural validity of the PROMIS surveys and the HIT-6 supported unidimensionality of most surveys, although not all. Strong support for the unidimensionality for the HIT-6, and PROMIS Depression, Cognitive Concern, Anger, Anxiety and Belly Pain scales was observed. Some surveys tap multidimensional constructs (e.g., PROMIS Nausea/Vomiting), while other surveys may include an item that does not hang together well with the other items (e.g., PROMIS Fatigue).

The assessment of convergent validity in this study focused on the strength of correlation between the HIT-6 and PROMIS surveys with theoretically-similar items from two previously validated legacy measures: the HCV-PRO and the MSAS. Almost all surveys demonstrated moderate (0.40 – 0.70) to strong (>0.70) correlations with items tapping theoretically-similar constructs from the HCV-PRO and MSAS, thus providing evidence that these instruments measure the purported construct. These findings are consistent with another recent psychometric study conducted in cirrhotic patients that documented satisfactory convergent and discriminant validity for the same PROMIS measures²⁰. A few weak to moderate correlations were observed for the HIT-6 and PROMIS Belly Pain, which may be because the general “pain” item we used to evaluate headache and abdominal pain was too nonspecific. The absence of other legacy measures or items impeded our ability to adequately examine convergent validity for a couple of scales, representing one limitation of this study.

Finally, we explored known-groups validity in patients with and without cirrhosis, with varying levels of liver disease, and with a range of comorbid medical conditions. Differences in PRO scores were negligible between patients with no cirrhosis and mild cirrhosis, however, patients with advanced cirrhosis (MELD>12) had worse symptom scores than patients with no or minimal cirrhosis, by 1.9 to 4.1 point differences. In particular, it is reasonable to expect that gastrointestinal symptoms (nausea, abdominal pain, diarrhea) and pain would be worse in patients with advanced cirrhosis due to decompensation compared to those with minimal or no cirrhosis (point differences ≥3). This finding is consistent with Bajaj et al. who found differences between patients with compensated and decompensated cirrhosis on several of the same PROMIS measures²⁰. The lack of differentiation in this study between patients with no cirrhosis and those with mild cirrhosis may be because the former group had stage 2–3 fibrosis, while the latter group had early stage 4 fibrosis; the small difference in liver disease stage may explain the similarity in symptoms in these two groups. Unfortunately, liver disease staging data are unavailable to substantiate this hypothesis, although we know that at the time of enrollment into this study, many payers only covered DAA therapy in patients with advanced fibrosis or cirrhosis. Future studies with the PROMIS measures could determine if other items from the larger item banks could provide better differentiation between these fibrosis levels. In a second analysis of know-groups validity, we found evidence of an association between the PROMIS measures and patients’ number of comorbidities, such that patients with more comorbidities reported worse symptoms⁵¹. These analyses support known-groups validity by differentiating between patients with varying degrees of comorbidities and liver disease.

The PROMIS measures, HIT-6, and HCV-PRO complement other PROs that have been used to study the subjective experiences of patients with HCV. A recent literature review identified 22 concepts, almost all precise symptoms, described as important to patients that have been inadequately examined in HCV clinical trials⁵. Of 18 PRO measures utilized in previous studies, only four were satisfactorily developed and/or validated in patients with HCV, and three are quality of life instruments. As for specific symptoms, only the Fatigue Severity Scale had previously been validated in patients with HCV⁵³. Investigators can now use the current PROMIS and HIT-6 measures to evaluate several additional precise symptoms that are important to patients yet completely under-studied.

Our study has limitations that point to future directions for improvements in PRO measurement in the field of HCV. All patients were English speaking. Survey data were collected via various data collection modalities and our limited data preclude evaluation of social desirability or effects of data collection modality. Other legacy instruments were not utilized in the parent study due to concern about patient burden and could be an area of future investigation to explore convergent and discriminant validity. Most of the PRO measures showed floor effects, ostensibly because many patients were not experiencing a specific symptom. There is a chance that these measures may not be sensitive enough to capture very mild side effects of DAA therapy or subtle symptom improvement associated with viral cure. However, investigators could seek to improve the surveys by identifying additional items from the large PROMIS item banks to create greater variability at the healthier end of the spectrum. Likewise, future studies might identify other items from the PROMIS banks that help surveys better distinguish between patients with no cirrhosis and early cirrhosis. There may be other symptoms of HCV that were not evaluated in this study (e.g., itching). These PROs were chosen to capture a range of burdensome symptoms; however it would be interesting to explore positive patient experiences after being cured of HCV, that are not captured by these surveys (e.g., positive growth, psychological freedom, happiness) that are not just the absence of symptoms, but altogether different human experiences. Future studies might evaluate the responsiveness of the PRO measures over time and in other patient subpopulations who might have worse symptomatology (e.g., patients with certain sociodemographic characteristics or those with alcohol, drug use and mental health issues)^54–58.

To conclude, the current study found the psychometric properties of 10 PROMIS short forms, the HIT-6, and the HCV-PRO performed satisfactorily in patients with chronic HCV. Slight modifications to the surveys may enhance performance among healthy subjects and better distinguish between patients with and without cirrhosis. Our study indicates that these symptom PROs can be used in clinical research investigations of similar patients and clinical settings.

Table 6.

Known Groups Validity: Correlations between PRO scores and Number of Comorbidities

Measure	Number of Comorbidities
	Spearman’s correlationa (95% CI)
HIT-6 scoreb	0.31 [0.25, 0.37]
PROMISb
Fatigue	0.39 [0.34, 0.44]
Pain Interference	0.53 [0.48, 0.57]
Sleep Disturbance	0.41 [0.36, 0.46]
Depression	0.30 [0.24, 0.36]
Cognitive Concerns	0.33 [0.27, 0.38]
Anger	0.29 [0.23, 0.35]
Anxiety	0.30 [0.24, 0.36]
Belly Pain	0.34 [0.28, 0.40]
Diarrhea	0.20 [0.14, 0.27]
Nausea/Vomiting	0.29 [0.23, 0.35]
HCV-PROc	−0.48 [−0.53, −0.43]

Note: CI=Confidence Interval; Sample size ranges from 887–961.

^aSpearman’s rank correlation coefficient (Rho)

^bHigher values of HIT-6 and PROMIS T-scores indicate more severe symptoms.

^cHigher values indicate higher functional well-being.