Impact of National Institutes of Health Gastrointestinal PROMIS^® Measures in Clinical Practice: Results of a Multicenter Controlled Trial

Abstract

OBJECTIVE

The National Institutes of Health (NIH) created the Patient Reported Outcomes Measurement Information System (PROMIS^®) to allow efficient, online measurement of patient-reported outcomes (PROs), but it remains untested whether PROMIS improves outcomes. Here, we aimed to compare the impact of gastrointestinal (GI) PROMIS measures vs. usual care on patient outcomes.

METHODS

We performed a pragmatic clinical trial with an off-on study design alternating weekly between intervention (GI PROMIS) and control arms at one Veterans Affairs (VA) and three university-affiliated specialty clinics. Adults with GI symptoms were eligible. Intervention patients completed GI PROMIS symptom questionnaires on an e-portal one week before their visit; PROs were available for review by patients and their providers prior to and during the clinic visit. Usual care patients were managed according to customary practices. Our primary outcome was patient satisfaction as determined by the Consumer Assessment of Healthcare Providers & Systems (CAHPS) questionnaire. Secondary outcomes included provider interpersonal skills (Doctors’ Interpersonal Skills Questionnaire [DISQ]) and shared decision-making (9-item Shared Decision Making Questionnaire [SDM-Q-9]).

RESULTS

There were 217 and 154 patients in the GI PROMIS and control arms, respectively. Patient satisfaction was similar between groups (p>.05). Intervention patients had similar assessments of their providers’ interpersonal skills (DISQ 89.4±11.7 vs. 89.8±16.0, p=.79) and shared decision-making (SDM-Q-9 79.3±12.4 vs. 79.0±22.0, p=.85) vs. controls.

CONCLUSIONS

This is the first controlled trial examining the impact of NIH PROMIS in clinical practice. One-time use of GI PROMIS did not improve patient satisfaction or assessment of provider interpersonal skills and shared decision-making. Future studies examining how to optimize PROs in clinical practice are encouraged before widespread adoption.

INTRODUCTION

Patients often seek care because they experience symptoms that negatively impact health-related quality of life (HRQOL). Healthcare providers must elicit, measure, and interpret patient symptoms as part of their clinical evaluation. To assist with this goal, researchers have developed and validated a wide range of patient-reported outcomes (PROs) across diseases, with a focus on chronic illnesses.(1–3) These PROs, which measure any aspect of a patient’s biopsychosocial health that comes directly from the patient, may help direct care and improve outcomes. When PROs are collected systematically, efficiently, and in the right place at the right time, they may enhance the patient-provider relationship at the center of chronic disease care, improve communication, and help make shared decisions.(4–6)

However, despite the promise of using PROs to guide patient care, there are important challenges to applying PROs in routine practice.(7–12) For example, it can be time consuming to collect PROs from patients and securely transmit the data into the electronic health record (EHR), making it untenable for use in busy practices. There are also many PROs to choose from, with a lack of measurement standards across questionnaires. Furthermore, clinicians note that it can be difficult to understand and act upon PRO scores. When coupled with limited evidence from previous research that administering PROs truly impacts patient outcomes,(2) these challenges limit widespread use of PROs in clinical practice; most providers instead opt for informal measurement of symptoms and function.

In this context, the National Institutes of Health (NIH) created the Patient Reported Outcomes Measurement Information System (PROMIS^®) in 2004 with the goal of developing, validating, and disseminating a toolbox of publicly-available PROs that cover the breadth and depth of the human illness experience while overcoming technical challenges of applying PROs in practice (www.nihpromis.org).(13) Using modern psychometric techniques, such as item response theory and computerized adaptive testing,(14, 15) PROMIS offers state-of-the-art psychometrics, establishes common-language benchmarks for symptoms across conditions, and identifies clinical thresholds for action and meaningful clinical improvement or decline. PROMIS questionnaires are administered electronically and efficiently, allowing implementation in busy clinical settings. Because of the extraordinary burden of illness from digestive diseases, the PROMIS consortium added a gastrointestinal (GI) item bank, which our group developed.(16) Using the NIH PROMIS framework, we constructed and validated eight GI PROMIS symptom scales using data from over 2000 subjects.(16–18)

However, despite over a decade of NIH PROMIS development, it remains unclear whether implementing GI PROMIS, let alone any PROMIS measures, can improve patient outcomes vs. usual care. In this study, we conducted an NIH-supported multicenter controlled trial of PROMIS vs. usual care in clinical practice. Specifically, we employed GI PROMIS measures in diverse patients with active GI symptoms, collected the results via a patient-provider e-portal, and presented the data at the point of care. We hypothesized that compared to usual care, use of GI PROMIS would enhance the patient-provider interaction, leading to improved patient satisfaction and higher patient assessment of provider interpersonal skills and shared decision-making.

METHODS

Study Overview

We performed a pragmatic, multicenter clinical trial comparing use of validated GI PROMIS questionnaires(16) vs. usual care in diverse patients with active GI symptoms, including those with abdominal pain, bowel incontinence, bloating/gas, constipation, diarrhea, dysphagia, heartburn/reflux, and nausea/vomiting. We administered the GI PROMIS questionnaires through a secure, online, patient-provider e-portal (see Appendix Figure 1 for sample screenshots). The portal collected the PRO data and converted responses into a symptom “heat map” (Figure 1) that visually compared each patient’s symptoms against the general U.S. population.(16, 18) Both patients and providers could view this heat map on the portal prior to and during the clinic visit.

FIGURE 1.

Sample “heat map” report of GI PROMIS scores and history of present illness (HPI). Patients complete PROMIS items on the e-portal, and the results are converted into a GI PROMIS symptom heat map and HPI. Patients’ PROMIS scores are compared to the general U.S. population with benchmarks to add interpretability to the scores, similar to a lab test. Both the heat map and HPI are viewable on the e-portal for both the patient and healthcare provider prior to the clinic visit.

To enhance clinical applicability of GI PROMIS, the e-portal auto-composed a complete GI history of present illness (HPI) report triggered off the PROMIS symptoms. Patients were guided through a set of questions measuring the timing, severity, frequency, location, quality, and character of each reported GI PROMIS symptom, along with relevant comorbidities, family history, and alarm features.(19, 20) Once the questions were completed, the information was transformed into a full narrative GI HPI that accompanied the PROMIS heat map (Figure 1). In a previous head-to-head trial comparing GI PROMIS-directed computerized HPIs vs. physician HPIs, we found that the computerized HPIs were rated by blinded reviewers to be of higher quality and more thorough, complete, succinct, and relevant.(19) However, the previous trial did not measure the impact of the PROMIS on patient outcomes.

By tying GI PROMIS scores to a focused HPI, using specific GI symptoms with benchmarked interpretation, and directly presenting the results to the provider at the point of care, we attempted to optimize the impact of using PROMIS. In this manner, the current study sought to overcome traditional critiques of using PROs in clinical practice: i.e., technical difficulties of transmitting to the EHR, interpretability, data visualization issues, and clinical actionability.

Study Design, Patients, and Setting

We used a pragmatic, off-on study design alternating weekly between the PROMIS intervention and control arms. Patients who visited the following clinics were eligible for the study: (i) Cedars-Sinai Medical Center general GI clinic; (ii) West Los Angeles Veterans Affairs (WLAVA) Medical Center general GI clinic; (iii) University of Michigan functional GI and motility clinic; and (iv) University of Michigan scleroderma clinic (selected because scleroderma patients have a high prevalence of GI symptoms). The Cedars-Sinai and WLAVA GI clinics are academic teaching practices staffed by GI attending physicians; the initial evaluation in these clinics were primarily conducted by GI specialty fellows, internal medicine residents, or physician assistants. Conversely, attending physicians primarily staffed the GI and scleroderma clinics at the University of Michigan.

We enrolled patients aged ≥18 years who were scheduled for an initial visit or had not been seen in the clinic within the past eight months. Patients were also required to read and write English and possess basic point-and-click computing skills.

During the control weeks, patients were treated according to all customary practices. In the intervention weeks, eligible patients were mailed a letter one week prior to their appointment inviting them to log on to the e-portal to complete GI PROMIS. Eligible intervention patients who did not complete PROMIS before their visit were also approached during the day of their appointment by research staff and again invited to access the e-portal on a clinic computer prior to seeing their physician. Clinic providers were informed to access the e-portal and view the GI PROMIS symptom heat map for patients that completed PROMIS. In keeping with our pragmatic approach to the study, providers were not mandated to use the PROMIS data or PROMIS-directed HPI. Rather, providers were allowed to make individual decisions on how to employ the PROMIS data report, if at all.

Within 24 hours of completing the clinic visit, patients were sent the post-visit questionnaires to measure their satisfaction with the visit as well as their assessment of their providers’ interpersonal skills and shared decision-making. This study was approved by the Institutional Review Boards (IRBs) at all sites (Cedars-Sinai IRB Pro00041476; University of Michigan IRB HUM00063094; WLAVA IRB PCC#2013-111563).

Primary and Secondary Outcomes

The primary outcome was patient satisfaction as measured by the Consumer Assessment of Healthcare Providers and Systems Clinician & Group Survey 2.0 (CG-CAHPS).(21) Because the CG-CAHPS is a global assessment of patients’ satisfaction with their medical care over the past year, we used selected items that were applicable for assessing patient satisfaction after a single visit. Patients were reminded to answer the questions thinking about their most recent visit to the GI or scleroderma clinic. The answer options for most selected CG-CAHPS items were “Yes, definitely,” “Yes, somewhat,” and “No.” We employed a “top box” approach which is commonly used when reporting CG-CAHPS data(22); a positive response included only “Yes, definitely” while negative responses included “Yes, somewhat” or “No.”

Our secondary outcomes were patient assessments of provider interpersonal skills and shared decision-making. Patients completed the Doctors’ Interpersonal Skills Questionnaire (DISQ) to assess their provider’s interpersonal skills.(23) The DISQ comprised 12 items, each scored on a 5-point scale where 1=“Poor” and 5=“Excellent.” We converted each item to a 100-point scale and averaged the scores for the 12 items to calculate an overall interpersonal skills score.

We used the 9-item Shared Decision Making Questionnaire (SDM-Q-9) to assess patient shared decision-making.(24) The SDM-Q-9 contained 9 items, each scored on a 6-point scale where 1=“Completely Disagree” and 6=“Completely Agree.” Similar to the DISQ, we converted each item to a 100-point scale and averaged the 9 scores to calculate an overall shared decision-making score. Again, for both DISQ and SDM-Q-9, patients were informed to answer the questions thinking about their most recent GI or scleroderma clinic visit.

Covariates

We also collected information on potentially confounding patient- and provider-level variables. Patient-level factors included age, gender, and race/ethnicity. We also collected provider-level factors, including site of care and provider level of training.

Sample Size Calculation

Our primary objective was to measure differences in CG-CAHPS provider rating scores between groups. Although CG-CAHPS is widely used and accepted as a measure of patient satisfaction with outpatient visits, we are unaware of data measuring the minimally clinically important difference (MCID) on the scale. Therefore, the sample size was calculated to achieve an effect size of 0.5 (a half standard deviation difference) in mean CG-CAHPS provider rating scores between groups – an effect size that is moderate and generally correlates with the MCID.(25, 26) Assuming a two-tailed 5% significance level with a power of 80%, the minimum sample size needed to show an effect size of 0.5 was 64 patients per group.

Statistical and Sensitivity Analyses

Statistical analyses were performed using Stata 13.1 (StataCorp LP, College Station, TX). A two-tailed p-value <.05 was considered significant. Our primary analyses were performed from the intention-to-treat perspective. For intervention patients who completed GI PROMIS, but did not return the post-visit outcome surveys, we assumed their outcomes (CG-CAHPS provider rating, DISQ, and SDM-Q-9) were no different than controls. Specifically, the missing outcome data for this group was imputed to the corresponding mean value calculated from controls for each item. Because this assumption biases towards the null, we also performed a sensitivity analysis using a per-protocol approach where we excluded patients without follow-up data.

For bivariate analyses, we used the two-sample t-test and chi-squared test to compare means and proportions, respectively, between groups. We performed a multivariable logistic regression model to identify patient characteristics that were independent predictors of completing PROMIS prior to the clinic visit.

We used linear regression to generate an adjusted p-value and to evaluate differences in CG-CAHPS provider ratings between groups while adjusting for potential confounding patient- and provider-level factors. We used similar approaches when comparing the remaining CG-CAHPS items (Firth logistic regression) and the DISQ and SDM-Q-9 items (linear regression).

RESULTS

Study Population

Figure 2 shows patient flow through the clinical trial. Overall, 502 patients were assigned to the control arm and 3 (0.6%) had missing demographic data. Of the 499 with complete covariate data, 154 (30.9%) completed the post-visit outcome questionnaires. Significant differences were seen between completers and non-completers with respect to age, gender, and site of care; no difference in race/ethnicity was noted between groups (Appendix Table 1). Table 1 presents the demographics of patients in the control arm.

FIGURE 2.

Flow diagram of enrolled patients. For the control group, intention-to-treat and per-protocol analyses included those who returned the post-visit outcome questionnaires. For the GI PROMIS arm, the intention-to-treat analyses included those who completed GI PROMIS and who showed for their visit; missing outcome data was imputed to the corresponding mean value calculated from controls for each item. Per-protocol analyses for the GI PROMIS arm only included those who completed GI PROMIS and the post-visit outcome questionnaires.

TABLE 1.

Patient demographics.

Variable	Control arm (n = 154)	GI PROMIS arm (n = 217) *	p-value
Age (years)	58.7 ± 15.8	54.1 ± 16.3	.007
Male	67 (43.5%)	109 (50.2%)	.20
Race/ethnicity:			.24
Caucasian	100 (64.9%)	151 (69.6%)
African American	32 (20.8%)	28 (12.9%)
Asian	5 (3.3%)	8 (3.7%)
Latino	9 (5.8%)	11 (5.1%)
Other/unknown	8 (5.2%)	19 (8.8%)
Site of care:			.08
West Los Angeles VA GI clinic	55 (35.7%)	77 (35.5%)
Cedars-Sinai GI clinic	13 (8.4%)	11 (5.1%)
University of Michigan GI clinic	77 (50.0%)	100 (46.1%)
University of Michigan scleroderma clinic	9 (5.8%)	29 (13.4%)
Provider level of training:			.29
GI or rheumatology attending	89 (57.8%)	138 (63.6%)
GI fellow	37 (24.0%)	52 (24.0%)
Internal medicine resident or GI PA	28 (18.2%)	27 (12.4%)

Data are presented as mean ± standard deviation or n (%).

Columns may not add up to 100% due to rounding.

GI, gastrointestinal; PA, physician assistant; PROMIS, Patient Reported Outcome Measurement Information System; VA, Veterans Affairs.

^*Four of the 221 individuals who completed PROMIS no-showed for their clinic appointment.

For the intervention group, 594 (0% missing demographic data) were invited to complete GI PROMIS prior to their clinic visit. Among those invited, 221 (37.2%) accessed the e-portal and completed the questionnaires. A majority of the patients who completed PROMIS attended their clinic appointment (217/221; 98.2%). Of the 217 individuals who completed PROMIS and attended their clinic visit, 112 (51.6%) completed the post-visit outcome assessments. Significant differences were seen in age and site of care between those who did and did not return the surveys; no differences were seen in gender and race/ethnicity between groups (Appendix Table 2). In Table 1, we list the demographics of those in the GI PROMIS arm.

Predictors of Completing GI PROMIS

Table 2 shows the results from the multivariable regression on completion of GI PROMIS prior to the clinic visit. Age and gender were not independent predictors of completing PROMIS. African Americans were less likely to access the e-portal vs. whites (odds ratio [OR] 0.44; 95% confidence interval [CI] 0.26, 0.74); no differences were seen between whites and the remaining racial/ethnic groups (Latino, Asian, Other/Unknown). Conversely, patients seen at the University of Michigan GI clinic were more likely to complete PROMIS vs. patients at the WLAVA GI clinic (OR 7.96; 95% CI 4.19, 15.1).

TABLE 2.

Predictors of completing PROMIS.

Variable	Completed GI PROMIS (n = 221)	OR [95% CI] *
Age (years)	-	1.00 [0.98, 1.01]
Gender:
Male	113 (29.1%)	reference
Female	108 (52.7%)	1.25 [0.72, 2.16]
Race/ethnicity:
Caucasian	152 (48.6%)	reference
African American	28 (18.4%)	0.44 [0.26, 0.74]
Asian	9 (60.0%)	2.38 [0.76, 7.43]
Latino	13 (19.7%)	0.52 [0.26, 1.05]
Other/unknown	19 (39.6%)	1.03 [0.52, 2.03]
Site of care:
West Los Angeles VA GI clinic	81 (23.4%)	reference
Cedars-Sinai GI clinic	11 (29.7%)	1.02 [0.44, 2.37]
University of Michigan GI clinic	100 (79.4%)	7.96 [4.19, 15.1]
University of Michigan scleroderma clinic	29 (34.1%)	1.07 [0.53, 2.14]

Data are presented as n (%).

CI, confidence interval; GI, gastrointestinal; OR, odds ratio; PROMIS, Patient Reported Outcome Measurement Information System; VA, Veterans Affairs.

^*The multivariable logistic regression included all variables in the table.

Primary and Secondary Outcomes

Intention-to-Treat Analyses

Table 3 presents the CG-CAHPS provider rating scores for the GI PROMIS and control arms in the intention-to-treat analysis. After adjusting for confounders, we found no difference in provider rating between groups. Tables 4 and 5 list differences in provider interpersonal skills and shared decision-making, respectively. We found no difference in DISQ scores between the PROMIS and control arms. Both groups also had similar shared decision-making scores as by the SDM-Q-9.

TABLE 3.

Patient satisfaction assessment.

CG-CAHPS item	Control arm	GI PROMIS arm	Adjusted p-value
Provider explained things in a way that was easy to understand *	122/139 (87.8%)	91/101 (90.1%)	.75 †
Provider listened carefully *	126/139 (90.7%)	92/101 (91.1%)	.90 †
Provider gave patient easy to understand information about health questions or concerns *, ‡	108/132 (81.8%)	85/97 (87.6%)	.25 †
Provider seemed to know the important information about patient’s medical history *, ‡	95/132 (72.0%)	72/97 (74.2%)	.52 †
Provider showed respect for what patient had to say *, ‡	121/132 (91.7%)	91/97 (93.8%)	.18 †
Provider spent enough time with patient *	121/139 (87.1%)	93/101 (92.1%)	.23 †
Provider rating (0–10 scale) §	8.93 ± 1.65	8.88 ± 1.15	.94 ||

Data are presented as n (%) or mean ± standard deviation.

CG-CAHPS, Consumer Assessment of Healthcare Providers & Systems Clinician & Group Survey; GI, gastrointestinal; PROMIS, Patient Reported Outcome Measurement Information System.

^*Per-protocol analysis (n=240; there was incomplete CG-CAHPS data for 26 patients).

^†The Firth logistic regression model adjusted for patient- (age, gender, race/ethnicity) and provider-level factors (site of care, level of training).

^‡These questions were not required for the eleven patients who stated that they did not talk with their provider about any health questions or concerns.

^§Intention-to-treat analysis (n=345; there was incomplete data for this item for 26 patients).

^||The linear regression model adjusted for patient- (age, gender, race/ethnicity) and provider-level factors (site of care, level of training).

TABLE 4.

Patient assessment of provider interpersonal skills.

DISQ item	Control arm (n = 144)	GI PROMIS arm (n = 209)	Adjusted p-value *
Overall satisfaction with provider	89.7 ± 17.7	88.9 ± 13.4	.67
Warmth of provider’s greeting	90.3 ± 16.4	89.5 ± 12.2	.72
Ability to listen to the patient	90.4 ± 17.2	89.5 ± 12.7	.56
Adequacy of explanations to patient	89.4 ± 16.7	88.9 ± 12.7	.63
Extent of reassurance provided to patient	87.8 ± 18.9	87.6 ± 13.9	.86
Confidence in provider’s ability	90.6 ± 17.5	90.3 ± 13.2	.77
Opportunity for patient to express concerns and fears	89.7 ± 17.7	88.9 ± 13.5	.68
Respect shown to patient	91.3 ± 16.3	92.5 ± 9.9	.33
Time given for visit	88.5 ± 18.9	88.5 ± 13.3	.86
Consideration of patient’s personal situation in treatment or advice	89.7 ± 18.5	88.8 ± 13.4	.66
Concern for patient as a person	89.9 ± 16.9	89.3 ± 12.7	.68
Recommendation of provider to friends	90.3 ± 18.5	89.4 ± 13.4	.67
Average DISQ score	89.8 ± 16.0	89.4 ± 11.7	.79

Data are presented as mean ± standard deviation.

DISQ scores are on a 100-point scale. Complete DISQ data was unavailable for 18 patients.

DISQ, Doctors’ Interpersonal Skills Questionnaire; GI, gastrointestinal; PROMIS, Patient Reported Outcome Measurement Information System.

^*The linear regression model adjusted for patient- (age, gender, race/ethnicity) and provider-level factors (site of care, level of training).

TABLE 5.

Patient assessment of shared decision-making.

SDM-Q-9 item	Control arm (n = 118)	GI PROMIS arm (n = 185)	Adjusted p-value *
Disclosure that a decision needs to be made	81.8 ± 22.5	82.5 ± 13.0	.82
Formulation of equality of partners	79.4 ± 23.7	79.8 ± 14.4	.81
Equipoise statement	77.8 ± 25.6	79.2 ± 13.4	.85
Informing on the options’ benefits and risks	78.7 ± 25.1	78.2 ± 14.7	.41
Investigation of patient’s understanding and expectations	83.2 ± 23.3	82.9 ± 13.2	.75
Identification of preferences	76.0 ± 25.5	76.3 ± 15.0	.98
Negotiation	75.8 ± 25.8	75.7 ± 15.7	.87
Shared decision	76.7 ± 26.3	76.9 ± 15.8	.84
Arrangement of follow-up	81.2 ± 24.6	82.1 ± 12.9	.81
Average SDM-Q-9 score	79.0 ± 22.0	79.3 ± 12.4	.85

Data are presented as mean ± standard deviation.

SDM-Q-9 scores are on a 100-point scale. Complete SDM-Q-9 data was missing for 68 patients.

GI, gastrointestinal; PROMIS, Patient Reported Outcome Measurement Information System; SDM-Q-9, 9-item Shared Decision Making Questionnaire.

^*The linear regression model adjusted for patient- (age, gender, race/ethnicity) and provider-level factors (site of care, level of training).

Per-Protocol Analyses

Because the intention-to-treat analysis biases results towards the null, we also performed a per-protocol analysis. Here, there were 154 individuals in the control group and 112 in the GI PROMIS arm. There were no differences between groups with respect to age, gender, race/ethnicity, site of care, or provider level of training (all p>.05).

The results were similar: there was no difference in CG-CAHPS provider ratings between groups (control 8.93 ± 1.65 vs. GI PROMIS 8.84 ± 1.64; adjusted p=.76). Both groups had similar patient satisfaction scores for the remaining CG-CAHPS items (Table 3). Patient assessment of provider interpersonal skills (Appendix Table 3) and shared decision-making (Appendix Table 4) were also similar between both arms.

DISCUSSION

To our knowledge, this is the first controlled trial evaluating the impact of PROMIS on patient outcomes in clinical practice. Despite theoretical benefits of measuring GI PROs to drive clinical decision-making, we found no differences in patient satisfaction or assessment of provider interpersonal skills and shared decision-making between those in the NIH GI PROMIS and control arms. These results suggest that simply measuring GI PROMIS scores may be insufficient to meaningfully improve patients’ interaction with the healthcare system. These results are also in line with a systematic review that found inconsistent benefits of applying PROs in clinical practice.(2)

There are several possible explanations for our negative results. First, despite the wide use of CG-CAHPS for assessing patient satisfaction, the MCID for the scale is unknown; it is possible that the study was underpowered to detect a significant and meaningful difference between groups. Because of this issue, we calculated the sample size to achieve a moderate effect size of 0.5, which prior research found generally correlates with the MCID.(25, 26) Second, patients reported high levels of satisfaction in both arms of this trial. This result may have led to a “ceiling effect”; it is possible that PROMIS on its own may not offer incremental improvements among patients who are already satisfied with their provider. Third, only 30.9% and 51.6% of patients in the control and intervention arms, respectively, completed the post-visit outcome questionnaires; we cannot know if outcomes would be different in survey non-responders. Fourth, we focused on a proximal outcome of patient satisfaction after a single clinic visit; it is possible that longitudinal use of GI PROMIS (i.e., to track GI symptom improvement and response to therapies) may have led to improved patient satisfaction over time. Lastly, in keeping with the study’s pragmatic approach, we did not mandate or assess the use of PROMIS scores or PROMIS-directed HPI reports by providers. It is possible that some clinicians did not employ the report and managed the patient according to their customary practices. While we could have tested the efficacy of GI PROMIS in a tightly controlled setting by mandating that all providers use PROMIS and incorporate it into their patient assessment, we instead sought to test the effectiveness of PROMIS in a setting that more resembles the “real world.”

Our study also has limitations with respect to external generalizability. We only evaluated patients with GI symptoms, so we cannot know whether using other PROMIS questionnaires, such as those for fatigue, physical function, or pain, among many others, would also fail to show a difference vs. usual care. Moreover, our trial was conducted solely in clinics affiliated with academic universities. It is possible that outcomes may be different when GI PROMIS is employed in non-university based clinics, but that must be formally tested and it is the subject of our future research.

Despite these limitations, we found no differences between groups. Even with post hoc analyses searching for differences on an item-by-item basis, the groups were equivalent. This is consistent with existing literature that administering PROs, although conceptually appealing, often fails to meaningfully improve patient outcomes vs. usual care.(2) Notably, we attempted to overcome this problem by tying GI PROMIS to a full narrative GI HPI, offering the reports on a computer interface viewable in the clinic, making the results available both before and during the clinic to patients and their providers, and visualizing the scores with a heat map that displays percentile scores vs. the general U.S. population. Furthermore, we tested a focused use case where clinical benefit should be evident – measuring GI symptoms in patients presenting with disorders affecting the GI tract. Despite these multiple efforts to bolster the potential of GI PROMIS, and our enthusiasm for PROMIS as consortium investigators, we found no difference between groups.

Moreover, despite offering patients access to the e-portal one week prior to their visit, only one-third of patients completed the PROMIS assessments. A likely contributing factor was the “untethered” nature of the e-portal employed in this study, as it was not integrated into the EHR. However, we approached non-completers in the clinic itself and offered help to complete GI PROMIS on a clinic computer while waiting for the doctor, yet most still were uninterested. Notably, our low uptake is similar to findings from Wagner and colleagues who tested the feasibility of using PROMIS through a “tethered” e-portal among women receiving gynecologic oncology outpatient care.(27) They found that only 37% of PROMIS assessment requests sent via their EHR portal were completed by patients.(27) In addition to system-level issues, patient-level factors may have also contributed to the low intervention uptake. While a systematic review found that patients generally have positive attitudes towards e-portals, issues including security concerns, preconceived beliefs about technology, among others, continue to pose important barriers for widespread e-portal adoption.(28)

In our study, we also noted differential uptake of the GI PROMIS intervention by patient characteristics. Namely, we found that African Americans were 56% less likely to complete PROMIS on the e-portal prior to their visit (adjusted p=.002). There was also a trend towards lower use of PROMIS among Latinos compared to whites, but this difference did not quite reach statistical significance (adjusted p=.07). These findings are consistent with a number of past reports that also found racial/ethnic disparities in e-portal use.(29–31) Efforts to better understand and to address these disparities are critical, as the increasing prevalence of e-portals and other digital health interventions may continue to widen the healthcare gap between whites and minorities.

There were also different rates of GI PROMIS uptake among the four clinical sites. For example, patients at the University of Michigan GI clinic were eight times more likely to use PROMIS than patients at the other sites. The reason behind this is unclear. It is possible that the University of Michigan GI clinic cared for patients that were more “tech-savvy” and willing to use the e-portal. It is also possible that physicians at this clinic were stronger champions of PROMIS, or have a different bond with their patients than those at other clinics; this could not be directly measured. These differences indicate that cultural differences among clinical settings may influence use of PROs such as PROMIS.

While GI PROMIS did not appreciably improve patient-centric outcomes, there are other potential benefits to using PROMIS that were not assessed for as part of this study. For instance, we did not measure provider satisfaction; it is possible that clinicians with access to the GI PROMIS reports were more satisfied with the clinic encounter. Similarly, we did not evaluate clinic visit efficiency. Having the PROMIS scores and PROMIS-directed HPI in hand prior to seeing the patient in the exam room may have allowed clinicians to conduct a more efficient and meaningful clinic visit, and may also have reduced charting and documenting time. These are areas that are the subjects of future research.

Even though our findings are “negative”, they are still relevant for the field of PRO science. The results of this study may inform future research and policy on how best to implement GI PROMIS and other PROs in clinical practice. For example, the passage of the Medicare Access and CHIP Reauthorization Act in 2015 provided the Centers for Medicare & Medicaid an opportunity to update the Medicare EHR Incentive Programs, otherwise known as “Meaningful Use” (MU). One of the goals of the next MU iteration is to reward providers for the outcomes that technology helps them achieve with patients.(32) It remains to be seen how these outcomes will be defined as well as the role of PROs, but it will be important for policy makers to recognize that EHR PRO collection alone may be insufficient to improve patient outcomes.

In summary, this is the first multicenter controlled trial evaluating the impact of PROMIS on patient outcomes in clinical practice. We found that use of NIH GI PROMIS did not improve patient satisfaction or assessment of provider interpersonal skills and shared decision-making. These negative findings may help guide investigators and policy makers in optimizing use of PROs in future clinical practice.

STUDY HIGHLIGHTS.

What is current knowledge?

• The National Institutes of Health (NIH) Patient Reported Outcomes Measurement Information System (PROMIS^®) is a toolbox of highly reliable, precise patient-reported outcome measures that cover the breadth and depth of human health and illness.

• Due to the extraordinary burden of digestive diseases, the NIH PROMIS consortium developed and validated gastrointestinal (GI)-specific PROMIS measures.

• It is unclear whether implementing GI PROMIS, let alone any PROMIS measures, can improve patient outcomes vs. usual care.

What is new here?

• This is the first multicenter controlled trial evaluating the impact of PROMIS on patient outcomes in clinical practice.

• One-time use of GI PROMIS did not improve patient satisfaction or assessment of provider interpersonal skills and shared decision-making.