Abstract
Introduction
PROMIS® Computer Adaptive Testing (CAT) is a testing tool for tracking patient-reported outcome measures (PROM) with a goal to improve individual and population outcomes. Patients’ health literacy (HL) may affect how they are able to complete PROM. We hypothesized that patients with low HL were less likely both to complete surveys and to do so via online patient portal (OPP).
Methods
A retrospective cohort study of 3926 orthopaedic patients in a rural state was performed. Completion rate and completion method both were subjected to log-binomial regressions with patient demographics and HL as predictor variables.
Results
PROM surveys were completed by 2166 (55.2%) of patients, including 512 completed via OPP and 1654 completed via in-clinic tablet (ICT). Compared to high HL patients, low HL patients had a 23% higher non-completion rate, and low HL completers were 63% less likely to use OPP. Age and gender had significant (P < 0.05) associations with completion method, but not completion rate, whereas Area Deprivation Index (ADI) had significant associations with both. Compared to White patients, Black patients had a 25% higher non-completion rate, and Black completers were 49% less likely to use OPP.
Discussion
Our analysis shows that health literacy, demographics, and socioeconomic status affect both whether and how patients fill out PROM surveys. Patients with low HL were less likely to complete PROM surveys, and less likely to use the OPP when they did.
Conclusion
Our results demonstrate that patients’ health literacy, demographics, and socioeconomic status affect both whether they complete their PROMIS® CAT and what method they complete it with. Additional efforts should be made to understand these factors, accommodate patients, and facilitate accurate and complete PROM responses, especially in hospitals that serve diverse and socioeconomically disadvantaged patients.
Keywords: health literacy (HL), patient-reported outcome measure (PROM), computer adaptive testing (CAT), socioeconomic status, area deprivation index (ADI), online-patient portal (OPP), in-clinic tablet (ICT), did not complete (DNC), data collection
Introduction
Patient-reported outcome measures (PROM) are a vital aspect in modern healthcare, as they provide objective measures that provide healthcare teams information to better understand patient’s health status, quality of life, functional status, and comorbid factors. The advent of the Patient-Reported Outcomes Measurement Information System Computer Adaptive Testing (PROMIS® CAT) has enabled greater efficiency in collection of PROM. PROMIS® consists of a series of domain-based surveys that provide a more universal assessment of health outcomes than legacy disease-specific PROM instruments. 1 The CAT format substantially shortens survey length, as the survey adapts based on each subsequent patient answer to avoid unnecessary questions. Uncertainty remains regarding the best way to capture outcomes scores for all patients, especially in the outpatient setting. Current strategies used to obtain PROM include online patient portals (OPPs), where an email/phone number links patients to a web connection for survey completion prior to a clinical visit. Alternatively, the patient can complete the survey at the time of visit using an in-clinic tablet (ICT).2,3 Completion of questionnaires before clinic visits using an OPP is advantageous for clinic flow and patient visit length due to less time being spent completing PROM surveys at the time of visit.
Health literacy (HL) is defined as “the ability to obtain, process, and understand basic health information and services needed to make appropriate health decisions.”4,5 Health literacy and other patient demographics may play a role in the ability of patients to complete forms and utilize electronic methods for data collection. 6 Older patients have trouble with both digital and functional literacy due to age-related cognitive declines, compounding the effect these disparities have on the health literacy of elderly patients. 7 Patients with lower health literacy may not be as able to understand medical language and may have difficulties filling out medical forms.4,5,7 Health literacy is complex, and understanding may fluctuate between healthcare interactions and the delivery of healthcare information.4,5,7,8 Demographic and socioeconomic factors may also play a role in the ability of the patient to fill out electronic forms; patients in rural states may have not have a smartphone or have no internet access, preventing them from filling out forms online.
The aim of our study was to examine factors associated with successful completion of PROM questionnaires by orthopaedic patients. Our two hypotheses were as follows: (1) patients with low HL are less likely to complete PROMIS® CAT forms, and (2) patients with low HL who do complete the PROMIS® forms are less likely to use the OPP and more likely to use an ICT. We also sought to determine which baseline demographic and patient factors contribute to both whether the patient completed the PROM survey and what method they used.
Methods
After obtaining institutional review-board approval, patient data (age, gender, race, address, and health literacy) were obtained through our institution’s central repository. All 4601 hip, knee, and spine patient encounters from our outpatient orthopaedic clinics were collected over a 1-year period (January 1 to December 31, 2020). Our institution, a tertiary medical center in a rural state, first deployed PROMIS® CAT to the spine and total joint clinics, which is why these patient groups were selected for this study. Patients that were given a PROMIS® intake assessment as part of their outpatient orthopedic clinic visit were selected, and all duplicate patient encounters were removed so that each patient was only recorded once and those that were out of state were excluded due to differences in state-level Area Deprivation Index (ADI) score for a total of 3926 total patients included in this study (see Chart 1). If patients were seen on more than one encounter, we included only their first PROMIS® survey-completion method.
Chart 1.
Strobe Flow Chart for Patient Study Inclusion Based on Residency.
As part of standard intake clinic workflow, patients were asked to complete a PROMIS® CAT in two domains: Pain Interference v. 1.1 and Physical Function v. 2.0. All patients with an email address on record were emailed a secure link, via OPP, to complete PROMIS® questions prior to their appointment. This was an optional (but encouraged) survey collected when patients logged in or set up their new appointment, and if they did not complete before their scheduled appointment, they were given a tablet in the clinic and encouraged to complete the survey in-person. Patients were given instructions from the front desk staff on how to complete the PROM surveys as well as information on their clinical purpose. The OPP used in this study was MyChart, part of the Epic electronic health record system (Verona, WI).
Health literacy (HL) was determined using the Single Question Health Literacy Screening Tool question. 9 This questionnaire asks patients: “How confident are you filling out medical forms by yourself?” Responses range from “Extremely” to “Not at all,” with each being assigned a numerical value of 1-5, respectively. Answers from this questionnaire were dichotomized into high-HL (1-2) or low-HL (3-5). 9 This methodology has been validated in previous health literacy studies. 10 The HL screening is required in our institution every 3 years for any patient seen in the inpatient or outpatient setting as a best-practice notification, and only patients who answered this question were included in our analysis.
Using the University of Wisconsin’s Neighborhood Atlas, patient addresses were used to score each patient’s ADI for both the state and national levels. 11 For clarity, a lower ADI score indicates better socioeconomic status. Because a PO box would provide a null value, the patients that had only a post office box or “bad address” listed in the address section in the original patient list were cross-referenced by medical record number (MRN) in the electronic health record database and the address listed on the photo ID provided at the latest encounter was used to search for ADIs. Patients were omitted if ADI scores were unavailable because of no address listed, having no valid address on their photo ID, or being from out-of-state to keep continuity in state ADI levels.
To examine non-completion rate statistically, we divided patients by whether they completed vs Did Not Complete (“DNC”) the PROMIS® CAT. To examine completion method statistically, we took the patients that completed the PROMIS® Assessment Survey and subdivided them by whether they used the OPP or an ICT to complete it. We considered completion of at least one of the two PROMIS® CATs as sufficient for inclusion. All analysis was performed with SAS v9.4 software (SAS Institute Inc., Cary, NC, USA), and all comparisons employed a 2-sided alpha = 0.05 significance level.
Our initial analysis proceeded as follows. First, we summarized the continuous factors (age, national ADI scores, and state ADI scores) by completion status (DNC vs completed) as means and standard deviations (SDs) and compared them for status-group differences with Wilcoxon’s rank-sum test. Then we summarized and compared them the same way by completion method (OPP vs ICT). The categorical factors [factor levels] were Gender [male and female], Race [Black, White, and Other], dichotomized HL [low and high], and Subspecialty [hip/knee and spine]. For each factor, first we summarized the data within factor levels as the number (percentage) DNC and the number (percentage) that completed the survey, and used Pearson’s chi-square test to assess DNC-rate differences between factor levels among all participants. Then we repeated this process to assess OPP-rate differences between factor levels among participants that completed the survey.
After this initial analysis, we conducted multivariable analysis using log-binomial regression to examine whether HL in particular was independently significant for either DNC rate or OPP rate after adjusting for all other factors being studied.12,13 We chose log-binomial regression over the more commonly used logistic regression in order to compare rates of DNC and OPP use directly instead of through odds ratios. In both of our multivariable models, we included national ADI but excluded Arkansas ADI because of the high collinearity between them. To address the question of whether the patterns of multivariable association with either DNC rate or OPP-use rate change between subspecialties, we undertook interaction analysis. Briefly, we augmented the original or “main effects” multivariable model with five additional interaction terms, one for the Subspecialty interaction with each of the five other factors. Then we compared the augmented model to the main-effects model via Likelihood Ratio Test (LRT) for the improvement in how well the augmented model fits the data. If the LRT yielded a statistically significant result, then we used the augmented model to estimate separate rate ratios (95% confidence intervals) for the hip/knee submodel and the spine submodel, and displayed these estimates side-by-side accompanied by interaction P-values to indicate which multivariable rate ratios differed significantly between subspecialties.
Results
Of the 3926 patients surveyed, 512 patients (13.0%) completed the questionnaire via OPP, 1654 patients (42.1%) completed via ICT, and 1760 patients (44.8%) did not complete their questionnaire. More than half (54%) of the 378 patients with low health literacy (HL) failed to complete their questionnaire, compared to 44% of the 3548 patients with high HL (P < 0.001; Table 1). Of the 2166 patients that completed their questionnaire (“the completers”), 9% with low HL (16 of 174) used the OPP, compared to 25% of completers (496 of 1992) with high HL (P < 0.001; Table 1).
Table 1.
Sociodemographic Characteristics of Participants Versus (1) Whether They Completed and (2) How They Completed Their PROMIS Assessment Survey.
| Participant characteristic | Whole study (N = 3926) | DNC (N = 1760) | Completed: (N = 2166) | P † | Used OPP: (N = 512) | Used ICT: (N = 1654) | P † |
|---|---|---|---|---|---|---|---|
| Age at encounter | Mean (SD) | Mean (SD) | Mean (SD) | 0.228‡ | Mean (SD) | Mean (SD) | 0.024‡ |
| Years | 61.8 (14.2) | 62.0 (14.6) | 61.7 (13.9) | 60.4 (13.7) | 62.1 (13.9) | ||
| Gender | Group total | N (%) a | N (%) a | 0.210 | N (%) b | N (%) b | 0.033 |
| Male | 1619 | 745 (46.0%) | 874 (54.0%) | 186 (21.3%) | 688 (78.7%) | ||
| Female | 2307 | 1015 (44.0%) | 1292 (56.0%) | 326 (25.2%) | 966 (74.8%) | ||
| Race | Group total | N (%) a | N (%) a | <.001 | N (%) b | N (%) b | <.001 |
| Black | 808 | 432 (53.5%) | 376 (46.5%) | 50 (13.3%) | 326 (86.7%) | ||
| Other | 117 | 50 (42.7%) | 67 (57.3%) | 13 (19.4%) | 54 (80.6%) | ||
| White | 3001 | 1278 (42.6%) | 1723 (57.4%) | 449 (26.1%) | 1274 (73.9%) | ||
| Health literacy | Group total | N (%) a | N (%) a | <.001 | N (%) b | N (%) b | <.001 |
| Low | 378 | 204 (54.0%) | 174 (46.0%) | 16 (9.2%) | 158 (90.8%) | ||
| High | 3548 | 1556 (43.9%) | 1992 (56.1%) | 496 (24.9%) | 1496 (75.1%) | ||
| National ADI | Mean (SD) | Mean (SD) | Mean (SD) | <.001 ‡ | Mean (SD) | Mean (SD) | <.001 ‡ |
| (Scaled 1-100) | 66.7 (22.9) | 69.0 (22.2) | 64.9 (23.4) | 58.7 (23.7) | 66.8 (22.9) | ||
| State ADI | Mean (SD) | Mean (SD) | Mean (SD) | <.001 ‡ | Mean (SD) | Mean (SD) | <.001 ‡ |
| (Scaled 1-10) | 4.48 (2.90) | 4.74 (2.90) | 4.28 (2.89) | 3.54 (2.78) | 4.50 (2.89) | ||
| Subspecialty | Group total | N (%) a | N (%) a | <.001 | N (%) b | N (%) b | <.001 |
| Spine | 940 | 555 (59.0%) | 385 (41.0%) | 144 (37.4%) | 241 (62.6%) | ||
| Hip/knee | 2986 | 1205 (40.4%) | 1781 (59.6%) | 368 (20.7%) | 1413 (79.3%) |
DNC, did not complete; OPP, online patient portal; ICT, in-clinic tablet; ADI, area deprivation index. Higher ADIs are associated with lower socioeconomic status.
†P-values are from either Pearson’s chi-square test or ‡Wilcoxon’s rank-sum test; P-values in bold are significant at P < 0.05.
aPercentage of the Group Total.
bPercentage of the patients that completed the PROMIS survey.
With respect to the other sociodemographic factors (Table 1), average ages were only 0.3 years different between those who completed the questionnaire and those who did not (P = 0.228), but within the completers, the average age was 1.7 years lower among those who used the OPP (P = 0.024). Similarly, the survey completion rate in all patients was only 2% different between males and females (P = 0.210), but among those who completed it, the rate of OPP use was almost 4% lower in males (P = 0.033). In terms of race, fewer than half of Black patients (46.5%) completed the assessment compared to 57.4% of White patients and 57.3% of Other races (P < 0.001). Among the completers, the rates of OPP use were 13.3% for Black patients, 26.1% for White patients, and 19.4% for members of Other races (P < 0.001). Relative to patients who completed the questionnaire, participants who did not had higher average ADI scores, i.e., more than 4 percentiles higher on National ADI and almost half a decile higher on Arkansas ADI (P < 0.001 for both). Among the patients who completed it, those who did so with OPP scored an average of 8.1 percentiles lower on National ADI and 0.96 deciles lower on Arkansas ADI than those who used ICT (P < 0.001 for both). Finally, Table 1 shows that the survey non-completion rate was 1.46-fold higher among spine patients (59.0%) than among hip/knee patients (40.4%) (P < 0.001), but also shows a 1.81-fold higher rate of OPP use among survey completers that were spine (37.4%) vs hip/knee (20.7%) (P < 0.001).
Table 2 shows side-by-side the results of univariable and multivariable log-binomial regressions of DNC rate on sociodemographic factors. Compared to patients with high HL, those with low HL had a 1.23-fold higher DNC rate (P < 0.001) in univariable analysis, but this increase in DNC rate was sharply attenuated to 1.11-fold (P = 0.042) after multivariable adjustment (Table 2). The effect of Age on DNC rate showed the opposite behavior. A 10-year increase in age produced only a 1% increase in DNC rate (rate ratio = 1.010; P = 0.427) in univariable analysis that turned into a 5% increase (rate ratio = 1.049; P < 0.001) after multivariable adjustment (Table 2). With respect to National ADI, the four-percentile elevation in Table 1’s DNC group became in Table 2 a 4.5% increase in DNC rate with each 10-percentile increase (univariable rate ratio = 1.045; P < 0.001), most of which persisted after covariate adjustment (multivariable rate ratio = 1.035; P < 0.001). Finally, the DNC-rate ratios for Gender, Race, and Subspecialty show only small differences between their univariable and multivariable values (Table 2).
Table 2.
Log-Binomial Regressions of Rate of DNC (Did Not Complete) on Sociodemographic Factors (N = 3296 Participants).
| Outcome: DNC rate | Univariable regression results§ | Multivariable regression result§ | ||||
|---|---|---|---|---|---|---|
| Sociodemographic factor | Rate ratio | 95% CI† (lower, upper) | P ‡ | Rate ratio | 95% CI† (lower, upper) | P ‡ |
| Age | 0.427 | |||||
| 10-year increase | 1.010 | (0.985, 1.036) | 1.049 | (1.023, 1.075) | <.001 | |
| Gender | 0.211 | 0.400 | ||||
| Male | 1.046 | (0.975, 1.122) | 1.029 | (0.963, 1.100) | ||
| Female (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
| Race | <.001 | <.001 | ||||
| Black | 1.256 | (1.163, 1.355) | 1.210 | (1.121, 1.306) | ||
| Other | 1.004 | (0.810, 1.243) | 1.030 | (0.836, 1.270) | ||
| White (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
| Health literacy | <.001 | 0.042 | ||||
| Low | 1.231 | (1.113, 1.360) | 1.108 | (1.008, 1.218) | ||
| High (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
| National ADI | <.001 | <.001 | ||||
| 10-point increase | 1.045 | (1.029, 1.062) | 1.035 | (1.019, 1.052) | ||
| Subspecialty | <.001 | <.001 | ||||
| Spine | 1.463 | (1.366, 1.567) | 1.488 | (1.391, 1.592) | ||
| Hip/knee (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
DNC, did not complete. ADI, area deprivation index. Rate Ratio, the ratio of DNC rates.
§Regression results are from Univariable and Multivariable log-binomial regressions of DNC rate on the sociodemographic factors. The Univariable Rate Ratios for categorical factors are the ratios of the corresponding DNC rates from Table 1.
†95% Confidence Interval.
‡P-value.
P-values in bold are significant at P < 0.05.
However, when we performed interaction analysis on the multivariable DNC-rate model of Table 2, we uncovered significant evidence (LRT chi-square = 14.40, DF = 6; P = 0.025) for a difference between subspecialties in how the sociodemographic factors affect DNC rate. To examine this difference further, we estimated rate ratios for all factors within each subspecialty; Table 3 shows the resulting two multivariable submodels. Race displayed the most significant change between subspecialties in how it affected DNC rate (interaction P = 0.011). With White patients as a reference, Black hip/knee patients had a 33% increase in DNC rate while Other hip/knee patients had a 12% decrease (Race P < 0.001), whereas Black spine patients had only a 7% increase in DNC rate while Other spine patients had a 15% increase (Race P = 0.375) (Table 3). Age also displayed a significant change between subspecialties (interaction P = 0.035). A 10-year increase in age raised the DNC rate by almost 8% in hip/knee patients (rate ratio = 1.079; P < 0.001), but by only 2% in spine patients (rate ratio = 1.023; P = 0.212) (Table 3). None of the other sociodemographic factors exhibited a significant change between subspecialties in how they affected DNC rate.
Table 3.
Multivariable Log-Binomial Regression “Submodels” by Subspecialty of DNC Rates Versus Sociodemographic Factors, With Tests for Factor Interactions With Subspecialty.
| Outcome: DNC rate | Interaction P⁑ | Multivariable submodel estimates§ for Subspecialty = hip/knee | Multivariable submodel estimates§ for subspecialty = spine | ||||
|---|---|---|---|---|---|---|---|
| Sociodemographic factor | Rate ratio | 95% CI† (lower, upper) | P ‡ | Rate ratio | 95% CI† (lower, upper) | P ‡ | |
| Age | 0.035 | <.001 | 0.212 | ||||
| 10-year increase | 1.079 | (1.042, 1.118) | 1.023 | (0.987, 1.059) | |||
| Gender | 0.529 | 0.689 | 0.249 | ||||
| Male | 1.018 | (0.933, 1.111) | 1.064 | (0.958, 1.182) | |||
| Female (ref) | 1.000 | (—, —) | 1.000 | (—, —) | |||
| Race | 0.011 | <.001 | 0.375 | ||||
| Black | 1.325 | (1.202, 1.462) | 1.074 | (0.942, 1.223) | |||
| Other | 0.880 | (0.635, 1.219) | 1.149 | (0.901, 1.465) | |||
| White (ref) | 1.000 | (—, —) | 1.000 | (—, —) | |||
| Health literacy | 0.732 | 0.068 | 0.293 | ||||
| Low | 1.130 | (0.996, 1.283) | 1.091 | (0.934, 1.275) | |||
| High (ref) | 1.000 | (—, —) | 1.000 | (—, —) | |||
| National ADI | 0.543 | 0.003 | 0.003 | ||||
| 10-point increase | 1.031 | (1.010, 1.052) | 1.042 | (1.014, 1.070) | |||
Likelihood Ratio Test comparing this model that contains subspecialty interactions to the multivariable main-effects model of Table 2 for the improvement in how well it fits the data: Chi-square = 14.40, DF = 6; P = 0.025.
DNC, did not complete. ADI, area deprivation index. Rate Ratio, the ratio of DNC rates.
§Submodel estimates are from the Multivariable log-binomial regression of DNC rate on sociodemographic factors when the regression model includes the interactions of Subspecialty with the five other factors. The Rate Ratio, †95% Confidence Interval, and ‡P-value describes the sociodemographic factor’s relationship to the DNC rate within each subspecialty, whereas the ⁑Interaction P-value indicates whether that sociodemographic factor’s relationship changes significantly between subspecialties. P-values in bold are significant at P < 0.05.
On the 2166 participants that completed the PROMIS® survey successfully, we performed both univariable and multivariable log-binomial regressions of OPP-use rate on sociodemographic factors; Table 4 shows the results side-by-side. For gender, race, health literacy, and subspecialty, we note that the univariable rate ratios from Table 4 successfully reproduce the ratios of corresponding OPP-use rates from Table 1. The two major takeaways from Table 4 are that (1) all six sociodemographic factors examined had statistically significant associations with rate of OPP use, and (2) the covariate-adjusted rate ratios from multivariable analysis showed only small differences from their unadjusted values. Compared to survey completers with high HL, those with low HL had a large (63%) observed reduction in OPP-use rate (ratio = 0.369; P < 0.001) that remained large (55%) after covariate adjustment (ratio = 0.448; P < 0.001) (Table 4). Spine survey completers enjoyed a large increase in the rate of OPP use compared to hip/knee completers, with respective relative rates of 1.81 and 1.72 before and after covariate adjustment (P < 0.001 for both; Table 4). Compared to White survey completers, Black and Other completers respectively used the OPP 49% and 25% less often before adjustment, and 40% and 30% less often after adjustment (Race P < 0.001 for both; Table 4). A 10-percentile increase in National ADI led to similar-sized decreases in OPP-use rate of 9.9% before covariate adjustment and 8.4% after adjustment (Table 4). Likewise, a 10-year increase in the age of survey completers led to similar-sized decreases in OPP-use rate of 5.9% (P = 0.023) before covariate adjustment and 5.6% (P = 0.027) after adjustment (Table 4). And male survey completers were roughly 16% less likely to use the OPP than female survey completers, both before covariate adjustment (ratio = 0.843; P = 0.033) and after covariate adjustment (ratio = 0.836; P = 0.019). Finally, when we performed interaction analysis on the multivariable OPP-use model, we found scant evidence for interactions with subspecialty (LRT chi-square = 3.18, DF = 6; P = 0.7853 for all five interactions collectively, with interaction P-values ranging from 0.144 to 0.937 for each interaction individually).
Table 4.
Log-Binomial Regressions of Rate of OPP (Online Patient Portal) Use Versus Sociodemographic Factors Among the 2166 Participants That Completed the PROMIS Assessment Survey.
| Outcome: OPP-use rate | Univariable regression results§ | Multivariable regression result§ | ||||
|---|---|---|---|---|---|---|
| Sociodemographic factor | Rate ratio | 95% CI† (lower, upper) | P ‡ | Rate ratio | 95% CI† (lower, upper) | P ‡ |
| Age | 0.023 | 0.027 | ||||
| 10-year increase | 0.941 | (0.893, 0.991) | 0.944 | (0.899, 0.992) | ||
| Gender | 0.033 | 0.019 | ||||
| Male | 0.843 | (0.720, 0.988) | 0.836 | (0.719, 0.973) | ||
| Female (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
| Race | <.001 | <.001 | ||||
| Black | 0.510 | (0.390, 0.669) | 0.595 | (0.453, 0.781) | ||
| Other | 0.745 | (0.454, 1.221) | 0.699 | (0.432, 1.130) | ||
| White (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
| Health literacy | <.001 | <.001 | ||||
| Low | 0.369 | (0.230, 0.593) | 0.448 | (0.280, 0.719) | ||
| High (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
| National ADI | <.001 | <.001 | ||||
| 10-point increase | 0.901 | (0.876, 0.928) | 0.916 | (0.891, 0.943) | ||
| Subspecialty | <.001 | <.001 | ||||
| Spine | 1.810 | (1.546, 2.120) | 1.719 | (1.472, 2.008) | ||
| Hip/knee (reference) | 1.000 | (—, —) | 1.000 | (—, —) | ||
OPP, online patient portal. ADI, area deprivation index. Rate Ratio, the ratio of OPP-use rates.
§Regression results are from Univariable and Multivariable log-binomial regressions of OPP-use rate on the sociodemographic factors. The Univariable Rate Ratios for categorical factors are the ratios of the corresponding OPP-use rates from Table 1. †95% Confidence Interval. ‡P-value. P-values in bold are significant at P < 0.05.
Discussion
Our study found that health literacy may be an important factor in determining if and how patients choose to complete PROM. Patients with low health literacy were 1.23-fold less likely to complete the PROM survey in any manner, and when they did complete it, they were 2.71-fold less likely to use the OPP. Previous studies have shown that patients with poor health literacy have worse patient reported outcomes, worse health status, and lower quality of life. 14
The impact of health literacy on successful completion of electronic PROM instruments has not been well reported. With the advent of the PROMIS® CAT, a digital device is required to enter questionnaire responses, which may bias results when tracking surgical outcomes. Providers taking care of patients in rural states with lower socioeconomic status and higher ADI may find it more difficult to collect results. As insurers and Medicare shift to reimbursement models based on PROM collection rates, this may influence the ability of physicians to demonstrate value in less affluent clinical practices. While electronic patient portals can foster greater patient satisfaction and better communication for patients outside of the ambulatory visit, a greater reliance on digital tools has the potential to increase discrepancies in engagement with those that have limited access or understanding, thereby exacerbating existing health disparities. 15 Recognizing that those who did not fill out their PROM at all, and those who did not fill it out via OPP, as both being more likely to have poorer health literacy could allow for accommodations in order to hopefully achieve better physician-patient communication and improved health outcomes. The literature regarding disparities in digital health is new and evolving, but a set of practical action steps has been proposed by Smith and Magnani. They advocate intuitive design of digital health interfaces, emphasizing ease of use that incorporates patient feedback on usability. Clinic staff should make a specific effort to ask patients regarding their preferred method of electronic communication.
The number of patients who did not complete PROMIS® CAT surveys was notable. Our findings that this group of patients had lower HL, worse ADI, and were more likely to be Black suggest that, even with our efforts to provide multiple methods for PROM completion, we were still missing these crucial data on a large subset of at-risk patients. Each institution likely will face unique challenges to achieving 100% PROM completion, but we do think our findings are likely common to many hospitals. Further educating front-facing clinic staff to help patients understand the reasons for PROM completion and helping them navigate the ICT interface are strategies to mitigate lack of response.
Effective provider-patient communication is essential for patients to share information more openly with their healthcare providers, leading to more proper diagnoses, better adherence to treatment plans, and higher patient satisfaction. 16 Early recognition of patients with potentially lower health literacy may facilitate better communication and use of clinical time during patient encounters. Strategies to achieve this goal include having prepared printed materials, scheduling slightly longer appointments, and allocating time for the patient-provider teach-back method, which involves asking patients to repeat key information in their own words to help patients retain what they have learned during their visit.4,17 Identifying patients who haven’t completed their PROM survey via OPP, i.e., completed it before their visit, and recognizing the potential link of not completing it before their visit to lower health literacy as well as lack of access to broad-band internet service, may allow adequate preparation ahead of scheduled appointments to allocate resources to provide more effective communication as well as provide a functioning device (the ICT) that they can complete their PROM survey. Further investigation is needed to improve PROM survey collection in low health literacy groups. This may include better patient portals with easier access and further education as to the importance of patient portal use.
A previous study by Long et al shows there are multiple distinct patient-level factors that affect PROM completion. Failure to consider these factors in PROM design and implementation may lower completion rates or prevent accurate completion, undermining PROM validity. In our population, unappreciated factors such as patient vision, print literacy, and aptitude with digital platforms likely contributed to one’s ability to answer PROM questions and follow written instructions. Given our similar patient demographics, these factors affect minority populations at disproportionate rates, which may contribute to widening existing health disparities. 18
Bernstein et al showed that the rate of completion of PROM varies according to sociodemographic variables. This variability could bias clinical outcomes research in orthopaedic surgery. 19 Our results also show that socioeconomic factors affect PROM collection methods. Average ADI was shown to be approximately a full decile lower for those who submitted via OPP than those who submitted via ICT for both the state and national level (Table 1). This could be the result of confounding with variables tested as well as those not tested, such as computer and internet access. It has been reported that 15-24% of Americans lack broadband and internet connectivity. Up to 40% of low-income households lack an at-home internet subscription, requiring them to use cellular data plans (that may be limited) or public Wi-Fi to access the internet, thereby limiting their ability to use online health technology.15,20 We also noted that Black and Other non-White patients completed PROM via OPP less frequently than Caucasian patients. Previous literature has shown that access to internet is a social determinant of health and that minorities are less likely to have broadband internet accesses, in both urban and rural settings. 20 Lower socioeconomic status and higher ADI have both been shown to be predictive of hospital readmission rates, and the association of impaired health literacy in these statistics should not be discounted.21,22
Our analysis also found that, among those who completed the PROM assessment, each 10-year increase in age led to a 6% decrease in the probability of using the OPP. This could be explained by age-related changes in cognitive function that lower the reading ability of older age groups, and health literacy can be markedly lower in older patients, thereby leading to worse functional health. 7 Digital literacy has been shown to be a predictor of health as healthcare becomes more digitally based; thus the essential skills that a patient must have to benefit from information technology multiply and become more complex.15,23 Recognizing that elderly patients can have greater difficulties with functional literacy, digital literacy, and health literacy can allow for implementing measures to help them make more informed decisions about their health.
We found that spine patients completed PROM via the OPP almost 50% more frequently than hip and knee patients. We also noticed that spine patients tended to be almost 7 years younger and with a slightly higher composition of White patients than hip/knee patients, which may help explain why the effects of age and race on non-completion rate were stronger in the hip/knee subgroup. This is reminiscent of what was found in a Danish study investigating the characteristic differences in hip, knee, and spine pain patients. 24 Their study also found that more back-pain patients had a <3-month time between onset of pain and seeking treatment than knee/hip patients (19% vs 5-7%, respectively). 24 Moreover, spine patients in their study had a higher level of education. 24 Wright et al identified that social deprivation does not affect PROMIS® scores in patient populations across orthopaedic subspecialties equally. Notably, spine patients did not show significant differences in their PROs based on social deprivation, while total joints patients did. Understanding that different pathology subgroups of orthopaedic patients may have different health literacy and digital literacy capacities merits consideration in patient care, as well as further research investigation into why.
Our study has several limitations. We collected data during part of the COVID-19 pandemic. This could have created a situation in which prospective patients may have canceled appointments, been more motivated to use the OPP over the ICT, or even refused the survey all together due to hygiene concerns, possibly skewing results. Patients had to “op in” to set up a portal either on a smartphone or computer, and provided an email address to be contacted via the OPP. We do not have the information that would allow us to know if all patients set up this feature. Our study did not account for the lack of broadband or email access that plagues rural southern states. 25 The “2020 Broadband Deployment Report” produced by the Federal Communications Commission found that 22% of the rural population does not have access to viable broadband internet service, regardless of ability to pay for service. 26 Therefore even if email correspondences were attempted to be sent to all patients, it is likely that not all patients received such messages and thus were unable to set up online portal access. Inability to access the internet is a major determinate in a patient’s ability to use OPPs, and racial minorities and elderly patients are more likely to live in areas that have poor broadband internet access, and therefore be less likely to access an OPP. 24 Sisodia et al showed that during the COVID-19 pandemic, profound racial disparities occurred due to relying on portal use for PROM completion. 27
Further possible limitations to the study include patients with marginal reading ability that may not acknowledge they need help with reading comprehension. Patients may be in denial or ashamed of their need for help with their literacy problem, or patients just may not understand the question being asked.4,28 This may influence their response to the validated health literacy screening tool we utilized. Finally, the rate of PROM non-response was quite high; we attribute this to reluctance of patients to undertake a new aspect of their clinic experience (including touching a tablet in clinic during the pandemic), however, this provided an opportunity to evaluate demographics of “non-responses” in a large, real-world study population.
Conclusion
Our results demonstrate that patients’ health literacy, demographics, and socioeconomic status affect both whether they complete their PROMIS® questionnaire and what method they use when they do. Additional efforts should be made to understand these factors, accommodate patients, and facilitate accurate and complete PROM responses; this is especially true in hospitals serving diverse and socioeconomically disadvantaged patients.
Acknowledgements
Statistical analysis was performed by Mr Eric Siegel (UAMS statistician).
Authors’ Note: Institutional IRB #262252 at The Univeristy of Arkasnas for Medical Sciences; accepted 6/25/21.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project described was supported by the Translational Research Institute (TRI), grant UL1 TR003107 through the National Center for Advancing Translational Sciences of the National Institutes of Health (NIH). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. In addition, the Bill and Betty Petty Research Fund at the University of Arkansas for Medical Sciences supported this work.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
IRB Approval: UAMS approval # 262252.
ORCID iDs
Simon C. Mears https://orcid.org/0000-0002-0068-0283
Eric R. Siegel https://orcid.org/0000-0001-9824-6612
Jeffrey B. Stambough https://orcid.org/0000-0003-0950-5227
Statements and Declarations
References
- 1.Cheung EC, Moore LK, Flores SE, Lansdown DA, Feeley BT, Zhang AL. Correlation of PROMIS with orthopaedic patient-reported outcome measures. JBJS Rev. 2019;7(8):e9. [DOI] [PubMed] [Google Scholar]
- 2.Lizzio VA, Dekhne MS, Makhni EC. Electronic patient-reported outcome collection systems in orthopaedic clinical practice. JBJS Rev. 2019;7(7):e2. [DOI] [PubMed] [Google Scholar]
- 3.Makhni EC, Bozic KJ. Team approach: clinical outcome collection, done practically. JBJS Rev. 2018;6(9):e5. [DOI] [PubMed] [Google Scholar]
- 4.Weiss BD. Health Literacy: A Manual for Clinicians. Chicago, IL: American Medical Association, American Medical Foundation; 2003:52. [Google Scholar]
- 5.Koh HK, Berwick DM, Clancy CM, et al. New federal policy initiatives to boost health literacy can help the nation move beyond the cycle of costly “crisis care”. Health Aff. 2012;31(2):434-443. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Perez JL, Mosher ZA, Watson SL, et al. Readability of orthopaedic patient-reported outcome measures: is there a fundamental failure to communicate? Clin Orthop Relat Res. 2017;475(8):1936-1947. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Baker DW, Gazmararian JA, Sudano J, Patterson M. The association between age and health literacy among elderly persons. J Gerontol. 2000;55(6):S368-S374. https://academic.oup.com/psychsocgerontology/article/55/6/S368/623814 [DOI] [PubMed] [Google Scholar]
- 8.Nutbeam D. The evolving concept of health literacy. Soc Sci Med. 2008;67(12):2072-2078. [DOI] [PubMed] [Google Scholar]
- 9.Chew LD, Griffin JM, Partin MR, et al. Validation of screening questions for limited health literacy in a large VA outpatient population. J Gen Intern Med. 2008;23(5):561-566. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hadden K, Martin R, Prince L, Barnes CL. Patient health literacy and diabetic foot amputations. J Foot Ankle Surg. 2019;58(5):877-879. [DOI] [PubMed] [Google Scholar]
- 11.Kind AJH, Buckingham WR. Making neighborhood-disadvantage metrics accessible — the neighborhood Atlas. N Engl J Med. 2018;378(26):2456-2458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wacholder S. Binomial regression in GLIM: estimating risk ratios and risk differences. Am J Epidemiol. 1986;123(1):174-184. [DOI] [PubMed] [Google Scholar]
- 13.Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol. 2005;162(3):199-200. [DOI] [PubMed] [Google Scholar]
- 14.Katz P, Dall'Era M, Trupin L, et al. Impact of limited health literacy on patient-reported outcomes in systemic lupus erythematosus. Arthritis Care Res. 2021;73(1):110-119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Sieck CJ, Sheon A, Ancker JS, Castek J, Callahan B, Siefer A. Digital inclusion as a social determinant of health. NPJ Digit Med. 2021;4(1):52. doi: 10.1038/s41746-021-00413-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ha JF, Longnecker N. Doctor-patient communication: a review. Ochsner J. Spring; 2010:10(1), 38-43. [PMC free article] [PubMed] [Google Scholar]
- 17.BodenheimerTeach-Back T. A Simple Technique to Enhance Patients’ Understanding. Fam Pract Manag. 2018. Jul/Aug;25(4):20-22. [PubMed] [Google Scholar]
- 18.Long C, Beres LK, Wu AW, Giladi AM. Patient-level barriers and facilitators to completion of patient-reported outcomes measures. Qual Life Res. 2022;31(6):1711-1718. [DOI] [PubMed] [Google Scholar]
- 19.Bernstein DN, Karhade AV, Bono CM, Schwab JH, Harris MB, Tobert DG. Sociodemographic factors are associated with patient-reported outcome measure completion in orthopaedic surgery: an analysis of completion rates and determinants among new patients. JB JS Open Access. 2022;7(3):e2200026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Perzynski AT, Roach MJ, Shick S, et al. Patient portals and broadband internet inequality. J Am Med Inf Assoc. 2017;24(5):927-932. doi: 10.1093/jamia/ocx020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kind AJ, Jencks S, Brock J, et al. Neighborhood socioeconomic disadvantage and 30-day rehospitalization: a retrospective cohort study. Ann Intern Med. 2014;161(11):765-774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Hu J, Kind AJH, Nerenz D. Area deprivation Index predicts readmission risk at an urban teaching hospital. Am J Med Qual. 2018;33(5):493-501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.van Kessel R, Wong BLH, Clemens T, Brand H. Digital health literacy as a super determinant of health: more than simply the sum of its parts. Internet Interv. 2022;27:100500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Shah N, Kramer J, Borrelli B, et al. Patient characteristics in knee, hip and back pain patients participating in patient education and exercise therapy. a comparison of 46,273 patients from the glad registries in Denmark. 2020. doi: 10.1016/j.joca.2020.02.276 [DOI] [Google Scholar]
- 25.Stucka M, King L. In Arkansas, Many Still Lack Broadband Access. Brunswick: Times Record. https://www.swtimes.com/story/news/local/2021/07/08/arkansas-broadband-expansion-lacking/117402376/. Accessed December 30, 2021. [Google Scholar]
- 26.35 FCC rcd 8986 (11), Apr 24, 2020.
- 27.Sisodia RC, Rodriguez JA, Sequist TD. Digital disparities: lessons learned from a patient reported outcomes program during the COVID-19 pandemic. J Am Med Inf Assoc. 2021;28(10):2265-2268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Morris NS, MacLean CD, Chew LD, Littenberg B. The single item literacy screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract. 2006;7:21. [DOI] [PMC free article] [PubMed] [Google Scholar]