Response to survey directed to patient portal members differs by age, race, and healthcare utilization

Abstract

Health care systems are increasingly utilizing electronic medical record—associated patient portals to facilitate communication with patients and between providers and their patients. These patient portals are growing in recognition as potentially valuable research tools. While there is much information about the response rates and demographics of internet-based surveys as well as the demographics of patients who are portal members, not much is known about the response rate of internet-based surveys directed to a group of patient portal members or the demographics of which portal members respond to internet-based surveys issued within that specific population. The objective of these analyses was to determine the demographics of patient portal users who respond to an internet-based survey request. We hypothesized that respondents would more likely be: (1) older (65+), (2) European American, (3) married, (4) female, (5) college educated, (6) have higher medical care utilization, (7) have more comorbidities, and (8) have a private practice primary care physician (as opposed to a salaried group practice primary care physician). We found that our respondents tended to be older, of European geographic ancestry, and more frequent users of healthcare. While patient portal members are an easily identifiable and contactable group that are potentially valuable participants for research, it is important to understand that respondents to surveys solicited from this sampling frame may not be entirely representative. It will be important to develop strategies to more fully engage populations that represent the target population in order to increase overall and subgroup response rates.

INTRODUCTION

Health care systems are increasingly utilizing electronic medical record—associated patient portals to facilitate communication with patients and between providers and their patients. These patient portals are growing in recognition as potentially valuable research tools. While there is much known about the response rates and demographics of respondents involved in internet-based surveys as well as the demographics of patients who are portal members, not much is known about the response rate of internet-based surveys directed to a group of patient portal members or the demographics of which portal members respond to internet-based surveys issued within that specific population.

Considering research into the possible advantages or disadvantages of internet-based surveys compared to traditional in person, telephone or self-administered paper-based methods, the results are mixed. While some studies have shown response rates of internet-based surveys to be similar or superior to those of paper-based surveys,^1,2 several have shown the former to have a lower response rate.^3–6 However, internet-based surveys may have advantages such as quicker responses and longer answers to open-ended questions.⁵ Additionally, it has been shown that although email addresses are often more readily available for younger patients and females,² the factors affecting response to internet-based surveys are similar to those for other survey modes.⁶ For example, Simone et al showed that respondents to their internet survey posted on oncolink.org regarding pain intervention in radiation oncology patients were predominantly white, female, and well-educated.⁷ It is not known whether patient portal populations have different survey response patterns with respect to demographics.

Although the demographics of patient portal members are similar to that of those who typically respond to surveys (older, white, and married),^8–13 it is unknown if respondents to surveys are representative of the entire portal population. The objective of this study was to determine the demographics of patient portal users who respond to an internet-based survey request. Based on previous studies that have found demographic differences in these categories,^6,7 we hypothesized that respondents would more likely be: (1) older (65+ years), (2) European American, (3) married, (4) female, (5) college educated, (6) have higher medical care utilization, and (7) have more comorbidities. Finally, our healthcare system was particularly interested in if respondents would be different from nonrespondents in having a private practice primary care physician (as opposed to a salaried group practice primary care physician).

METHODS

Survey

A one-time email with a link to a REDCap survey was sent to 10 015 Henry Ford Health System patient portal (EPIC MyChart) users. These patients were randomly selected from 138 197 individuals who had active accounts and had logged in at least once in the previous calendar year. They were randomly selected from within strata defined by physician practice type, race, sex, and marital status. This survey included questions regarding preferences related to the timing of release of both routine and potentially sensitive (eg, biopsy, genetic, or sexually transmitted disease) test results. Two weeks were allowed for survey completion and no reminders were sent. Prior to initiation of this survey, we planned to evaluate the demographics of survey responders versus nonresponders independent of investigating the answers to the survey questions themselves. The responses to the survey questions are the subject of a separate study and will be published independently. This study was approved for waived consent by the Henry Ford Health System Institutional Review Board.

Patient characteristics

All patient characteristics analyzed in this report were captured from the electronic medical records. Medical record numbers of those who were sent the survey were used to link to the appropriate electronic medical records. Respondent age was determined by date of birth and categorized into three groups: less than 40, between 40 and 65, and over 65 years. Race was categorized as white, black, and other. Sex was categorized as female or male. Marital status was categorized as married or other. The Charlson Index, a measure of the number of comorbidities present, which ranges from 0 to 13, was ascertained to assess comorbidity.¹⁴ As described above, co-morbidities were obtained from the patients’ electronic medical records and included diagnoses from both inpatient and outpatient encounters. The majority of respondents had a score lower than 2 so this characteristic was categorized as 0, 1, or greater than 1. Healthcare utilization was assessed both by the average yearly number of primary care visits in the preceding 2 years as well as for the average yearly number of specialty care visits. These characteristics were categorized as one or fewer visits and more than one visit. Insurance was categorized into four groups: commercial, Medicare, Medicaid, and other. The category of primary care physician status was divided into the three types of physicians present within Henry Ford Health System: those associated with the salaried medical group (HFMG), those associated with suburban satellite hospitals (Employed), and those associated with the physician network (HFPN, private practice). Patient addresses at the time of the survey obtained from the patients’ electronic medical records were mapped to the 2010 US Census Bureau census tract using a commercial package assembled by Mapping Solutions, LLC Lansing, MI. (Patients are asked to update their address at every patient encounter.) The geocoded addresses with census tract were then used in combination with the American Community Survey (ACS, 2008–2012) to assign socioeconomic variables (median household income, educational attainment). Education was defined as the percentage of the respondents’ census block with a high school diploma or higher and was categorized as less than 80%, 80–89%, and greater than 90%. Socioeconomic status was defined as census block median income and was categorized as less than $35k/year, $35–55k/year, and greater than $55k/year.

Statistical analyses

Odds ratios and 95% confidence intervals were estimated using univariate logistic models fitted to quantify associations between various patient characteristics and whether the patient responded to the survey. Multiple logistic regression was also used to model these associations and generate odds ratios adjusted for the other variables included in the model. McFadden’s pseudo R-squared was calculated for goodness of fit.¹⁵ Statistical significance was set at P < 0.05. All analyses were performed in R.

RESULTS

The survey had an overall response rate of 13% (n = 1303). A summary of the patient characteristics overall and by response category can be seen in Table 1. Univariate and adjusted regression models are displayed in Table 2. Univariate analysis suggested that older age, white race, being married, more comorbidity, more doctor visits, Medicare insurance, and higher census tract education and income levels were all significantly associated with a higher response rate. Patients with a physician at a suburban satellite hospital (the “employed” group) were less likely to respond.

Table 1.

Patient characteristics by response categories (% are among cells, not across rows)

Characteristic	All (n = 10 015)	Responded to survey
		Yes (n = 1327)	No (n = 8688)
Age
<40	2880 (29%)	156 (12%)	2724 (31%)
40–64	5118 (51%)	654 (49%)	4464 (51%)
≥65	2017 (20%)	517 (39%)	1500 (17%)
Race
White	4468 (45%)	764 (58%)	3704 (43%)
Black	2624 (26%)	212 (16%)	2412 (28%)
Other	2923 (29%)	351 (26%)	2572 (30%)
Gender
Male	4700 (47%)	644 (49%)	4056 (47%)
Female	5313 (53%)	682 (51%)	4631 (53%)
Marital status
Not married	5065 (51%)	604 (46%)	4461 (51%)
Married	4950 (49%)	723 (54%)	4227 (49%)
Charlson index
0	5779 (68%)	715 (61%)	5064 (69%)
1	1697 (20%)	261 (22%)	1436 (20%)
>1	1009 (12%)	195 (17%)	814 (11%)
Annual average of specialty care visits
0–1	7755 (77%)	894 (67%)	6862 (79%)
>1	2260 (23%)	434 (33%)	1826 (21%)
Annual average of primary care physician (PCP) visits
0–1	5386 (54%)	672 (51%)	4714 (54%)
>1	4629 (46%)	655 (49%)	3974 (46%)
Insurance
Commercial	5283 (67%)	608 (58%)	4675 (68%)
Medicare	1582 (20%)	366 (35%)	1216 (18%)
Medicaid	786 (10%)	58 (6%)	728 (11%)
Other	284 (4%)	20 (2%)	264 (4%)
PCP group
HFPN	3341 (33%)	489 (37%)	2852 (33%)
Employed	3338 (33%)	262 (27%)	2976 (34%)
HFMG	3336 (33%)	476 (36%)	2860 (33%)
Percent with high school diploma or higher (in census block)
<80%	3277 (33%)	353 (27%)	2852 (33%)
80–89%	3297 (33%)	416 (31%)	2976 (34%)
≥90%	3441 (34%)	558 (42%)	2860 (33%)
Median household income (based on census block)
<$35k	3699 (37%)	396 (30%)	3303 (38%)
$35k–$55k	2907 (29%)	391 (30%)	2516 (29%)
>$55k	3396 (34%)	536 (41%)	2860 (33%)

Table 2.

Univariate and adjusted associations between patient characteristics and whether patient responded to survey

Characteristic	Univariate logistic model		Multiple logistic model*
	OR (95% CI)	P-value	OR (95% CI)	P-value
Age (10 y increase)	1.47 (1.41–1.52)	<0.001	1.40 (1.31–1.49)	<0.0001
Race (vs White)
Black	0.43 (0.35–0.50)	<0.001	0.50 (0.41–0.61)	<0.001
Other	0.66 (0.58–0.76)	<0.001	0.74 (0.63–0.87)	<0.001
Female (vs Male)	0.93 (0.83–1.04)	0.194	1.05 (0.92–1.21)	0.465
Married (vs Not)	1.26 (1.12–1.42)	<0.001	1.00 (0.87–1.15)	0.993
Charlson index (vs 0)
1	1.29 (1.10–1.50)	0.001	0.91 (0.74–1.12)	0.364
>1	1.70 (1.42–2.02)	<0.001	0.96 (0.81–1.15)	0.686
Average annual doctor visits (>1 vs 1 or fewer)
Specialty	1.83 (1.61–2.07)	<0.001	1.32 (1.12–1.54)	<0.001
PCP	1.16 (1.03–1.30)	0.014	1.22 (1.03–1.44)	0.019
Insurance (vs Commercial)
Medicare	2.31 (2.00–2.67)	<0.001	0.93 (0.76–1.14)	0.484
Medicaid	0.61 (0.46–0.81)	<0.001	0.75 (0.55–1.02)	0.068
Other	0.58 (0.37–0.92)	0.022	0.67 (0.40–1.12)	0.125
PCP (vs HFMG)
Employed	0.73 (0.63–0.85)	<0.001	0.92 (0.78–1.09)	0.331
HFPN	1.03 (0.90–1.18)	0.669	1.04 (0.83–1.30)	0.737
Percentage of census block with HS diploma or higher (vs <80%)
80–89%	1.20 (1.03–1.39)	0.020	1.04 (0.79–1.24)	0.769
≥90%	1.60 (1.39–1.85)	<0.001	1.27 (0.92–1.74)	0.140
Census block median household income (vs <$35k)
$35k–$55k	1.30 (1.12–1.50)	<0.001	0.94 (0.72–1.24)	0.679
> $55k	1.56 (1.36–1.80)	<0.001	0.93 (0.69–1.26)	0.639

^*Adjusted for all variables on the table, McFadden’s pseudo R-squared = 0.30 (indicating very good fit).¹⁵

After adjustment for all variables, only the associations with age, race, and doctor visits remained statistically significant. A 10-year increase in age was associated with an odds ratio for survey participation of 1.40 (P < 0.001). Participant portal members who self-identified as “Black” had an inverse odds ratio of 0.50 and “Other” race had an odds ratio of 0.74 (both P < 0.001). Patients who averaged more than one visit to a specialist per year over the last 2 years were more likely to participate with an odds ratio of 1.32 (P < 0.001), and the odds ratio for primary care provider visits was similar (OR = 1.22, P = 0.02).

DISCUSSION

Previous research has shown that there are demographic differences in respondents that prefer online surveys compared to those that prefer other more traditional methods (such as mail and phone).^2,16,17 In a variety of studies, those preferring web-based surveys tend to be younger,^2,16,17 more educated,^2,16,17 and have higher incomes.¹⁶ However, Sinclair et al hypothesized that internet surveys may be more effective in specialized groups where email lists are readily available.⁶ The motivation behind this study was to determine if there were demographic differences in respondents to a survey solicited to a group who had already expressed an interest in engaging online through a patient portal.

We found that not only were there demographic differences in those that responded to a survey within this group, they were not entirely consistent with previously published findings. Jones et al have shown that when given the choice between telephone and internet surveys, those who chose internet were younger, more educated and had higher incomes. Respondents to our survey tended to be older, while those that historically prefer internet surveys have been younger. Our respondents were also more likely to be white, although racial/ethnic differences have not previously been shown.¹⁷ Although we did not find any significant differences with regard to education or income, those who resided in a census block compromised of a population with a higher education level were slightly more likely to respond.

Another interesting finding was that our respondents tended to be more frequent users of care from both specialist and primary care physicians. This particular population may also be more frequent users of their patient portal and therefore more likely to respond to the survey. This may bias a survey solicited in this fashion to include a sample that is likely to be less healthy than the general population. Alternatively, this sample may not be less healthy than the general population but instead may more regularly seek care, both through primary and specialty care.

Subgroup analyses were also performed to look at associations among those who received health insurance from Health Alliance Plan (HAP, an Healthcare Maintenance Organization where a large number of members work in the health care system), Blue Cross Blue Shield and Medicaid. This analysis showed that minority groups with HAP and Blue Cross insurance were less likely to respond (as they were in the primary analyses), but within the Medicaid population this was not true. While no differences were found regarding marital status in the primary analysis, married HAP members were less likely to respond. Last, those with higher specialty care usage were more likely to respond if they had HAP and Blue Cross insurance, but not if they had Medicaid. It is perhaps not surprising that those with different types of health care insurance may exhibit different behaviors when responding to a survey from their health care provider.

There are many possible causes for demographic differences in responding, such as lack of internet access, fear of technology, or limited technological proficiency. As the use of internet-based methods for survey administration increases, it will be increasingly important to be aware of the possible demographic differences in respondents and adjust methods accordingly. For example, the literature shows that single mode surveys are far less effective than multimode methods^18–20 and that web-based survey response rates are improved when followed by letter mailings and/or phone calls.²¹ Providing incentives for survey completion may also increase the response rate. Utilizing multiple modes of survey administration is even more important in diverse populations.

The overall response rate was similar to other internet surveys. However, we expected a greater response rate due to the fact that this population was theoretically already engaged online with the health system. The response rate was likely affected by a short time given to respond (2 weeks) and lack of reminders. The lack of a reminder is also a limitation in that we compare our results to other studies in which varying levels of reminder messaging was utilized. Additionally, education and income were determined from census tract and were not directly obtained from potential respondents. While this method has been shown to be more accurate than using zip code level data, it is not as accurate as microlevel data and therefore is a limitation of this analysis.²² Other limitations of this study include that it was performed within a single health system and within the metro Detroit area only. Also, we are unable to ascertain the effect that internet access or digital divide may have affected these results, but the study population was selected from active users of the patient portal so we assume they or an associate that provides assistance have some level of internet access and digital knowledge. Despite these limitations, the results of this method clearly show that respondents to a survey of patient portal users may not be entirely representative and it may be necessary to develop alternative strategies to obtain a representative sample from such populations.

CONCLUSION

Internet surveys are gaining popularity as research tools as they are significantly less expensive and faster than more traditional methods. Patient portal members are an easily identifiable and contactable group that are potentially valuable participants for research; however, it is important to understand that respondents to surveys solicited from this sampling frame may not be entirely representative. It will be important to develop strategies to more fully engage representative populations to increase overall and subgroup response rates. For example, it may be worthwhile to employ mixed methods (mailed, phone call, and internet surveys) or oversample particular populations if you require a representative sample.

FUNDING

This work was supported by AHRQ Grant #1R24HS022417-01.

AUTHOR CONTRIBUTIONS

Authors contributed to the manuscript in all of the following ways:

• Substantial contributions to the conception or design of the work (Rubinfeld, Conway, Allard, and Johnson); or the acquisition, analysis, or interpretation of data for the work (Peltz-Rauchman, Divine, and McLaren)

• Drafting the work (Peltz-Rauchman and Divine) or revising it critically for important intellectual content (McLaren, Rubinfeld, Conway, Allard, and Johnson)

• Final approval of the version to be published (all authors)

• Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved (all authors)

COMPETING INTERESTS STATEMENT

The authors have no competing interests to declare.