Abstract
Background
In recent literature, patient portals are considered as important tools for the delivery of patient-centered care. To date, it is not clear how patients would conceptualize a patient portal and which health information needs they have when doing so.
Objective
This study aimed (1) to investigate health information needs, expectations, and attitudes toward a patient portal and (2) to assess whether determinants, such as patient characteristics, health literacy, and empowerment status, can predict two different variables, namely the importance people attribute to obtaining health information when using a patient portal and the expectations concerning personal health care when using a patient portal.
Methods
We conducted a cross-sectional survey of the Flemish population on what patients prefer to know about their digital health data and their expectations and attitudes toward using a patient portal to access their electronic health record. People were invited to participate in the survey through newsletters, social media, and magazines. We used a questionnaire including demographics, health characteristics, health literacy, patient empowerment, and patient portal characteristics.
Results
We received 433 completed surveys. The health information needs included features such as being notified when one’s health changes (371/396, 93.7%), being notified when physical parameters increase to dangerous levels (370/395, 93.7%), observing connections between one’s symptoms or diseases or biological parameters (339/398, 85.2%), viewing the evolution of one’s health in function of time (333/394, 84.5%), and viewing information about the expected effect of treatment (349/395, 88.4%). Almost 90% (369/412) of respondents were interested in using a patient portal. Determinants of patients’ attachment for obtaining health information on a patient portal were (1) age between 45 and 54 years (P=.05); (2) neutral (P=.03) or interested attitude (P=.008) toward shared decision making; and (3) commitment to question physicians’ decisions (P=.03, R2=0.122). Determinants of patients’ expectations on improved health care by accessing a patient portal were (1) lower education level (P=.04); (2) neutral (P=.03) or interested attitude (P=.008) toward shared decision making; and (3) problems in understanding health information (P=.04; R2=0.106).
Conclusions
The interest in using a patient portal is considerable in Flanders. People would like to receive alerts or some form of communication from a patient portal in case they need to act to manage their health. Determinants such as education, attached importance to shared decision making, difficulties in finding relevant health information, and the attached importance in questioning the decisions of physicians need to be considered in the design of a patient portal.
Keywords: cross-sectional studies, consumer health informatics, electronic health record, patient portal, personal health records
Introduction
The expression “no decision about me, without me,” as formulated by the British National Health Service, emphasizes the importance of patient-centered care and shared decision making [1]. Patient-centered care, which is an important feature of high-quality health care, is defined as “providing care that is respectful of and responsive to individual patient preferences, needs, and values, and ensuring that patient values guide all clinical decisions” [2]. The Institute of Medicine considers patient-centered care as one of the 6 objectives to be achieved to improve the quality of US health care [2]. It can be promoted with the help of patient-centered health information technologies [3], such as a patient portal.
A patient portal is known to improve the quality of and access to health care [4]. In addition, a patient portal exerts a positive influence on health care utilization [5]. A patient portal can be defined as “An electronic application through which individuals can access, manage and share their health information, and that of others for whom they are authorized, in a private, secure, and confidential environment” [6,7]. More specifically, a patient portal can be described as an application that is linked (tethered) to the electronic health record of the doctor [8,9]. The functions of a patient portal often include a medication list, test results, a list of allergies, a problem list, social history, major illness, lifestyle, family history, and links to personalized information [7,9].
Although patient portals are considered important tools for the development of patient-centered care, the current use is not optimal, and portals are still less patient-centered than they could be [10-12]. Known barriers to the use of portals for patients and providers include security and privacy issues, the negative impact on the workflow and limited user-friendliness [4,13]. Benefits associated with the use of a patient portal are increased convenience and satisfaction for patients [14,15].
Patient safety could be improved by identifying errors in medication lists [16,17]. Moreover, providing Web access could lower the threshold for the patient-clinician contact [13]. In addition, the quality of care can be improved by the sustained use of some features of a patient portal. For example, Web-based prescription refills and secure messaging have an impact on some physiological measures important for the management of type 2 diabetes [18].
Patients’ interest and ability to use a patient portal are influenced by age, health literacy, and level of education. Patients are more likely to adopt a patient portal if its features align with their information needs and with the functionalities they require [19].
To date, however, it remains unclear what patients’ information needs and functionality requirements exactly are. In addition, participants in this study had no prior experience with patient portals as they do not exist in Belgium. The survey intro was the first exposure people had to the concept of a patient portal. Therefore, this study aimed to investigate patients’ information needs with regard to the concept of a patient portal and its presumed use to access their health data. Furthermore, this study aims to assess patients’ expectations and attitudes regarding a patient portal and its use to access electronic health records in primary care. Finally, we investigate whether patients’ information needs, expectations, and attitudes regarding a patient portal correlate with patient-related determinants such as personal characteristics, health literacy, and empowerment status.
Methods
Study Design
We conducted a cross-sectional survey on the digital health information needs of patients and their expectations and attitudes toward using a patient portal to access their electronic health record in primary care among inhabitants of the Dutch-speaking part of Belgium (Flanders).
Study Population and Sample Size
The study population included inhabitants of the Flemish part of Belgium. For precise estimation of our results, we calculated the sample size using the free Web-based software Raosoft [20]. For a population size of 6,471,996, with a 5% margin of error and a 50% response distribution, we needed 385 completed surveys.
Design and Domains of the Survey
Our open and voluntary survey was a convenience sample and consisted of 2 parts (Multimedia Appendix 1 for the English version and Multimedia Appendix 2 for the Dutch Version).
The first part was based on the Health Information Technology Evaluation Collaborative (HITEC) Consumer Health Information Technology Survey as designed by Patel et al [21]. The second part was based on findings from previous qualitative research. The first page of the survey contained the informed consent and a mandatory checkbox for participants to acknowledge that they have read and agree with the informed consent. The final page of the survey contained a mandatory checkbox for participants to acknowledge they wanted to send the information to the researchers. If not, the data were not stored. Multimedia Appendix 3 provides a detailed description of the survey methodology according to the Checklist for Reporting Results of Internet E-Surveys [22].
Part 1 of the Survey
The HITEC Consumer Survey, developed in the United States, aimed to characterize consumer attitudes toward personal health records and included questions on potential personal health record use, preferences, and attitudes toward personal health records, the use of computers and the internet, experience with health care, health characteristics, and demographics [21].
One researcher translated the HITEC Consumer Survey from English to Dutch, and 2 researchers with adequate English proficiency translated the Dutch survey independently from each other back to English to reduce language bias to a minimum [23,24]. Small differences in the translation were solved by consensus. In the Dutch translation, a patient portal was defined as “an electronic online tool to view and manage patients’ health care information stored in the electronic health record of their general physician” [8]. Of note, 2 questions were not included in the Dutch translation of the questionnaire—one about payment options and one about private insurance companies—because these were not applicable to the Belgian situation. A few other questions were slightly changed because of the different demographic situation in Belgium compared with that in the United States. For example, questions inquiring about ethnicity were reformulated to match the common Dutch terminology. Instead, of asking “what is your race” or “are you of Latino or Hispanic origin or descent,” we asked, “What is your mother’s, father’s, and your country of birth?”
Part 2 of the Survey
The questions in this part of the survey were based on the findings of a previous qualitative study (Multimedia Appendix 4). This study aimed to understand what motivated people to search for health information. For this purpose, cultural probes and the Lillidots method were used [25]. Cultural probes are objects, such as a diary or camera, provided to participants to self-report data; cultural probes perform similar to astronomic probes because they are also left behind and return with fragmentary data [26,27]. In the study mentioned above, cultural probes were used to sensitize participants about digital health data to make it easier for them to express their feelings and experiences regarding health information. Participants were then interviewed to learn what health topics they wanted to know more about and what their questions and expectations were concerning these topics. For this purpose, an ideation method, called the Lillidots method [25], was combined with a strategy to anthropomorphize the technology at hand (digital health data in this case). This method, detailed elsewhere [28], resulted in insights about the type of health data people are interested in, questions they have in this respect, and their motivations to ask these questions. The second part of the survey was based on these motivations and included questions such as “In your relationship with your doctor, how important is it for you to be able to question their decision” and “Please indicate how important each of the following aspects would be to you, if you had access to all the data in your patient portal.”
In a nutshell, the second part of the survey was developed based on the themes that emerged from the previous qualitative study: (1) health awareness; (2) coping; (3) effective care; (4) empowerment; (5) good health; (6) patient rights; and (7) getting recognition. These themes grouped several items, such as people want to understand whether and when care is necessary and compare possible treatments (effective care) or what reimbursements they are entitled to (patient rights). To validate these findings with a larger population a questionnaire was constructed based on these items (part 2 of the survey). Combining part 1 and 2 allowed us to use a quantitative research instrument with our bottom-up qualitative research method.
Pilot
A pilot was performed in a small sample of 6 Dutch-speaking Belgians to establish whether the survey was feasible, particularly regarding the ease of use and time of completion. All participants appraised the survey as user-friendly and completed it in an average time of 20 minutes, which was considered acceptable. We did not include any pilot data in the large sample for analysis.
Data Collection
The data were collected between March 25 and September 1, 2016. Inhabitants of the Flemish part of Belgium were invited to participate through several channels—the member magazine of a health insurance with a print run expanding up to 1,200,000 copies, only this channel gave readers an opportunity to choose between Web-based reply and paper; the Web-based newsletter of the same health insurance; the newsletter of a website called “Health and Science,” which is an independent and evidence-based website to inform patients about health-related topics (12,000 subscribers); and the website of a well-known Flemish weekly magazine (knack.be), which receives 54,769 unique visitors and 141,304 (2.58 per visitor) page views per day [29] and their social media channels (Facebook and Twitter). In addition, some students of the Faculty of Social Sciences used their social networks to recruit participants. Two reminders were sent by the health insurance and the “Health and Science” newsletter to obtain the greatest possible response. People without access to a computer or the internet could phone to request a paper copy of the survey. This ensured involving as many people as possible and having a representative sample of the population to minimize selection bias. The paper surveys we received were manually entered into the database containing the results from the Web-based surveys.
Analyses
Descriptive Statistics (Frequencies)
In this study, descriptive statistics (frequencies) were used to assess the following:
Demographics (age, gender, mother tongue, employment status, education, income, internet use, and residence),
Health characteristics (answers to questions on self-rated health status, the presence of a chronic disease, >3 annual visits to the primary care physician, and prescribed medication use),
Health literacy (answers to questions on difficulties with finding relevant and reliable health information),
Patient empowerment (answers to questions on the quality of health care received in the last 5 years and questions about the importance of shared decision making and being able to question physicians’ decisions),
The potential use of a patient portal, attitude and expectations when using a patient portal (answers to questions on one’s interest in using a patient portal, types of information expected to be found in a patient portal, Web-based activities concerning health, frequency of using a patient portal, perceived usefulness of a patient portal, difficulty with using a patient portal, granting access to one’s own patient portal, and importance of certain features of a patient portal).
Determinant Analyses
To investigate whether determinants such as patient characteristics, health literacy, and patient empowerment predict the importance people attach to obtaining health information when accessing a patient portal and the expectations and attitudes of people toward the use of a patient portal, we used linear and logistic regression models. First, bivariate regression models were used to investigate associations between all variables. Only predictor variables that were significantly associated (P ≤.10) with the dependent variable in bivariate analyses were used for the multivariate analyses.
Bivariate and multivariate linear regression models were used to predict 2 different dependent variables. For one of these “the importance people attach to obtaining health information when using a patient portal,” we used the mean sum score of 5-point Likert scale answers (ranging from 1 “very important” to 5 “very unimportant”) to 12 questions asking about the importance of doing certain things with the help of a patient portal (Cronbach alpha=.897). We considered the mean sum score as an interval variable and used it as a dependent variable in a linear regression model. [30]
For the second dependent variable “expectations concerning one’s personal health care when using a patient portal,” we used the mean sum score of answers to 9 questions asking for the level of expected improvement on different aspects of health care when using a patient portal (ranging from 1 “will greatly improve” to 5 “will greatly worsen”; Cronbach alpha=.871). We used the mean sum score to compensate for missing data in ≥1 of the questions. Preliminary analyses were performed to ensure there was no violation of the assumption of normality, linearity, and multicollinearity. If a correlation (>.6) between independent variables was detected, only the variable with the greatest influence on the R2of the model was included in the multivariate analyses.
We used bivariate and multivariate logistic regression models to predict the attitude toward using a patient portal. We used answers to the question asking for the interest in using a patient portal as the dependent variable. This ordinal variable with 5 categories was recoded into a dichotomous variable (interested and not interested or neutral)
As predictor variables, we used answers to the questions asking for patient characteristics (age, employment status, family income, health status, education, and gender), patient empowerment (shared decision making, questioning the decisions of physicians, satisfaction with health care received the last 5 years), and health literacy (finding relevant information about health, evaluating the reliability of health information, and problems in understanding health information; Multimedia Appendix 5).
Except for the internal consistency (Cronbach alpha) of our composite dependent variables, we did not include any psychometric measures for our predictor variables because of the gap in the availability of sound psychometric measures for evaluating patient-facing eHealth technologies [31]. All statistical analyses were performed with IBM SPSS Statistics version 24 for Windows (IBM Corp, Armonk, NY, USA).
Missing Data
To calculate the frequencies, missing data were excluded, and percentages were based on the number of nonmissing values. For the regression models, we used independent variables that had a low percentage of missing data (<15%). If values of any of the independent variables included missing data, the entire case was excluded for the analysis.
Ethical Approval
This study was approved by the Social and Societal Ethics Committee of the Faculty of Social Sciences of KU Leuven on July 14, 2015 (first part of the survey) and on September 24, 2015 (second part of the survey) with the grant number G-2015 07 272.
Results
Demographics and Health Characteristics
The survey was completed by 433 people. While 10 surveys were submitted on paper, 423 were completed through the Web (Table 1). The completion rate of the Web-based survey was 91% (Multimedia Appendix 3). Differences between the 10 surveys submitted on paper and those completed on the Web were mainly age of respondents, age-related properties (eg, employment status), and their internet use. The mean age in the group of the paper surveys (n=10) was 68.6 years (SD 10.146), while it was 53 years (SD 16.497) in the Web-based group (n=423). All respondents of the paper surveys group were not working anymore (9/10 retired and 1/10 disabled), and 40% (4/10) did rarely or never used the internet, while only 0.2% (1/423) of the Web-based group rarely or never used the internet. The mean age of all the respondents (n=433) was 53.28 (SD 16.451) years.
Table 1.
Characteristics | Participants, n (%)a | |
Male | 187 (47.1) | |
Dutch-speaking | 394 (99.0) | |
Age (years) | ||
|
18-34 | 72 (19.1) |
|
35-44 | 39 (10.4) |
|
45-54 | 53 (14.1) |
|
55-64 | 95 (25.3) |
|
>65 | 117 (31.1) |
Employment status | ||
|
Employed | 164 (41.3) |
|
Student | 21 (5.3) |
|
Unemployed | 51 (12.8) |
|
Retired | 161 (40.6) |
Education | ||
|
Secondary school | 130 (32.7) |
|
Bachelor degree | 159 (40.1) |
|
Master degree or higher | 108 (27.2) |
Family income | ||
|
€ <30,000 | 114 (28.9) |
|
€ 30,000-60,000 | 100 (25.3) |
|
€ >60,000 | 58 (14.7) |
|
Does not know or prefers not to disclose | 123 (31.1) |
Internet use | ||
|
Internet use ≥1 time per day | 423 (97.9) |
|
Searched information on health or disease on the Web | 406 (94.4) |
Residence description | ||
|
Rural | 202 (50.6) |
|
Urban | 197 (49.4) |
Self-rated health status | ||
|
Excellent or very good | 121 (28.0) |
|
Good or fair | 276 (63.9) |
|
Poor | 35 (8.1) |
Chronic medical condition | 213 (49.4) | |
Visited primary caregiver >3 times in a year | 214 (50.1) | |
Taking prescribed medication | 300 (69.4) |
aNumbers may not sum to totals because of missing data. Percentages were calculated considering the missing data.
Almost 92% (397/432) of participants reported that they were in excellent or good health, although 49.4% (213/431) reported they had a chronic disease, and 69.4% (300/432) were taking prescribed medication (Table 1).
Health Literacy and Patient Empowerment
Finding relevant health information was considered difficult by 26.2% (106/404) of respondents, and assessing the reliability of the health care information found was considered difficult by 48.3% (195/404). More than 93% (377/404) of participants reported that they found shared decision making important, and 89.9% (363/404) of respondents thought it was important to be able to question the decisions made by physicians (Table 2).
Table 2.
Health literacy and patient empowerment | Participants, n (%)a | |
Satisfaction with the quality of health care received in the last 5 years | ||
|
Satisfied | 366 (84.5) |
|
Neutral | 39 (9.0) |
|
Dissatisfied | 28 (6.5) |
Finding relevant health information | ||
|
Difficult | 106 (26.2) |
|
Not easy or not difficult | 149 (36.9) |
|
Easy | 149 (36.9) |
Assessing the reliability of Web-based health care information | ||
|
Difficult | 195 (48.3) |
|
Not easy or not difficult | 116 (28.7) |
|
Easy | 93 (23.0) |
Importance of shared decision making | ||
|
Important | 377 (93.3) |
|
Neutral | 18 (4.5) |
|
Not important | 9 (2.2) |
Importance of being able to questions physicians’ decisions | ||
|
Important | 363 (89.9) |
|
Neutral | 37 (9.1) |
|
Not important | 4 (1.0) |
aNumbers may not sum to totals because of missing data. Percentages were calculated considering the missing data.
Patient Portal’s Potential Impact and Features
Most respondents were interested in the potential use of a patient portal (369/412, 89.6%; Table 3). The information that most people wanted to see in their patient portal were test results (381/410, 92.9%), current medication (345/410, 84.1%), immunization records (338/410, 82.4%), and their past medical visits, procedures, and surgeries (338/410, 82.4%). This corresponds with the Web-based health-related activities people are most interested in, namely viewing their medical records, test results, medication list (384/405, 94.8%); requesting appointments, referrals, and prescription refills (376/403, 93.3%); and signing up for reminders for preventive medicine (360/399, 90.2%).
Table 3.
Patient portal characteristics | Participants, n (%)a | |
Interested in using a patient portal | 369 (89.6) | |
Types of information people would prefer to have in their patient portal | ||
|
My allergies | 263 (64.1) |
|
Test results (eg, X-rays, blood tests, etc) | 381 (92.9) |
|
Immunization records (list of vaccines received) | 338 (82.4) |
|
Medication I have taken or am currently taking | 345 (84.1) |
|
List of doctors or health care providers I have seen | 279 (68.0) |
|
Family history of health problems | 236 (57.6) |
|
Medical problems | 322 (78.5) |
|
Medical visits or surgeries or medical procedures I have had | 338 (82.4) |
|
Lifestyle choices (eg, smoking history and exercise) | 138 (33.7) |
|
Information from devices that help me monitor my health | 199 (48.5) |
Activities I am doing or would like to do on the internet | ||
|
View medical records, test results, and medication list | 384 (94.8) |
|
Add notes to my medical record | 312 (78.6) |
|
Request appointments, referrals, prescription refills | 376 (93.3) |
|
Communicate with my doctor and receive reports by mail | 358 (88.8) |
|
Fill out paperwork before or after a physician visit | 331 (83.0) |
|
Sign up for reminders for preventive medicine (eg, flu shot) | 360 (90.2) |
|
Learn about opportunities to participate in medical research | 327 (82.6) |
|
Access my child’s or parents’ medical record if I am primary caretaker | 315 (81.2) |
|
Communicate with other people with similar health problems | 197 (50.8) |
|
Receive educational materials related to my health | 304 (79.4) |
|
Record my representative to manage my health care when I am not able | 337 (83.6) |
Expected frequency of patient portal use | ||
|
At least 1 time per week | 61 (14.8) |
|
1 time per month | 159 (38.6) |
|
Every 3-6 months | 159 (38.6) |
|
Rarely or never | 33 (8.0) |
How many people think the use of a patient portal will improve the followingb | ||
|
Security and privacy of my medical information | 90 (22.3) |
|
Communication between my doctors and myself | 321 (78.7) |
|
My understanding of my own health | 300 (73.7) |
|
My sense of control over my own health care | 311 (76.4) |
|
My worries about my own health care | 236 (58.1) |
|
The safety of my care (freedom from errors) | 278 (68.6) |
|
My satisfaction with my health care | 275 (68.1) |
|
The overall quality of my health care | 292 (72.1) |
|
The overall costs of my health care | 192 (47.4) |
Difficulty to use a patient portal to view and manage your health information and care | ||
|
Difficult | 22 (5.3) |
|
Easy | 292 (70.9) |
|
Neutral | 98 (23.8) |
Who would you give permission to view information in your patient portal | ||
|
Designated family members or friends | 226 (54.6) |
|
My primary care doctor | 391 (94.4) |
|
Other doctors or health care providers who care for me | 332 (80.2) |
|
My health insurance | 26 (6.3) |
|
My employer | 1 (0.2) |
|
The government | 8 (1.9) |
|
No-one | 18 (4.3) |
Importance of certain features when using a patient portalc | ||
|
Compare recent personal health data with health data from the past | 311 (78.1) |
|
Compare personal health data with medical standards | 291 (73.5) |
|
Compare personal health data with anonymous data from other patients | 146 (36.9) |
|
Compare personal health data with anonymous data from the Flemish population | 146 (37.1) |
|
See connections between your symptoms, your disease(s), your biological parameters | 339 (85.2) |
|
See connections between your health and the presence of environmental factors | 284 (71.7) |
|
View the evolution of your health in function of time | 333 (84.5) |
|
View information about the expected effect of treatment on your personal health | 349 (88.4) |
|
View information on the expected impact of your lifestyle on your personal health | 309 (79.0) |
|
Provide your data anonymously so that regional problems can be detected | 250 (63.1) |
|
Be notified when certain physical parameters evolve toward dangerous levels | 370 (93.7) |
|
Be notified when your health changes | 371 (93.7) |
aNumbers may not sum to totals due to missing data. Percentages were calculated considering the missing data.
bUsed as a composite dependent variable: “expectation concerning one’s personal health care when using a patient portal.”
cUsed as a composite dependent variable: “importance attached to obtaining health information when using a patient portal to access health data.”
Respondents were less interested in seeing their lifestyle choices (138/410, 33.7%) and information from devices to help monitor their health (199/410, 48.5%; Table 3).
The perceived impact of patient portal use varied. Only 22.3% (90/404) of respondents believed that the patient portal use would improve the security and privacy of their medical data, and 47.4% (192/405) of participants thought that using a patient portal would reduce the overall cost of their health care.
The majority (391/414, 94.4%) would give their primary care doctor permission to view information in their patient portal. The potential features of a patient portal that were considered important by respondents were being notified when certain physical parameters evolve toward dangerous levels (370/395, 93.7%), being notified when their health changes (371/396, 93.7%), being able to view the expected impact of treatment on personal health (349/395, 88.4%), being able to see connections between symptoms, disease(s), biological parameters, etc (339/398, 85.2%), and being able to view the evolution of their health in function of time (333/394, 84.5%). Consumers were less interested in comparing their personal health data with anonymous data from other patients (146/396, 36.9%) and with anonymous data from the Flemish population (146/394, 37.1%).
Determinant Analysis of the Importance People Attribute to Obtaining Health Information When Using a Patient Portal
Bivariate linear regression showed that the importance people attributed to obtaining health information when using a patient portal to access health data (dependent) was significantly associated (cutoff P≤.10) with age, employment status, self-rated health status, the interest in shared decision making, the importance of being able to question the decisions of physicians, the difficulty in finding relevant health information, the difficulty in assessing the reliability of health information, and the difficulty in understanding health information. Due to the collinearity between age and employment status, only age was used for multivariate analysis.
In the multivariate linear regression, age, shared decision making, and the importance of being able to question the decisions of physicians were significant (cutoff P≤.05). People attributed greater importance to receiving health information if they were aged 45-54 years compared with those who were aged 18-34 years, if they had an interested or neutral attitude toward shared decision making compared with having a negative attitude toward shared decision making and if they found it important to be able to question the decisions made by physicians compared with finding this unimportant (R2=0.122; Table 4).
Table 4.
Multivariate regression models | P value | Beta (95% CI) | ||
Importance of obtaining health information | ||||
|
Shared decision making | |||
|
|
Important | .03 | –.558 (–0.969 to –0.147) |
|
|
Neutral | .008 | –.500 (–0.952 to –0.048) |
|
|
Unimportant (constant) | —a | — |
|
Questioning decisions of physicians | |||
|
|
Important | .03 | –.642 (–1.222 to –0.061) |
|
|
Neutral | .12 | –.467 (–1.052 to 0.119) |
|
|
Unimportant (constant) | — | — |
|
Agebin years | |||
|
|
18-34 (constant) | — | — |
|
|
35-44 | .069 | –.186 (–0.386 to 0.014) |
|
|
45-54 | .047 | –.183 (–0.363 to –0.002) |
|
|
55-64 | .10 | –.126 (–0.278 to 0.026) |
|
|
>65 | .099 | –.124 (–0.271 to 0.023) |
Expectations concerning personal health care | ||||
|
Education | |||
|
|
High school or lower | .04 | –.155 (–0.303 to –0.007) |
|
|
Bachelor degree | .84 | –.015 (–0.153 to 0.124) |
|
|
Master degree or higher (constant) | — | — |
|
Shared decision making | |||
|
|
Important | .008 | –.566 (–0.983 to –0.148) |
|
|
Neutral | .03 | –.521 (–0.989 to –0.052) |
|
|
Unimportant (constant) | — | — |
|
Finding relevant health information | |||
|
|
Difficult (constant) | — | — |
|
|
Not difficult or not easy | .017 | .180 (0.032 to 0.328) |
|
|
Easy | .022 | .197 (0.028-0.366) |
|
Problems in understanding health information | |||
|
|
Frequently | .99 | .002 (–0.234 to 0.237) |
|
|
Sometimes | .037 | –.141 (–0.272 to –0.009) |
|
|
Rarely (constant) | — | — |
aNot applicable.
bOwing to collinearity with employment status, only age was used for the multivariate regression.
Determinant Analysis of Expectations Concerning Personal Health Care When Using a Patient Portal
Bivariate linear regression showed that expectations concerning personal health care when using a patient portal (dependent) were significantly associated (cutoff P≤.10) with age, employment status, education, the interest in shared decision making, the importance of being able to question the decisions of physicians, the difficulty in finding relevant health information, the difficulty in assessing the reliability of health information, and the difficulty in understanding health information.
Due to the collinearity between age and employment status, only the employment status was used for multivariate analysis. Multivariate linear regression showed that expectations concerning personal health care when using a patient portal (dependent) was significantly associated (cutoff P≤.05) with education, shared decision making, the difficulty in finding relevant health information and problems in understanding health information (Table 4). People expected an improvement in their individual health care when using a patient portal if they had lower levels of education (high school degree or lower) compared with highly educated participants (master degree or higher), an interested or neutral attitude toward shared decision making compared with a negative attitude toward shared decision making and if they sometimes had problems to understand health information compared with rarely having problems in understanding health information. People expected an impairment in their health care when using a patient portal if they found it easy or had a neutral attitude toward finding relevant health information compared with thinking it is difficult to find this information (R2=0.106; Table 4).
Determinant Analysis of the Interest in Using a Patient Portal
Owing to the low variance between the independent and dependent variables, the determinant analysis showed results with very high uncertainty and was not conclusive.
Discussion
Principal Findings
This study used a cross-sectional survey design to investigate health information needs, expectations, and interest of people accessing a patient portal to view their health data. The health information needs in this context are mainly features such as being notified when one’s health changes and being notified when physical parameters increase to dangerous levels. In addition, people are interested to see connections among their symptoms, disease, and biological parameters. Moreover, they prefer to have an overview of how their health evolves and to have access to information about the expected effect of treatment on their health. These findings are in line with the patients’ desire for direct communication through a patient portal that was described in previous work [32].
The importance that people attribute to obtaining health information when using a patient portal is partially predicted by age and the level of patient empowerment, namely the importance of shared decision making and questioning physicians’ decisions. Expectations concerning the impact on personal health care when using a patient portal are influenced by the level of education, interest in shared decision making, and the difficulty people experience in finding and understanding the relevant health care information. These findings confirm that screening for eHealth literacy and providing training in the use of a patient portal could help in improving the experience and expectations people have when using a patient portal [33,34].
People expect that the use of a patient portal can improve the communication between their physician and themselves. However, only a few people think that the use of a patient portal will improve the security and privacy of their medical information. In this study, the expressed interest in using a patient portal was high, with almost 90% of respondents interested in the use of a patient portal, although there was no functional patient portal available in Belgium at the time the survey was submitted. The fact that people in Flanders express the need to be notified when their health changes highlights the interest of people in receiving some form of communication or alert through their patient portal as to when they need to act to manage their health care. This corresponds with previous research that states that receiving intelligent alerts is an important feature in the conceptual design of an integrated shared decision personal health record [35]. A patient-directed clinical decision system that is integrated into a patient portal could be useful for this purpose as the ongoing research hypothesizes [36].
Although earlier research shows that incorporating patients’ lifestyle is important for patient portals to become more user-centric [37], only a third of our respondents were interested to have data about lifestyle choices in their patient portal. A potential explanation might be that people when conceptualizing a patient portal consider themselves to be aware of their lifestyle and therefore do not feel the need to find this information in a patient portal. Patients in Flanders, who have no experience with a patient portal, might think about a patient portal as a unidirectional channel where they can find and consult their health information, where people who have actually used a patient portal, value it as a bidirectional tool [11].
Another important finding is the fact that almost 9 of 10 respondents consider it important to be able to question the decisions made by physicians. This finding, together with the importance of shared decision making (93% consider this important), emphasizes the fact that people in Flanders are critical health care users who greatly appreciate patient empowerment and endorses the results from a former study where Belgian inhabitants attained fairly high empowerment scores [38].
Previous research showed that people think the use of a patient portal can improve the communication between their physician and themselves [39], the understanding of their own health [14], the sense of control over their health care [40], and the overall quality of their health care [4]. In literature, there is some skepticism about the influence of a patient portal on the security and privacy of health data and the total costs of health care [13,41,42]. People are mostly interested in test results, medication schemes, immunization records, and a history of medical visits and procedures [21,43]. These findings are consistent with ours, where about three-quarters of respondents believed that a patient portal could improve the doctor-patient communication, the understanding and sense of control of their health, and the quality of care.
Only a small percentage of respondents (22.3%) think the use of a patient portal can improve the privacy and security of medical information and almost half of them believe a patient portal can lower the cost of health care. People consider information about lifestyles choices and data from devices to help monitor their health to be the least interesting information in a patient portal. The lack of interest to have data in a patient portal that is gathered from devices to monitor one’s health corresponds with findings from a previous study, which states that “tools alone are not enough” and engaging patients in the use of a patient portal requires a patient-centered approach [44].
Strengths and Limitations
This study is one of the first to investigate the health information needs of patients on conceptualizing a patient portal to access digital health data. In contrast to other studies in this field, which often focus on a functional patient portal, we had to start from the very beginning, as there was no functional patient portal available in Belgium at the time our survey was submitted [32,45-48]. In addition, the use of cultural probes and ideation techniques, frequently used in design and human computer interaction research, provides insights into the way people would like to interact with digital health data. It creates the opportunity to design a patient portal that considers the health information needs expressed by future users. Our survey was distributed across a large region in Belgium and included respondents of all age categories.
Despite our efforts to reduce selection bias to a minimum, we could not recruit participants from different ethnic backgrounds. Our respondents almost exclusively spoke Dutch. This is not in line with the demographics in Flanders, where a significant part of the population (8.1%) is of foreign origin. As almost 98% of respondents used the internet at least once a day, we could not find much evidence for the so-called “digital divide,” which is described in previous research as an important barrier in the implementation of a patient portal [49]. One potential explanation for this could be the Web-based nature of the survey and the fact that reminders to participate were mostly made with the help of social media and Web-based newsletters. This means our findings may not be representative of some groups in the Flemish population who rarely use the internet. Finally, our 2 statistical models had a low predictive power with an R2 of 0.122 (predicted importance people attribute to obtaining health information when using a patient portal) and.106 (predicted expectations toward personal health care when using a patient portal). Although these R2 values were low, this is not unusual in social research [50].
Conclusions
This study yields a range of relevant aspects to consider when designing a patient portal. First of all, people express the need for a patient portal and personal access to it. Second, people like to receive alerts or some form of communication to foster their health management. Finally, several patient characteristics influence people’s attitude toward a patient portal. As such, middle-aged people and those with a positive attitude toward shared decision making attach more importance to obtaining health information. People with a lower education level or with low health literacy expect an improvement in their health care by using a patient portal.
Acknowledgments
MyHealthData is a project cofunded by IMEC. Project partners are Mintlab, EBMPracticeNet, CEBAM, HealthConnect, Landsbond der Christelijke Mutualiteiten, and MindBytes, with project support from Agentschap Innoveren & Ondernemen. The authors gratefully acknowledge Martial Luyts and Anna Ivanova for their guidance with the statistical analyses.
Abbreviations
- HITEC
Health Information Technology Evaluation Collaborative
Survey part 1 and part 2.
Survey part 1 and part 2, Dutch translation.
The Checklist for Reporting Results of Internet E-Surveys (CHERRIES).
Qualitative study that forms the basis for the second part of the survey.
Predictor variables for regression models.
Footnotes
Conflicts of Interest: None declared.
References
- 1.NHS . Equity and Excellence: Liberating the NHS. London: The Stationery Office/Tso; 2010. [Google Scholar]
- 2.Institute of Medicine (US) Committee on Quality of Health Care in America . Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press; 2001. [PubMed] [Google Scholar]
- 3.Barry MJ, Edgman-Levitan S. Shared decision making--pinnacle of patient-centered care. N Engl J Med. 2012 Mar 01;366(9):780–1. doi: 10.1056/NEJMp1109283. [DOI] [PubMed] [Google Scholar]
- 4.Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res. 2015;17(2):e40. doi: 10.2196/jmir.3703. http://www.jmir.org/2015/2/e40/ v17i2e40 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL. Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care. 2007 Jul;13(7):418–24. http://www.ajmc.com/pubMed.php?pii=3340 .3340 [PubMed] [Google Scholar]
- 6.Markle. NY, USA: Markle Connecting for Health; 2003. Jul 1, [2018-06-24]. Personal Health Working Group Final Report https://www.markle.org/publications/1429-personal-health-working-group-final-report . [Google Scholar]
- 7.Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc. 2006 Mar;13(2):121–6. doi: 10.1197/jamia.M2025. http://paperpile.com/b/kOVAHn/jlUbm .M2025 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gheorghiu B, Hagens S. Use and Maturity of Electronic Patient Portals. Stud Health Technol Inform. 2017;234:136–141. [PubMed] [Google Scholar]
- 9.Bates DW, Wells S. Personal health records and health care utilization. JAMA. 2012 Nov 21;308(19):2034–6. doi: 10.1001/jama.2012.68169.1392549 [DOI] [PubMed] [Google Scholar]
- 10.Otte-Trojel T, de Bont A, van de Klundert J, Rundall TG. Characteristics of patient portals developed in the context of health information exchanges: early policy effects of incentives in the meaningful use program in the United States. J Med Internet Res. 2014;16(11):e258. doi: 10.2196/jmir.3698. http://www.jmir.org/2014/11/e258/ v16i11e258 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gerard M, Fossa A, Folcarelli PH, Walker J, Bell SK. What Patients Value About Reading Visit Notes: A Qualitative Inquiry of Patient Experiences With Their Health Information. J Med Internet Res. 2017 Jul 14;19(7):e237. doi: 10.2196/jmir.7212. http://www.jmir.org/2017/7/e237/ v19i7e237 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Black H, Gonzalez R, Priolo C, Schapira MM, Sonnad SS, Hanson CW, Langlotz CP, Howell JT, Apter AJ. True "meaningful use": technology meets both patient and provider needs. Am J Manag Care. 2015 May 01;21(5):e329–37. https://www.ncbi.nlm.nih.gov/pubmed/26167781 .86163 [PubMed] [Google Scholar]
- 13.de Lusignan S, Mold F, Sheikh A, Majeed A, Wyatt JC, Quinn T, Cavill M, Gronlund TA, Franco C, Chauhan U, Blakey H, Kataria N, Barker F, Ellis B, Koczan P, Arvanitis TN, McCarthy M, Jones S, Rafi I. Patients' online access to their electronic health records and linked online services: a systematic interpretative review. BMJ Open. 2014;4(9):e006021. doi: 10.1136/bmjopen-2014-006021. http://bmjopen.bmj.com/cgi/pmidlookup?view=long&pmid=25200561 .bmjopen-2014-006021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Fisher B, Bhavnani V, Winfield M. How patients use access to their full health records: a qualitative study of patients in general practice. J R Soc Med. 2009 Dec;102(12):539–44. doi: 10.1258/jrsm.2009.090328. http://europepmc.org/abstract/MED/19966130 .102/12/539 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Delbanco T, Walker J, Darer JD, Elmore JG, Feldman HJ, Leveille SG, Ralston JD, Ross SE, Vodicka E, Weber VD. Open notes: doctors and patients signing on. Ann Intern Med. 2010 Jul 20;153(2):121–5. doi: 10.7326/0003-4819-153-2-201007200-00008.153/2/121 [DOI] [PubMed] [Google Scholar]
- 16.Schnipper JL, Gandhi TK, Wald JS, Grant RW, Poon EG, Volk LA, Businger A, Williams DH, Siteman E, Buckel L, Middleton B. Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial. J Am Med Inform Assoc. 2012;19(5):728–34. doi: 10.1136/amiajnl-2011-000723. http://jamia.oxfordjournals.org/lookup/pmidlookup?view=long&pmid=22556186 .amiajnl-2011-000723 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Weingart SN, Hamrick HE, Tutkus S, Carbo A, Sands DZ, Tess A, Davis RB, Bates DW, Phillips RS. Medication safety messages for patients via the web portal: the MedCheck intervention. Int J Med Inform. 2008 Mar;77(3):161–8. doi: 10.1016/j.ijmedinf.2007.04.007.S1386-5056(07)00090-1 [DOI] [PubMed] [Google Scholar]
- 18.Shimada SL, Allison JJ, Rosen AK, Feng H, Houston TK. Sustained Use of Patient Portal Features and Improvements in Diabetes Physiological Measures. J Med Internet Res. 2016 Jul 01;18(7):e179. doi: 10.2196/jmir.5663. http://www.jmir.org/2016/7/e179/ v18i7e179 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Irizarry T, DeVito DA, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res. 2015;17(6):e148. doi: 10.2196/jmir.4255. http://www.jmir.org/2015/6/e148/ v17i6e148 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.http://www.raosoft.com/samplesize.html. [2017-05-14]. http://www.raosoft.com/samplesize.html .
- 21.Patel VN, Abramson E, Edwards AM, Cheung MA, Dhopeshwarkar RV, Kaushal R. Consumer attitudes toward personal health records in a beacon community. Am J Manag Care. 2011 Apr;17(4):e104–20. http://www.ajmc.com/pubMed.php?pii=48672 .48672 [PubMed] [Google Scholar]
- 22.Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) J Med Internet Res. 2004 Sep 29;6(3):e34. doi: 10.2196/jmir.6.3.e34. http://www.jmir.org/2004/3/e34/ v6e34 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Hambleton R, Rodgers J. Item Bias Review. Practical Assessment, Research & Evaluation. 1995:1–3. [Google Scholar]
- 24.Pouwer F, van der Ploeg H, Bramsen I. Dwalingen in de methodologie II. Bias door vragenlijsten. Nederlands tijdschrift voor geneeskunde. 1998;142(27):1556–8. [PubMed] [Google Scholar]
- 25.De Roeck D, Slegers K, Stappers P, Standaert A. Infusing digital creativity in design, the low fidelity way. Proceedings of the Chi Sparks; Chi Sparks Nederland; 2011; Arnhem, The Netherlands. 2011. Jun 23, [Google Scholar]
- 26.Gaver B, Dunne T, Pacenti E. Design: Cultural probes. Interactions. 1999 Jan;6(1):21–29. doi: 10.1145/291224.291235. [DOI] [Google Scholar]
- 27.Boehner K, Vertesi J, Sengers P, Dourish P. How HCI interprets the probes. The SIGCHI Conference on Human Factors in Computing Systems; 2007; San Jose, California, USA. 1240789: ACM; 2007. pp. 1077–86. [Google Scholar]
- 28.Vandenberghe B, Slegers K. Anthropomorphism as a Strategy to Engage End-Users in Health Data Ideation. The 9th Nordic Conference on Human-Computer Interaction; 2016; Gothenburg, Sweden. 2971500: ACM; 2016. pp. 1–4. [Google Scholar]
- 29.Hype Stat. [2018-05-06]. Knack.Be - Info https://hypestat.com/info/knack.be .
- 30.Norman G. Likert scales, levels of measurement and the "laws" of statistics. Adv in Health Sci Educ. 2010 Feb 10;15(5):625–632. doi: 10.1007/s10459-010-9222-y. [DOI] [PubMed] [Google Scholar]
- 31.Wakefield BJ, Turvey CL, Nazi KM, Holman JE, Hogan TP, Shimada SL, Kennedy DR. Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis. J Med Internet Res. 2017 Oct 11;19(10):e346. doi: 10.2196/jmir.7638. http://www.jmir.org/2017/10/e346/ v19i10e346 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Alpert JM, Krist AH, Aycock RA, Kreps GL. Applying Multiple Methods to Comprehensively Evaluate a Patient Portal's Effectiveness to Convey Information to Patients. J Med Internet Res. 2016 May;18(5):e112. doi: 10.2196/jmir.5451.v18i5e112 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Price-Haywood EG, Harden-Barrios J, Ulep R, Luo Q. eHealth Literacy: Patient Engagement in Identifying Strategies to Encourage Use of Patient Portals Among Older Adults. Popul Health Manag. 2017 Dec;20(6):486–494. doi: 10.1089/pop.2016.0164. [DOI] [PubMed] [Google Scholar]
- 34.Coughlin SS, Stewart JL, Young L, Heboyan V, De Leo G. Health literacy and patient web portals. Int J Med Inform. 2018 May;113:43–48. doi: 10.1016/j.ijmedinf.2018.02.009.S1386-5056(18)30079-0 [DOI] [PubMed] [Google Scholar]
- 35.Davis S, Roudsari A, Courtney KL. Designing Personal Health Record Technology for Shared Decision Making. Stud Health Technol Inform. 2017;234:75–80. [PubMed] [Google Scholar]
- 36.Cutrona SL, Sreedhara M, Goff SL, Fisher LD, Preusse P, Jackson M, Sundaresan D, Garber LD, Mazor KM. Improving Rates of Influenza Vaccination Through Electronic Health Record Portal Messages, Interactive Voice Recognition Calls and Patient-Enabled Electronic Health Record Updates: Protocol for a Randomized Controlled Trial. JMIR Res Protoc. 2016 May 06;5(2):e56. doi: 10.2196/resprot.5478. http://www.researchprotocols.org/2016/2/e56/ v5i2e56 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Alpert JM, Krist AH, Aycock RA, Kreps GL. Designing User-Centric Patient Portals: Clinician and Patients' Uses and Gratifications. Telemed J E Health. 2017 Dec;23(3):248–253. doi: 10.1089/tmj.2016.0096. http://europepmc.org/abstract/MED/27333468 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Tambuyzer E, Van Audenhove C. Is perceived patient involvement in mental health care associated with satisfaction and empowerment? Health Expect. 2015 Aug;18(4):516–26. doi: 10.1111/hex.12052. http://europepmc.org/abstract/MED/23425015 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.King G, Maxwell J, Karmali A, Hagens S, Pinto M, Williams L, Adamson K. Connecting Families to Their Health Record and Care Team: The Use, Utility, and Impact of a Client/Family Health Portal at a Children's Rehabilitation Hospital. J Med Internet Res. 2017 Apr 06;19(4):e97. doi: 10.2196/jmir.6811. http://www.jmir.org/2017/4/e97/ v19i4e97 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Delbanco T, Walker J, Bell SK, Darer JD, Elmore JG, Farag N, Feldman HJ, Mejilla R, Ngo L, Ralston JD, Ross SE, Trivedi N, Vodicka E, Leveille SG. Inviting patients to read their doctors' notes: a quasi-experimental study and a look ahead. Ann Intern Med. 2012 Oct 2;157(7):461–70. doi: 10.7326/0003-4819-157-7-201210020-00002. http://europepmc.org/abstract/MED/23027317 .1363511 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.de Lusignan S, Ross P, Shifrin M, Hercigonja-Szekeres M, Seroussi B. A comparison of approaches to providing patients access to summary care records across old and new europe: an exploration of facilitators and barriers to implementation. Stud Health Technol Inform. 2013;192:397–401. [PubMed] [Google Scholar]
- 42.Vodicka E, Mejilla R, Leveille SG, Ralston JD, Darer JD, Delbanco T, Walker J, Elmore JG. Online access to doctors' notes: patient concerns about privacy. J Med Internet Res. 2013;15(9):e208. doi: 10.2196/jmir.2670. http://www.jmir.org/2013/9/e208/ v15i9e208 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Clark SJ, Costello LE, Gebremariam A, Dombkowski KJ. A national survey of parent perspectives on use of patient portals for their children's health care. Appl Clin Inform. 2015;6(1):110–9. doi: 10.4338/ACI-2014-10-RA-0098. http://europepmc.org/abstract/MED/25848417 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Ryan BL, Brown JB, Terry A, Cejic S, Stewart M, Thind A. Implementing and Using a Patient Portal: A qualitative exploration of patient and provider perspectives on engaging patients. J Innov Health Inform. 2016 Jul 04;23(2):848. doi: 10.14236/jhi.v23i2.848. [DOI] [PubMed] [Google Scholar]
- 45.Otte-Trojel T, de Bont A, Rundall TG, van de Klundert J. What do we know about developing patient portals? a systematic literature review. J Am Med Inform Assoc. 2016 Apr;23(e1):e162–8. doi: 10.1093/jamia/ocv114.ocv114 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Ronda MCM, Dijkhorst-Oei L, Rutten GEHM. Patients' Experiences with and Attitudes towards a Diabetes Patient Web Portal. PLoS One. 2015 Jun;10(6):e0129403. doi: 10.1371/journal.pone.0129403. http://dx.plos.org/10.1371/journal.pone.0129403 .PONE-D-15-00899 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Lazard AJ, Watkins I, Mackert MS, Xie B, Stephens KK, Shalev H. Design simplicity influences patient portal use: the role of aesthetic evaluations for technology acceptance. J Am Med Inform Assoc. 2016 Apr;23(e1):e157–61. doi: 10.1093/jamia/ocv174.ocv174 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Coughlin SS, Prochaska JJ, Williams LB, Besenyi GM, Heboyan V, Goggans DS, Yoo W, De Leo G. Patient web portals, disease management, and primary prevention. Risk Manag Healthc Policy. 2017;10:33–40. doi: 10.2147/RMHP.S130431. doi: 10.2147/RMHP.S130431.rmhp-10-033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Graetz I, Gordon N, Fung V, Hamity C, Reed ME. The Digital Divide and Patient Portals: Internet Access Explained Differences in Patient Portal Use for Secure Messaging by Age, Race, and Income. Med Care. 2016 Aug;54(8):772–9. doi: 10.1097/MLR.0000000000000560. [DOI] [PubMed] [Google Scholar]
- 50.Moksony F. Small is beautiful. The use and interpretation of R 2 in social research. Szociologiai Szemle. 1999:130–8. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Survey part 1 and part 2.
Survey part 1 and part 2, Dutch translation.
The Checklist for Reporting Results of Internet E-Surveys (CHERRIES).
Qualitative study that forms the basis for the second part of the survey.
Predictor variables for regression models.