Abstract
Background
General practices (GPs) in England have recently introduced a nationwide electronic personal health record (ePHR) system called Patient Online or GP online services, which allows patients to view parts of their medical records, book appointments, and request prescription refills. Although this system is free of charge, its adoption rates are low. To improve patients’ adoption and implementation success of the system, it is important to understand the factors affecting their use of the system.
Objective
The aim of this study is to explore patients’ perspectives of factors affecting their use of ePHRs in England.
Methods
A cross-sectional survey was carried out between August 21 and September 26, 2017. A questionnaire was used in this survey to collect mainly quantitative data through closed-ended questions in addition to qualitative data through an open-ended question. A convenience sample was recruited in 4 GPs in West Yorkshire, England. Given that the quantitative data were analyzed in a previous study, we analyzed the qualitative data using thematic analysis.
Results
Of the 800 eligible patients invited to participate in the survey, 624 (78.0%) returned a fully completed questionnaire. Of those returned questionnaires, the open-ended question was answered by 136/624 (21.8%) participants. A total of 2 meta-themes emerged from participants’ responses. The first meta-theme comprises 5 themes about why patients do not use Patient Online: concerns about using Patient Online, lack of awareness of Patient Online, challenges regarding internet and computers, perceived characteristics of nonusers, and preference for personal contact. The second meta-theme contains 1 theme about why patients use Patient Online: encouraging features of Patient Online.
Conclusions
The challenges and concerns that impede the use of Patient Online seem to be of greater importance than the facilitators that encourage its use. There are practical considerations that, if incorporated into the system, are likely to improve its adoption rate: Patient Online should be useful, easy to use, secure, and easy to access. Different channels should be used to increase the awareness of the system, and GPs should ease registration with the system and provide manuals, training sessions, and technical support. More research is needed to assess the effect of the new factors found in this study (eg, lack of trust, difficulty registering with Patient Online) and factors affecting the continuing use of the system.
Keywords: electronic personal health records, tethered personal health records, patient portal, patient online, technology acceptance, technology adoption, qualitative research, mobile phone
Introduction
Background
Over the past 2 decades, there has been a rapid and widespread diffusion of electronic personal health records (ePHRs) in health care institutes [1]. The Markle Foundation defines ePHRs as web-based portals that enable users to access their medical records stored by their health care providers [2]. Other services can be added to ePHRs, such as booking appointments, requesting referrals, messaging health care providers, requesting medication refills, and educational materials [3,4]. Several benefits may be gained from using ePHRs, such as empowering patients [5,6], increasing their adherence to medication [7,8], improving their self-management [8,9], enhancing patient-provider relationships and communications [10,11], decreasing adverse events and allergic reactions [11,12], and avoiding duplicated tests [11,12].
General practices (GPs) in England started implementing ePHRs in 2003 when patients were enabled to access their full records through kiosks installed in some GPs. These kiosks allow patients to check their demographic information, consultations, test results, letters, and allergies [13].
In 2007, the National Health Service (NHS) offered patients in England access to their Summary Care Records (SCR) through HealthSpace [14-16]. HealthSpace is a secure web-based personal health record that has several functions: booking or canceling hospital appointments, recording and charting health indicators (eg, vital signs, weight, peak flow), calendar with email reminders, NHS address book, links to educational sources, secure messaging, and access to the SCR [15,17]. The SCR is a summary of key health information (allergies, adverse reactions, current medications, and main diagnoses) extracted from patient electronic medical records held by their general practitioners, and it is stored centrally and accessible by authorized NHS staff in urgent situations [14,16]. Because of the low adoption rate and technical issues, HealthSpace was shut down in December 2012 [18].
In 2015, the NHS implemented ePHRs under a program called Patient Online or GP online services, which enables users to book appointments, request prescription refills, and access coded information in their medical records such as demographics, medications, allergies, test results, problems list, immunizations, and medical and surgical procedures [19]. Currently, it is the largest ePHR in England, given that it has been implemented in more than 99% of GPs [19]. As the system is provided by different companies, it is called by different names such as Patient Access, Patient Services, The Waiting Room, and SystemOnline [19]. GP online services have been introduced in the United Kingdom at a time when funding for the NHS is under pressure. Given the context of austerity, individual practices have limited resources to support the rollout of GP online services.
Research Problem and Aim
Despite the potential benefits of ePHRs, their adoption rate in England was only 28% by the end of June 2019 [20]. Identifying the factors affecting patients’ use of ePHRs is important to improve patients’ adoption and the implementation success of ePHRs [21-25]. A systematic review of 97 studies found that factors affecting patients’ use of ePHRs in England have not been examined, and there is a lack of qualitative studies (8%) in this topic [26]. Accordingly, this study aimed to explore patients’ perspectives of factors affecting their use of ePHRs (Patient Online) in England.
Methods
Data Collection
A cross-sectional survey was conducted between August 21 and September 26, 2017. In this study, a self-administered questionnaire was used to collect quantitative data through closed-ended questions and qualitative data through an open-ended question (Multimedia Appendix 1). The qualitative data provide the focus of this study. Note that the findings from the quantitative analysis of the survey data were presented in a previous paper [27]. The survey gained health research authority approval before starting data collection (The Research Ethics Committee reference number: 17/SC/0323).
Sample
A convenience sample of patients was recruited from 4 GPs in West Yorkshire, England. Patients were eligible to participate if they (1) lived in England and were registered at 1 of the 4 GPs, (2) were aged 18 years or older, and (3) had not used Patient Online before (nonusers).
Analysis
The qualitative data were analyzed using thematic analysis. Given the exploratory nature of this study, an inductive approach was used to generate themes directly from the data [28]. The analysis was performed following the steps proposed by Braun and Clarke [29]: (1) familiarizing with the data through scrutinizing and rescrutinizing the transcript; (2) coding data systematically; (3) generating subthemes and themes from codes; (4) checking the fit of those themes and subthemes to the original utterances and drawing an initial thematic map; (5) refining and regrouping some inappropriate codes and generating meta-themes from the themes for more granular grouping; and finally, (6) defining and naming subthemes, themes, and meta-themes. We followed the guidelines of Braun and Clarke, as these are considered the most systematic guide for conducting thematic analysis to date [30,31]. The analysis was carried out by the first author (AA), and the validity of codes and themes was checked by another author (BB). AA and BB discussed codes and themes. Where AA and BB had differing views on the code labels and/or thematic content, these discrepancies were resolved through discussion. In all cases, agreement was reached between AA and BB. Microsoft Excel was used to manage the analysis process.
Results
Collected Data
Out of the 800 eligible patients invited to participate in the survey, 624 (78%) participants completed the questionnaire. Of those participants, 136 (21.8%) answered the open-ended question. The 136 comments contained 221 utterances. A comment refers to the whole text written by a participant as a reply to our question, whereas an utterance refers to a part of the comment that has one idea or thought. In total, 3 of the 221 utterances were excluded because 2 utterances were illegible and the meaning of 1 utterance was not discernible. The final number of utterances included in the thematic analyses was 218. The excluded utterances were all part of longer comments, and for that reason, the final number of comments remained 136. Subsection 3.1 summarizes the characteristics of the respondents, and Subsection 3.2 presents the findings of the thematic analysis.
Participants’ Characteristics
Table 1 summarizes the characteristics of the participants who answered the open-ended question and those who did not. Those who responded to the question had a mean age of 43.7 years (SD 18.3). More female participants answered the question than male participants (80/136, 58.8% were females). The majority of the respondents had a White ethnicity (107/136, 78.7%), had an income of less than US $40,000 per year (95/136, 69.8%), and had access to the internet (112/136, 82.4%). In terms of education, 39.7% (54/136) of the respondents had a bachelor’s degree or higher.
Table 1.
Characteristics | Value, n (%) | |
Age (years), mean (SD) | 43.7 (18.3) | |
|
18-24 | 19 (14.0) |
|
25-34 | 35 (25.7) |
|
35-44 | 23 (16.9) |
|
45-54 | 20 (14.7) |
|
55-64 | 17 (12.5) |
|
65-74 | 12 (8.8) |
|
≥75 | 10 (7.4) |
Sex | ||
|
Male | 56 (41.2) |
|
Female | 80 (58.8) |
Ethnicity | ||
|
White | 107 (78.7) |
|
Asian | 14 (10.3) |
|
Black | 6 (4.4) |
|
Mixed | 7 (5.1) |
|
Others | 2 (1.5) |
Income (US $) | ||
|
<20,000 | 55 (40.4) |
|
20,000-29,999 | 24 (17.6) |
|
30,000-39,999 | 16 (11.8) |
|
40,000-49,999 | 9 (6.6) |
|
50,000-59,999 | 6 (4.4) |
|
60,000 or more | 5 (3.8) |
|
Prefer not to say | 21 (15.4) |
Education | ||
|
Up to secondary school | 13 (9.6) |
|
Secondary school | 31 (22.8) |
|
College/Diploma | 38 (27.9) |
|
Bachelor’s degree | 38 (27.9) |
|
Master’s degree | 10 (7.4) |
|
Doctoral degree | 6 (4.4) |
Internet access |
|
|
|
Yes | 112 (82.4) |
|
No | 24 (17.6) |
Findings of Thematic Analysis
In total, 2 meta-themes were generated as a result of the thematic analysis. The first meta-theme consists of 5 themes and relates to utterances explaining why patients do not use Patient Online (Figure 1). The second meta-theme pertains to utterances about why patients use Patient Online, and it contains 1 theme: encouraging features of Patient Online. The following sections contain more details about all 6 themes.
Theme 1: Concerns About Using Patient Online
The first theme, concerns about using Patient Online, is made up of 7 subthemes: (1) concerns about privacy and security, (2) difficulty accessing Patient Online, (3) difficulty using Patient Online, (4) lack of trust in Patient Online, (5) difficulty registering, (6) technical concerns, and (7) the inability of Patient Online to save money and time (Figure 1).
The security and privacy of Patient Online was a major concern for respondents. Their concerns were attributed to the recent NHS hack attacks, worries that their data will be accessed by third parties, and uncertainty about the security measures of Patient Online:
I believe that Patient Online has/ will have too many privacy issues, look what happened when the NHS was hacked.
Participant #9
Only concern is confidentiality of System One as I am aware CIA [Central Intelligence Agency] are now using the system.
Participant #30
The second subtheme shows that difficulty accessing (logging in) the system can be a barrier to its use. The main reasons given for difficulty accessing Patient Online were the inability to find its URL link and forgetting passwords and log-in details:
I tried to use the system but I can never find the correct link...
Participant #120
...I always forget my password.
Participant #35
The third subtheme was generated from comments about difficulty using Patient Online. Although the previous subtheme reflects patients’ concerns about logging on to Patient Online, this subtheme represents their worries about using the system after logging in to it (ie, ordering prescriptions, managing appointments, checking their records). According to some respondents, these concerns are exacerbated when nobody can help in using the system:
I don’t know if this would be easy to use.
Participant #5
If people experience a difficulty and do not know where to find help, or who to ask, they may give up trying.
Participant #49
The fourth subtheme indicates that some patients did not trust Patient Online to do what they want it to do. They doubted that an appointment would actually be booked for them if they booked via Patient Online:
...I don’t trust the service.
Participant #9
...I am not sure I would entirely trust it...
Participant #123
Concerns with difficulty registering with Patient Online were brought together to form the fifth subtheme. These concerns were attributed to the fact that they have to visit the practice in person with their ID to be able to register to use the system. To ease the registration process, a respondent suggested that the registration to Patient Online becomes part of the patient registration in practice:
You also have to make a trip to the surgery with ID to be able to use the service.
Participant #28
I think more effort should be made to encourage patients to sign up for this, and the process should be more streamlined-perhaps done as a matter of course when registering.
Participant #7
The sixth subtheme encompasses utterances that show concern regarding the technical difficulties of Patient Online. Technical issues here refer to technical errors that people believe they will face when using Patient Online:
Technology goes wrong and does not tell you why.
Participant #58
The last subtheme brought together utterances from some respondents who were worried about the inability of Patient Online to save money and time. This is reflected in the utterances of the seventh subtheme, that is, respondents, especially those who live near the practice doubted that using Patient Online saves money and time:
In my experience many of these things do not end up saving people’s time and money. So I don’t think I’ll be using this except infrequently.
Participant #38
It would not save travel costs because I live next to it.
Participant #85
Theme 2: Issues About Awareness of Patient Online
The utterances in this theme suggest that if respondents had more knowledge or awareness about Patient Online, they would use it. This theme consists of 2 subthemes: lack of awareness of Patient Online and advertising about Patient Online. In the first subtheme, the respondents stated that the lack of knowledge about the system’s presence, what it is about, how to use, and how to access it was the main reason for not using it:
To be honest, I’ve never heard of Patient Online before and that may be why people haven’t used it.
Participant #88
Not been shown what it is about and how to use it.
Participant #80
In the second subtheme, several respondents attributed their lack of awareness of Patient Online to the lack of advertisement about it. For this reason, they acknowledged the essential role of the publicity of Patient Online in increasing people’s awareness of it:
It is not openly advertised in the surgery that Patient Online is available.
Participant #28
...may not be enough advertisement.
Participant #62
Theme 3: Challenges Regarding Internet and Computers
The third theme refers to issues regarding prerequisites for using Patient Online (ie, a computer and internet access). Respondents identified 3 challenges regarding the internet and computers, which form the 3 subthemes within this theme. The first challenge is the lack of internet or computer access. Many respondents attributed the nonuse of Patient Online to not having internet or computer access:
Those who don’t have access to the internet may not use it.
Participant #57
Although many respondents have access to the internet and computers, they have limited skills in using them, and this is the second challenge:
I can’t use a computer so I can’t use Patient Online.
Participant #2
The third challenge is the lack of use of internet or computers. This subtheme indicates that some users may have access to computers and internet and the required skills but do not frequently use them:
I do not use computers of any kind.
Participant #75
Theme 4: Perceived Characteristics of Nonusers
The fourth theme was generated from utterances about who is less likely to use Patient Online. The 3 main characteristics of nonusers were related to age, use of GP services, and income. These characteristics formed 3 subthemes, in addition to an extra subtheme that encompasses infrequently reported characteristics.
Age was the most commonly reported characteristic of nonusers. Respondents suspected the ability of older people to use Patient Online for different reasons: lack of computer and internet skills, lack of internet access, lack of awareness of how to use the system, lack of confidence in using it, lack of technology use, and their preference for face-to-face contact:
Elderly people may have no understanding or knowledge of how to use a computer or the internet.
Participant #69
Older people may not use it as they don’t have access to internet or know how to use services.
Participant #116
In the second subtheme, respondents attributed the nonuse of the system to lack of use of GP services in general, such as consultations and medications:
I’ve never used it as it’s rare that I attend the surgery and I’m not on any medication.
Participant #132
Low income formed the third subtheme. Respondents stated that people need enough income to have internet access or get training to be able to use computers and the internet:
I do not have enough income/benefits...
Participant #20
The last subtheme encompasses characteristics of other people who are more likely to be nonusers of Patient Online and those who live near the practice, illiterate people, people who cannot read in English, and people who forget to use Patient Online:
I would use Patient Online more often if I lived further away from the surgery.
Participant #15
I am not good at reading or spelling so online would not be good for me.
Participant #70
Theme 5: Preferring Personal Contact
Preferring personal contact was identified as the main justification for not using Patient Online. Patients prefer personal contact because they think it is more reliable, easier, provides an instant reply, and is important in urgent conditions:
It is more reliable to speak to someone directly about their medical records rather than using online.
Participant #29
Picking up the phone and speaking to someone is easier.
Participant #135
Theme 6: Encouraging Features of Patient Online
Within this theme, respondents identified features of Patient Online that may encourage them to use the system. One of the main features of Patient Online is that it is useful for different people, such as students, people with mobility needs, people who cannot reach the practice, and busy people:
I feel that it would be particularly useful for students.
Participant #63
Another feature mentioned by respondents is ease of access. Some respondents thought that Patient Online could be more accessible if it was a mobile app. It is noteworthy that mobile apps were not available for accessing GP online services at the time of data collection. Later, patients have been enabled to access GP online services via a mobile app called NHS App [32]:
A mobile application would be more accessible...
Participant #95
Respondents reported other features of Patient Online, which may encourage people to use it, namely, secure, quick, user-friendly interface, convenient, and less stressful:
If it is secure and fast then people will use it, I suppose.
Participant #68
If the interface is not user-friendly people might not be encouraged to use Patient Online.
Participant #82
Discussion
Principal Findings
The aim of this study is to explore why patients in England choose to use ePHRs. Participants identified one leading cause that encouraged them to use Patient Online, which relates to its features being useful, easy to access, secure, quick, user-friendly interface, convenient, and less stressful. However, patients identified many reasons for not using Patient Online, which were categorized into 5 themes: concerns about using Patient Online, lack of awareness of Patient Online, challenges regarding internet and computers, perceived characteristics of nonusers, and preference for personal contact.
In the first theme, concerns about using Patient Online, the most prominent reason for not using Patient Online was privacy and security concerns. This may be attributed to the fact that ePHRs typically contain personal and sensitive information, and patients have previously been shown to be concerned about the accessibility of these data [33]. The hack attacks that happened to the NHS 4 months before data collection may have exacerbated these concerns in this sample. This finding is consistent with the results of the quantitative data in the original study [27], where perceived privacy and security significantly affected patients’ intention to use Patient Online. This factor was also found in other quantitative studies [33-36] and qualitative studies [37-43].
Participants also raised their concerns about difficulty logging on to Patient Online because of losing its URL and forgetting passwords and log-in details. This issue posed a challenge for patients because they were given new complex passwords and usernames to access Patient Online. Although passwords can be changed through the system, usernames are fixed. This effect of difficulty accessing the system has also been demonstrated in several studies [41,44,45].
Other worries were reported about difficulty using Patient Online, especially when there is no one to help. This may be attributed to the fact that patients need adequate computer and internet skills to use Patient Online. They may also need to access it without any help from others to protect their privacy. This factor was also found in quantitative analysis in the original study [27], where effort expectancy (ie, ease of use) and behavioral intentions were significantly associated. Furthermore, numerous quantitative and qualitative studies have shown similar findings regarding this factor [37,38,40,41,45-47].
Participants expressed their concerns about the difficulty they experienced registering with Patient Online. Indeed, it could be argued that the process of registration with Patient Online is less flexible than several systems (eg, MyChart, PatientSite, My Health Manager, My Health at Vanderbilt), where patients can register with the system using email, websites, or phone and with no need to visit the practice. To the best of our knowledge, this factor was not found in previous studies. This may be because of the ease of registration with other systems.
The inability of Patient Online to save money and time was a concern for some participants, especially those who live near the practice. This concern may have made patients feel that Patient Online is not useful for them. Thus, this factor is related to perceived usefulness, which was the most influential factor according to the quantitative analysis in the original study [27]. The effect of this factor was also demonstrated by quantitative studies [46-49] and other qualitative studies [37,40,41,45,50].
Finally, 2 further concerns in this group were raised by participants, a lack of trust in Patient Online to book appointments or request medication refills and the technical issues that some patients reported when using Patient Online. To the best of our knowledge, neither of these factors have been reported in previous studies.
In the second theme, lack of awareness of different aspects of Patient Online was an influential factor in not using the system. Lack of advertising about Patient Online was the main reason for this lack of awareness. Although 3 of the 4 GPs had advertisements about Patient Online visibly displayed on screens or brochures in the waiting room during the study, some patients still reported a lack of awareness of the system. This factor was in line with the findings of previous quantitative studies [51,52] and qualitative studies [37,41,45,53].
With regard to the third theme, 3 challenges related to computers and the internet were identified. The first is the lack of internet or computer access. This factor was represented by the construct facilitating condition in the quantitative analysis, and it was found to significantly affect the actual use of Patient Online [27]. Furthermore, previous studies have shown a significant deleterious effect of a lack of internet [54-59] and computer access [38,46,52,57].
The limited skills in using the internet or computers was the second challenge in this group. This challenge may have produced reports that patients found Patient Online difficult to use. Hence, this factor is related to perceived ease of use (ie, effort expectancy), which was the most influential factor according to the quantitative analysis in the original study [19]. Numerous studies have supported this effect of computer literacy [38,40,41,53,60] and internet literacy [61,62].
The last challenge was the lack of using internet or computers. This challenge may also be related to perceived ease of use, as those who rarely use computers and the internet may perceive the system difficult to use. Several previous studies showed similar findings regarding the effect of lack of computer use [43,46,62] and internet use on the adoption of ePHRs [39,43,54,63-65].
Regarding the fourth theme, participants determined the following characteristics of nonusers of Patient Online, which were consistent with findings of previous studies: older people [61,66-69], who rarely use GP services [55,66,68,70,71], who have low income [46,52,72,73], who live near the practice [70], and who have lower literacy levels [46,52,72,73].
In the last theme, participants justified their nonuse of the system by indicating their preference for personal contact with their GP. This was attributed to the perceived advantages of personal contact over the system. This factor was found in other studies [40,51,52].
Strengths
This study enabled us to explore new factors that were not examined by the quantitative part of the study (eg, lack of awareness) and previous studies (eg, lack of trust). Furthermore, this study allowed us, to some extent, to support and explain some relationships proposed in the quantitative study (eg, performance expectancy, perceived privacy, security).
To the best of our knowledge, this study had the largest sample size in comparison with all qualitative studies on this topic. This allowed us to explore a wide range of patients’ perspectives on the adoption of ePHRs.
Limitations
This study collected data from 4 GPs implementing the same ePHR (ie, SystemOnline), which may limit the generalizability of this study to other practices implementing other ePHRs (ie, Patient Access, Patient Services, The Waiting Room, Engage Consult, and Evergreen Life/i-Patient). However, it should be noted that all these systems provide the same services to the patients (ie, booking appointments, requesting prescription refills, and viewing health records), and no participant had used any of them before. As a result, the participants in this study were unlikely to have made comparisons between the different systems.
Although the qualitative data collected by an open-ended question helped in exploring factors affecting patients’ use of Patient Online, such data may not be equivalent to qualitative data collected by interviews or focus groups. Thus, we could not deeply understand the adoption process of Patient Online. However, this qualitative analysis did not aim to understand in depth the phenomenon of interest; rather, it aimed only to help in identifying other factors not included in the model and explaining the findings of the quantitative study. As answering the open-ended question was voluntary, there may be an element of self-selection.
As the open-ended question was put after closed-ended questions, participants’ answers to the open-ended question may be influenced by this order. This order was based on researchers’ recommendations that questionnaires should start with the most interesting and easy-to-answer questions, and open-ended and demographic questions should be presented at the end of the questionnaire [74-76].
Practical Implications
We believe that adoption of GP online services will significantly increase in the future, given that many factors identified in this study will be automatically and considerably mitigated by time. Specifically, the proportion of patients who are more comfortable with the use of computers, smartphones, electronic systems, and the internet will increase in the future given their increased spread over the world. Thus, these services may be desired and expected by patients. However, developers, marketers, and GPs still play a crucial role in increasing the adoption of GP online services.
During system development, patients should be involved in the process to identify the features that make the system useful and easy to use. Some participants pointed out that the system will be useful when it allows them to book walk-in appointments, communicate with their doctors, and select the required doctor. As Patient Online currently enables patients to choose the required doctor, developers should consider adding these services, which are provided by many ePHRs (eg, MyChart, MyHealtheVet, Patient Gateway) [46,66,77]. Furthermore, users of such systems should be informed and reassured about the different security measures that are in place (eg, strong firewalls, encouragement to use complex and long passwords), and it should be made clear that the provision of GP online services is strictly controlled by legislation to safeguard personal data. To ease logging on to the system, developers should develop a system that allows patients to access it through their fingerprints or face recognitions, instead of using complex usernames and passwords. It is noteworthy that the NHS App, which has been recently developed, is the only system that enables patients to access GP online services using fingerprints or face recognitions [32].
To increase the awareness of the system, its functionality, and its benefits, marketers should improve their publicity through different channels, such as public media (eg, television, radio, newspapers, magazines), social media (eg, Facebook, Twitter, YouTube), emails, mails, automated messages on the practices’ telephone system, and advertisements in general public areas (eg, shopping centers, health care settings, highway streets, universities). Face-to-face communication is considered as one of the most effective channels in marketing to persuade potential adopters to adopt an innovation [78,79]. Thus, all staff in practice (eg, physicians, nurses, receptionists) should offer the system to patients during their visits. GP staff may not be keen on publicizing online services because of a lack of incentives and time. Therefore, consideration should be given to providing incentives and resources for GPs to increase patients’ awareness of GP online services.
Although patients have been recently enabled to sign up in the GP online services without visiting their surgeries through only the NHS App [80], they still need to visit their surgeries in person to register to use GP online services provided by other systems (eg, SystemOnline, Patient Access). To ease signing up in these systems, GPs should allow patients to register on web or through phone and make the signing up procedure a part of patient registration in the practice. GPs may enhance patients’ perceptions of usefulness, ease of use of the system, and their trust in it by helping them in using a beta version of the system through a computer in a waiting room. GPs should provide online assistance, technical support, manuals, and training to allow patients to solve any technical issues that face them when using the system, thereby decreasing their technical concerns. GPs should collaborate with other parties (eg, Patient Online providers and government bodies) to provide computers and/or internet access at affordable prices for those who do not have them and cannot afford them. Given that many UK GPs report being overstretched and limited funding has been provided to support the rollout of GP online services, consideration should be given to providing incentive programs (eg, Meaningful Use policy as issued by the US government). Incentive programs could be used to encourage GPs to publicize their online services and encourage patients to use them.
Recommendations for Future Research
As this study could not provide a deep understanding of the adoption process of Patient Online, a deeper understanding of the adoption of online services could be gained through further qualitative work using interviews or focus groups. Several factors were revealed in this analysis but were not part of the conceptual model in the quantitative study, namely, awareness of Patient Online, lack of trust in the system, difficulty registering, disability, lack of use of GP services, and distance to the GPs. Future studies should consider adding these factors to the model and quantitatively examine them. Finally, more research is needed to identify the factors affecting the continuing use, as long-term viability and eventual success of information technology count on its continuing use more than initial use [81-83].
Conclusions
This research explored patients’ perspectives regarding factors influencing their use of Patient Online. We found about 20 factors grouped into 6 themes. The findings of this study supported the findings of the quantitative study (eg, performance expectancy, effort expectancy, perceived privacy). This study found new factors that were not examined by the quantitative part of the study (eg, lack of awareness) and previous studies (eg, lack of trust).
The challenges and concerns that impede the use of Patient Online seem to be greater than the facilitators that encourage its use. To foster use, several practical implications were suggested: Patient Online should be useful, easy to use, secure, and easy to access; different channels should be used to increase the awareness of the system; and GPs should ease registration with the system and provide manuals, training sessions, and technical support. More research is needed to quantitatively assess the effect of the new factors found in this study (eg, lack of trust, difficulty registering with Patient Online) and factors affecting continuing use of the system.
Acknowledgments
The authors would like to thank Dr Hamish Fraser for his help in the initial stages of the study, especially for his contribution to the development of the study design and setting up of links with practices. The authors would also like to thank Tracey Farragher for her help in assessing the nonresponse bias. The publication of this study was funded by the Qatar National Library.
Abbreviations
- ePHR
electronic personal health record
- GP
general practice
- NHS
National Health Service
- SCR
summary care record
Appendix
Questionnaire.
Footnotes
Authors' Contributions: AA and PG developed the protocol. AA collected the data. AA analyzed the data under BB’s guidance and supervision. AA and ZS drafted the manuscript, and it was revised critically for important intellectual content by all authors. All authors approved the manuscript for publication and agree to be accountable for all aspects of the work.
Conflicts of Interest: None declared.
References
- 1.Wade-Vuturo AE, Mayberry LS, Osborn CY. Secure messaging and diabetes management: experiences and perspectives of patient portal users. J Am Med Inform Assoc. 2013 May 1;20(3):519–25. doi: 10.1136/amiajnl-2012-001253. http://europepmc.org/abstract/MED/23242764. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.The Personal Health Working Group: Final Report. Markle Foundation. 2003. [2020-12-21]. https://www.markle.org/sites/default/files/final_phwg_report1.pdf.
- 3.Nazi KM, Hogan TP, McInnes DK, Woods SS, Graham G. Evaluating patient access to electronic health records. Med Care. 2013;51:S52–6. doi: 10.1097/mlr.0b013e31827808db. [DOI] [PubMed] [Google Scholar]
- 4.Pai H, Lau F, Barnett J, Jones S. Meeting the health information needs of prostate cancer patients using personal health records. Curr Oncol. 2013 Dec;20(6):e561–9. doi: 10.3747/co.20.1584. http://www.current-oncology.com/index.php/oncology/article/view/1584/1338. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ertmer A, Uckert F. User acceptance of and satisfaction with a personal electronic health record. Stud Health Technol Inform. 2005;116:861–6. [PubMed] [Google Scholar]
- 6.Laranjo L, Rodolfo I, Pereira AM, de Sá AB. Characteristics of innovators adopting a national personal health record in Portugal: cross-sectional study. JMIR Med Inform. 2017 Oct 11;5(4):e37. doi: 10.2196/medinform.7887. https://medinform.jmir.org/2017/4/e37/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rice R. Healthcare Leaders' Lived Experiences Regarding the Implementation of Electronic Personal Health Records. New York, USA: Walden University; 2014. [Google Scholar]
- 8.Vermeir P, Degroote S, Vandijck D, Van Tiggelen H, Peleman R, Verhaeghe R, Mariman A, Vogelaers D. The patient perspective on the effects of medical record accessibility: a systematic review. Acta Clin Belg. 2017 Jun;72(3):186–94. doi: 10.1080/17843286.2016.1275375. [DOI] [PubMed] [Google Scholar]
- 9.Smith AB, Odlum M, Sikka M, Bakken S, Kanter T. Patient perceptions of pre-implementation of personal health records (PHRS): a qualitative study of people living with HIV in New York city. J HIV/AIDS Soc Serv. 2012 Oct;11(4):406–23. doi: 10.1080/15381501.2012.735166. [DOI] [Google Scholar]
- 10.Ochoa IA, Kitayama K, Uijtdehaage S, Vermillion M, Eaton M, Carpio F, Serota M, Hochman ME. Patient and provider perspectives on the potential value and use of a bilingual online patient portal in a Spanish-speaking safety-net population. J Am Med Inform Assoc. 2017 Nov 1;24(6):1160–4. doi: 10.1093/jamia/ocx040. http://europepmc.org/abstract/MED/28460130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Alyami M. Removing Barriers in Using Personal Health Record Systems. 15th International Conference on Computer and Information Science; ICIS'16; 25 August 2016; Okayama, Japan. 2016. [DOI] [Google Scholar]
- 12.Morton A. Examining Acceptance of an Integrated Personal Health Record (PHR) New York, USA: University of Maryland, Baltimore; 2011. [Google Scholar]
- 13.Fisher B, Fitton R, Poirier C, Stables D. Patient record access--the time has come! Br J Gen Pract. 2007 Jun;57(539):507–11. https://bjgp.org/cgi/pmidlookup?view=long&pmid=17550687. [PMC free article] [PubMed] [Google Scholar]
- 14.Kahn JS, Aulakh V, Bosworth A. What it takes: characteristics of the ideal personal health record. Health Aff (Millwood) 2009;28(2):369–76. doi: 10.1377/hlthaff.28.2.369. [DOI] [PubMed] [Google Scholar]
- 15.Pagliari C, Detmer D, Singleton P. Potential of electronic personal health records. Br Med J. 2007 Aug 18;335(7615):330–3. doi: 10.1136/bmj.39279.482963.AD. http://europepmc.org/abstract/MED/17703042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Greenhalgh T, Wood GW, Bratan T, Stramer K, Hinder S. Patients' attitudes to the summary care record and HealthSpace: qualitative study. Br Med J. 2008 Jun 7;336(7656):1290–5. doi: 10.1136/bmj.a114. http://europepmc.org/abstract/MED/18511764. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Greenhalgh T, Hinder S, Stramer K, Bratan T, Russell J. Adoption, non-adoption, and abandonment of a personal electronic health record: case study of HealthSpace. Br Med J. 2010 Nov 16;341:c5814. doi: 10.1136/bmj.c5814. http://europepmc.org/abstract/MED/21081595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.de Lusignan S, Ross P, Shifrin M, Hercigonja-Szekeres M, Seroussi B. A comparison of approaches to providing patients access to summary care records across old and new europe: an exploration of facilitators and barriers to implementation. Stud Health Technol Inform. 2013;192:397–401. [PubMed] [Google Scholar]
- 19.Start Using GP Online Services. NHS. 2016. [2020-12-21]. https://www.nhs.uk/using-the-nhs/nhs-services/gps/gp-online-services/
- 20.Patient Online Management Information (POMI) TinyURL. 2019. [2020-12-21]. https://tinyurl.com/ybptnhsu.
- 21.Fung V, Ortiz E, Huang J, Fireman B, Miller R, Selby JV, Hsu J. Early experiences with e-health services (1999-2002): promise, reality, and implications. Med Care. 2006 May;44(5):491–6. doi: 10.1097/01.mlr.0000207917.28988.dd. [DOI] [PubMed] [Google Scholar]
- 22.Huygens MW, Vermeulen J, Friele RD, van Schayck OC, de Jong JD, de Witte LP. Internet services for communicating with the general practice: barely noticed and used by patients. Interact J Med Res. 2015 Nov 24;4(4):e21. doi: 10.2196/ijmr.4245. https://www.i-jmr.org/2015/4/e21/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Najaftorkaman M, Ghapanchi A, Talaei-Khoei A. Analysis of Research in Adoption of Person-Centred Healthcare Systems. Open Repository. 2014. [2020-12-30]. https://openrepository.aut.ac.nz/handle/10292/8104.
- 24.Or CK, Karsh B, Severtson DJ, Burke LJ, Brown RL, Brennan PF. Factors affecting home care patients' acceptance of a web-based interactive self-management technology. J Am Med Inform Assoc. 2011;18(1):51–9. doi: 10.1136/jamia.2010.007336. http://europepmc.org/abstract/MED/21131605. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Logue MD, Effken JA. Modeling factors that influence personal health records adoption. Comput Inf Nurs. 2012:1. doi: 10.1097/nxn.0b013e3182510717. [DOI] [PubMed] [Google Scholar]
- 26.Abd-Alrazaq AA, Bewick BM, Farragher T, Gardner P. Factors that affect the use of electronic personal health records among patients: a systematic review. Int J Med Inform. 2019 Jun;126:164–75. doi: 10.1016/j.ijmedinf.2019.03.014. [DOI] [PubMed] [Google Scholar]
- 27.Abd-Alrazaq A, Bewick BM, Farragher T, Gardner P. Factors affecting patients' use of electronic personal health records in England: cross-sectional study. J Med Internet Res. 2019 Jul 31;21(7):e12373. doi: 10.2196/12373. https://www.jmir.org/2019/7/e12373/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Boyatzis R. Thematic Analysis and Code Development. Thousand Oaks, CA: Sage Publications; 1998. [Google Scholar]
- 29.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006 Jan;3(2):77–101. doi: 10.1191/1478088706qp063oa. [DOI] [Google Scholar]
- 30.Howitt D. Introduction to Qualitative Methods in Psychology. London, UK: Pearson; 2016. [Google Scholar]
- 31.Wilson S, MacLean R. Research Methods and Data Analysis for Psychology. New York, USA: McGraw-Hill Education; 2011. [Google Scholar]
- 32.The NHS App. 2020. [2020-12-21]. https://www.nhs.uk/using-the-nhs/nhs-services/the-nhs-app/
- 33.Daglish D. Electronic Personal Health Records: A Matter of Trust. McMaster University. 2013. [2020-12-30]. https://macsphere.mcmaster.ca/handle/11375/13325.
- 34.Rao M. Factors Affecting Health Care Technology Use in Baby Boomers: A Quantitative Study. New York, USA: Northcentral University; 2014. [Google Scholar]
- 35.Whetstone M, Goldsmith R. Factors influencing intention to use personal health records. Intl J of Pharm Health Mrkt. 2009 Apr 3;3(1):8–25. doi: 10.1108/17506120910948485. [DOI] [Google Scholar]
- 36.Ozok AA, Wu H, Gurses AP. Exploring patients’ use intention of personal health record systems: implications for design. Int J Hum-Comput Interact. 2017 Jan 6;33(4):265–79. doi: 10.1080/10447318.2016.1277637. [DOI] [Google Scholar]
- 37.Dontje K, Corser WD, Holzman G. Understanding patient perceptions of the electronic personal health record. J Nurs Pract. 2014 Nov;10(10):824–8. doi: 10.1016/j.nurpra.2014.09.009. [DOI] [Google Scholar]
- 38.Mishuris RG, Stewart M, Fix GM, Marcello T, McInnes DK, Hogan TP, Boardman JB, Simon SR. Barriers to patient portal access among veterans receiving home-based primary care: a qualitative study. Health Expect. 2015 Dec;18(6):2296–305. doi: 10.1111/hex.12199. http://europepmc.org/abstract/MED/24816246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Patel VN, Dhopeshwarkar RV, Edwards A, Barrón Y, Sparenborg J, Kaushal R. Consumer support for health information exchange and personal health records: a regional health information organization survey. J Med Syst. 2012 Jun;36(3):1043–52. doi: 10.1007/s10916-010-9566-0. [DOI] [PubMed] [Google Scholar]
- 40.Tieu L, Sarkar U, Schillinger D, Ralston JD, Ratanawongsa N, Pasick R, Lyles CR. Barriers and facilitators to online portal use among patients and caregivers in a safety net health care system: a qualitative study. J Med Internet Res. 2015 Dec 3;17(12):e275. doi: 10.2196/jmir.4847. https://www.jmir.org/2015/12/e275/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Turner A, Osterhage K, Joe J, Hartzler A, Lin L, Demiris G. Use of patient portals: personal health information management in older adults. Stud Health Technol Inform. 2015;216:978. [PubMed] [Google Scholar]
- 42.Damron BL, Kelly LS. Short-term exposure of laying hens to high dietary sodium chloride levels. Poult Sci. 1987 May;66(5):825–8. doi: 10.3382/ps.0660825. https://linkinghub.elsevier.com/retrieve/pii/S0032-5791(19)52719-5. [DOI] [PubMed] [Google Scholar]
- 43.Nguyen Q, Bartlett G, Rodriguez C, Tellier P. Young adults on the perceived benefits and expected use of personal health records: a qualitative descriptive study. J Innov Health Inform. 2016 Apr 18;23(1):171. doi: 10.14236/jhi.v23i1.171. doi: 10.14236/jhi.v23i1.171. [DOI] [PubMed] [Google Scholar]
- 44.Ronda MC, Dijkhorst-Oei L, Rutten GE. Patients' experiences with and attitudes towards a diabetes patient web portal. PLoS One. 2015;10(6):e0129403. doi: 10.1371/journal.pone.0129403. https://dx.plos.org/10.1371/journal.pone.0129403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Hess R, Bryce CL, Paone S, Fischer G, McTigue KM, Olshansky E, Zickmund S, Fitzgerald K, Siminerio L. Exploring challenges and potentials of personal health records in diabetes self-management: implementation and initial assessment. Telemed J E Health. 2007 Oct;13(5):509–17. doi: 10.1089/tmj.2006.0089. [DOI] [PubMed] [Google Scholar]
- 46.Emani S, Yamin CK, Peters E, Karson AS, Lipsitz SR, Wald JS, Williams DH, Bates DW. Patient perceptions of a personal health record: a test of the diffusion of innovation model. J Med Internet Res. 2012 Nov 5;14(6):e150. doi: 10.2196/jmir.2278. https://www.jmir.org/2012/6/e150/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Tavares J, Oliveira T. Electronic health record patient portal adoption by health care consumers: an acceptance model and survey. J Med Internet Res. 2016 Mar 2;18(3):e49. doi: 10.2196/jmir.5069. https://www.jmir.org/2016/3/e49/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Klein R. An empirical examination of patient-physician portal acceptance. Eur J Inf Syst. 2017 Dec 19;16(6):751–60. doi: 10.1057/palgrave.ejis.3000719. [DOI] [Google Scholar]
- 49.Lazard A, Watkins I, Mackert M, Xie B, Stephens K, Shalev H. Design simplicity influences patient portal use: the role of aesthetic evaluations for technology acceptance. J Am Med Inform Assoc. 2016 Apr;23(e1):e157–61. doi: 10.1093/jamia/ocv174. http://europepmc.org/abstract/MED/26635314. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Zickmund SL, Hess R, Bryce CL, McTigue K, Olshansky E, Fitzgerald K, Fischer GS. Interest in the use of computerized patient portals: role of the provider-patient relationship. J Gen Intern Med. 2008 Jan;23(Suppl 1):20–6. doi: 10.1007/s11606-007-0273-6. http://europepmc.org/abstract/MED/18095039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Goel MS, Brown TL, Williams A, Cooper AJ, Hasnain-Wynia R, Baker DW. Patient reported barriers to enrolling in a patient portal. J Am Med Inform Assoc. 2011 Dec;18(Suppl 1):i8–12. doi: 10.1136/amiajnl-2011-000473. http://europepmc.org/abstract/MED/22071530. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Lyles CR, Harris LT, Jordan L, Grothaus L, Wehnes L, Reid RJ, Ralston JD. Patient race/ethnicity and shared medical record use among diabetes patients. Med Care. 2012;50(5):434–40. doi: 10.1097/mlr.0b013e318249d81b. [DOI] [PubMed] [Google Scholar]
- 53.Osborn CY, Mayberry LS, Wallston KA, Johnson KB, Elasy TA. Understanding patient portal use: implications for medication management. J Med Internet Res. 2013 Jul 3;15(7):e133. doi: 10.2196/jmir.2589. https://www.jmir.org/2013/7/e133/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Nambisan P. Factors that impact Patient Web Portal Readiness (PWPR) among the underserved. Int J Med Inform. 2017 Jun;102:62–70. doi: 10.1016/j.ijmedinf.2017.03.004. [DOI] [PubMed] [Google Scholar]
- 55.Ancker JS, Barrón Y, Rockoff ML, Hauser D, Pichardo M, Szerencsy A, Calman N. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med. 2011 Oct;26(10):1117–23. doi: 10.1007/s11606-011-1749-y. http://europepmc.org/abstract/MED/21647748. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Perzynski A, Roach M, Shick S, Callahan B, Gunzler D, Cebul R, Kaelber DC, Huml A, Thornton JD, Einstadter D. Patient portals and broadband internet inequality. J Am Med Inform Assoc. 2017 Sep 1;24(5):927–32. doi: 10.1093/jamia/ocx020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Ronda MC, Dijkhorst-Oei L, Rutten GE. Reasons and barriers for using a patient portal: survey among patients with diabetes mellitus. J Med Internet Res. 2014 Nov 25;16(11):e263. doi: 10.2196/jmir.3457. https://www.jmir.org/2014/11/e263/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Roblin DW, Houston TK, Allison JJ, Joski PJ, Becker ER. Disparities in use of a personal health record in a managed care organization. J Am Med Assoc. 2009 Sep 1;16(5):683–9. doi: 10.1197/jamia.m3169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Woods SS, Forsberg CW, Schwartz EC, Nazi KM, Hibbard JH, Houston TK, Gerrity M. The association of patient factors, digital access, and online behavior on sustained patient portal use: a prospective cohort of enrolled users. J Med Internet Res. 2017 Oct 17;19(10):e345. doi: 10.2196/jmir.7895. https://www.jmir.org/2017/10/e345/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Gu Y, Day K. Propensity of people with long-term conditions to use personal health records. Stud Health Technol Inf. 2013;188:46–51. [PubMed] [Google Scholar]
- 61.Gordon NP, Hornbrook MC. Differences in access to and preferences for using patient portals and other ehealth technologies based on race, ethnicity, and age: a database and survey study of seniors in a large health plan. J Med Internet Res. 2016 Mar 4;18(3):e50. doi: 10.2196/jmir.5105. https://www.jmir.org/2016/3/e50/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Butler J, Carter M, Hayden C, Gibson B, Weir C, Snow L. Understanding adoption of a personal health record in rural health care clinics: revealing barriers and facilitators of adoption including attributions about potential patient portal users and self-reported characteristics of early adopting users. AMIA Annual Symposium Proceedings; AMIA'13; May 6-9, 2013; New York, USA. 2013. pp. 152–161. [PMC free article] [PubMed] [Google Scholar]
- 63.Arcury TA, Quandt SA, Sandberg JC, Miller DP, Latulipe C, Leng X, Talton JW, Melius KP, Smith A, Bertoni AG. Patient portal utilization among ethnically diverse low income older adults: observational study. JMIR Med Inform. 2017 Nov 14;5(4):e47. doi: 10.2196/medinform.8026. https://medinform.jmir.org/2017/4/e47/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Kim S, Kim HR, Bae J, Kim Y. The consumers' perceptions and requirements for personal health records in Korea. J Korean Soc Med Inform. 2009;15(3):273. doi: 10.4258/jksmi.2009.15.3.273. [DOI] [Google Scholar]
- 65.Sanders MR, Winters P, Fortuna RJ, Mendoza M, Berliant M, Clark L, Fiscella K. Internet access and patient portal readiness among patients in a group of inner-city safety-net practices. J Ambulat Care Manag. 2013;36(3):251–9. doi: 10.1097/jac.0b013e31829702f9. [DOI] [PubMed] [Google Scholar]
- 66.Ancker JS, Nosal S, Hauser D, Way C, Calman N. Access policy and the digital divide in patient access to medical records. Health Pol Technol. 2017 Mar;6(1):3–11. doi: 10.1016/j.hlpt.2016.11.004. [DOI] [Google Scholar]
- 67.Goel MS, Brown TL, Williams A, Hasnain-Wynia R, Thompson JA, Baker DW. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med. 2011 Oct;26(10):1112–6. doi: 10.1007/s11606-011-1728-3. http://europepmc.org/abstract/MED/21538166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Jhamb M, Cavanaugh KL, Bian A, Chen G, Ikizler TA, Unruh ML, Abdel-Kader K. Disparities in electronic health record patient portal use in nephrology clinics. CJASN. 2015 Oct 22;10(11):2013–22. doi: 10.2215/cjn.01640215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Leveille SG, Mejilla R, Ngo L, Fossa A, Elmore JG, Darer J, Ralston JD, Delbanco T, Walker J. Do patients who access clinical information on patient internet portals have more primary care visits? Medical Care. 2016;54(1):17–23. doi: 10.1097/mlr.0000000000000442. [DOI] [PubMed] [Google Scholar]
- 70.Garrido T, Kanter M, Meng D, Turley M, Wang J, Sue V, Scott L. Race/ethnicity, personal health record access, and quality of care. Am J Manag Care. 2015 Feb 1;21(2):e103–13. https://www.ajmc.com/pubMed.php?pii=85966. [PubMed] [Google Scholar]
- 71.Martinez M, Baum A, Saldaño A, Gómez A, Luna D, de QF. Predictive variables of the use of personal health record: the Hospital Italiano de Buenos Aires study. Stud Health Technol Inf. 2013:1171. [PubMed] [Google Scholar]
- 72.Ancker JS, Osorio SN, Cheriff A, Cole CL, Silver M, Kaushal R. Patient activation and use of an electronic patient portal. Inform Health Soc Care. 2015;40(3):254–66. doi: 10.3109/17538157.2014.908200. [DOI] [PubMed] [Google Scholar]
- 73.Cahill J, Lin L, LoBiondo-Wood G, Armstrong T, Acquaye A, Vera-Bolanos E, Gilbert MR, Padhye NS. Personal health records, symptoms, uncertainty, and mood in brain tumor patients. Neurooncol Pract. 2014 Jun;1(2):64–70. doi: 10.1093/nop/npu005. http://europepmc.org/abstract/MED/26034618. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Bowling A. Research Methods in Health: Investigating Health and Health Services. New York, USA: McGraw-hill education; 2014. [Google Scholar]
- 75.Matthews B, Ross L. Research Methods: a Practical Guide for the Social Sciences. New York, USA: Pearson Education; 2010. [Google Scholar]
- 76.Shaughnessey J, Zechmeister E, Zechmeister J. Research Methods in Psychology. New York, USA: McGraw-Hill Education; 2014. [Google Scholar]
- 77.Shimada SL, Brandt CA, Feng H, McInnes DK, Rao SR, Rothendler JA, Haggstrom DA, Abel EA, Cioffari LS, Houston TK. Personal health record reach in the Veterans Health Administration: a cross-sectional analysis. J Med Internet Res. 2014 Dec 12;16(12):e272. doi: 10.2196/jmir.3751. https://www.jmir.org/2014/12/e272/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Andrews J, Shimp T. Advertising, Promotion, and Other Aspects of Integrated Marketing Communications. Tenth Edition. Boston, USA: Cengage Learning; 2017. [Google Scholar]
- 79.Kotler P, Armstrong G. Principles of Marketing. Upper Saddle River, USA: Pearson Higher Ed; 2017. [Google Scholar]
- 80.NHS login. 2019. [2020-12-21]. https://www.nhs.uk/using-the-nhs/nhs-services/nhs-login/
- 81.Bhattacherjee A. Understanding information systems continuance: an expectation-confirmation model. MIS Q. 2001 Sep;25(3):351. doi: 10.2307/3250921. [DOI] [Google Scholar]
- 82.Forquer HA, Christensen JL, Tan AS. Predicting continuance-findings from a longitudinal study of older adults using an eHealth newsletter. Health Commun. 2014;29(9):937–46. doi: 10.1080/10410236.2013.833580. http://europepmc.org/abstract/MED/24446900. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Nijland N, van Gemert-Pijnen JE, Kelders SM, Brandenburg BJ, Seydel ER. Factors influencing the use of a web-based application for supporting the self-care of patients with type 2 diabetes: a longitudinal study. J Med Internet Res. 2011 Sep 30;13(3):e71. doi: 10.2196/jmir.1603. https://www.jmir.org/2011/3/e71/ [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Questionnaire.