Abstract
Introduction:
Clinicians must provide anticipatory guidance pregnant clients would find useful but might not seek out independently. Client-facing health information resources should a) satisfy clients’ health self-management queries, and b) provide anticipatory guidance at developmentaly appropriate times. Care Guide (by our technology partner, Maternity Neighborhood™) is an online maternity education platform positioned to meet pregnant clients’ information needs through high-quality, curated content paired with secure provider/client messaging. The research version of Care Guide is called Maternity Information Access Point (MIAP). Little is known about how clients perceive or engage with maternity education delivered via patient portal or personal health record.
Methods:
This qualitative study employed focus groups and four week field-testing periods with English- and Spanish-speaking pregnant women enrolled in Medicaid. User satisfaction and system usability were evaluated through self-report instruments.
Results:
Twelve of the 16 participants logged usage of MIAP, with amount of usage varying widely. Satisfaction (4.3/5) and usability (4.7/5) were rated highly. Weekly content push emails were a popular feature; participants agreed the content was relevant, timely, and useful. Forgetting passwords and lack of experience with technology were barriers to use.
Discussion and Conclusion:
Gestational age captured in the electronic health record can support automated pushing of content containing highly relevant anticipatory guidance. Platform features that guide the user through content can be leveraged to promote continued user engagement. Users’ desire for easy access to content must be balanced against the need to safeguard protected health information. Digital newcomers may require in-person technical support.
Keywords: Electronic Health Records, Prenatal Education, Information Seeking Behavior, Patient Education as Topic, Health Communication
Introduction
Pregnancy is a time of high information need as maternity care clients manage the health of their changing bodies and look ahead to birth and parenting. Pregnant clients’ motivation to get answers to their maternity questions is evident from their frequent consultation of numerous online and offline informational sources [1,2]. However, information needs during the maternity care episode extend beyond those identified by the client; clinicians and health educators must provide anticipatory guidance to cover material that the client would find useful but might not seek out independently [3] In fact, comprehensive and timely anticipatory guidance is a feature of prenatal care that is highly valued by clients [4] but may not be fully realized during a time-constrained office visit [5]. Therefore, client-facing health information resources can be a valuable adjunct to clinical visits [6] provided (1) that they satisfy clients’ health self-management queries, and (2) that they provide anticipatory guidance at developmentally appropriate times.
Publicly available pregnancy and childbirth websites and mobile apps are popular informational resources [3,7]. Their easy availability is an advantage but many contain advertisements [7] and inaccurate information [8,9] and thus are perceived by users as not being completely trustworthy [8,10]. Popular commercial sites such as Babycenter.com and WhatToExpect. com provide anticipatory guidance via weekly subscription emails about fetal and maternal development tailored to the user’s gestational age. However, given that maternity clients do not always discuss their findings with their care providers [3,10,11], it may be difficult for providers to insure that the anticipatory guidance their clients receive in this way is accurate, complete, and appropriately timed.
Educational content delivered by way of patient portals and Personal Health Records (PHRs) has the potential to mitigate some of the drawbacks of publicly available sites by putting the sponsoring organization or clinician in control of the content and the timing of its delivery Additionally, the opportunity to find one’s own clinical data on the same platform as content vetted by a provider may be appealing for clients. Although the concept of delivering informational content via portal or PHR is not new [11], there are few mentions in the literature of this format being applied to maternity care [12,13] and of these, two describe only system features [13,14] and one is a study protocol [15]. The only one that examined users’ (n = 12) experiences reported that the most popular user activity was viewing informational resources but got mixed reviews for usability [16]. Thus, little is known about how clients perceive or engage with the maternity education components of portals or PHRs.
The goal of our research program is to ensure that online maternity information resources are accessible to and meet the communication and learning needs of the women at greatest risk of health disparities. As such, the aim of the qualitative study reported here was to explore the feasibility and acceptability of an online maternity-education platform among Medicaid-enrolled women. To that end, we partnered with Maternity Neighborhood, the maker of a maternity-specific electronic health record (EHR) by the same name. Integrated with the EHR is a maternity education platform called Care Guide. This mobile-ready site is designed to meet pregnant clients’ information needs through the provision of high-quality, curated, browsable and searchable educational content paired with secure provider and client messaging. Key data elements such as gestational age are shared between the EHR and Care Guide so that content can be pushed to users automatically via email based on their stage of pregnancy Clinicians have the option of loading their own content into their practice’s Care Guide as well as the ability to push content to individual clients manually as clinical needs arise. The research version of Care Guide is known as Maternity Information Access Point (MIAP). The theoretical framework for this study was Coiera’s common ground theory of health care communication, which posits that “for computational tools to be of value, they have to share ground with human beings. Users need to know how to use the system and the system needs to be fashioned to users’ needs.” [14] In this paper we report selected findings from MIAP that relate to the use of electronic health data.
Methods
This qualitative study consisted of four pairs of focus groups each bookending a four week fieldtesting period. Two pairs of groups were conducted in English and the other two in Spanish. The study was conducted in New York City from September through December 2015. Approval was obtained from the Columbia University Medical Center Institutional Review Board.
Participants
Eligible participants were ages 18 years or older, ≤ 35 weeks pregnant, enrolled in Medicaid, able to speak English or Spanish, and had a Wi-Fi enabled device. Potential participants were referred by local organizations that provide doulas or early Head Start services to low-income women. Participants received Target gift cards ($50 for each of two sessions) in compensation for their time.
Data Collection
The following data were collected at intake by paper-and-pencil self-report: basic demographic data, information about internet access, and a measure of eHealth literacy (eHEALS) [15]. The Newest Vital Sign (NVS) [16], a measure of health literacy was also administered by study staff at intake. During the follow-up sessions, we collected the Wellness Portal Survey [17] as a measure of user satisfaction and selected items from the Newest Vital Sign (NVS) [18,19], as a measure of system usability Both were adapted to reference MIAP and are scored from 1 to 5 where 1 signals dissatisfaction/lack of usability 3 is neutral, and 5 indicates satisfaction/usability (see Appendix). All focus group sessions were audio recorded and conducted by trained facilitators according to approved focus group guides. Audio recordings were supplemented with notes taken by study staff. Maternity Neighborhood provided a log file of all MIAP activity during the study period. Each log file entry included a time and date stamp, the type of event (e.g., resource read, message sent by client), the user’s ID, the user’s gestational age on that date, and the name of the resource accessed, if applicable.
Language
At the time of the study, a limited number of Spanish-language resources came preloaded onto Care Guide, but the interface was available only in English. Spanish-speaking participants were provided with written instructions for navigating the few English-language menu items they would encounter. In order to ensure parity of content across languages, additional Spanish-language content was loaded onto MIAP from publicly available sources including womenshealth.gov, March of Dimes, and Lamaze International. Existing Spanish translations were used for the NVS [19], eHEALS [20], and UTAUT [21]. All other study materials were translated by the bilingual study staff and checked by at least one additional native Spanish speaker.
Procedure
Topics of initial focus group sessions included barriers and facilitators to online access and information-seeking, preferred information formats, and desired features of an education platform and content attributes. Participants received a mobile hotspot to ensure adequate connectivity, as well as instructions for using the hotspot and logging in to MIAP. They were asked to log in and view a resource or send a message to study staff at least once a week. To the extent that time permitted, study staff provided basic technical support to participants who required assistance.
During field testing, study staff pushed content via email, as shown in Figure 1, to participants once a week. Prior to the start of the study, each informational resource was tagged with an optimal gestational age, and a schedule was developed so that there would be at least one resource sent in each language for every gestational week. The platform calculates gestational age based on the user’s estimated date of delivery. It has the functionality to automatically push content based on gestational age, but because of an inability to toggle the language of the interface content was pushed manually so that users received content only in their preferred language.
The follow-up focus group sessions served to elicit participants’ experiences using MIAP, including their suggestions for improvement. Participants unable to attend follow-up sessions were interviewed individually. Focus group sessions were followed by peer debriefing sessions for the study staff in attendance.
Data Analysis
Audio recordings of focus group and peer debriefing sessions were transcribed and checked for accuracy. Conventional content analysis was employed to analyze the transcripts and notes in their original language using TAMS Analyzer 4.0. Two coders analyzed the transcripts independently, then discussed the codes to consensus. The principal investigator (AA) used an inductive approach to cluster codes into thematic categories and subcategories. Quantitative data were analyzed with simple descriptive statistics. Log files of each participant’s activities on the platform were analyzed to determine the following: (1) number of login sessions, (2) number of messages sent by user, (3) total number resources viewed, (4) average number of resources viewed per login session, (5) number of articles received in pushed emails, (6) number of articles read from pushed emails, and (7) number of articles read that did not originate from a pushed email.
Results
Participants
Sixteen women—nine English speakers and seven Spanish speakers—participated in the focus groups (see Table 1). Their average age was 26.0 (SD = 5.6) and their average gestational age at the start of the study was 24 weeks, 0 days (range 13 weeks, 5 days to 32 weeks, 2 days). The Spanish-speaking participants were more likely to be experienced mothers (1 nulliparous, 6 multiparous) than the English-speaking participants (6 nulliparous, 3 multiparous). Participants were evenly split between Hispanic and non-Hispanic, most (62 percent, n = 10) were married or living with a partner, and they had an average income of $4,809 (SD = $4,842) per household member per year. Most (69 percent, n = 11) had high-speed internet at home, just under half (44 percent, n = 7) had unlimited mobile data, most (69 percent, n = 11) used Android devices, and the great majority (88 percent, n = 14) rated themselves as either “intermediate” or “expert” users on the internet. There is some likelihood, as measured by the NVS, that the majority of participants (69 percent, n = 11) have limited health literacy. However, based on their eHEALS scores (M = 31.5, SD = 4.4) they felt confident about their ability to find and use health resources on the internet.
Table 1.
CHARACTERISTIC | N | (%) |
---|---|---|
PREFERRED LANGUAGE AND PARITY | ||
English, nulliparous | 6 | (38) |
English, multiparous | 3 | (18) |
Spanish, nulliparous | 1 | (6) |
Spanish, multiparous | 6 | (38) |
ETHNICITY | ||
Hispanic | 8 | (50) |
Non-Hispanic | 8 | (50) |
PARTNERSHIP | ||
Married or living with partner | 10 | (62) |
Single, never married or separated | 6 | (38) |
HIGH-SPEED INTERNET AT HOME | ||
Yes | 11 | (69) |
No | 5 | (31) |
MOBILE DATA PLAN | ||
Unlimited data | 7 | (44) |
Throttled* | 2 | (12) |
Limited or unknown | 7 | (44) |
MOBILE PLATFORM | ||
Android | 11 | (69) |
iPhone | 5 | (31) |
HOW WOULD YOU RATE YOURSELF AS A USER ON THE INTERNET? | ||
Beginning user | 2 | (12) |
Intermediate user | 7 | (44) |
Expert user | 7 | (44) |
NEWEST VITAL SIGN (NVS) SCORE | ||
0 to 1, high likelihood of limited literacy | 1 | (7) |
2 to 3, possibility of limited literacy | 10 | (62) |
4 to 6, almost always adequate literacy | 5 | (31) |
M | (SD) | |
Participant Age | 26.0 | (5.6) |
Gestational age, weeks/days | 24w 0d | (6w 5.5d) |
Gestational age, days | 168 | (47.5) |
Income per household member per year (n = 12) | $4,809 | ($4,842) |
eHealth Literacy Scale (eHEALS) score** (n = 15) | 31.5 | (4.4) |
*In throttled plans, users may use unlimited amounts of data but speeds are substantially reduced at a predetermined threshold of data usage (e.g., 1GB).
** Possible scores range from 8 to 40; higher scores indicate higher e-literacy.
Log Files
Log file data revealed that 12 of the 16 participants logged in to MIAP during the field-testing period. The 12 users averaged 6.3 login sessions (range 2–13) each and accessed an average of 14.7 informational resources (range 3–37), or about 2.5 resources per login session. Users received between 5 and 14 resources over the course of four content push emails (M = 9.7). On average, participants viewed over a third (M = 3.6, 37 percent) of the resources pushed to them. Ten of the 12 users viewed resources that were not pushed to them, for an average of 11.1 (range 2–31) resources not associated with a content push email. Three users sent the study staff one message each and two users sent two messages.
User Satisfaction and Usability
Overall, the 12 users rated satisfaction and usability highly using a 1 to 5 scale where higher scores are desirable. The average across items for the Wellness Portal Survey was 4.3. Only two items averaged scores less than 4: “I understand the instructions on how to send messages with MIAP” (M = 3.8), and “MIAP helped me improve my health” (M = 3.9). The highest scoring items were “I can navigate MIAP easily” (M = 4.7) and “Information I get from MIAP is exactly what I need to make more informed health decisions” (M = 4.6). The average score across UTAUT usability items was 4.7. The lowest scoring item was “Using MIAP increases my learning productivity” (M = 4.3). Items 1, 5, 6, 10, and 12 tied for the highest average rating of 4.8.
Focus Groups
In focus group discussions, participants said that they very much liked receiving pushed content weekly because they found the content to be relevant, easy to understand, and useful to them at their stage of pregnancy. Some reported that the weekly email prompted them to engage with the platform and, that once logged in, they followed links from the pushed resource to “related” resources. Otherwise, participants reported engaging with the platform primarily through scrolling through the library of resources; very few used the available search or filtering functionality (see Figure 2). Participants cited the need to remember a password and log in as a barrier to use. Because of this, and because most used their phones to access MIAP, they said would have preferred an app that keeps them logged in over the existing mobile-ready website. None seemed concerned about potential loss of privacy that staying logged in would imply but none were using it in a real-world context in which their EHR data also would be accessible through the app.
Few participants felt that all their information needs where met by their care provider, and many participants described using a tenacious approach to finding the additional information they sought. They repeatedly spoke of consulting multiple sources of information to triangulate and establish the trustworthiness of the information they found. Most said that knowing that content had been vetted by their care provider would increase their confidence in its trustworthiness, and several volunteered that they had largely stopped consulting other sites after gaining access to MIAP. Several participants told anecdotes about the value of the anticipatory guidance they received from MIAP. One woman was surprised to learn that her nasal congestion could be pregnancy related and another was relieved to learn that her bleeding gums and back pain were likely to resolve after delivery.
Overall, English- and Spanish-speaking participants shared similar thoughts and experiences across the various focus group topics. The Spanish-speaking participants did report occasional difficulties with understanding materials if they were available only in English. As a research team, we also noticed that there were more digital newcomers among the Spanish-speaking participants. Generally they required more one-on-one in-person support to do things like adjust settings on their phones, set up passwords, navigate email, log in to the platform, and send messages. These difficulties were not primarily language-related as their phones were set up in Spanish. Rather, the ones who required assistance appeared to be relatively recently arrived in the United States and simply had limited familiarity with mobile technology and the internet. Time constraints on participants’ availability prevented study staff from providing as much support as they felt participants needed.
Discussion
In this study we found that an online maternity education platform was feasible and acceptable to the majority of the Medicaid-enrolled women invited to use it. Key among our findings was that MIAP helped to meet users’ needs for anticipatory guidance in a timely way primarily via content push emails. We envision that this popular feature could be extended to other data elements within the EHR to support clients’ learning needs for a variety of health conditions.
We found substantial variation in the amount of engagement with the platform among the participants. Use patterns appeared to be consistent with the level of enthusiasm the participants demonstrated in focus groups for acquiring information: one woman who felt that her care provider gave her all the information she needed never logged on, whereas another who described herself as a “research nut” logged the most encounters and resources viewed. Our interpretation of this finding is that MIAP provided an opportunity for women to meet their information needs to the extent that they perceived them but without being intrusive.
Somewhat in contrast to their self-depictions as aggressive seekers of information, participants’ actual usage patterns tended to be relatively passive in nature. Rather than searching for specific terms or using filtering features, most participants opted to scroll through the resource library or respond to the prompt provided by content push emails. Their reported use of links to “Related” resources (see Figure 2) suggests that this style of engagement could be optimized by providing users with readymade paths through the content. This may be done explicitly by suggesting topical curricula or small content bundles to the user based on their knowledge and interests, or more indirectly through deliberate curation and placement of content bundles.
Users’ desire for a mobile app and password-free access to the platform in place of a passwordprotected mobile-ready website raises the issue of how to balance ease of access with maintenance of data privacy and security on platforms that hold protected health information (PHI). The need to use passwords appeared to deter one or more participants from engaging with MIAP entirely. The need for passwords for platforms containing PHI remains, but supplementation with fingerprint authentication [22] or other methods yet to emerge may reduce the burden on the user. Platforms can explain clearly to the user the possible implications of a data breach so that users can at least understand the importance of creating and using a strong password.
In addition to passwords, other potential barriers to use include language and a low level of comfort or experience with technology. It is critically important that both content and interfaces be made available in the languages dominant among the population being served. The principal investigator, who has worked multiple times with dual-language materials, attests that the challenge that such a strategy represents is outweighed by its tremendous value. With respect to experience with technology even a website with few menu items and straightforward functionality presented a usability challenge to digital newcomers. Future generations will be digital natives, but for the moment, the needs of digital newcomers must be taken into account and adequate supports provided. Online guided tutorials are not enough; the most novice users will need a person at their side to help them get online.
Limitations
This study has several limitations. Chief among these is that the use of MIAP was not integrated with participants’ clinical care. We would expect users to be more engaged if given the opportunity to interact online with their actual care provider. Given the low volume of messages sent by participants, the prospect of messaging with research staff appears to have been a poor proxy for messaging with one’s own provider. Also, the sample size was relatively small. Nevertheless, findings across groups and participants were rather consistent, suggesting that this limitation was a modest one. In addition, the high satisfaction and usability ratings participants gave MIAP may have been influenced by social desirability bias or acquiescence bias. We worked to minimize these types of bias by asking open-ended questions and soliciting both negative and positive opinions but cannot rule out the possibility of their influence on our findings. Lastly we were unable to confirm nonusage by specific participants until after the focus groups had concluded, and as such lost the opportunity to inquire directly if their non-use was due to lack of interest in the platform or if participants experienced barriers to its use. Therefore, our conclusions about passwords and lack of experience with technology are based more on our observations and on indirect mentions by participants rather than from their explicit statements that these were the primary reasons for nonuse.
Generalizability and Future Work
Participants’ receptiveness to tailored pushed content raises the question of the generalizability of the concept. It is possible that there are other situations in which patients would appreciate receiving pushed content based upon one or more data elements in the EHR. A patient education platform such as the one described here would not be necessary for implementation; the full text and images of the content could be emailed directly to patients who opt in. For example, following the maternity care episode, guidance on infant development and vaccinations could be pushed to parents automatically based on the date of delivery Indeed, reminders and anticipatory guidance regarding vaccines based on birthdate have relevance throughout the lifespan such as for the shingles vaccine that is recommended for most adults age 60.
Many other data elements could be leveraged for the purpose of pushing content including diagnosis codes, medications, items from the problem list, procedure codes, and hospital discharge. Certain new diagnoses, such as diabetes, are similar to pregnancy in that they impose many new selfmanagement learning needs that cannot be satisfied in a single office visit. A programmed schedule of diabetes education content could be triggered by the initial appearance of a diagnosis code. The initial appearance of an element such as a new prescription for prednisone or the inclusion of nausea on the problem list could trigger delivery content with tips for dealing with medication side effects or managing symptoms. Algorithms can be used to look for specific combinations of data elements such as certain procedure codes plus discharge from the hospital to then trigger delivery of anticipatory guidance specific to particular stages of recovery. For example, guidance on wound healing could be pushed days or weeks following discharge for a surgical procedure.
Successful implementation of tailored pushed content based on EHR elements would depend on buy-in from patients and clinicians, and thoughtful design. For instance, a content push should be suppressed when delivery would be inappropriate (e.g., a patient is new but their diagnosis is not) or a nuisance (e.g., receipt of the same content twice following the delivery of twins). Verification by a clinician prior to a content push might be desirable in situations where complex judgments must be made about whether or not a patient is a good candidate for receiving the pushed content.
The future work planned for MIAP includes making the platform available as a mobile app. As such, additional options become available for alerts and for users to customize when and how they prefer to receive such alerts. The move to the app format is a good opportunity to add an interactive “welcome” tutorial to make the app accessible to digital newcomers. Lastly, we plan to tailor content modules to users’ demonstrated competency gaps as assessed by knowledge and attitude questions.
Conclusion
Gestational age captured in the EHR can support automated pushing of content containing the highly relevant anticipatory guidance that maternity clients need to support optimal self-management during pregnancy. Platform structure that guides the user through a curated content bundle can be leveraged to promote continued user engagement. Users’ desire for easy access to content must be balanced against the need to safeguard PHI. Digital newcomers may require in-person technical support to gain access to digital health resources.
Acknowledgements
This study was funded by the Columbia University Provost’s Grants Program for Junior Faculty Who Contribute to the Diversity Goals of the University. Many thanks to Maternity Neighborhood, Eva Rose Asaan Warner Ancient Song Doula Services, Healthy Start Brooklyn, Northern Manhattan Perinatal Partnership, Samantha Stonbraker, Heidi Luft, and Caroline Handschuh.
Appendix
Users Satisfaction: Wellness Portal Survey, Adapted from Chou et al., 2010 [20]
Response options ranged from 1 = strongly disagree to 5 = strongly agree with a neutral midpoint
I can navigate MIAP easily.
I can find information I need easily and quickly in MIAP
I understand the health information present in MIAP
I understand the instructions on how to send messages with MIAP
The information I find on MIAP is just what I need for managing my pregnancy.
The information I find on MIAP is arranged well. I always know where to look for what I need.
Information I get from MIAP is important to me.
Information I get from MIAP helps me improve my health.
Information I get from MIAP helps me participate more in my own health care.
Information I get from MIAP is exactly what I need to make more informed health care decisions.
MIAP is a valuable resource for me.
MIAP improves my interaction with my health care provider
MIAP helped me improve my health.
MIAP would likely help me to continually improve my health and wellbeing if I continued using it.
System Usability: Selected UTAUT Items, Adapted from Venkatesh et al., 2012 [21]
Response options ranged from 1 = strongly disagree to 5 = strongly agree with a neutral midpoint
I find MIAP useful for learning about pregnancy and childbirth.
Using MIAP increases my chances of getting the information that is important to me.
Using MIAP enables me to learn about pregnancy and childbirth more quickly.
Using MIAP increases my learning productivity.
Learning to use MIAP is easy for me.
My interaction with MIAP is clear and understandable.
I find MIAP easy to use.
It is easy for me to become skillful at using MIAP
I have the resources necessary to use MIAP
I have the knowledge necessary to use MIAP
MIAP is compatible with other technologies I use.
I can get help from others when I have difficulties using MIAP
References
- 1.Arcia A. US nulliparas’ perceptions of roles and of the birth experience as predictors of their delivery preferences. Midwifery. 2013. August 31;29(8):885–94. [DOI] [PubMed] [Google Scholar]
- 2.Sayakhot P, Carolan-Olah M. Internet use by pregnant women seeking pregnancy-related information: a systematic review. BMC Pregnancy Childbirth. 2016. March 28;16(1):1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Team TT, Carter MC, Corry M, Delbanco S, Foster TC, Friedland R, Gabel R, Gipson T, Jolivet RR, Main E, Sakala C. 2020 vision for a high-quality, high-value maternity care system. Womens Health Issues. 2010. February 28;20(1):S7–17. [DOI] [PubMed] [Google Scholar]
- 4.Doherty ME. Midwifery care: Reflections of midwifery clients. J Perinat Educ. 2010. January 1;19(4):41–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Lagan BM, Sinclair M, George Kernohan W. Internet use in pregnancy informs women’s decision making: a web-based survey. Birth. 2010. June 1;37(2):106–15. [DOI] [PubMed] [Google Scholar]
- 6.Daniels M, Wedler JA. Enhancing Childbirth Education through Technology. Int J Childbirth Educ. 2015. July 1;30(3). [Google Scholar]
- 7.Lewallen LP, Côté-Arsenault DY. Implications for nurses and researchers of Internet use by childbearing women. Nurs Womens Health. 2014. October 1;18(5):392–400. [DOI] [PubMed] [Google Scholar]
- 8.Hansen C, Interrante JD, Ailes EC, Frey MT, Broussard CS, Godoshian VJ, Lewis C, Polen KN, Garcia AP, Gilboa SM. Assessment of YouTube videos as a source of information on medication use in pregnancy. Pharmacoepidemiol Drug Saf. 2016. January 1;25(1):35–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Narasimhulu DM, Karakash S, Weedon J, Minkoff H. Patterns of internet use by pregnant women, and reliability of pregnancy- related searches. Matern Child Health J. 2016. July 25:1–8. [DOI] [PubMed] [Google Scholar]
- 10.Huberty J, Dinkel D, Beets MW, Coleman J. Describing the use of the internet for health, physical activity, and nutrition information in pregnant women. Matern Child Health J. 2013. October 1;17(8):1363–72. [DOI] [PubMed] [Google Scholar]
- 11.Pagliari C, Detmer D, Singleton P Potential of electronic personal health records. BMJ. 2007. August 18;335:330–333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Choemprayong S, Oh S, Sheble L. Interfaces for the Personal Pregnancy Health Records (PregHeR) System: facets in time. AMIA Ann Symp Proc. 2006;2006:885–. [PMC free article] [PubMed] [Google Scholar]
- 13.Viand KS. A Personal Health Record Module for Pregnant Women: System Development and User Adoption Study Ontario, Canada: McMaster University; 2014. [Google Scholar]
- 14.Coiera E. When conversation is better than computation. JAMIA. 2000. May-Jun;7(3):277–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res. 2006. Oct-Dec;8(4). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, et al. Quick assessment of literacy in primary care: The Newest Vital Sign. Ann Fam Med. 2005;3:514–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Chou AF, Nagykaldi Z, Aspy CB, Mold JW. Promoting patient- centered preventive care using a wellness portal: preliminary findings. J Prim Care Community Health. 2010. July 1;1(2):88–92. [DOI] [PubMed] [Google Scholar]
- 18.Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, et al. Quick assessment of literacy in primary care: The Newest Vital Sign. Ann Fam Med. 2005;3:514–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Venkatesh V, Thong JYL, Xu X. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Q. 2012. March;36(1):157–78. [Google Scholar]
- 20.Paramio Perez G, Almagro BJ, Hernando Gomez A, Aguaded Gomez JI. [Validation of the eHealth Literacy Scale (eHEALS) in Spanish University Students]. Rev Esp Salud Publica. 2015. May-Jun;89(3):329–38. [DOI] [PubMed] [Google Scholar]
- 21.Madera MM, Torres Nabel LC, Quevedo Huerta LN. Estudio de traducción y confiabilidad del instrumento de la Teoría Unificada de la Aceptación y Uso de la Tecnología (UTAUT). Revista Apertura. 2012;4(12). [Google Scholar]
- 22.Martínez-Pérez B, De La Torre-Díez I, López-Coronado M. Privacy and security in mobile health apps: a review and recommendations. J Med Sys. 2015. January 1;39(1):1–8. [DOI] [PubMed] [Google Scholar]