Sociotechnical challenges to developing technologies for patient access to health information exchange data

Abstract

Background

Providing patients with access to their medical data is widely expected to help educate and empower them to manage their own health. Health information exchange (HIE) infrastructures could potentially help patients access records across multiple healthcare providers. We studied three HIE organizations as they developed portals to give consumers access to HIE data previously exchanged only among healthcare organizations.

Objective

To follow the development of new consumer portal technologies, and to identify barriers and facilitators to patient access to HIE data.

Methods

Semistructured interviews of 15 key informants over a 2-year period spanning the development and early implementation of three new projects, coded according to a sociotechnical framework.

Results

As the organizations tried to develop functionality that fully served the needs of both providers and patients, plans were altered by technical barriers (primarily related to data standardization) and cultural and legal issues surrounding data access. Organizational changes also played an important role in altering project plans. In all three cases, patient access to data was significantly scaled back from initial plans.

Conclusions

This prospective study revealed how sociotechnical factors previously identified as important in health information technology success and failure helped to shape the evolution of three novel consumer informatics projects. Barriers to providing patients with seamless access to their HIE data were multifactorial. Remedies will have to address technical, organizational, cultural, and other factors.

Introduction

Tools to help patients access and use their own medical data are widely expected to empower them to manage their health better.^1–7 The federal ‘meaningful use’ program^{8 9} and initiatives such as patient-centered medical homes¹⁰ are driving investments in electronic patient portals to enable better data sharing and communication with patients.^{11 12} The blue button initiative helps Veterans Administration patients download their own records.¹³ Meanwhile, consumer interest in technologies for accessing health information is reflected in continued growth in health-related internet searches,^14–17 social networking,^14–17 and smartphone apps.¹⁸

As more patients take advantage of these initiatives to access their medical data, a key challenge will be to help them to access and integrate data from multiple healthcare organizations. It is possible that health information exchange (HIE) organizations might be able to offer a solution. In recent years, HIE organizations across the USA have developed infrastructures for healthcare providers to exchange patient data securely.^19–21 As a result, increasing numbers of providers are beginning to be able to access patient data from outside their own organization.^22–25 Such existing HIE infrastructures could be uniquely situated to help patients address the challenge of collecting and exchanging data across healthcare organizations. Growing numbers of HIE organizations are beginning to offer some ability for patients to view their data.²⁶

Novel technology development would be needed for patients to access and use HIE data. However, it is very difficult to develop and implement new health information technology (IT). In a high-profile case, Google discontinued its Google Health product, a platform allowing patients to aggregate and exchange their own data, after disappointing adoption.²⁷ Internationally, it has been estimated that fewer than half of health IT projects succeed.^{28 29} A variety of sociotechnical causes has been identified, ranging from technical (such as data standards) to organizational and cultural (such as failure to fit the culture of healthcare).^28–32 Useful frameworks of sociotechnical factors involved in health IT success or failure have been developed by Brender et al³¹ and Kaplan and Harris-Salamone,³⁰ using consensus of expert opinion.

An opportunity to study the process of consumer technology development arose in 2010 when several HIE organizations in New York State launched initiatives to develop products for patient access to HIE data. We anticipated that these projects would encounter many of the sociotechnical issues identified in studies of clinical information systems within healthcare organizations. In addition, it seemed likely that the efforts to provide patients with data across multiple healthcare organizations would lead to additional complexity. Following recommendations in an AMIA consensus paper for additional qualitative research on processes throughout the health IT lifecycle,³⁰ we designed a prospective qualitative study to follow the organizations as they developed the new technologies.

Our objective was to follow the development of new consumer information technologies prospectively from a sociotechnical perspective, with a focus on identifying barriers and facilitators to patient access to medical data across multiple healthcare organizations.

Methods

Setting and participants

In 2010, New York State grant funding was released to five regional health information organizations (RHIO) to launch projects to give patients access to community-wide clinical data. We recruited one urban, one largely rural, and one suburban organization in order to maximize diversity. The Brooklyn Health Information Exchange serves the New York City borough of Brooklyn; Southern Tier HealthLink covers rural areas and small cities in central New York State; and the Long Island Patient Information Exchange covers largely suburban Long Island.

All three RHIO already offered HIE services through which providers could query patient data from healthcare organizations across the community. The RHIO planned to use the grants to give patients access to these HIE data.

Sampling

In a snowball sampling approach, RHIO leaders were asked to recommend key informants, and these informants were invited to recommend additional internal and external contacts. Sampling and data analysis were conducted in parallel, and sampling was concluded when the researchers concluded that no new themes were arising and therefore data saturation had been achieved.³³ The Weill Cornell Institutional Review Board approved this study, and all participants granted informed consent.

Semistructured interview development

The interview guide asked about the technology, challenges to its development, and solutions or adaptations made in response to challenges (see supplementary appendix 1, available online only). As prompts, we named areas mentioned in published literature^28–31 (technical issues and functionality, organizational issues, economic factors, user behavior, national and state policy, and legal issues), concluding with an open-ended question allowing informants to propose additional topics.

Data collection

As we were interested in prospectively following the projects, we conducted interviews at multiple time points. A set of baseline interviews lasting 60–90 min was conducted shortly after grant funds were awarded, either in person or by telephone. A set of follow-up interviews lasting 60–90 min was conducted by telephone 8–13 months later at project go-live.

In addition to the interviews, we also reviewed the funding request for proposals, RHIO grant applications, RHIO websites, and consent forms, and viewed demos of the products. Every 4–8 weeks during the project, a 30-min unstructured telephone update was conducted with the project lead for each project. These brief interviews were not coded but instead used to help the researchers follow the development or implementation milestones and identify when materials such as consent forms, information on the RHIO website, or product demos were available for review.

Analysis

All baseline and follow-up interview recordings were transcribed and individual identifiers stripped from the transcripts. Two researchers (one with graduate training in public health and the other with graduate training in informatics) developed the initial code book using a grounded theory approach³³ by reviewing three early transcripts. The researchers then coded the remaining transcripts separately and met to reach consensus on each. Codes were applied at the paragraph level (meaning that individual codes might appear multiple times in a single transcript); codes were not mutually exclusive (meaning that individual paragraphs sometimes received more than one code). Codes were captured in ATLAS.ti (Scientific Software Development, Berlin, Federal Republic of Germany).

The codes were categorized according to the Brender framework of sociotechnical success and failure factors.³¹ This framework, developed through a Delphi study with expert participants involved in the European Federation for Medical Informatics Special Topics Conference, was considered appropriate for the current study because of its comprehensive listing of factors involved in both the success and failure of novel health IT. For simplicity, we collapsed three of Brender's categories into related categories to create nine categories:

• functionality (functions offered by the software and their fit to user needs);

• technical issues (technical support for these functions, including data and messaging standards, and system performance);

• organizational/managerial issues (planning, resource allocation, communication processes between developers and users, and implementation processes);

• culture (culture of healthcare and of patients);

• policy/strategy (national, statewide, and/or institutional policy and strategy);

• legal issues (HIPAA, state law, and other legal frameworks);

• economic issues (organizational and vendor resources, broader economic climate);

• education (user education and training on the system);

• behavior/user acceptance (user attitude, motivation, acceptance).

Codes were also classified by whether they were mentioned in conjunction with a challenge to the development and implementation of the new technology, a facilitator to development and implementation, or a solution developed in response to a challenge. We assumed that codes appearing frequently represented topics important to the interviewees. To provide a semiquantitative view of the importance of each code category, we graphed the relative percentage of all codes represented by each category in the baseline interviews and again in the follow-up interviews (figure 1).

Figure 1.

Types of challenges mentioned by informants changed over time. For example, technical barriers were mentioned much more often at follow-up than at baseline, while organizational, economic, and policy barriers decreased in importance.

In a separate analysis, codes were linked into themes using a grounded theory approach. These themes are listed in the results section as themes 1 to 4.

In the final step of the analysis, the Brender code categories as well as themes 1 to 4 were presented to a subset of the key informants, and their comments were used to revise the findings, in a member-checking process designed to improve the accuracy of interpretation and credibility of results from the perspective of those interviewed.³⁴

Results

In total, 19 interviews were conducted with 15 key informants (seven at RHIO A, four at RHIO B, and five at RHIO C). Informants were project managers, directors, project leads, membership and outreach staff, as well as product developers from software vendors contributing to the projects. Several of the informants had advanced training in medicine, public health, law, nursing, or software engineering; others did not have graduate degrees and described themselves as public outreach or patient advocates. A total of 162 pages of transcripts was analyzed.

Project summaries

All projects were completed within the grant period. However, substantive adaptations arose in response to the challenges, so that final products were significantly narrower in scope than planned.

Project 1 was conceived as a personal health record (PHR) populated with HIE data, which would give patients electronic control over which providers could access their data. (In New York, providers must obtain patient consent before accessing HIE data about the patient, although consent may be waived in emergencies.³⁵) However, the project changed significantly to an HIE consent management portal, functionality enabling patients to download HIE data into an existing PHR account, and an audit system allowing patients to track accesses of their data. Shortly after go-live, about 1600 patients had logged into the consent management portal, and 267 downloaded data from the HIE into a PHR account.

Project 2 was also intended to be a PHR populated by HIE data, with electronic management of HIE consent. However, as a result of challenges including the acquisition of a key software vendor, the project evolved into a secure patient–provider messaging center that allowed patients to search for and message providers, who could send selected HIE data to patients.³⁶ Although software was developed and field-tested, the project was put on indefinite hold after the RHIO underwent a merger.

Project 3 was designed as a HIE-populated PHR with consent management functions, with the addition of analytics to process HIE data for reminders and alerts to both patients and providers. As a result of technical, legal, and cultural issues encountered, the product became narrower in scope, using only a subset of the HIE data. Approximately 3000 patients registered to use the portal.

Sociotechnical challenges

In discussions of challenges to the projects, the dominant code category was achieving functionality that met the needs of both patients and providers (figure 1). Determining what functions should be offered involved iteration between goals and pragmatic realities, as well as discussions with multiple stakeholders. ‘I think everyone has a different idea of what a PHR is and what a patient portal is. If you ask a few people, you’ll probably get three different answers,’ one informant said.

Technical challenges were one of many issues mentioned at baseline, but had become the second most common category at follow-up (figure 1). Interviews suggested that over time, informants gained experience in addressing technical barriers. The most serious technical barrier was insufficient application of data standards, which was revealed when the organizations tried to process HIE data for inclusion in a PHR or in analytics. For example, laboratory data were only infrequently coded with LOINC codes. Equally serious were challenges to accurately identifying patients within and across institutions.

Legal issues also gained prominence over time. An issue mentioned frequently was the complexity of operationalizing access to data as defined by HIPAA and other laws. For example, federal clinical laboratory improvement amendments laboratory regulations³⁷ were cited by informants to explain why laboratory results had to be delivered to the ordering physician rather than to patients.

Organizational and management issues were mentioned frequently at baseline but had declined in importance at follow-up, when staffing and structures had become more settled. For example, early in the process, the RHIO encountered difficulties with vendors, ranging from contracts falling through to vendors being acquired or discontinuing products. By follow-up, these vendors had been replaced with other vendors or in-house developers. Remaining challenges pertained to coordination between the vendors and the RHIO, as well as the need for mutual learning between them.

Adapting to the culture of healthcare as well as to multiple cultures of patients remained a constant concern. For example, the RHIO were unable to provide translations into the dozens of languages spoken by patients in New York City. The federal ‘meaningful use’ EHR incentive program (classified as a policy/strategy issue) was mentioned occasionally. For example, one informant believed that it became harder to engage vendors because they had little time or energy to devote to projects unrelated to ‘meaningful use’, and stage I of meaningful use did not require the specific functions under development by the RHIO.

Economic concerns mentioned at baseline were less prominent at follow-up after grant resources had been allocated. Other domains (behavior and education) were less frequently mentioned at either baseline or follow-up; several informants suggested that these issues might become more common after roll-out as organizations sought to increase adoption.

Sociotechnical facilitators and solutions developed

Informants also identified facilitators to the technology development, implementation, and adoption. At baseline, the most commonly mentioned facilitators involved organizational/managerial strengths (figure 2, left). These included organizational experience with HIE technologies and vendors, access to sources of expertise within and outside the RHIO (including stakeholder boards and consultants), strong relationships with member organizations (hospitals and other healthcare providers), and, for one RHIO, a close partnership with a vendor. Notably, by follow-up, organizational strengths were infrequently mentioned, and instead the functionality of the new software was the most common facilitator. This was because by this point, RHIO had changed their projects in response to the challenges, and informants now identified the new functionality as a facilitator for success. For example, one RHIO considered the consumer-friendly user interface developed by its vendor as a facilitator to adoption.

Figure 2.

Types of facilitators and solutions also changed over time. For both facilitators and solutions, organizational factors were considered important at baseline, whereas software functionality was considered most important at follow-up.

The most common solutions involved narrowing project scope (dropping functionality, reducing the number of users, or retaining manual processes instead of developing automated ones), and reallocating resources (including bringing development work in house rather than contracting it out, as well as reprioritizing). ‘We focus on pieces and say, ok we're going to do it this way to get it out there and come back later when we have a revenue stream to enhance this’, one informant said.

Just as with the facilitators, the solutions (figure 2, right) were most frequently described in terms of organizational/managerial strengths at baseline, but at follow-up were most frequently linked to software functionality. For example, at baseline several informants described iterative development methods (an organizational/managerial factor) as a solution for the challenges they faced. By follow-up, development was complete and informants were confident that the functionality had addressed the challenges. Examples of solutions are described next.

Theme 1: managing patient identity and access

The goal of broad access to the technologies was complicated by problems with patient identification. Patients had multiple records across and even within healthcare institutions providing data to the RHIO. Master patient index algorithms often produced two or three matches for a single patient. Clinician users could review a minimal dataset about several potential matches to identify the appropriate records, but it was recognized as a privacy breach (as well as highly confusing) to expose these matches to patients. In addition, allowing patients to self-register could allow them to access others’ data.

To address the problems, two RHIO developed semi-manual enrollment processes that allowed more control over registration. In one, registration was conducted in person by RHIO staff who verified the patient’s identity and identified the correct records. In another, participating providers invited patients to enroll. These processes were considered necessary until better solutions could be found, but one informant noted, ‘That's not really scalable.’

Theme 2: providing value-added services for patients

Most informants believed that value to patients would come not solely from access to data but from services and functions to help them use and communicate about it. For example, the organizations hoped to parse HIE data so that they could be organized and de-duplicated, allowing patients to view trends over time rather than a series of information downloads. Another example of a service that was believed to add value to the patient was a set of algorithms that would analyze HIE data in order to send patients reminders about preventive care or alerts about medications. The informants generally believed that these sorts of services represented the primary way the data could be made be useful to patients, who were unlikely to understand the raw medical data. However, most of these ‘added value’ functions had to be significantly scaled back because of serious problems in data standards. Two projects ended up relying on batch information downloads that were not further parsed for patients. The remaining project found that the analytics package, developed to run on well-structured claims data, was unable to process some of the HIE data. In the end, only a subset of the HIE data was used in the analytics. One vendor representative noted, ‘Everybody is coding to a different standard. Just exchanging data is easy, but exchanging data and using it for something that is clinically valid is not easy.’

Theme 3: supporting healthcare organization workflow and processes

All three organizations expressed commitment to empowering patients by giving them access to HIE data. However, over time, they also became increasingly aware that these activities might interfere with the activities of healthcare organizations rather than supporting them. For example, informants were concerned about giving patients access to potentially sensitive data. By follow-up, one RHIO leader said, ‘The feedback we get from physicians really relates to one issue: the patient seeing information before I do.’ As a second example, it became clear that patients would have to contact the contributing healthcare organizations, not the HIE provider, if they wanted to correct data inaccuracies. However, it was not always clear how to help patients understand this complexity of HIE data, and how to facilitate patient communication with healthcare organizations in a way that was understandable and convenient for both the patient and the healthcare organization. As a result of challenges such as these, the HIE developers reduced the amount of data flowing to the patient, through measures such as permitting patients to view encounters, allergies, medications, and claims but not laboratory results. According to a RHIO leader, ‘We hope to show the patient as much data as possible, but not everything, because it is believed in the clinical community that that is not appropriate.’ Another solution was to replace automatic data releases with approaches in which healthcare providers selectively released data to patients. One informant said, ‘Let’s get out of the relationship between patient and physician as far as delivering information, and get into the position of enabling communication, but not being in the middle of it and being the traffic cop.’

Theme 4: protecting privacy and operationalizing consent

Novel consent issues were raised by the new technologies. One informant said, ‘There’s just a whole lot of frontier stuff that I hope we will all learn from.’ For example, state and federal law provided no guidance about whether providers should be able to use the technologies to make initial contact with patients with whom they did not have a relationship, and vice versa. In addition, it was not clear whether data entered by the patient should be released to providers who had been consented to view data supplied by healthcare organizations. To address these gaps, one RHIO decided to permit patients and providers to contact each other through RHIO technology. The RHIO that permitted patient-entered data added a consent asking patients which providers should be authorized to view these data.

By follow-up, the RHIO that was using HIE data for analytics had also encountered a new consent-related question. It was now possible for a healthcare provider to receive patient consent to view clinical data, while not having consent to view patient-entered data. To prioritize privacy, the RHIO reworked its analytics approach to ensure that providers did not indirectly learn about hidden patient-entered data by receiving a preventive care reminder or a medication alert generated from those data.

Discussion

This prospective qualitative study provides a novel perspective on how sociotechnical factors previously identified as important to health IT success or failure shaped the development of new patient portals designed to integrate data from several sources. Over the development period, projects designed to provide patients with one-stop access to their own data from multiple healthcare organizations were significantly modified. These modifications reduced the amount of data available to patients, instituted enrollment processes that limited the number of patients who could be authorized to use the system, and in one case involved data access via multiple steps (downloading HIE data into a patient portal) rather than a single step. These modifications resulted in narrowed project scope and more limited patient access to data.

The multifactorial reasons for these modifications suggest that broadening patient access to their own healthcare data will continue to be challenging. Technical barriers were serious, including challenges in managing patient identity in the absence of a national unique identifier, as well as inconsistent use of standards by the many healthcare organizations that contributed data to HIE organizations. Organizational and managerial factors included events such as software vendor mergers and product discontinuations, as well as relationships with individual healthcare organizations and organizational structure to support software development. Factors related to healthcare culture and law led the organizations to rethink patient data access policies to ensure that HIE data releases appropriately supported patient–provider relationships and care processes. These findings suggest that technical development alone is unlikely to achieve the goal of providing patients with seamless access to personal data from multiple institutions to better understand and manage their healthcare. Continued policy development is likely to be needed, including stronger promotion of data and messaging standards, and efforts will continue to be affected by continued evolution of the health IT marketplace. Future research in this area that included data from national and state policy makers, legal experts and lawmakers, and consumers could better represent the full spectrum of sociotechnical interactions that contribute to technology development.

This study makes several contributions. The Brender sociotechnical framework used here³¹ was developed not through primary data collection but through expert opinion, and the current study is the first to demonstrate that when it is applied to qualitative data on the development of new health IT, its categories are all employed and no new categories emerge. Thus, this study provides empirical validation of this expert-derived framework. In addition, this study extends that framework in important ways. First, it demonstrates that the sociotechnical framework identified primarily through analysis of clinical information systems at single centers is equally applicable to the development of consumer technologies developed at the regional level. The new technologies studied here were intended to aggregate data from many healthcare organizations, adding a significant level of technical and legal complexity that was not explicitly addressed in the original Brender framework. Furthermore, the framework was originally developed in the context of physicians and other highly educated employees within a single healthcare organization, who shared common organizational goals and operated under a single organizational policy. The members of the general public for whom the technologies in the study were developed represent the gamut of educational levels and are unlikely to share common goals. Nevertheless, we found that the framework was easily extended to cover the development of these consumer technologies.

Finally, this study is unusual in that we collected qualitative data prospectively during technology development, which minimizes biases introduced by hindsight. By comparing early and late datasets, this study provides a nuanced view of how the prominence of different sociotechnical factors can change over time. For example, we found that although technical barriers were mentioned at baseline, they became a far more important theme after 8–13 months of development, suggesting that their importance may not have been fully recognized at the beginning of the project. It seems likely that in a purely retrospective study with data collected at the end of the project, informants might have mistakenly recollected that technical barriers were recognized as equally critical throughout the project lifespan. As recently recommended by an AMIA consensus paper, it would be helpful for future research to examine processes throughout the health IT lifecycle,³⁰ and prospective qualitative data collection would be one method for accomplishing this goal.

Few other studies are available looking at the consumer health IT development process from a sociotechnical perspective. One example is the work of Gaskin and colleagues,³² who applied a sociotechnical viewpoint in examining the process of adapting a commercially available PHR to interface with a hospital EHR. Barriers they identified included ones similar to ours, including difficulties with coordination (in our case, relationships with commercial vendors) and differing conceptions of PHR functions (in our case, reflected in the prominence of the ‘developing functionality’ code). However, technical barriers encountered in the projects we studied appeared more severe than the ones documented by Gaskin et al.³² This can be attributed to the challenges of aggregating data across multiple healthcare institutions, with resulting patient identity and data standards challenges.

After grant funding concluded, changes in the landscape further affected the portal projects. The public–private partnership leading HIE policy in New York, the New York eHealth Collaborative, announced plans in 2013 for a statewide patient portal to access HIE data. Two RHIO in the study currently maintain their portals but halted marketing, partly in anticipation of the statewide portal. In addition, as mentioned earlier, the third RHIO in the study underwent a merger before beginning to market its portal. These consolidations provide continuing evidence of rapidly changing organizational and policy-level factors affecting health IT development in the consumer domain.

Limitations

As all three projects in this study were grant funded, the organizations had protected resources for project development, as well as strong incentive to meet project goals. This may limit applicability to commercially developed products or initiatives developed by organizations using in-house resources, in which economic issues may be more salient.³⁸ Another limitation is that the study followed the projects through development and early implementation, without long-term follow-up to determine sustainability or long-term adoption and use trends. In addition, this study was conducted in a state with relatively advanced HIE infrastructures as a result of extensive state-level funding.²¹ The analytic plan focused on comparisons between baseline data and follow-up data, and thus emphasized differences over time rather than differences between organizations. Snowball sampling was employed, which may result in oversampling cooperative participants as well as members of the social networks of the initial interviewee sample.³⁹ Interviews were conducted by two researchers, who subsequently conducted coding and analysis in tandem, with member-checking of results by several of the key informants, but nevertheless biases or inaccuracies in data interpretation are possible.

Conclusions and implications

In this prospective qualitative study, we showed that sociotechnical factors previously identified as important in health IT success and failure helped to shape the evolution of three novel consumer informatics projects. These projects all sought to provide patients with one-stop access to aggregated data collected from multiple healthcare organizations. Barriers to achieving this goal were multifactorial, including technical issues, rapidly changing organizational factors, issues in supporting the culture of healthcare and adapting to the culture of patients, and others. This implies that there will be no simple solution, and that progress will require continued policy development in multiple domains.