Bridging Clinical Researcher Perceptions and Health IT Realities: A Case Study of Stakeholder Creep

Abstract

Purpose

We present a case report detailing a challenge in health information technology (HIT) project implementations we term “stakeholder creep”: not thoroughly identifying which stakeholders need to be involved and why before starting a project, consequently not understanding the true effort, skill sets, social capital, and time required to complete the project.

Methods

A root cause analysis was performed post-implementation to understand what led to stakeholder creep. HIT project stakeholders were given a questionnaire to comment on these misconceptions and a proposed implementation tool to help mitigate stakeholder creep.

Findings

Stakeholder creep contributed to an unexpected increase in time (3-month delayed go-live) and effort (68% over expected HIT work hours). Four main clinician/researcher misconceptions were identified that contributed to the development of stakeholder creep: 1) that EHR IT is a single group; 2) that all EHR IT members know the entire EHR functionality; 3) that changes to an EHR need the input of just a single EHR IT member; and 4) that the technological complexity of a project mirrors the clinical complexity. HIT project stakeholders similarly perceived clinicians/researchers to hold these misconceptions. The proposed stakeholder planning tool was perceived to be feasible and helpful.

Conclusions

Stakeholder creep can negatively affect HIT project implementations. Projects may be susceptible to stakeholder creep when clinicians/researchers hold misconceptions related to HIT organization and processes. Implementation tools, such as the proposed stakeholder checklist, could be helpful in preempting and mitigating the effect of stakeholder creep.

1. INTRODUCTION

Between 2008 and 2014, the adoption of electronic health record (EHR) systems among non-Federal acute care hospitals increased from 9.4% to 75.5%,[1] fueled by Meaningful Use and the promise of EHRs for improving healthcare quality.[2, 3] EHRs can offer a means for scalable, sustainable implementation of health system interventions and quality improvement (QI).[4–6] However, important challenges remain for researchers, clinicians, and health information technology (HIT) staff implementing such projects in healthcare delivery organizations.

While the challenges of first installing an EHR are well-studied,[3, 7–11] much less research exists on project risk factors related to modifying already established EHRs to implement new interventions or process improvements. Previous research has highlighted how clinical information systems projects can fail due to scope creep,[12] defined as “Not thoroughly defining the scope of the new system and the requirements before starting, consequently not understanding the true work effort, skill sets and technology required to complete the project.”[13] Others have reported delays in the implementation of clinical decision support systems due to insufficient engagement of stakeholders who were not invited or who chose not to get involved.[14] In contrast, we present a case illustrating stakeholder creep, an under-studied EHR QI implementation challenge that can lead to project delays and unchecked increases in stakeholder involvement beyond a clinical or research team’s expectations and budgeted resources. Analogous to scope creep, we herein define stakeholder creep as “Not thoroughly identifying which stakeholders need to be involved and why before starting a project, consequently not understanding the true effort, skill sets, social capital, and time required to complete the project.” When applying alternative definitions of scope creep, the analogy holds. For example, when scope creep is defined as “Failure to define and maintain original success criteria,” [15] stakeholder creep is then defined as “Failure to originally identify and subsequently maintain the set of parties needed for success”. Similarly, when scope creep is defined as an expansion of the set of features needed, stakeholder creep is then defined as an expansion of the set of parties needed as another potential source for project failure, or cost and time overages.

While likely familiar to many HIT teams already, stakeholder creep may surprise and complicate clinical and research partnerships. This study analyzes the impact of stakeholder creep, identifies potentially causal misconceptions and knowledge gaps, and proposes a tool to anticipate necessary stakeholders and mitigate creep.

1.1. Case description

Our team of clinical researchers implemented Quit Connect Health, [16] a QI intervention providing referrals to state-sponsored tobacco quit lines from specialty (rheumatology) clinics. Nurses and medical assistants (MAs) were given workflows to assess all patients’ smoking status, current smokers’ readiness to quit or cut back, and ready smokers’ willingness to be electronically referred to the quit line to receive free counseling and cessation medication. The project involved a change in nurse and MA roles (providing tobacco-cessation education and quit line referrals), workflows (added assessment and referral steps), and technology (EHR-based decision support alerts and a bidirectional interface with the quit line vendor).

Our team expected that the development and implementation process for Quit Connect Health would be similar to BP Connect Health,[17, 18] a similar project for follow-up of specialty clinic high blood pressures that we completed a year prior. We estimated needing only the input of a few EHR HIT stakeholders and that the implementation would be straightforward given our familiarity with the process. Planning for Quit Connect Health with health system leaders began in May 2015 and aimed to go live in January 2016.

The actual go-live date was late April 2016, three months later than the goal and seven weeks later than an interim revised goal. The number of stakeholders grew beyond what was expected, leading to unanticipated project work and time demands. Despite experienced intervention and HIT teams and the use of standard project management tools, including a project charter and Gantt chart, something had been missed during the initial planning process, motivating a post-implementation analysis.

2. MATERIALS AND METHODS

2.1. Data sources and methods: stakeholder creep

Primary project data sources included all available hand-written and electronic support documentation (n > 136 documents), relevant emails from the implementation period, and health system personnel effort documentation related to the initial estimate and final development time logged by the HIT team.

The post-implementation analysis investigated the scope and characteristics of the stakeholder expansion to better understand who was brought into the project, when, and why. A stakeholder group was defined as any committee or group of people with an organizationally designated role that contributed toward work or oversight of the project. Stakeholder groups were classified as IT or non-IT. Data included stakeholder group name, role (design/planning, approval/oversight, or build/support), and the date each was identified as a new stakeholder.

2.2. Data sources and methods: perception misalignment

To understand what led to underestimating the number of stakeholders, the core research intervention team performed a root cause analysis. We first reviewed meeting agendas and project management Gantt charts from throughout the project. We next identified root cause baseline misconceptions regarding EHR HIT organization that negatively impacted the implementation by asking 5 Why’s and creating a fishbone diagram [19] examining five work system domains [20]. For example, part of our underestimation of the total time needed for implementing changes to the EHR resulted from assuming that the single EHR HIT committee that we had worked with on a prior, clinically similar project would be able to implement all of the changes for Quit Connect as well. In reality, because the EHR functionalities impacted by this project were different, different EHR IT groups needed to be involved. Our misconception was that EHR HIT members each know and are capable of changing any functionality within the EHR. In this manner, we identified four different key misconceptions: 1) that EHR IT is a single group; 2) that all EHR IT members know the entire EHR functionality; 3) that changes to an EHR need the input of just a single EHR IT member; and 4) that the technological complexity of a project mirrors the clinical complexity.

To assess whether these researcher/clinician misconceptions were commonly encountered by HIT members during other EHR HIT project implementations, we then administered an anonymous questionnaire (Supplementary File 1) to the core EHR HIT project team members (n = 11, including IT analysts and two physician informaticists), assessing their experiences of discordance between researcher/clinicians’ perceptions and HIT members’ perceptions. The first part of the questionnaire presented the four potential misconceptions and corresponding realities identified during the root cause analysis as two anchors on a 10-cm continuous scale. Participants were asked to mark where they felt the beliefs of most clinicians/researchers were positioned versus those of most IT members. The positions of these marks were compared across respondents using paired t-tests.

The latter part of the questionnaire requested responses to a project management tool that the intervention team developed after root cause analysis to prospectively identify appropriate HIT stakeholders before an EHR QI project. The tool was a checklist of the possible stakeholder groups at our organization for an EHR-based QI project implementation, along with a brief note regarding their responsibilities (condensed version in Table 1; see https://www.hipxchange.org/QuitConnectHealth for downloadable checklist with team descriptions). We developed this checklist iteratively, starting with our initial project scoping charter and project management Gantt chart and refining the checklist by reviewing meeting agendas and email correspondence. The questionnaire presented condensed and detailed tool versions, asking respondents to assess their feasibility and helpfulness on a 10-point Likert scale, where 10 was highly feasible or highly helpful, respectively.

Table 1.

Condensed stakeholder involvement checklist

HEALTH SYSTEM IT □Decision support team □Ambulatory team □Orders management team (inpatient) □Results team □Clinical documentation team (inpatient) □Hospital outpatient department team □Radiology team □Imaging warehouse team □Cardiology team □Anesthesiology team □Surgery team □Medication team □Patient portal team □Patient identification management team □Health information management team □Coding team □Long-term care and home health team □Internal messaging team □Hospital billing team □Physician billing team □Interface team □Network team □Security team □System environment manager □Server team

HEALTH SYSTEM ADMINISTRATION □Ambulatory, inpatient, or primary care leadership group □Clinical division/department chair □Clinician champions □Pharmacy leadership group □CMIO/clinical informatics □Purchasing director □Medical board and/or delegation protocol committee □Health system legal team □Clinic staffing manager □IT resource allocation group

REPORTING □Reporting team

TRAINING □Health system education and training team

OTHER HEALTH SYSTEM TEAMS □Laboratory

EXTERNAL □Community health partner

3. RESULTS

3.1. Stakeholder creep

Figure 1 illustrates the expansion of stakeholder involvement. Although the research team anticipated interacting with three HIT teams, the project ended with 37 distinct stakeholder groups of which two primarily contributed to design and planning, 14 to build and support, and 21 to approval and oversight. There were several stepwise spikes in the number of stakeholders as the initial HIT groups added stakeholders for EHR modifications and approvals that had not been scoped. In some cases, HIT experts requested input from new stakeholders as they discovered further concerns or alerted us to additional organizational processes. Stakeholder creep continued until one month before go-live as technical aspects of the project evolved. The number of work hours by HIT staff needed for the project similarly grew from the initial estimate of 202 hours to a final logged time of 340 hours, a 68% increase.

Figure 1. Expansion of the set of stakeholder groups involved over the course of the project.

Each dot represents discovery of a new stakeholder group for the project. Open dots represent non-IT stakeholder groups while black dots represent IT stakeholder groups. The background color behind each dot represents the group’s primary role in the project (black = design, white = approval, gray = build). The vertical dashed line on the right represents the actual go-live date of the project.

3.2. Perception misalignment

The root cause analysis identified four main misconceptions held by our intervention team about EHR HIT (Table 2) that likely contributed to the eventual stakeholder creep. The response rate to the questionnaire was 82% (n = 9 of 11 mailed questionnaires; 1 incomplete). Perceived beliefs of researchers/clinicians and EHR HIT informatics staff differed significantly on all four items. As hypothesized, respondents on average perceived researchers/clinicians’ beliefs to be closer to the “misconception” and EHR HIT informatics staff’s beliefs to be closer to the “reality”. The maximally discordant misconceptions were that “EHR IT is a single group” and that “all EHR IT members know the entire EHR functionality.”

Table 2.

Project members’ perceived differences in the beliefs of researchers/clinicians versus IT members about EHR IT

Misconception (1)		Reality (10)	Researchers/clinicians* mean (SD)	HIT team* mean (SD)	Diff. mean (SD)	p-value
EHR IT is a single group	↔	EHR IT consists of multiple, specialized groups	2.76 (3.10)	9.50 (0.53)	6.39 (2.99)	0.003
All EHR IT members know the entire EHR functionality	↔	EHR IT members specialize more in some parts of the functionality than others	3.66 (2.38)	9.05 (0.91)	5.16 (2.59)	0.005
Changes to an EHR need the input of just a single EHR IT member	↔	Changes to an EHR need the input of many EHR IT members	4.76 (2.55)	9.37 (0.60)	4.63 (2.86)	0.01
The technological complexity of a project mirrors the clinical complexity (e.g., if the workflow is simple, IT build will be simple)	↔	The technological complexity of a project often differs from the clinical complexity	3.39 (3.00)	7.35 (2.86)	4.88 (4.16)	0.03†

^*The value represents the distance (cm) from the left end of the 10-cm visual analog scale, where the left end was the misconception and the right end the reality as identified by the researchers.

^†When physician informaticists were excluded from analysis, the difference was no longer significant (p= 0.09).

On the 10-point scale, respondents rated the stakeholder tool as highly feasible (mean 8.57 ± 1.40) and helpful (8.71 ± 1.38) to plan projects within the same organization. The tool was rated moderately feasible (6.86 ± 2.48) and helpful (7.14 ± 2.27) in a scenario when a new organization is provided a pre-populated checklist prior to starting an implementation project.

Open text responses regarding the feasibility and helpfulness of the proposed tool were positive. Several respondents expressed optimism about the tool helping identify stakeholders and project components. One wrote, “Identifying stakeholders is a critical step. A stakeholder register would be extremely helpful.” Some respondents expressed concern about the tool’s helpfulness at other organizations: “Due to the complexity of organizations, I’m not sure the same types of groups would be responsible/involved with certain components.”

4. DISCUSSION

4.1. Effect of stakeholder creep

Rapidly expanding stakeholder involvement in this QI implementation led to several key learning opportunities. One key lesson was the time and resource cost of underestimating stakeholder involvement on an EHR-based QI implementation. More stakeholders meant more time to complete the unanticipated work steps, more time and effort for the project team to coordinate their roles, and more delays due to competing group demands, meeting schedules, and the various processes each group required. Subjective morale of both the HIT team and the clinician researchers also dipped due to unanticipated additional effort and time.

In part, the clinical researchers held misconceptions that likely contributed to this lack of anticipation. These misconceptions shared a common theme of underestimating the complexity of EHR IT. Just as medicine consists of a series of specialists, so does the EHR team, with each member focusing on separate but related functionality. This organizational attribute is known in the literature as “functional differentiation”, and measures of it have mixed significance across studies of successful adoption in health care.[21] When a project requires changes across a spectrum of functionality, as was the case with Quit Connect Health, the number of stakeholders can expand quickly to support each piece of functionality. The broader the spectrum of functionality involved, the more oversight and approval is needed for the project to ensure safe implementation, which was also seen in our study with over half of the stakeholder groups representing “approval” stakeholders. For the average clinician researcher, these stakeholders are particularly likely to be unanticipated if the project begins prior to a structured identification of stakeholders with the guidance of senior staff familiar with the HIT organization.

The stakeholder checklist tool that we developed aims to fill this gap by supporting needs analysis for HIT projects before they begin. Senior HIT leaders were involved in initial project resource allocation approval but less involved with project coordination due to the number of organizational HIT demands. Retrospectively, senior HIT leaders agreed that the stakeholder checklist would be a useful tool for them and front-line project managers to prevent stakeholder creep.

Our finding that the expanding number of stakeholders was associated with delays and increased time is consistent with an international Delphi study of software project risks, which identified the number of organizational units involved in a project as one of five key risk factors.[13] Although having necessary organizational structures in place for HIT projects can promote project success, factors inhibiting success include organizational inflexibility and poorly fitting organizational domains.[22] Our findings add to the list of inhibiting factors ambiguity about organizational structures, roles, and relevance to a project. While project scope ambiguity (often called “scope creep”) is a well-recognized risk factor, ambiguity about which stakeholders need to be involved in a project, or “stakeholder creep”, has not been formally recognized among risk factors influencing the success of clinical informatics projects, although it is an issue many practitioners and HIT staff know from experience.[12] This risk factor may be particularly relevant when organizations undergo rapid expansion, integration, or other organizational changes, as occurred here.

4.2. Mitigating stakeholder creep at the project level

Stakeholder creep as a risk factor could be mitigated if all clinician researchers or innovators thoroughly understood the technical and organizational complexity of HIT systems, but such a dissemination of knowledge will take time. Until such HIT familiarity is commonplace among medical professionals, it is important to provide tools to help address this knowledge gap and the problems that it can cause. Our stakeholder planning tool aims to address stakeholder creep by providing a checklist of available stakeholders and their roles to be prospectively filled out by the clinician/researcher, clinical informaticists, and HIT staff at the onset of a project. This tool facilitates stakeholder coordination and early engagement and provides a visual aid for educating researchers and clinicians on the organizational structure and spectrum of EHR functionalities potentially affected by their project. This visualization could align expectations for an EHR-based implementation and promote better understanding of EHR HIT complexity (Figure 2). To the extent that HIT requires a mutual transformation of a health care organization and its EHR,[9] such opportunities to bridge the knowledge gap between medicine and HIT are crucial for productive collaboration.

Figure 2. Bridging clinical researcher perceptions and health IT realities.

Clinical researchers may perceive health IT staff as a single group knowing the entire EHR functionality. In reality, health IT staff are organized into differentiated teams, each focused on specific EHR functionality. Our stakeholder checklist tool aims to help align perceptions with reality, helping clinical researchers and health IT teams partner to anticipate needs and mitigate stakeholder creep.

The tool may also facilitate the creation of cross-functional IT teams formed around a single project rather than specific pieces of functionality. The latter approach is often the result of the initial organization of IT teams around specific modules during the initial EHR implementation. As organizations mature and move into optimization, the development of cross-functional teams organized around operational domains or projects may be of greater benefit. Our organization is currently pursuing such changes, and this tool may help inform the compositions of the new teams. In another organization implementing Quit Connect, we recently tested the tool where it helped reduce implementation time to under three months.

4.3. The role of the organization

Our findings suggest that EHR modification is a social process where the set of stakeholder groups that may claim authority over aspects of a project is variable and path-dependent. However, given that this social process occurs within the context of an organization, there is an opportunity to mitigate stakeholder creep at the organizational and project levels. One way to do so is by engaging clinical leaders in IT governance and rational process design. [23] Furthermore, maturing the enterprise architecture to the “rationalized processes” stage can support standardized processes for EHR modification projects such that clinical/research teams would experience a single interface rather than needing to navigate the entirety of the HIT organizational structure. [24] Finally, stakeholder creep may be reduced by complementing project-level management with organizational-level governance using checklists such as our proposed tool.

4.4. Study limitations

Despite the strengths of a mixed-methods approach, there are limitations to this study. First, this study represents only one team’s experience; the misconceptions identified here and stakeholder creep would benefit from study in other organizations and projects. Second, different organizations may have different EHR IT structures, limiting transferability of the tool. We suggest that the tool be tailored to a healthcare system’s EHR HIT and operational structure before use. Overall, however, the consistency of the questionnaire responses indicate that our findings may reflect common underlying challenges in HIT implementation.

5. CONCLUSIONS

This case study assessed the impact of “stakeholder creep”, the unanticipated growth in stakeholders involved in an EHR QI project. This was linked to several misconceptions held by the clinical research team regarding HIT organizational complexity. We developed a stakeholder identification and selection checklist tool that could align expectations in future projects as healthcare organizations transition from implementing EHRs to innovating with them. HIT and HIT staff provide a valuable infrastructure for optimizing an increasingly complex clinical world. Only by strengthening the partnerships between researchers, clinicians, and HIT will we fully realize the potential of EHRs to improve quality and safety.

Summary table.

What was already known on the topic

• Scope creep is a risk factor for HIT project implementations

• Engagement of stakeholders is crucial for HIT project implementations

What this study added to our knowledge

• Stakeholder creep is a risk factor for HIT project implementations that can cause unexpected increases in time and effort

• Stakeholder creep may be caused in part by clinician/researcher misconceptions regarding HIT complexity

• Project planning tools, such as a stakeholder checklist, may help mitigate stakeholder creep by facilitating timely prospective identification and engagement of stakeholders

Highlights.

• Stakeholder creep is a risk factor for HIT projects analogous to scope creep.

• Clinician/researcher misperceptions of HIT teams may cause stakeholder creep.

• A stakeholder involvement planning tool is proposed to mitigate stakeholder creep.