Table 2.

Quotes supporting restraining force themes.

Barriers / Restraining Forces
EHR implementation variation	… multiple departments, multiple analysts, multiple clinicians involved … configuring Epic, and as a result you may have the same data element representing multiple data concepts… We observed seeing seven different variations of length of stay, and if you start doing analytics on length of … imagine how poor quality of analytics you’re going to have (Expert 1) …using mammogram as an example, if you pull mammogram against all EHRs you’re going to get anything from a yes to a date to BI-RADS score to a full narrative. It could be anything, and from a CDS standpoint, there’s nothing there that you can work with. (Expert 2) The way people implemented it at a one site, the resources might be modified for that site compared to another for the same resource specification. (Expert 3) Healthcare has the challenges of… one piece of data meaning one thing in your place and meeting another thing in my place, and that semantic interoperability is the promise of FHIR. Given the variability of implementation, it’s the challenge too. (Expert 3) …everybody’s order catalog is different ... it’s hard to find or to determine exactly what to hook up to. Then you have to fit into the logic flow. (Expert 3)
Limited EHR vendor support	It is really in the hands of the EHR vendors how well and how reliably consistent they are with FHIR standard, and what we’re observing is that they’re not… they may have multiple versions of the same standard implemented… even if the vendor has good intentions (Expert 1) The biggest challenges we’re seeing is how those standards …on the FHIR side is interpreted very differently from vendor to vendor. (Expert 2) Right now, people oftentimes take a hybrid approach. That is they may have their own data store in their own format and whatnot, but on top of that, they will add a FHIR-edge server… which can help that site do transformations. (Expert 3) At the first site … when we had to go live there weren’t enough [FHIR] services available (Expert 4)
Ontology Variation	FHIR is not actually explicit about terminologies … Majority of cases they’re pretty standard terminology, like ICD-10 codes… but as we are observing it doesn’t prevent people from using other terminology codes that are not directly compatible between two systems. (Expert 1) …rather than us allowing multiple observations within the system to have the same LOINC code they started using a custom coding system (Expert 2) How can they get lab vendors to send everything consistently, regardless of what vendor it’s coming from, in a way that it can be loaded and stored consistently… and pulled consistently? (Expert 2)
Workforce knowledge	Both FHIR for data as well as FHIR for the computable practice guideline. Right now, there aren’t many people who can do the whole thing, start with paper [guideline] and end up with code (Expert 3) I would say at our organization the IT folks know how to use FHIR … but two years ago when we were integrating services… it wasn’t that way (Expert 4) The expertise needed to develop and interface is still to niche. We have trouble staffing for it. They said use native functionality of our EHR to implement this tool, so that’s what we did. (Expert 5) I think we as an organization have started turning more towards how do we develop people in-house … how do we build a workforce that’s knowledgeable of healthcare data standards, knowledgeable of clinical decision support (Expert 5)
Limited data for testing	…realistic sample patients to test … This is not a problem that can be solved by one institution or another. To really avoid the bias problem in either design or implementation … or pathway specs, you want to have robust big data to examine and test with, and that’s still tricky. (Expert 3) …the problem with these test environments is the data is all made [up]. It’s not the same as the live environment, so sometimes things don’t work in the test environment, but they would work in the live environment, but it’s better to flush it out and make sure that it’s not a problem with the code and it’s not just something with that environment. (Expert 4) …have more of those environments, including with realistic patient data. Maybe synthetic data is a way to think about solving this. (Expert 5)