Primary Care Practitioners’ Perceptions of Electronic Consult Systems

Key Points

Question

What is the association between implementing electronic consult systems for specialty requests (eConsult) and perceptions of frontline primary care clinicians in safety-net health systems?

Findings

In this qualitative study of 40 primary care clinicians, interviewees consistently endorsed that eConsult shifted some burden of specialty work to them. Many felt that this burden was worth the effort to improve timeliness of care and their ability to manage specialty conditions, while others were frustrated by the broadened responsibility.

Meaning

Though eConsult is associated with improved specialist access, it simultaneously created new challenges for clinicians to manage; delivery transformations can create new barriers as they solve existing problems.

Abstract

Importance

Safety-net health systems across the country are implementing electronic consult (eConsult) systems in which primary care practitioners (PCPs) submit all requests for specialty assistance electronically to be reviewed and discussed with specialists. Evidence suggests that eConsult systems can make significant improvements in specialty access, but the outcomes of these systems for frontline PCPs is poorly understood.

Objective

To understand PCP perceptions of the results of eConsult initiation on PCP workflow, specialist access, and patient care.

Design, Setting, and Participants

Qualitative interviews were conducted from December 1, 2016, to April 15, 2017, with 40 safety-net PCPs in Los Angeles County who use the Los Angeles County Department of Health Services (DHS) eConsult system. Interviewees were recruited to include diversity in PCP type, practice setting, and employer (DHS employed vs DHS affiliated). Participants were interviewed about their perceptions of clinical workflow, access to specialists, relationships with specialists, and referral decision making.

Main Outcomes and Measures

Perceptions of the results of eConsult, including positive and negative themes and remaining perceived gaps in specialty care.

Results

Of the 40 participants, 27 (68%) were women; 24 (60%) PCPs performed 5 or more eConsults per week. Primary care practitioners’ perceptions of eConsult clustered around 4 main themes: access and timeliness of specialty care, shift of work to PCPs, relationships with specialists, and eConsult interface issues. Many PCPs praised the improved timeliness of specialist input with eConsult, as well as the added clinical and educational value of dialogue with specialists, particularly compared with the limitations of the prior referral process. However, PCPs also consistently perceived that eConsult shifted some of the work of specialty care to them. Many PCPs believed that this extra burden was worth the effort given the benefits of eConsult, such as improved timeliness of care and ability to manage specialty conditions. In contrast, others were frustrated by the increased administrative burden, broadened clinical responsibility, and restructuring of specialty care delivery.

Conclusions and Relevance

While associated with improved specialty care access, eConsult systems simultaneously created new challenges for PCPs, such as an increased burden of work in providing specialty care. Primary care practitioners varied in their enthusiasm for these workflow changes with diverging perceptions of the same processes. Our findings provide insights on challenges future primary care transformation efforts may face.

This qualitative study describes the perceptions of primary care practitioners on the use of an electronic consult system for specialty referrals in a safety-net health system.

Introduction

Access to timely, high-quality specialty care is a fundamental component of a well-functioning health system, yet safety-net health care professionals face persistent challenges delivering such care.^1,2 Poor access to specialty referrals can result in preventable morbidity and mortality. For example, delayed management of chronic kidney disease is associated with increased mortality and lower access to transplantation.^3,4,5 Access to specialty care is of particular concern for safety-net services, such as federally qualified health centers and public hospitals, that play a critical role in providing care for disadvantaged populations. Lack of timely specialty access is likely associated with worse health outcomes in underserved communities.^1,6,7

To address these challenges, in 2012 the Los Angeles (LA) County Department of Health Services (DHS), the second largest safety-net health system in the United States with 4 hospitals and over 500 000 primary care patients, implemented a transformative series of innovations in specialty care. The main goal of these innovations was to improve an inefficient specialty care system plagued with long wait times and high no-show rates. For example, an internal audit of specialty access in 2011 found that 25% of appointments for gastroenterology and urology had wait times exceeding 9 months. Appointment no-show rates exceeding 40% in specialty clinics were commonplace. The set of interventions included development of extensive expected practices around specialty care requests for common issues,⁸ centralizing scheduling for specialty care, and developing a new system for communication from primary to specialty care named eConsult (electronic consult), which is the primary focus for this study.

eConsult is a process improvement using health information technology to transform the traditional approach to specialty referrals. Instead of submitting simple referral requests with little clinical information to specialty departments, primary care practitioners (PCPs) using eConsult submit clinical inquiries electronically to specialist reviewers through an online portal, opening a dialogue between PCPs and specialists about patient management (a process also described as preconsultation exchange by the American College of Physicians).^9,10 Specialist reviewers then provide recommendations to PCPs, triage the urgency of requests, and often manage requests through electronic dialogue with the referring PCP without a face-to-face visit.

Early evidence suggests that eConsult can reduce wait times, improve physician satisfaction, and even serve as a form of continuing education for PCPs.^{11,12,13,14,15,16,17} Given the potential of eConsult, new systems have been expanding in health systems across the United States, such as Mayo Clinic Health System, the Veterans Administration, and Zuckerberg San Francisco General Hospital and Trauma Center.^18,19,20,21 Despite the growing popularity of eConsult, research on the PCP experience of eConsult transformation on workflow, specialist access, and patient care is limited. Prior work has largely focused on PCP interactions with eConsult in a single specialty^{22,23,24,25,26,27} or used surveys to elicit PCP feedback,^17,28 both of which are unable to capture important nuances of PCP experiences in a large, system-wide delivery transformation.

To inform other health systems considering implementation of eConsult, we qualitatively analyzed 40 semistructured PCP interviews to understand the perceived benefits and harms of eConsult implementation on PCPs. The large-scale adoption and transformative nature of eConsult make it a potentially instructive archetype for other health systems seeking to adopt similar innovations in specialty care delivery.

Methods

Setting and Context

In this qualitative study, we conducted interviews focused on experiences with eConsult among PCPs in LA County who were employed either by DHS-owned facilities or other community health centers (non-DHS clinics). Non-DHS community health centers still rely on DHS system specialists to provide care for uninsured patients in their role as safety-net providers for LA County. LA County DHS oversees 4 hospitals and 20 primary care sites staffed by DHS-employed physicians, with an additional 190 non-DHS community health centers in the region. Prior to eConsult implementation, referrals from both DHS and non-DHS PCPs to DHS specialists were mailed, faxed, or electronically sent forms to specialist department staff. After passing through both utilization management and clinical review, department staff would then schedule appointments for patients on a first-come, first-served basis, generally by mailing patients a letter (ie, no telephone calls to patients) with an appointment time without conferring with the patient about their availability and without clinical input regarding urgency. Dialogue between PCPs and specialists or between schedulers and patients was rare.

The DHS implemented eConsult along with other associated specialty care reforms to improve on this prior system, targeting several challenges described in Table 1. The eConsult intervention consisted of an electronic platform separate from the electronic medical record that PCPs logged into to submit store-and-forward requests for specialty assistance that were reviewed by specialist reviewers. Specialist reviewers were assigned to a set of primary care practices for which they reviewed all eConsults for 6 months or more. Reviewers were trained and expected to reply promptly to all eConsult requests and engage in a dialogue with PCPs to meet specialty care needs in the most effective, patient-centered manner possible. Primary care practitioners and DHS specialists were not directly compensated for entering or reviewing eConsults. After eConsult implementation, eConsult was required for all nonemergent, outpatient consultations to DHS specialists, in contrast to other eConsult systems where its use is an optional pathway for specialty input.^29,30

Table 1. How eConsult and Associated Interventions Aimed to Address Challenges With Prior Referral System^a.

Challenge Before eConsult	Unclear Referral Guidelines/Variation in Referral Practices	No Standard/Timely Way to Ask Questions to Specialist	Difficulty Tracking Patients After Referral	Difficulty Scheduling Appointments	Long Wait Times for Specialty Visits
Relevant intervention	Expected practices to facilitate care by PCPs prior to entering an eConsult	eConsult created platform for PCPs to dialogue with specialists	eConsult system tracks outcomes of in-person visit requests	Centralized scheduling system that directly contacts patient after eConsult dialogue completed	eConsult mandated use to allow specialist triage of referrals, potentially resolving requests without in-person visit
PCP perceptions before eConsult and associated interventions	NA	“You weren't able to have a dialogue with the specialists about what studies that they wanted or what test that they wanted done before their referral, before they evaluated the patient.”	“Before eConsult, it was a little bit of a black hole in terms of what happened to your referral. We wouldn't really get much feedback and a lot of the times referrals would sit for a long time…until months later.”	“Before… our patients would sometimes wait a year or more for an appointment to be scheduled and then they would still wait another couple of months for the appointment to actually happen.”

Abbreviations: NA, not applicable; PCP, primary care practitioner.

^aeConsult is an electronic system for PCPs to submit clinical inquiries to specialists for management via electronic dialogue. Requests can either be resolved through dialogue alone or converted to face-to-face referrals.

LA County DHS implemented eConsult from 2012 to 2015 across primary and specialty care practice sites on a staggered basis. Primary care practitioners employed by DHS completely transitioned to the eConsult model of specialty care for their entire patient panels (DHS PCPs). Non–DHS-employed PCPs at community health centers (non-DHS PCPs) used eConsult only for specialty access for uninsured patients who were enrolled in the My Health LA program (approximately 10%-30% of their panel), and otherwise used existing procedures for referring their insured patients to non-DHS specialists.

Qualitative Procedures

Interview Guide Development

We developed an interview guide informed by prior conceptual work on subspecialty referrals^{14,16,31,32,33} and refined through pilot interviews with PCPs and discussion with DHS leadership (P.G., H.F.Y.). The interview guide focused on PCP perceptions of the results of eConsult implementation on clinical workflow and workload, care coordination and access to specialists, clinician and staff relationships with specialists, referral decision making and comfort with managing a broader array of diagnoses, and satisfaction with specialty access. The final interview guide was constructed for goal interview length of 30 to 45 minutes (eMethods in the Supplement). Interviews were conducted by telephone from December 1, 2016, to April 15, 2017. Participants provided oral informed consent and were offered a $50 Amazon.com gift card via email for their participation. This study was approved as human subjects research by the Harvard Medical School, LA County DHS, and University of Pittsburgh institutional review boards.

Participant Recruitment

Eligible participants were DHS and non-DHS primary care physicians, physician assistants, and nurse practitioners practicing in clinics and hospitals directed by consenting medical directors across LA County. To recruit participants, we contacted 30 clinic medical directors identified by DHS leadership from across the DHS network; 18 medical directors consented to recruitment in their clinics and they forwarded recruitment emails to their clinic PCPs. From these 18 consenting clinical practices, we recruited participants through responses to medical director emails until we reached a prespecified sample size of 20 DHS and 20 non-DHS professionals, aiming to balance representation of PCP types and practice setting (Table 1). We chose this sample size because we expected that it would provide adequate saturation of major themes.

Qualitative Analysis

A trained investigator with qualitative research background conducted all interviews (M.S.L.). After participant consent was obtained, the interviews were recorded and then transcribed and deidentified. The research team then used thematic content analysis to identify the main themes and subthemes. A preliminary codebook was developed based on the first 5 interviews.³⁴ Two investigators then coded each interview (M.S.L. and K.N.R. or M.L.B.) using qualitative analysis software (Dedoose, version 7.5.3). The coded interviews were then compared for agreement and finalized through consensus, with iterative refinement of the codebook. The final results focused on dominant themes shaping PCPs’ attitudes toward eConsult as well as remaining perceived gaps in specialty care. Drawing from these themes, we developed a series of design questions based on interviewee perceptions of the current system and potential opportunities for improvement.

Results

We interviewed 40 PCPs, including 12 internists, 17 family practitioners, and 11 advanced practice clinicians (Table 2). Of the 40 participants, 27 (68%) were women; 24 (60%) PCPs performed 5 or more eConsults per week. In discussing specialty care prior to eConsult implementation, PCPs described numerous challenges, including uncertainty after referral submission, long wait times, and no way to “curbside” or discuss patient care with specialists (Table 1). As one PCP explained: “Prior to eConsult we were writing all our referrals on paper and there was little to no communication …we were pretty much in the dark.”

Table 2. PCP Characteristics.

Characteristic	DHS (n = 20)	Non-DHS (n = 20)	Total (N = 40)
Clinician type, No.
Physicians	15	14	29
Internal medicine	10	2	12
Family medicine	5	12	17
Advanced practice cliniciansa	5	6	11
Practice setting, No.
Hospital-based	17	1	18
Community clinic	3	19	22
Years of experience, No.
<5	10	7	17
5-15	6	9	15
>15	4	4	8
Years in current practice, No.
<2	11	6	17
2-5	4	9	13
>5	5	5	10
Estimated eConsults per week, No.b
<5	4	9	13
5-10	9	5	14
>10	6	4	10
Variable	1	2	3
Race/ethnicity, No.
White	6	8	14
African American	2	3	5
Asian American	6	7	13
Hispanic/Latino	5	2	7
Other	1	0	1
Sex, No.
Male	6	7	13
Female	14	13	27

Abbreviations: DHS, Department of Health Services; PCP, primary care practitioner.

^aAdvanced practice clinicians included physician assistants and nurse practitioners.

^beConsult is an electronic system for PCPs to submit clinical inquiries to specialists for management via electronic dialogue. Requests can either be resolved through dialogue alone or converted to face-to-face referrals.

Three qualitative themes emerged in which PCPs expressed opposing views on the perceived results of eConsult on (1) access and timeliness of specialty care, (2) the shift of specialty care to the PCP, and (3) relationships with specialists. Primary care practitioners also expressed strong views regarding health information technology interface issues and remaining gaps in specialty care delivery.

Access and Timeliness

Primary care practitioners had divided perceptions on the result of eConsult implementation on specialty access and timeliness (Table 3). Some believed that patients received in-person visits more quickly through eConsult (“Now that we’re able to refer patients and they can get their appointments within like a month or even 2 months, it's really nice to not have to send them to the emergency room all the time”) and also noted more timely care for patients who did not require a specialist visit (“If it’s a patient that doesn’t really need much specialist care… it actually saves time because they don’t need to wait for an appointment to see the specialist anymore”).

Table 3. Key Qualitative Themes^a.

Theme	Perceptions on eConsult
	Positive	Negative
Access and timeliness	Improves timeliness, access to specialist input	Delays or obstructs access to visits
	Improves timeliness of face-to-face specialty visits	Specialists are now “gatekeepers”
	Reduces unnecessary visits	Lack of follow-up after eConsult submitted
	Referrals more meaningful	Scheduling issues and scheduling timeliness are problematica
Shift of specialty care to PCPs	Increased comfort managing conditions	Increased burden of prereferral workup
	Specialty interchange is educational	PCP asked to do more testing and evaluation than PCP thinks appropriate for PCP to do
	PCPs more accountable	Administrative burden “not worth the effort”
	Publishing of expected practices assist with referral processa	Expected practices do not help the referral processa
Relationships with specialists	Increased communication with specialists	Some specialists not helpful or responsive
	Specialists as educators	Impersonal communication and lack of trust with specialists
		Specialists as gatekeepers

Abbreviation: PCP, primary care practitioner.

^aThis staement is an item discussed by PCPs that was external to the eConsult platform (eConsult is an electronic system for PCPs to submit clinical inquiries to specialists for management via electronic dialogue. Requests can either be resolved through dialogue alone or converted to face-to-face referrals).

At the other extreme, some PCPs stated that eConsult was an obstruction that slowed access to care: “It's a roadblock for me.” More specifically, PCPs perceived delays in specialty care from the interactive nature of the eConsult approach to referrals. For example, some mentioned delays due to specialist-requested additional testing (“They ask me a bunch of questions that I don't know and I would have to see the patient again to get”). Given these concerns with timeliness, PCPs described instances when their perceived remaining gaps in access to specialty care prompted them to modify their workflow (Table 4), such as sending patients to the emergency department for urgent subspecialty needs.

Table 4. Remaining Gaps in Specialty Care Delivery With Example PCP Responses.

Remaining Gap	PCP Response
	Category	Quotation
Timeliness/access: need to address urgent specialty care needs	Contact specialist department directly	“Anything that we need to do expeditiously, I wouldn't even go through eConsult.a I asked the residents to page the fellows, or someone on the service directly, because there's never any guarantee how quickly you'll have a reply through eConsult if you need something quickly.”
	Send patient to ED or urgent care	“If it's a little more urgent then we have to either send patients to urgent care or the ED.”
		“If I know they need specialist care, then I’ll send them to the urgent care… for them to be seen… and do an internal referral because that’s a lot easier.”
Shift of work to PCP	Exaggerating severity or using key words	“I always felt I had to embellish their medical condition.”
		“I probably adjusted the way I presented the eConsult, and played up things to the ones that I was concerned wouldn't be accepted.”
	Discussing with specialist prior to eConsult	“I’ll sort of curbside one of my specialty colleagues to ask them about the case… to see if it makes sense for me to do any consult and what to actually put in the eConsult.”
	Avoiding certain specialty reviewers	“I think this particular reviewer did not want to see patients. I don’t know. In the future, if I saw her as the name of the reviewer for that particular week, I actually would hold off on submitting my referral for a week.”
	Requesting rereview	“When the reviewer declines to see the patient against my better judgment, and I can’t convince them they really need to, and I can’t get on the phone with them, present the case… so sometimes I have to ask my medical director if we can forward the eConsult to a different reviewer.”

Abbreviations: ED, emergency department; PCP, primary care practitioner.

^aeConsult is an electronic system for PCPs to submit clinical inquiries to specialists for management via electronic dialogue. Requests can either be resolved through dialogue alone or converted to face-to-face referrals.

Primary care practitioners also frequently raised issues regarding scheduling specialist appointments, although implementation of the centralized scheduling system was a distinct initiative from the eConsult system. Some PCPs cited difficulties in reliably contacting patients: “The specialist accepted the referral months ago, and then you see the patient in clinic and you find out that… no one had contacted them.” In contrast, other PCPs suggested that the scheduling process improved after eConsult and centralized scheduling.

Shift of Work to PCPs

Primary care practitioners perceived a shift in the onus of specialty care to the PCP, with a range of reactions to this change (Table 3). Some PCPs appreciated the education that they received through dialogue with specialists, which they believed could increase their scope of practice: “I definitely learned a lot from interacting with specialists and my threshold for contacting them has actually gotten higher because of all the knowledge that they share with me.” Others described a higher level of accountability in communicating their needs to the specialist: “It used to be easier to just be like, ‘I don't know, the neurologist will deal with that.’ Now, I have to make a really good argument to see a neurologist … there's less passing the buck on things that are difficult to deal with.”

In contrast, other PCPs believed that the previsit requirements requested by specialty reviewers were a burdensome shift of work to PCPs: “Another issue is where the consultant seems to require a very, very long list of things that need to be done … it starts to feel like you're almost like the support staff. It's actually wasting multiple primary care visits to accomplish what [the specialist] could in 1 session.” To address this perceived “gatekeeping,” some PCPs reported exaggerating patient symptoms or relying on key phrases to get a face-to-face visit approved, avoiding submitting eConsults if possible, and requesting second reviewers (Table 4).

In discussing this shift of specialty care to primary care, PCPs also often frequently discussed DHS’ expected practices, which is a separate initiative that established management pathways for common referrals to be referenced prior to opening an eConsult dialogue. Some believed that these expected practices helped to facilitate prereferral workup and reduced unnecessary referrals, while others believed that they mandated tasks that were not helpful to patient care: “I know the test they want before sending them [to the specialist], but that's not helping me manage [patients].”

Relationships With Specialists

Primary care practitioners also expressed a range of perspectives of how eConsult affected their relationships with specialists (Table 3). Some believed that eConsult dialogue strengthened communication and relationships with the specialist: “It really has increased communication, and it's also been really good for my learning as well. I'm getting direct recommendations from the specialists.” However, others viewed interactions with specialist as more antagonistic: “Often the reviewers… aren't looking at your eConsult with the mind of ‘I'm here to help the PCP.’ … It sometimes can feel a little bit insulting.” Potentially compounding this, some PCPs were put off by the impersonal nature of eConsult communication: “The eConsult is like a text messaging or email platform where you don't have to look at the person and you can kind of be a jerk if you want to be and just put up unnecessary hoops.” Ultimately, redefining communication and relationships with specialists was viewed as an important effect of eConsult. Summarizing her opinion of the eConsult system, one PCP stated, “This is all relational, right? We forget that when we build these tools.”

Frustration With Information Technology

Most PCPs were frustrated to some degree with the administrative burden of eConsult and interface issues, including lack of integration between eConsult and electronic health records (Table 3). As a PCP described, “It's very, very difficult because it takes a while to log in to separate systems.” Primary care practitioners were also frustrated by the time involved to upload images on the eConsult interface. Many still experienced difficulty receiving notes after visits: “It still feels like there is still a black hole. We don't really know what happens … and we don't get records for like 6 to 12 months.”

Overall Perception of eConsult

Ultimately, PCPs who reported positive attitudes toward eConsult generally perceived improved timeliness of care, seemed willing to accept the shift of specialty care, and reported improved relationships through eConsult. These PCPs acknowledged challenges but believed that eConsult was worth the effort because of improved care compared with the previous flawed referral system (“I think it's increased my job satisfaction … in a broad sense. The amount of time that it takes to do the eConsults probably negatively [affects] my satisfaction. I would say the general benefits outweigh the negative feelings.”)

In contrast, PCPs who overall had a negative attitude toward eConsult resented the increased administrative burden and restructured specialty care delivery. These PCPs did not endorse added value for their effort entering eConsults or they perceived additional delays: “Why do I have to… log in to another system, answer these extra questions, put in the same questions that I wrote in the referral, click Send to have it say, ‘okay approved for face-to-face visit’? It's the same end point.”

eConsult Design Questions

Drawing from interviewee comments regarding eConsult’s successes and drawbacks, we developed a list of questions for health systems seeking to implement eConsult. This list can be used as a tool to understand the multiple layers of decisions with the potential to affect PCP perception of eConsult (Box).

Box. eConsult Implementation Planning Questions.

1. eConsult System/Process Design

A. PCP Interface Design

• Have log-in steps been minimized?

• Has interface with EHR been optimized?

• Has entry of duplicate information been minimized?

• Can PCPs indicate their desired outcome of the eConsult? Should they be able to?

• Can PCPs track status of eConsult after entry? Is that a desired feature?

• Is there a mechanism for specialists to flag responses that are urgent vs nonurgent?

B. Process Design

• How quickly can PCPs expect a response?

• What process should PCPs use if a more urgent specialty care need arises?

• Is there a way for other clinicians in the PCP’s group to join an ongoing dialogue if a different clinician is covering or sees the patient in follow-up?

• Is there a way to transfer an ongoing dialogue to a different specialist if determined to be more appropriate?

• When should a dialogue be closed?

• Should the eConsult platform only be used for previsit conversations, or could it be used for ongoing comanagement conversations?

• Do all eConsults require the same depths of information or might some (ie, referral for colonoscopy for routine screening) be more automated in terms of PCP data entry and specialist review?

• Will PCPs (and specialists) receive administrative time or some form of compensation to enter and respond to eConsults?

• Should administrative support or care coordinators be part of the workflow of eConsults?

2. Pre/Post eConsult Process Decisions

A. Pre-eConsult Processes

• Are expected practices or referral guidelines or pathways available?

• If so, are they easily available at the point of care, or do they require extra log-ins?

• Is there an alternative referral process for urgent referrals or particularly routine referrals?

B. Post-eConsult Processes

• If a visit is recommended by the specialist reviewer, how is that decision communicated to the PCP or patient?

• If a visit is recommended by the specialist reviewer, what are the scheduling processes?

• If a visit is recommended by the specialist reviewer, is documentation of the eConsult dialogue available to the specialist who sees the patient?

• How are visit notes shared back with PCPs referring through eConsult?

3. eConsult Implementation/Change Management Decisions

A. Implementation Planning/Messaging

• What is the desired outcome of eConsult? How is this desired outcome to be communicated with specialists, PCPs, and patients?

• Has implementation training adequately discussed the potential value of the eConsult dialogue?

• Would PCPs benefit from training in the appropriate amount of information to enter in eConsult?

• How personable are PCP-specialist relationships, and how will this change relationships? Will PCPs know their specialist reviewers? Would there be value in meet-and-greet sessions?

B. Maintenance/Feedback

• What processes are in place to receive feedback from PCPs and specialists about the process?

• Are there processes to monitor speed or quality of specialist reviewer responses?

Discussion

eConsult implementation in DHS elicited a range of positive and negative perceptions among frontline clinicians. Three areas of conflict were the perceived effect of eConsult on access and timeliness of care delivery, the acceptability of the shift of responsibility of specialty care to the PCP, and the result for relationships with specialists. Many PCPs, particularly those who were enthusiastic about the mission of eConsult, believed that it improved the timeliness of specialty input and appreciated the opportunity to manage a broader spectrum of care. Others believed that eConsult slowed the process of obtaining a referral as the key goal and were frustrated by the need to follow specialists’ electronic suggestions. Our results highlight how delivery transformations can create new perceived barriers to care and resistance to change among frontline users.

Our analysis suggests that closing the gap between objective improvement in access at the population level and individual PCP’s perceptions may require addressing assumptions about the goals of eConsult and the role of PCPs in providing specialty care. Despite data demonstrating that eConsult reduced wait times for specialty consultation,¹² PCPs held varied perceptions of eConsult’s outcome on access to care. Many interviewees referred to having their eConsults accepted or approved for a visit, implying that a successful eConsult is one that results in a specialist visit. In contrast, from the perspective of health system leadership, an eConsult that does not result in a visit is a marker of a successful eConsult if the patient’s needs were adequately addressed without a face-to-face visit. Thus, perceptions of reduced wait times empirically achieved by eConsult may be tempered by perceived unsuccessful eConsults where in-person visits were obstructed or denied.

The shift of responsibility of specialty care with eConsult was also controversial. While some PCPs felt empowered in addressing more of their patients’ needs, others felt taken advantage of as “support staff” performing additional testing requested by specialists. These perceptions show how eConsult called into question the respective responsibilities of PCPs and specialists. These results contrast with broadly positive PCP experiences in other eConsult systems where the use of an eConsult dialogue is discretionary rather than required for any referral.^23,28,35 The trade-off with discretionary systems vs the DHS mandatory approach is that PCPs use eConsult relatively infrequently and the system-wide result of such an approach may be limited.

Many of the negative themes that emerged from PCP perceptions of eConsult appeared to be old problems with a “new face,” not necessarily worsened by eConsult but more readily apparent to PCPs. Prior to eConsult, many challenges in specialty care were masked by the opaque referral process. For example, while PCPs had little formal interaction with specialists, eConsult created a new set of relationships with specialists for PCPs to manage, which many criticized as impersonal or brusque. However, this was not a challenge created by eConsult, but a latent problem that eConsult made visible. Another example is the sharing of notes and records between PCPs and specialists. The pre-eConsult system was often described as a black hole with little communication. Despite the capacity to share notes and images within eConsult, the awkward information technology interface for uploading images and sharing notes became a focal point of frustration for PCPs.

Limitations

There are important limitations to this study. As a qualitative analysis, this research is hypothesis-generating and descriptive, and we are unable to make causal claims about the outcome of eConsult beyond reporting the perceptions of interviewees. In addition, we did not interview PCPs prior to eConsult implementation, and PCPs’ memories of that period may be subject to recall bias. In addition, we only interviewed PCPs, although we recognize that the eConsult system also affected specialists and patients. Last, we attempted to capture a broad diversity of perspectives by including DHS and non-DHS PCPs; however, our study was not designed to compare these groups but rather to obtain a heterogeneity of perspectives.

Despite these limitations, there are several key lessons from this study that can help inform other health systems seeking to adopt eConsult systems or similar wide-scale health information technology changes. First, transformative changes will likely create new problems as they solve old challenges. For example, while PCP-specialist communication was not a major concern when little back-and-forth dialogue was occurring prior to eConsult, eConsult created new opportunities for friction between PCPs and specialists. Thus, further training for both PCPs and specialists on best practices for eConsult communication (eg, providing as much information up front as possible) could be helpful.

Second, we observed that if clinicians believed in the mission and purpose of eConsult and specialty care transformation, they were more likely to endorse positive experiences with eConsult. It is worth further exploring whether a different implementation process focusing on making the larger case for eConsult could influence PCP perceptions of eConsult’s inherent value. Last, we found that any factor affecting clinician workflow, even if tangential to an intervention, could significantly affect perceptions of the intervention. In this case, frustrations with patient scheduling and information technology interface awkwardness significantly influenced PCP perceptions of eConsult.

Conclusions

The implementation of eConsult as a transformative delivery innovation had a range of positive and negative consequences for PCPs’ day-to-day practice. Informed by our analysis, our list of guiding questions (Box) highlights decisions that health systems implementing eConsult must make, either intentionally or by default. For health systems planning to implement eConsult, the LA County DHS eConsult experience can offer important lessons in anticipating challenges with PCP expectations, workflow changes, and new problems emerging with health information technology adoption.

Supplement.

eMethods. Evaluating the LA DHS eConsult Program: Interview Guide Draft.