Understanding the Potential for Patient Engagement in Electronic Consultation and Referral Systems: Lessons From One Safety Net System

Abstract

Objective

To understand patient, primary care clinician (PCC), and subspecialist perspectives on potential, unexplored roles for patients in electronic consultation and referral (eCR) systems.

Data Sources

Primary focus group and survey data collected April–November 2015. Zuckerberg San Francisco General Hospital (ZSFG) is part of an integrated public health delivery system. Its mature eCR system was first implemented in 2005.

Study Design

This mixed‐methods study synthesizes patient, subspecialist, and PCC perspectives through two patient focus groups in English, Spanish, and Cantonese (n = 6); subspecialist focus groups (n = 2); and an electronic survey of all PCCs (n = 222/634, 35 percent response).

Data Collection/Extraction Methods

Focus groups were audio‐recorded and transcribed. Two researchers coded the transcripts to identify recurrent themes. Survey data were analyzed using summary and bivariate statistics.

Principal Findings

Patients expressed minimal desire to directly engage in eCR, instead of emphasizing their PCC's role in advocating, informing, and finding health solutions. Subspecialists requested more consistent communication to patients about the electronic consultation process. Most PCCs (52 percent) supported patient engagement in the eCR process, particularly patient ability to track consult status and securely message with subspecialists.

Conclusions

Results suggest a continuum of opportunities for patients and their caregivers to engage in eCR systems.

Access to subspecialty care for patients is often a challenge for primary care clinicians (PCCs), particularly those practicing in safety net settings (Cook et al. 2007; Forrest et al. 2007; Kim et al. 2009). For patients in the safety net, wait times for subspecialty care can be as long as a year. Excessive wait time risks harm to patients who need access to subspecialty expertise and leads to inefficiencies that waste resources. Such challenges have impressed the need for newer models of care that improve access and efficiency and reduce waste. Telehealth, defined by the U.S. Department of Health and Human Services as the use of electronic information and telecommunication technologies to support and promote long‐distance clinical health care, patient and professional health‐related education, public health, and health administration, offers promise. In particular, electronic consultation and/or referral systems (eCRs), which are telehealth modalities focused on asynchronous communication among providers, represent new models of care that specifically address challenges with specialty care access. eCRs include electronic consultation systems, electronic referral systems, and integrated electronic consultation and referral systems. Electronic consultations are asynchronous provider–provider consultations that occur within an electronic health record or a Web‐based portal. They encourage bidirectional communication among providers, allowing specialists to address consultative needs without an in‐person visit or ensuring adequate diagnostic workup prior to a face‐to‐face patient visit when necessary. Electronic consultation systems may be used in parallel to referral processes (either electronic—known as electronic referrals—or paper/fax based) that carry the expectation of a specialty care visit. Alternatively, they may be integrated into the overall referral process, creating an integrated electronic consultation and referral system, in which all requests for specialty expertise are reviewed by specialists who respond with appropriate forms of consultation (Tuot et al. 2015a).

eCR systems have been implemented across the United States and internationally (Esquivel et al. 2012; Palen et al. 2012; Scherpbier‐de Haan et al. 2013; Liddy et al. 2015; Tuot et al. 2015a) and have successfully created a paradigm shift from a focus on “access to specialty visits to access to specialty expertise” (Chen, Murphy, and Yee 2013). Consistent with the Quadruple Aim (Bodenheimer and Sinsky 2014) of specialty care delivery, eCR systems have improved population health (increased access to subspecialty care, reduced wait times for in‐person specialty appointments, optimized efficiency in specialty care delivery, improved PCC capacity for complex decision making, and enhanced dialogue between PCCs and subspecialist providers) (Kim et al. 2009; Kim‐Hwang et al. 2010; Straus et al. 2011; Chen, Murphy, and Yee 2013; Sewell et al. 2013; McGeady et al. 2014; Tuot et al. 2015b), and enhanced provider satisfaction with the referral process, while reducing costs (Tuot et al. 2015b; Liddy, Drosinis, and Keely 2016; Olayiwola et al. 2016).

To date, eCR programs have focused on communication between PCCs and specialists, and the process has been mostly invisible to patients and their caregivers. Understanding patient satisfaction and engagement with eCR systems is crucial to unlocking this model's full potential to achieve the Quadruple Aim. Early studies suggest receptiveness among patients to alternative forms of communication with specialists. A Kaiser Permanente study in Colorado found that patients were equally satisfied with specialty consultations whether they occurred through traditional consultation request or via a virtual visit (Palen et al. 2012). Similarly, in a study of patient perspectives on use of electronic consultation to enhance access to endocrinology care in Canada, nearly 50 percent of patients considered an electronic consultation an acceptable alternative to an initial face‐to‐face visit (Keely, Traczyk, and Liddy 2015).

Given the potential for patient engagement in the eCR model to affect their health care experience, we sought to learn more about patient understanding of eCRs, as well as potential roles for patients and their caregivers in eCR systems, through a mixed‐methods study of patients, PCCs, and subspecialists.

Methods

Design

Between April 2015 and October 2015, we conducted focus groups with patients (n = 6 groups) and subspecialist physicians (n = 2 groups) and surveyed primary care clinicians within one health network. The study was approved by the UCSF Committee on Human Research (protocol number 14‐15193).

Setting

Zuckerberg San Francisco General Hospital and Trauma Center (ZSFG) is the acute care hospital for the San Francisco Health Network (SFHN), San Francisco's integrated publically funded delivery system. Serving over 120,000 underserved patients annually, ZSFG provides a full range of ambulatory subspecialty services. Referrals to subspecialty care come from a network of 14 SFHN primary care clinics as well as from a consortium of 10 independent federally qualified health centers. All requests for ZSFG subspecialty services are made through the integrated electronic consultation and referral platform, known as eReferral. eReferral was first implemented at ZSFG in 2005 with gastroenterology. PCCs can now submit electronic consultations to 83 subspecialty services. In the 2015–2016 academic year, there were nearly 87,000 electronic consultation submissions.

Subspecialty reviewers consider each submission for a consultative request and can immediately schedule an appointment, respond with questions, request additional evaluation, or provide management suggestions (Figure 1). This process may require multiple exchanges between the referring PCC and the specialist reviewer until they reach consensus on the best solution for the patient. While most “never scheduled” appointments reflect a consensus by the PCC and specialist reviewer that an in‐person subspecialty visit is not needed, some may also reflect resolution of the medical problem, thus eliminating the need for a specialist consultation, or a patient being lost to follow‐up in the health care system.

Figure 1.

eReferral Workflow and Consultative Options [Color figure can be viewed at http://wileyonlinelibrary.com] Source: Kim‐Hwang et al. 2010. Reprinted with permission.

Study Protocol

A 10‐member advisory committee provided input on the focus group and survey instruments, key research questions, recruitment strategy, and other study logistics. The committee included two patient advisors, one subspecialist eReferral reviewer, two primary care health network staff specializing in patient experience, one clinic medical director, one primary care health network director, one health plan medical officer, and one health services researcher.

Patient focus groups consisted of patients from the practice settings in which eReferral is used. Patients were eligible to participate if aged 18 years or older, fluent in one of the three predominant language groups at ZSFG (English, Spanish, or Cantonese) and had a consultation request submitted by their PCC using the eReferral platform within the previous 12 months. Eligible patients were identified from the electronic medical record. They were recruited by telephone and provided written consent for participation. Two patient focus groups were conducted between June and October 2015 in each of the three predominant patient languages. They ranged in size from two to seven participants, and each lasted between one and two hours.

Focus group facilitators used a semistructured focus group guide that allowed patients to draw upon their personal experiences as well as to imagine their responses to a variety of scenarios (Morgan 1996). Open‐ended interview questions centered around overall experience with subspecialist consultations, satisfaction with subspecialty visits, perceived coordination of PCC and subspecialty care, communication with and between PCCs and specialists about the electronic consultation process, potential patient or family/caregiver roles in the consultation process, and desirability of such potential roles. During the course of the focus groups, the eCR process was described to patients after their understanding of the process was explored. Patients completed brief demographic surveys and received $25 in remuneration at the end of the focus group.

Subspecialist physician focus groups were conducted during two separate ZSFG eReferral subspecialty reviewer retreats held in April 2015. Each group consisted of five to seven subspecialty reviewers who self‐selected and consented for the focus group discussion, which lasted approximately 30 minutes. Open‐ended focus group questions explored subspecialist communication with patients about the electronic consultation process, potential for patient or family/caregiver role in the consultative process, and general eReferral satisfaction. Subspecialist participants were not remunerated for their participation.

Patient and subspecialist focus groups were conducted by experienced language‐concordant facilitators and included a language‐concordant note‐taker. Each facilitator had previous experience leading focus groups and received additional training by a senior member of the research team in the principles of group facilitation.

Primary Care Clinician Survey

In October 2015, we administered an electronic survey to all PCCs in 35 practices that utilize eReferral to refer patients to subspecialists at ZSFG. PCCs were emailed an invitation to complete the survey electronically, with up to four reminder emails sent to nonresponders. Survey measures examined PCC perceptions of the potential for patient or family/caregiver roles in the electronic consultation process. Basic demographics on practice setting, experience, and training were also collected. The first 150 PCCs to complete the survey were given a $10 gift card.

Analysis

Focus groups were audio‐recorded and professionally transcribed, and then verified by written field notes. Transcripts were analyzed using thematic synthesis in modified grounded theory (Grbich 2012; Charmaz 2014) using an iterative and collaborative process. A preliminary codebook was developed from reviewing a subset of transcripts. Two of three research team members independently coded each transcript (MK and EL or TW) and agreed upon initial codes. As additional concepts emerged, team members collaboratively adjusted the codebook then identified both descriptive and analytic themes. Discrepancies in coding were resolved by consensus and discussed with study co‐PIs (JNO, DST). Atlas.ti 7.5 software was used to facilitate the coding process (ATLAS.ti Version 7.5 2015).

PCC survey data were analyzed using Stata 13.1 (Stata 13.1 2015). Questions asked on a 0 to 5 scale (strongly disagree to strongly agree) were recoded to disagree, neutral, or agree to simplify interpretability. We used chi‐squared tests to evaluate potential differences by training level (attending physician, resident physician, or nurse practitioner/physician assistant).

Results

Patient Characteristics

A total of 29 patients participated in six patient focus groups (Table 1); 6 patients spoke English, 10 spoke Spanish, and 13 spoke Cantonese. Nearly two‐thirds of study participants were female, with an age range of 27 to 79 years (mean = 57.7, standard deviation = 12.3). Eighty‐three percent of patients reported that their PCC spoke the same language as they did. Over one‐half of participants reported that they had a computer at home and over 60 percent reported that they had Internet access on their mobile phone or tablet devices. About one in five patients reported that they were signed up for the SFHN patient portal.

Table 1.

Demographics of Study Participants (Patients, PCCs, Subspecialists)

Patient focus groups (N = 29 patients)
Age	57.7 (mean) 12.3 (SD)
Gender (N, %)
Male	10	34%
Female	19	66%
Language
English	6	20%
Spanish	10	35%
Cantonese	13	45%
Computer at home	16	55%
Internet on mobile device	18	62%
Signed up for patient portal	6	21%
Language‐concordant PCC	24	83%
Subspecialist focus groups (N = 11 subspecialists)
Age	45 (mean) 7.4 (SD)
Post‐training years in practice	15.2 (mean) 9.6 (SD)
Gender (N, %)
Male	3	27%
Female	8	73%
PCC survey (N = 222)
Practice setting
Hospital‐based outpatient clinic	128	59%
SFHN Community Primary Care clinic	59	27%
Independent federally qualified health center consortium	30	13%
Training
Nurse practitioner/physician assistant (NP/PA)	51	23%
Attending physician	99	45%
Resident/fellow	69	32%
Specialty
Family medicine	103	47%
Internal medicine	86	39%
Other primary care	32	14%
Time Spent with eReferral per week
<10 minutes	30	14%
10 to <30 minutes	92	43%
30 to <60 minutes	57	27%
60+ minutes	33	16%

Specialist Characteristics

Two focus groups were conducted with a total of 11 subspecialist reviewers, all of whom regularly use the eReferral system to provide specialty expertise (Exhibit 2). The following subspecialties were represented: Trauma, Surgery, Internal Medicine, Diabetes, Genetics, Transgender Health, Geriatrics, Occupational Therapy/Rehabilitation, Physical Therapy, and Urology. Over 70 percent of participating subspecialists were female; subspecialists ranged in age from 35 to 57 years (mean = 45, standard deviation = 7.4) and had been in practice for 2 to 30 years after training (mean = 15.1, standard deviation = 9.6) (Table 1).

Primary Care Clinician Characteristics

Two hundred and twenty‐two of 634 PCCs in the SFHN responded to the electronic survey (response rate: 35 percent). Fifty‐nine percent of PCCs practiced in a ZSFG hospital‐based outpatient clinic, 27 percent in an SFHN community primary care clinic, and 14 percent in a local consortium of 10 independent federally qualified health centers that often refer to ZSFG for subspecialty care. Forty‐five percent were attending physicians, 42 percent were resident physicians, and 23 percent were nurse practitioners or physician assistants. Nearly one‐half of PCCs reported family medicine as their primary specialty and 39 percent internal medicine. Thirty percent had been in practice between 1 and 10 years, 24 percent between 11 and 20 years, and 18 percent for over 20 years. Nearly half of PCCs (43 percent) reported spending between 10 and 30 minutes using eReferral each week; nearly one‐third (27 percent) reported spending between 30 and 60 minutes using eReferral per week (Table 1).

Patient Focus Group Findings

Patients found the term “eReferral” consistently confusing or unfamiliar, but they readily described instances of PCCs communicating with other clinicians by computer and also noted examples of PCCs providing information communicated from subspecialists.

We noted four overall themes from patient focus groups regarding their experiences, expectations, and attitudes toward the eReferral system (Table 2):

Table 2.

Summary of Patient Focus Group Themes and Subspecialist Focus Group Themes

Patient Focus Group Themes

Subspecialist Focus Group Themes

Concerns about care coordination: would not want eCR communication to detract from care coordination for reasons such as time delays between appointments, disagreements between providers, or deflection of care. Desire to be informed: would like PCC to lead decisions about if eCR is best approach and also close the loop after the appointment occurs or relaying subspecialist recommendations if not scheduled Fears of missing information: would want the eCR communication to the subspecialist through the PCC to adequately represent their concerns or complaints; patients worry about forgetting information if there is a delay in getting a subspecialist appointment Desire for a provider that helps patients find resolution to their health problems: a strong relationship and good communication with the PCC and patient can be the foundation of a successful eCR system due to provider accountability for their care, not dismissing patients’ concerns, and feeling that the PCC is caring

Need for better patient communication about eReferral occurrence and process: better patient preparation for appointments, either through PCC communication to patients and/or subspecialist outreach to patients, confronting language barriers Mismatch between patient and subspecialist expectations: not knowing why an appointment is scheduled, expectation of surgical procedure at the initial consultation Engagement of patients and caregivers in eReferral: ability to get information more directly, potential for security and confidentiality breaches, exacerbating technology and language barriers

1. While patients noted some benefits from eReferral and awareness of electronic communication occurring, they also expressed some concern that eReferral may further detract from care coordination or lead to responsibility for their care being deflected among providers. For example, one patient expressed concern that a provider using eReferral might say, “I asked someone and he told me to give you this. If something happens to you, it's not my responsibility because the other doctor prescribed it”(Spanish focus group 1).

2. Patients desire that PCCs determine whether eReferral communication is appropriate but also want to be informed about the process. Most patients felt PCCs were best positioned to determine whether an eReferral communication versus direct in‐person visit was appropriate for their situation. For example, one patient commented, “I tell [my doctor], ‘If you think I need that and it's for my own benefit, just do it’” (Spanish focus group 2) and another stated, “I'd be very satisfied to not have to see a specialist if that were offered to me, if that seemed to be appropriate therapy” (English focus group 1). Patients also noted interest in being informed, commenting for example, “I think probably what should happen just as a safeguard is to let the patient know: I have had this discussion. If you feel the need to speak to this specialist personally, we can set that appointment up” (English focus group 2).

3. Additionally, although many patients trust the decision on how the subspecialist is engaged to their PCC, there were fears that dyadic communication between the clinicians may filter the full clinical picture or their concerns. One patient specifically described concerns that the subspecialist might not be privy to important information about their condition through eReferral: “And if I feel like my doctor is brushing off that information, is not communicating other symptoms … you know, these are the only four symptoms that matter and so I'm just going to give those to the specialist, at that point I might feel like wow, there's more information that's not getting through” (English focus group 1).

4. Patients generally appreciate the opportunity for in‐person communication with subspecialist clinicians, yet noted that a strong primary care clinician–patient relationship could establish a positive perception of virtual delivery of specialty care. One patient stated as follows: “that's really sort of the problem that I have with electronic referrals or consultations. I personally prefer human contact” (English focus group 2). However, another patient highlighted, “a good doctor will explain to you so you can understand it … that makes you feel comfortable and you can walk out of there saying, ‘Okay, now I'm getting somewhere’” (English focus group 1).

Overall, patients were generally accepting of the current eCR model in which PCCs and subspecialty clinicians communicate about their clinical conditions and did not express strong interest in being inserted into that interaction. This was most often due to a high level of trust and confidence in PCC care, but it was also due to limited health literacy and existing language and/or technology barriers that would make it challenging to participate in the electronic communication. However, patients also expressed the following prerequisites for greater acceptance of an eCR system: clear care coordination and clinician accountability, patients informed about the process with the opportunity to have their questions answered, patient health concerns appropriately and comprehensively communicated between the PCC and subspecialist, and a strong, trust‐based, language‐concordant PCC–patient relationship.

Subspecialist Focus Group Findings

Subspecialist focus groups pointed to three themes (Exhibit 3):

1. There is little consensus on how best communication about the eReferral occurrence and process should occur. While the subspecialists believe good communication about the process of electronic consultation is important, they are not clear how PCCs communicate with their patients on this model of care, nor do subspecialists believe they are consistent themselves. Some subspecialists believed that communication about eReferral should occur as a part of action planning when closing out any patient appointment in primary care, while others felt that this may be cumbersome to achieve every visit. Another suggested a “triage” approach in which communication about eCR processes occurs based on likelihood that an in‐person visit would ultimately be scheduled.

2. The perceived inconsistency in both clarity and content of eReferral processes to patients results in frequent mismatches between patient and subspecialist expectations. One subspecialist mentioned that at least half of patients are not aware of the electronic consultation submitted for them by their PCC when they ultimately meet the subspecialist, and many others in the group concurred. Subspecialists also described examples where patients did not understand the point of a subspecialty in‐person visit. For example, patients could arrive at an ambulatory surgery appointment expecting surgery or did not attend the in‐person surgery appointment because they were not ready for a surgery, which, at that point, had not been recommended.

3. Regarding the engagement of patients and/or their caregivers in the eCR process, the subspecialists acknowledged several challenges, for example, (1) patients who do not use the Internet would not be able to access the electronic communications, (2) electronic communications may pose potential security and confidentiality concerns, (3) language barriers for non‐English‐speaking patients, and (4) perception of a lack of reimbursement for the time/effort required by subspecialists to communicate via the eCR versus face‐to‐face visit time. Some subspecialists felt that patient involvement in eReferral would lead to confusion in the patient record and result in back‐and‐forth communication that could be resolved more efficiently with an in‐person visit. But it was also noted that clarifying certain questions directly with patients without needing the PCC in the middle could be advantageous, for example, to ask whether the patient had a preference for one of two available surgeons.

Overall, subspecialists believe that the eReferral model has been transformative for patient care. While they do not envision broadly including patients in the direct electronic dialogue between clinicians, they do believe that enhanced communication between clinicians and patients, as well as clarity on processes and expectations at the point of eCR submission, would optimize the experience for patients.

PCC Survey Findings

About half of PCCs (52 percent) supported patient and caregiver involvement in the electronic consultation process in some capacity (Figure 2). Similarly, about half of PCCs anticipated modest or great value for each of the four survey items describing potential patient engagement capabilities: patients participating in decision about whether an in‐person visit is needed (56 percent), ability to view specialist consultative communication (53 percent), receipt of a copy of the consultation request (47 percent), and contributing to the electronic consultative dialog via the eCR platform (45 percent). Anticipated value significantly differed by training status for patients participating in deciding whether an in‐person visit is needed (68 percent residents vs. 52 percent attending physicians vs. 46 percent NP/PA, p‐value = .001) and patients contributing to the consultative dialog (51 percent residents vs. 45 percent attending physicians vs. 37 percent NP/PA, p‐value = .004). The most commonly requested eCR functionalities in open‐ended responses included patient ability to view the status of their electronic consultation request and to directly message securely with subspecialists.

Figure 2.

Primary Care Clinician (PCC) Survey Responses Depicting Perception of Value and Potential Risks for Patient Engagement in eReferral

Many PCCs (71 percent) expressed concerns that workload would increase if patients were engaged with the eCR process. Relatively few (32 percent) PCCs agreed/strongly agreed that patient or caregiver engagement in eReferral would increase medicolegal liability, although open‐ended responses suggested that the ability for patients to view the PCC–specialist dialogue could increase potential misunderstandings given patients’ lack of medical knowledge and also reduce the openness of communication between PCCs and subspecialists.

Only 22 percent of PCCs agree or strongly agree that patients and their caregivers currently understand the eCR process, and most PCCs (76 percent) agree or strongly agree that involving patients or caregivers in that process would require significant training. As patients themselves also noted, PCCs frequently noted salient patient engagement barriers due to high proportions of non‐English‐speaking patients (n = 72 open‐ended comments), low literacy and low health literacy (n = 59 open‐ended comments), and limited access to computers, phones, and the Internet (n = 45 open‐ended comments).

Table 3 summarizes patient, PCC, and subspecialist perspectives on potential benefits and barriers to patient engagement in eCR systems.

Table 3.

Potential Benefits and Barriers to Patient Engagement in eCR Systems

	Patient Perspective (6 Focus Groups)	PCC* Perspective (Survey of 222 Providers)	Subspecialist Perspective (2 Focus Groups)
Potential benefits of patient engagement in eCR systems
More informed patients	Could improve understanding of where information is coming from (PCC or specialist); understand timing and expectations for upcoming subspecialist visits.	Could improve setting expectations for subspecialist visits and acceptability of management recommendations.	Could enhance patient activation and care planning, especially for time‐sensitive issues (e.g., cancer patients) and where advanced preparation is needed.
Improved communication through a secure platform	Systems could create space for patient questions, feedback, or clarifications; could also ensure patient concerns relayed appropriately between clinicians.	Secure messaging potentially beneficial for patients to provide information directly to specialists without relaying through PCC.	Secure messaging could minimize patient confusion on expectations of in‐person specialty visits.
Leveraging desired features	eCR engagement may also cultivate higher uptake of patient portals, which would provide patients with access to laboratory results and medication lists; may allow patients to know whether/when a specialty care appointment has been scheduled, so they can prepare for them.	Patient‐facing eCR systems could convey the patients’ status and understanding within the eCR process (e.g., question pending, answered, appointment scheduled) for patient and PCC benefit.	eCR systems could incorporate patient self‐assessments or symptoms diaries providing useful information for the treatment plan.
Potential barriers to patient engagement in eCR systems
Restricted access to technology	Not all patients have access to computers, and most desire simple, easy‐to‐use platforms (e.g., mobile accessible and intuitive)	Computer literacy may pose challenges for some patients and other patient‐facing technologies (e.g., patient portals) generally achieve low uptake.	Limited computer access for patients may impede ability to engage in eCR systems.
Limitations in English proficiency and health literacy	Communication in preferred language frequently not available. Patients note that terminology frequently does not make sense without medical training or interpretation.	Concerns about misunderstandings if patient involved due to low literacy or low health literacy. Potential that patient involvement will limit frankness of communication.	There is a need to be cautious about wording between clinicians if patient will be viewing. Concerns about patient expectations vis‐à‐vis frequency and need for eCR communication.
Time burden for patients and providers	Patients often need to leverage family members for access to technology or for translation, which are not always available. Patients also recognize the increased provider burden created by asynchronous messaging.	Concerns about increased workload as result of increased patient messaging.	Concerns about increased workload for PCCs and specialists around increased patient messaging, especially in regard to the effort needed to prevent misunderstandings.

eCR, electronic consult and referral systems; PCCs, primary care clinicians.

Discussion

This is, to our knowledge, the first study to triangulate patient, primary care clinician, and subspecialty provider perspectives on patient engagement with eCR systems. Although this triangulation did not identify any areas of convergence about specific roles that patients may play in an eCR system, all groups emphasized the importance of consistent and improved communication in all of the relationships. For example, while many PCCs believed that providing patients the ability to directly interact with subspecialists in an electronic exchange would be valuable, patients did not express this as a preference and subspecialists had reservations about this capability. Regardless, each study group suggested ideas that could enhance the experience of patients who are part of eCR systems such as: informing patients of the details on the process as well as their options, while ensuring their concerns are not diluted in the information exchange (patients); creating consistent messaging about eCR processes and expectations with patients (subspecialists); and engaging patients in the decision‐making process around if an in‐person visit is needed (PCCs).

Our results corroborate past findings showing an openness on the part of patients to alternative forms of subspecialist consultations (Palen et al. 2012; Keely, Traczyk, and Liddy 2015). They go beyond these past studies to show that PCCs and subspecialists also express a cautious openness to exploring new models of patient engagement in the eCR model of care delivery. Notably, clinicians also described a need to improve communications around existing systems to make the process more transparent to patients. The lack of awareness of the patients in this study about the system supports this recommendation.

Our study was limited by its focus on one eCR system. Newer systems may have different barriers and opportunities for patient engagement that we did not identify here. Additionally, although our patient and subspecialist focus groups were diverse in language and specialty, respectively, the small sample size may not adequately capture the perspectives of all patients or subspecialists in our system. Also, we stratified the patient focus groups by language and not age. The median age of patients was 57 years old, which may have influenced the perspectives on technology adoption. It is possible that a younger cohort of patients would have had more interest and willingness to engage with technology. Future studies should consider age‐based sampling. Finally, our limited time for focus groups with the subspecialists may have blunted further perspectives on this topic, although we believe this was a useful springboard for future discussion.

Initially, we sought to develop a prototype for patient engagement in eCR systems. Based on the findings and lessons from this study, we believe there are many potential strategies for engaging patients. Rather than having one approach, patient engagement may be considered as a continuum. A truly patient‐centered eCR system would enable patients or those responsible for their care to nimbly interact with the system in different ways, at different times, and for different needs. For example, we propose that patient engagement build on a foundation of clear PCC–patient communication around eCR processes and expected follow‐up. For minimal engagement, systems could add patient ability to view appointments or send a quick clarifying response to a subspecialist question. Enhanced patient engagement could include opportunities for greater communication and decision making, such as detailed correspondence between a patient and rheumatologist on the nature and timing of fatigue.

Our findings from a long‐standing, multispecialty eCR system suggest that patient engagement in eCRs could unlock much potential for care improvements. Patient involvement will be critical not only in our questions about eCR innovation but also in our solutions and strategies as we leverage the eCR model to achieve all components of the Quadruple Aim.

Supporting information

Appendix SA1: Author Matrix.