Abstract
Objectives:
In response to limited access to specialty care in safety-net settings, an integrated delivery system and three safety-net organizations in the Denver, CO, metropolitan area launched a unique program in 2013. The program offers safety-net providers the option to electronically consult with specialists. Uninsured patients may be seen by specialists in office visits for a defined set of services. This article describes the program, identifies aspects that have worked well and areas that need improvement, and offers lessons learned.
Methods:
We quantified electronic consultations (e-consults) between safety-net clinicians and specialists, and face-to-face specialist visits between May 2013 and December 2014. We reviewed and categorized all e-consults from November and December 2014. In 2015, we interviewed 21 safety-net clinicians and staff, 12 specialists, and 10 patients, and conducted a thematic analysis to determine factors facilitating and limiting optimal program use.
Results:
In the first 20 months of the program, safety-net clinicians at 23 clinics made 602 e-consults to specialists, and 81 patients received face-to-face specialist visits. Of 204 primary care clinicians, 103 made e-consults; 65 specialists participated in the program. Aspects facilitating program use were referral case managers’ involvement and the use of clear, concise questions in e-consults. Key recommendations for process improvement were to promote an understanding of the different health care contexts, support provider-to-provider communication, facilitate hand-offs between settings, and clarify program scope.
Conclusion:
Participants perceived the program as responsive to their needs, yet opportunities exist for continued uptake and expansion. Communitywide efforts to assess and address needs remain important.
INTRODUCTION
Limited access to specialty care services for the uninsured and underinsured is a national concern. Up to 25% of visits to community health centers result in medically necessary referrals for care not provided on-site at their facilities.1 Poor access to specialty care is especially pronounced for the uninsured, who often rely on hospital Emergency Departments for specialized services.1,2 Although insurance coverage has expanded under the Affordable Care Act, an estimated 23 million people in the US remain uninsured, including undocumented immigrants and those with gaps in insurance coverage.3 Individuals benefiting from Medicaid expansion also face an increasingly difficult time accessing specialty care. There is an ongoing need to identify ways to increase access to specialty care for underserved populations.
Optimal specialty care access in community health centers focuses on comprehensiveness (bringing services into primary care) and coordination (building relationships with services outside primary care).4 This integration can occur through development of informal relationships and partnerships with hospitals, contracts with specialists to work on-site, use of telehealth or electronic communication, collaboration with a teaching community, and participation in integrated systems.5
Multiple specialty care access programs have been implemented nationally to meet the needs of uninsured and underinsured patients.6 Specialist consultation can provide information to reduce clinical uncertainty, to increase access to specific procedures, to permit comanagement of complex patients, and to transfer patients from primary to specialty care.6 Components of specialty care programs include referral hubs, teleconferencing programs, and electronic-consultation (e-consult) programs. Specialty care referral hubs connect safety-net primary care clinics to volunteer specialists and are primarily funded by grants and donated services; many specialists are linked to academic medical centers.2,7,8 Teleconferencing programs may or may not include direct patient engagement. One example that has been adopted nationally and internationally is the Extension for Community Healthcare Outcomes (ECHO) model, which began as a way to expand access to treatment of hepatitis C in rural communities by allowing primary care clinicians to discuss a patient case with specialists virtually.9,10 E-consult programs link primary care clinicians in the safety net to specialists through secure messaging; some of these programs are limited to electronic exchange, whereas others include referral for face-to-face visits.11,12
E-consult systems can reduce the need for in-person specialist visits, expand primary care clinicians’ scope of work, and improve aspects of care coordination while maintaining informational value.13,14 Other potential benefits include improved communication between clinicians and patients, and between clinicians facilitating information transfer and aiding decision-making processes, as well as improved care quality, less travel time, and better timeliness of care.15 Additional research is needed to assess potential cost savings and to fully evaluate the impact on practice workflow.16,17 Prescreening through e-consult has been proposed to improve the efficiency and utility of in-person specialty care referrals.18 Given the potential efficiencies, there have been calls to identify which clinical circumstances lend themselves well to asynchronous e-consult.6
Recognizing the need for specialty care access for underserved patients in the Denver, CO, metropolitan area, Kaiser Permanente (KP) Colorado (KPCO) collaborated with the Colorado Health Institute in 2010 to 2011 to conduct a statewide specialty care needs assessment in the Colorado health care safety net, as well as an associated demand study in three metropolitan Denver federally qualified health centers.19,20
The needs assessment confirmed barriers to specialty care access, especially for uninsured patients. Access was particularly difficult for neurosurgery, orthopedics, and endocrinology but was impaired for many more medical and surgical specialties.19 The assessment also identified e-consult as a potentially useful resource for primary care clinicians. Drawing on the findings from the needs assessment and meetings with local existing programs to improve specialty care access, the KP Safety Net Specialty Care Program was launched in March 2013. Also considered in launching the program was the capacity of the KPCO specialty departments to take part in the program, as determined by the Operations Department leadership.
This evaluation describes the program during the first 20 months of implementation, identifies aspects that have worked well and areas for process improvement, and shares lessons learned for other emerging specialty care access programs.
METHODS
Evaluation Overview and Study Design
From December 2014 through June 2015, we conducted an evaluation of the Safety Net Specialty Care Program to synthesize and document lessons learned and to identify areas for program development and process improvement. This evaluation received ethical approval from the KPCO institutional review board (CO-12-1840). We used a mixed-methods design, combining descriptive quantitative information with qualitative interviews.
Setting
A nonprofit integrated health care delivery system, KPCO primarily serves the Denver metropolitan area and north and south of Denver along the Colorado “Front Range.” To develop and to implement the program, KPCO collaborated with 3 federally qualified health centers in the Denver metropolitan area: Clinica Family Health Services, Metro Community Provider Network, and Salud Family Health Centers. Together, these centers serve a population of more than 150,000 patients, of which approximately 57,500 are uninsured adults, according to the 2014 Uniform Data System.21 The 3 centers serve a patient population that includes a sizable number of uninsured migrant, low-income working, and homeless individuals and families.
Description of the Program
The program includes three components: e-consults, face-to-face visits, and continuing medical education (CME).
E-Consults
E-consults are electronic medical consultations between primary care clinicians in the safety net and KPCO specialists, conducted through a clinical messaging portal enabled by a third party and compliant with the Health Insurance Portability and Accountability Act. Consultations are available to primary care clinicians for any uninsured adult patient with a condition relevant to one of the eight participating specialty departments: allergy/immunology, cardiology, dermatology, endocrinology, gastroenterology, ophthalmology, pulmonology, and rheumatology. E-consults are submitted to a specialist on call from the designated department, who then replies electronically.
Face-to-Face Visits
When warranted, a specialist may suggest a face-to-face visit from a menu of available services to supplement an e-consult. The menu of face-to-face services addresses selected conditions that can be resolved in the KPCO system and focuses on treatments that can improve a patient’s health within approximately 90 days. This care is provided at no cost to the patient; however, referring safety-net practices take responsibility for prescriptions—including injections, infusions, and durable medical equipment—and for arranging transportation for patients. If needed, a face-to-face visit may require 1 or more follow-up visits. After a face-to-face visit, the patients are reconnected to their primary care provider for ongoing health care needs.
Continuing Medical Education
As part of the program, KPCO specialty care clinicians provide periodic in-person CME lectures on topics identified by safety-net clinicians. Topics have included new guidelines and best practices on topics such as hepatitis C, preoperation physical examinations, and polycystic ovarian syndrome. This component of the program is not the focus of this article because only 1 CME session took place during the initial 20-month period.
Community Partners and Staff
The 3 participating safety-net organizations were selected on the basis of their participation in the 2010–2011 needs assessment, leadership support, strong existing relationships with KPCO, geographic proximity to KPCO specialty care centers, and proficiency with electronic medical records, which were needed to capture meaningful, comparable, and accurate information on specialty care referrals.20 Although having proficiency with electronic medical records was an initial requirement, the e-consult platform does not interface with each safety-net organization’s own electronic health record system. Table 1 summarizes the roles and staffing of community partners for the program during the initial 20-month period.
Table 1.
Safety Net Specialty Care Program community partner organizations and roles
| Organization | Staffing | Role |
|---|---|---|
| Kaiser Permanente Colorado (KPCO) | Administrative staff (0.6 FTE) | Evaluate initial e-consults; ensure that e-consults are addressed promptly; assist with scheduling face-to-face visits; track specialists’ time and the cost of direct care visits |
| Community Benefit and Relations staff (1.5 FTE) | Facilitate communication between safety-net and KPCO leaders and internal and external stakeholders; oversee maintenance of legal, compliance, budgetary, and information technology security issues related to the program | |
| Community Benefit-supported clinician time (0.3 FTE) | Manage communication and relationships with medical leadership and the participating specialty departments | |
| Specialists | Review and respond to incoming e-consults as needed; see uninsured patients for face-to-face visits, when considered necessary | |
| Colorado Community Health Network | Administrative staff person (0.4 FTE) | Liaison to all safety-net clinicians; track and summarize e-consults on a monthly basis; help safety-net partners identify topics for continuing education; maintain the contract with NetChemistry Inc, a company based in Newport Beach, CA, which designed the software platform for e-consult exchanges (initially, and then this was transferred to KPCO) |
| Safety-net organizations | Internally defined (physicians, nurse practitioners, physician assistants, and care coordinators) | Clinicians submit questions via e-consult platform to KPCO specialists; in some safety-net organizations, a care coordinator fills out the information in the e-consult platform. An identified person at each safety-net organization coordinates with the KPCO administrative staff person to arrange face-to-face visits |
E-consults = electronic consultations; FTE = full-time equivalent.
Administrative Process and Financial Accounting
The program is supported by an ongoing financial commitment from KPCO’s Community Benefit and Relations (CB&R) department.
Before launching the program, the CB&R staff identified the secure messaging vendor and facilitated legal, compliance, and information technology security verification. Although specialist physicians are salaried, their departments are reimbursed by KPCO CB&R for physician time dedicated to answering e-consults, with the reimbursement based on the average departmental specialist’s salary. Each e-consult, which may include multiple exchanges about the same patient, is reimbursed at the average departmental salary for a 30-minute period. The value of direct care services is based on the Medicare fee schedule, and these services are paid for by CB&R through its Community Medical Financial Assistance program.22 In-person interpreter services are funded by KPCO CB&R and provided by KP translation services or contracts. The role of Denver-based Colorado Community Health Network as administrator was also supported by KPCO CB&R through grant funding.
Evaluation Data Collection
All e-consults conducted between November 1, 2014, and December 31, 2014, were retrospectively reviewed to determine the number of exchanges, response time, reason or reasons for the e-consult, specialist’s response, and subsequent safety-net primary care provider’s response to the specialist. We interviewed a purposive sample of safety-net clinicians and staff, participating KPCO specialists, and patients. Brief interviews also were conducted with key program staff at Colorado Community Health Network and KPCO. All interviews were conducted over the phone, with the exception of one in-person interview with a safety-net clinician at an administrative building. Interviews with clinicians took 10 to 15 minutes and with patients took up to 30 minutes. Interview questions are included in the Sidebar: Interview Guide for Patients and Clinicians in the Safety Net Specialty Care Program Evaluation, available from: www.thepermanentejournal.org/files/16-022Questionnaire.pdf.
Interviews were conducted with 18 of 204 safety-net clinicians (9%) and 3 referral coordinators (1 from each of the safety-net organizations) including physicians, nurse practitioners, and physician assistants. At least 5 clinicians from each of the safety-net organizations were interviewed, including those with high use (on 4 or more occasions), low use (1–3 times using the system), and no experience using the e-consult system. Safety-net clinicians were asked about their decision to use the program, why clinicians might not use the program, how they use the program, usefulness of advice, ability to convey patients’ needs and preferences, information that should be included in e-consults, for what types of advice the program is appropriate, patients’ hesitations about and experience with face-to-face visits, turnaround time, how the program fits within workflow, and recommendations. Referral coordinators were asked about their role in the system and how referrals work elsewhere in the community.
Twelve of 65 participating KPCO specialists (18%) were interviewed. At least 1 specialist was interviewed from each of the 8 participating departments, and others were interviewed if they were available. Specialists were asked the same questions as safety-net primary care clinicians and were also asked about the quality of questions asked via electronic communication, differences in communication with clinicians within KPCO, whether they had used an alternative form of communication, and potential program expansion.
Ten of 81 patients (12%) who had received at least 1 face-to-face visit were interviewed. Patients were randomly selected within strata representing each specialty department and safety-net organization. Patients were asked about specialty care received before their KPCO visit, concerns about their visit, information and arrangements before their visit, interactions with KPCO specialists and staff on the day of their visit, their understanding of the visit process and follow-up plan, care they received afterward, and the extent to which their situation was resolved.
Data Analysis
We described e-consults by specialty department, by clinician type, and by each clinician. Face-to-face visits were described by specialty department, by safety-net clinic, and by patient characteristics. Two reviewers (MPF and EAB) independently categorized e-consult exchanges and classified e-consults into categories that reflected common primary-specialty care interactions. Differences were resolved through discussion.
Patient interviews were recorded and reviewed for themes and representative quotes.23 Safety-net clinicians and staff and KPCO specialists were interviewed individually, and detailed notes were taken during interviews. Interview notes were transcribed and coded using a thematic analysis approach aided by qualitative data management software (ATLAS.ti Version 7 for Windows, Scientific Software Development GmbH, Berlin, Germany). The primary coder started the process by closely reading all the transcripts from interviews with clinicians and looking for emerging themes. A codebook was generated and updated after a second reading of all transcripts, and a final codebook was used to code all interviews. A general inductive approach was used to analyze the interviews with clinicians,24 and the primary categories are described and presented with illustrative quotes in the Results section.
RESULTS
Utilization Summary: E-Consults
Between May 2013 and December 2014, 103 of 204 (50%) safety-net clinicians at 23 clinics in the 3 participating organizations used the Safety Net Specialty Care Program. Among those who used the program at least once, the mean number of e-consults by clinician was 5.6, with a range of 1 to 31. During the same period, 65 specialists from the 8 participating specialty departments took part in the program.
Figure 1 illustrates e-consults between the program’s launch in May 2013 and December 2014, by participating specialty department. There was a lag time of 2 months, with the first e-consult submitted in May 2013. Of the 602 e-consults, gastroenterology received the most with 153, followed by dermatology, cardiology, endocrinology, and rheumatology, all with 90 e-consults or more, and then ophthalmology, pulmonology, and allergy/immunology. Primary changes to the program during the time period are highlighted.
Figure 1.
E-consults and program timeline from May 2013 to December 2014.
13 = 2013; 14 = 2014; Aug = August; Dec = December; E-consults = electronic consultations; Feb = February; GI = Gastroenterology; Jan = January; Jul = July; Jun = June; Oct = October; Mar = March; Nov = November; PA = physician assistant.
Utilization Summary: Face-to-Face Visits
Table 2 summarizes characteristics of patients receiving face-to-face visits between August 2013, when face-to-face visits were initially offered, and December 2014. Table 2 includes the menu of available services for face-to-face visits. During this period, 81 patients were seen at least once and 58 received follow-up visits. Of the 81 patients, 54 (67%) were women, and the average age was 45 years (standard deviation = 13 years). Gastroenterology provided the most visits (n = 45, 56%), followed by cardiology (n = 17, 21%) and ophthalmology (n = 13, 16%). Fifty-one patients (63%) used an interpreter during their face-to-face visit. Most interpretation requests were for Spanish, with additional requests for Cantonese, Mandarin, Nepali, and Vietnamese. An additional 37 individuals were offered appointments but were not seen. Four patients canceled, 4 no longer needed services, 19 were not reachable, 5 declined, 2 did not show up for the appointment, 1 person’s records were not received (a requirement before face-to-face care), and 2 became insured through Medicaid and were thus no longer eligible.
Table 2.
Characteristics of patients in face-to-face visits, August 2013 to December 2014
| Patient characteristics | Patients (N = 81)a |
|---|---|
| Mean age, years (SD) | 45 (13) |
| Sex | |
| Men | 27 (33) |
| Women | 54 (67) |
| Specialty department visited | |
| Allergy/Immunology Services available for asthma, allergic rhinitis, food allergy, chronic sinusitis, urticaria |
1 (1) |
| Cardiology Services available for echocardiograms, Holter monitor, event monitor, treadmill, and nuclear treadmill |
17 (21) |
| Dermatology Services available for acute and chronic disorders of the skin, nails, and hair |
3 (4) |
| Endocrinology Services available for hypothyroidism, hyperthyroidism, medication adjustment recommendations for diabetes Types 1 and 2, osteoporosis, hyperparathyroidism, vitamin D deficiency, low testosterone levels |
0 (0) |
| Gastroenterology Services available for select symptomatic and diagnostic colonoscopies |
45 (56) |
| Ophthalmology Services available for cataract evaluation and removal, severe conjunctivitis unresponsive to treatment, severe dry eyes, and pterygium |
13 (16) |
| Pulmonology Services available for chronic obstructive pulmonary disease |
0 (0) |
| Rheumatology Services available for gout | 2 (2) |
| Interpreter service used during visit | |
| Yes | 51 (63) |
| No | 30 (37) |
| Home safety net | |
| Clinica Family Health Services: Site 1 | 59 (73) |
| Metro Community Provider Network: Site 2 | 17 (21) |
| Salud Family Health Centers: Site 3 | 5 (6) |
Data are presented as no. (%) unless otherwise indicated.
SD = standard deviation.
Review of e-Consult Exchanges
Table 3 summarizes the sample of 54 e-consults. On average, there were 2.3 electronic exchanges per consultation. The primary reasons for e-consults were as follows: treatment recommendations (44%), advice about further evaluation (43%), and diagnosis confirmation (41%). Electronic responses from specialists included advice about medication management (41%), continued monitoring (32%), and laboratory evaluation and procedure recommendations (24% each). Of this sample, 22% of patients were recommended for face-to-face visits. Most specialist recommendations were not followed by explicit responses from primary care clinicians. Response time varied by department, with average turnaround time being 3 days at the beginning of the program and 4 days by the end of 2014.
Table 3.
Summary of review of electronic exchanges, November to December 2014
| Electronic consult (e-consult) factors | E-consultsa |
|---|---|
| Patients receiving e-consults | 54 (100) |
| Mean number of electronic exchanges | 2.3 |
| E-consults with attached image or chart | 32 (59) |
| Reason for safety-net provider originated e-consult | |
| Diagnosis confirmation | 22 (41) |
| Treatment recommendations | 24 (44) |
| Advice about further evaluation | 23 (43) |
| Specialist response | |
| Medication management | 22 (41) |
| Continued monitoring | 17 (32) |
| Recommended procedure | 13 (24) |
| Laboratory evaluation | 13 (24) |
| Recommended face-to-face visit at Kaiser Permanente | 12 (22) |
| Safety-net provider response | |
| No response | 41 (76) |
| Accepts care plan | 8 (15) |
| Other | 5 (9) |
Data are presented as no. (%) unless otherwise indicated. Percentages do not add to 100 because multiple factors in the same category applied in some e-consults.
Interview Findings
Table 4 summarizes characteristics of interview participants. The 18 safety-net clinicians had an average of 10.4 years (range = 0.7–26 years) in practice, and the 3 case managers were relatively experienced (range = 4–15 years). The 12 KP specialists had an average of 12.9 years in practice (range = 1–29 years). Three men and 7 women patients were interviewed, averaging 48 years of age (standard deviation = 13 years).
Table 4.
Characteristics of interview participants: patients, safety-net primary care clinicians and staff, and Kaiser Permanente specialists
| Characteristic | Safety-net clinicians and staff (n = 21) | Kaiser Permanente specialists (n = 12) | Patients (n = 10) |
|---|---|---|---|
| Sex, no. (%) | |||
| Men | 6 (29) | 9 (75) | 3 (30) |
| Women | 15 (71) | 3 (25) | 7 (70) |
| Mean age, years (SD) | — | — | 48 (13) |
| Mean years in practice (SD) | 10 (9) | 13 (9) | N/A |
| Primary care clinician type, no. (%) | |||
| Physician (MD, DO) | 11 (53) | N/A | N/A |
| Midlevel (NP, PA) | 7 (33) | NA | N/A |
| Referral case managers | 3 (14) | N/A | N/A |
| Specialty department, no. | |||
| Allergy/Immunology | N/A | 1 | 1 |
| Cardiology | N/A | 1 | 2 |
| Dermatology | N/A | 2 | 1 |
| Endocrinology | N/A | 1 | 0 |
| Gastroenterology | N/A | 3 | 3 |
| Ophthalmology | N/A | 1 | 3 |
| Pulmonology | N/A | 1 | 0 |
| Rheumatology | N/A | 2 | 1 |
DO = doctor of osteopathy; MD = medical doctor; N/A = not applicable; NP = nurse practitioner; PA = physician assistant; SD = standard deviation; — = not collected.
In keeping with the evaluation goal of informing process improvement, we identified the following primary themes from interviews with clinicians and patients: the program’s responsiveness to a need in the community, how and why the program is being used, the patient experience, and specific areas for process improvement.
Responsiveness to Community Need
Both clinicians and patients agreed that the program responds to a defined need. A safety-net primary care clinician said: “Knowing what the next steps are for diagnosis and workup [is important]. … The face-to-face visits completely change the patients’ lives. They have the workup and diagnosis that never would have been possible without a face-to-face visit.” A referral case manager at a safety-net organization commented: “We don’t have anyone who will see our uninsured patients. … It is great for [the] doctors and especially the patients.”
For KPCO specialists, the program is seen as rewarding and an opportunity to feel like they are making a difference in the community. One said: “It is really nice to do this kind of work. The patients are very appreciative. They are so grateful. You feel like you are really doing something good with this program.”
Patients described having limited options in accessing specialty care and that specialty care services were prohibitively expensive: “I did not have money, I did not have insurance. … I was falling into a depression, and they took me out of it.”
Program Applications
Safety-net primary care clinicians found the e-consult portion of the program to be useful for interpreting results, defining next steps for diagnosis and additional tests, reassuring that they were on track, referring patients for procedures, and as an alternative to in-person specialty care referrals. In addition to seeking advice, primary care clinicians used the program with the hope of having patients be seen for a face-to-face visit if needed. A safety-net primary care clinician said: “Typically there is a plan. Either the consultant confirms diagnosis, or [s/he] would tell us: ‘You might think about this—Plan A if B, or otherwise try this.’ They have sent a plan to follow that I have used prior to going back for more advice. Including 1 to 2 steps in the future is quite useful.”
Primary care clinicians were conscious of their patients’ limited resources, and the e-consult exchange in some cases provided them assurance that they need not refer their patients to a specialist. A safety-net clinician stated: “It gives us a little more assurance if something is necessary vs unnecessary. It reduces the financial risk to [the patients].” Another safety-net clinician remarked that the e-consult was sufficient and a patient did not have to be seen for a face-to-face visit: “I got really good advice. We got the patient on [medications], and so [s/he] didn’t need to be seen.”
Safety-net primary care clinicians also described factors detracting from use of the program. These included time required to complete the e-consult, limited knowledge about the program, the added step to log into the system, considering a phone call or face-to-face visit to have more value, ability to have their questions answered elsewhere, uncertainty about patient eligibility, patients’ increased access to Medicaid (making them ineligible for the program), and patient needs beyond the scope of the program. One safety-net clinician who had used the program on multiple occasions mentioned technology challenges: “Getting data to a specialist can be challenging. I have to take a photo, send it to my work e-mail, download it to my desktop, and then put it into e-consult.”
Neither e-consults nor direct specialty care fully addressed the lack of resources in caring for safety-net patients. For example, one specialist mentioned that his/her advice in some cases might not be applicable, such as “when we ask for imaging or [laboratory studies] that the patient can’t afford to pay for.” A safety-net clinician described the limitations of the face-to-face menu of services in the program: “The response was ‘the patient just needs surgery’, but that is not an option [financially]. … Sometimes there is a disconnect.”
Aspects Facilitating Program Use
Safety-net clinicians made several suggestions to improve program efficiency. For example, one safety-net system uses referral case managers to improve e-consult referral efficiency; the primary care clinician flags a patient’s chart in the internal medical records system, and a referral case manager submits the e-consult. After proving effective at one clinic, this process was extended to all clinics in that safety-net organization. A safety-net clinician observed how the e-consult program has become a part of the workflow: “It is now part of our regular referral process, so it has improved.” Another clinician described the time savings of having referral case managers put information into the KP system as “life-changing.”
Safety-net clinicians also emphasized the importance of clear clinical questions to optimize e-consult communication and including the program as a part of training for new clinicians to encourage their use of the system. A safety-net clinician said: “I try to ask a pointed clinical question. I try to be very specific, and that is probably why I get a detailed response.”
Department-Specific Use
Demand for e-consult and face-to-face care differed by department. Gastroenterology experienced the most requests for face-to-face care, primarily for procedures. “We do not get so many questions. We get a lot of cases where they just need a ‘scope.’ ” The Ophthalmology Department also focused on face-to-face visits: “For Ophthalmology, typically the primary care clinician does not treat the patient. They do not ask how to treat the patient. In the e-consults, they describe the patient in great detail and then add a picture. I would not personally make treatment recommendations third hand. I would not treat without seeing the patient in person.” Cardiology also experienced a relatively large number of e-consults that resulted in face-to-face visits.
By viewing images attached to e-consults, Dermatology provided primarily electronic advice, supplemented by a few face-to-face visits for dermatologic procedures. Likewise, Endocrinology and Rheumatology provided primarily electronic advice through e-consults. Rheumatology recommendations were limited somewhat by high medication costs. A Pulmonology Department specialist considered chest-imaging review via e-consult to be the most appropriate use of the program for that department.
Turnaround time for e-consults also varied by department. One KP specialist pointed out: “It depends on the department, almost everything in [endocrinology] is not acute. We try to reply in 24 hours. For [cardiology], sometimes it probably needs to be in less time.”
Volume and Potential to Expand
Many specialists believed there was room to expand the program beyond the current volume of e-consults; however, this perception varied by department, with high-volume or lower-staff-ratio departments expressing interest in maintaining the program at its current size. Safety-net clinicians commented that expansion to other specialties would be beneficial, specifically mentioning Neurology and Orthopedics.
The Patient Experience
Overall, patients reported positive experiences in face-to-face visits with specialists and the KPCO system. They expressed gratitude recognizing that the cost of services would have been too expensive had they tried to obtain care elsewhere. Logistic challenges were mitigated by providing patients with information on system and visit navigation ahead of time and providing in-person or phone-based language interpreters. Transient logistic challenges that were subsequently resolved included a “no show” of an interpreter and incorrectly being charged for care.
When asked about other options, patients reported that they did not have other options for obtaining specialty care. An Ophthalmology Department patient stated: “The operation was very expensive. … I didn’t have the money. … [My eyes] really bothered me. … It would have cost me around $10,000, and they did it for free. I am really happy.”
Definitive tests were considered very beneficial. One patient remarked: “One simple study makes the difference between being okay or not.” In some cases patients’ health issues were not resolved after their face-to-face visit; some sought care elsewhere afterward, and others described an ongoing need for care. A number of patients expressed hope or the expectation of being seen again by a KPCO specialist in the short or long term.
Areas for Process Improvement
Both specialty and primary care clinicians identified areas for process improvement.
Different Contexts
Given the differences in health care delivery contexts, both KPCO specialists and safety-net primary care clinicians believed that increased sharing of information about each other’s expectations and limitations could increase the value of specialty recommendations. Specifically, clinicians mentioned that sharing internal KPCO specialty guidelines and clarifying available on-site resources at the safety nets would be helpful.
A safety-net clinician said: “I know from providers who work within the KPCO system that there are guidelines: after you do this—x, y, and z—then you can make a referral. It would be helpful to know them. Potentially, if those internal standards were shared with us, it might be useful.” A parallel comment from a KPCO specialist included the need for “providing more information to specialists about where community health center providers are coming from, what resources they have access to, [and] what training they have (eg, if they know how to do a biopsy), to know what kind of advice to give them and what to ask them to do. … Also, it would be good to know, do they have pathology services available?”
Clinician-to-Clinician Communication
Some KPCO specialists were uncertain about the usefulness of their advice because of limited feedback from safety-net primary care clinicians. One KPCO specialist commented: “With the e-consults, I have not received communication back. I have not received any feedback. I assume that they get the answer, and that is what they need. It is more of a question-and-answer communication.” Some safety-net clinicians wished for feedback regarding the quality of submitted e-consults. Specialists believed that the questions were appropriate and comparable to the kinds of questions that they received from KPCO primary care clinicians.
Back-and-forth exchanges in an e-consult could also be seen as inefficient. A KPCO specialist recommended: “Just look at it once instead of the back-and-forth. Then we have to follow-up to ask about the specifics. ‘Oh, did you ask this?’ The exchanges take a lot of time.” Another specialist said: “I think it may almost be better to do a phone call than to do the back-and-forth over e-mail since that can be cumbersome. I wonder whether maybe telephone advice or a telephone consultation line wouldn’t be better.” Several KPCO specialists mentioned that they had used the phone to communicate, whereas others had left their numbers but had not received calls.
Patient Hand-off and Ownership
Although patients received information on navigating the KPCO system, pre- or postvisit information did not always make it to the safety-net clinicians. “For one, the patient was supposed to go for a face-to-face [visit], but there were difficulties with communication, and I am not sure what happened. It was for an [echocardiogram], and I am not aware of the patient going.” In a different situation, a safety-net primary care clinician said: “I just received [laboratory] results. The patient may or may not have received something. I was looking for a specialist’s plan. It is good for me to see a note, … and it looks bad if I see the patient again and I don’t know what was recommended.” In another example, a safety-net clinician commented: “The patient came back and shared information with me. Over the e-consult program I received a message ‘We will see this patient for a face-to-face visit’ and then nothing came back afterward, or at least not that I am aware of.”
These logistic challenges were seen as similar to referrals elsewhere in the community: “You get a message, ‘Your patient will be seen for a face-to-face’ [visit] … . There is not good communication around [when] your patient has an appointment, and then, what [happens] next? It is one of the more frustrating parts of the program—not knowing what happens from there. In the community, with other specialists, it is difficult across the board.” One solution suggested by a safety-net clinician was as follows: “A point person at my clinic could help, or having something in the system with the status, for example, if the patient has been contacted or not.”
Program Scope and Processes
Both safety-net clinicians and KPCO specialists expressed uncertainty about patient eligibility, the process for having a face-to-face visit, and the specific services that could be offered during a face-to-face visit. One safety-net clinician recommended: “There is nothing in the [e-consult] system about how to initiate a face-to-face visit. More specific information and a step-by-step process of this would be helpful.” Another clinician suggested: “I would recommend sharing how often patients get a face-to-face [visit]. It makes it more hopeful for us.” A KPCO specialist was somewhat uncertain about qualifying services for a face-to-face visit: “The challenge was when it was converted to a face-to-face [visit]. E-mails are helpful, about 90% can be done via e-mail—can be taken care of that way. I can do a face-to-face appointment and get a history and do a physical [examination], but after that, getting anything else is what I can’t guarantee.”
DISCUSSION
This partnership between an integrated delivery system and 3 safety-net organizations illustrates a novel approach to providing specialty care services to uninsured individuals in the Denver area. Although the scope of the Safety Net Specialty Care Program was limited by a finite menu of services and specialties, lessons learned from the first 20 months’ experience will inform program expansion and can be applied to other specialty care/safety-net partnerships.
The evaluation showed a number of positive aspects. For several of the specialty departments, such as dermatology, endocrinology, and rheumatology, there were a large number of e-consults and a small number of face-to-face visits that occurred. As such, the e-consult platform presented an alternative to referring patients for in-person appointments with a specialist. Before the program, uninsured patients at the three safety-net organizations did not have a defined mechanism for receiving face-to-face specialty care, at no cost, for the menu of services in the eight participating departments.
Findings from this evaluation suggest several possible dimensions for program improvement and expansion. There is room for expansion in participating specialty departments and the menu of services for face-to-face visits. Other safety-net organizations, specialties, and services may wish to adopt e-consults. Because only half of eligible safety-net primary care clinicians used the program, there is potential for continued program uptake. Uptake may be facilitated by having clinicians share information on program efficiencies (eg, referral case managers) and by acting on recommendations for process improvement (eg, communication about care delivery contexts). More facile communication between clinicians and systems will help build professional relationships—a process that should become self-enhancing. Increased comfort with, and investment in, the e-consult system should prompt ongoing communication about the program’s scope.
The biggest challenge for the program is that although it responds to a defined community need and hopes to expand, it is neither designed for nor capable of resolving all specialty care needs. On an individual level, the current menu of services has limitations, and goals for expansion are unlikely to be easily or completely resourced. The limited scope affects not only uninsured patients who do not meet clinical eligibility criteria but also safety-net patients whose Medicaid insurance disqualifies them from program access. Specialty care needs for Medicaid patients have been highlighted by the Medicaid expansion under the Affordable Care Act and remain a community concern.25 Community-facing delivery systems, such as KPCO, have an opportunity to collectively address specialty care gaps. One effective mechanism may be through regional health alliances focused on improving specialty care access for underserved populations. For example, a safety-net specialty care initiative in California required that grant recipients build on existing collaborative efforts or form coalitions, recognizing that addressing specialty care access requires the involvement of multiple institutions in the community.26
This experience offers a number of lessons for organizations interested in implementing similar programs. Factors considered to be essential to the program’s launching and continued implementation include support from KPCO leadership; dedicated administrative staff time; approval from KPCO Legal, Compliance, and Information Technology Security departments; engaged specialists and safety-net clinicians; established working relationships with safety-net partners; a secure messaging vendor and e-consult platform; and proficiency with electronic medical records. Factors that enabled success were communication about the program and support within departments and safety-net sites, a defined menu of services, and the use of referral case managers in some safety-net settings. Factors that limited uptake include inadequate clinician awareness of the program, the need to log on to another site, and confusion about the scope of the program and which cases are eligible.
This evaluation has a number of limitations. Interviews were conducted with a small number of individuals because of the scope of the evaluation, and this article represents only the perspectives of clinicians and patients who were interviewed. Safety-net primary care clinicians who had not used the e-consult system were underrepresented relative to those who had used the system; their insights may have been particularly useful on how to increase uptake. The three participating safety-net organizations are all federally qualified health care centers and are the largest safety nets in the Denver metropolitan area unaffiliated with the integrated Denver Health and Hospitals system; smaller safety-net practices may have different e-consult and referral patterns, and they may have fewer staff or resources necessary to streamline successful adoption of the program, such as referral coordinators. The evaluation was not structured with a comparison group as it is offered to all clinicians and all eligible patients at the three participating safety-net organizations. Unresolved issues were identified during interviews with patients; however, this information was not captured for all patients who received face-to-face visits. In the future, follow-up interviews with each patient after their care experience would be a way to quantify the extent to which unresolved issues affect patients. There are a number of additional aspects that will be important to evaluate in the future, including patient outcomes, the extent to which Emergency Department utilization is reduced, and the CME portion.
This evaluation raises new operational questions for specialty care programs. For example, it will be important to ask how to most efficiently meet the demand for procedural face-to-face visits, ensure that e-consults are focused on consultative questions that substitute for visits rather than procedure requests, and minimize time-consuming technology-related hassles such as logging onto the e-consult system.2 It will also be crucial to identify and assess new collaborations that can provide hospital and inpatient services as part of specialty care for safety-net patients. Finally, ongoing, community-based, specialty care needs assessments are important to identify high-priority areas for future intervention.
CONCLUSIONS
To our knowledge, the Safety Net Specialty Care Program is one of the first examples nationally of an integrated delivery system partnering with safety-net clinics to offer specialty care and is one of a small number of programs using a combination of e-consults and face-to-face visits. This combination offers a new approach to offer specialty care advice to safety-net clinicians and needed specialty care services for uninsured patients. Key process improvement recommendations from the first 20 months of implementation are to understand the difference in context between the specialty care and the safety-net settings, to support communication and relationship building between clinicians, to identify ways to make hand-offs between settings smoother, and to clarify program scope. Although the program meets the needs of some patients with limited access to specialty care services, ongoing participation in communitywide efforts to evaluate and to address continued needs remains important.
Acknowledgments
The authors would like to thank the participating primary care clinicians and staff from the three safety-net organizations who dedicated time to implement and provide ongoing feedback about this program. Specifically we thank Edward Farrell, MD, and Karen Funk, MD, from Clinica Family Health Services; Tillman Farley, MD, from Salud Family Health Centers; and Jane Lose, MSN, CNM, ANP, Debra Preller, MD, and Christine McLemore, DO, from Metro Community Provider Network. The late Virgilio Licona, MD, from Salud Family Health Centers also assisted.
The authors thank the Colorado Community Health Network in Denver for its administrative role in implementing the program and for providing support during the program evaluation.
The authors also thank the Kaiser Permanente Colorado Community Benefit and Relations Department and Operations Department, and the Colorado Permanente Medical Group leadership for their support for this program. The evaluation was funded by the Kaiser Permanente Colorado Community Benefit and Relations Department.
Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.
Footnotes
Disclosure Statement
The author(s) have no conflicts of interest to disclose.
References
- 1.Cook NL, Hicks LS, O’Malley AJ, Keegan T, Guadagnoli E, Landon BE. Access to specialty care and medical services in community health centers. Health Aff (Millwood) 2007 Sep-Oct;26(5):1459–68. doi: 10.1377/hlthaff.26.5.1459. DOI: https://doi.org/10.1377/hlthaff.26.5.1459. [DOI] [PubMed] [Google Scholar]
- 2.Hall MA. Organizing uninsured safety-net access to specialist physician services. J Health Care Poor Underserved. 2013 May;24(2):741–52. doi: 10.1353/hpu.2013.0076. DOI: https://doi.org/10.1353/hpu.2013.0076. [DOI] [PubMed] [Google Scholar]
- 3.Summer L. The impact of the Affordable Care Act on the safety net [Internet] Washington, DC: AcademyHealth; 2011. Apr, [cited 2016 Jun 6]. Available from: www.academyhealth.org/files/FileDownloads/AHPolicybrief_Safetynet.pdf. [Google Scholar]
- 4.Olayiwola JN, Bodenheimer T, Dubé K, Willard-Grace R, Grumbach K. Facilitating care integration in community health centers: A conceptual framework and literature review on best practices for integration into the medical neighborhood [Internet] San Francisco, CA: UCSF Center for Excellence in Primary Care; 2014. Mar, [cited 2016 Jun 6]. Available from: www.blueshieldcafoundation.org/sites/default/files/publications/downloadable/BSCF_Facilitating_Care_Integration_Mar_2014.pdf. [Google Scholar]
- 5.Neuhausen K, Grumbach K, Bazemore A, Phillips RL. Integrating community health centers into organized delivery systems can improve access to subspecialty care. Health Aff (Millwood) 2012 Aug;31(8):1708–16. doi: 10.1377/hlthaff.2011.1261. DOI: https://doi.org/10.1377/hlthaff.2011.1261. [DOI] [PubMed] [Google Scholar]
- 6.Kirsh SR, Ho PM, Aron DC. Providing specialty consultant expertise to primary care: An expanding spectrum of modalities. Mayo Clin Proc. 2014 Oct;89(10):1416–26. doi: 10.1016/j.mayocp.2014.04.016. DOI: https://doi.org/10.1016/j.mayocp.2014.04.016. [DOI] [PubMed] [Google Scholar]
- 7.Spatz ES, Phipps MS, Wang OJ, et al. Expanding the safety net of specialty care for the uninsured: A case study. Health Serv Res. 2012 Feb;47(1 Pt 2):344–62. doi: 10.1111/j.1475-6773.2011.01330.x. DOI: https://doi.org/10.1111/j.1475-6773.2011.01330.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Shuman AG, Aliu O, Simpson K, et al. Patching the safety net: Establishing a free specialty care clinic in an academic medical center. J Health Care Poor Underserved. 2014 Nov;25(4):1810–20. doi: 10.1353/hpu.2014.0173. DOI: https://doi.org/10.1353/hpu.2014.0173. [DOI] [PubMed] [Google Scholar]
- 9.Arora S, Thornton K, Murata G, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med. 2011 Jun 9;364(23):2199–207. doi: 10.1056/NEJMoa1009370. DOI: https://doi.org/10.1056/NEJMoa1009370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.University of New Mexico School of Medicine; Project ECHO . Project ECHO: A revolution in medical education and care delivery [Internet] Albuquerque, NM: Project ECHO; c2016. [cited 2016 Jun 6]. Available from: http://echo.unm.edu/. [Google Scholar]
- 11.Chen AH, Kushel MB, Grumbach K, Yee HF., Jr Practice profile. A safety-net system gains efficiencies through “eReferrals” to specialists. Health Aff (Millwood) 2010 May;29(5):969–71. doi: 10.1377/hlthaff.2010.0027. DOI: https://doi.org/10.1377/hlthaff.2010.0027. [DOI] [PubMed] [Google Scholar]
- 12.Scheibe MM, Imboden JB, Schmajuk G, et al. Efficiency gains for rheumatology consultation using a novel electronic referral system in a safety-net health setting. Arthritis Care Res (Hoboken) 2015 Aug;67(8):1158–63. doi: 10.1002/acr.22559. DOI: https://doi.org/10.1002/acr.22559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bergus GR, Emerson M, Reed DA, Attaluri A. Email teleconsultations: Well formulated clinical referrals reduce the need for clinic consultation. J Telemed Telecare. 2006;12(1):33–8. doi: 10.1258/135763306775321434. DOI: https://doi.org/10.1258/135763306775321434. [DOI] [PubMed] [Google Scholar]
- 14.Palen TE, Price D, Shetterly S, Wallace KB. Comparing virtual consults to traditional consults using an electronic health record: An observational case-control study. BMC Med Inform Decis Mak. 2012 Jul 8;12:65. doi: 10.1186/1472-6947-12-65. DOI: https://doi.org/10.1186/1472-6947-12-65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Rodriguez KL, Burkitt KH, Bayliss NK, et al. Veteran, primary care provider, and specialist satisfaction with electronic consultation. JMIR Med Inform. 2015 Jan 14;3(1):e5. doi: 10.2196/medinform.3725. DOI: https://doi.org/10.2196/medinform.3725. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Vimalananda VG, Gupte G, Seraj SM, et al. Electronic consultations (e-consults) to improve access to specialty care: A systematic review and narrative synthesis. J Telemed Telecare. 2015 Sep;21(6):323–30. doi: 10.1177/1357633X15582108. DOI: https://doi.org/10.1177/1357633X15582108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kim Y, Chen AH, Keith E, Yee HF, Jr, Kushel MB. Not perfect, but better: Primary care providers’ experiences with electronic referrals in a safety net health system. J Gen Intern Med. 2009 May;24(5):614–9. doi: 10.1007/s11606-009-0955-3. DOI: https://doi.org/10.1007/s11606-009-0955-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mehrotra A, Forrest CB, Lin CY. Dropping the baton: Specialty referrals in the United States. Milbank Q. 2011 Mar;89(1):39–68. doi: 10.1111/j.1468-0009.2011.00619.x. DOI: https://doi.org/10.1111/j.1468-0009.2011.00619.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Colorado Health Institute; Center for the Study of the Safety Net . Colorado safety net specialty care assessment: Final report [Internet] Denver, CO: Colorado Health Institute; 2010. 2011. Mar, [cited 2016 Jun 6]. Available from: www.coloradohealthinstitute.org/uploads/downloads/2010_SpecialtyCare.pdf. [Google Scholar]
- 20.2010 Colorado specialty care demand study: Final report [Internet] Denver, CO: Colorado Health Institute; 2011. [cited 2016 Oct 13]. Available from: www.coloradohealthinstitute.org/key-issues/detail/safety-net-1/2010-colorado-safety-net-specialty-care-assessment-final-report. [Google Scholar]
- 21.2014 Health Center Data: Colorado Program Grantee Data [Internet] Rockville, MD: Health Resources and Services Administration. US Department of Health and Human Services; 2014. [cited 2016 Oct 13]. Available from: http://bphc.hrsa.gov/uds/datacenter.aspx?q+d&year=2014&state=CO#glist. [Google Scholar]
- 22.Fee schedules—general information [Internet] Baltimore, MD: Centers for Medicaid and Medicare Services; 2015. May 19, [cited 2016 Jun 6]. Available from: www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/FeeScheduleGenInfo/index.html. [Google Scholar]
- 23.Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the analysis of qualitative data in evaluation: A procedure for the rapid identification of themes from audio recordings (RITA) Am J Eval. 2014 May 27;36(1):118–32. DOI: https://doi.org/10.1177/1098214014536601. [Google Scholar]
- 24.Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval. 2006 Jun;27(2):237–46. DOI: https://doi.org/10.1177/1098214005283748. [Google Scholar]
- 25.Felland LE, Lechner AE, Sommers A. Improving access to specialty care for Medicaid patients: Policy issues and options [Internet] New York, NY: The Commonwealth Fund; 2013. Jun 6, [cited 2016 Jun 16]. Available from: www.commonwealthfund.org/publications/fund-reports/2013/jun/improving-access-to-speciality-care. [Google Scholar]
- 26.Canin L, Wunsch B. Specialty care in the safety net: Efforts to expand timely access [Internet] Oakland, CA: California Health Care Foundation; 2009. May, [cited 2016 Jun 6]. Available from: www.chcf.org/~/media/MEDIA%20LIBRARY%20Files/PDF/PDF%20S/PDF%20SpecialtyCareOverview.pdf. [Google Scholar]