Abstract
Objective
Health information technology (HIT) holds promise in increasing access to rheumatologists by improving the quality and efficiency of referrals, but few studies have examined its use for this purpose. We evaluated the use and impact of a novel electronic referral (eReferral) system in rheumatology in a safety net health system.
Methods
We examined eReferrals over four years. Our primary outcome was use of pre-consultation exchange, defined as back-and-forth communication between referring and specialty care providers, facilitating triage of referrals, requests for more information, or resolution of questions without a visit. We calculated the proportion of eReferrals that underwent pre-consultation exchange, time to reviewer response, and number of visits scheduled. To increase generalizability, we selected a random sample of eReferrals to undergo additional blinded, adjudicated review to assess agreement on appropriateness for pre-consultation exchange.
Results
Between 2008–2012, 2,383 eReferrals were reviewed and 2,105 were eligible for analysis. One-quarter of eReferrals were resolved without a clinic visit. The proportion of eReferrals undergoing pre-consultation exchange increased over time (55% in 2008 vs. 74% in 2011). The volume of referrals steadily increased over time. Reviewer response time averaged between 1–4 days. In the random sample of eReferrals that underwent adjudicated review, agreement between reviewers was high (kappa 0.72).
Conclusion
HIT-enabled pre-consultation exchange was used for a majority of eReferrals and facilitated communication between referring clinicians and rheumatologists. This redesigned system of care allowed for triage of a high number of referrals, with large numbers of referrals determined to be appropriate for pre-consultation exchange.
Despite an aging population and increasing burden of musculoskeletal disease, the United States faces a shortage of rheumatologists (1). To increase access and efficiency, new models of care may be needed, including greater pre-visit collaborations between referring and specialty care physicians. Pre-consultation exchange is a novel model proposed by the American College of Physicians (ACP) to facilitate effective, patient-centered communication between primary and specialty care providers (2, 3, 4). Pre-consultation exchange is a process that facilitates communication among providers during the primary care to specialty care referral process. The goal of pre-consultation exchange is to maximize the efficiency of specialty visits both by answering clinical questions that may not require a formal patient visit and streamlining the pre-specialty visit workup (2,5). While studies regarding pre-consultation exchange and health information technology (HIT) exist for other medical subspecialties such as hepatology (3), to our knowledge, there are no studies examining HIT solutions for pre-consultation exchange in rheumatology. It therefore remains unknown whether this tool would be acceptable or useful in ambulatory rheumatology care.
Two studies from the early 2000s have examined the use of pre-appointment management in rheumatology. In the first study, which involved review of paper-based referrals, only 59% of 279 referred patients actually required a rheumatology consultation (6). In the second, a multi-faceted strategy that aimed to eliminate appointment backlogs using interventions such as redesigning appointment scheduling processes and creating pre-appointment templates for osteoarthritis referrals reduced wait times for the third available rheumatology appointment from about 60 days to less than 2 days (7). However, these studies did not incorporate the use of HIT-enabled electronic iterative communication between the specialist and referring provider.
The safety net health system at San Francisco General Hospital (SFGH) was an early adopter of HIT to facilitate specialty care referrals in order to maximize the scarce resource of specialty providers (5, 8). In this study, we aimed to describe the nature and quality of iterative pre-consultation exchanges with rheumatology between 2008 and 2012, and to determine the impact and appropriateness of pre-consultation exchange in rheumatology ambulatory care.
PATIENTS AND METHODS
The eReferral Program
eReferral was implemented at SFGH in 2007 to facilitate two-way electronic communication between referring and specialty providers for new patient referrals. Primary care and other referring providers initiate all new specialty referral requests through eReferral. The program is embedded within the electronic health record and the information contained in the electronic health record for each patient is available for specialist review. Most patients have their primary care records, laboratories and imaging studies available for review. A smaller subset only has limited information available in the electronic medical record (EMR). About 25% of the patients referred to SFGH are from a Clinic Consortium and these providers do not use our EMR. In these cases, the referring providers type in or fax clinical data. The system automatically populates an electronic form with relevant demographic information, and the reason for consultation is entered as free text in addition to relevant history, physical examination findings, laboratory and other data. The specialist reviewer then evaluates the data and consultation request and determines the outcome of the referral using iterative communication with the requesting provider. All provider communication via eReferral is captured in real time and the system is recorded within the electronic health record. The reviewing rheumatologist has protected and compensated time dedicated to reviewing eReferrals, funded at approximately a 10% full-time employee equivalent of effort. This compensation is generally commensurate with the time spent performing the reviews. The percent effort for reviewing eReferrals can be divided amongst specialty providers at the discretion of individual departments. Each division handles this differently, and in rheumatology, one primary reviewer was preferred. Across the eReferral system, the average time for reviewers to complete an eReferral is 8.1 minutes per referral (2.5 to 15.5 minutes) (9,10). In rheumatology, the approximate review time was also 8 minutes, with a range of approximately 1–40 minutes.
Once a primary care provider initiates an eReferral, several outcomes are possible at the discretion of the reviewing specialist (Figure 1). First, a patient may be scheduled directly into the rheumatology clinic and the reviewing specialist is able to instruct the scheduler to make a routine, expedited, or urgent appointment. Second, the consult may be deemed unsuitable for rheumatology (e.g. the patient may be better served by a referral to another specialty or may not require a specialty consultation). Third, additional information may be requested from the referring provider (e.g. clarification of the referral question to assist with triage, additional history, exam, or results). Fourth, the clinical question posed through the eReferral may be answered entirely within the eReferral system without a scheduled visit. The latter three outcomes constitute pre-consultation exchange (5, 8, 9) (Figure 1).
Figure 1.
Rheumatology eReferral workflow and pre-consultation exchange workflow.
Study Setting and Population
SFGH is a safety net hospital which serves as the primary source of specialty care for uninsured, underinsured and many Medicaid patients in San Francisco. SFGH provides over 200,000 specialty visits annually, serving a network of 25 primary care clinics, including 4 that are based at SFGH. In the ambulatory setting, approximately 35% of patients have Medicaid coverage, 17% Medicare coverage and 37% are uninsured, with the remainder covered through a variety of local and state insurance programs. Approximately one-third of patients do not speak English (11). Within this system, the rheumatology service has over 3,000 patient visits per year.
Using a dataset containing all rheumatology eReferrals, we examined two populations of patients. The first included all patients with an eReferral between January 1, 2008 and May 31, 2012 (“total population”). We excluded eReferrals made by rheumatologists themselves while serving on the Internal Medicine inpatient service, duplicate referrals (as determined by duplicate medical record numbers) and referrals made for dual-energy X-ray absorptiometry (DEXA) scans since these are not performed by the rheumatology service.
Because a single rheumatologist reviewed almost all eReferrals during the study period (JI), we performed additional analyses on a random subsample of patients to increase the robustness and generalizability of our findings. This second population included a 10% random sample of the entire referral cohort (“random sample”). We performed a more detailed, adjudicated clinical review to evaluate the appropriateness and outcome of pre-consultation exchange in this sample (Figure 2).
Figure 2.
Description of exclusion criteria and study populations in rheumatology eReferral study.
Outcomes
In the total population, the primary outcome measures were the proportion of eReferrals that underwent pre-consultation exchange during the study period (2008–2012) and the time to eReferral response by the reviewing rheumatologist.
For the smaller, random sample, each eReferral was rated on its perceived appropriateness for pre-consultation exchange by two reviewing rheumatologists who were blinded to the actual outcome of the eReferral (GS, MM, JG, JY, MR, NR). Pre-consultation exchange appropriateness ratings were modified from a published scale used by Sewell et al. (3). This included rating scales assessing 1) clarity of referral request (not difficult at all to identify reason for consultation, somewhat difficult to identify reason for consultation, very difficult to identify reason for consultation), 2) ability to identify reason for consultation (yes, no), and 3) if the reason for consultation was explicitly stated (yes/no; if yes, categorize into 1 of 36 categories). If the referral was rated as appropriate for pre-consultation exchange, reviewing rheumatologists also judged whether immediate scheduling was recommended. Finally, for those eReferrals that were not felt to require immediate scheduling, reviewers were asked to indicate the reasons pre-consultation exchange might be appropriate (need for additional testing; clinical question requires clarification; rheumatology visit may not be needed). Any disagreements on ratings between reviewers were adjudicated through discussion.
Additional variables
Additional variables examined included patient age, sex, and primary language spoken (English, Spanish, Cantonese, Mandarin, Russian, other/unknown). Referring provider characteristics included the referral source (community clinic or from a clinic physically located at SFGH) and training level (physicians versus other clinicians, which included nurse practitioners or other midlevel providers).
Statistical Analysis
Patient and eReferral characteristics were calculated as mean +/− SD for continuous variables and percentages for categorical variables. To assess the level of agreement for ratings generated by the adjudicated review in the random sample, we calculated a kappa statistic. A value of p<0.05 was considered statically significant for all statistical tests. Statistical analyses were performed using the SPSS statistical software package (IBM/SPSS, Chicago, IL, version 19).
Ethical Considerations
The Committee on Human Research at the University of California, San Francisco and San Francisco General Hospital approved this study.
RESULTS
Between 2008 and 2012, 2383 eReferrals were received by the rheumatology service and 2105 of these were eligible for analysis (Figure 2). The average age of new patient referrals was 53 years (SD 13), 65% were female and 45% were non-English-speaking. The majority of eReferrals were made from community providers (providers outside the physical constrain of SFGH) as opposed to providers physically located at SFGH. Over three quarters of referring providers were physicians, while 22.7% were nurse practitioners or mid-level providers (Table 1).
Table 1.
Characteristics of new patients referred to ambulatory rheumatology clinic and referring providers from January 1, 2008 through May 31, 2012 in ambulatory rheumatology clinic at San Francisco General Hospital (SFGH).
| Characteristics | N=2015 |
|---|---|
| Patient Characteristics | |
| Age, years (SD) | 53 (13) |
| Female, N (%) | 1367 (64.9) |
| Primary language, N (%) | |
| English | 1158 (55.0) |
| Spanish | 316 (15.0) |
| Cantonese | 212 (10.0) |
| Mandarin | 27 (1.3) |
| Russian | 17 (0.8) |
| Other/unknown | 376 (17.7) |
| eReferral Characteristics | |
| Referral Source | |
| Community provider | 1376 (65.4) |
| SFGH provider | 729 (34.6) |
| Referring provider type | |
| Physicians | 1628 (77.3) |
| Nurse practitioner or midlevel providers | 477 (22.7) |
During the study period, the number of total new eReferrals steadily increased, with a slight decrease in the latter six months of the review period. In every year, a substantial number of eReferrals were not scheduled because it was determined they did not need an appointment; this totaled approximately 25% of all eReferrals throughout the study period (Figure 3). A majority of eReferrals underwent pre-consultation exchange, increasing from 55% in the second half of 2008 to 74% in the second half of 2011. The average time to consultant response to primary care providers for each new referral was between 1–4 days (Figure 4). Of the eReferrals that underwent pre-consultation exchange, 63% were subsequently scheduled for a face-to-face appointment. Wait times were stable at 70–80 days (data not shown) even though patient volume increased towards the latter half of the study period.
Figure 3.
Frequencies of new referrals and scheduled referrals from January 1, 2008 through May 31, 2012 in ambulatory rheumatology clinic.
The first periods of 2008 reflect when eReferral was still used inconsistently for rheumatology referral.
Figure 4.
Proportion of referrals undergoing pre-consultation exchange and time to consultant response for all new ambulatory rheumatology referrals from January 1, 2008 through May 31, 2012.
In the random sample of eReferrals retrieved for more detailed clinical review (N=257), the top five reasons for referral were potential inflammatory arthritis, rheumatoid arthritis, pain, systemic lupus erythematosus (SLE) and crystalline arthropathies. In our adjudicated review, 61% (n=156) of eReferrals were rated as appropriate for pre-consultation exchange. Reviewers had good agreement on these ratings (kappa of 0.72, p<0.0001). Of the eReferrals rated as appropriate for pre-consultation exchange, pain was the most frequent consultation question. Referrals for crystal arthropathies were least likely to be rated as appropriate for pre-consultation exchange and rated more appropriate for direct scheduling. The most common reasons for pre-consultation exchange were the need for additional testing as determined by the reviewing rheumatologist or clarification regarding clinical information before triaging the referral or scheduling a visit (Table 2). Approximately one third of the eReferrals were felt not to require a scheduled clinic visit (Table 2).
Table 2.
Pre-consultation exchange appropriateness ratings for the five most common diagnoses in a random sample of eReferrals between 2008–2012.
| Common eReferral Diagnoses | Total eReferrals N (%) |
eReferrals recommended for immediate scheduling N (%) |
eReferrals recommended for pre-consultation exchange N (%) |
|---|---|---|---|
| Potential inflammatory arthritis | 42 (17) | 15 (35) | 27 (65) |
| Rheumatoid arthritis | 35 (14) | 15 (43) | 20 (57) |
| Pain | 31 (13) | 2 (6) | 29 (94) |
| Systemic lupus erythematosus | 20 (8) | 9 (45) | 11 (55) |
| Crystalline arthropathy | 17 (7) | 11 (65) | 6 (35) |
DISCUSSION
In this study, we found that pre-consultation exchange was used for a majority of rheumatology eReferrals in our safety net health system and the use of pre-consultation exchange was found to be appropriate for a majority of referrals (61%). eReferral allowed 25% of referrals to rheumatology to be triaged or managed without a scheduled appointment. We also found that agreement between rheumatologists reviewing eReferrals was generally high, which lends support to the generalizability of the review process.
Communication among clinicians is central to the provision of high-quality health care, and the need for tools that facilitate such communication has grown as specialty care has increased in the United States. The data reported here demonstrate that the use of pre-consultation exchange utilizing an HIT-enabled solution was well utilized after implementation, with almost three-quarters of rheumatology referrals undergoing such exchange as the system matured, an increase from only one-half of rheumatology referrals undergoing such exchange at system implementation. This increase is suspected to be multifactorial, as both primary care providers and the reviewing specialist became more comfortable with the system. Primary care providers are now using eReferral for virtual co-management of chronic conditions (5, 9, 10). In addition, there was rapid care communication from the specialist to the primary care provider, with an average rheumatologist response time ranging 1 to 4 days. Corroborating our findings in rheumatology, eReferral has been found effective in streamlining the referral process in other specialties (3,5,8,12). In addition, our data support the findings in two earlier studies done in rheumatology demonstrating that use of non-electronic pre-appointment management can increase access to rheumatologists (6, 7). We were unable to provide precise estimates of changes in wait times, because during the first periods when eReferral was implemented in rheumatology, there was a simultaneous initiation of a new rheumatoid arthritis clinic. Since this clinic increased the number of providers, the observed decreases in wait time cannot be attributed solely to eReferral. However, other specialty clinics across SFGH were able to document a real decrease in wait times, as outlined in the paper by Chen et al. (10). The stable wait time between 70 to 80 days in the setting of an increased volume of patient referrals suggests that eReferral improve access to specialty care, however, this is preliminary evidence and further work is needed to definitively show this. With the rapid expansion of electronic health records, incorporation of electronic referral and communication systems can likely be scaled more rapidly to facilitate specialty care access for patients.
Provisions in the Affordable Care Act (14) and the National Quality Strategy (15) highlight the urgency of making health care more patient-centered, including improving provider-to-provider communication. Furthermore, accountable-care organizations (ACO) have been proposed as mechanisms to improve quality, reduce costs, and increase access to health care. As health care systems try to integrate the delivery of subspecialty care with primary care, there is increasing importance placed on electronic referral management that facilitates pre-consultation exchange (9,10,15). The use of HIT to facilitate these changes, particularly with the use of pre-consultative exchange, will likely be needed for specialties like rheumatology moving forward. The use of eReferral also has had a positive effect with regards to primary care physicians. In previous surveys of primary care physicians utilizing the eReferral system, 71% felt that eReferral improved care, 71% felt that it provided important guidance for pre-visit evaluation and 89% said they felt that it made tracking referrals easier for them. However, 41% also found the eReferral system more time consuming than the previous paper-based referral system (9, 10). Therefore, the use of HIT and pre-consultation exchange has the potential to help care for the increasing number of new rheumatology referrals, in the setting of rheumatology workforce shortages (1), an aging population, and an increase in the number of Americans with health insurance under the Affordable Care Act.
This study had several limitations. First, there was no validated tool available for reviewing and rating the referrals. However, we used a clinically logical system that has been used previously in other studies (3). Second, no outcomes data on the cohort of patients who were not scheduled in rheumatology was available for analysis. This is an important area requiring further study, since some of these patients may have actually needed rheumatologic care. Third, this study was performed in a retrospective manner, utilizing a chart-review process for data analysis. Fourth, this study was conducted in a safety-net health system with a salaried reviewer, which could limit its generalizability to other health system models. While this system has been widely and successfully implemented at SFGH throughout all departments, the effect of widespread rheumatology-specific provider adoption of similar technology in other health systems is not known. We did not quantitatively evaluate the burden of the additional work review of all referrals had on our single reviewer; the burden of the additional work could affect widespread utilization of this system. Further investigations into the use of HIT and pre-consultation exchange in rheumatology should focus on its effects on provider efficiency, valuation, quality, cost, and improvement of consultations as the eReferral system continues to mature.
In conclusion, this is the first study to comprehensively evaluate the appropriateness of iterative pre-consultation exchange in rheumatology. This study is an important step in evaluating the potential of HIT to facilitate communication between rheumatology and primary care clinicians. Our data suggest that using an iterative electronic referral system that facilitates communication between referrers and rheumatologists has the potential to improve the triage and efficiency of new patient visits, particularly in settings where rheumatologists are a scarce resource.
Significance and Innovations.
The use of health information technology and pre-consultation exchange has the potential to help care for the increasing number of new rheumatology referrals and improve access to care.
Using an iterative electronic referral system that facilitates communication between referrers and rheumatologists has the potential to improve the triage and efficiency of new patient visits.
Acknowledgments
Grant Support:
Research reported in this manuscript was supported by the American College of Rheumatology’s Ephraim P. Engleman Endowed Resident Research Preceptorship and the National Institute of Arthritis and Musculoskeletal and Skin diseases under award numbers K23 AR060259 and P60 AR053308. The content is solely the responsibility of the authors and does not necessarily represent the official views of the American College of Rheumatology or the National Institutes of Health. Funding from the Rosalind Russell Medical Research Center for Arthritis at the University of California, San Francisco also supported this work.
The authors wish to thank Matt Reimert, MD and Nicole Richman, MD for their assistance in performing the blinded, adjudicated review of eReferrals for this study in addition to Richard Shaw MA, PhD, for his assistance with statistical analysis.
Footnotes
Disclosures: No financial or other commercial relationships.
References
- 1.Deal CL, Hooker R, Harrington T, Birnbaum N, Hogan P, Bouchery E, Klein-Gitelman M, Barr W. The United States rheumatology workforce: supply and demand, 2005–2025. Arthritis Rheum. 2007 Mar;56:722–9. doi: 10.1002/art.22437. [DOI] [PubMed] [Google Scholar]
- 2.Kirschner N, Greenlee MC. Policy Paper. American College of Physicians; 2010. The Patient-Centered Medical Home Neighbor: The Interface of the Patient-Centered Medical Home with Specialty/Subspecialty Practices. [Google Scholar]
- 3.Sewell JL, Guy J, Kwon A, Chen AH, Yee HF., Jr Preconsultation exchange for ambulatory hepatology consultations. Am J Med. 2013;126:523–8. doi: 10.1016/j.amjmed.2012.12.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Foy R, Hempel S, Rubenstein L, Suttorp M, Seelig M, Shanman R, et al. Meta-analysis: effect of interactive communication between collaborating primary care physicians and specialists. Ann Intern Med. 2010;152:247–58. doi: 10.7326/0003-4819-152-4-201002160-00010. [DOI] [PubMed] [Google Scholar]
- 5.Chen AH, Murphy EJ, Yee HF., Jr eReferral--a new model for integrated care. N Engl J Med. 2013;368:2450–3. doi: 10.1056/NEJMp1215594. [DOI] [PubMed] [Google Scholar]
- 6.Harrington JT, Walsh MB. Pre-Appointment Management of New Patient Referrals in Rheumatology: A key Strategy for Improving Health Care Delivery. Arthritis Care & Research. 2001;45:295–300. doi: 10.1002/1529-0131(200106)45:3<295::AID-ART263>3.0.CO;2-3. [DOI] [PubMed] [Google Scholar]
- 7.Newman ED, Harrington TM, Olenginski TP, Perruquet JL, McKinley K. The rheumatologist can see you now”: Successful implementation of an advanced access model in a rheumatology practice. Arthritis Rheum. 2004;51:253–7. doi: 10.1002/art.20239. [DOI] [PubMed] [Google Scholar]
- 8.Kim Y, Chen AH, Keith E, Yee HF. Not Perfect, but Better: Primary Care Providers’ Experiences with Electronic Referrals in a Safety Net Health System. J Gen Intern Med. 2009;24:614–9. doi: 10.1007/s11606-009-0955-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kim-Hwang JE, Chen AH, et al. Evaluating Electronic Referrals for Specialty Care at a Public Hospital. J Gen Intern Med. 2010;25:1123–8. doi: 10.1007/s11606-010-1402-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Chen AH, Kushel MB, Grumbach K, Yee HF. Practice Profile: A Safety-Net System Gaines Efficienies Through ‘eReferrals’ To Specialists. Health Affairs. 2010 May;29:5. doi: 10.1377/hlthaff.2010.0027. [DOI] [PubMed] [Google Scholar]
- 11.San Francisco General Hospital Foundation. [Accessed September 12, 2013];About San Francisco General Hospital and Trauma Center. http://sfghf.net/about_sfgh.html.
- 12.Sewell JL, Day LW, Tuot DS, Alvarez R, Yu A, Chen AH. A Brief, Low-cost Intervention Improves the Quality of Ambulatory Gastroenterology Consultation Notes. Am J Med. 2013 Aug;126:732–8. doi: 10.1016/j.amjmed.2013.02.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Affordable Care Act. Title III-Improving the Quality and Efficiency of Healthcare. [Accessed September 12, 2013];Subtitle F-Health Care Quality Improvements. 2010 Mar 23; http://www.hhs.gov/healthcare/rights/law/index.html.
- 14.National Health Strategy. [Accessed September 12, 2013]; http://www.nhsq.info/
- 15.Greenberg JO, Barnett ML, Spinks MA, Dudley JC, Frolkis JP. The “medical neighborhood”: integrating primary and specialty care for ambulatory patients. JAMA Intern Med. 2014;174:454–7. doi: 10.1001/jamainternmed.2013.14093. [DOI] [PubMed] [Google Scholar]