Errors in Completion of Referrals among Urban Older Adults in Ambulatory Care

Abstract

RATIONALE, AIMS, AND OBJECTIVES

Clinical care often requires referrals, but many referrals never result in completed evaluations. We determined the extent to which referral-based consultations were completed in a U.S. medical institution. Factors associated with completion were identified.

METHOD

In cross-sectional analysis, we analyzed billing records and electronic and paper-based medical records, for patients 65 or more years of age receiving healthcare between July 2000 and June 2002 in an integrated, urban, tax-supported medical institution on an academic campus. All referrals in ambulatory care, scheduling of consultation within 180 days, and completion were assessed. We conducted multivariate survival analysis to identify factors associated with completion.

RESULTS

We identified 6,785 patients with encounters. Mean age was 72 years, with 66% women, 55% African-American, and 32% Medicaid-eligible. Of 81% with at least one primary-care visit in ambulatory care, 63% had at least one referral. About 8% of referrals required multiple orders before an appointment was scheduled. Among 7,819 orders for specialty consultation in ambulatory care, 71% led to appointments, and 70% of appointments were kept (completed = 0.71*0.70 or 50%). Scheduling of consultations varied (12% to 90%) by specialty. Medicare, singular orders, location of referral, and lack of hospitalization were independently significantly associated scheduling of appointments.

CONCLUSIONS

Among older adults studied, half of medical specialty referrals were not completed. Multiple process errors likely contribute to these results, including missing information, misguided referrals, and faulty communications. Information systems offer important opportunities to improve the referrals process.

INTRODUCTION

In ambulatory medical care, creating a referral for specialty care requires both articulation of a problem and effective communication of the information. An appropriate referral incorporates information about the patient and the reason for referral, such as a question to be answered or a request for a procedure or treatment. As the information in the referral flows to people and sites of care, a cohesive communications system must ensure integrity of data flow. Although up to 20% of patients are referred to specialists,^{1, 2} specialty consultants and referring clinicians often disagree about reasons for consultation, and faulty exchanges of information are common.^3–5 Consequently, many referrals are either inappropriately conceived or fail to be completed as intended.⁶

A “wrong plan” or a failure of a planned action to be completed as intended defines error, according to the Institute of Medicine.⁷ A fraction of errors occurs before the consultant evaluates the patient, such as when an appointment is being arranged for the patient to visit the consultant. Addressing such errors requires knowing how many there are and in which patients or sites the errors occur. We characterized completion of primary-care referrals among older patients in a U.S. academic medical center. We hypothesized that at least 25% of referrals would not lead to evaluation of patients. This would have implications for quality of care and redesign of healthcare delivery.

METHODS

Setting

We conducted a two-year cross-sectional analysis of patients 65 or more years of age receiving healthcare in an integrated, urban, tax-supported, Midwestern, county-based institution in the U.S. The institution’s hospital sits on the campus of an academic medical center. Participants received health services between 01 July 2000 and 30 June 2002. All orders for consultation are recorded in an electronic system,⁸ which also stores records of clinic encounters and hospitalization. Data include demographic details such as date of birth, race, gender, and insurance. We extracted these data for participants, for all ambulatory-care referrals during the study period. The university’s Institutional Review Board approved the study.

Classification of specialties

To include analysis of whether completion of consultation was related to specialists’ use of invasive procedures, we provided a simple classification of specialties according to relative predominance of invasive procedures: “frequent” (anesthesiology, breast clinic, breast oncology clinic, dental, neurosurgery, orthopedics, otolaryngology, plastic surgery, surgery, trauma, urology, vascular surgery), “occasional” (allergy, burn clinic, cardiology, continence clinic, dermatology, gastroenterology, nephrology, obstetrics and gynecology, ophthalmology, pain clinic, podiatry, pulmonary, rehabilitation medicine, rheumatology), or “infrequent” (anticoagulation clinic, endocrinology, geriatrics, hematology, house calls, infectious diseases, neurology, neuropsychology, nutrition, occupational therapy, oncology, physical therapy, psychiatry, weight clinic).

Outcomes

Since filling clinics to capacity could interfere with prompt scheduling, we conservatively allowed 180 days from time of referral for appointments to be scheduled. Outcomes were fraction of referrals leading to an appointment within 180 days of referral, and fraction of referrals and appointments leading to specialty consultations (kept visits). Although our institution does not provide care for all residents of the community, nearly all referrals in our system lead to in-house consultation visits that can be counted. To determine whether findings may have changed since the time when these data were collected, we also spot-checked a separate sample of nearly 20,000 orders from 2005.

Specialty consultation in ambulatory care

We characterized participants by age, gender, and Medicaid status. The state-administered Medicaid program primarily serves individuals with low income or certain disabilities. We identified fractions of participants with a visit to a primary-care clinic in any of the institution’s community health centers. Since specialties receiving few referrals might handle referrals differently from the rest of the group, we excluded referrals to a specialty averaging less than one per month. For the remainder, we summarized outcomes by targeted specialty and patients’ gender, race, age, invasive procedures, insurance, hospitalization, location at time of referral, and whether multiple referrals for a specialty occurred. Multiple orders may make completion of consultation more likely and may reflect greater perceived importance. We did not examine reasons for referral or identify whether specific referrals (e.g., gynecology) were generated to transfer primary care. For many patients in the institution, Medicaid or a specially designated, tax-supported, local fund serve as primary or secondary payer. The federally administered U.S. Medicare program provides at least partial healthcare insurance coverage to nearly all Americans 65 or more years of age and to small groups of others with selected medical conditions. In this study, having Medicare was defined as documentation of Medicare as a primary or secondary payer.

Some subjects did not have six months of followup, due to leaving or having a specialty consultation near the period’s tail. To account for varying follow-up times, we used survival analysis, modeling time to scheduled appointment. Scheduled appointments were defined as appointments scheduled within 180 days of referral. For referrals with no corresponding specialty appointment, appointment time was set as the minimum of 180 days or days to the last recorded encounter. To identify factors independently associated with completion of specialty visits, specialty, gender, race, age, insurance, hospitalization, whether multiple orders were generated, and location from which referral was generated were included in a proportional hazards regression model. We omitted frequency of invasive procedures, due to colinearity with specialty. P-values of 0.05 or less were considered statistically significant.

RESULTS

We identified 6,785 patients with encounters in the hospital or ambulatory care. Mean age was 72 years; 66% were women, 55% were African-American, and 32% were Medicaid recipients.

Specialty consultation in ambulatory care

The mean number of primary-care and specialty visits in ambulatory care over two years was 7 and 11, respectively. Of patients with primary ambulatory-care visits, 3,472 (63%) had at least one order for consultation, excluding orders for primary care and geriatrics. Among these, the mean number of unique specialty consultation orders was two. About 8% of unique orders required more than one order for the same specialty before an appointment was scheduled. Among 7,819 referrals in ambulatory care, 71% led to appointments, and 70% of appointments were kept. Specialty appointments scheduled within 180 days of referral varied widely by specialty, from 12% to 90%. Table 1 shows referrals’ additional characteristics. In unadjusted analysis, appointments corresponding to referrals were significantly more likely to be kept among specialties with frequent (76%) or infrequent (76%) but not occasional (66%) procedures, patients with Medicare (71% vs. 63%), patients not previously hospitalized (71% vs. 55%), and patients referred from certain clinical sites, such as gynecology (93%) or emergency (82%). Primary-care medicine and geriatrics clinics had the lowest rates of kept visits among scheduled appointments: approximately 70%. Examining a separate set of 19,294 orders from 2005 showed that 54% of orders for specialty consultation led to an appointment.

Table 1.

Characteristics of referrals in ambulatory care (N=7,819)

		Percentage of referrals scheduled within 180 days	Kept visits
Characteristic	Referrals N (%)		Percentage of scheduled appointments	Percentage of referrals*
Gender
Female	5567 (71)	71	71	51
Male	2249 (29)	70	68	48
Race				**
White	3339 (44)	72	71	51
African-American	4156 (55)	71	70	50
Other	100 (1.3)	68	74	50
Age (years)
65–69	3238 (41)	71	70	49
70–74	2209 (28)	72	72	52
75–79	1356 (17)	70	69	49
80–84	611 (7.8)	70	72	51
≥ 85	405 (5.2)	71	66	47
Invasive procedures		**	**	**
Frequent	1814 (23)	72	76	55
Occasional	4033 (52)	74	66	49
Infrequent	1972 (25)	64	76	48
Medicaid				**
No	4976 (64)	71	71	51
Yes	2843 (36)	71	69	49
Medicare			**	**
No	719 (9.2)	69	63	43
Yes	7100 (91)	71	71	51
Any hospitalization		**	**	**
No	7193 (92)	73	71	52
Yes	626 (8.0)	53	55	29
Patient’s location at time of order		**	**	**
Primary Care	6303 (81)	72	69	50
Emergency	515 (6.6)	68	82	55
Geriatrics	557 (7.1)	66	69	46
Gynecology	19 (0.2)	79	93	74
Women’s Health	56 (0.7)	66	78	52
Other subspecialty	236 (3.0)	61	75	45
Other	133 (1.7)	64	84	53
Multiple orders for consultation		**		**
No	7387 (95)	71	70	50
Yes	432 (5.5)	65	72	47

^*Percentage of referrals kept = percentage of referrals scheduled × percentage of scheduled appointments kept

^**p ≤ 0.05 for column-variable (i.e., among values represented by this row)

Survival analysis

Results of survival analyses for ambulatory patients are shown in Table 2. Many targeted specialties had significantly lower rates of scheduling and kept visits than the reference category. Also significantly associated with appointments scheduled within 180 days of referral were Medicare (hazard ratio, HR, of 1.1), hospital admission (HR 0.5), multiple orders per referral (HR 0.6), and location of referral. Female gender (HR 1.1) and Medicare (HR 1.2) were significantly associated with more kept visits, while Medicaid (HR 0.9), a hospital admission (HR 0.6), and multiple orders per referral (HR 0.8) were significantly associated with a lower rate of kept visits.

Table 2.

Multivariate survival analysis of referrals in ambulatory care (N= 7,819)

Characteristic	Percentage of referrals scheduled within 180 days Hazard ratio	Kept visits
		Percentage of scheduled appointments Hazard ratio	Percentage of referrals Hazard ratio
Specialty, clinic, or site targeted by referral	*	*	*
Hematology	0.79	0.87	0.88
Dermatology (reference)	1.00	1.00	1.00
Otolaryngology	1.18**	1.16	1.15
Rheumatology	0.51**	0.79	0.77**
Cardiology	0.66**	0.85	0.88
Nephrology	0.77**	0.59**	0.66**
Surgery	0.74**	0.90	0.85
Vascular surgery	0.77**	0.66**	0.69**
OB/GYN	0.70**	0.80**	0.78**
Nutrition	0.80**	0.74**	0.63**
Plastic surgery	0.85	1.52	1.16
Podiatry	0.49**	0.64**	0.61**
Neurology	0.42**	0.67**	0.61**
Continence clinic	0.50**	1.02	0.75
Pulmonary	0.47**	0.68**	0.66**
Anticoagulation clinic	0.67**	3.00**	1.38
Oncology	1.01	2.30**	1.50**
Trauma	0.80	0.61	0.63
Geriatrics	0.52**	1.11	0.82
Ophthalmology	0.33**	0.45**	0.42**
Orthopedics	0.47**	0.84**	0.65**
Urology	0.52**	1.13	0.80**
Neurosurgery	0.39**	0.68**	0.54**
Physical therapy	0.44**	1.40**	0.69**
House calls	0.35**	1.76	0.87
Rehabilitation medicine	0.23**	0.70	0.34**
Psychiatry	0.20**	1.15	0.41**
Endocrinology	0.19**	0.57**	0.29**
Gastroenterology	0.14**	0.58**	0.22**
Occupational therapy	0.15**	1.79**	0.27**
Pain clinic	0.10**	0.71	0.17**
Dental	0.07**	1.18	0.18**
Neuropsychology	0.07**	0.40	0.08**
Gender, female	1.03	1.12**	1.12**
Race		*	*
White (reference)	1.00	1.00	1.00
African-American	0.98	1.01	0.99
Other	1.22	1.70**	1.66**
Age Categories
65–69 (reference)	1.00	1.00	1.00
70–74	1.01	1.02	1.02
75–79	0.98	0.94	0.93
80–84	1.00	1.00	0.97
≥ 85	1.13	0.94	0.99
Medicare	1.13**	1.15**	1.22**
Medicaid	1.03	0.92**	0.94
Any hospitalization	0.45**	0.60**	0.40**
Multiple orders for consultation	0.62**	0.75**	0.75**
Patient’s location at time of referral	*	*	*
Primary (reference)	1.00	1.00	1.00
Emergency	1.36**	2.40**	1.72**
Geriatrics	1.16**	1.10	1.19**
Gynecology	1.71**	4.82**	3.12**
Women’s health	1.38	1.57**	1.61**
Other subspecialty	0.96	1.21	1.00
Other	1.24	1.56**	1.40**

^*p ≤ 0.05 for column-variable

^**p ≤ 0.05 for value compared to reference value for this variable

OB/GYN = obstetrics/gynecology

DISCUSSION

In many medical institutions, access to specialty services is problematic. Although specialists often provide key diagnoses or treatment, delays in diagnosis have been linked to poor clinical outcomes.⁹ Unlike a general primary-care population, most older patients with ambulatory-care visits experienced referral within two years. Despite high prevalence of referral among this population, only 71% of referrals led to appointments, and only 70% of appointments were kept, resulting in an overall completion rate of only 0.71 × 0.70 or 50%.

Much of the 50% failure rate stems from errors in care. Failures can occur through various pathways, including lost paperwork, faulty communication, and missing clerical or clinical details. Referrals requiring multiple orders and those for patients with previous hospitalization were less likely to be independently associated with scheduling an appointment. Hospitalized patients are sicker and in some institutions may be more difficult to schedule, due to readmission or difficulties finding patients to confirm appointments. In our institution, however, once a decision to refer is made, appointments are scheduled without consulting patients. Although not counted here, multiple orders for specialty consultation in ambulatory care may include referrals generated upon hospital discharge. In these cases, a subsequent duplicate referral from an ambulatory-care site may not lead to a duplicate appointment, and the computed scheduling rate would decrease. This sequence of events likely represents a faulty transition as the patient moves from hospital to ambulatory care. In another study of 569 discharge summaries, for example, only 27% were received by the patients’ primary-care providers.¹⁰ Such lapses of information flow across sites of care should prompt plan managers to address transition points in greater detail.

Although many failures of referral stem from errors in care, some do not. Differences between scheduled appointments and kept visits often reflect patients’ choices, comorbid illnesses, or social support, such as transportation to see specialists. In completion of referral, variation by anticipated frequency of invasive procedures may be related to financial incentives or patients’ expectations or motivations to pursue care; verifying this would require investigation. Although this study’s findings—especially by specialty—would likely differ somewhat among institutions, overall trends are similar in many regions of the U.S. For example, our findings are similar to those of an earlier study reporting that 63% of 5,172 patients referred from family practice kept appointments.¹¹ The variation by specialty should prompt plan administrators to conduct similar analyses and pay particular attention to specialties with the worst outcomes.

Better information systems are likely to improve several aspects of care.^12–20 For clinicians, systems can be designed to identify multiple orders or appointments²¹. Systems can provide more effective training about referrals, more complete clinical information, and better tools for decision support^{22, 23} and documentation.^{24, 25}

Communications systems are also critical for effective referral. Systems for electronic messaging between primary- and secondary-care clinicians can be especially useful.²⁶ Walter Reed Army Medical Center implemented an “Ask a Doc” system based on electronic mail, with average response times to specialty consultation of less than one day.²⁷ Although messaging systems help, referrers and specialty consultants benefit from talking personally and agreeing on goals.^{28, 29} One study of 85 practices by Forrest et al. showed a 3-fold increase in completion of referrals when referring physicians scheduled appointments and directly communicated with specialists.³⁰ Systems interventions that target multiple points in the process of specialty consultation may improve outcomes further.

For patients, several institutions have created electronic portals to facilitate arranging appointments. Regardless of patients’ ages, this could increase patients’ activation, decrease time needed for staff to arrange appointments, and increase the rate of kept visits. Alternative approaches may be needed to accommodate patients with low health literacy. For older or disabled patients, programs offering transportation or other social support services may increase the chance that a scheduled appointment will be kept.

Our study has limitations. We did not assess reasons for failed orders and did not have data about individual clinics’ criteria for scheduling or whether referring clinicians knew about the criteria. We could not assess whether patients were aware of orders or had discussions with clinicians about plans for referral. We also did not assess patients’ distances from the medical center or severity of medical conditions, which may be related to referrals or outcomes. We did not assess clinical outcomes and whether outcomes depend on completion of consultation. Since self-referrals constitute a minority of referrals, however, a referral’s failure would usually preclude any chance for a specialist to improve clinical outcomes.

In summary, half of specialty referrals for older adults in our system were not completed. Although many factors contribute, the findings beg for better information systems that can retrieve or verify clinical and demographic information and improve quality and efficiency of communication among departments and with patients. To improve care, the systems interventions needed to coordinate referral and specialty consultation across an institution need support from institutional leaders, as well as collaboration among clinical teams and systems professionals.