Specialty Referral Completion Among Primary Care Patients: Results From the ASPN Referral Study

Abstract

PURPOSE This study describes referral completion from the perspectives of patients and primary care physicians and identifies predictors of adherence to the referral recommendation.

METHODS We observed a cohort of 776 referred patients from the offices of 133 physicians in 81 practices and 30 states. Referring physicians and patients completed self-administered questionnaires at the time of the referral decision and 3 months later.

RESULTS Physicians reported that 79.2% of patients referred had a specialist visit, and 83.0% of patients indicated they completed the referral. The most common reasons for not completing the referral were “lack of time” and patient belief that the “health problem had resolved.” The κ statistic for patient-physician agreement on referral completion was 0.34, indicating only fair concordance. Patients in Medicaid plans were less likely than others to complete the referral, and more likely to experience a health plan denial. A longer duration of the patient relationship with the primary care physician and physician/staff scheduling of the specialty appointment were both positive predictors of referral completion.

CONCLUSIONS About 8 in 10 patients referred from primary care complete a specialty referral within 3 months. Findings from this study suggest that referral completion rates may be increased by assisting patients with scheduling their specialty appointments and promoting continuity of care.

INTRODUCTION

Specialty referrals are one of several management options available to primary care physicians. The effectiveness of specialty referrals as a management tool depends not only on the quality of physician decision making—when and for whom to obtain specialty care—but also on the likelihood that patients referred choose to complete the referral by attending one or more specialist visits.

Although much is published on the determinants of primary care physician referral decision making,^1–3 much less is known about referral completion. Based on chart audits and physician survey, prior studies have found that 63% to 82% of patients referred from primary care settings ultimately attend a consultation with the type of specialist to whom they were referred.^4–6 Beyond these descriptive estimates, there is little information on the associations among patient, referring physician, and health care delivery system characteristics with referral completion.

A necessary step toward improving the quality and outcomes of the specialty referral process is developing a better understanding of the determinants of referral completion. To fill this knowledge gap, we report findings from the ASPN Referral Study, a prospective cohort study that examined the content and results of physicians’ referral decisions.^3,7–9 The study was specifically designed to (1) estimate and contrast rates of referral completion from the perspectives of patients and primary care physicians; (2) identify patient, physician, and delivery system predictors of referral completion; and (3) understand from the perspective of patients why some chose not to or could not complete the referral.

METHODS

Physician Recruitment

Physician recruitment was directed to family physician and general internal medicine members of the Ambulatory Sentinel Practice Network (ASPN), physicians affiliated with Medical Group Management Association, other practice-based research networks (Minnesota Academy of Family Physicians Research Network, the Wisconsin Research Network, and the Dartmouth COOP), and the larger community of physicians. The study was publicized by means of direct mailings to physicians, articles in research network newsletters, notices in journals, and presentations at conferences. Telephone contact was made with physicians expressing interest. Physicians were included in the study if they practiced in the United States, were not in training, and participated in both the baseline and follow-up surveys.

Data Collection

The Johns Hopkins School of Public Health’s Committee on Human Research and the Colorado Multi-Institutional Review Board approved study protocols, physician disclosure letters, and patient informed consent forms. Data were collected from August 1997 to April 1999. Before recruiting patients, physicians completed a questionnaire with demographic and practice characteristic items. Each practice selected a coordinator who communicated with the research team, learned study protocols, trained office staff, monitored data quality, and completed a questionnaire about the organizational and financial components of their practices.

During 15 consecutive workdays, physicians and their staff invited all patients who were referred to a specialist to participate in the study. We defined a referral as a physician’s decision to send the patient to see a specialist practitioner (physicians and non-physicians with specialized skills were included) for a face-to-face encounter. Referrals to laboratory and imaging facilities, emergency departments, hospitals for inpatient admission, and curbside consultations in which the referring physician obtains oral advice from a specialist without a patient visit were excluded from the study.

For each referral, physicians and patients (or caregivers for children and youth) completed a questionnaire, and office staff recorded its occurrence in a referral log. The physician baseline questionnaire had items on the type of specialist referred to; the physician’s reason for referral and expectation of the consultant, the expected duration of the specialist’s involvement, urgency of the referral, and duration of the relationship with the patient; and whether the physician or staff made an appointment with the specialist, and whether the physician transferred relevant patient information to the specialist. Only sociodemographic items from the patient baseline questionnaire were used in this analysis.

Approximately 3 months after each referral was made, follow-up surveys were conducted with referred patients and their referring physicians. Practices mailed questionnaires to patients who participated in the baseline survey and who were among the first 10 referrals made by the physician. To enhance response rate, a $2 bill was sent with the patient questionnaire; patients who completed the questionnaire were entered into a $500 lottery.

Because we kept a record of the universe of referred patients, we had a list of all eligible patients. For the 6% of patients with more than one referral, we randomly selected 1 referral for inclusion in the follow-up survey. There were 1,022 patients eligible for the follow-up survey. Of these, 796 patients returned a completed questionnaire (response rate 78%). The questionnaire asked whether patients saw the specific specialist type (eg, cardiologist, general surgeon, etc) to whom they were referred. We customized each patient questionnaire with the name of the referring physician, the date when the referral was made, and the type of specialist referred to.

Referring physicians used medical records to complete follow-up questionnaires and determine referral completion status. Physicians completed questionnaires for 776 of the 796 patients (97% response rate) who participated in the patient follow-up survey. These 776 patients constituted the final patient sample.

Data Analysis

The physician and patient follow-up surveys provided independent assessments of whether the referral was completed. To estimate interobserver concordance, we calculated the percentage of agreement and κ statistic, which corrects for chance agreement.¹⁰ Using patient reports as the criterion, we also estimated the sensitivity, specificity, and predictive values of physician reports of referral completion. These statistics provide information on the degree to which physicians are aware of patient-reported referral completion.

Cross-tabulations with Pearson χ² analysis were done to identify characteristics of patients, physicians, practices, and health plans that may be associated with patient-reported referral completion. An α level of .20 was used to report statistically significant associations with referral completion. This critical value level was selected, rather than the more conventional .05 level, to ensure that we included all variables that may be important covariates in the multivariable logistic regression predictive model.¹¹

The referral completion predictors that were significant in univariate analysis were entered into a stepwise logistic regression analysis with forward selection. The analysis goal was to identify the most parsimonious set of predictors that explains variability in the chances of referral completion. The unit of analysis was the referred patient. We used the generalized estimating equation¹² to account for the clustering of visits within physicians. Data analysis was conducted using STATA statistical software, version 8.¹³ Groups of variables were entered into the regression model as follows: referral decision, physician-patient relationship, physician, practice, and then patient characteristics. The likelihood ratio test¹¹ was used to determine whether the variable was a significant predictor of referral completion. A critical value of 0.05 was selected as the criterion for retaining variables in the final regression model.

RESULTS

The study sample was composed of 776 patients referred by 133 physicians from 81 unique practices located in 30 states. The mean age of patients was 48.2 years (SD, 19.6), and 59.9% were female (Table 1 ▶). Only 3.7% of patients were uninsured, and 11.0% had Medicaid financing. Most referrals (64.0%) were expected to be short-term (less than 3 months). The most common reason for a referral was to obtain a therapeutic procedure (eg, a surgery) or diagnostic test (eg, an endoscopy). Almost one-half of the patient sample had a 5-year or longer relationship with their referring physicians. Referring physicians scheduled the specialty appointment for 78.8% of referrals made and sent to the specialist relevant patient information for 85.3%.

Table 1.

Characteristics of Study Sample

Characteristics	No.	%
Patient characteristics (n = 776 referred patients)*
Age
0–19 y	70	9.0
20–39 y	179	23.1
40–49 y	146	18.8
50–59 y	133	17.2
60–69 y	120	15.5
≥ 70 y	127	16.4
Sex
Male	311	40.1
Female	465	59.9
Patient/parent education
Less than high school degree	121	15.6
High school or some college	374	48.2
College or technical school graduate	207	26.7
Graduate or professional degree	74	9.5
Annual family income
<$20,000	215	30.7
$20,000–$40,000	218	31.1
$40,000–$60,000	123	17.5
>$60,000	145	20.7
Patient race
White	693	91.8
Nonwhite	62	8.2
Health plan characteristics
Health plan payer (n = 648)
Private	408	63.0
Medicaid	71	11.0
Medicare	145	22.3
Uninsured	24	3.7
Gatekeeping and primary care payment type (n=630)
No gatekeeping	294	46.7
Gatekeeping with fee-for-service payment	177	28.1
Gatekeeping with capitated payment	159	25.2
Referral decision characteristics (n = 776 referrals)
Type of specialist referred to
Medical subspecialist	259	33.4
Surgical subspecialist	362	46.6
Obstetrician/gynecologist	34	4.4
Nonphysician clinician	89	11.5
Mental health professional	32	4.1
Primary expectation of specialist
Consultation for advice	81	10.6
Perform a test or procedure	385	50.3
Shared responsibility for ongoing management	220	28.7
Transferred responsibility for ongoing management	80	10.4
Expected duration of specialist involvement
<3 mo	493	64.0
3–12 mo	173	22.5
>12 mo	104	13.5
Urgency of referral—how soon patient should see specialist
<1 wk	265	34.6
1 wk – 1 mo	406	52.9
At the patient’s discretion	96	12.5
Duration of relationship with referring physician
<3 mo	92	12.0
3 mo – 5 y	295	38.5
≥5 y	379	49.5
Referring physician/staff scheduled specialist appointment
No	163	21.2
Yes	605	78.8
Referring physician gave patient the name of a specialist
No	87	11.3
Yes	686	88.7
Referring physician transferred patient information to specialist
No	113	14.7
Yes	655	85.3
Referring physician characteristics (n = 133 physicians)
Years in practice
<5 y	28	21.1
6–14 y	38	28.6
≥15 y	67	50.4
Sex
Male	106	79.7
Female	27	20.3
Specialty
Family physician	123	92.5
General internist	10	7.5
Referring practice characteristics (n = 81)
Location
Urban	67	84.8
Rural	12	15.2
US census division
Northeast	12	14.8
Midwest	28	34.6
South	24	29.6
West	17	21.0
Size
Solo, 2- to 3-physician practice	30	37.0
Large practice (>3 physicians)	51	63.0

* Numbers may not sum to 776 for each variable because of missing data.

Physicians reported that 79.2% (615 of 776) of patients referred had a specialist visit, and 83.0% (644 of 776) of patients reported that they completed the referral (Table 2 ▶). According to 132 referred patients who did not see a specialist, the most common, non–mutually exclusive reasons for choosing not to complete the referral were (1) the patient’s belief that the health problem had resolved (47.5%), (2) lack of time (37.3%), and (3) the patient disagreed with the physician on the need for referral (26.5%). Patients who requested the referral were more likely to complete the referral than those who did not (92.4% vs 81.9%; P=.02).

Table 2.

Patient-Physician Agreement on Referral Completion

	Patient-Reported Referral Completion
Referring Physician-Reported Referral Completion	Yes No. (%)	No No. (%)	Totals
Yes	551 (85.6)	64 (48.5)	615
No	93 (14.4)	68 (51.5)	161
Totals	644 (100.0)	132 (100.0)	776

For 2.3% of all referrals made, patients said that their health care insurance plan did not approve the referral. Patients with Medicaid financing were more likely than their insured counterparts to have their referral denied by their health plan (7.1% vs 1.2%; P<.01).

Patients and physicians agreed on referral completion for 79.3% (615 of 776). The κ statistic for patient-physician agreement was 0.34, representing only fair agreement. Using patients’ reports as the reference standard, the sensitivity of physician reports of referral completion was 85.5%, the specificity was 51.5%, the positive predictive value was 89.6%, and the negative predictive value was 42.2%.

All patient, referral decision, health plan, physician, and practice variables, which are listed in Table 1 ▶, were tested for statistical association with patient-reported referral completion; only those with a P value of less than .20 are shown in Table 3 ▶. Variables not associated with referral completion were (1) patient characteristics—age, sex, education, and income; (2) health plan—gatekeeping arrangements and capitation payment; (3) referral decision—type of specialist referred to, primary expectation of specialist, urgency, referring physician gave name of specialist to patient, and referring physician transferred information to specialist; and (4) physician and practice—sex, census division, and practice size.

Table 3.

Associations Between Patient-Reported Referral Completion and Patient, Referral Decision, Physician, Practice, and Health Plan Characteristics: Univariate Analyses

Predictor	Completed/Total Referrals* No.	Referral Completion %	P Value
Overall	644/776	83.0
Patient race
White	581/693	83.8
Nonwhite	46/62	74.2	.06
Medicaid health plan
Yes	53/71	74.6
No	591/705	83.8	.07
Expected duration of specialist involvement
<3 mo	418/493	84.8
3–12 mo	141/173	81.5
>12 mo	80/104	76.9	.13
Duration of patient-physician relationship
<3 mo	69/92	75.0
3 mo – 5 y	240/295	81.4
≥5 y	325/379	85.8	.04
Referring physician/staff scheduled appointment
No	122/163	74.8
Yes	515/605	85.1	<.01
Referring physician years in practice
<5 y	95/125	76.0
6–14 y	191/229	83.4
≥15 y	358/422	84.8	.07
Referring physician specialty
Internist	41/56	73.2
Family physician	603/720	83.8	.04
Referring physician practice location
Urban	555/674	82.3
Rural	72/80	90.0	.08

Note: Associations with P <.20 only are shown in this table.

* For each bivariate association, denominators may vary because of missing values.

A stepwise logistic regression with forward selection was done to select the most parsimonious set of predictors of patient-reported referral completion (Table 4 ▶). Patients in Medicaid plans were less likely than all others to complete the referral. On the other hand, a longer duration of relationship with the primary care physician and scheduling of the specialty appointment by the referring physician’s office staff were both positive predictors of referral completion.

Table 4.

Predictors of Referral Completion: Logistic Regression Analysis

Predictors of Referral Completion	Adjusted Odds Ratio	95% Confidence Interval
Medicaid health plan
Yes	0.51	0.28–0.92
No	Referent
Duration of patient-physician relationship
<3 mo	Referent
3 mo – 5 y	1.65	0.87–3.13
≥5 y	2.14	1.14–4.00
Referring physician/staff scheduled appointment
No	Referent
Yes	1.91	1.19–3.06

Note: Predictor variables were retained in the final model only if they significantly improved model fit (P <.05 for the likelihood ratio test). Logistic regression with the generalized estimating equation was used to estimate odds ratios and their 95% confidence intervals.

DISCUSSION

According to both patients and referring physicians, about 8 in 10 referrals are completed within 3 months. We attempted to minimize the effects of recall bias by using a follow-up period of 3 months; customizing each patient questionnaire with the specific name of the referring physician, the date when the referral was made, and the specific type of specialist referred to; and having referring physicians use patients’ medical records to complete questionnaires.

Physicians in this study volunteered to participate, and many were active in primary care practice-based research networks. Their referral practices may systematically differ from all US physicians because of this self-selection process. Our physician sample has been compared with a nationally representative sample of primary care physicians who participated in the National Ambulatory Medical Care Survey, and results suggested similar rates of referral overall and by age, sex, and disease status between the 2 groups.⁹ Even though all physician participants were interested in research on specialty referrals, the potential direction of any bias on referral completion estimates is unclear.

We believe the effects of nonresponse bias are likely to be minimal, although 22% of patients did not return the follow-up survey. Physician-estimated referral completion for this subgroup was 76%, which is close to the 80% physician-based estimate for the sample who participated in the follow-up survey. Responders and nonresponders to the patient follow-up questionnaire did not differ significantly in terms of payer status or likelihood that physicians or their staff scheduled the specialty appointment. The nonresponders were more likely than responders to report a relationship of less than 5 years with their primary care physician (39% vs 50%, respectively).

Doctor-patient agreement on which patients actually attended a specialty appointment was only fair. Physicians’ perceptions of whether the referral was completed were sensitive, but not specific. In other words, the largest source of disagreement resulted from physicians believing that patients had completed the referral, while the patients reported that they had not. We found that physicians were no better than a chance flip of a coin in reporting on referral completion when patients had, in fact, not seen the specialist. This low level of agreement could result from primary care physicians not following up on all referred patients or incomplete transfer of information from specialists back to primary care practices, leading to a lack of medical record documentation. It should also be noted that we had no source other than patient and physician reports for verifying whether the referral was actually completed, which raises the question of which source is most valid. We elected to use the patient as the criterion in this study, because referral completion is an event that they experience directly, whereas physicians are in effect proxy observers.

Referral completion awareness could be improved with electronic records that automatically update the patient’s primary care records once a specialist appointment has occurred. Such an electronic system could also be used to notify physicians when patients cancel a specialty appointment, perhaps with the reason for the cancellation appended to the notice. Lack of physician knowledge of referral completion could lead to overuse quality problems, resulting from inadvertent duplication of tests, procedures, or prescribing of medications. False-positive physician reports, on the other hand, could result in patients not receiving needed services.

Scheduling the appointment with the specialist at the time the referral was made had a strong positive effect on referral completion. Other researchers have found a similar association between assistance in scheduling the specialist’s appointment and referral completion for youth referred from school-based health centers,⁵ patients referred from an emergency department to outpatient psychiatry,¹⁴ and patients referred for mammograms.¹⁵ Staff or physician scheduling of the specialty referral appointment is low-cost, practical, and, as this study shows, an important determinant of referral completion.

Research from other countries has found that longer waiting times increase the chances of nonattendance for a new referral visit.¹⁶ One reason why specialty appointment waiting times may decrease referral completion is that patients’ health problems can resolve during the wait, which was the case for about one-half of all patients who did not complete the referral in our study. Another study on nonattendance in a pediatric dermatology clinic found similar rates of health problem resolution.¹⁷ It is unclear, however, whether physicians would similarly report that the problem prompting referral had in fact resolved.

Medicaid insurance was associated with reduced chances of completing the referral. The proportion of patients with Medicaid financing (11%) in this study was low, which suggests that practices with higher proportions would experience reduced rates of referral completion. Other research has found that Medicaid beneficiaries are less likely than those with commercial insurance to access specialty care for chronic care management.¹⁸ One explanation for why referral completion rates were so low for Medicaid patients is that their health plans were more likely than their insured counterparts to deny the referral. Another explanation that we did not examine is that Medicaid patients may have more difficulty scheduling a specialist appointment because of low clinician acceptance of Medicaid payment. Nonetheless, Medicaid financing should alert physician organizations to patients who may need special efforts to facilitate referral completion.

A longer relationship between the patient and the referring physician was another important positive predictor of referral completion. Better continuity between doctors and patients has been associated with reduced rates of primary care no-shows.¹⁹ One reason for this association may be that a longer relationship offers greater opportunities for doctors and patients to align their goals regarding health care treatment and outcomes.²⁰ A shared understanding of common goals may increase patients’ trust that their physicians are acting on their behalf, thereby increasing patient’s motivation to follow specialty referral recommendations made by their doctor. Patient motivation to see the specialist, according to our results, is a key reason for attending the appointment: 9 in 10 patients requesting the referral completed it.

Referral completion involves a negotiation process between doctors and patients. This dialogue is influenced by continuity of the doctor-patient relationship. Not all patients choose to participate in their physicians’ management decisions: 1 in 4 patients who did not complete the referral disagreed with their doctor on the need for a specialist, and about 1 in 2 stated the problem prompting the referral had resolved. Findings from this study suggest that referral completion rates may be increased by assisting patients with scheduling their specialty appointments and promoting continuity of care.