Change in appropriate referrals to nephrologists after the introduction of automatic reporting of the estimated glomerular filtration rate

Abstract

Background:

Use of the serum creatinine concentration, the most widely used marker of kidney function, has been associated with under-reporting of chronic kidney disease and late referral to nephrologists, especially among women and elderly people. To improve appropriateness of referrals, automatic reporting of the estimated glomerular filtration rate (eGFR) by laboratories was introduced in the province of Ontario, Canada, in March 2006. We hypothesized that such reporting, along with an ad hoc educational component for primary care physicians, would increase the number of appropriate referrals.

Methods:

We conducted a population-based before–after study with interrupted time-series analysis at a tertiary care centre. All referrals to nephrologists received at the centre during the year before and the year after automatic reporting of the eGFR was introduced were eligible for inclusion. We used regression analysis with autoregressive errors to evaluate whether such reporting by laboratories, along with ad hoc educational activities for primary care physicians, had an impact on the number and appropriateness of referrals to nephrologists.

Results:

A total of 2672 patients were included in the study. In the year after automatic reporting began, the number of referrals from primary care physicians increased by 80.6% (95% confidence interval [CI] 74.8% to 86.9%). The number of appropriate referrals increased by 43.2% (95% CI 38.0% to 48.2%). There was no significant change in the proportion of appropriate referrals between the two periods (−2.8%, 95% CI −26.4% to 43.4%). The proportion of elderly and female patients who were referred increased after reporting was introduced.

Interpretation:

The total number of referrals increased after automatic reporting of the eGFR began, especially among women and elderly people. The number of appropriate referrals also increased, but the proportion of appropriate referrals did not change significantly. Future research should be directed to understanding the reasons for inappropriate referral and to develop novel interventions for improving the referral process.

Until recently, the serum creatinine concentration was used universally as an index of the glomerular filtration rate (GFR) to identify and monitor chronic kidney disease.¹ The serum creatinine concentration depends on several factors, the most important being muscle mass.¹ Women as compared with men, and elderly people as compared with young adults, tend to have lower muscle mass for the same degree of kidney function and thus have lower serum creatinine concentrations.^2,3 Consequently, the use of the serum creatinine concentration is associated with underrecognition of chronic kidney disease, delayed workup for chronic kidney disease and late referral to nephrologists, particularly among women and elderly people. Late referral has been associated with increased mortality among patients receiving dialysis.^3–11

In 1999, the Modification of Diet in Renal Disease formula was introduced to calculate the estimated GFR (eGFR).^12,13 This formula uses the patient’s serum creatinine concentration, age, sex and race (whether the patient is black or not). All of these variables are easily available to laboratories except race. Laboratories report the eGFR for non-black people, with advice to practitioners to multiply the result by 1.21 if their patient is black. Given that reporting of the eGFR markedly improves detection of chronic kidney disease,^14,15 several national organizations recommended that laboratories automatically calculate and report the eGFR when the serum creatinine concentration is requested.^16–19 These organizations also provided guidelines on appropriate referral to nephrology based on the value.

Although several studies have reported increases in referrals to nephrologists after automatic reporting of the eGFR was introduced,^20–26 there is limited evidence on the impact that such reporting has had on the appropriateness of referrals. An increase in the number of inappropriate referrals would affect health care delivery, diverting scarce resources to the evaluation of relatively mild kidney disease. It also would likely increase wait times for all nephrology referrals and have a financial impact on the system because specialist care is more costly than primary care.

We conducted a study to evaluate whether the introduction of automatic reporting of the eGFR by laboratories, along with ad hoc educational activities for primary care physicians, had an impact on the number and appropriateness of referrals to nephrologists.

Methods

Study design

We used a quasi-experimental design with interrupted time-series analysis to estimate the effects of automatic reporting of the eGFR on the number and appropriateness of referrals.²⁷ Data were collected at multiple times before automatic reporting began, to identify the underlying trend in referrals, and after reporting began, to determine the estimated effect of the intervention after accounting for the underlying trend. The study was approved by The Ottawa Hospital Research Ethics Board.

Participants

We included all patients referred by primary care physicians to the nephrology service of The Ottawa Hospital during the year before automatic reporting was introduced (Mar. 7, 2005, to Mar. 6, 2006) and the year after its introduction (Mar. 7, 2006, to Mar. 6, 2007). We excluded patients if they were referred by specialists, were already known to the nephrology service or did not have chronic kidney disease. A referral was considered to be for chronic kidney disease if the patient was referred because of decreased kidney function (e.g., decreased creatinine clearance, renal failure, renal disease or decreased GFR) or proteinuria (including albuminuria).

Setting

The study was conducted at The Ottawa Hospital, a tertiary care teaching hospital with 18 nephrologists. The hospital’s nephrology service is the only one in Ottawa and serves the majority of nephrological needs in the Champlain Local Health Integration Network, the regional health authority of eastern Ontario. The region has about 825 primary care physicians sending referrals. Referrals to two nephrologists who practise at a secondary care centre were not included in this study.

The area served by the Champlain Local Health Integration Network has a population of 1 176 600, of whom 12.5% are 65 years or older, 17% are immigrants and 13% are from visible minorities.²⁸ Given the distance to other referral centres, most of the residents receive health care within the region.²⁸ Because 90% of the nephrologists in the Champlain Local Health Integration Network practise at The Ottawa Hospital, the sample is likely representative of the local population.

Intervention

The primary intervention was the province of Ontario’s introduction of automatic reporting of the eGFR by laboratories for all adult outpatients, which began in the first week of March 2006. When a serum creatinine concentration was requested, the laboratory provided both that value and the eGFR. All reports of eGFRs were accompanied by a prompt from the laboratory that included an explanation of eGFR ranges.¹⁷

Concurrent with the introduction of automatic reporting of the eGFR, the nephrology service mailed an algorithm to all primary care physicians in the Champlain Local Health Integration Network (Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.110678/-/DC1). This algorithm explained the interpretation of the eGFR and appropriate parameters for referrals to nephrology, based on the value. In addition, ad hoc educational sessions (lectures and workshops) were provided to the primary care physicians to discuss interpretation of the eGFR results and parameters for referral to nephrology.

Data collection

A trained research assistant (M.C.-F.) collected data from the referral packages using a standard two-part form developed for the study. The form was based on the Canadian guidelines for referral to nephrology.¹⁶ To verify the accuracy of the data, one of us (A.A.) performed an audit of a random sample of 5% of the referral packages throughout the data-collection period. The inter-rater agreement was strong (kappa = 0.90, 95% confidence interval [CI] 0.87 to 0.94).

Outcome measures

The primary outcome was the number of appropriate referrals per two-week interval. A referral was considered appropriate if any of the following criteria were present at referral: the eGFR was less than 30 mL/min per 1.73 m²; there was progressive decline in the eGFR (> 20% fall within one year); the protein excretion was greater than 1 g in a 24-hour collection of urine or the urine protein–to–creatinine ratio was greater than 100 mg/mmol or the urine albumin–to–creatinine ratio was greater than 60 mg/mmol; the referring primary care physician indicated that he or she was unable to achieve treatment targets (e.g., blood pressure could not be controlled); or the primary care physician provided a specific reason for referral, such as hyperkalemia. We recalculated the eGFR values using the abbreviated (four-variable) Modification of Diet in Renal Disease formula for both study periods.¹³

Prespecified secondary outcome measures included the proportion of appropriate referrals and the total number of referrals from primary care physicians per two-week interval.

Data for the otucome measures were entered (by M.C.-F.), and the accuracy of the data entry was verified through an audit (by A.A.) of a random sample of 5% of the total sample. The inter-rater agreement was strong (kappa = 0.94, 95% CI 0.92 to 0.97).

Statistical analysis

Continuous data are expressed as means with standard deviations or as medians with interquartile ranges (IQRs). Categorical data are presented as numbers and percentages. Separate time-series analyses were performed for the primary and secondary outcome measures. The Durbin–Watson statistic was used to test for autocorrelation. No significant autocorrelation was detected. However, because time-series data are typically autocorrelated, we undertook a conservative approach and used regression analysis with autoregressive errors. We divided the data biweekly to maximize the number of observations and the amount of information per data point.

To explore whether or not the automatic reporting had any effect on referrals, we fitted three interrupted time-series models of the following form:

$$\begin{array}{l}{Y}_i=\alpha +{\beta }_1{T}_i+{\beta }_2{R}_i+{\beta }_3{T}_i{R}_i+{\upsilon }_i\\ {\upsilon }_i=-\phi {\upsilon }_{i-1}+{\varepsilon }_i\\ {\varepsilon }_i\sim N(0,{\sigma }^2)\end{array}$$

In each model, Y_i represents the outcome (the number of appropriate referrals;the total number of referrals; or the proportion of appropriate referrals). The variable T_i represents the time period (T_i = 1, … 52). The indicator variable R_i is “1” after and “0” before the introduction of automatic reporting. Finally, υ_i is the autoregressive error term. The model postulates two different lines, one before reporting began and one after reporting began.

The parameters β₂ and β₃ represent the changes in intercept and slope, respectively, after the intervention. If the intervention had no effect, we would expect both of these parameters to be zero. To determine whether or not there was an effect, we used a likelihood ratio test to determine whether or not β₂ and β₃ were equal to zero. Furthermore, if the reporting did have an effect, we were able to estimate what the referral patterns would have been had automatic reporting not been introduced, by extrapolating the line defined by Y_i = α + β₁T_i beyond the point when reporting began (Figure 1). This allowed us to describe the effect of the intervention in terms of the total number of additional referrals over the following year that were due to reporting of the eGFR. We also assessed determinants of appropriate referral.

Figure 1:

Example of the calculation of the difference in the number of referrals to nephrologists before and after the intervention (the introduction of automatic reporting of the estimated glomerular filtration rate). The circles indicate the observed number of referrals. The difference in the number of referrals was calculated by subtracting the observed number after the intervention from the predicted number (dashed line) at biweekly intervals.

Results

Of 5093 referrals during the study period, referral packages were retrieved for 4999 (98.2%). We excluded 2327 because the patient was referred by a specialist (n = 1182), was already known to the nephrology service (n = 717) or was not being referred for chronic kidney disease (n = 428) (Figure 2).

Figure 2:

Selection of referrals to the nephrology service in the year before and the year after the introduction of automatic reporting of the estimated glomerular filtration rate.

Characteristics of the 2672 patients referred for chronic kidney disease by primary care physicians are shown in Table 1. In the year after automatic reporting of the eGFR was introduced, the total number of referrals increased significantly among patients 80 years and older (percentage-point change 8.0; p < 0.001) and among women (percentage-point change 12.6; p < 0.001). The eGFR values at the time of referral were significantly lower in the year after than in the year before reporting began (difference in the mean eGFR values −4.5 mL/min per 1.73 m², 95% CI −2.7 to −6.2). The serum creatinine concentration and urine protein excretion did not differ significantly between periods (Table 1).

Table 1:

Characteristics of 2672 patients referred for chronic kidney disease by primary care physicians before and after automatic reporting of the estimated glomerular filtration rate by laboratories was introduced

Characteristic	Period; no. (%) of referrals*		p value
	Before reporting began n = 1207	After reporting began n = 1465
Age, yr
< 50	220 (18.2)	218 (14.9)	0.02
50–64	272 (22.5)	309 (21.1)	0.4
65–79	504 (41.8)	564 (38.5)	0.1
≥ 80	211 (17.5)	374 (25.5)	< 0.001
Median (IQR)	68.9 (55.2–80)	71.2 (58.4–80.2)	< 0.001
Sex, female	503 (41.7)	795 (54.3)	< 0.001
Serum creatinine, μmol/L	n = 1137	n = 1352
> 150	417 (36.7)	492 (36.4)	0.9
≤ 150	720 (63.3)	860 (63.6)	0.9
Median (IQR)	133 (105–166)	135 (109–168)	0.1
Estimated GFR, mL/min per 1.73 m2	n = 1137	n = 1352
< 30	214 (18.8)	322 (23.8)	0.003
30–59	644 (56.6)	796 (58.9)	0.3
≥ 60	279 (24.5)	234 (17.3)	< 0.001
Median (IQR)	44.7 (33.3–59.7)	41.8 (30.7–54.6)	0.01
Protein excretion, g/d	n = 588	n = 697
Median (IQR)	0.17 (0.08–0.45)	0.14 (0.04–0.44)	0.9

Note: GFR = glomerular filtration rate, IQR = interquartile range.

^*Unless stated otherwise.

Time-series analysis showed a relative increase of 80.6% (95% CI 74.8% to 86.9%) in the total number of referrals for chronic kidney disease in the year after automatic reporting began (Table 2, Figure 3). The number of appropriate referrals increased by 43.2% (95% CI 38.5% to 48.2%) after automatic reporting began (Table 2, Figure 4). There was no significant change in the proportion of appropriate referrals between the two study periods (−2.8%, 95% CI −26.4% to 43.4%) (Table 2, Figure 5). The proportion of appropriate referrals fell in the first month after automatic reporting began and then rose gradually until the end of the year, when it was higher than values in the year before reporting began (Figure 5).

Table 2:

Number of total referrals, number of appropriate referrals and proportion of appropriate referrals for chronic kidney disease in the year after automatic reporting of the estimated glomerular filtration rate began

Outcome measure	Observed	Expected* (95% CI)	Difference (95% CI)	Effect size of intervention, % change above expected (95% CI)
Total no. of referrals	1465	811 (784 to 838)	654 (626 to 682)	80.6 (74.8 to 86.9)
Total no. of appropriate referrals	799	558 (539 to 577)	241 (221 to 261)	43.2 (38.5 to 48.2)
Proportion of referrals that were appropriate	0.55	0.56 (0.38 to 0.74)	−0.016 (−0.17 to 0.20)	−2.8 (−26.4 to 43.4)

Note: CI = confidence interval.

^*Expected numbers and proportions were derived from regression analysis with autoregressive errors.

Figure 3:

Total number of referrals for chronic kidney disease received before and after the introduction of automatic reporting of the estimated glomerular filtration rate in March 2006. Open circles represent the total number of referrals at biweekly intervals. The solid line represents the regression line; the screened zone represents the 95% prediction interval.

Figure 4:

Number of appropriate referrals for chronic kidney disease received by the nephrology service from primary care physicians during the year before and the year after the introduction of automatic reporting of the estimated glomerular filtration rate by laboratories in March 2006. Open circles represent the number of referrals observed at biweekly intervals. The solid line represents the regression line; the screened zone represents the 95% prediction interval.

Figure 5:

Proportion of appropriate referrals for chronic kidney disease received before and after the introduction of automatic reporting of the estimated glomerular filtration rate. Open circles represent the proportion of appropriate referrals at biweekly intervals. The solid line represents the regression line; the screened zone represents the 95% prediction interval.

Determinants of appropriate referrals are listed in Table 3. The number of referrals received for patients whose eGFR was less than 30 mL/min per 1.73 m² was higher after than before reporting began (percentage-point change 5.0, 95% CI 1.8 to 8.2). There were fewer specific issues raised by family physicians after reporting began (percentage-point change −3.7, 95% CI −7.3 to −0.2). There was no significant difference between the two study periods in the number of appropriate referrals based on the presence of proteinuria or progressive chronic kidney disease (Table 3).

Table 3:

Criteria of appropriateness observed in referral packages before and after automatic reporting of the estimated glomerular filtration rate began^*

Criterion	Period; no. (%) of referrals		Percentage-point change (95% CI)
	Before reporting began	After reporting began
Estimated GFR < 30 mL/min per 1.73 m2	n = 1137‡ 214 (18.8)	n = 1352‡ 322 (23.8)	5.0 (1.8 to 8.2)
Proteinuria†	n = 588§ 85 (14.5)	n = 697§ 113 (16.2)	1.7 (−2.2 to 5.7)
Progressive chronic kidney disease	n = 661¶ 150 (22.7)	n = 796¶ 185 (23.2)	0.5 (−3.8 to 4.9)
Family physician unable to achieve treatment target	n = 1207** 49 (4.1)	n = 1465** 77 (5.3)	1.2 (−0.4 to 2.8)
Specific question by family physician	n = 1207** 397 (32.9)	n = 1465** 427 (29.2)	−3.7 (−7.3 to −0.2)

Note: CI = confidence interval, GFR = glomerular filtration rate.

^*A referral was judged to be inappropriate if it met none of the criteria.

^†Protein excretion > 1 g/d.

^‡Number of referrals in which estimated GFR could be calculated.

^§Number of referrals in which quantification of protein excretion was performed.

^¶Number of referrals in which determination of progressive chronic kidney disease was possible.

^**Total number of referrals.

Interpretation

In our study, the introduction of automatic reporting of the eGFR by laboratories, accompanied by ad hoc educational activities for primary care physicians, was associated with an increase in the number of appropriate referrals for chronic kidney disease to the nephrology service. However, the proportion of appropriate referrals did not change significantly because the number of inappropriate referrals also increased. The total number of referrals increased, especially among elderly people and women.

With automatic eGFR reporting, there was a concern that inappropriate referrals to nephrology would also increase after the automatic reporting began.^29,30 Although we found no change in the proportion of appropriate to inappropriate referrals overall, the time-series analysis showed that the proportion of appropriate referrals fell in the first month after automatic reporting began and then rose gradually until the end of the year. Whether this trend was sustained beyond the end of the study is uncertain. The initial fall in the proportion of appropriate referrals may have been due to primary care physicians being unfamiliar with the new test and the time needed to adjust to the change in practice. Our data showed that the introduction of automatic reporting of the eGFR led to increases in both appropriate and inappropriate referrals.

Noble and colleagues compared the appropriateness of referrals in the 3 months before and the 12 months after reporting of the eGFR was introduced in Australia.²² Similar to our study, they reported an absolute increase in the number of appropriate referrals after automatic reporting began. However, unlike our finding of no change in the proportion of appropriate referrals, they observed a significant decrease in the proportion, from 74.3% before to 65.2% after reporting began. Their results may have differed from ours because of the sample size, differences in the study population, geographic location and classification criteria for appropriateness.

In both our study and the study by Noble and colleagues,²² a substantial number of referrals were considered inappropriate according to the referral guidelines developed by the respective societies.^16,18 Increases in the number of inappropriate referrals will increase wait times for nephrology consultations, delaying care for patients who need urgent attention, and will increase costs to the health care system. Factors that may be responsible for inappropriate referrals may include lack of knowledge about appropriate referrals; lack of resources to manage patients with mild chronic kidney disease in primary care; patients’ demands to see a specialist; and different perspectives about what constitutes an appropriate referral.^31,32 Further research may help to delineate the reasons for inappropriate referral.³³ Interventions to decrease inappropriate referrals would help to limit the total number of referrals, thus limiting unnecessary consumption of expensive resources.

The total number of referrals increased by 80.6% in our study. Most of the increase was attributed to referrals of elderly and female patients, which is consistent with the findings of Jain and coworkers²¹ and Noble and colleagues.²² Several studies have shown the poor performance of the serum creatinine concentration alone in screening for chronic kidney disease, especially among elderly people and women.^2,3,14 With the elderly population increasing, this is an important finding.

Strengths and limitations

The strengths of our study include the large sample and the robust methodology used. The biggest threat to time-series analysis is an event that independently has an effect similar to that of the intervention at the time of the intervention.²⁷ It is highly unlikely that the number of patients in our study would have abruptly increased at the time automatic reporting was introduced. Moreover, Jain and coworkers²¹ observed no change in the number of referrals to dermatology services in Ontario during the same study period as ours. A second common threat to time-series analysis is seasonal trend. Because we collected data for one year before and one year after automatic reporting began, we consistently captured potential seasonal variations in referrals in both periods. A third threat to time-series analysis is a change in the way records are kept.²⁷ During the period studied, there was no known change in the administrative process of tracking nephrology referrals at The Ottawa Hospital.

Five limitations to our study should be noted. First, data were collected retrospectively; nevertheless, we were able to retrieve 98% of referral packages. Second, our study was limited to a single nephrology service; however, almost all patients in our region are referred to this service. Third, the educational interventions were not studied independently from the automatic reporting. Although this would have provided information about the effect of each intervention individually, we do not believe that a change in laboratory reporting should be undertaken without appropriate education of the target audience. Fourth, we did not account for race when we calculated the eGFR using the Modification of Diet in Renal Disease formula. This should not have affected our findings substantially, because only 3.9% of the population of Ontario is black³⁴ and the same approach was used for both study periods. Finally, we did not have follow-up data on the outcomes of the patients.

Conclusion

The introduction of automatic reporting of the eGFR by laboratories, along with ad hoc educational activities for primary care physicians, was associated with an increase in the total number of referrals for chronic kidney disease, especially among women and elderly people. The number of appropriate referrals also increased; however, the proportion of appropriate referrals did not change significantly because of the increasing number of inappropriate referrals. Future research should be directed to understanding the reasons for inappropriate referral and to develop novel interventions for improving the referral process.