Abstract
The authors estimated the supply of organ donors in the U.S. and also according to organ procurement organizations (OPOs). They estimated the number of donors in the U.S. to be 16,796. Estimates of the number of potential donors for each OPO were used to calculate the level of donor efficiency (actual donors as a percent of potential donors). Overall, donor efficiency for OPOs was 35 percent; the majority was between 30- and 40-percent efficient. Although there is room to improve donor efficiency in the U.S., even a substantial improvement will not meet the Nation's demand for organs.
Introduction
Although over 70,000 Americans awaited a solid organ transplant in 2000, only 22,953 transplants were performed (United Network for Organ Sharing, 2003). It is generally believed that this disparity is due to the inability of the organ procurement system to tap more efficiently the potential supply of organs from deceased individuals who are medically suitable for donation. Prior estimates of the donor supply range between 6,900 and 15,000 potential donors. However, these estimates are based on data from the late 1980s or early 1990s (Garrison, Bentley, and Raque, 1991; Evans, Orians, and Ascher, 1992; Nathan et al., 1991; Ojo et al., 1999), from a limited number of service areas (Nathan et al., 1991; Garrison, Bentley, and Raque, 1991), from extrapolations generated from data on specific organs (Nathan et al., 1991; Ojo et al., 1999), or from death records (Evans, Orians, and Ascher, 1992), which lack sensitivity with respect to identifying medically suitable organ donors. A more current and valid estimate of the potential number of organ donors is needed. Moreover, the lack of a reasonable estimate of donor potential also makes it difficult to evaluate the performance of organizations that are responsible for procuring organs in the United States.
OPOs are not-for-profit entities, assigned to specific geographical service areas, that coordinate the acquisition and distribution of organs (U.S. General Accounting Office, 1998). CMS sets performance standards for OPOs according to five criteria: organ donors, kidneys recovered, kidneys transplanted, extrarenal organs recovered, and extrarenal organs transplanted. For each criterion, performance is expressed as a function of the population within the OPO service area (for example, the number of organ donors per million population (DPMP)1.
The approach used to assess OPO performance has been criticized because it assessed performance based on the total population in a service area rather than the potential number of donors (Shafer, Kappel, and Heinrichs, 1997; U.S. General Accounting Office, 1998; Christiansen et al., 1998; Luskin et al., 1999; Ojo et al., 1999). Although a population-based measure is readily available and inexpensive to produce (U.S. General Accounting Office, 1997), it is problematic because donor pools can differ across service areas due to variation in characteristics that define potential organ donors (Ojo et al., 1999). Both OPOs and CMS recognize the limitations of the approach used to assess performance and both parties are interested in identifying viable alternatives (U.S. General Accounting Office, 1997). Alternative performance measures, such as measuring procurement and transplantation as a function of the number of deaths, the number of potential donors based on medical record review, and the number of potential donors based on statistical estimates for each ser vice area have been proposed (U.S. General Accounting Office, 1998).
We estimated the potential supply of organ donors using a previously validated prediction model that estimated the number of potential donors for individual hospitals based on publicly available data (Christiansen, Gortmaker, and Williams, 1998). We aggregated the hospital estimates and determined the overall potential for the U.S. and for each OPO service area. We evaluated the performance of OPOs by comparing their estimates of donor efficiency (actual donors per estimated potential donors) and we compared donor efficiency with the measure, DPMP, previously used by CMS to certify OPOs. Finally, we propose a strategy for certification of OPOs based on donor efficiency.
Methods
Study Population
We used 1998 American Hospital Association (AHA) data to select hospitals for analysis based on previously described criteria (Christiansen, Gortmaker, and Williams, 1998), as shown in Table 1. Eligible hospitals included non-Federal, acute-care facilities with an intensive care unit and at least 50 beds (N = 3,120).
Table 1. Hospital Exclusion Criteria: 1998.
Reason for Exclusion | Number of Hospitals |
---|---|
Total Hospitals in American Hospital Association Data Set | 6,247 |
Total Number of Eligible Hospitals | 3,120 |
Type of Hospital1 | 1,089 |
Federal Government Hospital | 280 |
Located in Territories Not Assigned to an Organ Procurement Organization Service Area2 | 4 |
No Intensive Care Unit | 2,094 |
Number of Staffed Beds < 50 | 1,899 |
Excluded the following types: hospital unit of an institution (prison hospital, college infirmary, etc.); hospital unit within an institution for the mentally retarded; psychiatric; tuberculosis and other respiratory diseases; eye, ear, nose, and throat; rehabilitation; orthopedic; other specialty treatment; children's hospital unit of an institution; children's psychiatric; children's tuberculosis and other respiratory disease; children's eye, ear, nose, and throat; children's rehabilitation; children's orthopedic; children's other specialty; institution for mental retardation; and alcoholism and other chemical dependency.
Excluded Marshall Islands, Guam, and American Samoa.
SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.
Estimating Potential Number of Donors
For three OPO service areas, Christiansen and colleagues (1998) used publicly available data on the characteristics of hospitals (bed size, case-mix index, status as a trauma center, and medical school affiliation) and hierarchical Poisson regression (Christiansen and Morris 1997a) to develop a model to estimate the potential number of donors for individual hospitals located in the service areas. They then aggregated the hospital estimates to produce an estimate of the number of potential donors for each OPO. The criterion for their model (number of potential organ donors per hospital) was obtained from a review of hospital records. A potential donor was defined as a medically suitable patient who had conditions consistent with brain death, as defined by the President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research (1981). Patients were excluded as potential donors if they were age 70 or over or exhibited one or more International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (Centers for Disease Control and Prevention, 2003) codes that are contraindications for organ donation. A list of the ICD-9-CM codes appears in Table 2. These exclusion criteria reflected practice within each of the three service areas studied. Evaluation of the model suggested that it would predict well when used in service areas other than those used in the study (Christiansen, Gortmaker, and Williams, 1998).
Table 2. ICD-9-CM Exclusion Criteria.
Code | Condition |
---|---|
010.00-018.9 | Tuberculosis |
042.0-044.9 | Human Immunodeficiency Virus (HIV) Infection with Specified Conditions |
46.1 | Jakob-Creutzfeldt Disease |
54.5 | Herpetic Septicemia |
070.2-070.3 | Hepatitis B Surface Antigen |
140.0-190.9 | Malignant Neoplasm |
192-208.9 | Malignant Neoplasm of Other and Unspecified Parts of the Nervous System |
284.0-284.9 | Aplastic Anemia |
286.0-286.4 | Hemophilia |
288 | Agranulocytosis |
710.0-710.9 | Diffuse Diseases of Connective Tissue |
765.00-765.03 | Extreme Immaturity |
765.1 | Preterm Infant of Birth Weight 1,000-2,499 Grams |
780.6 | Pyrexia of Unknown Origin |
795.8 | Positive Serologic or Viral Culture Findings for HIV |
NOTE: ICD-9-CM is International Classification of Diseases, Ninth Revision, Classified Modification.
SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.
We estimated the number of potential donors for an OPO service area comprised of i = 1, …, k hospitals using:
Trauma center missing refers to an indicator equal to 1 if information on the trauma center status of a hospital was missing or equal to 0, otherwise. The total number of potential donors per OPO was calculated as Θ=ΣΘi and the estimate of the standard error associated with the prediction was defined as ω=√(Σ(Θi2/2.58)+Θi) (Christiansen et al., 1998).
We obtained the number of staffed hospital beds from the AHA (1998) data set or, if missing, from the Hospital Blue Book (1998). We obtained hospital case-mix values from a publicly available file (Centers for Medicare & Medicaid Services, 2003). The case-mix index is a measure of the costliness of cases treated by a hospital relative to that of the national average for all inpatients covered by Medicare, using diagnostic-related group weights as a measure of the relative costliness of cases (Christiansen et al., 1998). We were unable to link 98 hospitals to data in the case-mix file. For these hospitals and for six whose case-mix values were typographical errors, we used the mean case-mix value (1.37). The variables, trauma center (whether the hospital was a certified trauma center) and medical school affiliation, were obtained from the AHA data set; both were represented by dichotomous variables coded as 0 (no) or 1 (yes). If trauma center status was missing from the AHA data set, we obtained it from the Hospital Blue Book (1998) for all, but one hospital.
Statistical Analysis
Using the equation previously mentioned, we calculated the potential number of donors per eligible hospital for 1998. Based on the hospital's geographic location, we calculated the potential number of donors and the 95 percent confidence interval (CI) for each OPO service area. We obtained the actual number of donors and the population for each OPO service area for 1998 (United Network for Organ Sharing, 2003). For each OPO, we calculated a measure of donor efficiency (number of actual donors ÷ number of potential donors * 100) and DPMP, the standard of performance used by CMS. We calculated the 95-percent CI for the efficiency measure, using the Delta method (Casella and Berger, 1990). This method can result in a CI that is outside the bounds of the parameter; in this case, the ratio is bounded by 0 and 1. We truncated a CI at 1 for one OPO. We assessed the degree of association between donor efficiency and DPMP by calculating the Spearman rank-order correlation (Marascuilo and McSweeney, 1977).
Finally, we describe how donor efficiency estimates might be used to evaluate performance of OPOs for the purpose of certification. This approach takes into account that donor efficiency is based on an estimate of donor potential, and therefore there is a level of uncertainty associated with the efficiency measure. We used probability models (Christiansen and Morris, 1997b) for the unknown parameter representing the number of potential donors to calculate the likelihood that an OPO meets a prespecified performance criterion. Then we demonstrate how this information might be used to evaluate performance.
Results
Potential Donors and Donor Efficiency
Table 3 lists the estimated number of potential donors, the number of actual donors, donor efficiency, and DPMP according to OPO. For 1998, we estimated that there were 16,796 (95 percent CI = 16,105, 17,481) potential donors in the U.S. Across OPOs, the number of potential donors ranged from a high of 937 (95 percent CI = 754, 1121) to a low of 53 (95 percent CI = 23, 83).
Table 3. Number of Potential Donors (95-Percent Confidence Interval), Actual Number of Donors, and Donor Efficiency (95-Percent Confidence Interval)1.
OPO | Potential Donors | 95-Percent CI Lower | CI Upper | Actual Donors | Donor Efficiency | 95-Percent CI Lower | CI Upper | DPMP | Prob (Donor Efficiency < 25 Percent) |
---|---|---|---|---|---|---|---|---|---|
Total | 16,796 | 16,105 | 17,481 | 5,793 | 34.5 | 24.2 | 44.8 | 21.3 | — |
1 | 53 | 23 | 83 | 43 | 81.6 | 34.9 | 100.0 | 21.7 | 0.00 |
2 | 55 | 31 | 79 | 36 | 65.5 | 36.9 | 94.0 | 9.1 | 0.00 |
3 | 107 | 62 | 153 | 70 | 65.1 | 37.5 | 92.8 | 24.4 | 0.00 |
4 | 234 | 174 | 293 | 152 | 65.1 | 48.5 | 81.6 | 23.9 | 0.00 |
5 | 60 | 21 | 98 | 31 | 52.1 | 18.2 | 85.9 | 21.4 | 0.00 |
6 | 216 | 117 | 315 | 112 | 51.9 | 28.1 | 75.6 | 40.7 | 0.00 |
7 | 111 | 67 | 154 | 56 | 50.7 | 30.9 | 70.4 | 29.4 | 0.00 |
8 | 389 | 287 | 491 | 190 | 48.9 | 36.0 | 61.7 | 25.9 | 0.00 |
9 | 113 | 55 | 171 | 52 | 46.0 | 22.5 | 69.6 | 25.6 | 0.00 |
10 | 201 | 140 | 261 | 92 | 45.9 | 32.0 | 59.7 | 21.4 | 0.00 |
11 | 389 | 293 | 484 | 178 | 45.8 | 34.6 | 57.1 | 24.3 | 0.00 |
12 | 69 | 38 | 99 | 31 | 45.1 | 25.1 | 65.2 | 17.9 | 0.00 |
13 | 664 | 530 | 798 | 298 | 44.9 | 35.8 | 53.9 | 28.9 | 0.00 |
14 | 388 | 290 | 487 | 174 | 44.8 | 33.4 | 56.2 | 28.1 | 0.00 |
15 | 175 | 84 | 265 | 78 | 44.6 | 21.5 | 67.7 | 30.4 | 0.00 |
16 | 443 | 357 | 528 | 197 | 44.5 | 35.9 | 53.1 | 17.7 | 0.00 |
17 | 282 | 198 | 366 | 123 | 43.7 | 30.7 | 56.7 | 24.1 | 0.00 |
18 | 299 | 215 | 383 | 124 | 41.5 | 29.9 | 53.1 | 29.1 | 0.00 |
19 | 111 | 61 | 161 | 45 | 40.6 | 22.4 | 58.8 | 22.5 | 0.00 |
20 | 192 | 116 | 269 | 77 | 40.0 | 24.0 | 56.0 | 26.5 | 0.00 |
21 | 197 | 129 | 264 | 78 | 39.7 | 26.1 | 53.3 | 25.3 | 0.00 |
22 | 224 | 152 | 296 | 86 | 38.4 | 26.1 | 50.8 | 26.7 | 0.00 |
23 | 197 | 135 | 259 | 74 | 37.6 | 25.7 | 49.4 | 17.8 | 0.00 |
24 | 331 | 241 | 422 | 124 | 37.4 | 27.2 | 47.7 | 24.7 | 0.00 |
25 | 140 | 76 | 203 | 52 | 37.2 | 20.3 | 54.1 | 19.9 | 0.02 |
26 | 442 | 346 | 538 | 162 | 36.6 | 28.7 | 44.6 | 21.5 | 0.00 |
27 | 251 | 173 | 329 | 92 | 36.6 | 25.3 | 48.0 | 20.0 | 0.00 |
28 | 247 | 141 | 352 | 87 | 35.3 | 20.2 | 50.3 | 23.6 | 0.03 |
29 | 316 | 235 | 397 | 109 | 34.5 | 25.6 | 43.3 | 15.8 | 0.00 |
30 | 212 | 140 | 284 | 72 | 33.9 | 22.5 | 45.4 | 21.7 | 0.02 |
31 | 186 | 113 | 260 | 63 | 33.9 | 20.5 | 47.2 | 30.2 | 0.04 |
32 | 354 | 257 | 451 | 117 | 33.0 | 24.0 | 42.1 | 26.0 | 0.01 |
33 | 303 | 215 | 391 | 100 | 33.0 | 23.4 | 42.7 | 25.9 | 0.01 |
34 | 77 | 39 | 115 | 25 | 32.5 | 16.4 | 48.7 | 21.8 | 0.12 |
35 | 250 | 174 | 326 | 80 | 32.0 | 22.3 | 41.7 | 18.1 | 0.03 |
36 | 634 | 497 | 772 | 200 | 31.5 | 24.7 | 38.4 | 21.0 | 0.01 |
37 | 326 | 214 | 439 | 101 | 31.0 | 20.3 | 41.7 | 20.2 | 0.08 |
38 | 475 | 345 | 604 | 146 | 30.7 | 22.4 | 39.1 | 26.1 | 0.05 |
39 | 350 | 257 | 443 | 106 | 30.3 | 22.2 | 38.3 | 16.7 | 0.06 |
40 | 212 | 128 | 296 | 64 | 30.1 | 18.2 | 42.1 | 22.4 | 0.15 |
41 | 240 | 154 | 326 | 71 | 29.6 | 18.9 | 40.2 | 20.5 | 0.16 |
42 | 163 | 92 | 233 | 47 | 28.9 | 16.3 | 41.4 | 19.1 | 0.24 |
43 | 236 | 139 | 333 | 68 | 28.8 | 17.0 | 40.6 | 22.1 | 0.23 |
44 | 106 | 45 | 167 | 30 | 28.2 | 12.0 | 44.5 | 15.6 | 0.32 |
45 | 303 | 234 | 372 | 85 | 28.0 | 21.6 | 34.4 | 19.5 | 0.14 |
46 | 201 | 125 | 276 | 56 | 27.9 | 17.4 | 38.4 | 22.4 | 0.27 |
47 | 98 | 45 | 150 | 27 | 27.7 | 12.8 | 42.6 | 18.2 | 0.35 |
48 | 740 | 583 | 896 | 199 | 26.9 | 21.2 | 32.6 | 18.0 | 0.24 |
49 | 154 | 86 | 222 | 41 | 26.7 | 14.8 | 38.5 | 18.1 | 0.39 |
50 | 937 | 754 | 1121 | 247 | 26.4 | 21.2 | 31.5 | 20.7 | 0.29 |
51 | 868 | 718 | 1017 | 228 | 26.3 | 21.8 | 30.8 | 19.7 | 0.28 |
52 | 261 | 165 | 357 | 67 | 25.7 | 16.2 | 35.1 | 14.6 | 0.44 |
53 | 196 | 121 | 271 | 50 | 25.6 | 15.8 | 35.4 | 19.0 | 0.46 |
54 | 514 | 408 | 620 | 130 | 25.3 | 20.1 | 30.5 | 12.7 | 0.46 |
55 | 343 | 222 | 465 | 86 | 25.0 | 16.2 | 33.9 | 19.8 | 0.49 |
56 | 96 | 36 | 156 | 24 | 25.0 | 9.4 | 40.6 | 15.8 | 0.50 |
57 | 142 | 69 | 215 | 34 | 24.0 | 11.7 | 36.4 | 14.7 | 0.56 |
58 | 355 | 255 | 455 | 83 | 23.4 | 16.8 | 30.0 | 18.0 | 0.68 |
59 | 142 | 73 | 210 | 33 | 23.3 | 12.0 | 34.6 | 19.1 | 0.61 |
60 | 187 | 95 | 279 | 42 | 22.5 | 11.4 | 33.5 | 19.5 | 0.66 |
61 | 77 | 36 | 118 | 16 | 20.8 | 9.7 | 31.9 | 13.5 | 0.73 |
62 | 162 | 103 | 222 | 32 | 19.7 | 12.5 | 26.9 | 15.1 | 0.87 |
Donor efficiency is the number of actual donors divided by the number of potential donors times 100.
NOTES: CI is confidence interval. OPO is organ procurement organization. DPMP is donors per million population.
SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.
Donor efficiency ranged from a high of 81.6 (95 percent CI = 34.9 percent, 100.0 percent) to a low of 19.7 (95 percent CI = 12.5 percent, 26.9 percent). For more than one-half of the OPOs (53 percent), donor efficiency ranged between 30 and 40 percent; for four OPOs, efficiency was nearly two-thirds or greater (Table 3). For three of these four OPOs, the donor efficiency estimates have wide CIs due to the small number of eligible hospitals (15, 17, and 23 hospitals) within each service area, suggesting that there is a high level of uncertainty associated with the estimate of donor efficiency for these OPOs. Six OPOs were less than 25 percent efficient (Table 3).
Figure 1 displays donor efficiency plotted against DPMP. The Spearman rank-order correlation between the measures was moderate (r=0.57).
Evaluation of OPO Performance
Because the method used to derive donor efficiency is based on an estimate of donor potential, there is a level of uncertainty associated with the efficiency measure. However, one can calculate the probability that donor efficiency is greater than or equal to a specific criterion (say, 25 percent or one out of four potential donors actually donates) by estimating distributions for the unknown parameter, the number of potential donors. We assume that the true number of potential donors at an OPO follows an approximate Gaussian distribution (Colton, 1974) with a mean equal to our estimated number of potential donors and an estimated standard deviation previously noted. Based on these distributions, one can calculate the probability that the true donor efficiency rate is less than or more than a specific criterion. For demonstration purposes, we report the probability that the true rate is less than 25 percent. This probability, in conjunction with the measure of donor efficiency, might be used for purposes of evaluating OPO performance.
For example, if the probability is more than 50 percent (we are more certain than uncertain) that the donor efficiency for an OPO is 25 percent or less, then the OPO might be considered a poor performer. If the probability is less than 50 percent (we are more uncertain than certain) that the OPO is a poor performer, then CMS might reserve judgment with respect to imposing sanctions on the OPO due to a lack of certainty regarding its performance. In this case, CMS might decide to review hospital records in order to determine, with more certainty, the actual performance of the OPO. The selection of a probability criterion more than 50 percent is arbitrary. A higher value (for example, 75 percent) might be used if one wished to be more conservative with respect to imposing sanctions on OPOs.
The probability that an OPO's donor efficiency rate is less than 25 percent appears in Table 3 (donor efficiency <25 percent). Again, this is an artificial standard chosen here only to demonstrate how these estimates can be used to assess OPO performance. For example, the probability is 0.61 that OPO 59 had a true donor efficiency rate of less than 25 percent (Table 3). Based on a criterion of 25 percent, we would be more than 50-percent certain that 6 of the 62 OPOs (OPOs 57 to 62) were poor performers. In the case of OPO 56, where the probability (donor efficiency < 25 percent) =0.50), we might reserve judgment on performance, pending a review of hospital records. If we wished to be more conservative regarding the identification of poor performers and used a 75-percent level of certainty, then one of the 62 OPOs (OPO 62) would be labeled a poor performer. Table 4 provides a summary of the number of OPOs considered poor performers, according to various levels of donor efficiency and various levels of certainty.
Table 4. Number of Organ Procurement Organizations Considered Poor Performers According to Various Levels of Donor Efficiency and Various Levels of Certainty.
True Donor Efficiency Rate Less Than | Probability That Donor Efficiency Rate Is < True Rate | ||
---|---|---|---|
| |||
0.50 | 0.75 | 0.95 | |
0.25 | 6 | 1 | 0 |
0.50 | 55 | 50 | 34 |
0.75 | 61 | 59 | 54 |
SOURCES: Guadagnoli, E., Harvard Medical School, Christiansen, C.L., Boston University and the Edith Nourse Rogers Memorial Veterans Hospital, and Beasley, C.L., Institute for Healthcare Improvement, 2003.
Discussion
Our estimate of the potential donor pool in the U.S., 16,796 (95 percent CI = 16,105; 17,481), is in line with some past estimates derived from a variety of estimation methods. Considering that 5,793 donors provided organs for transplantation in 1998 (35 percent overall donor efficiency), room for improvement exists. If all OPOs that performed with less than 50 percent efficiency increased their performance to 50 percent and all remaining OPOs maintained their level of efficiency, we might expect 8,479 donors. Moreover, if all OPOs were able to attain a 75-percent level of efficiency, the number of donors would increase to 12,598. However, considering that 41,000 registrations were added to the waiting list in 2000 (Organ Procurement and Transplant Network, 2003), and assuming that each donor yields, on average, three organs for transplantation (U.S. General Accounting Office, 1998), the demand for organs would still exceed the potential supply. The addition of living donors to this pool would bring us closer to meeting demand; nevertheless, this calculation ignores the backlog of patients already on the waiting list.
Although the supply of cadaveric donors may never satisfy the demand for organs for transplantation, cadaveric donors will remain an important, if not the most important, supply of organs for the foreseeable future. Therefore, efforts to evaluate and improve the performance of OPOs are essential. The availability of an estimate of donor potential should allow policymakers to contemplate what a reasonable standard of performance is for OPOs. Past work suggests that 69 percent of the public would be likely to donate their own organs (Gallup Organization, 1993); refusal to give consent for donation ranges between 30 and 50 percent (Gortmaker et al., 1996; Ojo et al., 1999; Capron, 2001). Past work also suggests that consent rates of 74 percent are possible when hospital and OPO staff engage in specific processes when requesting organ donation (Cutler et al., 1993; Klieger et al., 1994; Beasley, Capposella, and Brigham, 1997; Gortmaker et al., 1998; Siminoff et al., 2001). To date, significant and lasting increases in rates of consent have not been reported, perhaps related to the difficulty of orchestrating the donation process consistently to include the recommended process steps. Therefore, policymakers should not assume that 100 percent efficiency is possible. Considering the proportion of the public that is willing to donate and the sensitive context in which the donation request takes place, perhaps 50 percent efficiency is the upper limit for the expected performance of OPOs. Only 7 of 62 OPOs met or exceeded this level in 1998.
The availability of a measure of donor efficiency also allows policymakers to identify OPOs that are good performers. Investigation into the practices of these OPOs might identify strategies and processes that could be transferred to those OPOs whose efficiency is low. A recent survey has indicated that there is wide divergence in consent practices among OPOs (Wendler and Dickert, 2001).
The measure of donor efficiency relies on statistical estimates, and therefore, there is some level of uncertainty associated with the donor efficiency rate for each OPO. Given this uncertainty, commonly used statistical approaches to assess performance, such as ranking OPOs based on donor efficiency rates or comparing the donor-efficiency rate for an OPO with the mean rate for all OPOs, are not appropriate. We describe a method that allows policymakers to evaluate the probability that an OPO's performance is actually below a particular standard. The utility of this approach relies on the ability of policymakers and other interested parties to agree on an acceptable standard and on a probability criterion to evaluate the standard. Similar methods for analyzing performance have been used by the Department of Veterans Affairs to report hospital performance with respect to a number of outcomes (Burgess et al., 2000; Burgess, Lourdes, and West, 2000. Nevertheless, the donor efficiency rate could, at a minimum, be a useful tool for internal quality improvement activities within an OPO.
The association between the donor efficiency measure and the current standard used by CMS, DPMP, was moderate. However, the use of the donor efficiency measure described in this article, in combination with an agreed-upon performance criterion and a probability statement that the criterion is met, improves the interpretation of OPO performance and provides a fairer assessment of performance than the approach currently in use. The latter assumes that donor pools do not vary with respect to the characteristics that define donors.
This study has several potential limitations. First, the model used to predict the potential number of donors was developed using data from three OPOs. Validation at the national level using more current data should be performed. The need to review hospital records for validation would make this an expensive undertaking; however, CMS has sponsored the abstraction of data from hospitals throughout the country (Marciniak et al., 1998). Second, although CMS used five criteria to evaluate OPO performance, we examined only one, primarily because the variables used to predict the other criteria are likely to differ from those we used to predict the number of potential donors. For example, rates of recovery for specific organs might vary due to the skill of OPO staff to clinically manage potential donors or to the characteristics of the donor population, such as the proportion of patients with hypertension.
Although the actual number of donors in the U.S. represents about one-third of the potential number of donors, the demand for organs is not likely to be met from cadaveric donors. Nevertheless, this should not stop efforts to increase the efficiency of OPOs. The approach we describe can be used to measure the performance of OPOs and to identify good and bad performance outliers. In addition, we need to pursue methods to increase the number of organs for transplantation. Some of these methods might relate to OPO practices and cadaveric donors (e.g., expanding the scope of legal powers of attorney for health care) (Wendler and Dickert, 2001), to legislative interventions (e.g., allowing presumed consent and mandated consent) (Murray and Youngner, 1994), or to clinical innovations such as expanding the criteria and strategies for transplantation, for example, using older donors, splitting a liver for transplantation between two adults (Gridelli and Remuzzi, 2000), increasing the use of living donors (Matas et al., 2000; Levinsky, 2000; Abecassis et al., 2000), and exploring xenotransplantation and organ engineering (Chapman and Bloom, 2001). The challenge to meeting the demand for organs through all of these methods is related not only to the process of implementation, but also, and more important, to resolving the ethical issues that these approaches provoke.
Acknowledgments
The authors want to thank Yang Xu for assistance with statistical programming.
Footnotes
Edward Guadagnoli is with Harvard Medical School. Cindy L. Christiansen is with Boston University and the Edith Nourse Rogers Memorial Veterans Hospital. Carol L. Beasley is with the Institute for Healthcare Improvement. The research in this article was supported in part by the Centers for Medicare & Medicaid Services (CMS) under HCFA Contract Number 99-0418. The views expressed in this article are those of the authors and do not necessarily reflect the views of Harvard Medical School, Boston University, Edith Nourse Rogers Memorial Veterans Hospital, Institute for Healthcare Improvement, or CMS.
CMS placed a moratorium on the OPO certification standards at the end of 2000. A new standard that satisfies both the transplant community and CMS has yet to be established.
Reprint Requests: Edward Guadagnoli, Ph.D., Harvard Medical School, Department of Health Care Policy, 180 Longwood Avenue, Boston, MA 02115. E-mail: guadagnoli@hcp.med.harvard.edu
References
- Abecassis M, Adams M, Adams P, et al. Consensus Statement on the Live Organ Donor. Journal of the American Medical Association. 2000;284(22):2919–2926. doi: 10.1001/jama.284.22.2919. [DOI] [PubMed] [Google Scholar]
- American Hospital Association. AHA Guide to the Health Care Field, 1998 Edition. Chicago, IL.: 1998. [Google Scholar]
- Beasley CL, Capossela CL, Brigham LE, et al. The Impact of a Comprehensive, Hospital-Focused Intervention to Increase Organ Donation. Journal of Transplantation Coordination. 1997;7(1):6–13. doi: 10.7182/prtr.1.7.1.96461077u64l1m5x. [DOI] [PubMed] [Google Scholar]
- Burgess JF, Jr, Christiansen CL, Michalak SE, Morris CN. Medical Profiling: Improving Standards and Risk Adjustments Using Hierarchical Models. Journal of Health Economics. 2000;19(3):291–309. doi: 10.1016/s0167-6296(99)00034-x. [DOI] [PubMed] [Google Scholar]
- Burgess JF, Jr, Lourdes V, West M. Profiling Substance Abuse Provider Trends in Health Care Delivery Systems. Health Services & Outcomes Research Methodology. 2000;1(3-4):253–276. [Google Scholar]
- Capron AM. Reexamining Organ Transplantation. Journal of the American Medical Association. 2001;285(3):334–336. doi: 10.1001/jama.285.3.334. [DOI] [PubMed] [Google Scholar]
- Casella G, Berger RL. Statistical Inference. Wadsworth & Brooks/Cole; Pacific Grove, CA.: 1990. [Google Scholar]
- Centers for Disease Control and Prevention. International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) Internet address: http://www.cdc.gov/nchs/about/otheract/icd9/abticd9.htm (Accessed 2003.)
- Centers for Medicare & Medicaid Services. Internet address: http://www.cms.gov/providers/hipps/ippswage.asp (Accessed 2003.)
- Chapman LE, Bloom ET. Clinical Xenotransplantation. Journal of the American Medical Association. 2001;285(18):2304–2306. doi: 10.1001/jama.285.18.2304. [DOI] [PubMed] [Google Scholar]
- Christiansen CL, Gortmaker SL, Williams JM, et al. A Method for Estimating Solid Organ Donor Potential by Organ Procurement Region. American Journal of Public Health. 1998;88(11):1645–1650. doi: 10.2105/ajph.88.11.1645. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Christiansen CL, Morris CN. Hierarchical Poisson Regression Modeling. Journal of the American Statistical Association. 1997a;92(438):618–632. [Google Scholar]
- Christiansen CL, Morris CN. Improving the Statistical Approach to Health Care Provider Profiling. Annals of Internal Medicine. 1997b;127(8 Pt 2):764–768. doi: 10.7326/0003-4819-127-8_part_2-199710151-00065. [DOI] [PubMed] [Google Scholar]
- Colton T. Statistics in Medicine. Little, Brown and Company; Boston, MA.: 1974. [Google Scholar]
- Cutler JA, David SD, Kress CJ, et al. Increasing the Availability of Cadaveric Organs for Transplantation: Maximizing the Consent Rate. Transplantation. 1993;56(1):225–228. [PubMed] [Google Scholar]
- Evans RW, Orians CE, Ascher NL. The Potential Supply of Organ Donors: An Assessment of the Efficacy of Organ Procurement Efforts in the United States. Journal of the American Medical Association. 1992;267(2):239–246. [PubMed] [Google Scholar]
- Gallup Organization. The American Public's Attitudes Toward Organ Donation and Transplantation. The Gallup Organization, Inc.; Princeton, NJ.: 1993. [Google Scholar]
- Garrison RN, Bentley FR, Raque GH, et al. There Is an Answer to the Shortage of Organ Donors. Surgery, Gynecology & Obstetrics. 1991;173(5):391–396. [PubMed] [Google Scholar]
- Gortmaker SL, Beasley CL, Brigham LE, et al. Organ Donor Potential and Performance: Size and Nature of the Organ Donor Shortfall. Critical Care Medicine. 1996;24(3):432–439. doi: 10.1097/00003246-199603000-00012. [DOI] [PubMed] [Google Scholar]
- Gortmaker SL, Beasley CL, Sheehy E, et al. Improving the Request Process to Increase Family Consent for Organ Donation. Journal of Transplantation Coordination. 1998;8(4):210–217. doi: 10.7182/prtr.1.8.4.2g64j1x161620765. [DOI] [PubMed] [Google Scholar]
- Gridelli B, Remuzzi G. Strategies for Making More Organs Available for Transplantation. New England Journal of Medicine. 2000;343(6):404–410. doi: 10.1056/NEJM200008103430606. [DOI] [PubMed] [Google Scholar]
- Hospital Blue Book. Official National Edition. Billian; Atlanta, GA.: 1998. [Google Scholar]
- Klieger J, Nelson K, David R, et al. Analysis of Factors Influencing Organ Donation Consent Rates. Journal of Transplantation Coordination. 1994;4(3):132–134. [Google Scholar]
- Levinsky NG. Organ Donation by Unrelated Donors. New England Journal of Medicine. 2000;343(6):430–432. doi: 10.1056/NEJM200008103430610. [DOI] [PubMed] [Google Scholar]
- Luskin RS, Buckley CA, Bradley JW, et al. An Alternative Approach to Evaluating Organ Procurement Organization Performance. Transplantation Proceedings. 1999;31(1-2):353–355. doi: 10.1016/s0041-1345(98)01658-3. [DOI] [PubMed] [Google Scholar]
- Marascuilo LA, McSweeney M. Nonparametric and Distribution-Free Methods for the Social Sciences. Brooks/Cole Publishing; Monterey, CA.: 1997. [Google Scholar]
- Marciniak TA, Ellerbeck EF, Radford MJ, et al. Improving the Quality of Care for Medicare Patients with Acute Myocardial Infarction: Results from the Cooperative Cardiovascular Project. Journal of the American Medical Association. 1998;279(17):1351–1357. doi: 10.1001/jama.279.17.1351. [DOI] [PubMed] [Google Scholar]
- Matas AJ, Garvey CA, Jacobs CL, Kahn JP. Nondirected Donation of Kidneys from Living Donors. New England Journal of Medicine. 2000;343(6):433–436. doi: 10.1056/NEJM200008103430611. [DOI] [PubMed] [Google Scholar]
- Murray TH, Youngner SJ. Organ Salvage Policies: A Need for Better Data and More Insightful Ethics. Journal of the American Medical Association. 1994;272(10):814–815. doi: 10.1001/jama.272.10.814. [DOI] [PubMed] [Google Scholar]
- Nathan HM, Jarrell BE, Broznik B, et al. Estimation and Characterization of the Potential Renal Organ Donor Pool in Pennsylvania: Report of the Pennsylvania Statewide Donor Study. Transplantation. 1991;51(1):142–149. doi: 10.1097/00007890-199101000-00022. [DOI] [PubMed] [Google Scholar]
- Ojo AO, Wolfe RA, Leichtman AB, et al. A Practical Approach to Evaluate the Potential Donor Pool and Trends in Cadaveric Kidney Donation. Transplantation. 1999;67(4):548–556. doi: 10.1097/00007890-199902270-00011. [DOI] [PubMed] [Google Scholar]
- Organ Procurement and Transplant Network. Internet address: http://www.optn.org (Accessed 2003.)
- President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research. Guidelines for the Determination of Death. Journal of the American Medical Association. 1981;246(19):2184–2186. [PubMed] [Google Scholar]
- Shafer TJ, Kappel DF, Heinrichs DF. Strategies for Success Among OPOs: A Study of Three Organ Procurement Organizations. Journal of Transplantation Coordination. 1997;7(1):22–31. doi: 10.7182/prtr.1.7.1.q63n04685n218417. [DOI] [PubMed] [Google Scholar]
- Siminoff LA, Gordon N, Hewlett J, Arnold RM. Factors Influencing Families' Consent for Donation of Solid Organs for Transplantation. Journal of the American Medical Association. 2001;286(1):71–77. doi: 10.1001/jama.286.1.71. [DOI] [PubMed] [Google Scholar]
- United Network for Organ Sharing. Internet address: http://www.unos.org (Accessed 2003.)
- U.S. General Accounting Office. Organ Procurement Organizations: Alternatives Being Developed to More Accurately Assess Performance. U.S. Government Printing Office; Washington, DC.: Nov, 1997. GAO/HEHS-98-26. [Google Scholar]
- U.S. General Accounting Office. Organ Donation: Assessing Performance of Organ Procurement Organizations. U.S. Government Printing Office; Washington, DC.: Apr, 1998. GAO/T-HEHS-98-131. [Google Scholar]
- Wendler D, Dickert N. The Consent Process for Cadaveric Organ Procurement: How Does It Work? How Can It Be Improved? Journal of the American Medical Association. 2001;285(3):329–333. doi: 10.1001/jama.285.3.329. [DOI] [PubMed] [Google Scholar]