Table 2. Results of Studies Validating Diagnoses of Heart Failure (HF) in Administrative Data.
| First Author, Year | Diagnostic Codes | Parameter | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | Kappa (95% CI) | LR+ | LR- | DOR | Quality |
| Levy [16], 1999 | ICD-9 402, 408, 425, 428, 429.3 | HF as a comorbidity (secondary diagnosis) of MI | 65 (59–72) | 78 (72–83) | 5.91 (3.47–10.05) | 0.39 (0.30-0.51) | 15.06 (7.47-30.37) | Medium | |||
| Goff [43], 2000 | ICD-9 428 | primary or secondary diagnosis | 62.79 (60.17–65.34) | 95.39 (94.65–96.03) | 83.48 (81.04–85.66) | 87.38 (86.28–88.35) | 13.61 (11.69–15.85 | 0.39 (0.36-0.42) | 34.90 (28.90-42.13) | High | |
| ICD-9 428 or 402 | primary or secondary diagnosis | 66.21 (63.63–68.69) | 93.28 (92.42–94.06) | 78.53 (76.04–80.84) | 88.18 (87.09–89.19) | 9.86 (8.69–11.18) | 0.36 (0.34-0.39) | 27.21 (22.95-32.26) | |||
| ICD-9 398.91, 402.x1, 404.x, 415.0, 416.9, 425.4, 428.x, 429.4, 514, 518.4, 786.0 | primary or secondary diagnosis | 67.08 (64.51–69.55) | 92.61 (91.71–93.42) | 77.11 (74.60–79.44) | 88.34 (87.28–89.33) | 0.72 | 9.08 (8.05–10.23) | 0.36 (0.33-0.38) | 25.53 (21.61-30.16) | ||
| ICD-9 428.x | females | 66 | 95.3 | 86.3 | 86.2 | ||||||
| ICD-9 428.x | males | 60.5 | 95.7 | 81.9 | 88.2 | ||||||
| ICD-9 428.x or 402.x | females | 68.8 | 92.2 | 79.7 | 86.8 | ||||||
| ICD-9 428.x or 402.x | males | 64.6 | 93.2 | 75.5 | 89.0 | ||||||
| ICD-9 398.91, 402.x1, 404.x, 415.0, 416.9, 425.4, 428.x, 429.4, 514, 518.4, 786.0 | females | 70.2 | 92.1 | 79.9 | 87.3 | ||||||
| ICD-9 398.91, 402.x1, 404.x, 415.0, 416.9, 425.4, 428.x, 429.4, 514, 518.4, 786.0 | males | 65.0 | 93.2 | 75.5 | 89.1 | ||||||
| Humphries [51], 2000 | ICD-9 428 | as pre-admission comorbid diagnosis | 66.1 | 94.8 | 50.6 | 97.2 | 0.53 | 12.71 | 0.36 | 35.28 | Medium |
| Austin a [53], 2002 | ICD-9 428 | primary diagnosis | 58.52 (57.20–59.83) | 96.78 (96.63–96.93) | 65.14 (63.78–66.46) | 95.79 (95.61–95.95) | 0.58 | 18.20 (17.29–19.17) | 0.43 (0.42-0.44) | 42.47 (39.51-45.65) | High |
| primary or secondary | 85.41 (84.44–86.33) | 84.28 (83.97–84.59) | 35.80 (34.98–36.63) | 98.25 (98.13–98.37) | 0.43 | 5.43 (5.31–5.56) | 0.17 (0.16–0.19) | 31.38 (29.01–33.94) | |||
| Szeto [50], 2002 | ICD-9 428.1, 428.0, 428.9 | in administrative database | 86.67 (58.39–97.66) | 100.00 (96.50–100.00) | 100.00 (71.66–100) | 98.00 (94.21–99.74) | 223.16 (14.1–3625.5) | 0.16 (0.05–0.49) | 1441.8 (65.77–31608.6) | Medium | |
| Onofrei [50], 2004 | ICD-9 402.01, 402.11, 402.91, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, 425.xx, 428.xx | HF defined as LVEF≤55% | 43.9 (41.3–46.6) | 99.5 (99.4–99.5) | 35.6 (33.3–37.9) | 99.6 (99.6–99.6) | 80.40 (73.98–87.37) | 0.56 (0.54–0.59) | 142.54 (126.33–160.84) | High | |
| HF defined as LVEF≤40% | 54.4 (50.8–57.9) | 99.4 (99.3–99.4) | 24.9 (22.9–27.0) | 99.8 (99.8–99.8) | 85.69 (78.81–93.17) | 0.46 (0.43–0.50) | 186.52 (160.54–216.71) | ||||
| Heckbert [55], 2004 | ICD-9 428 or 425 | 78.99 (75.96–81.74) | 97.72 (97.55–97.88) | 45.34 (42.70–48.01) | 99.49 (99.40–99.56) | 0.56 (0.53–0.59) | 34.67 (32.03–37.52) | 0.22 (0.19–0.25) | 161.27 (134.00–194.09) | High | |
| ICD-9 428 only | 48.59 (45.78–51.41) | ||||||||||
| ICD-9 425 only | 17.36 (11.75–24.76) | ||||||||||
| Rector b [49], 2004 | ICD-9 398.91, 402.01, 402.11, 402.91, 404.01, 404.11, 404.91, 404.03, 404.13, 404.93, 428.xx | from 1 year of data, diagnosis on 1 claim | 57.80 (50.93–64.38) | 92.83 (91.90–93.66) | 33.96 (29.20–39.06) | 97.18 (96.54–97.71) | 8.06 (6.83–9.51) | 0.46 (0.39–0.53) | 17.72 (13.15–23.89) | Medium | |
| from 1 year of data, diagnosis on ≥2 claims | 46.79 (40.06–53.64) | 94.67 (93.85–95.39) | 35.92 (30.39–41.83) | 96.54 (95.85–97.12) | 8.78 (7.19–10.72) | 0.56 (0.50–0.64) | 15.62 (11.51–21.19) | ||||
| Roger [42], 2004 | ICD-9 428, as primary diagnosis | against Framingham criteria | 82 (81.09–82.87) | High | |||||||
| against clinical diagnosis | 90 (89.28–90.67) | ||||||||||
| Khand [17], 2005 | ICD-10 I50.1, I50.2, I11.0, I11.1, I25.5, I42.9 | possible, probable, or definite; 1st-6th diagnostic positions | 86.67 (82.41–90.05) | High | |||||||
| probable or definite; 1st-6th diagnostic positions | 77.27 (72.29–81.60) | ||||||||||
| definite; 1st-6th diagnostic positions | 65.45 (60.02–70.53) | ||||||||||
| Ingelsson [34], 2005 | ICD-8 427.00, 427.10, 428.99; ICD-9 428; ICD-10 I50 | overall definite | 81.70 (76.91–85.71) | High | |||||||
| overall definite or questionable | 97.79 (95.31–99.03) | ||||||||||
| definite as primary diagnosis | 95.00 (89.58–97.79) | ||||||||||
| definite or questionable as primary diagnosis | 99.29 (95.49–99.96) | ||||||||||
| Lee [56], 2005 | ICD-9 428.x | vs. Framingham criteria, entire cohort; primary diagnosis | 94.33 (93.07–95.38) | High | |||||||
| vs. Framingham criteria, women; primary diagnosis | 94.62 (92.81–96.01) | ||||||||||
| vs. Framingham criteria, men; primary diagnosis | 93.91 (91.97–95.42) | ||||||||||
| vs. Carlson criteria, entire cohort; primary diagnosis | 88.60 (86.94–90.08) | ||||||||||
| vs. Carlson criteria, women; primary diagnosis | 89.35 (87.02–91.32) | ||||||||||
| vs. Carlson criteria, men; primary diagnosis | 87.83 (85.32–89.96) | ||||||||||
| Birman-Deych [15], 2005 | ICD-9 428.x, 398.91, 402.01, 402.11, 402.91, 404.01, 404.11, 404.91, 404.03, 404.13, 404.93 | any diagnostic position (current or past event) | 76.00 (75.19–76.80) | 97.95 (97.69–98.19) | 97.00 (96.61–97.34) | 82.41 (81.79–83.02) | 37.10 (32.87–41.87) | 0.25 (0.24–0.25) | 151.44 (132.90–172.57) | Medium | |
| primary position, during baseline hospitalization (current event) | 33.00 (32.13–33.89) | 99.41 (99.26–99.53) | 97.98 (97.46–98.40) | 63.15 (62.47–63.81) | 55.97 (44.59–70.24) | 0.67 (0.67–0.68) | 83.04 (65.95–104.56) | ||||
| any position, during baseline hospitalization (current event) | 83.00 (82.29–83.70) | 86.00 (85.38–86.60) | 85.00 | ||||||||
| any position, all hospitalizations within past 12 months of baseline (current event) | 86.00 (85.33–86.64) | 82.01 (81.33–82.68) | 80.64 (79.91–81.35) | 87.05 (86.43–87.65) | 4.78 (4.60–4.97) | 0.17 (0.16–0.18) | 28.01 (26.11–30.05) | ||||
| So [48], 2006 | ICD-9 428.x | as a comorbidity of MI | 81.8 (68.65–90.48) | 96.4 (91.75–98.81) | 90.0 (78.19–96.67) | 93.0 (87.52–96.60) | 22.58 (9.47–53.87) | 0.19 (0.11–0.33) | 119.70 (38.84–368.88) | Medium | |
| ICD-10 I09.9, I11.0, I13.0, I13.2, I25.5, I42.2, I42.5-I42.9, I43.x, I50.x, P29.0 | 80.0 (67.03–89.57) | 97.8 (93.78–99.55) | 93.6 (82.46–98.66) | 92.5 (86.92–96.18) | 36.80 (11.92–113.58) | 0.20 (0.12–0.35) | 180.00 (48.03–674.60) | ||||
| Henderson [46], 2006 | ICD-10 I50 | 1998–1999 | 87.0 4 (83.13–90.17) | 99.44 (99.22–99.60) | 89.89 (86.22–92.69) | 99.26 (99.02–99.45) | 0.88 | 155.87 (112.77–215.44) | 0.13 (0.10–0.17) | 1195.7 (769.3–1858.4) | High |
| 2000–2001 | 86.25 (79.71–91.00) | 99.71 (99.55–99.81) | 86.25 (79.71–91.00) | 99.71 (99.55–99.81) | 0.86 | 292.90 (192.10–446.59) | 0.14 (0.09–0.20) | 2123.9 (1148.9-3926.4) | |||
| Alqaisi c [45], 2009 | ICD-9 428.xx, 398.91, 402.01, 402.11, 402.91 | 65.00 (60.08–69.63) | High | ||||||||
| Chen [47], 2009 | ICD-10 I09.9, I11.0, I13.0, I13.2, I25.5, I42.0, I42.5- I42.9, I43.x, I50.x, P29.0 | 1st–16th diagnostic positions | 68.56 (63.24–73.45) | 99.32 (98.98–99.55) | 90.16 (85.65–93.41) | 97.20 (96.61–97.70) | 100.76 (67.72–149.92) | 0.32 (0.27–0.37) | 318.33 (201.73–502.32) | Medium | |
| kappa | 0.76 | ||||||||||
| prevalence-adjusted, bias-adjusted kappa | 0.94 | ||||||||||
| Heisler [52], 2009 | ICD-9 428, 428.0, 428.1, 428.9 | 0.00 (0.00–80.21) | 100.00 (99.32–100.0) | n/a (0/0) | 99.72 (98.87–99.95) | Low | |||||
| Merry [54], 2009 | ICD-9 428 | 43.04 (35.27–51.15) | 99.92 (99.86–99.95) | 80.00 (69.64–87.60) | 99.57 (99.45–99.64) | 530.43 (319.19–881.47) | 0.57 (0.50–0.65) | 930.44 (525.99–1645.9) | High |
a
Validation was conducted amongst a coronary care unit population.
b
Thirty-eight algorithms were evaluated in this study; the first parameter we selected achieved the highest sensitivity with specificity ≥90%, the second was chosen for comparison purposes.
c
Sixteen algorithms were evaluated in this study; the validity of the primary algorithm is reported here 95% CI = 95% confidence interval; DOR = diagnostic odds ratio; HF = heart failure; LR+ = positive likelihood ratio; LR- = negative likelihood ratio; LVEF = left ventricular ejection fraction; MI = myocardial infarction; NPV = negative predictive value; PPV = positive predictive value.