Table 2.
Results comparing the performance of the cut-off methods in four samples of 5,000 addresses each from Fife and Tayside
| NHS Fife Population 213 (4.3%) true care-home addresses Estimate (95% CI) | NHS Tayside Population 253 (5.1%) true care-home addresses Estimate (95% CI) | NHS Fife ≥65-year olds 556 (11.1%) true care-home addresses Estimate (95% CI) | NHS Tayside ≥65-year olds 431 (8.6%) true care-home addresses Estimate (95% CI) | |
|---|---|---|---|---|
| Phonics score area under the curve | 0.950 (0.930–0.970) | 0.863 (0.835–0.890) | 0.924 (0.909–0.939) | 0.934 (0.918–0.950) |
| Markov score area under the curve | 0.957 (0.937–0.976) | 0.935 (0.914–0.956) | 0.966 (0.956–0.977) | 0.986 (0.979–0.994) |
| Phonics score researcher-defined cut-off | 200 (cut-off ≥13) | 199 (cut-off ≥13) | 485 (cut-off ≥13) | 400 (cut-off ≥13) |
| Sensitivity | 87.4% (82.1–91.4) | 72.7% (66.7–78.0) | 84.9% (81.5–87.7) | 87.0% (83.4–89.9) |
| Specificity | 99.7% (99.5–99.8) | 99.6% (99.4–99.8) | 99.7% (99.4–99.8) | 99.5% (99.2–99.6) |
| Positive predictive value | 94.0% (89.5–96.7) | 92.4% (87.6–95.5) | 97.3% (95.3–98.5) | 93.8% (90.8–95.8) |
| Negative Predictive value | 99.4% (99.2–99.6) | 98.5% (98.1–98.8) | 98.1% (97.6–98.5) | 98.7% (98.4–99.1) |
| Phonics score maximising Youden’s J | 222 (cut-off ≥0.50) | 199 (cut-off ≥13.0) | 485 (cut-off ≥12.9) | 400 (cut-off ≥13.0) |
| Sensitivity | 90.2% (85.3–93.7) | 72.7% (66.7–78.0) | 84.9% (81.6–87.7) | 87.0% (83.4–90.0) |
| Specificity | 99.4% (99.1–99.6) | 99.7% (99.5–99.8) | 99.7% (99.5–99.8) | 99.5% (99.2–99.6) |
| Positive predictive value | 87.4% (82.1–91.3) | 92.5% (87.6–95.6) | 97.3% (95.3–98.5) | 93.8% (90.8–95.8) |
| Negative predictive value | 99.6% (99.3–99.7) | 98.6% (98.2–98.9) | 98.1% (97.7–98.5) | 98.8% (98.4–99.1) |
| Markov score researcher-defined cut-off | 201 (cut-off ≥29.6) | 194 (cut-off ≥29.6) | 501 (cut-off ≥29.6) | 418 (cut-off ≥29.6) |
| Sensitivity | 84.2% (78.5–88.7) | 69.2% (63.0–74.7) | 85.4% (82.2–88.2) | 90.3% (86.9–92.8) |
| Specificity | 99.6% (99.3–99.7) | 99.6% (99.4–99.8) | 99.4% (99.1–99.6) | 99.4% (99.1–99.6) |
| Positive predictive value | 90.0% (84.8–93.7) | 90.2% (84.9–93.8) | 94.8% (92.3–96.5) | 93.1% (90.1–95.2) |
| Negative Predictive value | 99.3% (98.9–99.5) | 98.4% (97.9–98.7) | 98.2% (97.8–98.6) | 99.1% (98.8–99.3) |
| Markov score maximising Youden’s J | 301 (cut-off ≥5.9) | 399 (cut-off ≥5.0) | 620 (cut-off ≥5.9) | 589 (cut-off ≥4.9) |
| Sensitivity | 90.7% (85.8–94.1) | 85.8% (80.7–89.7) | 92.6% (90.0–94.6) | 97.2% (95.1–98.5) |
| Specificity | 97.8% (97.3–98.2) | 96.2% (95.6–96.7) | 97.6% (97.1–98.1) | 96.3% (95.7–96.8) |
| Positive predictive value | 64.8% (59.1–70.1) | 54.4% (49.4–59.3) | 83.1% (79.8–85.9) | 71.1% (67.3–74.7) |
| Negative predictive value | 99.6% (99.3–99.7) | 99.2% (98.9–99.4) | 99.1% (98.7–99.3) | 99.7% (99.5–99.9) |
| Markov score sensitivity = specificity | 555 (cut-off ≥1.4) | 750 (cut-off ≥1.6) | 816 (cut-off ≥2.2) | 572 (cut-off ≥6.1) |
| Sensitivity | 92.6% (88.0–95.5) | 88.9% (84.2–92.4) | 93.9% (91.5–95.7) | 96.5% (94.2–97.9) |
| Specificity | 92.6% (91.8–93.3) | 88.9% (88.0–89.8) | 93.4% (92.6–94.1) | 96.6% (96.0–97.1) |
| Positive predictive value | 35.9 (31.9–40.0) | 30.0% (0.27–0.33) | 63.9% (60.6–67.3) | 72.7% (68.8–76.3) |
| Negative predictive value | 99.6 (99.4–99.8) | 99.3% (99.0–99.6) | 99.2% (98.9–99.4) | 99.7% (99.4–99.8) |
Number in bold is the number the method identified as a care-home using the cut-off value quoted. Area under curve—a measure of evaluating how well each test discriminates between care-home and non-care-home addresses. The closer the value is to 1, the better the performance of the test. Cut-offs are researcher-defined aiming to maximise positive predictive value (ideally >95%) with adequate sensitivity (ideally >80%), Youden’s J (sensitivity plus specificity minus one with values closer to one indicating better test performance), and sensitivity = specificity (which did not exist for Phonics so not shown). Sensitivity—the proportion of care-home addresses correctly identified by each method; Specificity—the proportion of non-care-home addresses correctly identified by each method; Negative predictive value—the proportion of addresses identified as non-care-home addresses which are not care-homes; Positive predictive value—the proportion of addresses identified as care-homes which are care-home addresses.