Table 4.
Potential Sources of Bias for 16 Validated General Prognostic Indices
| INDEX | Adequate description of sample, (Participation rate)a | Clearly defined, reproducible prognostic variablesb | Blinded measurement of potential prognostic variables and mortalityc | Completeness of potential predictorsd | Completeness of mortality outcomee | Model building is conceptually based, stability of model testedf |
|---|---|---|---|---|---|---|
| Community | ||||||
| Gagne 201131 | Partly Race/ethnicity not described (participation not optional in this administrative dataset) |
Partly ICD-9 codes have limited reproducibility |
Yes | NR | NR | Partly Stability of model not tested |
| Mazzaglia 200724 | Partly Race/ethnicity not described (Italian sample), participants not compared with non-participants |
Partly “Inadequacy of income” not well described |
Yes | NR | 99% | Yes |
| Carey 200415 | Partly No comparison of respondents to non- respondents |
Yes | Yes | 99.3% | NR | Yes |
| Carey 200814 | Yes (participation not optional in this administrative dataset) | Yes | Yes | 92% | NR | No Not conceptually based; stability of model not tested |
| Lee 200622 | Partly Participants not compared with non-participants (81% participation rate) |
Yes | Yes | NR | 98% | Yes |
| Schonberg 200930 | Partly Participants not compared with non-participants (74% participation rate) |
Yes | Yes | 95% | 97% | Yes |
| Nursing Home | ||||||
| Porock 200526 | Partly Comorbities not described (participation not optional in this administrative dataset) |
Yes | Yes | NRg | >99% linkage to Missouri death certificates | Yes |
| Flacker 200320 | Yes (participation not optional in this administrative dataset) | Yes | Yes | NR | 87% | Yes |
| Hospital | ||||||
| di Bari 201016 | Partly Race/ethnicity not described (Italian sample); “admitted for medical reasons” not clear; (participation not optional in this administrative dataset) |
Partly Admission to “day hospital” not clearly defined |
Yes | 91% linkage across 4 datasets, 0% after linkage | 91% linkage across 4 datasets, including mortality | No Not conceptually based; stability not tested |
| Fischer 200618 | Yes | Yes | Partly For validation, 10% blinded chart review with 100% agreement |
NR | 98% | Partly Final predictors for model selected a priori; stability not tested |
| Inouye 200321 | Partly Participants not compared with non-participants (86% participation rate) |
Partly ICD-9 codes have limited reproducibility |
Yes | >99% for all predictors | 100% | Yes |
| Pilotto 200825 | Partly Race/ethnicity not described (Italian sample), participants not compared with non-participants (80% participation rate) |
Yes | Yes | 90% | 82% | Yes |
| Teno 200033 | Partly Race/ethnicity and participation rate not described |
Yes | Yes | 81% | 100% | Yes |
| Levine 200723 | Partly Participation rate not reported |
Partly ICD-9 codes have limited reproducibility |
Yes | NR | >99% | No Not conceptually based; stability not tested |
| Walter 200132 | Partly Participation rate not described. |
Yes | Yes | 96% | 100% | Yes |
| Drame 200817 | Partly Race/ethnicity not described (French sample) (87% participation rate) |
Yes | Yes | NR | 92% | No Not conceptually based; stability of model not tested |
Sample description: study and source populations clearly defined and study sample clearly described (age, sex, race/ethnicity, comorbidities, baseline mortality rates), enrollment procedures clear and, unless administrative data, comparison of participants and non-participants (Yes/Partly/No/Unsure). Participation rates provided for studies requiring consent.
Prognostic variables defined: clear, reproducible measures (Yes/Partly/No/Unsure). ICD-9 codes rated partly due to concerns about reproducibility.43
Blinding: developers of the prognostic index were blinded to the measurement of potential prognostic variables and mortality outcomes (Yes/Partly/No/Unsure). Secondary analyses of existing data categorized as yes.
Completeness of Predictors: % sample with complete predictors
Completeness of Mortality: % sample with complete follow up or % successful linkage to vital statistics records (e.g. National Death Index)
Model building: Selection of potential predictors is conceptually based, and stability of model by varying assumptions and/or modeling techniques is tested (Yes/Partly/No/Unsure)
Abbreviations: NR=not reported; SS#= social security number; DOB=date of birth