Disease Risk Score (DRS) as a Confounder Summary Method: Systematic Review and Recommendations

Abstract

Purpose

To systematically examine trends and applications of the disease risk score (DRS) as a confounder summary method.

Methods

We completed a systematic search of MEDLINE and Web of Science® to identify all English language articles that applied DRS methods. We tabulated the number of publications by year and type (empirical application, methodological contribution, or review paper) and summarized methods used in empirical applications overall and by publication year (<2000, ≥2000).

Results

Of 714 unique articles identified, 97 examined DRS methods and 86 were empirical applications. We observed a bimodal distribution in the number of publications over time, with a peak 1979-1980, and resurgence since 2000. The majority of applications with methodological detail derived DRS using logistic regression (47%), used DRS as a categorical variable in regression (93%), and applied DRS in a non-experimental cohort (47%) or case-control (42%) study. Few studies examined effect modification by outcome risk (23%).

Conclusion

Use of DRS methods has increased yet remains low. Comparative effectiveness research may benefit from more DRS applications, particularly to examine effect modification by outcome risk. Standardized terminology may facilitate identification, application, and comprehension of DRS methods. More research is needed to support the application of DRS methods, particularly in case-control studies.

INTRODUCTION

Epidemiologic analyses often require investigators to control for many measured confounding variables. Restriction, stratification and matching allow for easily interpretable analysis, yet become complex as the number of variables for adjustment increase.¹ Adjustment using multivariable regression techniques has thus become a standard method to control for confounding. In addition to conventional multivariable regression methods that include the exposure and potential confounding variables in a single outcome model, two methods of confounder summary score techniques have been proposed: the exposure propensity score (EPS), and the disease risk score (DRS).^2-8 EPS reflect patients’ exposure probability conditional on measured confounders. This confounder summary score can then be used in place of the individual confounding variables in conventional adjustment methods, such as: matching, stratification, weighting, restriction, or as a covariate in the outcome model.^3,4,9 Use of EPS has increased exponentially since its introduction in 1983.^2,3 However, EPS is limited when exposure is rare and can be complicated when studying multiple exposures or multiple exposure levels.

The DRS is the prognostic analogue of the EPS, derived based on the predicted risk of disease outcome and was first proposed methodologically in 1976.⁵ Early simulation work published in 1979 concluded that the DRS method may overestimate the effect of confounders and thus bias results.¹⁰ A subsequent simulation published in 1989 concluded that overestimation of confounders may be rare, particularly when applying the DRS as a categorical variable.¹¹ Recent evidence also identifies that in the setting of a large number of exposed individuals and outcomes, conventional multivariable regression, EPS and DRS methods yield similar results provided covariates are not highly correlated with exposure.^3,6-8,12 Although the intention of EPS and DRS in summarizing confounders into a single summary score is similar, the logic behind the methods is distinct. EPS model the treatment selection process to balance treatment determinants, similar in concept to randomization in clinical trials. DRS do not share this feature of balancing baseline covariates across treatment groups. Rather, DRS seek to balance outcome determinants such that baseline outcome risk is similar between treatment groups. In contrast to EPS, DRS are not limited when exposure is rare or categorical, and they can provide a meaningful scale across which investigators can examine effect modification. Despite their advantages and even though they were initially proposed before EPS,^5,13-15 DRS have received less attention in the epidemiologic literature.¹⁶ We sought to systematically examine trends the use and application of DRS as a confounder summary method.

METHODS

We conducted a systematic literature search to identify all English language articles that utilized DRS confounder summary score methods in studies of humans. We searched the MEDLINE database from 1965 to May 2011 with keyword terms: “disease$ risk$ score$”, “summary$ risk$ score$”, “multivariate$ risk$ score$”, “confounder$ score$”, and “Miettinen$ confounder$ score$”; it was important to include terms in quotations to avoid inclusion of papers that may have used comorbidity indices, such as the Charlson Index,¹⁷ or a risk scoring system, such as the Framingham risk score.¹⁸ We then used Web of Science® to perform a citation search to identify papers that referenced seminal DRS method papers,^5,6,10,11 and an author search to identify articles published by investigators noted to have frequently used DRS (PG Arbogast and WA Ray). Two authors (MT, JJG) reviewed all abstracts to exclude articles that clearly did not meet eligibility criteria. Discrepancies were resolved by agreement. Full text articles were then reviewed to confirm eligibility. The number of eligible publications was plotted by calendar year and type, as: empirical application, methodological contribution or review paper. We then focused exclusively on the empirical applications and abstracted: author, journal, year of publications, terminology used to describe method, study design, sample size, exposure and outcome variable, number of outcome events, primary analytical methods to derive DRS (statistical method used, data type, number of covariates in model), and how DRS was applied. One author (MT) extracted all data and a second author (JJG) verified all extracted data. The area of study and methods of DRS derivation and application were tabulated overall, and stratified by calendar year of publication in print (before or after January 2000).

RESULTS

Of 714 unique articles identified, 97 studies were eligible: 8 methodological contributions,^{5-7,10-12,19,20} 3 review papers,^16,21,22 and 86 were empirical applications,^23-108Figure 1. We retained one abstract that had not yet been published in full form,¹² and excluded one abstract identified through the search that was recently published in a full-text article.^8,20 The keyword search identified 173 articles (20 relevant empirical), the major paper citation search identified 338 articles (73 relevant empirical), and the author search identified 320 articles (11 relevant empirical) – each method yielded unique empirical papers, and no paper was identified by all three search methods, Figure 2. The empirical studies were published between March 1976 and May 2010 with a bimodal distribution; 32 (37%) articles were published prior to 1990, 15 (17%) in the 1990s, and 39 (45%) published since 2000 (Figure 3).

Figure 1.

Flow Diagram of Systematic Search Results

Figure 2.

Venn Diagram of Search Result Yield of Empirical Applications, by Search Strategy, N=86

Figure 3.

Number of Disease Risk Score Confounder Summary Score Publications, by Year of Publication, N=97. Empirical application (solid, n=86), methodological contribution (diagonal stripes, n=8) and review papers (horizontal stripe, n=3); 35 papers published before 1990, 16 papers published between 1990 and 1999, and 46 papers published since January 2000.

Table 1 summarizes the DRS derivation and application methods, with full details presented for each paper in the online Appendix. The most common terminology used to describe DRS methods included the words and/or combinations of: 1. summary, 2. confounder, 3. Miettinen, and 4. score; and many included disease specific terminology. Cohort (47%) and case-control (42%) studies were the most common study designs. Studies of cancer risk (27%) and drug effects (24%) were the most common applications. Application focus changed over time, with environmental and social exposures/outcomes (32%) and cancer risk (19%) the most common before 2000 and drug exposures (46%) and skin cancer risk (36%) dominating since 2000.

Table 1.

Characteristics of disease risk score confounder summary method applications, N=86

			Year of Publication
	Total (n=86)		<2000 (n=47)		≥2000 (n=39)
	No.	%	No.	%	No.	%
Area of Study
Drug effects	21	24.4	3	6.4	18	46.2
Environmental and social exposure/disease	18	20.9	15	31.9	3	7.7
Hospital Care/Health Care	10	11.6	8	17.0	2	5.1
Cancer Risk	23	26.7	9	19.1	14	35.9
Skin Cancer Risk	15	17.4	1	2.1	14	35.9
Pregnancy outcomes	8	9.3	7	14.9	1	2.6
Surgical effectiveness	4	4.7	3	6.4	1	2.6
Ophthalmology	2	2.3	2	4.3	0	0.0
Study Design
Cohort	40	46.5	19	40.4	21	53.8
Case-Control	36	41.9	23	48.9	13	33.3
Cross-sectional	5	5.8	3	6.4	2	5.1
Clinical Trial	5	5.8	2	4.3	3	7.7
DRS Derivation (Regression Method)
Not Reported	9	10.5	6	12.8	3	7.7
Reported	77	89.5	41	87.2	36	92.3
Logistic	36	46.8	15	36.6	21	58.3
Discriminant	13	16.9	13	31.7	0	0.0
Linear	8	10.4	8	19.5	0	0.0
Poisson	7	9.1	0	0.0	7	19.4
Cox Proportional Hazards	6	7.8	1	2.4	5	13.9
Other	7	9.1	4	9.8	3	8.3
Primary DRS Application Method
Not Reported	3	3.5	2	4.3	1	2.6
Reported	83	96.5	45	95.7	38	97.4
Stratification	50	60.2	40	88.9	10	26.3
Included as covariate	29	34.9	5	11.1	24	63.2
Matching	2	2.4	0	0.0	2	5.3
Other	2	2.4	0	0.0	2	5.3
Primary DRS Application Variable
Not Reported	13	15.1	10	21.3	3	7.7
Reported	73	84.9	37	78.7	36	92.3
Continuous	5	6.8	2	5.4	3	8.3
Categorical*	68	93.2	35	94.6	33	91.7
2	4	5.9	1	2.8	3	9.1
3	19	27.9	8	22.8	11	33.3
4	13	19.1	7	20.0	6	18.2
5	19	27.9	16	45.7	3	9.1
10	12	17.6	3	8.6	9	27.3
20	2	2.9	0	0.0	2	6.0
Other	5	7.3	4	11.4	1	3.0

^*Some applications examined several categories and thus proportions add to greater than 100%

Appendix.

Detailed Description of Studies Utilizing the Disease Risk Score Confounder Summary Method, N=86

Author	Terminology	Study Design	N (subjects unless stated)	Exposure	Outcome(s)	No. Outcomes (cases in case-control)	Measure Reported	Primary DRS Derivation		Primary DRS Application
								Method	No. Var	Method	Variable
Applebaum 2007 (20)	Multivariate confounder score	Case-control	2,326	Arsenic	BCC and SCC	BCC, n=880 SCC, n=666	Odds ratio	“Multivariate confounder method” (controls only) and Miettinen method	6	Covariate in multi-variable model	Cat-4
Applebaum 2009 (21)	Multivariate confounder score	Case-control	559	Oral Contraceptives	SCC	261	Odds ratio	“Multivariate confounder method” (controls only) and Miettinen method	6	Covariate in multi- variable model	Cat-4
Axelson 1983 (22)	Miettinen c onfounder score technique	Cohort	84 pregnancies in 30 women	Work in tire building department of rubber factory	Abnormal pregnancy	16	Rate ratio	Linear	4	Stratified (M-H)	Cat-5
Boyce 1986 (23)	Confounder summarization procedure	Cohort	968	Social and cultural factors	Maternal and neonatal complications	Maternal, n=445 Neonatal, n=136	Relative risk	Logistic	7	Stratified and linear trend	Cat-5
Bravata 2010 (24)	Risk adjustment score	Cohort	1,487	Post-stroke or transient ischemic attach care processes (n=7)	Combined (in-hospital mortality, discharge to hospice or skilled nursing facility)	239	Odds ratio	Logistic	13	Only covariate with exposures in multi-variable model	Continuous
Chung 1982 (25)	Multivariate confounder score	Cohort	16,961	Past induced abortion	Spontaneous fetal loss in subsequent pregnancies	NR	Relative risk	Logistic	6	Stratified (M-H)	Cat-5
Cohen 1997 (26)	Multivariate risk score	Clinical trial	3,809	Various risk factors for bleeds	Bleeding	421	Odds ratio	Logistic	10	Stratified	Cat-4
Dash 2006 (27)	Multivariate confounder score	Cohort	196	Elective partial nephrectomy (PN) vs. radical nephrectomy (RN)	Disease-free survival	21	Hazard ratio	Cox	7	Only covariate with exposures in multi-variable model	Continuous
Daubs 1981 (28)	Multivariate risk score	Case-control	N(1)=1,274; N(2)=1,002	(1)Intraocular pressure; (2)Refractive error	(1)Myopia (2)Primary open angle glaucoma	(1)n=272 (2)n=672	Relative risk	Discriminant	3,11	Stratified (M-H)	Cat-4,5
Elwood 1978 (29)	Risk score	Case-control	6,391	Geographical and ethnic influence	Anencephalus	1,391	Risk ratio	Linear discriminant	11	Stratified	Cat-5
Ensrud 2008 (30)	Summary fracture risk score	Clinical trial	10,101	Raloxifene	Fractures: nonvertebral and clinical vertebral	Nonvertebral, n=866 Clinical vertebral, n=161	Hazard ratio	Logistic (placebo group)	14,3,2	Stratified	Cat-3
Fiebach 1990 (31)	Multivariate confounder score	Cohort	467	Admission for chest pain to stepdown unit vs. coronary care unit	Adverse outcomes ( death, life-threatening or other serious complications, or need for a major invasive procedure)	175	Relative risk	Logistic	>50 considered	Stratified	Cat-NR
Flodin 1986 (32)	Miettinen confounder score technique	Case-control	413	Various e nvironmental and occupational exposures	Acute myeloid leukemia	59	Rate ratio	Linear	8	Stratified (M-H)	Cat-3
Flodin 1987 (33)	Miettinen confounder score technique	Case-control	562	Various environmental and occupational exposures	Multiple myeloma	131	Rate ratio	Linear	10	Stratified (M-H)	Cat-NR
Flodin 1988 (34)	Miettinen confounder technique	Case-control	550	Various environmental and occupational exposures	Multiple sclerosis	83	Rate ratio	Linear	11	Stratified (M-H)	Cat-NR
Flodin 1988(35)	Miettinen confounder score technique	Case-control	542	Various environmental and occupational exposures	Chronic lymphatic leukemia	111	Rate ratio	Linear	8	Stratified (M-H)	Cat-NR
Flodin 1990 (36)	Miettinen confounder score technique	Case-control	158	Various environmental and occupational exposures	Acute myeloid leukemia	86	Rate ratio	Linear	14	Stratified (M-H)	Cat-NR
Fung 2002 (37)	Multivariate risk score	Cohort	107,975	Alcohol Intake	BCC	6,088	Relative risk	Logistic	NR	Stratified	Cat-3
Fung 2002 (38)	Risk score	Cohort	85,836	Vitamins and carotenoids	BCC	5,392	Relative risk	Logistic	NR	Stratified	Cat-3
Giles-Corti 2002 (39)	Multivariate summarizeation score technique or determinant scores	Cross-sectional	1,773	Individual, social, and physical variables	Physical activity	1,450	Odds ratio	Logistic	12	Stratified	Cat-3,10
Giles-Corti 2003 (40)	Multivariate summarizeation score technique or determinant scores	Cross-sectional	1,803	Individual, social, and physical variables	Walking	310	Odds ratio	Logistic	12	Stratified	Cat-3,10
Gillum 1978 (41)	Multivariate confounder summarizing score	Case-control	1,708	Sociocultural mobility	CHD, MI, Angina, and HTN	CHD, n=88 MI, n=78 Angina, n=48 HTN, n=319	Incidence rate ratio	Binary	14,19	Stratified (M-H)	Cat-5
Graham 2005 (42)	CV risk score	Case-control	39,639	NSAIDS	Serious CHD	8,143	Odds ratio	Logistic (unexposed)	32	Covariate in multi- variable model	Cat-10
Grijalva 2007 (43)	Summary risk score	Cohort	14,932	DMARDs	Non-persistence and adherence	Non-persistence, n=8,835	Hazard ratio and model coefficient	Persistence- Cox Adherence- Linear	48	Covariate in multi- variable model	Cat-5
Han 2004 (44)	Multivariate confounder score	Nested case-control	1,678	Genetic polymorphisms of XRCCI	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=300, SCC, n=286 melanoma, n=219	Odds ratio	Logistic	6	Covariate in multi-variable model	Cat-3
Han 2005 (45)	Multivariate confounder score	Nested case-control	1,679	Genetic polymorphisms of XPD	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=300, SCC, n=286 melanoma, n=219	Odds ratio	Logistic	6	Covariate in multi- variable model	Cat-2
Han 2006 (46)	Multivariate confounder score	Nested case-control	1,679	Genetic polymorphisms of p53 Codon 72	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=300, SCC, n=286 melanoma, n=219	Odds ratio	Logistic	NR	Covariate in multi- variable model	Cat-3
Han 2006 (47)	Multivariate confounder score	Nested case-control	1,562	Constitutional factors and sun exposure	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=283, SCC, n=275 melanoma, n=200	Odds ratio	Logistic	5	Covariate in multi- variable model	Cat-3
Han 2007 (48)	Multivariate confounder score	Nested case-control	1,678	Genetic polymorphisms of V16A on MnSOD gene	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=300, SCC, n=286 melanoma, n=219	Odds ratio	Logistic	6	Covariate in multi- variable model	Cat-3
Han 2007 (49)	Multivariate confounder score	Nested case-control	1,678	Genetic polymorphisms (vitamin D and folate)	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=300, SCC, n=286 melanoma, n=219	Odds ratio	Logistic	6	Covariate in multi- variable model	Cat-3
Hennekens 1976 (50)	Multivariate risk score	Case-control	1,298	Coffee drinking	Death due to congestive heart failure	649	Risk ratio	Linear discriminate	21	Stratified	Cat-5
Heyden 1980 (51)	Confounder summarization score	Cohort	1,165	Sex	Diabetes-related coronary mortality	124	Rate ratio	NR	6	Only covariate with exposures in multi-variable model	NR
Hill 2000 (52)	Confounder score	Cohort	150	Maternal smoking, drinking and familial susceptibility to alcohol dependence	Psychiatric disorders	NR	Odds ratio	Logistic	2	Covariate in multi- variable model	NR
Hirsch 2009 (53)	Estimated baseline risk	Clinical trial	1,139	In-hospital revascularization and early invasive vs. selective invasive treatment strategy	Long-term mortality	74	Hazard ratio	Cox	11	Stratified	Cat-3
Holman 1999 (54)	Comorbidity summarization score	Cohort	19,598	Transurethral resection of p rostate vs. open prostatectomy	Death	4,845	Rate ratio	Cox	21	Covariate in multi- variable model	Fractional polynomial of continuous variable
Hooton 1981 (55)	Risk strata	Cohort	169,518	Various risk factors	Nosocomial infection	NR	Incidence rates	Risk tree model	8,8,9,10	Stratified	Cat-13,10, 9, 6
Johnson 1992 (56)	Miettinen's multivariate confounder score	Cross-sectional	2,544	Adolescent smoking, weight changes, and binge-purge behavior	Secondary amenorrhea	215	Relative risk	NR	NR	NR	NR
Joseph 1996 (57)	Confounder score	Case-control	4,061	Major Tranquillizers	Death or near death from asthma	131	Relative risk	Logistic	NR	Covariate in multi- variable model	Continuous
Journois 2005 (58)	Multivariate c onfounder score	Cohort, historical comparator	64	Inhaled nitric oxide use in severe postoperative p ulmonary hypertension	Early postoperative mortality	27	Odds ratio reported as risk ratio	Logistic	NR	Matched	Continuous
Knowler 1980 (59)	Multivariate risk-indicator score	Cohort	163	Systolic blood pressure	Retinopathy	54	Rate difference and rate ratio	Linear	13	Stratified	Cat-5
Koopman 1991 (60)	Predictive risk score	Cross-sectional	3,408 households	Various risk factors	Dengue infection	NR	Odds ratio	NR	NR	Covariate in multi- variable model	NR
Levin 1980 (61)	Multivariate confounder score	Case-control	1,312	Past induced abortions	Pregnancy loss	240	Relative risk	Discriminant	26	Stratified	Cat-NR
Magnus 1979 (62)	Confounder summarizing score	Case-control	1,348	Light physical activity	Acute coronary events	473	Rate ratio	Linear discriminant	14	Stratified	Cat-10
Matroos 1979 (63)	Summary score	Case-control	1,390	Cigarette or cigar smoking	Acute coronary events	499	Rate ratio	Linear discriminant	18	Stratified	Cat-10
Matthai 1994 (64)	Predictive model	Clinical trial	2,166	Low- vs. high-osmolality contrast agents in cardia angiography	Adverse events	78	Odds ratio	Logistic	NR	Stratified	Cat-4
Miller 2006 (65)	Multivariate confounder score	Case-control	2,364	A23G single nucleotide polymorphism	Skin Cancer (BCC, SCC)	BCC, n=886 SCC, n=682	Odds ratio	NR	6	Covariate in multi- variable model	Cat-4
Nan 2008 (66)	Multivariate confounder score	Case-control	1,679	P53 codon 72 polymorphism and its interaction with melanocortin 1 receptor variants	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=300 SCC, n=286 Melanoma, n=219	Odds ratio	Logistic	6	Stratified	Cat-3
Nan 2009 (67)	Multivariate confounder score	Case-control	1,678	MDM2 polymorphism and its interaction with the p53 Arg72Pro polymorphism	Skin Cancer: 1.BCC, 2.SCC, 3.melanoma	BCC, n=300 SCC, n=286 Melanoma, n=219	Odds ratio	Logistic	6	Covariate in multi- variable model	Cat-2
Nelemans 1993(68)	Sun-sensitivity summary score	Case-control	324	Intermittent E xposure to Sunlight	Melanoma	141	Odds ratio	Logistic	6	Stratified	Cat-2
Olsen 1991 (69)	Multivariate confounder score	Cohort	1,752	Social Network Strength	All-cause and cardiovascular mortality	1,501	Hazard ratio	NR	NR	Stratified (Cox)	Cat-5
Orth-Gomer 1980 (70)	Multivariate confounder score	Case-control	150	Psychological stress	Ischemic heart disease	50	Relative risk	Linear discriminate	NR	Stratified (M-H)	Cat-5
Orth-Gomer 1980 (71)	Multivariate confounder score	Case-control	150	Pattern-A behavior	Ischemic heart disease	50	Odds ratio	Linear discriminate	NR	Stratification	Cat-5
Parker 2002 (72)	Risk group	Clinical trial	6,797	Enalapril	Combined (death or hospitalization for heart failure)	2,275	Relative risk	Logistic	NR	Stratified	Cat-3
Pater 1979 (73)	Miettinen's Multivariate confounding score	Cohort	1,419	Auxometry	5-year breast cancer recurrence	372	Odds ratio	Linear	NR	Stratified	Cat-3
Rajala 1980 (74)	Multivariate confounder summarizing score	Cross-sectional	212	Tooth brushing	Dental caries	NR	Prevalence difference	Linear discriminant	7	Stratified	Cat-4,3
Rantakallio 1992 (75)	Confounder score	Cohort	5,966	Maternal smoking in pregnancy	Delinquency in offspring	355	Risk difference and risk ratio	Logistic (unexposed)	NR	Stratified	Cat-5
Rantakallio 1995 (76)	Confounder Score	Cohort	20,097	Maternal Build	Pregnancy outcome (preterm births, perinatal and childhood deaths, birth weight)	NR	Odds Ratio	Logistic	6	Covariate in multi-variable model	Cat-3
Ray 2001 (77)	Summary CV risk score	Cohort	481,744	Antipsychotics	Sudden cardiac death	1,487	Rate ratio	Poisson (unexposed)	NR	Stratified and covariate in multi-variable model	Cat-4
Ray 2002 (78)	Summary CV disease risk score	Cohort	656,875	NSAIDS and Cox-2	Serious coronary heart disease	5,316	Rate ratio	NR	NR	Covariate in multi- variable model	NR
Ray 2004 (79)	Summary Score	Cohort	1,246,943 PY	Erythromycin	Sudden cardiac deaths	1,476	Rate ratio	NR	NR	Covariate in multi- variable model	Cat-10
Ray 2004 (80)	Summary CV Risk Score	Cohort	481,744	Cyclic anti- depressants	Sudden cardiac deaths	1,487	Rate ratio	Poisson (unexposed)	NR	Covariate in multi- variable model	Cat-10
Ray 2002 (81)	Summary R isk Score	Cohort	69,314	NSAIDS	Serious CHD	6,362	Rate ratio	Poisson (unexposed)	NR	Covariate in multi- variable model	NR
Ray 2009 (82)	Summary CV risk score	Cohort	279,900	Atypical antipsychotics	Sudden cardiac deaths	1,870	Incidence rate-ratio	Poisson (unexposed)	NR	Covariate in multi- variable model	Cat-20
Ray 2007 (83)	Baseline s ummary medical comorbidity score	Cohort	76,637	NSAIDs w protective cotherapy vs. coxibs	Peptic ulcer h ospitalization	1,223	Rate ratio	Poisson (former users with no gastroprotective therapy)	NR	Covariate in multi-v ariable model	Cat-10
Ray 2009 (84)	CV Risk Score	Cohort	48,566	NSAIDs	CV risk	3,600	Rate ratio	Poisson (noncurrent users)	NR	Covariate in multi- variable model	Cat– 20
Read 1983 (85)	Multivariate c onfounder score	Cohort	8,527	Clinical setting	Antenatal diagnostic procedures	3,786	Proportion	Multivariate least square regression	45	Stratified	Cat-4
Rosenberg 1982 (86)	Multivariate score	Case-control	1,447	Aspirin	Myocardial infarction	551	Odds ratio	Logistic (unexposed group)	17	Stratified	Cat-5
Rothman 1980 (87)	Summary confounder score	Case-control	17,099	Maternal age and birth rank	Breast Cancer	4,339	Relative risk	NR	11	Stratified	Cat – 5
Roumie 2008(88)	Vascular risk score	Cohort	336,906	NSAIDS	Stroke	4,354	Hazard ratio	Cox (non-users)	22	Covariate in multi- variable model	Cat-10
Roumie 2009 (89)	Summary risk score	Cohort	610, 001	NSAIDS	CV events	22,432	Hazard ratio	Cox (non-users)	24	Covariate in multi- variable model	Cat-10
Salonen 1985 (90)	Multivariate confounder score	Cohort	102	Health education program	Smoking cessation after myocardial infarction	25	Rate ratio	Logistic	NR	Stratified	Cat-3
Schachter 1982 (91)	Confounder summarization score	Case-control	883	Chlamydia trachomatis	Cervical neoplasia	383	Odds ratio	Logistic (whole population)	NR	Stratified	Cat-4
Scholer 1999 (92)	Risk score	Cohort	18,768,162 infant years	Socio-demographic factors	Infant Injury deaths	5,963	Rates	Point system based on adjusted stratum calculated by Poisson	5	Stratified	Cat-5
Shore 1979 (93)	Multivariate confounder score	Case-control	322	Hair Dye	Breast cancer	129	Relative risk	Discriminate	14	Stratified	Cat–6
Singer 1989 (94)	Multivariate baseline risk scores	Cohort	424	Diabetes	Mortality after myocardial infarction	94	Relative risk	Logistic (unexposed )	3	Stratified	Cat-3
Siu 1979 (95)	Multivariate c onfounding score	Cohort	433	Post-operative r adiotherapy	Mortality	NR	Odds ratio	Discriminant	11	Stratified	Cat-3
Solomon 2006 (96)	CV risk score	Cohort	98,370	NSAIDS	CV events	4,850	Relative risk	Cox (nonusers)	23	Stratified	Cat– 2
Stason 1976 (97)	Multivariate confounder summarizing score	Case-control	2,885	Alcohol consumption	Nonfatal myocardial infarction	399	Rate ratio	Linear discriminant (whole cohort)	20	Stratified	Cat-5
Strauss 1997 (98)	Multivariate c onfounder score	Cohort	4291	Tube feeding	Mortality	612	Rate ratio	Logistic (unexposed)	NR	Stratified	Cat-8
Strauss 1996 (99)	Multivariate c onfounder score	Cohort	7,241	Institutional placement vs. community living	Mortality	1,330	Mortality rates	Logistic (unexposed)	10	Stratified	Cat-8
Swan 1981 (100)	Multivariate score	Case-control	285	Oral contraceptives	Cervical carcinoma	69	Odds ratio	Linear discriminant	NR	Stratified	Cat-3
van Rossum 2001 (101)	Multivariate risk score	Cohort	19,019	Season	Death	8,347	Rate ratios	Logistic	NA	Stratified	Cat-3
van Staa 2008 (102)	Disease risk score	Cohort	1,172,341	NSAID	Myocardial infarction	31,019	Rate ratios	Poisson (controls)	21	Matched	Cat-10
van Staa 2001 (103)	Miettinen's multivariate confounder score	Cohort	244 235	Oral corticosteroid	Fracture	NR	Incidence rates	Logistic (whole cohort)	27	Stratified	Cat-5
Welsh 2008 (104)	Multivariate confounder score	Case-control	2464	Genetic variation in the histidase gene	Skin cancer (BCC, SCC)	SCC, n=702 BCC, n=914	Odds ratios	Logistic	5	Covariate in multi- variable model	Cat- 3
Wynder 1979 (105)	Miettinen confounder score method	Case-control	10,581	Filter cigarette usage	Lung and larynx cancer	1,034	Odds ratio	NR	NR	NR	NR

BCC=Basal Cell Carcinoma, Cat=Categorical, CHD=Coronary Heart Disease, M-H=Mantel-Haenszel, MI=Myocardial Infarction, NR=Not Reported, PY=Person Years, SCC=Squamous Cell Carcinoma, Var=Variables

DRS methods were not clearly reported in up to 15 percent of empirical papers, Table 1. Of the empirical papers reporting derivation methods, logistic regression (47%), followed by discriminant analysis (17%) where the most common, with a shift away from discriminant analysis and no study using this method since 2000. The majority of papers did not specify the cohort used to derive DRS (70%). Of the 26 papers with methodological detail, 85% created DRS in an “unexposed” or subgroup. DRS were most commonly used as a categorical variable (93%), with a shift from stratification prior to 2000 (89% of applications published before January 2000) to use as a covariate in regression models (63% of applications) since January 2000. The most common number of groups were 3 (28%), 5 (28%), 4 (19%), and 10 (18%).

DISCUSSION

We examined use of DRS as a confounder summary score method over time and identified a bimodal distribution in DRS application with a peak 1979-1980 and resurgence since 2000. This bimodal distribution is not surprising given early simulation efforts. In 1976, Miettinen proposed the creation of a ‘multivariate confounder score’ in the unexposed group to be applied in the full cohort to examine exposure effects adjusted for confounding variables.⁵ However, early simulation work by Pike in 1979 concluded that the DRS method may overestimate the effect of confounders and thus bias results.¹⁰ A subsequent simulation by Cook and Goldman published in 1989 concluded that overestimation of confounders may be rare, particularly when applying the DRS as a categorical variable.¹¹ These results likely rejuvenated interest and confidence in the methodology. The increase since 2000 may also partially relate to the recent increased demand and continued importance of comparative effectiveness research, particularly in the area of pharmacoepidemiology,^9,109,110 with 46 percent of DRS application papers related to drug safety and effectiveness since 2000.

Our results show great variation in DRS application with differences in methods of score derivation, utilization, and naming. The most common method of DRS derivation was originally discriminant analysis and since 2000, has been logistic regression. This follows trends of analyses seen in the last 40 years of health science research with older studies utilizing discriminant analysis, and more recent publications utilizing logistic and other regression analyses. Miettinen's original derivation of the DRS used a discriminant function and discussed Cox, logistic, and linear models as possible alternatives.⁵

About half (42%) of empirical studies applied DRS in a case-control study, however only two methodological contributions have examined DRS using a case-control study design.^10,12 Pike completed his simulation work that halted wide uptake of the DRS using a case-control study design,¹⁰ and a recent simulation published in abstract form concluded that the DRS may be appropriate in the case-control study when exposure is not highly correlated with its confounders.¹² Use of the propensity score in a case-control has been shown to introduce artificial effect modification and reduce control of confounding.¹¹¹ Further methodological work is needed to support DRS utilization in the case-control setting.

DRS were most commonly used as a categorical variable to control for confounding in the main outcome model. The number of categories varied, with 3 (28%) and 5 (28%) groups being the most common. Miettinen recommended that the initial analysis be completed with equal deciles and then adjacent strata combined to create five strata.⁵ In many cases, the most clinically relevant number of groups may be three with risk stratified into low, medium and high. However, few studies used DRS to communicate results by disease strata (23%). We believe this to be an underutilization of DRS benefits. The added advantage of graphical presentation may allow for easier communication of results and identify effect modification by baseline outcome risk. Stratifying results by disease risk strata may be particularly beneficial in drug effects studies to maximize the benefits and limit harms in patients. Oral bisphosphonates, as an example, are indicated to treat osteoporosis and reduce fracture risk among patients with low bone mineral density and/or major risk factors for fracture.¹¹³ Treating patients at low fracture risk may increase potential harms, with little benefit on fracture risk reduction. Prior evidence identifies little difference in fracture risk reduction between osteoporosis therapies among patients in low risk strata.^7,114 Interestingly, few studies used matching on the DRS as the means to implement DRS, yet matching on EPS is common.^3,4,9 In the presence of heterogeneous treatment effects, matching on the DRS or examining risk by DRS strata after adjusting for DRS in the regression, allows investigators to estimate exposure effects in well-defined populations and examine effect modification by disease (outcome) risk. Once treatment effect heterogeneity has been described, standardization methods such as matching on the DRS, stratifying the Cox proportional hazards model on DRS strata, or another method to adjust for the observed interaction in the regression model may be used if an overall treatment effect estimate is needed.

Our results show that, where reported, DRS were most commonly derived in a subgroup of the study population and then applied to the study population at large. These applications are similar to the recommendations made by Miettinen and Cook et al..^5,11 These authors argued the importance of creating the score in the unexposed group so that exposure does not bias the underlying risk of outcome. However, more recent empirical and simulation work has identified that DRS creation in the full cohort may be important in settings where exposure is highly correlated with covariates.^7,8 In other settings, such as in the context of new therapeutic agents, deriving DRS in an external historical cohort may be advantageous.¹¹⁵ Further research is needed to understand the relative advantages and disadvantages of different DRS approaches.

Our systematic review is subject to some limitations. First, although we completed a 3-step search, we recognize that due to the lack of standardized terminology to describe DRS methods, we may have missed some relevant applications. Indeed, upon discussion of our review with colleagues, five additional DRS applications were identified that were not found through our comprehensive search – three were early applications,^13-15 and two described DRS as “propensity score” for the outcome.^116,117 Although “propensity score” for the outcome is technically correct, this terminology makes the identification and interpretation of the DRS more challenging. Propensity Score became an official MeSH keyword heading in 2010, defined as the conditional probability of exposure to a treatment given observed covariates. As DRS applications increase, standardized terminology is recommended. Descriptions such as the “multivariate risk score” and “confounder score” may be confusing and vague, and therefore we encourage adoption of the recent terminology Disease Risk Score. Disease risk score is descriptive of the technique and unique versus propensity score and may thus minimize possible confusion between these two confounder summary score methods. Despite the limitations of our search strategy, it was interesting to note that all three search strategies (keyword, citation and author) captured different studies with only 18 of the 86 studies identified by two methods, and no paper identified by all three search methods. The citation search found the largest number of papers, 73 of the 86 papers. Despite potentially missing some applications, we feel our results and conclusions of the general trends would remain.

Second, given the lack of transparency or detail in how DRS was derived or applied, accurate description of some applications was difficult and supports our recommendation for improved transparency and a move toward standardized terminology in future applications. The great variation in the utilization of DRS highlights the need for further work. Additional simulation work is important to support the best means to utilize DRS, particularly in the case-control setting. Finally, although we summarized derivation methods, we did not consider the process of variable selection or its appropriateness. Considerations for variable selection in DRS are similar to those for EPS and a conventional multivariable regression strategy -- variables measured before the start of exposure are risk factors for the outcome of interest, i.e., are potential confounding variables.^118-120

In summary, we identified an increase in DRS application, yet underutilization of DRS to examine potential effect modification by disease risk. Comparative safety and effectiveness research may benefit from DRS to help target interventions to those who benefit most. However, more work is needed to guide DRS applications, particularly in the case-control setting. A move towards better transparency in DRS derivation and utilization, and standardization of terminology will facilitate DRS application and interpretation. We recommend that future work consider utilizing the terminology Disease Risk Score when describing confounder summary scores derived based on the primary outcome model.

Key Points (5 max).

• Disease Risk Scores (DRS) are confounder summary scores derived based on the probability of disease outcome and may be advantageous over other confounding adjustment techniques when exposure is rare, to study multiple exposures, and to study effect modification by outcome risk.

• Use of DRS confounder summary methods has increased, yet remains low. We observed a bimodal distribution in the number of publications over time, with a peak 1979-1980, and resurgence since 2000. Close to half of empirical applications since 2000 have been associated with pharmacoepidemiology.

• Great variation in DRS application exist with differences in methods of score derivation, utilization, and naming.

• There is a general lack of transparency in methods used to derive and apply DRS methods, and few studies used DRS to its full potential by examining effect modification by outcome risk.

• A move toward standardized terminology and providing methodological detail will facilitate DRS utilization and interpretation. We recommend that future work consider adopting the terminology Disease Risk Score when applying confounder summary scores derived based on the probability of disease outcome.