Skip to main content
. Author manuscript; available in PMC: 2017 Oct 1.
Published in final edited form as: J Am Geriatr Soc. 2016 Aug 22;64(10):2065–2073. doi: 10.1111/jgs.14253

Table 3. Considerations and Recommendations for Propensity Score Analysis.

Item Steps Considerations and Recommendations
1 Estimation of PS
1.1 Identify potential confounders

  1. Use subject-matter knowledge; do not rely on statistical criteria alone (e.g., p-value).

  2. Instrumental variables and intermediate variables should not be included in the PS model.
1.2 Estimate PS

  1. Logistic model is typically used; other machine learning techniques can be used alternatively.

  2. Model continuous variables in more flexible terms (e.g., multiple categories or splines).

  3. Consider clinically plausible interactions (e.g., interaction between confounders and time period).

  4. Avoid modeling strategies to build a best prediction model for treatment status; the main purpose of PS is to balance potential confounders between treated and untreated patients.
1.3 Evaluate PS model

  1. Use standardized difference, not significance testing (e.g., p-value), to assess balance in potential confounders between treated and untreated groups with similar PS values.

  2. Examine the distribution of PS between treated and untreated groups to assess the extent of overlap.

  3. C statistics do not inform whether PS models are correctly specified or include all confounders.
2 Estimation of treatment effect *
2.1 PS matching
2.1.1 Match untreated patient to
treated patient based on PS

  1. Matching algorithm: nearest neighbor matching vs. optimal matching, 1:k (k≥1) matching ratio, caliper (maximum difference in PS allowed within a matched pair), and matching with or without replacement.

  2. Choose the matching algorithm that results in the best covariate balance between the treatment groups.
2.1.2 Evaluate balance in potential
confounders

  1. Use standardized difference, not significance testing (e.g. p-value), to assess covariate balance in the PS-matched sample.
2.1.3 Estimate treatment effect

  1. Consider using appropriate statistical methods for matched data (e.g., paired t-test and McNemar test).

  2. When one is interested in estimating the treatment effect at the population level instead of individual matched-pair level, simply analyzing data ignoring matching process is acceptable.
2.2 PS weighting
2.2.1 Estimate PS-based weights

  1. Present the distribution of weights; pay attention to extreme weights.

  2. If extreme weights are present, check misspecification of PS logistic model; consider alternative methods for weight estimation or trimming to minimize the influence of extreme weights.

  3. Consider stabilized weight to improve precision of treatment effect estimate.
2.2.2 Estimate treatment effect

  1. Use a weighted regression to estimate the treatment effect.
2.3 PS stratification
2.3.1 Create PS strata

  1. Typically, 5 strata are used; increased number of PS strata leads to larger bias reduction.
2.3.2 Evaluate balance in potential
confounders

  1. Use standardized difference, not significance testing (e.g. p-value), to assess confounder balance within each PS stratum.
2.3.3 Estimate treatment effect

  1. Estimate the stratum-specific treatment effect. Lack of constant treatment effect across strata indicates true treatment effect variation by PS or residual confounding within the stratum.

  2. Calculate a weighted average of stratum-specific treatment effects if there is constant treatment effect across strata.

Abbreviations: ATE, average treatment effect; ATT, average treatment effect in the treated; PS, propensity score.

*

Covariate adjustment is omitted because it is no longer considered a best practice.