Table 5.
Factors associated with the proportion of significant effect sizes
| Model 1 | Model 2 | Model 3 | ||||
|---|---|---|---|---|---|---|
| Average Marginal Effect (AME) | Sample mean | Average Marginal Effect (AME) | Sample mean | Average Marginal Effect (AME) | Sample mean | |
| New studies (August 2015-Feb 2018) | 0.11 (0.13) | 0.32 (0.47) | 0.06 (0.17) | 0.30 (0.46) | 0.05 (0.16) | 0.26 (0.44) |
| Research design | ||||||
| Difference in Differences (DID) | −0.20** (0.08) | 0.43 (0.50) | −0.20** (0.10) | 0.44 (0.50) | − 0.24** (0.10) | 0.52 (0.51) |
| Interrupted Time Series (ITS) | −0.09 (0.10) | 0.12 (0.33) | −0.11 (0.11) | 0.13 (0.34) | −0.67*** (0.08) | 0.09 (0.30) |
| Randomized Control Trial (RCT) | −0.19* (0.10) | 0.41 (0.49) | −0.20* (0.11) | 0.10 (0.30) | −0.34*** (0.12) | 0.14 (0.35) |
| Country | ||||||
| US | −0.11* (0.07) | 0.41 (0.49) | −0.13* (0.07) | 0.41 (0.49) | 0.08 (0.11) | 0.40 (0.50) |
| Setting | ||||||
| Hospital | 0.06 (0.06) | 0.30 (0.46) | 0.05 (0.07) | 0.32 (0.47) | −0.24** (0.09) | 0.33 (0.48) |
| Incentive Schemes | ||||||
| Cost and quality | 0.11* (0.06) | 0.25 (0.43) | 0.13* (0.08) | 0.25 (0.44) | −0.34*** (0.08) | 0.21 (0.41) |
| Rewards for improvement | −0.12* (0.07) | 0.36 (0.48) | −0.12* (0.07) | 0.38 (0.49) | −0.09 (0.06) | 0.36 (0.49) |
| Use of incentives | ||||||
| Discretionary use | 0.01 (0.09) | 0.55 (0.50) | ||||
| Special use | 0.08 (0.12) | 0.06 (0.25) | ||||
| Size of payment to revenue ratio (%) | 0.01* (0.00) | 11.53 (9.18) | ||||
| Publication year fixed effects | Yes | Yes | Yes | |||
| Number of observations | 116 | 109 | 42 | |||
| Number of clusters | 62 | 57 | 25 | |||
| Average number of observations per cluster | 1.9 | 1.9 | 1.7 | |||
| (Min-Max) | (1–15) | (1–14) | (1–4) | |||
| Pseudo-Log | −59.91 | −55.66 | −16.99 | |||
| AIC | 1.33 | 1.37 | 1.57 | |||
| BIC | − 408.42 | −365.18 | − 84.57 | |||
Note: The omitted reference group for “Research Design” is before and after designs, controlled before and after, and case-control studies. The omitted reference group for “Country” is all other countries. The omitted reference group for “Setting” is primary care. The Omitted reference group for “Incentive schemes” is to pay for performance only. The omitted reference group for “Rewards for improvement” is not rewarding for improvement. The Omitted reference group for “Use of incentives” is using incentives for physician income. The standard errors are clustered in the scheme level and are in the parenthesis
∗p < 0.10, ∗ ∗ p < 0.05, ∗ ∗ ∗p < 0.01
AIC Akaike information criterion
BIC Bayesian information criterion