Skip to main content
. Author manuscript; available in PMC: 2018 Nov 28.
Published in final edited form as: JAMA. 2017 Nov 28;318(20):2011–2018. doi: 10.1001/jama.2017.17653

Table 3.

Rate of violations of required research practices for publications in 2015–2016 that used the National Inpatient Sample.

Research
Practice
No.
Required Research
Practices
Overall
(N = 120)
Estimates for the universe of
NIS studies
(N = 1082)
Impact factor <10
(N = 96)
Impact factor ≥10
(N = 24)
P-value*
(Fisher’s
exact
test)
Studies
violating
practice, n (%)
Eligible,
N
Projected
number of
studies
(95% CI)
Projected
percentage
of studies
(95% CI)
Studies
violating
practice, n (%)
Eligible,
N
Studies
violating
practice, n
(%)
Eligible,
N
1 Identifying observations as hospitalization events rather than unique patients 45 (37.5%) 120 437 (334 to 541) 40.4% (30.9 to 50.0) 41 (42.7%) 96 4 (16.7%) 24 0.02
2 Not performing state-level analyses 10 (8.3%) 120 77 (23 to 131) 7.1% (2.1 to 12.1) 7 (7.3%) 96 3 (12.5%) 24 0.42
3 Limiting hospital-level analyses to data from years 1988–2011 2 (11.8%) 17 12 (0 to 33) 8.2% (0.0 to 22.5) 1 (7.7%) 13 1 (25.0%) 4 0.43
4 Not performing physician-level analyses 3 (2.5%) 120 32 (0 to 68) 2.9% (0.0 to 6.2) 3 (3.1%) 96 0 (0.0%) 24 1.0
5 Not using non-specific secondary diagnosis codes to infer in-hospital events 62 (51.7%) 120 588 (484 to 693) 54.4% (44.7 to 64.0) 55 (57.3%) 96 7 (29.7%) 24 0.02
6 Using survey-specific analysis methods that account for clustering, stratification and weighting 79 (65.8%) 120 739 (642 to 837) 68.3% (59.3 to 77.3) 69 (71.9%) 96 10 (41.7%) 24 0.008
7 Accounting for data changes in trend analyses spanning major transition periods in the dataset (1997–1998 and 2011–2012) 21 (77.8%) 27 174 (97 to 251) 79.7% (62.5 to 97.0)§ 16 (80.0%) 20 5 (71.4%) 7 0.63
*

For comparison of studies in journals with impact factor <10 vs ≥10

Unless otherwise specified, results reflect estimated percentage of all 1082 studies using NIS data during the study period with a given violation. Therefore, an estimated 43 studies (95% CI 2 to 85) that correspond to the 5 studies excluded for using the NIS as a secondary source of data, are represented in the denominator for these percentages.

Percentage of the estimated 141 studies performing hospital-level analyses. These studies correspond to the 17 sampled studies that performed these analyses.

§

Percentage of the estimated 218 studies performing analyses spanning major data transitions. These studies correspond to the 17 sampled studies that performed these analyses.

NIS, National Inpatient Sample