Abstract
BACKGROUND:
Cardiac magnetic resonance imaging (MRI) is a non-invasive technique used to accurately and reproducibly measure biological parameters such as left ventricular mass. However, some subjects either refuse or are unable to complete testing, and the impact of excluding these missing data from predictive models is unknown.
METHODS:
Multiple imputation was applied to cardiac MRI data that were previously analyzed using a complete case approach. The model variables – 10 traditional cardiovascular risk factors and five sociodemographic variables – were used as a basis for imputation. Men and women were imputed separately. The primary focus was assessing the change in the cardiovascular predictors of left ventricular geometry and systolic function.
RESULTS:
Although 27% of participants were missing cardiac MRI data, multiple imputation returned results similar to those of a complete case analysis. These results were robust to the point of including additional variables in the imputation analysis above and beyond the model variables. The degree of variance explained by the models increased marginally but the statistical inference was altered for only two predictors out of 53 cardiovascular risk factors using multiple imputation.
DISCUSSION:
The results suggest that the cardiac MRI data in the Multi-Ethnic Study of Atherosclerosis (MESA) do not substantively change when missing data are handled using multiple imputation. Future analyses of cardiac MRI data may consider the complete case approach to be adequate despite the high rate of missing data in this population.
Keywords: Comparison of methods, Complete case, Magnetic resonance imaging, Multiple imputation
Abstract
HISTORIQUE :
L’imagerie par résonance magnétique (IRM) est une technique non effractive utilisée pour mesurer de façon précise et reproductible certains paramètres biologiques, tels que la masse ventriculaire gauche. Or, il arrive que des patients refusent ou sont incapables de mener le test à terme et on ignore quel est l’impact de l’exclusion de ces données manquantes des modèles prédictifs.
MÉTHODES :
Une technique d’imputation multiple a été appliquée aux données d’IRM cardiaques préalablement analysées à l’aide d’une approche par ensembles complets de données. Les variables du modèle, dix facteurs de risque cardiovasculaires classiques et cinq variables sociodémographiques, ont servi de base pour l’imputation. Les hommes et les femmes ont fait l’objet d’imputations distinctes. L’objectif principal était d’évaluer le changement des prédicteurs cardiovasculaires de la géométrie et de la fonction systolique ventriculaires gauches.
RÉSULTATS :
Bien que 27 % des participants aient présenté des données d’IRM cardiaque partielles, l’imputation multiple a généré des résultats semblables à ceux des sujets dont l’analyse des données était complète. Ces résultats se sont révélés robustes au point d’inclure des variables additionnelles dans l’analyse d’imputation en plus des variables du modèle. Le degré de variance expliqué par les modèles a augmenté accessoirement, mais l’inférence statistique a été modifiée pour seulement deux prédicteurs sur 53 facteurs de risque cardiovasculaires à l’aide de l’imputation multiple.
DISCUSSION :
Les résultats donnent à penser que les données d’IRM cardiaque de l’étude MESA (Multi-Ethnic Study of Atherosclerosis) ne changent pas substantiellement lorsque les données manquantes sont manipulées par imputation multiple. Les futures analyses des données d’IRM cardiaque pourraient considérer que l’approche par cas complet serait adéquate malgré le taux élevé de données manquantes dans cette population.
Cardiac magnetic resonance imaging (MRI) is a noninvasive technique used to accurately and reproducibly measure biological parameters such as left ventricular (LV) mass (1). The Multi-Ethnic Study of Atherosclerosis (MESA) (2) described the first large-scale application of cardiac MRI in a multiethnic cohort study.
However, a nontrivial number of participants did not undergo an MRI due to reasons such as claustrophobia, metal objects in the body or refusal to undergo the procedure (1). Of the 6814 MESA participants, 5004 (73%) completed the cardiac MRI procedure and had technically adequate data. This level of missing data is above the threshold in which missing data in other medical tests have been shown to introduce biased results (3). Therefore, it is important to assess whether the missing data have an impact on the results obtained from cardiac MRI data.
In cases of an important amount of missing data, the most robust current approach is multiple imputation, which tends to outperform alternative methods (4–7). However, in some cases, the reasons for missing data may be unrelated to the study outcome or the predictors of the outcome. In these cases, in which the missing data are completely random, less sophisticated approaches, such as the complete cases analysis, may be equally valid (5).
The goal of the present study was to compare the results of the test for the association of traditional cardiovascular risk factors with cardiac MRI results with and without the use of multiple imputation to account for missing data.
METHODS
MESA was a longitudinal, population-based study of 6814 men and women 45 to 84 years of age, from four distinct ethnic groups (African American, Asian, Caucasian and Hispanic). These participants, all without a clinically recognized cardiovascular disease (CVD) at baseline, underwent evaluations of demographics, risk factors and degree of subclinical CVD (2). The MESA study intended to detect risk factors that predict the transition from subclinical CVD to overt CVD in a multiethnic cohort.
The study was based on all 6814 MESA participants at baseline, of whom, 5004 had documented cardiac MRI data. Of the 5004 participants with cardiac MRI data, 4888 (98%) had no missing information on cardiac risk factors and were therefore suitable for a complete case analysis. There were no partially missing cardiac MRI data and all participants with any cardiac MRI measure had all measures available. The number of participants with missing data other than MRI data was so low (2%) that all missing data techniques should provide equivalent estimates (3). The cohort analyzed in the present study was previously described (1), as were the imaging techniques used (8).
To make the present study’s results as comparable as possible to previous work, the models used by Heckbert et al (1), who considered a complete case approach to cardiac MRI data, were used. Heckbert et al estimated the demographic and cardiovascular predictors of LV geometry and function. Comparing results of the present study directly with those of this previously published paper enabled a contrast to be made between the results obtained from multiple imputation and the previous work. All MRI outcome data for LV mass, LV end-diastolic volume, LV stroke volume, LV ejection fraction and cardiac output were assessed for normality to determine whether a transformation of the data was required to support using a multivariate normal approach to the imputation.
All models of LV geometry and function were estimated using linear regression, using the same approach as in the initial report by Heckbert et al (1). Imputation was conducted separately by sex because there was some evidence of effect modification. In imputing each model by sex, all interactions were implicitly included by sex into the imputation model. The inclusion of effect modification was rejected by Heckbert et al (1) after a Bonferroni correction for multiple testing (9). However, over-specification of an imputation model may improve parameter estimates (10) and, in general, it is common to use a more richly specified imputation model than the final analytical model (7,10).
The primary multiple imputation was considered to be based on the same variables used as possible predictors of the MRI outcome data, which included the following sociodemographic variables and cardiovascular risk factors: age, sex, race/ethnicity, clinic site, height, systolic blood pressure, diastolic blood pressure, current smoking, alcohol use, exercise, body mass index, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, impaired fasting glucose and diabetes. An expanded multiple imputation model was then considered, which also included family history of heart attack or stroke, high school education or less, the Framingham risk score for heart disease, waist circumference, serum and urine creatinine, blood glucose, heart rate, urine albumin, triglycerides, common carotid intima-media thickness, an emotional and social support index, and the Spielberger trait anger (11) and anxiety (13) scales. Finally, all classes of medications (n=45) used at baseline by at least 1% of the study population were considered in the imputation model.
The multiple imputation was performed using SAS version 9.1.3 (SAS Institute Inc, USA). Mixed-chain imputation was used with 1000 burn-in iterations and the Markov chain Monte Carlo option. Time plots and autoregression plots were assessed as diagnostic checks. Results were imputed separately for men and women, and included all variables (including outcome variables) in the imputation step (13). Ten imputed datasets were used to ensure that the effect estimates were not overly inaccurate due to Monte Carlo variability (14).
RESULTS
Of the 1810 participants with missing cardiac MRI data, data were missing due to ineligibility (28%; usually because of metallic fragment, implant or device), inability (55%; usually because of claustrophobia), refusal (12%), mechanical problem with the scanner (2%) or unknown (4%) (1).
The distribution of risk factors suggested that the participants who did not complete a cardiac MRI scan were at higher cardiovascular risk (Table 1). The results for the mean of the cardiac MRI variables were similar between the complete case and multiple imputation approaches (Table 2) with less than a 10% difference between the approaches for almost all estimates. This similarity continued even when the number of variables used in the imputation was dramatically increased. Small differences were seen in LV mass in men when the results were pooled by sex and no longer considered effect modification by sex in the imputation.
TABLE 1.
Estimates of cardiovascular risk factors based on cardiac magnetic resonance imaging (MRI): Results from the Multi-Ethnic Study of Atherosclerosis (MESA) from 2000 to 2002
| Predictors of LV function | Cardiac MRI performed (n=5004) | Cardiac MRI data missing (n=1810) | P* |
|---|---|---|---|
| Cardiovascular variables | |||
| Systolic BP, mmHg | 125.4 | 129.8 | <0.0001 |
| Diastolic BP, mmHg | 71.8 | 72.2 | 0.2486 |
| Antihypertensive drug use, % | 35.3 | 41.7 | <0.0001 |
| Current smoking, % | 12.7 | 14.0 | 0.1630 |
| Alcohol intake, drinks/week | 3.7 | 4.1 | 0.6592 |
| Exercise, MET h/week | 26.7 | 23.6 | 0.0041 |
| Body mass index, kg/m2 | 27.7 | 30.0 | <0.0001 |
| LDL cholesterol, mmol/L | 3.00 | 3.00 | 0.9029 |
| HDL cholesterol, mmol/L | 1.31 | 1.29 | 0.0197 |
| Lipid-lowering drug use, % | 15.9 | 16.8 | 0.3840 |
| Impaired fasting glucose, % | 26.7 | 31.0 | 0.0004 |
| Diabetes, % | 12.9 | 17.8 | <0.0001 |
| Demographic variables | |||
| Age, years | 61.5 | 63.9 | <0.0001 |
| Male sex, % | 47.6 | 45.9 | 0.2170 |
| Less than high school education, % | 34.6 | 41.7 | <0.0001 |
| Spielberger trait anger scale | 14.8 | 14.7 | 0.2585 |
| Spielberger trait anxiety scale | 15.9 | 15.8 | 0.5979 |
T test for equality of means for continuous variables and χ2 test for proportions between those with and without missing data. BP Blood pressure; HDL High-density lipoprotein; LDL Low-density lipoprotein; LV Left ventricular; MET Metabolic equivalent of task
TABLE 2.
Means for cardiac magnetic resonance imaging variables for men and women at baseline in the Multi-Ethnic Study of Atherosclerosis (MESA) from 2000 to 2002
|
Men |
Women |
|||||||
|---|---|---|---|---|---|---|---|---|
| Complete case (n=2382) | Multiple imputation 1 (n=831) | Multiple imputation 2 (n=831) | Multiple imputation 3 (n=831) | Complete case (n=2622) | Multiple imputation 1 (n=979) | Multiple imputation 2 (n=979) | Multiple imputation 3 – pooled (n=979) | |
| LV mass, g | 168.8 | 175.6 | 176.3 | 173.9 | 123.8 | 130.9 | 130.0 | 131.5 |
| LV end-diastolic volume, mL | 140.3 | 140.8 | 141.4 | 140.9 | 113.9 | 116.1 | 115.6 | 116.4 |
| LV stroke volume, mL | 92.7 | 92.6 | 92.8 | 92.9 | 80.8 | 82.5 | 81.9 | 82.7 |
| LV ejection fraction, % | 66.6 | 66.3 | 66.2 | 66.5 | 71.3 | 71.4 | 71.2 | 71.4 |
| Cardiac output, L/min | 5.97 | 6.00 | 6.09 | 6.04 | 5.45 | 5.56 | 5.61 | 5.60 |
Comparison between complete case analysis and three examples of multiple imputation (1 – model variables only; 2 – model variables, subclinical disease measures and medication information; 3 – pooling the men and women into the same imputation model). LV Left ventricular
The effect of imputation on the predictors of LV geometry and function was small (Tables 3 and 4). There was a small increase in the amount of variance explained by the model for four of the five measures of LV geometry and function. Of the 53 predictors tested across the five outcome variables, only two changes in inference occurred when missing data were handled using multiple imputation (diastolic blood pressure was no longer significantly associated with LV mass and systolic blood pressure squared became a significant predictor of LV ejection fraction). Both changes in inference were in marginal predictors and the point estimates were similar (Tables 3 and 4).
TABLE 3.
Multivariable analysis of traditional cardiovascular risk factors in relation to left ventricular (LV) mass and volume
| Predictors of LV mass |
LV mass, g |
LV end-diastolic volume, mL |
LV stroke volume, mL |
|||
|---|---|---|---|---|---|---|
| Complete case | Multiple imputation* | Complete case | Multiple imputation* | Complete case | Multiple imputation* | |
| Model R2, % | 59.6 | 60.3 | 43.8 | 44.5 | 34.9 | 36.0 |
| Systolic blood pressure, per 21 mmHg | 9.57 (8.44 to 10.70) | 9.69 (8.51 to 10.87) | 6.35 (5.29 to 7.42) | 6.23 (5.12 to 7.36) | 5.46 (4.73 to 6.18) | 5.43 (4.65 to 6.21) |
| Diastolic blood pressure, per 10 mmHg | −1.08 (−2.15 to −0.02) | −0.88 (−2.06 to 0.30) | −4.21 (−5.22 to −3.20) | −4.00 (−5.14 to −2.86) | −4.17 (−4.85 to −3.48) | −4.09 (−4.86 to −3.32) |
| Current smoking | 7.73 (5.55 to 9.92) | 7.55 (5.36 to 9.76) | 0.86 (−1.21 to 2.92) | 0.72 (−1.30 to 2.74) | −1.89 (−3.28 to −0.49) | −1.79 (−3.11 to −0.47) |
| Alcohol intake, per 9 drinks/week | 0.46 (−0.32 to 1.24) | 0.51 (−0.24 to 1.26) | −0.41 (−1.15 to 0.33) | −0.31 (−1.19 to 0.57) | −0.24 (−0.74 to 0.26) | −0.09 (−0.64 to 0.47) |
| Exercise, per 27 MET h/week | 0.93 (0.43 to 1.42) | 0.88 (0.28 to 1.47) | 0.64 (0.18 to 1.11) | 0.64 (0.18 to 1.09) | 0.51 (0.20 to 0.83) | 0.49 (0.16 to 0.83) |
| Body mass index, per 5 kg/m2 | 11.87 (11.03 to 12.71) | 11.51 (10.71 to 12.31) | 8.49 (7.69 to 9.28) | 8.43 (7.74 to 9.14) | 5.92 (5.39 to 6.46) | 5.88 (5.38 to 6.39) |
| LDL cholesterol, per 0.79 mmol/L | −1.13 (−1.83 to −0.42) | −1.07 (−1.86 to −0.29) | −0.89 (−1.55 to −0.22) | −0.92 (−1.62 to −0.22) | −0.63 (−1.09 to −0.18) | −0.68 (−1.11 to −0.25) |
| HDL cholesterol, per 0.38 mmol/L | 0.14 (−0.69 to 0.97) | 0.1 (−0.80 to 1.01) | 1.79 (1.01 to 2.58) | 1.84 (1.04 to 2.63) | 1.12 (0.59 to 1.65) | 1.17 (0.62 to 1.71) |
| Impaired fasting glucose (versus normoglycemic) | −1.25 (−2.96 to 0.46) | −0.8 (−2.42 to 0.83) | −3.85 (−5.47 to −2.23) | −3.74 (−5.43 to −2.05) | −2.44 (−3.54 to −1.35) | −2.42 (−3.66 to −1.19) |
| Diabetes (versus normoglycemic) | 3.44 (1.12 to 5.75) | 3.91 (1.53 to 6.30) | −4.65 (−6.83 to −2.47) | −4.76 (−7.31 to −2.21) | −4.57 (−6.05 to −3.09) | −4.53 (−6.17 to −2.90) |
Comparison of estimates (95% CIs) derived using multiple imputation versus complete case analysis to handle missing data.
This multiple imputation included all of the predictors described in Table 1. HDL High-density lipoprotein; LDL Low-density lipoprotein; MET Metabolic equivalent of task
Table 4.
Multivariable analysis of traditional cardiovascular risk factors in relation to left ventricular (LV) ejection fraction and cardiac output;
| Predictors of LV mass |
LV ejection fraction, % |
Cardiac output, L/min |
||
|---|---|---|---|---|
| Complete case | Multiple imputation | Complete case | Multiple imputation | |
| Model R2, % | 17.8 | 17.8 | 22.3 | 23.6 |
| Systolic blood pressure, per 21 mmHg | 2.71 (0.90 to 4.53) | 2.86 (1.03 to 4.69) | 0.78 (0.46 to 1.11) | 0.79 (0.45 to 1.13) |
| Systolic blood pressure, per 21 mmHg, squared | −0.13 (−0.27 to 0.01) | −0.14 (−0.27 to −0.00) | −0.04 (−0.07 to −0.02) | −0.04 (−0.07 to −0.02) |
| Diastolic blood pressure, per 10 mmHg | 2.63 (0.65 to 4.61) | 2.17 (0.17 to 4.18) | −0.05 (−0.11 to 0.01) | −0.04 (−0.11 to 0.01) |
| Diastolic blood pressure, per 10 mmHg, squared | −0.26 (−0.39 to −0.12) | −0.23 (−0.36 to −0.09) | ||
| Current smoking | −1.56 (−2.14 to −0.97) | −1.41 (−2.00 to −0.83) | −0.14 (−0.25 to −0.02) | −0.13 (−0.23 to −0.02) |
| Alcohol intake, per 9 drinks/week | 0.02 (−0.19 to 0.23) | 0.09 (−0.12 to 0.30) | −0.03 (−0.07 to 0.01) | −0.02 (−0.06 to 0.02) |
| Exercise, per 27 MET h/week | 0.02 (−0.11 to 0.16) | 0.01 (−0.12 to 0.15) | 0.00 (−0.03 to 0.02) | 0.00 (−0.03 to 0.02) |
| Body mass index, per 5 kg/m2 | −0.03 (−0.25 to 0.20) | −0.01 (−0.27 to 0.25) | 0.41 (0.37 to 0.46) | 0.41 (0.37 to 0.45) |
| LDL cholesterol, per 0.79 mmol/L | 0.02 (−0.17 to 0.21) | −0.01 (−0.19 to 0.18) | −0.04 (−0.08 to 0.00) | −0.04 (−0.07 to 0.00) |
| HDL cholesterol, per 0.38 mmol/L | −0.04 (−0.27 to 0.18) | −0.03 (−0.25 to 0.19) | 0.07 (0.02 to 0.11) | 0.07 (0.03 to 0.11) |
| Impaired fasting glucose (versus normoglycemic) | 0.20 (−0.25 to 0.66) | 0.16 (−0.30 to 0.61) | 0.02 (−0.07 to 0.11) | 0.01 (−0.09 to 0.12) |
| Diabetes (versus normoglycemic) | −0.86 (−1.47 to −0.23) | −0.78 (−1.40 to −0.16) | 0.11 (−0.01 to 0.24) | 0.13 (−0.01 to 0.26) |
Comparison of estimates (95% CIs) derived using multiple imputation versus complete case analysis to handle missing data. HDL High-density lipoprotein; LDL Low-density lipoprotein; MET Metabolic equivalent of task
DISCUSSION
Multiple imputation is the gold standard approach for handling missing data (5) but it involves dealing with additional analytical complexity. A complete case analysis can be more transparent and has the advantage of relying solely on the data actually observed. In the present example, an analysis using multiple imputation did not change any of the inference around the predictors of LV mass, meaning that the main goal of the previous analysis could be met with a complete case approach (1). It is important to note that the similarity of the estimates was preserved even when the number of variables used in the imputation model was greatly increased compared with the variables used in the statistical models, and different types of variables (eg, psychological) were added.
These findings are important because the covariates in Table 1 suggest that there is a relationship between cardiovascular health and missing cardiac MRI data. If having missing data is also related to LV geometry and function, this may have introduced confounding between the predictors and the outcome. However, as shown in Table 2, the imputed values are quite similar to the measured values and this is verified by the very small changes in the estimates seen in Tables 3 and 4. Verifying this, however, is a key element in confirming the validity of the complete case approach for cardiac MRI data (1).
The results of the present study are similar to those found in the Cardiovascular Health Study (CHS) (15) in which LV mass (measured using electrocardiography) was missing for 35% of participants at baseline. Consistent with the CHS data, the associations in the present study were preserved whether using complete case analysis or multiple imputation as an approach for handling missing data. This lack of difference between approaches is supported by other studies that have found particular substantive examples in which multiple imputation provided similar results to less sophisticated approaches (16–18).
However, it is critical to remember that the results of multiple imputation tend to be less biased than less sophisticated approaches to handling missing data (3–7). Therefore, when the complete case approach and multiple imputation show differences, the estimates from the imputed datasets are likely to be closer to the true values for the parameters (4). Usually, even when the missing data are not random, the estimates obtained from multiple imputation, while still biased, are less biased than other approaches (5). This suggests that the imputed estimates should be used when an important difference is observed and particular substantive areas should be checked to determine whether the approach for handling missing data may have important effects on the results.
The use of psychological scales in the imputation indirectly tested the hypothesis that psychological reasons for refusing a cardiac MRI could be a candidate for a hidden common cause of both missing data and LV geometry or function. However, the inclusion of these additional variables did not result in any advantage in the imputation because it yielded the same results as the imputation based solely on the sociodemo-graphic and cardiovascular risk factors included directly in the model.
Limitations
The present study has a number of important limitations. First, these results assume that there is no unknown or unmeasured variable that predicts both missing data and the outcome. While the estimates were robust to the point of inclusion of many additional candidate variables, it is always possible that none of these variables can serve as a proxy for this unknown variable. However, this is a common assumption of all missing data techniques (13).
Second, all members of the MESA cohort either had subclinical CVD or were disease-free at baseline. It is not clear whether these results can be generalized to populations with more severe CVD. In particular, the participants who were unable to undergo an MRI due to devices (1) may have been at greater risk for the outcome if the device was implanted to treat a clinically important manifestation of CVD.
Finally, it is possible that further differences could emerge in a longitudinal setting as opposed to the cross-sectional setting of the present study. Participants may withdraw from the study for reasons related to refusal to undergo a cardiac MRI; this is especially possible because low-compliers may be more likely to refuse to undergo a procedure and are also more likely to suffer adverse outcomes (19). We may also have observed different results if we had used different exposures to predict the results of the cardiac MRI because there may have been information in other variables that could have improved these estimates. However, attempts to further refine the variables used in the imputation in the present study (such as using finer categories for smoking and education level) did not result in detectable differences in the estimates.
CONCLUSION
The present study provides important confirmatory evidence that the inferences derived from the results of cardiac MRI data are broadly correct when missing data are handled using a complete case approach given the exposure variables considered.
Footnotes
FUNDING: This research was supported by contracts N01-HC-95159 through N01-HC-95165 and N01-HC-95169 from the National Heart, Lung, and Blood Institute. The authors thank the other investigators, the staff and the participants of the MESA study for their valuable contributions. A full list of participating MESA investigators and institutions can be found at <www.mesa-nhlbi.org>.
REFERENCES
- 1.Heckbert SR, Post W, Pearson GD, et al. Traditional cardiovascular risk factors in relation to left ventricular mass, volume, and systolic function by cardiac magnetic resonance imaging: The Multiethnic Study of Atherosclerosis. J Am Coll Cardiol. 2006;48:2285–92. doi: 10.1016/j.jacc.2006.03.072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bild DE, Bluemke DA, Burke GL, et al. Multi-Ethnic Study of Atherosclerosis: Objectives and design. Am J Epidemiol. 2002;156:871–81. doi: 10.1093/aje/kwf113. [DOI] [PubMed] [Google Scholar]
- 3.Barzi F, Woodward M. Imputations of missing values in practice: Results from imputations of serum cholesterol in 28 cohort studies. Am J Epidemiol. 2004;160:34–45. doi: 10.1093/aje/kwh175. [DOI] [PubMed] [Google Scholar]
- 4.Greenland S, Finkle WD. A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol. 1995;142:1255–64. doi: 10.1093/oxfordjournals.aje.a117592. [DOI] [PubMed] [Google Scholar]
- 5.Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychol Methods. 2002;7:147–77. [PubMed] [Google Scholar]
- 6.van der Heijden GJ, Donders AR, Stijnen T, Moons KG. Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: A clinical example. J Clin Epidemiol. 2006;59:1102–9. doi: 10.1016/j.jclinepi.2006.01.015. [DOI] [PubMed] [Google Scholar]
- 7.Rubin DB. Multiple imputation after 18+ years. J Am Stat Assoc. 1996;91:473–89. [Google Scholar]
- 8.Natori S, Lai S, Finn JP, et al. Cardiovascular function in multi-ethnic study of atherosclerosis: Normal values by age, sex, and ethnicity. AJR Am J Roentgenol. 2006;186(6 Suppl 2):S357–65. doi: 10.2214/AJR.04.1868. [DOI] [PubMed] [Google Scholar]
- 9.Bender R, Lange S. Adjusting for multiple testing – when and how? J Clin Epidemiol. 2001;54:343–9. doi: 10.1016/s0895-4356(00)00314-0. [DOI] [PubMed] [Google Scholar]
- 10.Meng XL. Multiple-imputation inferences with uncongenial sources of input (with discussion) Statist Sci. 1994;10:538–73. [Google Scholar]
- 11.Spielberger CD, Johnson EH, Russell SF, Crane RJ, Jacobs GA, Worden TJ. The experience and expression of anger: Construction and validation of an anger expression scale. In: Chesney MA, Rosenman RH, editors. Anger and Hostility in Cardiovascular and Behavioral Medicine. Washington: Hemisphere; 1985. [Google Scholar]
- 12.Spielberger CD. Manual for the State-Trait Anxiety Inventory. Palo Alto: Consulting Psychologists Press; 1983. [Google Scholar]
- 13.Moons KG, Donders RA, Stijnen T, Harrell FE., Jr Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59:1092–101. doi: 10.1016/j.jclinepi.2006.01.009. [DOI] [PubMed] [Google Scholar]
- 14.Schafer JL. Multiple imputation: A primer. Stat Methods Med Res. 1999;8:3–15. doi: 10.1177/096228029900800102. [DOI] [PubMed] [Google Scholar]
- 15.Arnold AM, Kronmal RA. Multiple imputation of baseline data in the Cardiovascular Health Study. Am J Epidemiol. 2003;157:74–84. doi: 10.1093/aje/kwf156. [DOI] [PubMed] [Google Scholar]
- 16.Engels JM, Diehr P. Imputation of missing longitudinal data: A comparison of methods. J Clin Epidemiol. 2003;56:968–76. doi: 10.1016/s0895-4356(03)00170-7. [DOI] [PubMed] [Google Scholar]
- 17.Bono C, Ried LD, Kimberlin C, Vogel B. Missing data on the Center for Epidemiologic Studies Depression Scale: A comparison of 4 imputation techniques. Res Social Adm Pharm. 2007;3:1–27. doi: 10.1016/j.sapharm.2006.04.001. [DOI] [PubMed] [Google Scholar]
- 18.Kang T, Kraft P, Gauderman WJ, Thomas D, Framingham Heart Study Multiple imputation methods for longitudinal blood pressure measurements from the Framingham Heart Study. BMC Genet. 2003;4(Suppl 1):S43. doi: 10.1186/1471-2156-4-S1-S43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Horwitz RI, Horwitz SM. Adherence to treatment and health outcomes. Arch Intern Med. 1993;153:1863–8. [PubMed] [Google Scholar]