Abstract
Background
Medical professionals and students are inadequately trained to respond to rising global obesity and nutrition-related chronic disease epidemics, primarily focusing on cardiovascular disease. Yet, there are no multi-site studies testing evidence-based nutrition education for medical students in preventive cardiology, let alone establishing student dietary and competency patterns.
Methods
Cooking for Health Optimization with Patients (CHOP; NIH NCT03443635) was the first multi-national cohort study using hands-on cooking and nutrition education as preventive cardiology, monitoring and improving student diets and competencies in patient nutrition education. Propensity-score adjusted multivariable regression was augmented by 43 supervised machine learning algorithms to assess students outcomes from UT Health versus the remaining study sites.
Results
3,248 medical trainees from 20 medical centers and colleges met study criteria from 1 August 2012 to 31 December 2017 with 60 (1.49%) being from UTHealth. Compared to the other study sites, trainees from UTHealth were more likely to consume vegetables daily (OR 1.82, 95%CI 1.04-3.17, p=0.035), strongly agree that nutrition assessment should be routine clinical practice (OR 2.43, 95%CI 1.45-4.05, p=0.001), and that providers can improve patients’ health with nutrition education (OR 1.73, 95%CI 1.03-2.91, p=0.038). UTHealth trainees were more likely to have mastered 12 of the 25 competency topics, with the top three being moderate alcohol intake (OR 1.74, 95%CI 0.97-3.11, p=0.062), dietary fats (OR 1.26, 95%CI 0.57-2.80, p=0.568), and calories (OR 1.26, 95%CI 0.70-2.28, p=0.446).
Conclusion
This machine learning-augmented causal inference analysis provides the first results that compare medical students nationally in their diets and competencies in nutrition education, highlighting the results from UTHealth. Additional studies are required to determine which factors in the hands-on cooking and nutrition curriculum for UTHealth and other sites produce optimal student — and, eventually, preventive cardiology — outcomes when they educate patients in those classes.
Keywords: Machine learning, nutrition, medical education, public health, medical student
Introduction
Cardiovascular disease is a serious condition that has a 30% global mortality annually and affects the quality of life of millions of people [1]. Almost half of these cases in the United States can be modified through behavioral determinants of health such as nutrition, physical activity, and smoking cessation [2]. Nutritional interventions, especially implementation of the Mediterranean Diet, have been shown to be effective as low-cost, preventive solutions to modify cardiovascular disease outcomes in multiple systematic reviews [3–9].
Unfortunately, less than half of medical providers in the United States provide nutrition guidance during primary care visits [10]. This can be traced to a lack of education: in one study, 94% of physicians agreed that it is their responsibility to counsel patients on nutrition, while only 14% felt that they were properly trained to do so [11]. This discrepancy begins as early as medical school, where only 19% of students believe they have had enough training to appropriately counsel patients on nutrition [12].
In order to fill this education gap, the Goldring Center for Culinary Medicine (GCCM) at Tulane University School of Medicine has developed the first medical school-based teaching kitchen to provide a quality culinary medicine program that can be replicated around the country. To create an evidence-based curriculum that has been tested in a real-world environment, GCCM worked with over 45 medical schools, colleges, and hospitals in order to start the largest and longest-running cohort trial of culinary medicine curriculum, Cooking for Health Optimization with Patients (CHOP).
CHOP is the very first trial to test the efficacy of hands-on cooking and nutrition education on medical student competency for nutrition counseling. Previous studies were limited in their design as they were either single-center studies [13–16] or lacked adequately powered sample sizes [14–20]. Other studies have had difficulties with data collection, such as loss to long-term follow up [13–15, 17–19] or lack of validated survey tools [13–16]. In order to add to the growing body of evidence for medical nutrition curriculum, CHOP has used an extensive network of institutions to overcome these study weaknesses.
The present study, the CHOP-Medical Students sub-study, is part of the overall CHOP trial. The purpose of this study was to determine if there is a difference in the efficacy of a practical culinary and nutrition curriculum to improve medical student nutrition counseling skills between UTHealth and other participating schools. If a difference is identifiable, then further studies can help determine which factors contribute to these differences to further improve the effectiveness of the culinary nutrition program in increasing patient counseling abilities of medical students.
Methods
Study design
CHOP-Medical Students is a prospective multi-site cohort study design. Any medical student that responded to the validated, electronic survey [21] and attended one of the first twenty collaborating medical schools that implemented the GCCM licensed curriculum, Health Meets Food, between August 1, 2012 to June 26, 2017, was included in the study. Any student that completed more than one survey per cycle each semester or failed to report the number of GCCM classes received, if any, was excluded.
Comparison groups
The standard group was 3,188 medical students from 19 other medical schools across the country who participated in the GCCM nutrition education curriculum. The comparison group was 60 students from UTHealth who participated in the GCCM nutrition education curriculum. The curriculum has a standard of 28 hours of instruction over 8 classes; each divided up by 30 minutes of pre-class lecture video preparation, 30 minutes of case-based learning prompted by the nutrition topics, 90 minutes of hands-on cooking, and 45 minutes of post-class problem-based learning sessions in which students both apply their knowledge to clinical scenarios and eat the food they prepared. This curriculum is different from standard nutrition education in medical schools today as there is no evidence-based nutrition education given within school curricula nationally. Currently there is no specific standard for nutrition in medical education according to the Liaison
Committee on Medical Education (LCME) and a survey of medical schools from 2008 indicated that only 25% have a dedicated nutrition component to medical school education [22].
Data source
A validated Likert scale-based, voluntary survey that was used to assess the competency of medical students on 25 nutrition topics is the primary data source for this study which included a validated MedDiet score [7, 8].
Power analysis
The sample size required to detect an odds ratio (OR) of at least 1.50 with a 95% confidence interval and 80% power assuming an allocation of 0.01 was at least 50 UTH students and 4,000 non-UTH students which was met.
Machine learning and statistical analysis
The first part of the analysis used machine learning within a supervised learning framework to test 43 algorithms with ten-fold cross-validation. The various algorithms were chosen based on data type. Algorithm performance has been assessed to have higher accuracy, lower root relative squared error with model acceptability set at 100% (for comparison among machine learning algorithms), and lower root mean squared error (for comparison to traditional statistical models) [23].
The second part of the analysis used novel integration of three traditional statistical methods. A panel analysis of longitudinal data with triply robust propensity-score (PS) adjusted multilevel mixed effects multivariable regression was conducted. Regression that includes PS as one of the adjusted variables is considered one of the top performing PS methods [24]. Furthermore, the mixed effects method contained both fixed effects and random effects components; allowing for control of time-invariant and time-variant traits, respectively [25]. This type of analysis has been proven in multiple studies to provide causal inference in observational trials [26–30]. Results were reported as fully adjusted odds ratios due to the complex data source and rare disease assumption [31, 32]. Statistical significance was set to a two-tailed p-value < 0.05. Machine learning analysis was performed in R 3.3.2 (The R Foundation for Statistical Computing, Vienna, Austria). Traditional statistical analysis was performed in STATA 14.2 (STATACorp, College Station, Texas, United States of America). Ethics approval was obtained through the Institutional Review Board (IRB) of Tulane University.
Results
Over the five-year study period, 4,026 surveys were completed by 3,248 participants who met the inclusion criteria. The mean age was 25.71+ 2.91 years old, 548 (61.43%) were female, 202 (22.65%) had formal nutrition education prior to medical school, 207 (23.21%) adhered to a special diet, 270 (30.27%) were in their clinical years, and 225 (25.22%) intended to enter a primary care specialty. Additional demographic data is provided in Table 2.
Table 2.
Demographic differences between CHOP - Medical Students and UT Houston students
| All Participants | UT Houston | |
|---|---|---|
| Age (years) | 25.71 ± 2.91 | 25.16 ± 3.28 |
| Number of female participants | 548 (61.43%) | 64 (68.82%) |
| Number of non-white participants | 50 (53.76%) | |
| Number of participants with formal nutrition educations | 202 (22.65%) |
UTHealth students accounted for 60 (1.49%) of the participants in the trial. Of the behavioral aspects of the survey, UT-Houston students were more likely to consume vegetables daily (OR 1.82, 95%CI 1.04-3.17, p=0.035), strongly agree nutrition assessment should be routine clinical practice (OR 2.43, 95%CI 1.45-4.05, p=0.001), and strongly agree that providers can improve patients’ health with nutrition education (OR 1.73, 95%CI 1.03-2.91, p=0.038).
UTHealth students were non-significantly more likely to have mastered 12 of the 25 competency topics with the top three being moderate alcohol intake (OR 1.74, 95%CI 0.97-3.11, p=0.062), dietary fats (OR 1.26, 95%CI 0.57-2.80, p=0.568), and calories (OR 1.26, 95%CI 0.70-2.28, p=0.446). All other variables that were adjusted for the regression analysis are provided in Table 1.
Table 1.
CHOP - Medical Students (N = 4,125 ): Propensity score - adjusted multi - level mixed effects panel analysis of UT Houston medical
| Outcome | OR | 95%CI | P-value | |
|---|---|---|---|---|
| Adherence | High/Medium vs. Low | 1.17 | 0.74-1.86 | 0.499 |
| Olive Oil | 1.66 | 0.83-3.29 | 0.151 | |
| Fruit | 1.35 | 0.85-2.12 | 0.203 | |
| Vegetables | 1.32 | 0.82-2.10 | 0.251 | |
| Vegetables/Fruits | 1.67 | 1.06-2.61 | 0.027 | |
| Legumes | 0.94 | 0.48-1.86 | 0.858 | |
| Seafood | 1.01 | 0.59-1.73 | 0.977 | |
| Alcohol | 1.01 | 0.43-2.38 | 0.985 | |
| Meat | 1.62 | 0.80-3.29 | 0.179 | |
| Whole Grains | 1.48 | 0.94-2.32 | 0.095 | |
| Fruit | 1.35 | 0.85-2.13 | 0.203 | |
| Vegetables | 1.82 | 1.04-3.17 | 0.035 | |
| Baked Goods | 1.74 | 0.92-3.28 | 0.089 | |
| Soft Drinks | 0.79 | 0.40-1.56 | 0.506 | |
| Routine Nutrition Counseling | 2.43 | 1.45-4.05 | <0.001 | |
| Specific Counseling | 1.73 | 1.03-2.91 | 0.038 | |
| MedDiet Impact | 1.84 | 1.17-2.91 | 0.009 | |
| MedDiet | 1.28 | 0.59-2.72 | 0.528 | |
| Dash Diet | 1.54 | 0.59-3.96 | 0.373 | |
| Vegetarian Diet | 1.11 | 0.49-2.46 | 0.807 | |
| Low Fat Diet | 1.83 | 0.76-4.40 | 0.178 | |
| High Protein Diet | 1.56 | 0.70-3.50 | 0.280 | |
| Serving Size | 1.17 | 0.57-2.39 | 0.668 | |
| Moderate Alcohol | 1.74 | 0.97-3.11 | 0.062 | |
| Eating Disorders | 0.99 | 0.54-1.82 | 0.977 | |
| Cholesterol | 1.72 | 0.94-3.14 | 0.077 | |
| Diabetes Diet | 1.01 | 0.51-1.99 | 0.974 | |
| Diabetes Weight Loss | 0.84 | 0.47-1.49 | 0.547 | |
| Obesity Weight Loss | 0.80 | 0.41-1.58 | 0.526 | |
| Omega Fats | 1.07 | 0.48-2.39 | 0.869 | |
| Dietary Fats | 1.26 | 0.57-2.80 | 0.568 | |
| Antioxidants | 1.51 | 0.78-2.94 | 0.223 | |
| Calories | 1.26 | 0.70-2.28 | 0.446 | |
| Hydration | 1.31 | 0.78-2.22 | 0.307 | |
| Celiac | 0.80 | 0.36-1.76 | 0.571 | |
| Food Allergies | 1.25 | 0.61-2.57 | 0.537 | |
| Glycemic Index | 0.96 | 0.38-2.44 | 0.932 | |
| Fiber | 0.96 | 0.48-1.89 | 0.895 | |
| Food Label | 1.35 | 0.84-2.17 | 0.210 | |
| Osteoporosis | 0.94 | 0.46-1.91 | 0.865 | |
| BMI | 1.14 | 0.69-1.88 | 0.605 | |
| Exercise | 1.20 | 0.76-1.89 | 0.442 |
Discussion
CHOP-Medical Students is the most extensive multi-site cohort study of nutrition 140 education and culinary medicine for medical students, including the only study to use machine learning and causal inference statistical analysis for longitudinal assessment.
The initial study showed that the GCCM curriculum improves medical student diets and competency on 25 evidence-based and clinically significant topics of nutrition education [21]. The purpose of this sub-study was to investigate if there are differences between schools that can be used to understand how curriculum implementation can be improved and tailored to individual programs in order to achieve the best outcomes.
UTHealth students were more likely to have superior diets and beliefs about nutrition education in healthcare (but no mastery of the competency in nutrition counseling for patients), suggesting that there is regional and/or institutional variation in the
implementation of the GCCM curriculum that can be studied to achieve optimal outcomes.
Cardiovascular disease in the United States is a clinically and financially costly multifactorial condition that requires a multifactorial treatment plan. Nutrition education has been proven to be an effective, low-cost intervention when treating cardiovascular conditions as a critical adjunct to pharmacological and/or procedural interventions; however, many physicians are not adequately trained to provide nutrition counseling in clinical practice.
GCCM developed a hands-on cooking nutrition counseling education program for medical students in order to provide future physicians the skills to address the nutritional determinants of cardiovascular disease. Studies have proven that this intervention can effectively improve students’ nutrition counseling competency over the standard medical student curriculum [21].
The present study has several strengths including being the largest prospective nutrition education and culinary medicine cohort study for medical trainees, robust use of machine learning-augmented causal inference statistics, and its novel findings that there is a significant variation among medical schools among diets and nutrition beliefs but not competencies. This study suggests the need for such multi-site cohorts in order to optimally inform strategies for personalizing such an evidence-based curriculum for the region and institution to best serve the needs of medical trainees and ultimately their current and future patients. This study does have notable limitations including its smaller sample size for UT-Health as the reference school and non-randomization of the trial.
Further prospective trials are required to validate and expand these results to eventually establish adequate nutrition education among medical trainees, as a sustainable capacity-building strategy to reversing the nutrition related cardiovascular disease epidemic.
Acknowledgements
The authors have no relevant financial disclosures. The authors would like to gratefully acknowledge the dedicated medical school, college, and hospital administrators, community members, medical professionals, and trainees who made this intervention possible to build healthier communities, together. Additionally, the authors are indebted to the remaining CHOP co-investigators: Dr. Darrin D’Agostino, Dr. Keith Argenbright, Dr. Jaclyn Albin, Dr. Milette Siler, Dr. Dennis Muscato, Dr. Louise Muscato, Rachel Bross, Dr. Nadia Aibani, Dr. Holly Johnson, Dr. David Martins, Dr. Barbara Tangel, Dr. Alyse Van Lieu, Dr. Sheryl Harnas, Dr. Ann Andrada, Dr. Eugenia Edmonds, Dr. Jeremy Beer, Dr. Tisha Lunsford, Dr. Margaret Kim, Dr. Irma Hasham, Dr. Debra Stulberg, Dr. Katherine Palmer, Dr. Jennifer Boryk-Ratner, Dr. Tomi Dreilbelbis, Dr. Shannon Kelleher, Dr. Richard Streiffer, Dr. Jennifer Clam, Dr. Jeannine Lawrence, Dr. Linda Knol, Dr. Melanie Tucker, Dr. John Higginbotham, Dr. Joan Han, Dr. Chantis Mantilla, Dr. Patricia Griffin, Dr. Lindsey Grant, Dr. Caroline Compretta, Dr. Loretta Jackson, Dr. Rob Karch, Dr. Rosemarie Lorenzetti, Dr. Konrad Nau, Dr. Mike Finan, Dr. Connie Cooper, Dr. Laurie Macaulay, Dr. Elaine Chen, Dr. John Billimek, Dr. David Kilgore, Dr. Beverly Wilson, Dr. Suzanne Bertollo, Dr. Shreela Sharma, Dr. Laura Moore, Dr. Deanna Hoelscher, Dr. Lawrence Deyton, Dr. Seema Kakar, Dr. Chris D’Adamo, Dr. Deborah Cohen, Louise Merriman, Dr. Kristen Mathieson, Dr. Emily Rosenthal, Dr. Swana De Gisel, Dr. Roma Gianchandani, Dr. Brigid Gregg, Dr. Dianne Habash, Dr. Farzaneh, Dr. Lori Whelan, Jennifer Newton, Dr. Ashley Bunnard, Dr. Tanmeet Sethi, Dr. Ala Shaikhkhalil, Dr. Rebecca Mathews, Dr. Ann Langston, Dr. Carolyn Nichols, Dr. Sharon Moore, Dr. Lisa Hamilton, Dr. Laura Robertson-Boyd, Dr. Leanne Mauriello, Dr. Lora Boroff, Dr. Kate Swanic, Dr. Kelly Fackel, Dr. Jim Schoen, Dr. Josie Elrod, Dr. Rebecca Collins, Jill Gallin, Annie Sanford, Dr. Daniel Fry, Gina DeVito, Dr. Leonard, Walsh, Dr. Evan Siau, Dr. Glenn Mack, Dr. Joan Hendrix, and Dr. Jonathan Woodward. Furthermore, the authors are indebted to those who provided their support, Ms. Elise LeBovidge.
Statement of Contributions
C.M.C. and D.M. designed the research study and analyzed the data. J.W., D.H., A.D., L.S., and T.H. conducted research and provided critical review. A.P. wrote the paper. J.T., H.B., A.N., T.D.N, and A.M. are responsible for the final content.
Funding disclosures
The implementation of the Health meets Food curriculum at UTHealth was funded by the Nourish Program at the UTHealth School of Public Health and the Michael & Susan Dell Center for Healthy Living.
Funding information
Funding is provided by the Nourish Program.
Compliance with Ethical Standards
Conflict of Interest
There are no conflicts of interest to report.
Ethical Approval
Ethical Approval was provided by Tulane IRB Proposal.
Informed Consent
Informed consent was provided by all participants in the trial.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.American Heart Association Statistics Committee and Stroke Statistics Subcommittee Heart disease and stroke statistics--2010 update: a report from the American Heart Association. Circulation. 2010;121(7):e46–e215. doi: 10.1161/CIRCULATIONAHA.109.192667. [DOI] [PubMed] [Google Scholar]
- 2.Mattioli AV, Palmiero P, Manfrini O, Puddu PE, Nodari S, Dei Cas A, Mercuro G, Scrutinio D, Palermo P, Sciomer S, di Francesco S, Novo G, Novo S, Pedretti RFE, Zito A, Parati G, Pedrinelli R, Farinetti A, Maiello M, Moscucci F, Tenaglia RL, Sucato V, Triggiani M, Cugusi L, Scicchitano P, Saba PS, Ciccone MM. Mediterranean diet impact on cardiovascular diseases: a narrative review. J Cardiovasc Med. 2017;18(12):925–935. doi: 10.2459/JCM.0000000000000573. [DOI] [PubMed] [Google Scholar]
- 3.Huedo-Medina TB, Garcia M, Bihuniak JD, Kenny A, Kerstetter J. Methodologic quality of meta-analyses and systematic reviews on the Mediterranean diet and cardiovascular disease outcomes: a review. Am J Clin Nutr. 2016;103(3):841–850. doi: 10.3945/ajcn.115.112771. [DOI] [PubMed] [Google Scholar]
- 4.Koloverou E, Esposito K, Giugliano D, Panagiotakos D. The effect of Mediterranean diet on the development of type 2 diabetes mellitus: a meta-analysis of 10 prospective studies and 136,846 participants. Metabolism. 2014;63(7):903–911. doi: 10.1016/j.metabol.2014.04.010. [DOI] [PubMed] [Google Scholar]
- 5.Ajala O, English P, Pinkney J. Systematic review and meta-analysis of different dietary approaches to the management of type 2 diabetes. Am J Clin Nutr. 2013;97(3):505–516. doi: 10.3945/ajcn.112.042457. [DOI] [PubMed] [Google Scholar]
- 6.Sofi F, Cesari F, Abbate R, Gensini GF, Casini A. Adherence to Mediterranean diet and health status: meta-analysis. BMJ. 2008;337:a1344. doi: 10.1136/bmj.a1344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Trichopoulou A, Costacou T, Bamia C, Trichopoulos D. Adherence to a Mediterranean Diet and survival in a Greek population. N Engl J Med. 2003;348(26):2599–2608. doi: 10.1056/NEJMoa025039. [DOI] [PubMed] [Google Scholar]
- 8.Estruch R, Ros E, Salas-Salvadó J, Covas MI, Corella D, Arós F, Gómez-Gracia E, Ruiz-Gutiérrez V, Fiol M, Lapetra J, Lamuela-Raventos RM, Serra-Majem L, Pintó X, Basora J, Muñoz MA, Sorlí JV, Martínez JA, Martínez-González MA, PREDIMED Study Investigators Primary prevention of cardiovascular disease with a Mediterranean diet. N Engl J Med. 2013;368(14):1279–1290. doi: 10.1056/NEJMoa1200303. [DOI] [PubMed] [Google Scholar]
- 9.Fitó M, Guxens M, Corella D, Sáez G, Estruch R, de la Torre R, Francés F, Cabezas C, López-Sabater MDC, Marrugatet J, et al. Effect of a traditional Mediterranean diet on lipoprotein oxidation: a randomized controlled trial. Arch Intern Med. 2007;167(11):1195–1203. doi: 10.1001/archinte.167.11.1195. [DOI] [PubMed] [Google Scholar]
- 10.Smith AW, Borowski LA, Liu B, Galuska DA, Signore C, Klabunde C, Huang TTK, Krebs-Smith SM, Frank E, Pronk N, Ballard-Barbash R. U.S. primary care physicians’ diet-, physical activity-, and weight-related care of adult patients. Am J Prev Med. 2011;41(1):33–42. doi: 10.1016/j.amepre.2011.03.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Vetter ML, Herring SJ, Sood M, Shah NR, Kalet AL. What do resident physicians know about nutrition? An evaluation of attitudes, self-perceived proficiency and knowledge. J Am Coll Nutr. 2008;27(2):287–298. doi: 10.1080/07315724.2008.10719702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Spencerm EH, Frank E, Elon LK, Hertzberg VS, Serdula MK, Galuska DA. Predictors of nutrition counseling behaviors and attitudes in US medical students. Am J Clin Nutr. 2006;84(3):655–662. doi: 10.1093/ajcn/84.3.655. [DOI] [PubMed] [Google Scholar]
- 13.Lewis KO, Frank GR, Nagel R, Turner TL, Ferrell CL, Sangvai SG, Donthi R, Mahan JD. Pediatric trainees’ engagement in the online nutrition curriculum: preliminary results. BMC Med Educ. 2014;14:190. doi: 10.1186/1472-6920-14-190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ray S, Udumyan R, Rajput-Ray M, Thompson B, Lodge KM, Douglas P, Sharma P, Broughton R, Smart S, Wilson R, Gillam S, van der Es MJ, Fisher I, Gandy J. Evaluation of a novel nutrition education intervention for medical students from across England. BMJ Open. 2012;2:e000417. doi: 10.1136/bmjopen-2011-000417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Afaghi A, Haj Agha Mohamadi AA, Ziaee A, Sarchami R. Effect of an integrated case-based nutrition curriculum on medical education at Qazvin University of Medical Sciences, Iran. Global J Health Sci. 2012;4(1):112–117. doi: 10.5539/gjhs.v4n1p112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Barss P, Grivna M, Al-Maskari F, Kershaw G. Strengthening public health medicine training for medical students: development and evaluation of a lifestyle curriculum. Med Teach. 2008;30(9-10):e196–e218. doi: 10.1080/01421590802334267. [DOI] [PubMed] [Google Scholar]
- 17.Schlair S, Hanley K, Gillespie C, Disney L, Kalet A, Darby PC, Frank E, Spencer E, Harris J, Jay M. How medical students’ behaviors and attitudes affect the impact of a brief curriculum on nutrition counseling. J Nutr Educ Behav. 2012;44(6):653–657. doi: 10.1016/j.jneb.2011.08.006. [DOI] [PubMed] [Google Scholar]
- 18.Conroy MB, Delichatsios HK, Hafler JP, Rigotti NA. Impact of a preventive medicine and nutrition curriculum for medical students. Am J Prev Med. 2004;27(1):77–80. doi: 10.1016/j.amepre.2004.03.009. [DOI] [PubMed] [Google Scholar]
- 19.Kohlmeier M, McConathy WJ, Cooksey Lindell K, Zeisel SH. Adapting the contents of computer-based instruction based on knowledge tests maintains effectiveness of nutrition education. Am J Clin Nutr. 2003;77(4S):1025S–1027S. doi: 10.1093/ajcn/77.4.1025S. [DOI] [PubMed] [Google Scholar]
- 20.Lebensohn P, Kligler B, Dodds S, Schneider C, Sroka S, Benn R, Cook P, Guerrera M, Dog TL, Sierpina V, Teets R, Waxman D, Woytowicz J, Weil A, Maizes V. Integrative medicine in residency education: developing competency through online curriculum training. J Grad Med Educ. 2012;4(1):76–82. doi: 10.4300/JGME-04-01-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Monlezun DJ, Leong B, Joo E, Birkhead AG, Sarris L, Harlan TS. Novel longitudinal and propensity score matched analysis of hands-on cooking and nutrition education versus traditional clinical education among 627 medical students. Adv Prev Med. 2015;2015:656780. doi: 10.1155/2015/656780. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Monlezun DJ, Dart L, Vanbeber A, Smith-Barbaro P, Costilla V, Samuel C, et al. Machine learning- augmented propensity score-adjusted multilevel mixed effects panel analysis of hands-on cooking and nutrition education versus traditional curriculum for medical students as preventive cardiology: multisite cohort study of 3,248 trainees over 5 years. Biomed Res Int. 2018;2018. [DOI] [PMC free article] [PubMed]
- 23.Garcia-Arce A, Rico F, Zayas-Castro JL. Comparison of Machine Learning Algorithms for the Prediction of Preventable Hospital Readmissions. J. Healthc. Qual. 2017. [DOI] [PubMed]
- 24.Elze MC, Gregson J, Baber U, Williamson E, Sartori S, Mehran R, Nichols M, Stone GW, Pocock SJ. Comparison of Propensity Score Methods and Covariate Adjustment: Evaluation in 4 Cardiovascular Studies. J Am Coll Cardiol. 2017;69(3):345–357. doi: 10.1016/j.jacc.2016.10.060. [DOI] [PubMed] [Google Scholar]
- 25.Verbeke GM. Linear Mixed Models for Longitudinal Data. New York: Springer; 2000. [Google Scholar]
- 26.Funk MJ, Westreich D, Wiesen C, Stürmer T, Brookhart MA, Davidian M. Doubly robust estimation of causal effects. Am J Epidemiol. 2011;173(7):761–767. doi: 10.1093/aje/kwq439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Yazdani A, Boerwinkle E. Causal Inference in the Age of Decision Medicine. J. Data Mining Genomics Proteomics. 2015;6(1). [DOI] [PMC free article] [PubMed]
- 28.Shardell M, Ferrucci L. Joint mixed-effects models for causal inference with longitudinal data. Stat Med. 2017. [DOI] [PMC free article] [PubMed]
- 29.Gunasekara FI, Richardson K, Carter K, Blakely T. Fixed effects analysis of repeated measures data. Int J Epidemiol. 2014;43(1):264–269. doi: 10.1093/ije/dyt221. [DOI] [PubMed] [Google Scholar]
- 30.Stuart EA. Matching methods for causal inference: A review and a look forward. Stat Sci. 2010;25(1):1–21. doi: 10.1214/09-STS313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Robbins AS, Chao SY, Fonseca VP. What’s the relative risk? A method to directly estimate risk ratios in cohort studies of common outcomes. Ann Epidemiol. 2002;12(7):452–454. doi: 10.1016/S1047-2797(01)00278-2. [DOI] [PubMed] [Google Scholar]
- 32.Greenland S, Thomas DC, Morgenstern H. The rare-disease assumption revisited. A critique of ‘estimators of relative risk for case-control studies. Am J Epidemiol. 1986;124(6):869–883. doi: 10.1093/oxfordjournals.aje.a114476. [DOI] [PubMed] [Google Scholar]