Table 3.

Key References About Learning Analytics for Clinician Educators

Citation	Why This Paper Is Important
Key conceptual papers
Bok HG, Teunissen PW, Favier RP, et al. Programmatic assessment of competency‐based workplace learning: when theory meets practice. BMC Med Educ 2013;13:123.13	This paper provides a good overview of what the concept programmatic assessment means and how it can help to lay basis for mapping out the type of assessment data that on might wish to gather.
Pusic MV, Boutis K, Hatala R, Cook DA. Learning curves in health professions education. Acad Med 2015;90:1.9	This provides a conceptual overview about learning curves and how they might apply to health professions education.
Cirigliano MM, Guthrie C, Pusic MV, et al. “Yes, and …” Exploring the future of learning analytics in medical education. Teach Learn Med 2017;29:368–72.12	This is a short paper that highlights the opportunities for using learning analytics in medical education.
Key implementation‐related papers
Chan TM, Sherbino J, Mercuri M. Nuance and noise: lessons learned from longitudinal aggregated assessment data. J Grad Med Educ 2017;9:724–9.6	This is an example of how assessment data might be aggregated and mapped, explaining to educators how this data might be harnessed, and what noise might surround the emergent data.
Ginsburg S, Regehr G, Lingard L, Eva KW. Reading between the lines: faculty interpretations of narrative evaluation comments. Med Educ 2015;49:296–306.54	This paper highlights the importance of qualitative data and how administrators and competency committee members might interpret these data.
McConnell M, Sherbino J, Chan TM. Mind the gap: the prospects of missing data. J Grad Med Educ 2016;8:708–12.8	This paper highlights a systems‐level analysis of how administrators and competency committee members might consider “missing data.” It provides a suggestion of applying a framework similar to Yvonne Steinert's KSALTS framework to analyze the problem that missing data presents.
Warm EJ, Held JD, Hellmann M, et al. Entrusting observable practice activities and milestones over the 36 months of an internal medicine residency. Acad Med 2016;91:1398–405.11	This is a worked example of a residency program with a large number of residents and how they visualized and used the data they acquired about workplace‐based assessments for educational purposes.