Skip to main content
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 2020 Feb 6;27(4):639–643. doi: 10.1093/jamia/ocz223

Metrics for assessing physician activity using electronic health record log data

Christine A Sinsky 1,, Adam Rule 2, Genna Cohen 3, Brian G Arndt 4, Tait D Shanafelt 5, Christopher D Sharp 5,6, Sally L Baxter 7,8, Ming Tai-Seale 9, Sherry Yan 10, You Chen 11, Julia Adler-Milstein 12, Michelle Hribar 2
PMCID: PMC7075531  PMID: 32027360

Abstract

Electronic health record (EHR) log data have shown promise in measuring physician time spent on clinical activities, contributing to deeper understanding and further optimization of the clinical environment. In this article, we propose 7 core measures of EHR use that reflect multiple dimensions of practice efficiency: total EHR time, work outside of work, time on documentation, time on prescriptions, inbox time, teamwork for orders, and an aspirational measure for the amount of undivided attention patients receive from their physicians during an encounter, undivided attention. We also illustrate sample use cases for these measures for multiple stakeholders. Finally, standardization of EHR log data measure specifications, as outlined here, will foster cross-study synthesis and comparative research.

Keywords: metric; operational efficiency; EHR log data; time studies, burnout

INTRODUCTION

Electronic health records (EHRs) have transformed the daily work of clinicians. Widely implemented over the past 2 decades, with an investment of approximately $36 billion in federal funding,1,2 EHRs are now ubiquitous in ambulatory (86%) and inpatient (96%) settings.3 Despite this investment and widespread adoption, the evidence for the impact of EHRs on quality, cost, and efficiency is mixed,4–9 and there is growing evidence of a negative impact on the work lives of physicians.10–16 In particular, the use of an EHR has been associated with higher rates of burnout.10,12,15 A direct observation time-motion study in 2016 found physicians in the ambulatory setting spend half of the workday on the EHR, requiring nearly 2 hours of EHR and desk work for every 1 hour of direct clinical face time with patients.17 Subsequent studies using EHR log data similarly found that approximately half of the physician workday is spent on the EHR.18–20 In addition, physicians may allocate 1-2 hours of personal time each night to EHR tasks.17,18 Special concerns have been raised about the time costs of several clinical tasks, including inbox management,21,22 computerized physician order entry,15 algorithm-generated alerts,22,23 visit-note documentation,17,18,24 and the overall volume of administrative burden mediated through the EHR.25,26

While EHR log data has shown promise in measuring time spent on these clinical activities, the use of EHR log data to further understand the clinical environment is a nascent science.27,28 All certified EHRs in the United States are required to maintain EHR logs, tracking who accesses which patient records at what time, and what they do in the record, such as view, edit, or delete patient information.29,30 While originally designed to support auditing of inappropriate record access, these logs provide an opportunity to study clinical activities unobtrusively and at scale, including how time is distributed among various tasks and role types, and how interventions, such as new policies, staffing models, or workflows, impact the time spent within the EHR. A systematic review identified 85 studies using EHR log data.31 Most did not detail their definitions of EHR task or methodology in estimating EHR time, nor did they validate their measures. Some researchers accessed raw data files from EHR logs, using extensive data cleaning and preprocessing to compute measures such as total EHR time. Others used “off-the-shelf” processed data provided by vendors, typically generated by proprietary algorithms and shaped by the assumptions and definitions which are not always known to the researcher or reader. This variation hinders cross-study synthesis and comparative effectiveness research.

Therefore, a network of researchers and other stakeholders working with EHR log data32 collaborated to standardize EHR audit log measures that capture different dimensions of the physician EHR work. In this commentary, we propose 7 core EHR use measures, identify potential use cases for these measures across multiple stakeholders, highlight unresolved methodologic considerations, and address future directions for research and use. This work builds on prior work33 outlining potential measures to evaluate the practice environment by refining and expanding the core measure set and providing detailed specification.

FACTORS GUIDING MEASURE SELECTION

We worked with multiple stakeholders to select measures that are feasible and relevant to clinical and operational decision making. We identified time as an important unit of measure, as it is a commodity in limited supply for physicians and other health professionals. We also recognized the need to normalize time-based data to allow comparisons across different degrees of clinical effort (ie, part time vs full time); we chose an 8-hour period of scheduled patient time as the unit of normalization. In designing these measures, we prioritized simplicity and feasibility over complexity and granularity, recognizing that there will be variations in clinical circumstances that are not fully captured with these initial metrics. In addition, while time alone does not capture all dimensions of work (eg, the content of work and appropriateness of tasks to different levels of training or certification), time-based metrics are relevant, interpretable, and form a foundation from which to expand upon in the future. We also recognize that these measures are best suited to application in the ambulatory setting and will need to be modified for use in the inpatient arena. Finally, while we have focused on physicians, future studies examining EHR use measures for additional roles (medical assistants, pharmacists, nurses, advance practice providers, etc.) will also be beneficial.

PROPOSED CORE EHR USE MEASURES

We propose 7 core measures of EHR use which reflect multiple dimensions of practice efficiency (Table 1): total EHR time (EHR-Time); work outside of work (WOW), often referred to as “pajama time”; time on documentation (Note-Time); time on prescriptions (Script-Time); inbox time (IB-Time); teamwork for orders (TWORD); and an aspirational measure for the amount of undivided attention patients receive from their physicians during an encounter, undivided attention (ATTN).

Table 1.

Proposed core EHR use measures

Measure Abbreviation Definition and example
Total EHR time EHR-Time8 Total time on EHR (during and outside of clinic sessions) per 8 h of patient scheduled time.
Example: A physician with 32 patient-scheduled hours per week, 20 h of EHR time during scheduled hours, 10 h of WOW each week would have EHR-Time8 of 30/32 × 8 = 7.5.
Work outside of work WOW8 Time on EHR outside of scheduled patient hours per 8 h of patient scheduled time.a
Example: A physician with 32 scheduled patient hours per week and a total of 10 h of EHR time outside of these scheduled hours, would have WOW8 = 10/32 × 8 = 2.5.
Time on encounter note documentation Note-Time8 Hours on documentation (note writing) per 8 h of scheduled patient time
Example: A physician with 32 scheduled patient hours per week and a total of 20 h of documentation time (both in the room with the patient and outside of the room) per week would have DocTime8 of 20/32 × 8 = 5.0.
Time on prescriptions Script-Time8 Total time on prescriptions per 8 h of patient scheduled time
Example: A physician spends 3 h per week on prescription work and has 24 h of patient scheduled time per week. Script8 = 3/24 × 8 = 1
Time on inbox IB-Time8 Total time on inbox per 8 h of patient scheduled time
Example: A physician spends 10 h per week on Inbox work and has 20 h per week of patient scheduled time. IB8 = 10/20 × 8 = 4
Teamwork for orders TWORD The percentage of orders with team contribution
Example: A physician working with a team that is empowered to pend, send orders by protocol, or operationalize verbal orders, may compose 25% of the orders from start to finish on their own, while the rest are pended or completed by team members for the physician’s co-signature. In this case, TWORD = 75%.
Undivided attention ATTN The amount of undivided attention patients receive from their physician. It is approximated by [(total time per session) minus (EHR time per session)]/total time per session.  
Example: A physician who is actively on the EHR 3 h of a 4-h clinic session would have a lower ATTN score (4-3)/4 = 0.25 than would a physician who was actively on the EHR 1 h of a 4-h clinic session. (4-1)/4 = 0.75.

EHR: electronic health record.

a

For consistency, and to avoid distortion owing to different session lengths, we define work outside of work precisely as that time outside of scheduled patient hours and do not include any “shoulder time” before or after clinic.

EXAMPLES OF THE UTILITY OF EHR USE MEASURES

EHR use measures have multiple uses within research and clinic administration (Table 2). First, EHR use measures can be used to assess the impact of new staffing models, such as advanced team-based care with in-room support.34 For example, a physician practicing with a strong, skilled team may be able to spend the majority of her time providing undivided attention to her patients, while her empowered staff enters orders, completes the billing invoice, drafts the preliminary visit note, and manages the team’s inbox. Another physician, practicing in a more “the doctor does it all” environment, performs the majority of data entry and inbox management on her own. EHR use measures can help to quantify this difference by establishing the degree to which WOW and Note-Time decrease and TWORD increases when moving to an advanced team-based model of care.

Table 2.

Sample use cases for EHR log data

Stakeholder Objective Examples of research questions
Health system leaders Improve the patient experience How do increased staffing ratios impact WOW8, Note-Time8, and the amount of direct face time patients receive from their physician (ATTN)?
How do decreases in WOW8, Note-Time8, and IB-Time8 impact appointment availability and continuity?
Improve access for patients How do staffing ratios impact the total workload for physicians (ie, patient scheduled time + WOW)?
Improve professional fulfillment Can EHR use patterns predict future risk of burnout, other dimensions of distress, or professional fulfillment?
Improve retention Can EHR use patterns predict future risk of leaving the organization?
Improve recruitment Physicians may choose to incorporate data regarding work outside of work and total EHR time when evaluating a potential employment offer.
Operations leaders Improve efficiency Does empowered inbox management by team members reduce physician/APP EHR time after hours?
Does advanced team-based care with in-room support35 reduce work outside of work?
Standardize FTE expectations How many patient contact hours is associated with a 40-h total work week in across different specialties?
How do changes in staffing ratio, team stability, and team skill level impact work outside of work? 
Understand the costs of compliance decisions How do requirements that only physicians do certain tasks such as order entry, medication reconciliation, and prohibitions against verbal orders impact WOW8?
Regulators Understand and evaluate trade-offs involved in a given policy under consideration Pilot test policies under consideration and evaluate their impact on EHR use metrics before finalizing and implementing.
How do requirements that only physicians do certain tasks such as order entry, medication reconciliation and prohibitions against verbal orders impact WOW8?
What are the time costs associated with prohibiting verbal orders?
What are the time costs of requiring 2-factor authentication and password revalidation for nonscheduled prescriptions?
Technology vendors Improve usability How does an electronic workflow for order entry impact time on orders?
Help clients measure and improve practice efficiency Provide insight and guidance to help organizations improve efficiency, patient satisfaction, and professional fulfillment for healthcare professionals

APP: advanced practice provider; ATTN : undivided attention; EHR: electronic health record; FTE: full-time equivalent; IB-Time8: time on inbox per 8 h of patient scheduled time; Note-Time8:time on encounter note documentation per 8 h of patient scheduled time; WOW8: work outside of work per 8 h of patient scheduled time.

EHR use measures can also inform expectations for patient-scheduled hours for a full-time equivalent position within a given specialty and clinical support environment. A physician sharing a single medical assistant with another physician may be able to accommodate 20 hours of patient-scheduled time per week, because of an additional 20 hours of WOW per week. In another practice within the same specialty, a physician dedicated 2 medical assistants may be able to accommodate 30 hours of patient-scheduled time per week because of a reduction in WOW to 10 h/wk as a result of greater support.

EHR use measures can be a component of predictive analytics. For example, with longitudinal tracking, health systems may find that WOW, IB-Time, and TWORD predict physician risk for distress, burnout, and intent to cut back on clinical effort or leave the practice. EHR use measures can likewise illuminate the impact of regulatory and compliance decisions upon clinician work. For example, will WOW and Note-Time change as a result of the anticipated CPT 2021 modification of documentation guidelines and codes,35 which eliminates history and exam from criteria in billing office visit codes?

Finally, EHR use measures provide discrete metrics that can be used by multiple stakeholders to promote EHR design improvements to facilitate efficiency of use. These stakeholders include EHR vendors, those responsible for EHR implementation, and those who integrate elements of the EHR designed for regulatory or billing purposes. EHR use measures could help inform understanding of efficient practices and improve future design.

CHALLENGES AND LIMITATIONS

Several definitional and methodologic considerations with the use of EHR log data remain unsettled, including those related to measure implementation, validity, normalization, and generalizability.

IMPLEMENTATION

1. Definition ofEHR usetime. The interval without user activity that triggers a time-out from active use will impact the resulting EHR use times. Currently, this interval may vary from 5 seconds, to 90 seconds, to 30 minutes. Determining time-out intervals that best represent user activity is challenging and may vary by setting or task, and one could even argue that they should not be included at all in some calculations. For example, when a physician is working from home and multitasking with family obligations, all of the EHR time, both active and idle, potentially adversely impacts the physician and the quality of family time.

2. Definition of work outside of work. Some vendors and researchers have used clock times, such as 7 pm to 7 am, to establish the window for work outside of work, but this does not account for variability in physician schedules. A physician who utilizes administration days or personal days off to complete documentation and inbox work would appear to have little WOW in this data model. In the ambulatory setting, an accurate assessment of WOW requires integration of the physician’s clinic schedule with the EHR log data.

3. The exclusion of “shoulder time(ie, prep time)immediately before or after clinic. Some stakeholders have excluded the 30 minutes before and after scheduled patient time from WOW calculations. This minimizes the apparent WOW for those who work shorter clinic sessions. For example, a physician with 16 hours of patient contact time per week distributed in two 8-hour days would have 2 hours of shoulder time subtracted from WOW, whereas a physician who distributed the same patient contact time across four 4-hour days would have 4 hours of shoulder time subtracted from WOW. To avoid these sources of distortion in the data, we define WOW precisely as that time on the EHR outside of scheduled patient hours.

4. The mapping of EHR actions (ie, time on a type of EHR screen) to work (ie, the total time required to complete a task). For example, the time to resolve an inbox task is not fully captured by the time on the inbox screen alone—it may also include reviewing the problem list, consulting the medication list, and assessing recent lab results. Our inbox measure captures the entire time on the task of inbox management as opposed to just the time spent on the inbox screen itself. The technical methodology for linking the time on multiple screens to capture time per task requires refinement.

5. Work outside of the EHR: Not every aspect of clinical work is performed via the EHR. The time associated with phone calls, messages, family meetings, paperwork, verbal communication with colleagues, and even direct face time with patients are components of a physician’s workday that are not directly captured by EHR time stamp data.

VALIDITY

6. EHR usemeasure validation. Options include face validity via end-user testing vs validation against an external standard, such as direct observation time-motion analysis. Manual time-motion observations are labor-intensive but currently represent the gold standard for validation.18,19,36,37 It is our hope that these core measures will be validated across different types of institutions, clinical specialties, and EHR vendors.

NORMALIZATION

7. EHR measure normalization. Candidate denominators include number of patients, clinic sessions, workdays, or hours of scheduled patient time. We elected to normalize to 8 hours of patient scheduled time because time is a finite resource for physicians. We acknowledge that volume is a consideration as well; a physician who is scheduled lightly during a clinic session (ie, 1 patient per hour) may be able to perform more documentation, inbox, and other EHR work during their session than is a physician who is scheduled more densely during their session (ie, 4 patients per hour). Therefore, we encourage researchers and vendors to report average clinic volumes and, as feasible, other measures of workload, along with the time-based measures.

8. Reporting period. We propose that EHR use data generally be collected over a 1-month time frame. There are tradeoffs between the amount of work required to collect and clean a larger dataset and the potential for a shorter period to reflect anomalies owing to seasonal care requirements, vacations, time on teaching service, changes in staffing, etc.

GENERALIZABILITY

9. Generalization to inpatient and procedural settings. Additional work is needed to adapt these measures to other settings. For example, when hospitalists and emergency room physicians work set shifts, WOW can be easily defined as any time outside of a scheduled shift and normalized to 8 hours of shift work. On the other hand, when physicians work in both ambulatory and hospital settings, it may be difficult to identify which WOW belongs to which setting.

10. Unintended negative consequences. Some physicians may not like the notion that “the company” is watching what they do at home or resent that measures make them appear inefficient. This could motivate them to shorten notes, cut corners, or find gaps in how the metric is calculated and do their work in that window to avoid being labeled an “inefficient physician.” In addition, it is important to note that the intention of these measures is to improve understanding of the physician work experience; we do not suggest that these new measures be included as requirements in any federal reporting programs.

CONCLUSION

The analysis of EHR log data is emerging as a tool to further understand the clinical environment and to optimize operational, technological, and policy decisions. We propose and define 7 core measures of EHR use and normalize the time measures to 8 hours of patient scheduled time. It is our intent that the measures provide insight and facilitate research regarding the efficiency of using EHRs in the practice environment, the effectiveness of teams, the impact of policies and regulations, and practice characteristics that contribute to physician distress or well-being. Improving the physician experience should, in turn, positively impact the patient experience as well. Furthermore, these standard measures will allow for better reproducibility and comparison of research studies. While we expect EHR use measures will need to adapt over time in response to advances in technology, changing clinician roles, and evolving regulatory policies, these measures provide a starting point to enhance consistency and reproducibility for analyses of current systems.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

AUTHOR CONTRIBUTIONS

Each author made substantial contributions to the conception or design of the work; was involved in drafting the work or revising it critically for important intellectual content; gave final approval of the version to be published; and has agreed to be accountable for all aspects of the work.

CONFLICT OF INTEREST

SLB acknowledges grant funding from National Institutes of Health/National Library of Medicine T15LM011271 and the Heed Ophthalmic Foundation. The remaining authors have no competing interests to declare.

References

  • 1. Thune J, Alexander L, Roberts P, Burr R, Enzi M. Where is HITECH’s $35 billion dollar investment going? Health Affairs Blog. https://www.healthaffairs.org/do/10.1377/hblog20150304.045199/full/ Accessed September 9, 2019.
  • 2. Adler-Milstein J, Jha AK.. HITECH act drove large gains in hospital electronic health record adoption. Health Aff (Millwood) 2017; 36 (8): 1416–22. [DOI] [PubMed] [Google Scholar]
  • 3.Office of the National Coordinator for Health Information Technology. Health IT Dashboard. https://dashboard.healthit.gov/quickstats/quickstats.php Accessed ¡September 9, 2019.
  • 4. Buntin MB, Burke MF, Hoaglin MC, et al. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood) 2011; 30 (3): 464–71. [DOI] [PubMed] [Google Scholar]
  • 5. Black AD, Car J, Pagliari C, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011; 8 (1): e1000387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Jones SS, Rudin RS, Perry T, et al. Health information technology: an updated systematic review with a focus on meaningful use. Ann Intern Med 2014; 160 (1): 48–54. [DOI] [PubMed] [Google Scholar]
  • 7. Adler-Milstein J, Huckman RS.. The impact of electronic health record use on physician productivity. Am J Manag Care 2013; 19 (10 Spec No): Sp345–52. [PubMed] [Google Scholar]
  • 8. Baumann LA, Baker J, Elshaug AG.. The impact of electronic health record systems on clinical documentation times: a systematic review. Health Policy 2018; 122 (8): 827–36. [DOI] [PubMed] [Google Scholar]
  • 9. Poissant L. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc 2005; 12 (5): 505–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Robertson SL, Robinson MD, Reid A.. Electronic health record effects on work-life balance and burnout within the I(3) population collaborative. J Grad Med Educ 2017; 9 (4): 479–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Friedberg MW, Chen PG, Van Busum KR, et al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy. Santa Monica, CA: RAND Corporation; 2013. [PMC free article] [PubMed] [Google Scholar]
  • 12. Babbott S, Manwell LB, Brown R, et al. Electronic medical records and physician stress in primary care: results from the MEMO study. J Am Med Inform Assoc 2014; 21 (e1): e100–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Holman GT, Waldren SE, Beasley JW, et al. Meaningful use’s benefits and burdens for US family physicians. J Am Med Inform Assoc 2018; 25 (6): 694–701. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Kroth PJ, Morioka-Douglas N, Veres S, et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (8): e199609. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016; 91 (7): 836–48. [DOI] [PubMed] [Google Scholar]
  • 16. Gardner RL, Cooper E, Haskell J, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (2): 106–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Sinsky C, Colligan L, Li L, et al. Allocation of physician time in ambulatory practice: A time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11): 753–60. [DOI] [PubMed] [Google Scholar]
  • 18. Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (5): 419–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Tai-Seale M, Olson CW, Li J, et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017; 36 (4): 655–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Young RA, Burge SK, Kumar KA, Wilson JM, Ortiz DF.. A time-motion study of primary care physicians’ work in the electronic health record era. Fam Med. 2018; 50 (2): 91–9. [DOI] [PubMed] [Google Scholar]
  • 21. Murphy DR, Meyer AND, Russo E, et al. The burden of inbox notifications in commercial electronic health records. JAMA Intern Med 2016; 176 (4): 559–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Tai-Seale M, Dillon EC, Yang Y, et al. Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records. Health Aff (Millwood) 2019; 38 (7): 1073–8. [DOI] [PubMed] [Google Scholar]
  • 23. Gregory ME, Russo E, Singh H.. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (3): 686–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Hripcsak G, Vawdrey DK, Fred MR, et al. Use of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc 2011; 18 (2): 112–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Erickson SM, Rockwern B, Koltov M, et al. Putting patients first by reducing administrative tasks in health care: a position paper of the American College of Physicians. Ann Intern Med 2017; 166 (9): 659–61. [DOI] [PubMed] [Google Scholar]
  • 26. Sinsky CA, Beasley JW, Simmons GE, et al. Electronic health records: design, implementation, and policy for higher-value primary care. Ann Intern Med 2014; 160 (10): 727–8. [DOI] [PubMed] [Google Scholar]
  • 27. DiAngi YT, Stevens LA, Halpern-Felsher B, Pageler NM, Lee TC.. Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers’ perceived control over their workload in the EHR. JAMIA Open 2019, 2 (2): 222–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Chen ES, Cimino JJ.. Automated discovery of patient-specific clinician information needs using clinical information system log files. AMIA Annu Symp Proc. 2003; 2003: 145–9. [PMC free article] [PubMed] [Google Scholar]
  • 29.ASTM International. ASTM E2147-01. Standard Specification for Audit and Disclosure Logs for Use in Health Information Systems West Conshohocken, PA: ASTM International; 2009. https://www.astm.org/DATABASE.CART/HISTORICAL/E2147-01.htm Accessed October 28, 2019.
  • 30.ASTM International. ASTM E2147-18. Standard Specification for Audit and Disclosure Logs for Use in Health Information Systems West Conshohocken, PA: ASTM International; 2018. https://www.astm.org/Standards/E2147.htm Accessed October 28, 2019.
  • 31. Rule A, Chiang MF, Hribar MR.. Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods. J Am Med Inform Assoc 2019; doi: 10.1093/jamia/ocz196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Center for Clinical Informatics and Improvement Research. National Research Network. https://medicine.ucsf.edu/center-clinical-informatics-and-improvement-research/national-research-network Accessed December 1, 2019.
  • 33. DiAngi YT, Lee TC, Sinsky CA, Bohman BD, Sharp CD.. NOvel metrics for improving professional fulfillment. Ann Intern Med 2017; 167 (10): 740–1. [DOI] [PubMed] [Google Scholar]
  • 34. Sinsky CA, Bodenheimer T.. Powering-up primary care teams: advanced team care with in-room support. Ann Fam Med 2019; 17 (4): 367–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.American Medical Associaiton. CPT Evaluation and Management. https://www.ama-assn.org/practice-management/cpt/cpt-evaluation-and-management Accessed October 30, 2019.
  • 36. Read-Brown S, Hribar MR, Reznick LG, et al. Time requirements for electronic health record use in an Academic Ophthalmology Center. JAMA Ophthalmol 2017; 135 (11): 1250–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Lopetegui M, Yen PY, Lai AM, Embi PJ, Payne PR.. Time Capture Tool (TimeCaT): development of a comprehensive application to support data capture for Time Motion Studies. AMIA Annu Symp Proc. 2012; 2012: 596–605. [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES