Measuring Electronic Health Record Use in Primary Care: A Scoping Review

Abstract

Background  Simple measures of electronic health record (EHR) adoption may be inadequate to evaluate EHR use; and positive outcomes associated with EHRs may be better gauged when varying degrees of EHR use are taken into account. In this article, we aim to assess the current state of the literature regarding measuring EHR use.

Objective  This article conducts a scoping review of the literature to identify and classify measures of primary care EHR use with a focus on the Canadian context.

Methods  We conducted a scoping review. Multiple citation databases were searched, as well as gray literature from relevant Web sites. Resulting abstracts were screened for inclusion. Included full texts were reviewed by two authors. Data from the articles were extracted; we synthesized the findings. Subsequently, we reviewed these results with seven EHR stakeholders in Canada.

Results  Thirty-seven articles were included. Eighteen measured EHR function use individually, while 19 incorporated an overall level of use. Eight frameworks for characterizing overall EHR use were identified.

Conclusion  There is a need to create standardized frameworks for assessing EHR use.

Background and Significance

Electronic health record (EHR) systems have the potential to significantly impact the practice of primary care by improving quality of care, reducing costs, and improving health outcomes; certain countries have provided funding/incentives to encourage their widespread adoption and use. 1 2 The most notable of such national programs is the United States' provision of “Meaningful Use” incentives, supported by the HITECH (Health Information Technology for Economic and Clinical Health) Act of 2009. 3 Canada has also created national initiatives to promote adoption. 4 Health information technology adoption rates among primary care physicians in countries such as the United Kingdom, Australia, and New Zealand are high (at, or above 95%). 5 As of 2015, 73% of Canadian family physicians had adopted an EHR. 6 However, adoption rates vary greatly between provinces, 7 and rates of multifunctional EHR use are much lower. 8

There are many reported benefits of EHRs over traditional article records, namely, increased workflow efficiency, error reduction, improved patient safety, and better communication between providers. 9 10 While some studies report generally positive improvements in care with EHR use, 11 12 others are mixed, 13 14 and some show negative effects on quality of care. 15 16 These mixed results may be due to inconsistent use of this technology in clinical practice. 17 18 19 There are indications that when more mature, or advanced EHR use exists, 18 including, for example, better integration with workflow, 20 positive effects may be realized. Therefore, it is important to be able to assess EHR use effectively, so that the link between use and outcomes can be further explored.

Current approaches to assessing the degree to which clinicians use their EHRs in day-to-day practice focus primarily on the use/nonuse of component parts of the EHR—i.e., functions of the EHR such as ordering laboratory tests. Therefore, unidimensional (for example, asking if the EHR is used yes or no), and binary measures of EHR use (for example, those that simply ask if an EHR function is used – yes or no) may be inadequate to evaluate actual use; positive outcomes associated with EHRs may be better gauged when multiple dimensions of EHR use are taken into account.

It is thus important to develop a comprehensive, multidimensional set of measures for evaluating EHR use. While Schoen et al define multifunctional EHR use as “Uses electronic medical records and at least two electronic functions in each of the following four domains: generating patient information, generating panel information, order entry management, and routine clinical decision support,” we propose that assessments of use can take into account more than just the use of multiple EHR functions. 5 Therefore, multidimensional assessments of electronic medical record (EMR) use could go beyond these traditional measures to include, taking into account the way that the EHR is used during patient encounters, exploring the way that different primary care practitioners use the EHR, understanding how the EHR is being used by a team of practitioners versus an individual, and evaluating degrees of function use. As a first step, we need to understand how primary care EHR use has been measured.

In Canada, while EHR use measures exist, 21 22 there is currently no nationally agreed upon framework for EHR use in the primary care setting. There are many provincially based incentive programs for EHR adoption, which employ differing frameworks for measuring use. 2 While approaches to measuring EHR use exist for other settings, 23 24 there is no universally accepted, standard way EHR use is assessed across all health care settings. There are some unique aspects of primary care, for example, that it is defined by first contact care provided on a longitudinal basis to a population of patients attached to a particular practice, where the practitioner is often the coordinator of care across settings, that make primary care EHRs and the data within them, different from medical records used in other settings. There is a body of literature which we seek to explore in this article, which focuses on measuring EHR use in the primary care setting. To date, there are no sources which comprehensively describe the sets of primary care EHR use measures reported in the literature. We focused on primary care EHRs with a view to define the review according to our interests in the type of EHR used in primary care practice, as well as the importance and scope of the setting in terms of the health care system. 25 26 Electronic records systems in health care are named inconsistently in the literature as EMRs 2 and EHRs. 3 11 For the purposes of this study, we use the term EHR, defined as computer-based systems that record and store a patient's health and medical information such as medical history, encounter notes, prescriptions, and laboratory reports. EHR systems also allow electronic sharing of such records with authorized health care providers. 27

Objective

This article conducts a scoping review of the literature to identify and classify measures of primary care EHR use with a focus on the Canadian context. In this context, measures are tools or instruments by which clinician use of EHRs in practice can be assessed.

Methods

This scoping review was performed following the methodological framework of Arksey and O'Malley 28 and Levac et al. 29 Scoping reviews “map rapidly key concepts underpinning a research area and the main sources and types of evidence available […] especially where an area is complex or has not been reviewed comprehensively before.” 28 The review followed six stages: identifying the research question; identifying relevant studies; study selection; charting data; collating, summarizing, and reporting results; and stakeholder consultations.

Identifying Relevant Studies

A preliminary search was performed on the following literature databases: PubMed, Cochrane, CINAHL, Web of Science, and Proquest in December 2015. No date restrictions were defined. Search queries included keywords relating to EHRs, primary care, and measuring use. If available (PubMed, Cochrane, and CINAHL), indexed subject headings were also used. A supplementary search was conducted in PubMed using the National Library of Medicine's custom query for EHRs. 30 A research librarian from The University of Western Ontario was consulted as we developed our search strategy. Once preliminary results were obtained, abstracts of 15 randomly selected articles were evaluated to ensure that the topic of interest was effectively captured. Search queries were refined based on this evaluation (see Appendix A ).

Study Selection

A total of 514 results were obtained from the combined searches (see Fig. 1 ). After the removal of 106 duplicates, 408 articles remained. Five additional texts identified based on prior knowledge of one author (A.L.T.) were added for a total of 413 unique articles, which were independently reviewed by A.L.T. and M.Z.H. Abstracts were included if: (1) the article included the primary care setting, (2) there was a focus on health information technology, and (3) the study focus/objective was to measure EHR use (please see Appendix A for keywords and subject headings we used in our search that correspond to each of these concepts). The latter criterion reflects our interest in measures of actual EHR use versus the simple availability of functions. Articles were excluded if the abstract could not be obtained. Following independent review of the abstracts, we resolved inclusion/exclusion disagreements in person by consensus. Of 413 articles, 84 had abstracts that met the review criteria.

Fig. 1.

Study selection.

Full texts of the 84 articles were obtained, and evaluated independently by A.L.T. and M.Z.H. Full texts were included if they: (1) explicitly included primary care practices, and (2) focused on applying a multidimensional set of measures for EHR use. Conflicts were reconciled in-person. As we were interested in measures that assessed overall EHR use as a more realistic reflection of day-to-day use, we excluded studies assessing only one EHR function. Of the 84 articles, 56 did not meet the review criteria. One article was excluded as the full text could not be accessed.

As a supplement, we also selected four articles identified through a gray literature search. The Web sites of governmental and professional organizations relating to eHealth/health information technology in Canada, the United States, the United Kingdom, Australia, and New Zealand were manually searched (see Appendix B ). The countries were chosen because they were English-speaking and had high levels of EHR adoption. 5 Types of literature excluded were: newsletters, press releases, commentaries, newspaper and magazine articles, and annual reports. The inclusion criteria for the gray literature were the same as that for full text articles.

Appendix B. Gray Literature Sites Searched.

International
International Medical Informatics Association (IMIA)	http://www.imia-medinfo.org/
Health Level Seven International (HL7)	http://www.hl7.org/
International Federation of Health Information Management Associations (IFHIMA)	https://ifhima.org/
Canada
COACH, Canada's Health Informatics Association	http://coachorg.com/
Canada Health Infoway	https://www.infoway-inforoute.ca/en/
National Institutes of Health Informatics (NIHI)	http://www.nihi.ca/
Canadian Institute for Health Information (CIHI)	https://www.cihi.ca/
Canadian Health Information Management Association (CHIMA)	https://www.echima.ca/
Canadian Medical Association (CMA)	https://www.cma.ca/
British Columbia Ministry of Health	http://www2.gov.bc.ca/gov/content/governments/organizational-structure/ministries-organizations/ministries/health
Doctors of BC	https://www.doctorsofbc.ca/
Alberta Medical Association	https://www.albertadoctors.org/
Alberta Netcare EHR	http://www.albertanetcare.ca/
Doctors Manitoba	https://docsmb.org/
Manitoba eHealth	http://www.manitoba-ehealth.ca/
Saskatchewan Medical Association	http://www.sma.sk.ca/
eHealth Saskatchewan	https://www.ehealthsask.ca/
Ontario Medical Association	https://www.oma.org/
eHealth Ontario	http://www.ehealthontario.on.ca/en/
Quebec Medical Association	https://www.amq.ca/en
Dossier Santé Québec (Quebec Health Record)	http://www.dossierdesante.gouv.qc.ca/
Newfoundland and Labrador Medical Association	http://www.nlma.nl.ca/
eDOCSNL (Newfoundland and Labrador)	https://edocsnl.ca//
Newfoundland and Labrador Centre for Health Information	https://www.nlchi.nl.ca//
United States
American Medical Association (AMA)	https://www.ama-assn.org/
American Medical Informatics Association (AMIA)	https://www.amia.org/
American Health Information Management Association (AHIMA)	http://www.ahima.org/
HealthIT.gov	https://www.healthit.gov/
Centers for Medicare and Medicaid Services (CMS)	https://www.cms.gov/
Healthcare Information and Management Systems Society (HIMSS)	http://www.himss.org/
HIMSS Analytics	http://www.himssanalytics.org/
Health Resources and Services Administration	https://www.hrsa.gov/
American Society of Health Informatics Managers (ASHIM)	https://www.ashim.com/
United Kingdom
National Health Service	http://www.nhs.uk/
UK Department of Health	https://www.gov.uk/government/organisations/department-of-health
British Medical Association (BMA)	https://www.bma.org.uk/
UK Council for Health Informatics Professionals (UKCHIP)	http://www.ukchip.org/
Institute of Health Records & Information Management	http://www.ihrim.co.uk/
BCS Health	http://www.bcs.org/
Professional Record Standards Body	http://theprsb.org/
UK Clinical Research Collaboration	http://www.ukcrc.org/
Australia
Australian Department of Health	http://www.health.gov.au/
Australian Digital Health Agency	https://www.digitalhealth.gov.au/
Standards Australia: eHealth	http://www.e-health.standards.org.au/
Health Informatics Society of Australia	https://www.hisa.org.au/
Australian Health Informatics Education Council	http://www.ahiec.org.au/
My Health Record Australia	https://myhealthrecord.gov.au/
Health Information Management Association of Australia	http://www.himaa2.org.au/
The Australasian College of Health Informatics	http://www.achi.org.au/
Australian eHealth Research Centre	https://aehrc.com/
Australian Commission on Safety and Quality in Health Care	https://www.safetyandquality.gov.au/
New Zealand
Health Informatics New Zealand (HiNZ)	http://www.hinz.org.nz/
New Zealand Medical Association (NZMA)	https://www.nzma.org.nz/
New Zealand Ministry of Health	http://www.health.govt.nz/
New Zealand Health IT (NZHIT)	http://www.healthit.org.nz/

Reference lists of included texts were manually screened for additional relevant citations. Six additional articles were identified. In total, 37 articles were included for review ( Fig. 1 ).

Data Extraction

A preliminary data charting form was developed and tested independently by A.L.T. and M.Z.H. on five randomly selected articles. Feedback from testing was used to refine the form (see Appendix C ), after which it was applied by M.Z.H. on the remaining articles. The following information was recorded, if available: author affiliation, country of origin, type of study, rationale of study, setting, unit of analysis, basis for measure development, and whether the measure applied had an overall classification of levels of use. If the article measured use of specific EHR functions, all functions evaluated and degrees of use were recorded and organized into categories (see Table 1 ).

Appendix C. Charting Strategy.

General characteristics
• Citation
• Author affiliation
• Country of origin
• Type of study
• Setting
• Unit of analysis
• Rationale of study
Measure characteristics
• Data collection method
• Nature of the measure
• Rational for developing measures
• Basis for measure development
• Functionalities measured (If applicable)
o Degree of use for each functionality measured
• Degrees of overall use measured

Table 1. Functions measured in review studies.

Reference	Administrative	Health information	Order generation	Information exchange	Decision support and care management	Patient support	Public health
Belz (2015) 31	X	X		X	X	X
Bowes (2010) 32	X	X			X
COACH (2013) 62	X	X	X	X	X	X	X
Djalali et al (2015) 64	X	X	X	X
Dombrosky (2014) 63	X	X	X	X	X	X	X
Felt-Lisk et al (2010) 56		X	X	X	X
Goetz Goldberg et al (2012) 33	X	X	X	X	X	X	X
Gordon et al (2015) 34	X	X	X	X	X	X
Hogan and Kissam (2010) 35		X	X	X	X	X	X
Hsiao et al (2013) 36		X	X	X	X	X
Lanham et al (2014) 37	X			X	X	X
Lanham et al (2012) 55	X			X	X	X
Li (2011) 38		X	X	X	X	X	X
Makam et al (2013) 39		X	X		X	X
McClellan et al (2013) 40		X			X
Meehan et al (2014) 41	X				X	X
Miller et al (2004) 52	X	X	X	X	X	X
OntarioMD (2015) 21	X	X	X	X	X	X	X
Ornstein et al (2015) 42		X			X	X	X
Paré et al (2015) 57	X	X	X	X	X		X
Pfoh et al (2012) 43	X	X	X	X	X		X
Poon et al (2010) 44		X	X	X	X
Price et al (2011) 22	X	X	X	X	X	X	X
Price et al (2013) 60	X	X	X	X	X	X	X
Rao et al (2011) 50		X	X	X	X
Raymond et al (2015) 58	X	X	X	X	X		X
Riddell et al (2014) 45	X	X	X	X	X	X	X
Rimmer et al (2014) 61	X	X	X	X	X	X	X
Shin and Sharac (2013) 46	X	X	X	X	X	X	X
Simon et al (2008) 51		X	X	X	X		X
Singh et al (2012) 54		X	X	X	X
Tagg (2015) 59	X	X	X	X	X	X	X
Wilcox et al (2008) 53	X	X	X	X	X	X
Wright et al (2014) 47	X	X	X	X	X	X	X
Xiao et al (2012) 48	X	X	X	X	X	X
Zhou et al (2009) 49		X	X	X	X
Total = 36	24 (67%)	33 (92%)	29 (81%)	31 (86%)	35 (97%)	24 (67%)	18 (50%)

Note: Not included is Gillies, 65 in which the stages of use described were broad and specific functions were not listed.

Functional categories were defined as follows:

Administration: Functions used outside clinical practice which do not directly pertain to health information (e.g., billing, scheduling).

Health Information: Viewing and recording patient information (e.g., medical history, visit notes, laboratory results).

Order Generation: Generating prescription, laboratory, imaging, and consultation orders.

Information Exchange: Electronically sending/receiving data with other health care organizations.

Decision Support and Care Management: Functions which enhance or process information to facilitate care and care management (e.g., drug or laboratory-related alerts, care reminders, viewing clinical guidelines).

Patient Support: Use of patient portals, patient education materials.

Public Health: Submitting data to patient registries, public health reporting.

Expert Consultation

In June and July 2016, we conducted four semistructured interviews with seven stakeholders (two of the interviews were conducted with a group of participants) who had EHR expertise. Participants included one clinician researcher, one provincial-level clinician knowledge user, and five knowledge users, all from Canada. In keeping with scoping review methodology, 29 prior to the interviews we circulated an executive summary of the review to participants. In the interviews, we asked participants questions focused on their experience with EHR use, their views on the findings of the scoping review, and their ideas regarding important considerations for measuring EHR use in Canada. The interviews were conducted by A.L.T. and M.Z.H. (see Appendix D ). Each interview was recorded and notes were taken; A.L.T. and M.Z.H. met after each interview to debrief and discuss the results. Ethics approval was received from the Health Sciences Research Ethics Board at The University of Western Ontario.

Appendix D. Interview Guide.

1. Characteristics of the participant

2. Could you share your views, in general, about measuring EHR use?

3. Could you share your thoughts about the preliminary findings of the scoping review?

4. What do you think is important to measure in terms of EHR use?

5. What do you think are important considerations for applying measures of EHR use in a Canadian context?

6. In what direction should further action be taken in regard to measuring EHR use in Canada?

Results

Study Characteristics

Of the 37 articles included, 26 reported on studies that were conducted in the United States. 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 Nine articles were Canadian, 21 22 57 58 59 60 61 62 63 one was from Switzerland, 64 and one was from the United Kingdom. 65 The dates of publication ranged from 2000 to 2015. An increasing trend in articles published per year was noted ( Fig. 2 ). Most studies were cross-sectional ( n  = 25). Six longitudinal studies evaluated changes in EHR use over a period of time. 22 34 36 41 56 61 One text was a discussion article presenting a model for measuring information maturity, 65 and one article featured a comparison of two national surveys. 38

Fig. 2.

Publication year of studies included in the review.

Setting

Thirteen studies were specific to primary care. Ten studies did not specify a setting, but included primary care physicians or practices in their analyses. Some specified a particular category of health organization, such as patient-centered medical homes ( n  = 2), 31 46 small- to medium-sized practices ( n  = 4), 33 40 45 56 and rural primary care practices ( n  = 2). 22 54 The perspectives of the studies reviewed varied in whether they measured EHR use at the level of the practice/clinic ( n  = 16) or at the level of the individual physician ( n  = 15). One study was conducted at the level of physician–nurse teams ( n  = 1). 55

Data Collection Method

Five studies used a mixed-methods approach to collect EHR use data. 33 37 46 52 55 Quantitative data collection methods were most frequently used, and included surveys ( n  = 21) administered by mail, phone, or the Internet. Five studies used electronic data extraction or reporting. 45 46 47 53 63 Electronic reporting includes the analysis of Meaningful Use attestation reports, which were required as of 2011 in the United States. Qualitative data collection methods, including in-person interviews ( n  = 8), 22 33 37 52 55 56 60 61 and onsite observation ( n  = 5), 33 37 41 52 55 were used less often.

Measuring EHR Use

Measuring use through assessing the utilization of individual EHR functions was the most prevalent metric encountered; however, there were a subset of studies which also included other factors involved in EHR use.

Use of EHR Functions

With the exception of Gillies, all studies measured use of specific EHR functions ( n  = 36). The functions fell into the seven broad functional categories of administration ( n  = 24), health information ( n  = 33), order generation ( n  = 29), information exchange ( n  = 31), decision support and care management ( n  = 35), patient support ( n  = 24), and public health ( n  = 18) (see Table 1 ). The specific functions that were encountered most frequently in each category are shown in Table 2 .

Table 2. Top two measured functions within each category.

Function	Number of articles
Administration
Performance reporting	12
Privacy and security	9
Health information
Viewing/recording medication lists	25
Viewing/recording laboratory results	24
Order generation
Prescription generation	18
Laboratory order generation	14
Information exchange
Electronic transmission of health information to other health providers	15
Electronic transmission of prescriptions to pharmacies	14
Decision support
Use of drug-related alerts	14
Use of care reminders	13
Patient support
Generation of patient education materials	11
Use of patient portals	9
Public health
Use of patient registries	8
Public health reporting	7

Metrics for degree of use of specific functionalities were most often binary, distinguishing between use and nonuse of functions ( n  = 15). Likert scales (e.g., “do not use,” “use sometimes,” “use all of the time”) to determine use frequency were also used ( n  = 7), 34 42 44 48 49 50 54 and ranged from 3 to 5 points.

The basis for the creation of the list of EHR functions for evaluation was often not stated. Only a few studies stated that a list of EHR functions was generated via a review of literature ( n  = 3). 57 58 64 Two articles used a set of functionalities that were determined by an expert panel. 35 54 The list of core EHR functions described by the Institute of Medicine was referenced by four studies as a basis for developing the lists used. 22 49 59 60

Other Measures of EHR Use

Other measures of EHR use reported included Makam et al, 39 where methods used by primary care physicians to record encounter notes within the EHR (dictation, free text, prepopulated dot phrases, electronic templates) were analyzed. This study also recorded the amount of time that physicians spent on clinical documentation in the EHR outside of their scheduled clinic time. The measurement framework used in Lanham et al 55 and Lanham et al 37 took into account the frequency that EHR use behaviors changed when new features were introduced.

Frameworks of Overall Measures of EHR Use

Fig. 3 shows our characterization of measures of EHR use. Almost half of the articles ( n  = 18) measured the use of each EHR function individually. In contrast, we identified 19 studies that categorized use of various EHR functions into maturity stages that characterized overall levels of use. Within the 19 studies, 8 distinct frameworks were used. Many were models of general EHR maturity within a practice, and included measures of use. In general, functions directly related to the clinical environment (note-taking, order generation) were at the lowest levels of use, followed by more complex local functions (administration, decision support, patient support), and finally, health-system integration functions (information exchange, public health) at the highest level. The measures of use were categorized based on their origins from either professional/government organizations, or individual authors (see Table 3 ).

Fig. 3.

Typology of measures of electronic medical record (EMR) use.

Table 3. Summary of frameworks for overall EHR use.

Developed by	Name	Articles used	Levels of use categorized	Functionality covered
Centers for Medicare and Medicaid Services—United States	Meaningful Use (Stage 1)	Hsiao et al (2013), 36 Meehan et al (2014), 41 Riddell et al (2014), 45 Shin and Sharac (2013), 46 Wright et al (2014), 47 Xiao et al (2012) 48	Meaningful Use status achieved if eligible provider meets all 15 core objectives and any 5 menu objectives	Core measures  - Uses CPOE for medication orders  - Implements drug to drug and drug allergy interaction checks  - Generates and transmits permissible prescriptions electronically (eRx)  - Records patient demographics  - Maintains an up-to-date problem list of current and active diagnoses  - Maintains active medication list  - Maintains active medication allergy list  - Records and charts changes in vital signs  - Records smoking status patients age 13+ 10.  - Implements one clinical decision support rule  - Reports ambulatory clinical quality measures  - Provides patients with an electronic copy of their health information  - Provides clinical summaries for patients for each office visit  - Exchanges key clinical information among providers of care  - Protects electronic health information Menu measures  - Submits electronic data to immunization registries  - Submits syndromic surveillance data to public health agencies  - Implements drug formulary checks  - Incorporates clinical laboratory test results as structured data  - Generates lists of patients by specific conditions for QI, outreach  - Sends reminders to patients for preventive/follow-up care  - Provides patients with timely electronic access to their health information  - Identifies and provides patient-specific education resources if appropriate  - Performs medication reconciliation at relevant transfers of care  - Provides summary of care record for each transition of care or referral
COACH—Canada	Canadian EMR Adoption and Maturity Model	COACH (2013), 62 Dombrosky (2014), 63 OntarioMD (2015) 21	Level 0: Paper-Based Level 1: Basic Electronic Record Keeping Level 2: Clinical Processes Level 3: Advanced Disease Management Level 4: Integrated Care Level 5: Population Impact	Practice management  - Appointment scheduling  - Practice billing  - Communication and coordination  - Business continuity and planning Information management  - Registration information  - Encounter documentation  - Data quality management  - Nomenclature consistency  - Document management (scanning)  - Privacy and security Patient results management  - Laboratory results  - Diagnostic image reports  - Hospital summary information  - Referrals and consults tracking (physician)  - Referrals and consults tracking (specialist) Diagnosis support  - Patient assessment tools  - Preventative/follow-up care  - Evidence-based resources Treatment planning support  - Care planning and coordination  - Medication management  - Complex care/chronic disease management Patient engagement and communication  - Patient education  - Self-care/comanagement Evaluation and monitoring  - Health quality indicator  - Public health reporting Health outcomes (jurisdictional)
eHealth Observatory—Canada	EMR Adoption Model	Price et al (2011), 22 Price et al (2013), 60 Tagg (2015) 59	Stage 0: Traditional paper-based practice Stage 1: Electronic reference material Stage 2: Partial use of computers at point of care for recording patient information Stage 3: Computer has replaced patient chart Stage 4: Advanced decision support Stage 5: Fully interconnected EMR	- Health information  - Laboratory management  - Diagnostic imaging  - Prescription management  - Referrals  - Decisions support  - Electronic communication  - Patient support  - Administrative processes  - Practice reporting
HIMSS Analytics	Ambulatory EMR Adoption Model	Belz (2015) 31	Stages 0–7	Stage 1:  - Desktop access to clinical information  - Unstructured data  - Multiple data sources  - Infraoffice/informal messaging  Stage 2:  - Desktop access to clinical information  - Orders and results  - Multiple data sources  - Infraoffice/informal messaging  Stage 3:  - Electronic messaging  - Clinical documentation  - Clinical decision support  Stage 4:  - CPOE  - Use of structured data  - Internal and external data sharing  Stage5:  - Personal health record connection  - Patient portal  Stage 6  - Clinical decision support  - Advanced decision support  - Proactive care management  - Structured messaging  Stage 7:  - Health information exchange  - EHR connectivity  - Business and clinical intelligence
Physician Information Technology Office—British Columbia	Clinical Value Model	Rimmer et al (2014) 61	Level 0: Front Office Administration Level 2: EMR Basics Level 3: Full EMR (considered baseline) Level 4: Proactive Care/Data-driven Practice Level 5: Community Shared Care	- Billing  - Scheduling  - Receive labs reports  - Record-keeping capabilities  - Structured data entry  - Registries  - Recalls  - Reminders  - Templates/flowsheets  - Guideline-informed care Data transfer between GP and specialists and other care providers
Lanham et al (2012)	EHR Use Categories	Lanham et al (2012), 55 Lanham et al (2014) 37	1. Low 2. Medium 3. High	- Reports, flow sheets, tracking, and trending features  - EHR-generated patient literature  - Macros/quick text feature  - EHR-enabled communication within practice  - EHR-enabled communication outside practice  - EHR-enabled communication with pharmacies  - Frequency to change EHR use with introduction of new features
Miller et al (2004)	Physician EMR User Types	Miller et al (2004), 52 Wilcox et al (2008) 53	1. Viewer 2. Basic User 3. Striver 4. Arriver 5. System Changer	- Viewing (chart notes, problem and medication lists, discharge summaries, laboratory results  - Order entry (prescriptions, laboratory tests, radiology studies, and referrals)  - Documentation (visit notes, other information)  - Care planning and management  - Patient-directed (patient education materials, patient viewing of EMR data, secure email with care providers)  - Billing and other administrative (insurance eligibility, visit level coding, referral management, and tracking)  - Messaging (email or other messaging system within and outside practice)  - Performance reporting (to internal and external audiences)
Gillies (2000)	General Practice Information Maturity Model	Gillies (2000) 65	Level 0: Paper-based Level 1: Computer system used by practice staff Level 2: Computer system used by practice staff and PHC team Level 3: Limited use of read codes Level 4: Use of coding policies and clinical protocols Level 5: Completely paperless	- NA

Abbreviations: CPOE, computerized physician order entry; EHR, electronic health record; EMR, electronic medical record; GP, general practitioner; PHC, primary health care.

Frameworks by Professional/Government Organizations

Centers for Medicare and Medicaid Services—United States

The Meaningful Use program was created through the 2009 HITECH act in the United States, and is implemented by the Centers for Medicare and Medicaid Services (CMS). The program consists of progressive stages of EHR use, each with requirements that change as EHR adoption matures. Six articles measured EHR use by the Meaningful Use Stage 1 criteria. 36 41 45 46 47 48 No article measured use by the Meaningful Use Stage 2 criteria (announced in 2012).

COACH—Canada

A white article by COACH (Canada's Health Informatics Association), described the Canadian EMR Adoption and Maturity Model (CEMRAMM), that was developed via a compilation of EHR adoption models used in four Canadian jurisdictions (Alberta, British Columbia, Manitoba, and Ontario), and proposed as a common model. 62 Six levels exist within this model, from 0 (article-based) to 5 (population impact). The CEMRAMM is also used in the Saskatchewan EMR Program, 63 and is nearly identical to the EMR Maturity Model used by the Ontario organization, OntarioMD. 21

eHealth Observatory—University of Victoria, Canada

Price et al 22 and Price et al 60 used the EMR Adoption Model developed at the eHealth Observatory at the University of Victoria, Canada. The framework defines six stages of EHR use which were made to be consistent with that of the Health Information Management Systems Society (HIMSS) EMR Adoption Model (EMRAM). Each stage is defined by the use of specific EHR functional categories, which were adapted from the Institute of Medicine's eight key capabilities of EHR systems. The eHealth Observatory EMRAM was also used by Tagg 59 in a survey of EHR use by family practices in Alberta.

Physician Information Technology Office (PITO)—British Columbia, Canada

The Clinical Value Model is a framework developed by the PITO program in British Columbia, which had the goal of supporting province-wide adoption of EHRs. This model consists of five stages, and was used in Rimmer et al. 61

HIMSS Analytics

The Ambulatory EMRAM (A-EMRAM) developed by HIMSS Analytics consists of eight stages of EHR Adoption for practices, from stage 0 (article chart based) to stage 7 (fully integrated and health information exchange-capable EHR). An evaluation of the stages of EHR adoption of patient-centered medical homes in the United States was conducted by Belz using this model.

Frameworks by Individual Authors

Lanham et al

Lanham et al 55 described three (low, medium, and high) categories of EHR use, depending on the degree of EHR feature use, as well as the level of EHR-enabled communication with others both within and outside the clinic. Also measured was the frequency at which physicians changed their usage habits when new features were introduced in the EHR. This model was subsequently used in Lanham et al. 37

Miller et al

In a study of physicians in solo and small practices, Miller et al 52 categorized five progressive EHR user types based on variety and degree of feature use: Viewer, Basic User, Striver, Arriver, and System Changer. Wilcox et al 53 adopted this model to categorize the EHR usage patterns of a larger sample of 3,348 physicians in the Intermountain Healthcare organization.

General Practice Information Maturity Model—United Kingdom

The General Practice Information Maturity Model (GPIMM) is described by Gillies. 65 This model outlines six levels (0–5) of information maturity in a general practice, from article-based to articleless. It takes into account the presence of EHR coding features, as well as which staff members in the practice use them.

Feedback from Stakeholder Consultations

There were two main areas of feedback from participants regarding the scoping review. First, participants in the stakeholder consultation phase of the scoping review identified resources which added to the currency of this review's content. The majority of the new resources that were identified were incorporated into the background section of this article. Second, participants offered new insights and perspectives regarding measuring EHR use, particularly in Canada.

In terms of the scoping review findings, participants noted the small number of studies overall, reflecting the early stage of this body of literature. The dominance of studies from the United States was discussed, as was the need for caution regarding the applicability of these studies in Canada's much different health care context. Participants echoed many of the points raised in the review, including: the importance of making a distinction between simple adoption, and full multifunctional EHR use; the fact that there are disparate existing EHR assessment tools, and that there is a need for a consensus across Canada on how to measure EHR use in a standard way; as well as the importance of measuring capabilities of EHR systems. There were strong views expressed about the need for ongoing nuanced measurement of EHR use (for example, determining what kind of EHR use would be needed for different patient groups, creating measures sensitive to different primary care contexts), and that these measurements should be linked to initiatives to advance EHR use among practitioners. Participants suggested that multiple initiatives could improve EHR use—including incentives in the primary care context, as well as quality improvement activities. The need to identify solutions to improving EHR use, potentially through research, was voiced.

Sharing the summary of the scoping review with stakeholders afforded the authors the opportunity to conduct the first steps of knowledge translation. In turn, the participants shared new resources with the authors and discussed important areas for future research. Taken together, the consultation process served to enhance the scoping review itself, while also affording participants and authors the opportunity for knowledge transfer and exchange.

Discussion

In the past, primary care EHR adoption in Canada lagged behind other countries such as the United Kingdom and the Netherlands. More recently, however, adoption levels among primary care practitioners have increased significantly, though there is variation among provinces. 7 A variety of support programs among the provinces encouraged adoption. 7 However, overall there is limited use of advanced features in the EHR among Canadian primary care practitioners. 6 Understanding advanced EHR use is a complex task, and currently there is a lack of information in the Canadian context in this regard. 57 Given the rising use of EHRs, and the link between advanced use and potential benefits to patient care, 18 20 it is important that we are able to measure EHR use effectively. Given the nature of the Canadian primary care system (e.g., that it is: recognized as the foundation of the system; the first point of contact; where care is coordinated; and where the majority of care is provided), 25 66 67 it is important that measures of EHR use are applicable in this context. Thus, we set out to explore existing EHR use measures in the literature within this review.

This review found a relatively small body of literature on measuring EHR use in primary care. The volume of this literature is developing, and the rate of publication of articles on this topic is increasing as the number of EHR users grows. The current literature focuses on assessing the use of specific functions within the EHR system.

Reviewed articles were relatively evenly divided between the practice and the physician-level perspective. Incentives programs such as Meaningful Use, as well as provincial programs in Canada, give incentives to eligible physicians, and develop measures intended for physician-level use. However, most of the eight frameworks identified in this review can be applied to both perspectives. For example, of the six articles which used the Meaningful Use framework, four were based on a practice-level perspective, and two were physician-level.

Within the articles in this review, surveys were the most widely used method for collecting data on EHR use. For quantitative studies, surveys are appropriate as they can facilitate larger sample sizes. However, survey studies can be subject to bias because they are based on self-report. Further inaccuracies can occur when a single physician is chosen to report on usage data for the entire practice (for example, Singh et al). 51 Direct observation and semistructured interviews can provide a more detailed evaluation of use, but are also more time-intensive. Most notably, this review highlights the potential of electronic reporting to evaluate use. Two articles used EHR audit data to determine levels of use. Three studies relied on Meaningful Use attestation reports, which are also reported electronically through the EHR system. Electronic reporting presents an objective and efficient method to evaluate use; however, the capability to do so must be present in the EHR software. Ongoing certification of EHR systems and vendors can ensure the consistency of functions available. 68

All measures encountered were focused on the use of different EHR functions. The types of functions measured varied between frameworks, with functions in public health, patient support, and administration being the least prevalent. This may be because these functions were perceived to have lesser value to clinical practice, and represent an extension of basic EHR functionality. The literature also shows that other measures of use exist, such as the change in use patterns in response to the addition of new EHR features. These measures should be taken into consideration when evaluating EHR use in the future. A recent development by HIMSS Analytics is the “Continuity of Care Maturity Model,” 69 which outlines the stages of interoperability, information exchange, and care coordination beyond the stages of the HIMSS A-EMRAM. The “Continuity of Care Maturity Model” gauges the next steps toward an interconnected network of care, after the widespread adoption and advanced use of EHRs is achieved.

This review also highlighted the importance of characterizing the overall level of EHR use of a practice or physician. Frameworks such as the HIMSS EMRAM and COACH's EMR Adoption and Maturity Model take a more user-oriented approach which measures overall use by successive levels. The implementation of adoption levels within this type of framework are an improvement over simpler frameworks that categorize advanced EHR use as a binary feature, for example, the CMS Meaningful Use requirements. A stepwise progression from basic to advanced use gives users a roadmap to follow, which can help to alleviate major barriers to the use of EHRs, including a lack of computer literacy in users, 7 and the perceived complexity of the EHR system. 70 Canadian provinces from which frameworks for EHR use were found in the review (BC, ON, SK) are among the provinces with the highest EHR adoption rates. 7

Another important consideration when measuring EHR use is the limitation of the EHR system itself. The degree of function use is limited by what functions exist in the system. As such, use should be evaluated in the context of EHR capability. Only a small number of studies in the review, such as Poon et al 44 and Gordon et al, 34 evaluated both the availability and use of functions. Current EHR product certification programs can ensure that systems adopted by health care providers can support advanced use. Use can also be limited by the infrastructure of the health care system within which the practice or physician is located. Jurisdictions that lack a network for sharing patient information limit the health information exchange capabilities of EHR users. Of note, the relationship between EHR use, EHR capability, and health care infrastructure is considered within the eHealth Observatory's EMR Adoption Framework. 22

Measuring EHR use has important implications in five main areas. The first is that individual users and teams could use an EHR use measure to assess their own level of the use of this technology, potentially supporting a stepwise progression in improvements in use. This could also enable comparisons among groups of users. Second, the motivation to use an EHR is tied to its value in clinical practice. 71 We know that a link exists between levels of EHR use and the potential benefits that may be realized in terms of care and patient outcomes. 18 20 Better measurement of EHR use could further establish the link between types of use and the outcomes that may be achieved, thus emphasizing the value of advanced EHR in practice. Third, understanding EHR use more comprehensively could allow the development of tailored resources for training or the creation of unique support programs among different jurisdictions. Fourth, exploring EHR use may lead to improved knowledge of the factors which impact a practitioner's ability to achieve advanced or improved use. 57 Finally, assessments of patterns and levels of EHR use among practitioners could inform the refining or creation of EHR software products that are better aligned with actual clinical practice. 72

Future Research

While there is mixed evidence of the impact of EHR use on clinical practice, 73 early research points to the importance of more mature, or advanced EHR use, 18 including integration with workflow, 20 in achieving positive effects. Further research could explore not only the impact of advanced EHR use on clinical practice and patient outcomes, but also the link between the use of specific EHR functions and their potential effects. With growing initiatives to assess the impact and uptake of EHRs, EHR research relating to care outcomes, barriers to use, and user satisfaction should apply a more nuanced view of EHR use. This more nuanced picture of use could capture the way a clinician uses the EHR during a patient encounter, for example, not just simply determining if an individual has used an EHR function or not. New research should therefore continue to explore aspects of EHR use by practitioners that go beyond assessing the use of individual software functions. Possibilities include exploring best practices in EHR use when interacting with patients, analyzing individual's use patterns, and examining changes in use behavior over time. Many unanswered questions remain, such as the appropriate use of EHR measures given different settings and systems, and ultimately, how best to help practitioners advance their use of EHRs.

Limitations

Additional studies relating to measuring EHR use in Canada have been released following completion of this review. 74 75 There may be studies missed by the manual gray literature search. Finally, the categorization of functions was created by A.L.T. and M.Z.H.; other categorizations of EHR functions exist. 22

Conclusion

With Canada and other select countries reaching high levels of EHR adoption, it is important to understand how these systems are being used in primary care. This scoping review explored the body of literature available regarding measuring the use of EHRs within primary care. We found that this literature is still in its early stages, but rapidly developing. Although certain frameworks were recurrent, many researchers are creating de novo measures. Therefore, there is a need to create more common, widely used, and validated framework for measuring EHR use. Creating a unified, multidimensional use assessment framework, and applying it consistently across jurisdictions, would have several benefits including: facilitating comparisons of levels of EHR use across primary care settings, supporting the identification of factors associated with higher and lower levels of EHR use, and limiting confusion about what framework to apply, thus simplifying the process of evaluating use.

This would also address the limitation of current frameworks which are mostly only applied in a limited capacity, and focus almost exclusively on use of EHR functions. Existing frameworks also do not consider the context of EHR system capability, which limits the degree of possible use. Most of the literature is focused on measuring use of specific EHR functions, and there is more work to be done in exploring other aspects of EHR use, such as how this technology is used during patient encounters, team versus individual use of the EHR, and the way different primary care practitioners use the EHR. We recommend that a nuanced view of EHR adoption be used in future studies, taking into account the variety of ways this technology is used in practice.

Clinical Relevance Statement

As adoption of EHRs in select countries increases, focus has shifted to measuring the use of these systems. By compiling and classifying current EHR use metrics, this review highlights the facets and nuances that must be considered when measuring EHR use. An understanding of these existing measures is essential to the development of unified and comprehensive EHR use metrics, which can facilitate the optimal use of EHRs by health care practitioners and organizations.

Multiple Choice Question

Within the current literature on that measures the use of EHR functions, which category of functionality is most often neglected?

1. Order Generation

2. Public Health

3. Administration

4. Patient Support

Correct Answer: The correct answer is b. Our review found that Public Health EHR functions were the most neglected, with only 18 of the 36 articles which measured use of EHR functions including this category (see Table 1 ).

Appendix A Database Search Queries

PubMed

(“general practice”[All Fields] OR “general practitioners”[All Fields] OR “general practitioner”[All Fields] OR “family physicians”[All Fields] OR “family physician”[All Fields] OR “family practice”[All Fields] OR “primary health care”[All Fields] OR “primary care”[All Fields] OR “primary healthcare”[All Fields] OR “internal medicine”[All Fields] OR “internist”[All Fields] OR “paediatrics”[All Fields] OR “pediatrics”[All Fields] OR “paediatrician”[All Fields] OR “pediatrician”[All Fields] OR Ambulatory[All Fields] OR “general practice”[MeSH Terms] OR “general practitioners”[MeSH Terms] OR “physicians, family”[MeSH Terms] OR “family practice”[MeSH Terms] OR “primary health care”[MeSH Terms] OR “primary health care”[MeSH Terms] OR “internal medicine”[MeSH Terms] OR “pediatrics”[MeSH Terms])

AND

(“computerised medical records”[All Fields] OR “computerised health records”[All Fields] OR “computerised patient records”[All Fields] OR “computerized medical records”[All Fields] OR “computerized health records”[All Fields] OR “computerized patient records”[All Fields] OR “electronic medical records”[All Fields] OR “electronic health records”[All Fields] OR “electronic patient records”[All Fields] OR “medical records systems, computerized”[MeSH Terms] OR “electronic health records”[MeSH Terms])

AND

(“Meaningful Use”[All Fields] OR “Advanced Use”[All Fields] OR “Maturity Model”[All Fields] OR “Adoption Model”[All Fields] OR “Clinical Value” OR “Clinical Quality”[All Fields] OR “CQM”[All Fields] OR “CQMs”[All Fields] OR “Multifunctional Electronic”[All Fields] OR “meaningful use”[MeSH Terms])

PubMed (with PubMed EHR search terms): https://www.nlm.nih.gov/services/queries/ehr.html )

(“general practice”[All Fields] OR “general practitioners”[All Fields] OR “general practitioner”[All Fields] OR “family physicians”[All Fields] OR “family physician”[All Fields] OR “family practice”[All Fields] OR “primary health care”[All Fields] OR “primary care”[All Fields] OR “primary healthcare”[All Fields] OR “internal medicine”[All Fields] OR “internist”[All Fields] OR “paediatrics”[All Fields] OR “pediatrics”[All Fields] OR “paediatrician”[All Fields] OR “pediatrician”[All Fields] OR Ambulatory[All Fields] OR “general practice”[MeSH Terms] OR “general practitioners”[MeSH Terms] OR “physicians, family”[MeSH Terms] OR “family practice”[MeSH Terms] OR “primary health care”[MeSH Terms] OR “primary health care”[MeSH Terms] OR “internal medicine”[MeSH Terms] OR “pediatrics”[MeSH Terms])

AND

((health information exchange[tw] OR hie[tw] OR rhio[tw] OR regional health information organization[tw] OR hl7[tw] OR health level seven[tw] OR “unified medical language system”[MeSH Major Topic] OR umls[tw] OR loinc[tw] OR rxnorm[tw] OR snomed[tw] OR icd9 cm[ti] OR icd 9 cm[ti] OR icd10[ti] OR icd 10[ti] OR metathesaurus[tw] OR patient card[tw] OR patient cards[tw] OR health card[tw] OR health cards[tw] OR electronic health data[tw] OR personal health data[tw] OR personal health record[tw] OR personal health records[tw] OR “health records, personal”[MeSH Major Topic] OR “health records, personal”[MeSH Major Topic] OR ehealth[tw] OR e-health[tw] OR “medical informatics applications”[MeSH Terms] OR “medical informatics applications”[MeSH Terms] OR “medical records systems, computerized”[MeSH Terms] OR “medical records systems, computerized”[MeSH Terms] OR computerized patient medical records[tw] OR automated medical record system[tw] OR automated medical record systems[tw] OR automated medical records system[tw] OR automated medical records systems[tw] OR computerized medical record[tw] OR computerized medical records[tw] OR computerized patient records[tw] OR computerized patient record[tw] OR computerized patient medical record[tw] OR electronic health record[tw] OR electronic health records[tw] OR “electronic health records”[MeSH Major Topic] OR “electronic health records”[MeSH Major Topic] OR electronic patient record[tw] OR electronic patient records[tw] OR electronic medical record[tw] OR electronic medical records[tw] OR electronic healthcare records[tw] OR electronic healthcare record[tw] OR electronic health care record[tw] OR electronic health care records[tw] OR “archives”[MeSH Major Topic] OR ehr[tw] OR ehrs[tw] OR phr[tw] OR phrs[tw] OR emr[tw] OR emr[tw] OR “health information systems”[MeSH Major Topic]) AND (medical record[ti] OR “medical records”[MeSH Terms] OR medical records[ti] OR patient record[ti] OR patient records[ti] OR patient health record[ti] OR patient health records[ti] OR “patient identification systems”[MeSH Terms] OR “patient identification systems”[MeSH Terms] OR “patient outcome assessment”[MeSH Major Topic] OR “patient discharge summaries”[MeSH Major Topic] OR healthcare record[ti] OR healthcare records[ti] OR health care record[ti] OR health care records[ti] OR health record[ti] OR health records[ti] OR hospital information system[tw] OR hospital information systems[tw] OR umae[ti] OR “attitude to computers”[MeSH Terms] OR medical informatics[ti])) OR ((“medical records systems, computerized”[MeSH Major Topic] OR “medical records systems, computerized”[MeSH Terms] OR computerized patient medical record[tw] OR computerized patient medical records[tw] OR automated medical record system[tw] OR automated medical record systems[tw] OR automated medical records system[tw] OR automated medical records systems[tw] OR computerized medical record[tw] OR computerized medical records[tw] OR computerized patient records[tw] OR computerized patient record[tw] OR electronic health record[tw] OR electronic health records[tw] OR electronic patient record[tw] OR electronic patient records[tw] OR electronic medical record[tw] OR electronic medical records[tw] OR electronic healthcare records[tw] OR electronic healthcare record[tw] OR electronic health care record[tw] OR electronic health care records[tw] OR “unified medical language system”[MeSH Major Topic] OR unified medical language system[tw] OR umls[tw] OR loinc[tw] OR rxnorm[tw] OR snomed[tw] OR icd9 cm[ti] OR icd 9 cm[ti] OR icd10[ti] OR icd 10[ti] OR Metathesaurus[tw] OR ehr[tw] OR ehrs[tw] OR phr[tw] OR phrs[tw] OR emr[tw] OR emrs[tw] OR meaningful use[tiab] OR meaningful use[tw] OR “meaningful use”[MeSH Major Topic]) AND (“J AHIMA”[Journal] OR “J Am Med Inform Assoc”[Journal] OR “AMIA Annu Symp Proc”[Journal] OR “Health Data Manag”[Journal] OR “Int J Med Inform”[Journal] OR “Yearb Med Inform”[Journal] OR “Telemed J E Health”[Journal] OR “Stud Health Technol Inform”[Journal]))

AND

(“Meaningful Use”[All Fields] OR “Advanced Use”[All Fields] OR “Maturity Model”[All Fields] OR “Adoption Model”[All Fields] OR “Clinical Value” OR “Clinical Quality”[All Fields] OR “CQM”[All Fields] OR “CQMs”[All Fields] OR “Multifunctional Electronic”[All Fields] OR “meaningful use”[MeSH Terms])

Cochrane

(“general practice”[All Fields] OR “general practitioners”[All Fields] OR “general practitioner”[All Fields] OR “family physicians”[All Fields] OR “family physician”[All Fields] OR “family practice”[All Fields] OR “primary health care”[All Fields] OR “primary care”[All Fields] OR “primary healthcare”[All Fields] OR “internal medicine”[All Fields] OR “internist”[All Fields] OR “paediatrics”[All Fields] OR “pediatrics”[All Fields] OR “paediatrician”[All Fields] OR “pediatrician”[All Fields] OR Ambulatory[All Fields] OR “general practice”[MeSH Terms] OR “general practitioners”[MeSH Terms] OR “physicians, family”[MeSH Terms] OR “family practice”[MeSH Terms] OR “primary health care”[MeSH Terms] OR “primary health care”[MeSH Terms] OR “internal medicine”[MeSH Terms] OR “pediatrics”[MeSH Terms])

AND

(“computerised medical records”[All Fields] OR “computerised health records”[All Fields] OR “computerised patient records”[All Fields] OR “computerized medical records”[All Fields] OR “computerized health records”[All Fields] OR “computerized patient records”[All Fields] OR “electronic medical records”[All Fields] OR “electronic health records”[All Fields] OR “electronic patient records”[All Fields] OR “medical records systems, computerized”[MeSH Terms] OR “electronic health records”[MeSH Terms])

AND

(“Meaningful Use”[All Fields] OR “Advanced Use”[All Fields] OR “Maturity Model”[All Fields] OR “Adoption Model”[All Fields] OR “Clinical Value” OR “Clinical Quality”[All Fields] OR “CQM”[All Fields] OR “CQMs”[All Fields] OR “Multifunctional Electronic”[All Fields] OR “meaningful use”[MeSH Terms])

CINAHL

(“General Practice” OR “General Practitioner” OR “Family Physician” OR “Family Practice” OR “Family Practitioner” OR “Primary Health Care” OR “Primary Care” OR “Primary Healthcare” OR “Internal Medicine” OR “Internist” OR “paediatrics” OR “pediatrics” OR “paediatrician” OR “pediatrician” OR “Ambulatory” OR (MH “Physicians, Family”) OR (MH “Primary Health Care”) OR (MH “Internal Medicine”) OR (MH “Pediatricians”) OR (MH “Pediatrics”))

AND

(“computerised medical records” OR “computerised health records” OR “computerised patient records” OR “computerized medical records” OR “computerized health records” OR “computerized patient records” OR “electronic medical records” OR “electronic health records” OR “electronic patient records” OR (MH “Computerized Patient Record”) OR (MH “Patient Record Systems”) OR (MH “Clinical Information Systems”))

AND

(“Meaningful Use” OR “Advanced Use” OR “Maturity Model” OR “Adoption Model” OR “Clinical Value” OR “Clinical Quality” OR “CQM” OR “CQMs” OR “Multifunctional Electronic” OR (MH “Meaningful Use”))

Web of Science

(“General Practi*” OR “Family Physician*” OR “Family Practi*” OR “Primary Health Care” OR “Primary Care” OR “Primary Healthcare” OR “Internal Medicine” OR “Internist*” OR “paediatric*” OR “pediatric*” OR “Ambulatory”)

AND

(“Medical Record* System*” OR “Health Record* System*” OR “Patient Record* System*” OR “Computeri?ed Patient Record*” OR “Computeri?ed Medical Record*” OR “Computeri?ed Health Record*” OR “Electronic Patient Record*” OR “Electronic Medical Record*” OR “Electronic Health Record*”)

AND

(“Meaningful Use” OR “Advanced Use” OR “Maturity Model” OR “Adoption Model” OR “Clinical Value” OR “Clinical Quality” OR “CQM” OR “CQMs” OR “Multifunctional Electronic”)

Proquest Theses and Dissertations

(“General Practi*” OR “Family Physician*” OR “Family Practi*” OR “Primary Health Care” OR “Primary Care” OR “Primary Healthcare” OR “Internal Medicine” OR “Internist*” OR “paediatric*” OR “pediatric*” OR “Ambulatory”)

AND

(“Medical Record* System*” OR “Health Record* System*” OR “Patient Record* System*” OR “Computeri?ed Patient Record*” OR “Computeri?ed Medical Record*” OR “Computeri?ed Health Record*” OR “Electronic Patient Record*” OR “Electronic Medical Record*” OR “Electronic Health Record*”)

AND

(“Meaningful Use” OR “Advanced Use” OR “Maturity Model” OR “Adoption Model” OR “Clinical Value” OR “Clinical Quality” OR “CQM” OR “CQMs” OR “Multifunctional Electronic”)