Abstract
Background
Electronic medical record (EMR) adoption in primary care has grown exponentially since their introduction in the 1970s. However, without their proper use benefits cannot be achieved. This includes: 1) the complete and safe documentation of patient information; 2) improved coordination of care; 3) reduced errors and 4) more involved patients. The use of EMRs is defined by practitioners using EMRs and their features to perform daily practice functions.
Objective
The purpose of this systematic review was to identify interventions aimed at improving EMR use in primary healthcare settings.
Methods
Ten online databases were searched to identify studies conducted in primary healthcare settings aimed at implementing interventions to observe the use of EMRs and directly measure the use of EMR functions or outcomes effected by the use of EMR functions.
Results
Of 2098 identified studies, 12 were included in the review. Results showed that interventions focused on the use of EMR functions, including referrals, electronic communication, reminders, use of clinical decision support systems and workflow management support functions, were five times more likely to show improvements in EMR use compared with controls. Interventions focused on data quality were five and a half times more likely to show improvements in EMR use compared with controls.
Conclusions
Individuals in primary healthcare settings aiming to improve EMR use would benefit from implementing interventions focused on EMR feature add-ons such as clinical decision support systems and customised referral templates, and provisions of educational materials, or financial incentives targeted at improving the use of EMR functions and data quality.
Keywords: electronic medical records, primary healthcare, benefits realisation, health information technologies
Summary.
What is already known?
The electronic medical record (EMR) system was developed originally in the early 1970’s as a means to store patient health information. Over time, and with the improvements in technology, EMRs are now capable of using stored patient health information to assist in the daily care provisions primary health care personnel provide to patients. This is done with the hopes of improving patient health care through creating higher quality patient data and improving primary health care center processes.
However, whether EMR use has been successful in improving the provision of patient care has not been made clear in the literature. Due to the importance of improving the use of EMRs with regard to patient outcomes, there has been recent interest on the part of organizations and governments to provide guidelines to improve EMR use. Improving EMR use requires targeted interventions aimed at the areas in which EMRs were created to function. There have only been a few studies in the literature that have been aimed at studying the effect of certain types of intervention on EMR use.
What does this paper add?
However, due to the high heterogeneity between those studies a meaningful meta-analysis was not feasible until this point where the synthesis of the results was possible through: the categorization of interventions using the EPOC taxonomy of interventions and the identification of possible intervention target areas to improve EMR use. This allowed for the meaningful grouping of the studies resulting in the ability to conduct the first meta-analysis in this field. This increases the power of the results and conclusions drawn from those results giving us a better understanding of the types of interventions that could be used to improve the use of EMRs.
Introduction
The past few decades have seen an expansion in the role of technology in healthcare reflected in the introduction of information technologies into the healthcare system. Electronic medical records (EMRs) are one form of information technology which can impact patient health outcomes.1 2 EMRs are computerised patient records introduced in the early 1970s as a way to organise, secure, complete and improve the quality of patient healthcare records.3 Around the turn of the century, EMRs gained attention because of the perceived and expected benefits they could offer the healthcare system which included: organisation of patient healthcare information, improving coordination of care as well as easier electronic access to medical information and expert opinion.4 5 With their creation and introduction into primary healthcare, EMRs were expected to have a positive impact on the quality of healthcare. This was expected to be realised through the use of EMRs to improve data quality through the recording of patient information and perform primary healthcare functions. This drove organisations and governments to create programme to promote the adoption of EMRs into the healthcare system.4
The distinction between EMR adoption and use is not clearly defined in the literature. However, for the purposes of this review, adoption of EMRs is defined as simply the introduction of EMRs into primary healthcare practice. The use of EMRs is the second step following adoption, involving practitioners using EMRs and their features to perform daily primary healthcare practice functions. A national survey in 2015 showed that the adoption of EMRs into primary healthcare practices is on the rise in Canada while EMR use is still low in comparison.6 7 The US Centers for Medicare and Medicaid Services (CMS) defined meaningful use as: ‘Using (EMRs) to: improve quality, safety, efficiency, and reduce health disparities. Engage patients and family. Improve care coordination, and population and public health. Maintain privacy and security of patient health information’.8 For the purposes of this review, improved EMR use is defined as using EMRs according to the above definition. Some studies suggest that to achieve noticeable improvements in patient health outcomes following adoption, improving the use of EMRs is necessary.9–12 Therefore, improving the use of EMRs to achieve desirable health outcomes has attracted recent attention.13 The mechanisms to improving EMR use however, have not yet been determined. Therefore, this systematic review aims to review the literature to identify interventions and their effect on improving EMR use in primary healthcare settings in hopes of bridging the gap between adoption and use of EMRs.
Methods
Search strategy
The search strategy was developed with limits to only include studies with human subjects conducted after 1970, the following databases were searched: MEDLINE Ovid, EMBASE Ovid, CINAHL, Cochrane Library and Web of Science. The grey literature was also searched through the following databases: Clinical Trials, NDLTD, CADTH, International Clinical Trials Registry, AHRQ. Finally, after applying the search strategy to all the mentioned databases and collecting the identified studies, snowballing was used as a supplementary search strategy. The full search strategy for all databases is listed in online supplementary appendix A.
bmjhci-2019-000023supp001.pdf (77.2KB, pdf)
Inclusion criteria
Research articles conducted after 1970 and published in the English language regardless of the location of the study. In addition, the following eligibility criteria were used to identify studies for inclusion:
Study focus: included studies were those that specifically focused on actual use of EMRs in primary healthcare, not simply earlier stages of implementation.
Intervention: studies with a clear intervention that was implemented or observed for the purpose of studying use or use patterns of EMRs were included.
Setting: included studies were only those conducted in a primary healthcare setting.
Outcome of interest: included studies had to have an outcome of interest related to measurements of use of EMR functions (number of uses, duration of use) as well as outcomes effected by EMR use such as number of referrals and completeness of patient records.
No restrictions based on study design or comparator groups were used. Opinion pieces, editorials and publications without an abstract were excluded, along with conference abstracts. After conducting the database searches, EPPI Reviewer V.4.0 (by EPPI-Centre, Social Science Research Unit, the Institute of Education, the University of London, UK) was used to automatically remove duplicates; subsequently, a manual search was conducted to remove any missed duplicates.14 The abstracts were screened independently by two reviewers, NH and MH, based on a list of screening questions derived from the eligibility criteria described above. Two reviewers, NH and AT, then independently conducted the full-text screening of the included studies. All screening questions are listed in online supplementary appendix B.
bmjhci-2019-000023supp002.pdf (37.6KB, pdf)
Data extraction
The first author’s name, year of the study and setting (location and country) were extracted to be used as study citation information. Information on the study population and participant composition was also extracted. In addition, extracted from each study were: intervention name, intervention type and a brief description of the intervention. In terms of outcomes, the outcome measured and a description of the outcome were also extracted. Lastly, information was extracted to allow for assessment of individual study bias.
Details of study interventions
A system was adopted in this review to categorise the wide variety of possible interventions that could be implemented to improve EMR use. Interventions for this systematic review were categorised using the Effective Practice and Organisation of Care (EPOC) taxonomy of interventions, which was published in the Cochrane Review by the EPOC Group in 2015.15 Interventions included in this review were placed into one of the following categories and are shown in figure 1:
Professional interventions: defined by EPOC as an intervention implemented with the goal of educating or furthering the knowledge of the target group in a specific area with the purpose of creating change.
Organisational interventions: defined by EPOC as interventions that target workflow, as well as those that create changes in an organisation’s framework. Therefore, included are any interventions that cause changes to the workflow of the primary healthcare practice through the healthcare professionals or structurally through changes to the EMR such as feature add-ons would be included.
Financial interventions: according to the EPOC definition, interventions were considered to be financial interventions if they provided an incentive for action.
Intervention target areas
This study is the first of its kind in this area of research (the first meta-analysis to study the effect of interventions on the use of EMRs). Outcomes for this study were grouped into target areas identified from the definition of EMR use which focuses on the importance of practitioners using EMRs and their features to carry out daily practice function and to maintain a level of patient record quality. These outcomes were measured in a similar population with similar interventions at similar settings and due to those similarities were grouped into target areas using a technique aimed at combining units of the outcomes.16 17 This technique has been used to group complex interventions in the literature and was adopted to be used in this review.17 18 For the purposes of this study, the areas targeted for change were called target areas and were made up of the identified units based on the defined purpose of EMRs to create a logical and meaningful way to group the complex outcomes identified from the included studies. Two target areas were identified in the included studies:
Use of EMR functions: describes the use of EMR functions directly in relation to duration and frequency of use. Examples of the functions include referrals, electronic communication, reminders triggered, use of clinical decision support systems as well as workflow management support functions.
Data quality: studies that described the level of data completeness for basic patient information including diagnostic, laboratory and prescription management information were included in this group.
The outcomes presented in the included studies were grouped by the target area of the intervention into either: 1) use of EMR functions or 2) data quality.
Statistical analysis
Using the above target areas as a guide, the effect sizes from individual studies were combined to create a summary effect size. Studies in which the interventions were aimed at the use of EMR functions all measured the use of those functions through reporting durations and frequency of use allowing for a meaningful summary effect estimate to be created. Similarly, studies with interventions aimed at improving data quality all described the levels of patient data completeness and were therefore grouped to create a single summary effect size. The outcome measures in each individual study were transformed into ORs to be included in the analysis using the data presented by the studies to generate 2-by-2 tables. Studies with multiple outcomes related to the same intervention target area were combined to be included into the analysis. Following that, the studies were separated by intervention target area into two different forest plots to create a meaningful meta-analysis. The statistical analysis including forest and funnel plots was completed using STATA V.13.0 (STATA, College Station, Texas, USA).19 All results were presented in forest plots and expressed in log-ORs because of the categorical nature of the outcomes of interest, using 95% CIs. Frequencies of outcomes along with the total number of participants were extracted. Some studies presented outcomes using regression coefficients. In this case, these coefficients were converted to ORs using the ln function.20 In addition, the ORs of the included studies were presented with their SEs in funnel plots to assess publication bias.
The random-effects model was used to conduct the meta-analysis due to its ability to account for in between study variation that arises from differences in study target population, study intervention and presentation of outcomes. It does that by assuming the true effect estimate varies between studies. Therefore, the random-effects model using the DerSimonian and Laird methods was used in STATA to create the forest plots.21
Risk of bias assessment
The Downs and Black tool was used to assesses risk of bias for individual studies included in this review. It is made up of 27 questions divided into subsections based on reporting, external validity, internal validity (bias and confounding) and power.22 The breakdown of the four subsections and a brief explanation of their importance are listed in online supplementary appendix C. The Downs and Black assessment scale was applied to the selected studies to determine the reliability, validity and power of the study. The Downs and Black checklist for bias assessment is presented in online supplementary appendix D. Scores were then calculated and combined into a risk of bias bar graph, as suggested by the Cochrane Handbook for Systematic Reviews, used to indicate the level of bias in each study.16
bmjhci-2019-000023supp003.pdf (14.8KB, pdf)
bmjhci-2019-000023supp004.pdf (99.2KB, pdf)
Results
After searching the databases in October 2015, 2098 abstracts were identified. From these 2098 abstracts, 659 duplicates were removed. This left 1439 titles for abstract screening. Following abstract screening, 19 studies were identified for full-text screening. Full-text screening was performed on the 19 retrieved studies. Twelve were identified that fit the previously specified inclusion criteria.23–34 The PreferredReporting Items for Systematic Reviews and Meta-Analyses flow chart was used to map out the study selection process as shown in figure 2. 35All 12 studies identified in this review were identified from initial electronic database screening.
Study characteristics
Of the identified studies more than half (n=7) were conducted in the USA.23 26–28 30 31 33 The remaining five were set in the UK (n=2),24 25 Finland (n=2)29 34 and Canada (n=1).32 All the included studies were conducted in a primary healthcare setting. In terms of study population size, the 12 included studies targeted 1564 primary care providers in 132 primary healthcare practices and included 578 071 patients. The characteristics of the included studies and the target population are listed in table 1. Of the 12 included studies, 4 were classified as purely organisational interventions27–30 and 5 were classified as professional interventions.23 24 32 34 The remaining three studies implemented mixed interventions, including at least two of the following: organisational, professional and financial.24 30 33 The studies, interventions and a brief description are listed in table 2. Eight of the included studies reported on the use of EMR functions, a more in-depth description of the EMR function and the outcome reported, along with an outcome measurement description is presented in table 3. All the studies in the data quality group focused on the effect of an intervention on improving the recorded data using an EMR.24–27 30 33 34 The completeness and accuracy of patient information are some of the markers used to measure data quality and were the areas most targeted by the included studies.35 A more in-depth description of the data quality area reported on and the outcome along with an outcome measurement description is presented in table 4.
Table 1.
Author | Setting | Study design | Number of Primary Care Providers (PCPs) | Composition | Number of patients |
Jerome et al40 | 1 primary healthcare centre Country: USA |
Prospective observational | 137 | Attending and resident physicians | * |
de Lusignan et al23 | *Primary healthcare centres Country: UK |
Retrospective observational | 576 | * | * |
de Lusignan et al24 | 84 primary healthcare centres Country: UK |
Quasi-experimental | 252 | 84 physicians 84 nurses 84 managers |
~20 000 19 470 preintervention 19 784 postintervention |
Pan et al25 | 2 family medicine residency training clinics Country: USA |
Quasi-experimental | 8 | 4 certified medical assistants 4 nurses |
525 patients 279 preintervention 246 postintervention |
Baer et al26 | 5 primary healthcare centres Country: USA |
Quasi-experimental | * | * | 15 495 |
Mavigilia et al27 | 18 outpatient clinics Country: USA |
Quasi-experimental | 359 | 187 physicians 108 nurses 64 other |
413 417 |
Kortteisto et al28 | 1 primary healthcare centre Country: Finland |
Randomised controlled trial | 48 | 15 physicians 24 nurses 9 other |
13 588 |
Nemeth et al29 | 8 primary healthcare centres Country: USA |
Mixed methods | 74 | * | 66 104 |
Kruse et al30 | 2 primary healthcare centres Country: USA |
Mixed methods | 36 | 21 physicians 3 nurses 12 physician trainees |
2894 |
Maddocks et al31 | 9 primary healthcare centres Country: Canada |
Randomised controlled trial | 24 | Physicians | 23 688 |
Davis et al32 | 1 primary healthcare centre Country: USA |
Retrospective observational | 36 | Residents | 360 patients 180 preintervention 180 postintervention |
Sweeney et al33 | 1 primary healthcare centre Country: Ireland |
Randomised controlled trial | 16 | 10 physicians 6 nurses |
22 000 |
*Represent missing data.
Table 2.
Author | Intervention | Intervention description |
Baer et al26 | Organisational intervention: web-based appraisal tool |
|
Mavigilia et al27 | Organisational intervention: KnowledgeLink |
|
Kortteisto et al28 | Organisational intervention: computer-based decision support system EBMeDS |
|
Kruse et al30 | Organisational intervention: electronic one-click referral button to tobacco use control centre |
|
Jerome et al40 | Professional intervention: focus groups driven by customised educational strategies |
|
de Lusignan et al24 | Professional Intervention: Primary Care Data Quality (PCDQ) Programme |
|
Pan et al25 | Professional intervention: feedback and education |
|
Maddocks et al31 | Professional intervention: 2-hour educational session |
|
Sweeney et al33 | Professional intervention: data management strategy |
|
de Lusignan et al24 | Mixed interventions: feedback of data quality markers and financial incentives |
|
Nemeth et al29 | Mixed interventions: electronic standing orders provided by a customised health template |
|
Davis et al32 | Mixed interventions: asthma template along with lectures and tutorials |
|
Table 3.
Author | Outcome | EMR feature | Outcome measurement description |
Jerome et al40 | Per cent change in use of EBM literature request | Decision support |
|
Baer et al26 | Per cent of new EMR generated reminders on colon and breast cancer screening | Decision support |
|
Mavigilia et al27 | Frequency of use of KnowledgeLink | Decision support |
|
Kortteisto et al28 | Change in number of reminders triggered | Alerts and reminders |
|
Nemeth et al29 | Per cent of nurses and nursing staff using the health maintenance template | Health template |
|
Kruse et al30 | Per cent of referrals through EMR to tobacco use control centre | Exchange of patient healthcare information |
|
Maddocks et al31 | Change in provided preventive care testing | Exchange of patient healthcare information |
|
Davis et al 32 | Per cent use of asthma template | Health template |
|
EMR, electronic medical record.
Table 4.
Author | Outcome | Data quality area | Outcome measurement description |
de Lusignan et al23 | Change over time in the score of 10 data quality markers | 10 data quality markers |
|
de Lusignan et al24 | Per cent change of completed patient records in blood pressure, cholesterol, smoking habits and patients asked to stop smoking | Completeness of patient information |
|
Pan et al25 | Per cent of new patient height, weight and blood pressure records that were complete | Completeness of patient information |
|
Baer et al26 | Per cent of new coded patient data of family history of cancer | Completeness of patient information |
|
Nemeth et al29 | Per cent of new coded patient data | Completeness of patient information |
|
Davis et al32 | Per cent documentation of asthma severity | Completeness of patient information |
|
Sweeney et al33 | Proportion of primary healthcare provider notes that were coded using the International Classification of Primary Care (ICPC-02) system | ICPC-02 coding system |
|
Meta-analysis results.
EMR, electronic medical record.
The study by de Lusignan et al24 was excluded from the meta-analysis because the results of the study were presented using regression coefficients and were not consistent with the other included studies.24 In addition, Kruse et al30 did not present preintervention scores and was therefore excluded from the meta-analysis.31 All other studies included in the analysis presented enough data to calculate ORs and SEs to assess publication bias.
Two forest plots were generated by STATA using the ORs. The first forest plot represents studies focused on the use of EMR functions as the intervention target area, displayed in figure 3. The overall effect estimate was a significant log-odds of 1.66 (95% CI 1.43 to 1.88; absolute OR=e1.66=5.26, rounded to 5). Therefore, personal, organisational and financial interventions directed at the use of EMR functions had a significant and favourable effect on improving EMR use. More specifically, interventions targeted at the use of EMR functions were five times more likely to show improvements in EMR use compared with the controls (figure 3). The second forest plot represents studies focused on data quality as the intervention target area and is presented in figure 4. All the studies depicted in this forest plot favoured the intervention. The overall effect estimate was a significant log-odds of 1.71 (95% CI 0.01 to 3.41; absolute OR=e1.71=5.53, rounded to 5.5). Therefore, personal, organisational and financial interventions directed at data quality had a significant and favourable effect on improving EMR use five and a half times more than the controls.
To evaluate the publication bias, two separate funnel plots for the use of EMR functions and data quality were produced using STATA (see figures 5 and 6, respectively) and then visually assessed for symmetry. Both funnel plots showed that studies were clustered at the top with only one each at the base of the funnel plot. This asymmetry in the funnel plot could be the result of publication bias. However, due to the small number of studies it is difficult to confidently exclude the presence of publication bias.
The risk of bias for individual studies showed a high reporting of results scores and low internal validity scores. The mean score for risk of bias in individual studies is 64% with an IQR of 60%. This shows a moderate risk of bias in the included studies as expressed in figure 7.
Discussion
The systematic review and meta-analysis were conducted to identify possible interventions focused on improving EMR use in primary healthcare settings. This review indicates that significant improvements in EMR use can be realised in primary healthcare settings where interventions targeting the use of EMR functions or data quality have been implemented. However, due to the possibility of publication bias, these results should be interpreted with caution. The findings of this review draw attention to four main themes in this area of study. Those themes are listed and discussed below.
Number of identified studies
In this review, only 12 studies of interventions focused on improving EMR use in primary healthcare were identified. Primary healthcare settings directly influence the majority of Canadians’ health outcomes.36 The importance of a well-functioning primary healthcare system was not reflected in the literature. Compared with the impact of this area on the health of the general population, the number of identified studies is surprisingly lacking. The deficiency in studies in the area of EMR use is possibly due to the focus in the field being on the adoption of EMRs. Even though studies have shown that adoption alone is not enough to access the EMR’s full potential, the shift to focus on improving EMR use is slow.8 11 12 36 In conclusion, one of the main hopes of this review is to draw attention to this gap in the literature. There should be a greater focus in the area of studies that can connect EMR availability to positive patient outcomes through improving EMR use with targeted interventions.
Lack of consistency
The area of EMR use is deficient in terms of available literature, and in the usability of this literature due to its lack of consistency in the information provided. Studies on the topic of EMR use vary in terms of interventions and approaches to assessing EMR use. Due to this being a relatively new field of study, there has been no standardisation of implementing interventions to improve EMR use established. This creates difficulties in synthesising those studies to create a useful meta-analysis.37 A standardised form of testing interventions to improve EMR use could create studies that are homogeneous enough to provide conclusions with greater power. In addition, there is no generally accepted evaluation method when discussing EMR use. In the future, studies would benefit from standardised interventions and a clearly defined way of evaluating meaningful use of EMRs.
Nature of the interventions
The predominant intervention type identified in this review used educational material, seminars and guidelines to target EMR use (professional interventions), which were identified in 8 of the 12 studies. This focus on professional interventions was found to be consistent with the literature given that the only other systematic review in this area,37 only included studies with educational interventions.37 However, perceived barriers to EMR use include lack of both financial incentives and useful EMR features.38 39 To address those barriers, the implementation of financial and organisational interventions is required. Therefore, there is a need for future studies to consider the other categories of interventions (organisational and financial) in the area of improving EMR use.
Focus of interventions
Both use of EMR functions and data quality received equal attention as target areas for interventions to improve EMR use. Even though the studies collected for this review represent two important areas for interventions to target in order to improve EMR use, the literature was found to be lacking in other areas that could be targeted to improve use areas such as: communication, workflow, knowledge/skills and technological support. The field of interventions and intervention target areas aimed at improving EMR use is still lacking in well-designed studies that cover all areas that effect EMR function and use.
Strengths and limitations
This review is aimed at a new and developing field. This is one of only two systematic reviews conducted in the area of improving EMR use.37 However, due to high heterogeneity in this area, previous reviews were unable to conduct a meaningful meta-analysis.37 In this review, a synthesis of the results was possible through: the categorisation of interventions using the EPOC taxonomy of interventions and the identification of possible intervention target areas to improve EMR use. A limitation of this study was that due to the new and wide geographic spread of information technology use in the health field, EMRs are identified differently in different countries, making it impossible to identify all the studies with one search term. In an attempt to learn all the possible terms that are used to refer to an EMR, a search was performed prior to the creation of the search strategies. Using those newly found terms, a search strategy was then created to be as inclusive as possible without straying from the inclusion/exclusion criteria.
Conclusion
This review reveals a lack of attention given to interventions aimed at improving EMR use in primary healthcare. This is also reflected in the absence of a generalised method to evaluate EMR use, as well as guidelines to implement interventions to improve this use. After an intensive and inclusive search of the literature, this systematic review found a relatively small number of included studies with high heterogeneity. However, it is still worth noting that the results of this meta-analysis indicate that it is beneficial for primary healthcare practice to implement organisational, professional and financial interventions. This can be achieved through investing in EMR feature add-ons, educational materials and financial incentives to improving EMR use.
Footnotes
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: None declared.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.Delpierre C, Cuzin L, Fillaux J, et al. A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? Int J Qual Health Care 2004;16:407–16. 10.1093/intqhc/mzh064 [DOI] [PubMed] [Google Scholar]
- 2.Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144:742–52. 10.7326/0003-4819-144-10-200605160-00125 [DOI] [PubMed] [Google Scholar]
- 3.Weed LL. Medical records that guide and teach. N Engl J Med Overseas Ed 1968;278:593–600. 10.1056/NEJM196803142781105 [DOI] [PubMed] [Google Scholar]
- 4.Tripathi M. EHR evolution: policy and legislation forces changing the EHR. Journal of AHIMA 2012;83:24–9. [PubMed] [Google Scholar]
- 5.Chang F, Gupta N. Progress in electronic medical record adoption in Canada. Canadian Family Physician 2015;61:1076–84. [PMC free article] [PubMed] [Google Scholar]
- 6.Collier R. National Physician Survey: EMR use at 75%. Canadian Medical Association Journal 2015;187:E17–E18. 10.1503/cmaj.109-4957 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.The College of Family Physicians of Canada, Canadian medical Association, Royal College . 2014 survey: national physician survey, 2014. Available: http://nationalphysiciansurvey.ca/surveys/2014-survey/
- 8.Canadian Medical Association . How can Canada achieve enhanced use of electronic medical records? 2014. Available: https://www.cma.ca/Assets/assets-library/document/en/advocacy/Enhanced-Use-of-EMRs-Discussion-Paper-Final-May-2014.pdf
- 9.Menachemi N, Collum TH. Benefits and drawbacks of electronic health record systems. Risk Manag Healthc Policy 2011;4:47–55. 10.2147/RMHP.S12985 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lau F, Price M, Boyd J, et al. Impact of electronic medical record on physician practice in office settings: a systematic review. BMC Med Inform Decis Mak 2012;12. 10.1186/1472-6947-12-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Linder JA, Ma J, Bates DW, et al. Electronic health record use and the quality of ambulatory care in the United States. Archives of Internal Medicine 2007;167:1400–5. 10.1001/archinte.167.13.1400 [DOI] [PubMed] [Google Scholar]
- 12.Graetz I, Reed M, Shortell SM. The next step towards making use meaningful: electronic information exchange and care coordination across clinicians and delivery sites. Medical Care 2014;52:1037–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. HealthIT.gov. Meaningful use definition and meaningful use objectives of EHRs: providers and professionals, 2015. Available: https://www.healthit.gov/providers-professionals/meaningful-use-definition-objectives
- 14.EPPI: Centre Home. UCL Institute of Education . Social science research unit, 2015. Available: http://eppi.ioe.ac.uk/cms/
- 15.Cochrane Effective Practice and Organisation of Care Review Group (EPOC) . Data collection checklist: EPOC Taxanomy, 2015. Available: http://epoc.cochrane.org/epoc-taxonomy
- 16.The Cochrane Collaboration Higgins J, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5, 2011. [Google Scholar]
- 17.Melendez-Torres GJ, Bonell C, Thomas J. Emergent approaches to the meta-analysis of multiple heterogeneous complex interventions. BMC Med Res Methodol 2015;15. 10.1186/s12874-015-0040-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lorenc T, Felix L, Petticrew M, Melendez GJ, et al. Meta-analysis, complexity, and heterogeneity: a qualitative interview study of researchers’ methodological values and practices. Syst Rev 2016;5. 10.1186/s13643-016-0366-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.StataCorp . Statistical software: release 13, 2015. [Google Scholar]
- 20.Borenstein M, Hedges L, Higgins J. Introduction to Meta-Analysis. United Kingdom: John Wiley & Sons Ltd, 2009. [Google Scholar]
- 21.DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177–88. 10.1016/0197-2456(86)90046-2 [DOI] [PubMed] [Google Scholar]
- 22.Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52:377–84. 10.1136/jech.52.6.377 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.de Lusignan S, Stephens PN, Adal N. Does feedback improve the quality of computerized medical records in primary care? Journal of the American Medical Informatics Association 2002;9:395–401. 10.1197/jamia.M1023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.de Lusignan S, Hague N, Brown A, et al. An educational intervention to improve data recording in the management of ischaemic heart disease in primary care. J Public Health 2004;26:34–7. 10.1093/pubmed/fdh104 [DOI] [PubMed] [Google Scholar]
- 25.Pan Q, Meng Y-X, Gibbons GH, et al. Effectiveness of an intervention to improve the documentation required for diagnosis of metabolic syndrome in clinics serving African-American patients. Qual Prim Care 2009;17:191–6. [PubMed] [Google Scholar]
- 26.Baer HJ, Schneider LI, Colditz GA, et al. Use of a web-based risk appraisal tool for assessing family history and lifestyle factors in primary care. J Gen Intern Med 2013;28:817–24. 10.1007/s11606-013-2338-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Maviglia SM, Yoon CS, Bates DW, et al. KnowledgeLink: impact of context-sensitive information retrieval on clinicians' information needs. J Am Med Inform Assoc 2006;13:67–73. 10.1197/jamia.M1861 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Kortteisto T, Raitanen J, Komulainen J, et al. Patient-specific computer-based decision support in primary healthcare--a randomized trial. Implement Sci 2014;9. 10.1186/1748-5908-9-15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Nemeth LS, Ornstein SM, Jenkins RG, et al. Implementing and evaluating electronic standing orders in primary care practice: a PPRNet study. J Am Board Fam Med 2012;25:594–604. 10.3122/jabfm.2012.05.110214 [DOI] [PubMed] [Google Scholar]
- 30.Kruse GR, Kelley JHK, Linder JA, et al. Implementation of an electronic health record-based care management system to improve tobacco treatment. J Gen Intern Med 2012;27:1690–6. 10.1007/s11606-012-2174-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Maddocks H, Stewart M, Thind A, et al. Feedback and training tool to improve provision of preventive care by physicians using EMRs: a randomised control trial. Inform Prim Care 2011;19:147–53. [DOI] [PubMed] [Google Scholar]
- 32.Davis AM, Cannon M, Ables AZ, et al. Using the electronic medical record to improve asthma severity documentation and treatment among family medicine residents. Fam Med 2010;42:334–7. [PubMed] [Google Scholar]
- 33.Sweeney J, Mchugh S, Perry IJ. Implementation and evaluation of a clinical data management programme in a primary care centre. Irish Medical Journal 2012;107:323–6. [PubMed] [Google Scholar]
- 34.Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Hogan WR, Wagner MM. Accuracy of data in computer-based patient records. J Am Med Inform Assoc 1997;4:342–55. 10.1136/jamia.1997.0040342 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Himmelstein DU, Woolhandler S, Hope WS. Hope and hype: predicting the impact of electronic medical records. Health Aff 2005;24:1121–3. 10.1377/hlthaff.24.5.1121 [DOI] [PubMed] [Google Scholar]
- 37.Goveia J, Van Stiphout F, Cheung Z, et al. Educational interventions to improve the meaningful use of electronic health records: a review of the literature: BEME guide No. 29. Med Teach 2013;35:e1551–60. 10.3109/0142159X.2013.806984 [DOI] [PubMed] [Google Scholar]
- 38.Gagnon M-P, Ghandour EK, Talla PK, et al. Electronic health record acceptance by physicians: testing an integrated theoretical model. J Biomed Inform 2014;48:17–27. 10.1016/j.jbi.2013.10.010 [DOI] [PubMed] [Google Scholar]
- 39.Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res 2010;10. 10.1186/1472-6963-10-231 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Jerome RN, Bettinsoli Giuse N, Rosenbloom ST, et al. Exploring clinician adoption of a novel evidence Request feature in an electronic medical record system. J Med Libr Assoc 2008;96:34–41. 10.3163/1536-5050.96.1.34 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjhci-2019-000023supp001.pdf (77.2KB, pdf)
bmjhci-2019-000023supp002.pdf (37.6KB, pdf)
bmjhci-2019-000023supp003.pdf (14.8KB, pdf)
bmjhci-2019-000023supp004.pdf (99.2KB, pdf)