Abstract
Introduction: Primary care providers have a role to play in supporting the development of healthy eating habits, particularly in a child’s early years. This study examined the feasibility of implementing the NutriSTEP® screen—a 17-item nutrition risk screening tool validated for use with both toddler and preschooler populations—integrated with an electronic medical record (EMR) in primary care practices in Ontario, Canada, to inform primary care decision-making and public health surveillance.
Methods: Five primary care practices implemented the NutriSTEP screen as a standardized form into their EMRs. To understand practitioners’ experiences with delivery and assess factors associated with successful implementation, we conducted semi-structured qualitative interviews with primary care providers who were most knowledgeable about NutriSTEP implementation at their site. We assessed the quality of the extracted patient EMR data by determining the number of fully completed NutriSTEP screens and documented growth measurements of children.
Results: Primary care practices implemented the NutriSTEP screen as part of a variety of routine clinical contacts; specific data collection processes varied by site. Valid NutriSTEP screen data were captured in the EMRs of 80% of primary care practices. Approximately 90% of records had valid NutriSTEP screen completions and 70% of records had both valid NutriSTEP screen completions and valid growth measurements.
Conclusion: Integration of NutriSTEP as a standardized EMR form is feasible in primary care practices, although implementation varied in our study. The application of EMR-integrated NutriSTEP screening as part of a comprehensive childhood healthy weights surveillance system warrants further exploration.
Keywords: child, obesity, electronic medical records, protective factors, NutriSTEP, surveillance system, feasibility, intervention research
Highlights
Primary care practices present an opportunity to identify nutrition risk in children using the NutriSTEP screening tool.
Successful implementation of an EMR-integrated NutriSTEP screen varied by primary care practice site.
Extraction of NutriSTEP data from EMRs is feasible; extracted data were of good quality.
Implementation of an EMR-integrated NutriSTEP screen presents an opportunity to improve the care and management of children and their families, as well as support population health outcomes and health system quality improvement.
Introduction
Roughly one-third of Canadian children and adolescents aged 5 to 17 years are living with excess weight or obesity.1,2 Because weight-related behaviours established in early childhood persist into adolescence and beyond,3 and consequences associated with overweight and obesity start early in life,4,5 it is important to intervene early.3 Given the complexity of childhood obesity, effective public health interventions require an approach that considers multiple factors that influence a child’s weight, including family, peer and environmental influences;3 these factors often lie outside the mandate of the health sector.6 Recognizing the important role nutrition plays in weight and well-being, Ontario’s Food and Nutrition Strategy7 recommends that children be screened using the NutriSTEP® screening tool. NutriSTEP is also recommended as a tool for primary care providers’ use in the routine assessment of children’s healthy eating behaviours as noted in the Primary Prevention of Childhood Obesity clinical practice guidelines.8 The NutriSTEP questionnaire is a validated screening tool used to identify nutritional risk and protective factors in both toddler (18–35 months) and preschooler (3–5 years) populations,9,10 and parent completion of NutriSTEP has been shown to increase parental knowledge of healthy eating.11 In addition to the screening tool, NutriSTEP implementation involves the provision of parent resource materials and community referrals for services to support parents of children identified as being at risk.12
Although it is traditionally implemented in community and public health settings by a variety of health and non-health practitioners, one Canadian study found that parents were interested in completing the NutriSTEP screen in health care settings. 13 Implementation in this manner would facilitate early intervention through the early identification of toddlers and preschoolers identified as being at risk. Collaborations between public health and primary care are becoming increasingly common and contribute to strengthened programs and services.14 Previous research evaluating the implementation of the paper-based NutriSTEP screen in a variety of primary care settings demonstrated use by primary care providers, primarily during their enhanced well-baby visits.15 This research also identified an interest on the part of participating primary care practices to have the NutriSTEP screen integrated into their EMRs to facilitate patient care and management and a willingness to centralize patient data to support a comprehensive childhood healthy weights surveillance system.15
The establishment of a comprehensive surveillance system has been identified as an essential component to primary prevention8 and evidence of an effective public health system.16,17 Enhanced collaborations and partnerships have the potential to inform primary prevention efforts of the public health system through sharing of relevant primary health care data. However, there is limited alignment between current public health surveillance systems’ objectives and corresponding data collected. For example, due to lack of data, the estimation of rates of overweight and obesity in children aged 5 years and younger is a critical information gap for public health in Canada.16,18,19 Public health professionals could potentially overcome this obstacle by accessing EMR data, such as measured height and weight data, collected during routine primary health care visits.
There is limited literature about the use of an EMR-integrated NutriSTEP screen in primary care practices and the necessary supports and processes for successful implementation. In an effort to address this gap, our study aimed to understand the experiences of primary care providers implementing an EMR-integrated NutriSTEP screen; identify factors associated with successful implementation; and assess data completeness. This study builds upon previous research20 that investigated the feasibility of accessing EMR data transmitted to a provincial registry21 and examined the implementation of the paper-based NutriSTEP screen in 10 primary care practices in Ontario.15
Methods
This feasibility study used both qualitative and quantitative methods.
Participants and settings
We recruited a convenience sample of family health teams and nurse practitioner practices through family health team, dietitian and professional networks through the promotion of a one-page advertisement shared using a variety of communication channels. Primary care practices were eligible for inclusion if they were current users of the Accuro digital EMR software (QHR Technologies, Kelowna, BC, Canada) and were willing to implement the EMR-integrated version of the NutriSTEP screen.
Implementation of the EMR-integrated NutriSTEP screening tool
The development of a standardized NutriSTEP form was led by staff at QHR Technologies, in consultation with the leads for the NutriSTEP screening tool. Functional elements of the standardized NutriSTEP form included automatic scoring of individual questions and overall total score, which was tested by members of the research team. A flag function was built into the standardized form as an option to remind primary care practices of children eligible for a NutriSTEP screen based on their age at the time of their visit. A purpose-built query function was also created for the extraction of discrete data elements of the patient EMR and was determined in collaboration with the research team. As a result of a licensing agreement between QHR Technologies and the University of Guelph, owner of the NutriSTEP screen, the EMR-integrated NutriSTEP screen was made available to all primary care practices using the Accuro EMR.
Research team members provided a 1-hour NutriSTEP training session to interested staff at participating primary care practices via webinar. The research team also developed a key message primer booklet for primary care providers, outlining detailed recommendations and follow-up responses corresponding to each of the 17 NutriSTEP questions. In addition, a variety of educational resources were provided for primary care providers to distribute (at their discretion) to parents based on their child’s NutriSTEP score and risk profile. Participating sites implemented the EMR-integrated NutriSTEP screen in a manner that best fit their practice. For primary care practices new to NutriSTEP, implementation began at a time that was convenient for them, once their training was completed.
Data collection and analysis
To understand the experiences of primary care providers implementing the EMRintegrated NutriSTEP screen, we conducted semi-structured interviews with key individuals identified (by their employer) as the person most knowledgeable about NutriSTEP implementation at their site. The implementation science framework developed by Durlak and DuPre,22 which was used as a theoretical basis for this research, informed the development of the interview guide. One author conducted a one-on-one, audio-recorded telephone interview with the person most knowledgeable about the current use of NutriSTEP at each site. An experienced transcriber transcribed all interviews verbatim. One author then checked the transcript of one interview against the audio recording for verification, and the remaining transcripts were considered accurate. Transcripts were analyzed thematically by one author with support of NVivo 10 qualitative software version 10 (QSR International Pty Ltd. 2012), with the coding structure established a priori based on a modified Durlak and DuPre22 framework and the research questions. This same author then analyzed each transcript according to the established coding structure. An iterative process was used to develop codes, whereby initial analyses informed the development of additional new codes; all transcripts were analyzed a second time using the newly revised coding structure.
Using the purpose-built query, discrete EMR data were extracted from the EMRs of the participating primary care practices between 20 June, 2016, and 7 July, 2017, by primary care practice staff and transferred to the agency of one member of the research team using a secure file transfer site. Descriptive statistics were generated using SAS version 9.3 (SAS Institute Inc., Cary, NC, USA). The NutriSTEP screen is a 17-item questionnaire that covers four attributes of nutritional status, including food and fluid intake, physical activity and sedentary behaviour, physical growth and development and factors affecting dietary intake and eating behaviours;12 variables of interest included individual NutriSTEP question score and overall total NutriSTEP score. Each NutriSTEP question has between two and five response options, and each response option is coded with a value ranging between zero and four.12 The sum of all individual NutriSTEP questions provides an indication of nutritional risk for the child, with a score of 20 or less indicating low risk, a score of 21 to 25 indicating moderate risk and a score of 26 or greater indicating high nutritional risk.12 The research team considered NutriSTEP data to be valid if primary care providers completed the appropriate screen for the child’s age (i.e. providers used a toddler screen for children aged 18–35 months and a preschooler screen for children aged 3–5 years). Furthermore, in this study, we allowed for a one-month buffer, whereby NutriSTEP data were considered valid if the respective screen for a child’s age group was within one month of the designated age range (i.e. 17–36 months for the toddler screen and 35–72 months for the preschooler screen).
Other variables of interest extracted from the EMRs included primary care practice site where the screen was completed; child’s date of birth; gender; postal code; date (of both NutriSTEP screen completion and height/length and weight measurements); and measured height/length and weight. We established weight-forage, weight-for-length and BMI-for-age z-scores for children up to 60 months using the World Health Organization’s (WHO) Child Growth Standards.23 We defined weight-for-age and BMI-for-age z-scores using the WHO’s Growth Reference Data for 5–19 Years24 for children 61 to 72 months of age. Growth status was determined using the Dietitians of Canada and Canadian Paediatric Society guidelines. 25 Growth status was determined to be invalid if height/length or weight variables were missing; height/length or weight measurements were deemed implausible; or measurement of height/length or weight was not timely. (After consultation with experts from the field, we decided that growth status calculations would be considered valid from records that collected a child’s height/length and weight measurements no more than 30 days apart).
Ethics approval process
Participating public health units with research ethics committees received their respective research ethics approval for this study. Further details of the research ethics process can be found in a report published on Public Health Ontario’s website.26
Results
Five primary care practices were recruited to implement the EMR-integrated NutriSTEP screen. Two of the sites had prior experience implementing the paper-based screen in their practices and declined to participate in the training webinar provided by the research team. Implementation of NutriSTEP varied by practice site (Table 1). The most common context for administration of the NutriSTEP screen was the 18-month enhanced well-baby visit (n = 3), though some practices also administered it at the 36-month visit (n = 2) and at the 4-year immunization appointment (n = 1); one primary care site administered the NutriSTEP screen only when nutritional risk was suspected (n = 1). Two practices completed the NutriSTEP screen directly into the EMR during their appointments, two practices had an EMR flag prompt front office administrative staff to provide parents with a paper-based NutriSTEP screen for their completion in the waiting room before their appointment, and one practice had parents complete the paper-based NutriSTEP screen during their appointment (when risk was suspected) and responses were entered into the EMR after the visit. Of the two practices that routinely requested that parents complete the paper-based screen before their appointment, one had the registered nurse review the screen with parents during their appointment and enter the NutriSTEP responses into the EMR after the visit, while the other had front office staff enter the NutriSTEP responses into the EMR, with a follow-up phone consult by their registered dietitian to discuss results. In our study, the NutriSTEP screen was administered or reviewed by a registered nurse (n = 3), a nurse practitioner (n = 1) or a registered dietitian (n = 1).
Table 1. Characteristics of participating primary care practice sites implementing the EMR-integrated NutriSTEP screen, Ontario, Canada, 2016–2017.
Experiences of primary care practices implementing the EMR-integrated NutriSTEP screen
Using the modified Durlak and DuPre22 framework (Table 2), we identified critical factors for successful implementation of the EMR-integrated NutriSTEP screen in participating primary care practices; they are described below.
Table 2. Factors associated with the implementation of the NutriSTEP screen in participating primary care practice sites, Ontario, Canada, 2016–2017.
Provider characteristics
According to Durlak and DuPre,22 provider characteristics include perceptions of a need for the innovation, perceived benefits or drawbacks of the innovation, selfefficacy and skill proficiency to implement the innovation as intended, and are important factors associated with successful implementation of a health promotion innovation. Overall, providers valued the NutriSTEP screen and felt it enhanced the traditional patient visit. This sentiment is described below by one participant.
I think it’s a huge value. I’m a big EMR user, [using] pathways and reminders. I think the Rourke and the well-baby visits are good, but they’re very generalized. We don’t look at how people eat, you know, we look at what they eat sometimes, but not how they eat, and promoting healthy habits. We have lots of obese children here, so I think it’s a good tool to actually get the conversation started about better nutrition and healthy eating habits. It’s nice to have. I like objective data…. it’s nice to have the scores, and say oh, hey, maybe this patient should go to a pediatrician, or whatever.
Having a staff member advocating for NutriSTEP use and incorporating the screen into appointments were identified as important factors by some practitioners. However, widespread implementation of NutriSTEP screening by all primary care providers did not always occur. As one respondent described, “For the other two physicians [who complete the well-baby visits but did not implement NutriSTEP], they have a nurse to assist, so they go through the Rourke, and the Nipissing, and all of those sort of things, and they didn’t really push or promote the NutriSTEP portion of it.” Some practitioners noted the voluntary nature of NutriSTEP as an influence on their decision not to administer the screen, instead choosing to use other, required screens, despite their limited nutritional scope.
Practitioners’ responses varied when a concern or a higher level of risk was identified, and included the provision of educational resources for parents, providing advice and detailing current guidelines and recommendations, referring families to a registered dietitian on staff for followup, or to another service provider in the community. In addition, primary care providers indicated that parents appreciated the opportunity to discuss nutritionrelated issues with practitioners at their scheduled appointments, regardless of their child’s nutritional risk score. The additional time required to complete the NutriSTEP screen in an existing visit was a challenge for some practices; while some practitioners were able to extend the visit time, others opted to have parents complete the screen on paper in the waiting room before their visit.
Characteristics of the EMR-integrated NutriSTEP screen
Durlak and DuPre22highlight compatibility and adaptability of a health promotion innovation as important features associated with successful implementation. The NutriSTEP screen was easily adapted and integrated into the current EMR; however, we did not consistently see the adaptation of the EMR-integrated NutriSTEP screen into existing workflows of primary care providers throughout all participating sites. Though having NutriSTEP in the patient EMR facilitated efficient storing and extraction of data, it was not important for all participating sites to have the screen completed electronically: two primary care practices chose to complete the NutriSTEP screen in paper format and later transfer the responses into the patient EMR. One of the practices found the direct completion of the screen in the EMR to be helpful, as illustrated below.
It’s easy to use. It even does the math for you, which I love, it’s kind of cool, it’s already in there, so nobody had to scan it and make text boxes, which might not sound like a big deal, but when the medical secretaries have to load a PDF that way, they hate it, and put 400 little text boxes, so it really, it made it easy to put it into play. The metrics were already set up, which is also equally as awesome, we didn’t have to figure out how to do that, again, it took some of that workload off everybody here.
Some providers noted the compatibility of the NutriSTEP screen to their health care appointments and found the screen facilitated the provider–patient conversations about healthy eating and healthy weights and provided an opportunity to discuss recommendations. As noted by one respondent, “I think that the NutriSTEP, in how it has been developed in the conversation style that you have it set as, is an easy approach for parents, and it’s a neutral approach. You’re getting them to just rate on average what they think from a day-today, and it opens up that conversation.”
Organizational capacity and community-level supports
Durlak and DuPre22 describe organizational capacity to support delivery and community- level supports such as administrative and referral supports as important considerations for successful implementation. Participants in this study identified administrative support as a critical factor for the implementation of the EMR-integrated NutriSTEP screen. Though administration varied across the five participating sites (Table 1), participants noted the value of the NutriSTEP screen in the clinical care and management of patients, as noted below by one participant.
The good part of it was it addressed some of the feeding issues that some people have, and so then I was able to refer to a dietitian with that. The dietitian loved getting NutriSTEP. They really like it, because otherwise they just get a script with your few notes, right…so [with NutriSTEP] they have something to go by.
For one participating site already implementing the NutriSTEP screen, interviewees identified a pre-existing partnership with public health unit staff and their ongoing support as an important factor for their implementation of this new innovation.
Systems supports for implementing the EMR-integrated NutriSTEP screen
Durlak and DuPre22 also describe systems supports, including training and technical support, as important factors for successful implementation of a health promotion innovation. In our findings, participants appreciated the training and educational resources provided by the research team; specifically, the educational resources and the key message primer booklet for primary care providers were well received and helpful in building provider confidence with nutrition-related conversations. As noted by one respondent, “I have some of your resources that I always carry with me. …I like that little book too, that’s a really nice little booklet that has each question, I really like that. I read that cover to cover, so I knew what I was doing, I thought I did, but making sure I knew everything.”
The EMR itself was also identified as an important factor for the implementation of the EMR-integrated NutriSTEP screen. Participating sites identified various functional aspects of the EMR that enhanced the clinical care and management of patients, including the use of flags and reminders in the patient EMR.
Quality of NutriSTEP data extracted from the EMR-integrated screen
In total, 282 patient records were successfully extracted from the EMRs of the five participating primary care practices; two records were identified as duplicates and excluded, resulting in 280 unique patient records available for analysis. The majority of records (74%, n = 206) were generated from one primary care practice, and one participating primary care practice did not yield any valid NutriSTEP completions (Table 3).
Table 3. Number of valid NutriSTEP screen and growth measurement records produced by five participating primary care practice sites, Ontario, Canada, 2016–2017.
Figure 1 illustrates the data processing flow of data extracted via the purposebuilt data query. Overall, 92% (n = 258) of records had valid NutriSTEP completions. Reasons for invalid NutriSTEP screens included age not within range at time of completion (n = 3), errors in using the appropriate NutriSTEP screen (i.e. incorrect screen for child’s age) (n = 5), missing date of birth (n = 1), and incorrect totalling of individual question scores (n = 13). Growth status was determined for approximately 81% of records (n = 228). Reasons for not being able to calculate growth status included missing height/ length or weight measurements (n = 5), invalid date of birth (n = 1), unbelievability of recorded height/length (n = 1), and the lack of recency or timeliness of height/ length and weight measurements (i.e. taken more than 30 days apart) (n = 45). Approximately 70% of records (n = 197) had both valid NutriSTEP completions and valid growth measurements. Additional details regarding data extraction findings are listed in Table 4. The dates for NutriSTEP screening, height/length measurement and weight measurement did not always coincide. It was later identified that the purpose-built query extracted records with a completed NutriSTEP screen and the most recent height/length and weight for the patient. The date for the most recent height/length and weight collected was not always the same date as the NutriSTEP screen completion date (Table 5); fewer than 50% of records had the same date for height/length, weight and NutriSTEP screen. Due to the small number of records, the research team did not examine the association between nutritional risk and growth status.
Table 4. Data extraction findings from the EMR-integrated NutriSTEP screen in five participating primary care practice sites, Ontario, Canada, 2016–2017.
Table 5. Difference in dates of EMR-collected height and weight and NutriSTEP screen among participating primary care practice sites, Ontario, Canada, 2016–2017.
Discussion
Overall, primary care providers valued the NutriSTEP screen and felt it positively contributed to the health care visit experience. Though the EMR proved useful for storing and extracting NutriSTEP data, additional work with the purpose-built query function is required to ensure extraction of appropriate data, particularly if EMR data are used to inform a child healthy weight surveillance system.
Implementation varied across participating sites. Having the NutriSTEP screen integrated into the EMR was not essential for its completion, as evidenced by some practices requiring parents to complete the screen in paper format prior to their visit, followed by manual entry by staff into the patient EMR afterwards. Previous research documented similar practices15,27 and found that additional provider time was required to scan paper-based screening results into patient EMRs. In the study conducted by Saviñon et al.,27 authors recommended the development of a software program to eliminate the administrative screening step and allow for linking discrete risk and protective factor data to other weight-related variables for a more comprehensive health assessment. Findings from our study demonstrated the feasibility of integrating such a tool as a standardized form and the ability to link NutriSTEP data with measured height/ length and weight data. In addition to facilitating appropriate referrals and care, integrating the NutriSTEP screen as an EMR form has the potential to streamline workflow and contribute to possible health care savings.28
NutriSTEP screening at one site was only conducted when a nutritional concern was identified or suspected. This non-routine implementation likely contributed to the limited number of screens completed. Given the low prevalence of nutrition risk in young children,9,10 it is not surprising to see so few screen completions when NutriSTEP is implemented in this manner. Furthermore, none of the three completed screens at this site were valid because the wrong screen was used for the patients’ age. The greatest number of valid screens was completed when NutriSTEP was routinely integrated into existing visits such as the 18-month enhanced well-baby visits. In our study, approximately 70% of screens were completed during this visit. Yet, time constraints remained an important consideration because participating practitioners faced challenges completing multiple tasks during this busy appointment. For two sites, this challenge was mitigated by asking parents to complete NutriSTEP in paper format in the waiting room before their appointment, resulting in the greatest number of screens completed. For one site, this manner of implementation proved very successful, contributing approximately three-quarters of all screen completions.
The low number of screens completed by the other participating practices could be due, in part, to the limited number of primary care providers integrating NutriSTEP into routine visits. While all sites had an individual who advocated for NutriSTEP implementation, other practitioners did not always use the screen, sometimes due to its voluntary nature. Currently, in Ontario, completion of the NutriSTEP screen as part of routine child health visits is not required; yet there remains the opportunity for NutriSTEP implementation during the enhanced 18-month wellbaby visit. Province-level support and direction requiring the completion of a comprehensive nutritional risk screen, such as NutriSTEP, would aid in greater uptake and use by primary care providers. Such support would present an opportunity to leverage existing province-level infrastructure and processes16 that would enhance access to relevant and timely surveillance data. Access to such data would improve the quality of care and management in primary care practices as well as population health assessment and surveillance efforts.
While our study did assess the quality of individual data variables captured through primary care practices’ EMRs, a data quality assessment of EMR data collected between sites, as recommended by Kahn et al.,29 was not conducted. Future research examining the use of EMR data for surveillance purposes should ensure the collection of consistent, accurate and reliable data across multiple sites and EMR platforms. 29 Future research might also consider the use of other frameworks to guide the assessment of widespread adoption and use. The Human–Organization–Technology (HOT–fit) framework proposed by Yusof et al.,30 for example, considers multiple factors that influence implementation categorized into four domains (i.e. Human, Organization, Technology and net benefits). This framework shares many common elements with the Durlak and DuPre22 framework; however, the HOT–fit framework30 provides additional detail for evaluating the technological aspects of an innovation. While our study demonstrated the ability to extract both NutriSTEP and height/length and weight data elements from the EMRs, the query extracted the most recent measured height/length and weight, which were not always measured on the same date of NutriSTEP completion, thereby limiting the ability to link NutriSTEP data with the child’s growth status. When considering the development of a provincial or national surveillance system informed by EMR data, technological aspects such as data quality are critical; therefore, it would be ideal if the query extracted these data based on the same visit date.
Strengths and limitations
This small-scale study provides an important contribution to the literature by providing insight into the varying implementation styles of an EMR-integrated NutriSTEP screen in primary care practices and potential factors that influence these workflows. Because this was a feasibility study, we used a convenience sampling method. As a result, our samples were small and nonrepresentative, and though in line with evidence of feasibility studies,31 the findings cannot be assumed to be generalizable to all primary care practices. In addition, the majority of quantitative data extracted were from one site, further limiting generalizability. Some participating primary care practices were current users of the (paper-based) NutriSTEP screening tool and therefore it is possible that their interest and willingness to implement the EMR-integrated NutriSTEP screen into their practices was different from those practices that were not current users. In addition, participating primary care practices implementing the EMRintegrated NutriSTEP screen were users of one EMR in particular and the experiences, barriers and enablers of participating sites may be different from those of sites using a different EMR.
Conclusion
Many interconnections exist between nutrition behaviour and growth status of children, and consideration of risk and protective factor data by primary care practitioners provides an opportunity for early identification, management and referral for individual support. There are still many challenges to consistent and accurate EMR use in primary care that must be addressed. Critical to population health intervention research is an understanding of factors that may influence outcomes.6 Our study identified several factors associated with the implementation of an EMRintegrated NutriSTEP screen in the primary care setting. While findings should be interpreted in the context of a smallscale study, they can inform further efforts to broaden its implementation to other primary care practices. Taken together, findings from our research suggest that it is feasible to integrate a validated nutrition screening tool into primary care EMRs, store the resulting data as discrete data elements for later extraction, and link them with other weight-related measures, allowing for comprehensive child health and weight assessments.
EMRs also present an opportunity to address the current gap in childhood healthy weights surveillance data for use in public health. This study highlighted the value of key partnerships with stakeholders such as EMR vendors, local public health units and primary care practices as important factors in such a screening program. Such collaborations should be considered if EMR data are to be used to inform a surveillance system that moves beyond BMI to improve population health.32 EMRs provide an opportunity for enhanced integration of preventive public health action and primary care provision, and bidirectional sharing of information30 through the development of a centralized surveillance system. The benefits of this system would extend beyond supporting clinical decision-making to include monitoring of population health outcomes and support quality improvement for an evidenceinformed health system; however, additional work is required to determine if the widespread collection of data from EMRs would result in accurate and representative estimates.33
Acknowledgements
The findings outlined in this manuscript come from a multi-phased Beyond BMI research project, which was funded by a Public Health Ontario (PHO) Locally Driven Collaborative Projects (LDCP) grant; no funding was provided for the development of this manuscript. The preliminary study occurred in 2014/15, followed by the EMR-integration study, which occurred in 2016/17. Findings from the larger Beyond BMI LDCP project are found in final reports provided to PHO as part of the LDCP Transfer Payment Agreements. These reports are publicly available on the PHO website (and cited in the current manuscript).
Conflicts of interest
The authors declare they have no competing interest. The University of Guelph holds the rights from Public Health Sudbury & Districts, Dr. Janis Randall Simpson, Dr. Heather Keller and Ms. Lee Rysdale to license the NutriSTEP screening tool, the Toddler NutriSTEP screening tool and related documentation, translations, trademarks, trade names or logos. Dr. Randall Simpson is one of the originators of NutriSTEP and receives royalties with respect to the licensing of questionnaires.
Authors' contributions
LA, KM, DM and JB designed the study. SJS, KM, KN, DM, LA, JRS and JB developed the data collection and analysis protocols. SJS and KN led data analysis with contributions from all other authors. LA took the lead in drafting and revising the manuscript, with critical feedback and input from KM, SJS, JRS and JB. All authors have read and provided final approval of this manuscript for publication.
The content and views expressed in this article are those of the authors and do not necessarily reflect those of the Government of Canada.
References
- Roberts KC, Shields M, Groh M, et al, et al. Overweight and obesity in children and adolescents: results from the 2009 to 2011 Canadian Health Measures Survey. Health Rep. 2012;23((3)):37–41. [PubMed] [Google Scholar]
- Rao DP, Kropac E, Do MT, et al, et al. Childhood overweight and obesity trends in Canada. Health Promot Chronic Dis Prev Can. 2016;36((9)):194–8. doi: 10.24095/hpcdp.36.9.03. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Queen’s Printer for Ontario. Toronto(ON): 2013. Addressing obesity in children and youth: evidence to guide action for Ontario. [Google Scholar]
- Ogden CL, Carroll MD, Kit BK, et al, et al. Prevalence of childhood and adult obesity in the United States, 2011–2012. Ogden CL, Carroll MD, Kit BK, et al. 2014;311((8)):806–14. doi: 10.1001/jama.2014.732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wheeler JJ, et al. Risk of obesity at 4 to 6 years of age among overweight or obese 18-month-olds: community-based cohort study. Can Fam Physician. 2013:e202–e208. [PMC free article] [PubMed] [Google Scholar]
- Hawe P, Potvin L, et al. What is population health intervention research. Can J Public Health. 2009;100((1)):Suppl I8–I14. doi: 10.1007/BF03405503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ontario Food and Nutrition Strategy Group. Toronto(ON): 2017. Ontario Food and Nutrition Strategy: a comprehensive evidence-informed plan for healthy food and food systems in Ontario. [Google Scholar]
- nd ed, et al. Registered Nurses’ Association of Ontario. Toronto(ON): 2014. Primary prevention of childhood obesity. [Google Scholar]
- Simpson J, Keller H, Rysdale L, et al, et al. Nutrition Screening Tool for Every Preschooler (NutriSTEP): validation and test-retest reliability of a parent-administered questionnaire assessing nutrition risk of preschoolers. Eur J Clin Nutr. 2008:770–80. doi: 10.1038/sj.ejcn.1602780. [DOI] [PubMed] [Google Scholar]
- Simpson J, Gumbley J, Whyte K, et al, et al. Development, reliability, and validity testing of Toddler NutriSTEP: a nutrition risk screening questionnaire for children 18–35 months of age. Appl Physiol Nutr Metab. 2015:877–86. doi: 10.1139/apnm-2015-0048. [DOI] [PubMed] [Google Scholar]
- Simpson J, Boere M, Dwyer J, et al, et al. Efficacy of nutrition risk screening with NutriSTEP among toddlers and preschoolers. Randall Simpson J, Boere M, Dwyer J, et al [Google Scholar]
- Toronto (ON): Nutrition Resource Centre and Janis Randall Simpson. NutriSTEP Implementation Toolkit [Google Scholar]
- Watson-Jarvis K, McNeil D, Fenton TR, et al, et al. Implementing the Nutrition Screening Tool for Every Preschooler (NutriSTEP) in community health centres. Can J Diet Pract Res. 2011;72((2)):96–8. doi: 10.3148/72.2.2011.96. [DOI] [PubMed] [Google Scholar]
- Valaitis R, et al. Canadian Foundation for Healthcare Improvement. Hamilton(ON): 2012. Strengthening primary health care through primary care and public health collaboration: final report for CFHI. [Google Scholar]
- Snelling SJ, Andrade L, Malaviarachchi D, et al, et al. risk and protective factors for child healthy weights assessed using NutriSTEP in primary health care. In: Beyond BMI Research Team. Beyond BMI: investigating the feasibility of using NutriSTEP and electronic medical records as a surveillance system for healthy weights including risk and protective factors in children. Toronto (ON): Locally Driven Collaborative Projects. Toronto(ON): 2015. Beyond BMI (Phase 2): risk and protective factors for child healthy weights assessed using NutriSTEP in primary health care; pp. risk and protective factors for child healthy weights assessed using NutriSTEP in primary health care–8. [Google Scholar]
- Ministry of Health and Long- Term Care. Toronto(ON): 2013. No time to wait: the healthy kids strategy. [Google Scholar]
- Groseclose SL, Buckeridge DL, et al. Public health surveillance systems: recent advances in their use and evaluation. Annu Rev Public Health. 2017:57–79. doi: 10.1146/annurev-publhealth-031816-044348. [DOI] [PubMed] [Google Scholar]
- Queen’s Printer for Ontario. Toronto(ON): 2013. Measuring the health of infants, children and youth for public health in Ontario: indicators, gaps and recommendations for moving forward. [Google Scholar]
- Brauer P, Royall D, Dwyer J, et al, et al. Obesity services planning framework for interprofessional primary care organizations. Prim Heal Care Res Dev. 2017;18((2)):135–47. doi: 10.1017/S1463423616000372. [DOI] [PubMed] [Google Scholar]
- Locally Driven Collaborative Projects. Toronto(ON): 2015. Beyond BMI: investigating the feasibility of using NutriSTEP and electronic medical records as a surveillance system for healthy weights including risk and protective factors in children. [Google Scholar]
- Jamal S, Moran K, Biro S, et al, et al. Locally Driven Collaborative Projects. Toronto(ON): Data quality assessment of child healthy weights from electronic medical records; pp. 1–31. [Google Scholar]
- Durlak JA, DuPre EP, et al. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41((3–4)):327–50. doi: 10.1007/s10464-008-9165-0. [DOI] [PubMed] [Google Scholar]
- WHO. Geneva(CH): The WHO child growth standards [Internet] Available from: https://www.who.int/childgrowth/standards/en/ [Google Scholar]
- WHO. Geneva(CH): The WHO growth reference data for 5-19 years [Internet] Available from: http://www.who.int/growthref/en/ [Google Scholar]
- A health professional’s guide for using the WHO growth charts for Canada. Dietitians of Canada and Canadian Paediatric Society. 2014 [Google Scholar]
- Locally Driven Collaborative Projects. Toronto(ON): 2017. Beyond BMI: building an EMR-based childhood healthy weights surveillance system to include nutritional risk and protective factors through the collection and EMR integration of NutriSTEP: final report (Phase 3) [Google Scholar]
- on C, Taylor JS, Canty-Mitchell J, et al, et al. Childhood obesity: can electronic medical records customized with clinical practice guidelines improve screening and diagnosis. J Am Acad Nurse Pract. 2012;24((8)):463–71. doi: 10.1111/j.1745-7599.2012.00735.x. [DOI] [PubMed] [Google Scholar]
- Hillestad R, Bigelow J, Bower A, et al, et al. Can electronic medical record systems transform health care. Health Aff. 2005;24((5)):1103–17. doi: 10.1377/hlthaff.24.5.1103. [DOI] [PubMed] [Google Scholar]
- Kahn MG, Raebel MA, Glanz JM, et al, et al. A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research. Med Care. 2012;50((7 Suppl 1)):S21–S29. doi: 10.1097/MLR.0b013e318257dd67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yusof MM, Kuljis J, Papazafeiropoulou A, et al, et al. An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit) Int J Med Inform. 2008:386–98. doi: 10.1016/j.ijmedinf.2007.08.011. [DOI] [PubMed] [Google Scholar]
- Bowen DJ, Kreuter M, Spring B, et al, et al. How we design feasibility studies. Am J Prev Med. 2009;36((5)):452–7. doi: 10.1016/j.amepre.2009.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- The National Academies Press. Washington(DC): 2012. Primary care and public health: exploring integration to improve population health. [PubMed] [Google Scholar]
- Birkhead GS, Klompas M, Shah NR, et al. Uses of electronic health records for public health surveillance to advance public health. Annu Rev Public Health. 2015;36((1)):345–59. doi: 10.1146/annurev-publhealth-031914-122747. [DOI] [PubMed] [Google Scholar]