A Time-Friendly, Feasible Measure of Nutrition Knowledge in Type 1 Diabetes: The Electronic Nutrition and Carbohydrate Counting Quiz (eNCQ)

Abstract

Background:

Greater knowledge about nutrition and carbohydrate counting are associated with improved glycemic control and quality of life in youth with type 1 diabetes (T1D). However, limited assessments of nutrition and carbohydrate knowledge have been developed, and existing measures can be time-consuming, overly broad, or not conducive to routine clinical use. To fill this gap, we developed and examined the feasibility of administering the electronic Nutrition and Carbohydrate Counting Quiz (eNCQ).

Method:

Ninety-two caregivers and 70 youth with T1D (mean age 12.5 years; mean time since diagnosis 5 years; English speaking) completed the 19-item eNCQ via tablet during a routine clinical visit. Completion time and item completion rates were used to assess feasibility. Relationships between eNCQ scores and patient demographics, diabetes management, and health outcomes were examined.

Results:

Participants took 10 minutes, on average, to complete the eNCQ. Total and Carbohydrate subscale scores (youth report) were negatively correlated with youth hemoglobin A1c (total r = –.38, carbohydrate r = –.38, Ps < .05), indicating that greater nutrition knowledge related to better glycemic control. Nutrition knowledge scores were generally high, but knowledge was negatively related to time since diabetes diagnosis (r = –.276, P < .05).

Conclusions:

Findings support feasibility of the eNCQ to assess nutrition knowledge in routine clinical care. Following additional acceptability and validity testing, the eNCQ may identify families in need of further nutrition education. Nutrition assessment is particularly indicated for youth over one year since T1D diagnosis, as these families displayed lower nutrition knowledge and may need continuing education to maintain diabetes-specific nutrition knowledge over time.

It is recommended that youth with type 1 diabetes (T1D) learn about healthy eating and carbohydrate counting immediately following diagnosis and at appropriate follow-up intervals.^1,2 Better nutrition and carbohydrate counting knowledge is associated with lower hemoglobin A1c (HbA1c) levels, better ability to cope with everyday stressors, and improved quality of life.^1,3-5 Moreover, evidence suggests that both youth and caregiver dietary knowledge may directly impact adherence behaviors, which, in turn, may lead to better treatment outcomes.^6,7 Thus, it is clinically important to have assessments that quantify nutrition knowledge in both caregivers and youth with T1D to identify families who are in need of further education to improve health and T1D outcomes.

To date, a few assessments of nutrition and carbohydrate counting knowledge have been developed for youth with T1D and their families. However, a challenge that exists for all of these assessments is how to administer them in a busy clinic setting. For example, the PedCarbQuiz⁵ provides a valid assessment of carbohydrate counting knowledge and insulin dosing in caregivers of youth with T1D. But this measure contains 78 items and can take up to 30 minutes to complete, making it impractical for clinical assessment. Similarly, the Diabetes Nutrition Knowledge Survey⁴ and Nutrition and Carbohydrate Counting Quiz (NutraCarbQuiz),⁸ while both relatively short, are administered via a paper-pencil format and require time to hand score, which may limit their use in clinic.

Thus, we discerned that an electronic or web-based administration method for a valid nutrition and carbohydrate counting assessment was needed to enable regular use in clinical care. Web-based instruments are fast and easy to administer, can be automatically scored, and scores can be automatically saved to a secure database or uploaded directly to a patient’s electronic medical record,^9-11 offering substantial benefits over a paper-pencil format. Moreover, automatic scoring and uploading reduces the risk of errors, improves data quality, and may enable real-time data use by providers. For example, a brief survey completed on a computer or tablet in the waiting room could be automatically scored and updated in the patient’s health record for use by the provider within minutes, facilitating tailored care or referral to additional services at the same visit.

Therefore, our purpose here was to develop and assess the feasibility of a tablet-based administration of an electronic version of the Nutrition and Carbohydrate Counting Quiz⁸ (eNCQ) in clinical practice. The NutraCarbQuiz (NCQ) is a valid measure of nutrition knowledge previously developed by our study team.⁸ We selected the NCQ for this study because it was designed to be brief (19 questions) and written at a 6th grade reading level. We predicted that an electronic version, the eNCQ, would be quick and effective to administer to caregivers and youth with T1D during a routine diabetes clinic visit. We had secondary aims to evaluate how nutrition knowledge as measured on the eNCQ would relate to patient demographic characteristics, T1D management, and health outcomes. We predicted that older youth and those reporting longer T1D duration would exhibit higher eNCQ scores, suggesting greater nutrition knowledge. We also predicted that greater nutrition knowledge demonstrated by caregivers and/or youth would be associated with lower HbA1c levels, more frequent blood glucose self-monitoring (SMBG), and lower body mass index (BMI). Our study results should inform clinical practice recommendations for assessing nutrition knowledge in routine care for youth with T1D.

Methods

Participants and Procedures

This study took place in a large Midwestern children’s hospital system. We identified eligible families via medical chart review and recruited them from one of four metropolitan clinic sites. Youth were eligible if they were between 1 and 21 years old, had a diagnosis of T1D for at least 6 months, and used either multiple daily injections or continuous subcutaneous insulin infusion. We excluded youth if they did not have T1D, did not use carbohydrate counting as part of their treatment regimen, were receiving medical treatments that could impact diabetes control (ie, chronic steroids, immunosuppressive therapy), or had a history of thalassemia affecting HbA1c levels, or if their family was non-English speaking. All study procedures received Institutional Review Board approval prior to subject recruitment.

A member of the Diabetes Clinic team approached youth and/or their caregiver at a clinic visit to obtain written parental consent and youth assent (or written consent for both when youth age ≥18 years). A team member who was not the youth’s primary provider always obtained informed consent. Following consent, caregivers completed a demographics questionnaire, while we collected clinical information from the youth’s electronic health record (EHR). Youth and their caregiver completed the eNCQ using an electronic tablet. For youth 11 years and older, both the youth and caregiver completed the eNCQ independently. However, for youth 10 years and younger, we only collected caregiver-reports because we expected children had less of a role in their diet. If two caregivers (ie, mother and father) attended the clinic visit, only one was asked to complete the eNCQ.

Measures

Demographic Information

This questionnaire included items about family environment, primary caregivers involved in the child’s T1D care, race/ethnicity, sex, and treatment regimen. Specific items about carbohydrate counting included (1) who primarily counts carbohydrates and (2) what resources are used for carbohydrate counting at home.

Clinical Information

We collected from the EHR youth’s date of birth, clinic visit dates, date of T1D diagnosis, insulin regimen, current age, height, weight, BMI z-score, blood pressure, annual lipid profile, number of documented hypoglycemic events in the past year, SMBG, and HbA1c level at the current appointment. For all youth, their HbA1c levels were processed on either the Tosoh G8 HPLC (Tosoh Bioscience Inc, San Francisco, CA) or the Afinion AS100 Analyzer (Orlando, FL). Both instruments are traceable to the Diabetes Control and Complications standard and report results as percentages.^12,13

Electronic Nutrition and Carbohydrate Counting Quiz (eNCQ)

The eNCQ is a 19-item self-report questionnaire that assesses applied carbohydrate counting and nutrition knowledge based on MyPlate recommendations (www.choosemyplate.gov). The eNCQ was adapted from a previous paper-pencil version (the NCQ) that was tested and modified by a team of 25 experts in pediatric endocrinology (eg, doctors, nurses, diabetes educators, dietitians). Previous versions of the NCQ showed acceptable internal consistency and concurrent validity with the Diabetes Nutrition Knowledge Survey and PedCarbQuiz.⁸ We previously demonstrated that the NCQ took an average of 10 minutes to complete and was predictive of glycemic control.⁸

The eNCQ is administered on an electronic tablet using REDCap, a secure, HIPAA-compliant web application for administering online surveys and storing data. Items test respondents’ ability to read nutrition labels, identify foods, demonstrate understanding of serving sizes and macronutrients, display knowledge of how foods impact blood glucose, and select foods to create a meal containing 60-grams of carbohydrate. For example, participants were asked, “Select the two foods that have the most carbs in a standard serving” (response options: raisins, turkey, broccoli, crackers, and/or steak) and “Which one of the following are sources of healthy fats?” (response options: natural cheese, salmon, lean roast beef, butter, hot dog). Additional items are displayed in Figure 1.

Figure 1.

Screenshots of REDCap user interface and eNCQ items.

Items are automatically summed to yield a Total score, as well as Nutrition and Carbohydrate Counting subscores, with higher scores indicating greater knowledge. Total scores can range from 0 to 35, while the Carbohydrate Counting subscale ranges from 0 to 24 and the Nutrition subscale ranges from 0 to 11. We also calculated accuracy scores based on the percentage of items each participant answered correctly. We used REDCap to record the start and stop times for participants and used these objective data points to calculate an average completion time for participants.

Statistical Analysis

We used descriptive and summary statistics to describe the study sample, nutrition knowledge scores, and survey completion times. We ran bivariate Pearson correlations to examine correspondence between eNCQ scores across reporters, and associations between eNCQ scores and youth outcomes. Pearson correlation coefficient effect sizes may be interpreted as small (r = .10), medium (r = .30), and large (r = .50).¹⁴

Results

Participants

We recruited 92 caregiver-youth dyads and 8 additional young adults aged 18 years or older between March 2015 and August 2015. Of the 92 caregivers and 100 youth/young adults recruited, 92 caregivers and 70 youth completed study measures (30 youth did not complete measures because they were <11 years old). We recorded HbA1c values for all 100 youth. But, we removed two youth because their HbA1c levels were ≥4 standard deviations above the mean and/or their eNQC scores were ≥3 standard deviations below the mean, leaving 68 youth and 90 caregivers in the final analyses. Youth had a mean age of 12.46 ± 3.69 years (range, 3-20) and mean T1D duration of 5.09 ± 3.51 years (range, 0.5-14.5). Of the youth, 54% were male, 95.0% were described by their caregiver or themselves as non-Hispanic white, and 74.5% used an insulin pump. Youth had a mean BMI z-score of 0.48 ± 0.89 and 26.5% were overweight or obese (BMI z-score > +1 standard deviation). Participants self-reported SMBG mean frequency of 6.08 ± 2.75 times per day. Youths’ HbA1c values ranged from 5.4 to 12.7% (mean = 8.39 ± 1.36%; 25.5% of HbA1c were in the recommended range of <7.5% based on American Diabetes Association guidelines).¹⁵

Time to Completion

Consistent with our prediction that the eNCQ would be relatively quick to complete, caregivers had a mean eNCQ completion time of 10.60 ± 3.88 minutes (range, 4-23 minutes) and youth had a mean completion time of 10.16 ± 5.66 minutes (range, 4-34 minutes). However, there were four caregivers and four youth who took ≥20 minutes to complete the eNCQ, accounting for the wide range in completion times. Caregivers and youth completed the eNCQ in the context of routine clinical care. Of caregivers, 100% completed all eNCQ items, while 95.7% of youth completed all eNCQ items.

eNCQ Scores

Caregivers had a total mean score of 29.21 ± 3.06 (range, 22-35; 83% accuracy) and youth had a mean score of 26.60 ± 3.62 (range, 18-34; 76% accuracy). On the Carbohydrate Counting and Nutrition subscales, caregivers had mean scores of 19.98 ± 2.22 (range, 15-24; 83% accuracy) and 9.23 ± 1.40 (range, 5-11; 84% accuracy), respectively. Youths’ mean scores on the Carbohydrate Counting and Nutrition subscales were 18.91 ± 2.53 (range, 12-24; 79% accuracy) and 7.69 ± 1.88 (range, 2.5-11; 70% accuracy), respectively.

Intercorrelations

We present correlations of caregiver and youth scores on the Total scale, Carbohydrate Counting, and Nutrition subscales in Table 1. First, we examined intercorrelations among the eNCQ total and subscale scores to examine its internal structure and identify any highly redundant scores. The results of these analyses demonstrated correlations among eNCQ total and subscale scores of .91 to .40 for caregiver report and .87 to .33 for youth report. These findings suggest that the eNCQ total and subscale scores are related but not completely redundant measures of knowledge. Next, we examined correlations between corresponding caregiver and youth reports to provide a measure of interrater reliability. These correlations ranged from .46 to .25, suggesting a moderate to large concordance and good interrater reliability.

Table 1.

Correlations Between Parent and Youth eNCQ Total and Subscale Scores.

Scales	Caregiver report			Youth report
	Total	Carbohydrate	Nutrition	Total	Carbohydrate	Nutrition
Caregiver report
Total	—
Carbohydrate	.91***	—
Nutrition	.75***	.40**	—
Youth report
Total	.46***	.41**	.37**	—
Carbohydrate	.35**	.33**	.25	.87***	—
Nutrition	.43**	.35**	.39**	.75***	.33**	—

^*P < .05. **P < .01. ***P < .001.

Consistent with our aim to examine the validity of the eNCQ, we present correlations between eNCQ scores and youth outcomes in Table 2. Significant associations between lower HbA1c and higher youth-report of the eNCQ Total and Carbohydrate Counting subscale (P < .01) provide evidence of criterion-related validity. This is further substantiated by the positive and significant correlations between frequency of SMBG and eNCQ scores (P < .02), save youth-reported nutrition scores (P = .17). Youth age and age at diagnosis were inversely related to caregiver eNCQ Total and Carbohydrate Counting subscale scores (P < .02). Although youth-reported eNCQ scores were not related to youth age and continuous age at diagnosis, after categorizing T1D duration for greater or less than one year, we found a negative association between a T1D duration of >12 months and youth report on the eNCQ Total and Carbohydrate Counting subscales (Ps ≤ .05). Caregivers also scored slightly higher on the Nutrition subscale if their child used an insulin pump (P < .05).

Table 2.

Intercorrelations Between Caregiver and Youth eNCQ Scores and Related Youth Outcomes.

Outcome variable	eNCQ caregiver report			eNCQ youth report
	Total	Carbohydrate	Nutrition	Total	Carbohydrate	Nutrition
Youth age	−.249*	−.261*	−.130	.063	.021	.094
Age at diagnosis	−.276**	−.269**	−.177	−.095	−.061	−.100
Diabetes duration >12 months	.026	.027	.014	−.242*	−.238*	−.146
HbA1c	−.197	−.149	−.195	−.383**	−.376**	−.233
Treatment regimen	.202	.141	.220*	−.050	−.097	.033
Daily BG checks	.334**	.316**	.227*	.296*	.296*	.174

^*P < .05. **P < .01.

eNCQ scores were not significantly related to youth sex, race, BMI z-score, blood pressure, cholesterol levels, or triglyceride levels. Associations between eNCQ scores and severe hypoglycemia were unable to be examined due to a low incidence rate (only one participant had a documented episode of severe hypoglycemia in the past year).

Discussion

Our results highlight the feasibility of administering the eNCQ, a tablet-based nutrition knowledge assessment, during routine clinical care. First, our data show that caregivers and youth were able to complete the eNCQ in about 10 minutes, making the eNCQ administration time three times shorter than the average administration time for other T1D nutrition assessments.⁵ Second, all caregivers and youth were able to complete the eNCQ during their clinic visit. Although information on wait times are limited in pediatric T1D, previous studies have found average wait times of 25 minutes in adult primary care and pediatric emergency room visits.^16,17 Thus, if similar wait times hold for pediatric T1D clinics, the eNCQ would be easy for families to complete while waiting to be seen. If wait times differ significantly in pediatric T1D clinics, our online format has potential for families to complete the eNCQ outside of clinic (eg, emailed link), and the feasibility of this approach should be examined in a future study. Third, our eNCQ utilizes the REDCap platform, which enables automatic scoring in real time. REDCap software also offers an open application programming interface (API) making it possible to export eNCQ results into an EHR, and options for customization, such as the creation of software hooks to track when the eNCQ was last completed by a family so that it could be automatically presented at the desired frequency (eg, annually).^18-20 In this way, the eNCQ has the potential to become embedded in a clinic’s routine assessment schedule. Future research should examine family-reported acceptability of regular eNCQ administrations.

We also show that the eNCQ has good criterion-related validity. Based on the literature, we expected total nutrition knowledge to be significantly associated with carbohydrate counting.^4,5,21 This hypothesis was confirmed. Especially large correlations were present between total nutrition knowledge and carbohydrate counting for both caregivers (r = .91, P < .001) and youth (r = .87, P < .001), suggesting that carbohydrate counting knowledge could be a good proxy measure of general nutrition knowledge, or that the eNCQ total score may be highly impacted by carbohydrate knowledge. Future research is needed to more fully describe how carbohydrate counting is related to general nutrition knowledge.

Further evidence of criterion-validity was the inverse correlation between youths’ eNCQ total scores and their HbA1c levels, and the positive association between youths’ eNCQ total scores and their SMBG. These findings highlight the importance of nutrition in supporting better glycemic control, which is highly predictive of long-term health outcomes for youth with T1D.²² Limited reports of hypoglycemia in our sample restricted our ability to investigate associations between nutrition knowledge and this acute complication. Future research should examine whether nutrition knowledge may be a protective factor against both severe hypoglycemia and diabetic ketoacidosis in youth. Moreover, contrary to the literature,²³ we did not find significant associations between eNCQ scores (youth/caregiver) and youth BMI, cholesterol, blood pressure, or lipids. This may indicate that the eNCQ successfully measures T1D-specific nutrition knowledge, but that other aspects of nutrition knowledge (eg, choosing lower fat and sodium foods) may underlie associations with cardiovascular and weight-related health.

Contrary to our hypothesis, youths’ eNCQ scores were inversely related to T1D duration. However, this result is supported by at least one previous study that found an inverse association between mealtime carbohydrate estimation accuracy and time since diagnosis.²⁴ It is possible that nutrition knowledge is difficult to retain over time, or that how youth estimate insulin doses drifts from carbohydrate counting early in T1D to experience-based dosing further out from diagnosis. We also found an inverse relation between caregivers’ eNCQ scores and child age and age at diagnosis, suggesting caregivers of younger children and those diagnosed at a young age in our sample demonstrated greater nutrition knowledge than caregivers of older children and those diagnosed at an older age. We suspect this association may be explained based on whom educators targeted for nutrition education (eg, parent or youth). For young children, caregivers are the natural education target because they will have responsibility for planning, shopping, and preparing meals for their child as well as carbohydrate counting. However, as youth grow older and eat more meals outside of the home, caregivers’ nutrition knowledge may be less applicable to daily T1D self-care, leading to challenges in retaining nutrition knowledge among caregivers of older youth. We believe the implications of our results support the importance of regular nutrition assessment and periodic nutrition refresher classes throughout childhood as well as family-based education to teach caregivers how to transfer nutrition knowledge to their child while maintaining adequate knowledge themselves.

Strengths of this study included the novel tablet administration of a nutrition assessment for families of children with T1D and the addition of completion time as a measure of feasibility. The study was further strengthened by recruiting participants from multiple clinic locations, and by collecting both youth- and caregiver-reports among patients 11-17 years old. This study also had some limitations. The sample was majority non-Hispanic white; however, this aligns with population norms for T1D in the United States.²⁵ The eNCQ was administered at only one time point, so additional research is needed to establish stability of nutrition knowledge and the test-retest reliability of the eNCQ. Only one measure of nutrition knowledge was administered, so future research is needed to compare the eNCQ against other validated measures of nutrition knowledge, carbohydrate counting, and/or general diabetes knowledge to establish its construct validity. To examine the treatment sensitivity of the eNCQ, future studies should assess nutrition knowledge using the eNCQ before and after nutrition education. In the future, investigators may also wish to directly assess patient-reported acceptability of the eNCQ, develop equivalent versions of the eNCQ to assess knowledge at regular intervals in clinic, develop a format that can be culturally tailored to families (eg, include culturally diverse foods), translate the eNCQ into other languages, and/or test whether the eNCQ can or should be completed by children younger than 11 years old. Finally, due to the exploratory nature of this study, we did not correct for multiple comparisons between eNCQ scores and youth outcomes, and these findings will need to be confirmed by independent samples.

Conclusion

Our quick, tablet-based eNCQ assessment may be feasibly and seamlessly incorporated into clinical practice and our initial validity results suggest that it may provide an accurate assessment of nutrition knowledge in families of youth with T1D. Others have demonstrated the ability to import patient- or caregiver-completed electronic surveys into EHRs.^18,19 Future research should examine acceptability of administering the eNCQ as a part of routine clinical care and the feasibility of importing these data into the EHR. Additional work is needed to expand the psychometrics of the eNCQ and to develop alternate forms. However, these initial results fill an important gap in routine T1D-specific nutrition assessment and, following additional sensitivity/specificity testing, may offer a timely tool to identify families of youth with T1D in need of additional nutrition education.