Assessment of a Short Diabetes Knowledge Instrument for Older and Minority Adults

Abstract

Purpose

The purpose of the study was to assess the performance of a short diabetes knowledge instrument (SDKI) in a large multi-ethnic sample of older adults with diabetes and to identify possible modifications to improve its ability to document diabetes knowledge.

Research Design and Methods

A sample of 593 African American, American Indian, and white female and male adults 60 years and older, with diabetes diagnosed at least two years prior, was recruited from eight North Carolina counties. All completed an interview that included a 16-item questionnaire to assess diabetes knowledge. A subsample of 46 completed the questionnaire a second time at a subsequent interview. Item-response analysis was used to refine the instrument to well-performing items. The instrument consisting of the remaining items was subjected to analyses to assess validity and test-retest reliability.

Results

Three items were removed after item-response analysis. Scores for the resulting instrument were lower among minority and older participants, as well as those with lower educational attainment and income. Scores for test-retest were highly correlated.

Conclusions

The SDKI (13 item questionnaire) appears to be a valid and reliable instrument to evaluate knowledge about diabetes. Assessment in a multi-ethnic sample of older adults suggests that this instrument can be used to measure diabetes knowledge in diverse populations. Further evaluation is needed to determine whether or not this instrument can detect changes in knowledge resulting from diabetes education or other interventions.

Diabetes is a chronic condition that frequently results in medical complications and presents challenges for optimal care of patients.¹ In 2010, 26.9% of all individuals aged 65 years and older had diabetes; 20% of adults newly diagnosed fell into this age group.² Treatment for diabetes typically is multifaceted and often delivered through Diabetes Self-Management Education (DSME) programs. DSME incorporates comprehensive care, skill training and behavioral strategies that can improve metabolic outcomes for patients with diabetes.³ Diabetes self-management requires that patients practice problem-solving, drawing upon their skills and knowledge related to the disease and to the effects of their behavior on glycemic control. While diabetes knowledge itself does not ensure that a patient will implement effective self-management, assessing patient knowledge is helpful in practice settings,⁴ as well as in research, to evaluate education mastery and provide individualized education to meet the needs of patients with diabetes and to evaluate the effectiveness of interventions.

The most widely used and validated diabetes knowledge instruments (Michigan Diabetes Knowledge Test and Diabetes Knowledge Scales)^5,6 predate the development of current oral medications, recommendations for dietary management, and concerns about certain co-morbidities.⁷ Thus, more up-to-date instruments for assessing diabetes knowledge are needed.

Several different types of diabetes knowledge instruments have been published more recently, including those aimed at specific populations, at simplifying aspects of older instruments, or expanding the domains of knowledge covered.^8–11 One instrument has been used in studies involving minority patients in the southeastern US.^12,13 This segment and region of the US population has high prevalence of diabetes as well as cardiovascular and cerebrovascular morbidity and mortality,¹⁴ and the need for programs to improve diabetes outcomes is significant. Therefore, a diabetes knowledge instrument that is acceptable, valid, and reliable in this population is needed. However, no evaluations of this knowledge instrument (which we label Short Diabetes Knowledge Instrument [SDKI]) have been published. According to its developers,^12,13 it was adapted from an earlier instrument,⁵ by updating medically out-of-date items. The new instrument retained a focus on testing knowledge of the definition of glycemic control, diabetic complications, means for controlling glucose, and diet. It updated the items by moving from an emphasis on the use of urine glucose testing, insulin, and rigid dietary exchanges to an emphasis on controlling blood glucose through diet, recognizing symptoms of abnormal blood glucose, and using healthy eating to prevent complications. Items on foot care and the importance of physical activity for preventing cardiovascular complications were added.

The study reported here, carried out as part of a larger study of diabetes beliefs and self-management in a multi-ethnic population of older adults in the Southeast, presents an opportunity to assess this diabetes knowledge instrument, the SDKI. Our goal is to examine its performance and identify possible modifications to improve its ability to document diabetes knowledge. This paper reports (1) item-response analysis, (2) construct validity, and (3) test-retest reliability of this instrument among a multi-ethnic sample of older adults with diabetes in the rural Southeast.

Methods

Sample

The research was conducted in eight south central North Carolina counties (Harnett, Hoke, Montgomery, Moore, Richmond, Robeson, Sampson and Scotland) in North Carolina. These counties were chosen because they contain large minority populations and because a high proportion of the population is below the federal poverty line. They represent variation on the urban-rural continuum.¹⁵ The total sample included 593 African American, American Indian, and white men and women 60 years or older, who had a diabetes diagnosis for at least two years, and were not receiving dialysis treatment. The goal of the sampling plan was to recruit 100 participants for each ethnic/gender cell, with each cell having participants spread across educational attainment categories (less than high school, high school, more than high school). Participant recruitment was designed to provide a representative sample. Participants were recruited from various organizations and locations within each county to represent site-based sampling.¹⁶

Data Collection

Initial data collection was completed from June 2009 through February 2010. Interviewers were completed in participants’ homes, unless they requested otherwise. Interviewers outlined the project objectives and obtained written informed consent. An incentive ($10) was given for completing the interview, which consisted of an interviewer-administered, fixed response questionnaire. The questionnaire collected data on personal characteristics age, ethnicity, education, income, diabetes status, and diabetes knowledge. Participants were asked if they had ever participated in formal diabetes education classes; no attempt was made to determine when these classes occurred or whether or not they were American Diabetes Associated recognized classes. Glycemic control (A1C) was assessed at the interview visit after all questionnaire data were collected, using a finger stick blood sample and the procedures for the handheld Bayer A1cNow+ machine.¹⁷

For test-retest reliability, additional data were collected in spring 2012 from 46 participants from the original sample, equally divided by ethnic, sex, and education groups. They were recruited in order from randomized lists of the original study participants. These participants completed a short face-to-face interview (Test) in which they responded to the diabetes knowledge questions described below. The interview was repeated one month later (Retest). Participants received an incentive of $10 for each interview. The Wake Forest School of Medicine Institutional Review Board approved all procedures.

Measures

The outcome measure was diabetes knowledge, as measured by the 16-item diabetes knowledge instrument developed and utilized by Samuel-Hodge, Keyserling, and colleagues.^12,13 This instrument was adapted from an older instrument⁵ that queried knowledge about self-management and long-term complications of diabetes. Each item was a multiple choice question with three or four possible responses. An additional response, “I don’t know” was listed on the interviewer’s score sheet, but was not read to participants as a possible answer. Respondents were read each question and its responses and instructed to choose the correct answer. They were instructed not to guess at an answer, but to say “I don’t know” if they did not know the answer. The interviewer entered the stated response to each question, including any answer of “I don’t know”, on the score sheet. Administration of the 16-items took less than 10 minutes. A computerized scoring rubric was employed in which a correct answer was scored 1, and an incorrect answer (including “I don’t know”) was scored 0. Scores for all items were summed. The possible range for the total score was 0 to 16, with higher scores indicating greater diabetes knowledge.

Analysis

Data analysis included descriptive statistics including percentage correct for individual items, item-scale correlation, item response theory (IRT)^18,19 analysis, bivariate analysis between the knowledge score and important factors, and internal consistency and reliability assessments. Rooted in educational and psychological testing, IRT-based measurement can be used to examine how individual questions perform within a knowledge-based test. The performance of a question can be summarized by the so-called item characteristic curve, which depicts the likelihood of a respondent correctly answering the question as a function of the knowledge level of the respondent. A flat curve, for example, implies that respondent with a high level of knowledge is as likely to get the question correct as someone with a low level of knowledge. This would suggest that the question should be eliminated from the test. The benefits of IRT-based testing include the sample invariance property of the measurement; in other words, the estimated item property is not dependent on the sample and that accurate test scores can still be achieved using different sets of items. Because IRT scaling assumes unidimensionality in the data, we also conducted dimensionality analysis using several model fit indexes: comparative fit index (CFI), Tucker–Lewis index (TLI), root mean square error approximation (RMSEA), and standardized root mean square residual (SRMR). CFI and TLI values of above 0.9, and RMSEA and SRMR of under 0.08 are considered acceptable fit.²⁰ Factor loadings were also examined to evaluate the clinical meaning of the existence of multiple dimensions, if any.

Bivariate analysis of the association between knowledge score and other factors, on the other hand, was used to provide evidence of construct validity of the instrument. The factors examined included sex, ethnicity, age, education level, income level, duration of diabetes, history of diabetes class attendance, and A1C control. Internal consistency of the score measure was evaluated using Cronbach’s alpha, and the correlation between test and retest scores was used to assess instrument reliability.^21,22

Results

By design, participants were evenly divided among white (212; 35.8%), African American (200; 33.7%), and American Indian (181; 30.5%) older adults (Table 1). Less than half (40.8%) reported ever taking diabetes education classes, and about half (51.0%) had A1C measures less than 7%.

Table 1.

Sample characteristics by sex and ethnicity (White, African American (AA), and American Indian (AI)). N=593.

	Total		Sex				Ethnicity
			Female		Male		White		AA		AI
	n	%	n	%	n	%	n	%	n	%	n	%
Ethnicity
White	212	35.8	123	33.6	89	39.2
African American	200	33.7	114	31.12	86	37.9
American Indian	181	30.5	129	35.3	52	22.9
Sex
Female	366	61.7					123	58.0	114	57.0	129	71.3
Male	227	38.3					89	42.0	86	43.0	52	28.7
Age, years
60–69	307	51.8	192	52.5	115	50.7	109	51.4	93	46.5	105	58.0
70–79	222	37.4	132	36.1	90	39.7	70	33.0	90	45	62	34.3
80+	64	10.8	42	11.5	22	9.7	33	15.6	17	8.5	14	7.7
Education
<High school	217	36.7	123	33.7	94	41.4	62	29.3	76	38.2	79	43.7
High school graduate	200	33.8	137	37.5	63	27.8	73	34.4	66	33.2	61	33.7
> High school	175	29.6	105	28.8	70	30.8	77	36.3	57	28.6	41	22.7
Poverty
≥ Poverty line	407	70.1	225	62.7	182	82.0	158	75.2	125	65.1	124	69.3
< Poverty line	174	29.9	134	37.3	40	18.0	52	24.8	67	34.9	55	30.7
Diabetes education
No diabetes class	351	59.2	222	60.7	129	56.8	126	59.4	113	56.8	112	61.5
Diabetes class	242	40.8	144	39.3	98	43.2	86	40.6	86	43.2	70	38.5
Duration of diabetes
< 10 years	227	40.1	146	42.1	81	37.0	82	41.2	72	37.9	73	41.2
≥ 10 years	339	59.9	201	57.9	138	63.0	117	58.8	118	62.1	104	58.8
A1C, %
<7	302	50.9	193	52.7	109	48.0	107	50.5	102	51.0	93	51.4
≥7, <8	158	26.6	95	26.0	63	27.8	67	31.6	50	25.0	41	22.7
≥8	133	22.4	78	21.3	55	24.2	38	17.9	48	24.0	47	26.0

Scores on the knowledge instrument ranged from 0 to 15, with a mean (± SD) of 10.0 (2.7). Cronbach’s alpha for standardized scores was 0.73, which was deemed acceptable.²³ Although this is not an exceedingly high alpha, the test is unlikely to be used in clinical trials that would require a higher level of internal consistency. Responses to individual items showed a substantial range in percent correct (Table 2). Only 6.8% reported a correct score for Q12 concerning healthy eating and animal fat consumption, while 89.2% correctly answered Q14 about which food could cause the biggest rise in blood sugar.

Table 2.

Diabetes knowledge items, with correct response. Number and percent of sample providing correct answer at baseline; number and percent providing consistently correct or incorrect responses for the SDKI at test and one month retest.

Item	Item text, with correct response	Baseline (n=593)			Test/Retest (n=46)
		n	% correct	Item-scale correlation	% consistent
Q1	When diabetes is not under control, the blood sugar: (4) Can be High or Low	450	75.89	0.3324	0.67
Q2	For someone with diabetes, regular exercise or physical activity may: (1) Help Lower your risk of heart problems	501	84.49	0.2972	0.85
Q3	An excellent range for fasting blood sugar is: (1) 80 – 120 MG/DL	464	78.25	0.3586	0.72
Q4	Rice is mostly: (2) Starch or carbohydrate	435	73.36	0.3475	0.74
Q5	Margarine is mostly: (3) Fat	372	62.76	0.3011	0.67
Q6	Which of these tests would tell you about your average blood sugar for the past 2 – 3 months? (2) Hemoglobin A1c (or A1c) test	407	68.87	0.3297	0.91
Q7	Which of the following is common with diabetes? (2) Problems with your heart and blood vessels	475	80.37	0.3759	0.83
Q8	If your blood sugar is low, you should not: (3) Drink a diet soda	305	51.43	0.2650	0.80
Q9	When people with diabetes on insulin get sick and cannot eat their meals, they should: (2) Continue to take their insulin	330	55.65	0.1656	0.67
Q10	When your blood sugar is high, you may: (1) Feel thirsty and have to go to the bathroom a lot	270	45.61	0.1917	0.80
Q11	High blood sugar may happen because you: (2) Ate too much	465	78.55	0.2841	0.67
Q12	Healthy eating for a person with diabetes means: (3) Eating less animal fat	40	6.76	−0.1188	--*
Q13	Low blood sugar may be caused by: (1) Too much insulin or pills	240	40.61	0.2024	--*
Q14	Which of these foods will cause the biggest rise in your blood sugar? (1) 1 glass of orange juice	527	89.17	0.3361	0.87
Q15	When you have diabetes you should not: (3) Soak your feet in warm water	188	31.76	0.0124	--*
Q16	Eating less animal or saturated fats can help to lower your chances of having problems with your: (1) Heart	472	79.86	0.2186	0.72

^*Items dropped from analysis

Dimensionality analysis for the one-dimensional (two-dimensional) factor models resulted in the following values: CFI=0.76 (0.91); TLI=0.72 (0.89); RMSEA =0.07 (0.04); SRMR=0.11 (0.08). The fit indexes, with the exception of RMSEA, were not satisfactory for the unidimensional model. However, further examination into the factor loadings revealed that only three questions (Q4, Q5, and Q15) were loaded on a second factor. Q4 and Q5 were both related to content of specific foods and therefore might pertain to a nuisance dimension. Q15 had low loadings (<0.15) on both factors.

Item response analysis showed that three of the sixteen questions, Q12, Q13 and Q15, performed poorly. Q12 and Q15, in particular, exhibited low level of item-scale correlation (Table 2). Examining item characteristic curves for these questions demonstrated that the items were not discriminating different knowledge levels of study respondents and that respondents likely were simply guessing at the answers. These three items were dropped from further analysis. Item response curves were recalculated for the remaining 13 items (Figure 1). These show that responses to the remaining items all differentiate those with low overall knowledge from higher overall knowledge. With the remaining thirteen items, scores on the knowledge instrument ranged from 0 to 13, with a mean of 9.63 (±2.68). Cronbach’s alpha for standardized scores was 0.75. The model fit statistics CFI, TLI, RMSEA, and SRMR for the unidimensional model for the 13-item test slightly improved to 0.78, 0.74, 0.07, and 0.10, respectively.

Figure.

The item characteristic curves (ICC) (solid line) and information curves (dash line) for diabetes knowledge items retained in the Short Diabetes Knowledge Instrument (SDKI). The ICC shows the probability of a respondent correctly answering the item as a function of the knowledge level (scale on left) of the respondent. The information curve shows the level of accuracy of the knowledge score (scale on right).

Bivariate analyses were designed to validate the instrument by comparing scores across participant characteristics frequently associated with differences in health knowledge. In these analyses, scores varied by ethnicity and age, with whites’ knowledge scores (9.71 ± 2.19) exceeding those of minority participants (8.82 ± 2.70 for African Americans, and 9.11 ± 2.59 for American Indians) and participants age 60 to 69 had higher scores (9.49 ± 2.45) than older participants (age 70–79: 8.91 ± 2.63; age 80+: 9.11 ± 2.32) (Table 3). Knowledge scores increased with higher levels of education and were higher for those with incomes exceeding the federal poverty level (9.46 ± 2.35 vs. 8.78 ± 2.79). Although neither was significant, scores tended to be higher among those with history of diabetes class attendance and A1C less than 8%.

Table 3.

Short Diabetes Knowledge Instrument (SDKI) Score: bivariate comparisons by participant characteristics and diabetes status, using 13 item instrument.

	N	Mean	SD	Statistic (t, F)	P Value
Total	593	9.23	2.5
Ethnicity	593			6.82	<.01
White	212	9.71	2.19
African American	200	8.82	2.70
American Indian	181	9.11	2.59
Sex	593			0.30	0.76
Female	366	9.25	2.61
Male	227	9.19	2.37
Age	593			3.57	0.03
60–69	307	9.49	2.45
70–79	222	8.91	2.63
80+	64	9.11	2.32
Education	592			21.27	<.01
<HS	217	8.38	2.71
HS	200	9.59	2.42
>HS	175	9.86	2.04
Income	581			9.13	<.01
Below poverty line	174	8.78	2.79
Above poverty line	407	9.46	2.35
Diabetes Class	593			1.25	0.21
Yes	242	9.38	2.27
No	351	9.12	2.67
A1C	593			2.27	0.13
<8%	300	9.31	2.57
≥8%	135	8.94	2.30

Mean knowledge scores for the test and retest were 8.20 (± 2.38) for the initial test and 8.43 (± 2.41) for the repeated test, respectively. Scores were highly correlated (r=0.75; p<.0001). The percent agreement calculated for consistently correct and incorrect responses to individual items at test and retest ranged from a 67% to 91% (Table 2).

Discussion

These analyses examined the properties of the original 16-item diabetes knowledge instrument using procedures designed to evaluate the performance of individual items in the context of the entire instrument, and the validity and reliability of the overall instrument. Three items were found to perform poorly and were eliminated to create the 13-item SDKI. The strength of using item response analysis is that one can be assured that responses to each of the remaining items differentiate high and low levels of overall knowledge. The SDKI has a satisfactory level of internal consistency. Its validity is demonstrated by its association with socio-demographic characteristics of the sample, particularly formal education and income. The sample used in this study varied, by design from about a third with less than a high school education to about a third with greater than a high school education. About 30% of participants had incomes below the poverty line. As expected, knowledge increased with educational attainment and income. Such characteristics are frequently associated with greater health knowledge and have been used to validate other knowledge instruments.^9,10,24 Responses to individual items, whether correct or incorrect, are highly consistent between test and retest administrations of the instrument; and overall test-retest reliability over one month is high. These findings suggest that the 13-item instrument is measuring a stable level of knowledge.

There is a clear need for an up-to-date instrument to evaluate diabetes knowledge. The DKN scale⁵ predated an emphasis on the link between diabetes and cardiovascular disease and an emphasis on cardiovascular disease risk reduction in diabetes education. It is now considered essential that patients with diabetes recognize that failure to control diabetes can lead to serious cardiovascular consequences.⁷ Dietary recommendations in the DKN⁵ and DKT⁶ scales were structured in terms of food exchanges and substitutions, terms no longer used. Rather than prohibiting specific foods, current dietary recommendations for diabetes management stress obtaining a mix of carbohydrates, protein, and fat through an individually selected diet, as well as reduction in saturated and trans fat.⁷ Questions about glucose testing in the older instruments were phrased in terms of urine tests, not blood glucose testing; and a preponderance of the medication questions asked about insulin rather than the oral medications more commonly used today.^5,6

This study must be evaluated in light of its limitations. The instrument used does not include all aspects of diabetes knowledge. In fact, the only item on foot care, an important aspect of diabetes self-management to prevent complications,⁷ was eliminated as a result of the item response analysis. A comparison of the items in the final instrument with current national standards for diabetes self-management education²⁵ shows that some of the core curriculum topics (e.g., incorporating nutritional management into lifestyle) are much more heavily covered by the instrument than are others (e.g., using medication safely and for maximum therapeutic effectiveness). The population from which the study was drawn was limited to a set of largely rural counties in one southeastern state. Therefore, it may not represent the larger US population of persons with diabetes. The sample included only older adults, so the instrument may not perform the same in samples of other age groups. Nevertheless, the study has several notable strengths. The sample was large and included considerable variation in ethnicity, income, and educational attainment. The age group represented experiences the highest prevalence of diabetes and is the age group currently experiencing the highest rates of new diagnoses.² The instrument studied is up-to-date, including information of diet, complications, and medications that are current.

Implications for Diabetes Educators

Assessment of diabetes knowledge is an important starting point for individualizing diabetes education, the approach advocated by the American Diabetes Association.⁷ Evaluating diabetes knowledge level of a particular patient using a set of basic questions can help objectively gauge the focus needed for that patient. Gaps identified in patient knowledge can be a springboard for healthcare providers to use for agenda setting. Tools that can assist diabetes educators to obtain baseline and follow-up knowledge information are needed in practice. For research, an abbreviated instrument to evaluate diabetes education outcomes is also provided with the Short Diabetes Knowledge Instrument (SDKI).

Current diabetes self-management requires both mastery of skills (e.g., medication use, home glucose monitoring) as well as motivation to engage consistently in practices (e.g., consuming a low fat diet, regular exercise) that can lower blood glucose levels. By assessing diabetes knowledge, diabetes educators can be better equipped to help their patients to master skills and increase motivation. Multiple diabetes knowledge questionnaires exist. The one described in this paper is more up-to-date than many, and shows validity and reliability among older adults, a growing population at high risk for diabetes and its complications.