Development and acceptability of a culturally competent skills and knowledge assessment tool for patients with diabetes mellitus

Abstract

Background

Patients newly diagnosed with type 2 diabetes mellitus (DM) and newly prescribed insulin need to learn essential self-care and management skills quickly. To optimize teaching, clinicians need to assess a patient’s basic understanding of DM and their skills. While DM patient assessments exist, this study reports the development of an assessment of patient DM management skills and knowledge, using feedback from DM clinicians, patients, and caregivers.

Research Design and Methods

A systematic search of Pubmed/Medline and Scopus (1980–2017) of DM knowledge assessments was performed. Twenty-four studies were identified. Content from the existing assessments was adapted to create a 12 item DM-Skills Knowledge Assessment (SKA) to assess a patient’s DM management skills and knowledge. To assess cultural humility, modified cognitive interviews were conducted in individual user sessions and semi-structured focus groups. Audio-transcripts of the interviews/focus groups were independently coded, and codes were grouped into key themes. Participant demographic characteristics were assessed.

Results

Five focus groups and eleven key informant interviews were conducted, including 10 DM clinicians, 12 patients/caregivers, and 15 laypersons. All 10 clinicians reported that the DM-SKA addresses the key domains of DM education deemed to be of highest importance during the transition from hospital to home and that their patients would be willing to complete the assessment. More than half of the patient/caregiver/layperson participants self-reported race/ethnicity other than non-Hispanic white and performed similarly to non-Hispanic white participants in understanding each item, willingness to complete the DM-SKA, and perception that family or community members would be willing to complete the DM-SKA. The DM-SKA has a baseline Flesch reading score of 81.3, indicating low complexity language.

Conclusion

DM clinicians agreed that the DM-SKA assesses all essential DM management skills. For patients/caregivers, it has acceptable literacy, cognitive validity, and culturally acceptable for racial/ethnic minority populations in the study, including elderly persons.

Introduction

More than 34 million adults in the United States (US) (13 % of US adult population) have diabetes mellitus (DM), and 27 % of adults over age 65 are diagnosed with diabetes [1]. While most adults with DM do not initially require insulin [1], [2], 11 % of all adults diagnosed with DM over age 20 require insulin within 1 year of diagnosis [1]. Ethnic and racial minorities in the US are at higher risk of developing DM [1], [3]. They also have a larger burden of DM complications, are less likely to achieve glycemic goals and receive recommended American Diabetes Association Standards of Care or national standards for Diabetes Self-Management Education and Support (DSMES) [4], [5], [6]. These racial ethnic disparities are well-known and driven primarily by social injustice, inequitable resource allocation, and social determinants of health [3], [5], [7], [8].

Patients newly diagnosed with DM, particularly those who are newly prescribed insulin, must acquire diabetes management skills quickly in order to provide essential self-care, particularly during the transition period from hospital to home, to prevent and/or delay DM and non-DM related complications (e.g., infection) [4], [9]. The benefits of patients with DM receiving DSMES on clinical outcomes, quality of life, and health care costs are well established [4]; however, actual delivery of DSMES is far from universal, most often due to a lack of resources, with a substantial proportion of patients with DM never receiving any DSMES [4].

Whether patients receive DSMES or not, clinicians need to assess a patient’s understanding of DM and DM management skills, to provide effective teaching to empower patients to understand their condition and make decisions to better it. Assessments of DM skills and knowledge have been developed and validated, with the Diabetes “Survival Skills” Knowledge Test [9] Diabetes Knowledge Questionnaire [10], Diabetes Knowledge Test [11], and Diabetes Numeracy Test [12] being the most widely used. However, none of these assessments have reported on any feedback from end users (patients/caregivers) or from DM clinicians, including diabetes care and education specialists, despite evidence that person-centered interventions and communications have been associated with improved foot care [13], and overall glycemic control [14]. Furthermore, none were developed with consideration and attention to cultural tailoring for racial and ethnic minorities, despite the known role of culture on health outcomes [15], [16] and the increasing number of patients with DM from racial and ethnic minorities. Nam et al. showed a significant link between culturally tailored DM education, specifically among ethnic minorities, and improved glycemic control using HbA1c as a surrogate marker [17].

The goal of this study was to develop a patient/caregiver-centered assessment tool, consistent with the current standards of DM care [4] and focused on assessing baseline knowledge and skills of patients, newly diagnosed with DM and newly requiring insulin. The purpose of the DM-SKA was to attempt to develop an assessment that meets the culturally diverse needs of patients.

Methods

Development of the Diabetes Mellitus Skills and Knowledge Assessment (DM-SKA)

A comprehensive and systematic search of the literature in PubMed, Medline, and Scopus, of assessments of DM knowledge or skills, published between 1980 and 2017, was performed. Medical study headings or “MeSH” terms included “diabetes mellitus,” “patient education,” “evaluation studies,” “education measurements/methods,” and “mental recall.” The educational content of each of assessment was mapped to the DM education domains from the American Diabetes Association Guidelines for Medical Care in Diabetes [18] and the Diabetes Knowledge Questionnaire (DKQ) from the Australian National Consensus group [10]. Specific items from educational domains related to essential DM knowledge and skills for immediate DM self-care of patients in transition from hospital to home were retained and reviewed. Items from domains such as acute and chronic complications, sick day management, nutrition, physical activity, and foot care were not retained, as these domains are not essential for patients during the immediate transition from hospital to home period. Each retained item was reviewed by 2 DM physicians (SK and AW) and 1 DM education specialist for content relevance and accuracy. The retained items were further reviewed with regards to overlap or redundancy between items and clarity.

Key Informant interviews

Approval of the study was obtained from the Northwestern University Institutional Review Board and all participants provided informed consent.

Study participants

Clinicians providing DM care (physicians, nurse practitioners, and certified diabetes care and education specialists) at Northwestern Medicine in Chicago, IL were recruited. Patients, hospitalized at Northwestern Medicine, who were admitted with hyperglycemia symptoms or DM complications and were newly diagnosed with Type 2 DM and newly prescribed insulin and their caregivers (family member, friend, or other individual who was identified as being prepared to help the patient after discharge to home with finger-stick glucose checks and/or insulin administration) were recruited. Laypersons participating in activities at a Senior Community Center in Chicago, who neither had a personal history of DM nor had required insulin, were also recruited.

Study procedures

Individual interviews with clinician participants were conducted by a clinician and research assistant and were audio-recorded. Clinician participants were asked, when reviewing the retained items, to (a) consider the knowledge and skill assessment needs of a patient “newly diagnosed with DM who is about to be discharged from hospital to home on newly prescribed insulin” and to suggest additional topics or domains and/or recommend any changes or deletions to the items and (b) whether they felt that patients would be (1) willing and (2) able to complete the items.

Patient, caregiver, and layperson participants took part in either a focus group of 4–6 persons or an individual interview, conducted by a clinician and research assistant. A standardized guide was used for the patient, caregiver, and layperson focus groups and interviews, which were audio-recorded. Patients, new to both diabetes and insulin therapy and with hyperglycemia symptoms or DM complications as the reason for hospitalization, were included. Caregivers and laypersons (non-hospitalized) were asked to consider the following scenario: “Imagine you were recently diagnosed with DM and told you were going to go home and will need to give yourself insulin injections.” At the beginning of each focus group session, the goals of the study were explained to participants. Then, study participants were asked to read each item and to describe the item “in their own words.” This task sought to assess patient/caregiver understanding and interpretation of each item [19]. Additional probing questions were then asked [19], specifically about the language used, including words, their general clarity, ease of understanding, and potential of understanding by “their friends, family or members of their community.” Participants were also asked if they felt “their friends, family or members of their community” would be willing to complete the DM-SKA items. Once feedback was elicited, patients, caregivers and laypersons were asked to complete the DM-SKA to the best of their ability.

Statistical analyses

The audio-recorded interviews and focus groups were transcribed. Transcripts were uploaded into MAXQDA software (VERBI, 20.0.0) for coding and analysis. Two reviewers (SH, JH), independently, inductively coded the same initial transcript and then reconciled their codes to create a code book, which was then reviewed and categorized by a DM physician. The codes were then applied to the remaining transcripts. Demographic characteristics of study participants were gathered in an Excel (Microsoft Office 365, 2011) database. Responses to the DM-SKA were scored with each correct answer = 1 and incorrect or incomplete answers = 0. Partial credit was given to questions with multiple correct answers.

Results

Development of the Diabetes Mellitus Skills and Knowledge Assessment (DM-SKA) literature review

Using the five MeSH terms, a total of 24 studies about knowledge assessments were identified. Items, related to the educational domains of the American Diabetes Association Guidelines for Medical Care in Diabetes [18] and the Diabetes Knowledge Questionnaire (DKQ) from the Australian National Consensus group [10], and source of the item are shown in Table 1.

Table 1.

Items identified by literature review.

	Item	Educational Domain	Source
1	What is diabetes?	Knowledge Introduction to Diabetes	Farrant et al. Computer based learning and assessment for diabetic patients
2	Which of the following is true?	Knowledge Introduction to Diabetes	Dunn et al. Development of the diabetes knowledge scales forms DKNA, DKNB, and DKNC
3	The glucose meter strip is placed into the glucose meter before or after pricking your finger?	Skills Blood glucose (a) understanding normal levels (b) self-monitoring	Dunn et al. Development of the diabetes knowledge scales forms DKNA, DKNB, and DKNC
4	Circle all the things you can try if not enough blood comes out of your finger after pricking with a lancelet	Skills Blood glucose (a) understanding normal levels (b) self-monitoring	Dunn et al. Development of the diabetes knowledge scales forms DKNA, DKNB, and DKNC
5	If you are at home and have problems turning on the meter, who can you first ask for help?	Knowledge Support services	Question generated by our research team, inspired from a question in the article by Farrant et al
6	Circle all the places where you can learn about finding a Certified Diabetes Educator	Knowledge Support services	Question generated by our research team, inspired from a question in the article by Farrant et al
7	If you begin to feel a low blood sugar reaction, you should:	Knowledge Hypoglycemia	Dunn et al. Development of the diabetes knowledge scales forms DKNA, DKNB, and DKNC
8	Which of these signs indicate that your blood sugar level is too low? Circle all that apply:	Knowledge Hypoglycemia	Farrant et al. Computer based learning and assessment for diabetic patients
9	Which of these signs show that your blood sugar level is too high? Circle all that apply.	Knowledge Hyperglycemia	Farrant et al. Computer based learning and assessment for diabetic patients
10	What to do if your blood sugar level gets too high?	Knowledge Hyperglycemia	Question generated by our research team
11	How long does the pen need to be held in the skin for?	Skills Insulin	Question generated by our research team
12	Circle all the places where you can inject insulin.	Skills Insulin	Question generated by our research team

Creation of the DM-SKA

Twelve items (Appendix 1) were adapted for the DM-SKA. The items address the six domains deemed by DM patients and DM experts to be of highest importance for patients with DM during the transition period from hospital to home. [20]. The six domains include general DM understanding, self-monitoring of blood glucose, understanding support services, identification and management of both hyper- and hypoglycemia, and insulin administration. The DM-SKA does not include any items that assess lifestyle changes, such as nutrition and physical activity, as these domains, while important, are more relevant for chronic management of DM rather than essential for immediate self-care.

As shown in Table 1, two items assess basic knowledge about DM and its long-term complications; two items assess knowledge about self-monitoring of blood glucose; two items assess knowledge about DM sources for support; two items assess recognition and self-management of hypoglycemia; two items assess recognition and self-management of hyperglycemia, including when to contact a clinician; and the final two items assess insulin self-administration skills.

End-User Feedback and Cultural Assessment

Of the 24 studies reviewed, none reported any data about end-user (patients, caregivers, DM clinicians) feedback and only 8 (33 %) [11], [12], [21], [22], [23], [24], [25] included any evaluation performed with a minority population. None included any information about the inclusion of racial-ethnically diverse end-users or obtaining their feedback.

End-user acceptability of the DM-SKA

Key Informant Interviews and Focus Groups

The overall response rate was 90 % (37/41) and included 10 DM clinicians, 12 patients/caregivers, and 15 laypersons (Table 2). Four patients, who were invited to participate while still hospitalized, declined to participate, describing themselves as “being overwhelmed”. Eleven individual interviews were conducted with a clinician, patient, or caregiver. Five semi-structured focus groups sessions were held: 2 sessions with 2 clinicians each, 2 sessions with 2 patients/caregiver each, and 1 session with 8 laypersons.

Table 2.

Demographics and Participant Characteristics.

	Providers N = 10	Patients/caregivers N = 12	Laypersons N = 15
Characteristics	4 MD, 2 APN, 2 CDE, 2RN	Inpatients new to insulin	3 with hx of DM skills
Age (mean years +/- SD)	37.4 ± 11.2	48.5 ± 11.2	65.1 ± 16.7
Age > 65yers of age	12.50 %	16.60 %	73.30 %
Race/ethnicity
Non-Hispanic Black	N/A	58.30 %	33.30 %
Non-Hispanic White	70 %	41.60 %	46.70 %
Hispanic	N/A	8.30 %	20 %
Asian	30 %	N/A	N/A
Highest Educational Level %
Graduate degree	100 %	60 %	60 %
Hba1c on diagnosis (mean +/- SD)	N/A	11.1 % ± 2.07	N/A
Discharged on insulin	N/A	100 %	N/A

Mean age of participating clinicians was 40.1 ± 12 years, with 12.5 % ≥ age 65. Mean age of the patients/caregivers/laypersons was 57.8 ± 17 years, with 44 % ≥ age 65 (Table 2). Patient/caregiver/layperson participants self-reported their race/ethnicity, with 46 % non-Hispanic white, 33 % non-Hispanic Black, 12 % Hispanic, and 9 % Asian American. Their highest educational degree attained ranged from high school diploma to doctorate, with 60 % being a college graduate (with bachelor’s degree or higher) and the remainder having a high school diploma.

As shown in Table 3, all patients/caregivers and laypersons (N = 27) demonstrated adequate understanding of each item and 96 % (N = 25) reported a willingness to complete the DM-SKA. When asked, 88 % of patients, caregivers, and laypersons reported that they thought that a family member, friend, or someone in their community would be willing to complete the DM-SKA items. Two patients, however, reported concerns that the DM-SKA felt like a “test” or “quiz.” Some younger (<age 65) patients/caregiver/laypersons (N = 4) raised concerns about the potential willingness of elderly family members to complete the DM-SKA with vision problems and older age being potential barriers. However, a willingness to complete the DM-SKA was endorsed by all participants who were > age 65. Overall, patients/caregivers and laypersons felt that the DM-SKA would be “for their own benefit.” When asked if it would bother them to take the test more than once, participants reported liking “reiteration of information”.

Table 3.

Focus Group and Key Informant Interview Results about the DM-SKA.

Patients/Caregivers/Laypersons	N = 27
Willingness to complete DM-SKA	96.2 % (N = 26)
Thought family member, friend or member of community willing to complete DM-SKA	88.8 % (N = 25)
Barriers identified by younger participants < 65 years of age	14.8 % (N = 4) Vision problems (N = 2) Older age (N = 2)
Need for assistance	22 % (N = 6) Vision problems (N = 2) Language barrier (N = 4)
Acceptability from older participants > 65 years of age N = 12	100 % (N = 12)
Mean score of DM-SKA for Patients and Caregivers N = 12	9.85 ± 1.5 out of 12

The DM-SKA has a baseline Flesch reading score of 81.3 [26], indicating low complexity language and a Flesch-Kincaid reading grade level of 5.3 [26]. The overall mean DM-SKA score of patient/caregiver/laypersons participants (N = 27) was 8.44 ± 2.47 out of 12. Questions with multiple correct answers had the most incorrect responses, specifically questions 4, 6, 8 and 9. Of participants who completed the DM-SKA (N = 27), 5 (20 %) needed assistance, 2 with vision difficulties and 3 with English as a second language. Assistance was provided by the interviewer (clinician or research assistant) reading aloud the questions and answer options to the participants and recording their responses.

As shown in Table 4, all clinicians reported that appropriate DM skill domains were included: “… some of the basics that I would say a patient should know before they leave the hospital.” They also reported feeling that patients would be willing and able to complete the DM-SKA. Two clinicians were concerned that patients might feel like the DM-SKA is a “test” or “quiz.” Clinicians offered differing suggestions about the best time to administer the DM-SKA: “… [should be] both pre and post-test” versus “…good time would be at follow-up in clinic.”

Table 4.

General Themes from the Focus Groups and Key Informant Interviews.

	Themes	Examples of Quotes
Providers	•Endorsed use of DM skills assessment •General endorsement of reading level •Appropriate skills domains covered •Varying opinions on timing of delivery/administration of DM-SKA •Concern with format/delivery	“I think they questions are good. I think they address some of the basics that I would say a patient should know before they leave the hospital” “…good time would be on follow-up in clinic” “I think that these are really important questions” “[should be] both pre and post-test” “maybe change the name to ‘check your understanding survey’
Patients Caregivers Laypersons	•Level and appropriateness adequate •Willingness to use •Concern with acceptability	“for their own benefit” “I like reiteration of information” “I need it” [how do you feel if you were given this test as part of diabetes education]” “Oh [the questions], they’re excellent” “people don’t want to feel like they are being tested”

Patients/caregivers and laypersons indicated that the level and appropriateness of the items were adequate [“questions are excellent”] but expressed some concern with acceptability [“…they don’t want to feel like they are being tested”].

Performance of the DM-SKA

More than half of the patient/caregiver/layperson participants self-reported race/ethnicity (33 % non-Hispanic Black, 12 % Hispanic, and 9 % Asian) other than non-Hispanic white. These participants performed similarly to non-Hispanic white participants in the understanding of each item, willingness to complete the DM-SKA, and perception that family or community members would be willing to complete the DM-SKA. Three participants of a race/ethnicity other than non-Hispanic white expressed difficulty completing the DM-SKA because of English being a second language. Mean DM-SKA scores of participants who reported their race/ethnicity other than non-Hispanic white was 8.27 ± 1.97 and was not statistically different from non-Hispanic White participants 8.78 ± 3.35 (p-value 0.32).

Discussion

This study reveals that few existing assessments of DM skills and knowledge gathered any end-user (patient, caregiver, DM clinicians) feedback or attended to cultural humility or competency. Furthermore, some previous assessments are now outdated and do not reflect current guidelines, while others were designed for and validated only for specific populations, suggesting that many are not generalizable to the current diverse US population with DM.

The twelve items, specifically related to knowledge and skills for immediate DM self-care and management, that comprise the Diabetes Mellitus-Skills and Knowledge Assessment (DM-SKA), were endorsed by all clinicians, patients, caregivers, and laypersons, particularly by older (>age 65) individuals, contrary to the perception of their younger family members.

All clinicians reported that their patients would likely be amenable to answering the DM-SKA items, that the language was simple, and that all necessary domains related to essential knowledge and skills for immediate DM self-care were addressed.

Patient, caregiver and layperson participants of varying educational levels and diverse racial/ethnic backgrounds were all able to verbalize an accurate understanding of the items, whether or not they knew the correct answer. Given that culture plays an important role in health behavior and health-related decision [15], it was important to include this data, even though based on our sample size, we cannot ensure the cultural competency of the DM-SKA. Culturally competent education and health promotion tools have been used to decrease health disparities [27] and engage communities in health promotion. Nearly all participants, regardless of education level or racial/ethnic background, indicated a willingness to complete the DM-SKA and endorsed that a family member, friend, or someone in their community would likely be willing to complete the DM-SKA.

Limitations and next steps

Limitations of our study include modest sample size and clinicians and patients being recruited from a single medical center. All patients had been recently hospitalized with new onset, uncontrolled type 2 DM, making the findings less generalizable to patients in the ambulatory setting. Over 60 % of the recruited patients and caregivers had a graduate degree, and not all races/ethnicities were represented in our results, again affecting generalizability. Furthermore, although we attempted to assess cultural competency of the DM-SKA, it was only evaluated in English, despite the growing number of non-English speaking patients with DM. Validation of the cultural competency of the DM-SKA with a larger sample size, making sure to include all ethnicities, and of certified translations of the DM-SKA is needed. Most importantly, studies that assess whether the DM-SKA is useful for clinicians to assess patients’ baseline DM skills and knowledge and determine its ability to detect improvement in skills and knowledge after additional education are necessary.

Conclusion

The DM-SKA is a tool that can be used to assess the DM skills and knowledge essential for patients with newly diagnosed type 2 DM and newly prescribed insulin to provide immediate DM self-care, when in the transition period from the hospital to home, including those with diverse educational levels and from diverse race/ethnicities.

In this peri and post-COVID-19 era, with many clinical care visits being conducted using virtual telehealth, it is of increasing importance for clinicians to be able to assess the knowledge and skills of patients with DM, in order to tailor and optimize their DM education and training.

CRediT authorship contribution statement

Stephanie Hakimian: Project administration, Resources, Writing – original draft, Writing – review & editing, Data curation, Formal analysis. Susan Karam: Conceptualization, Formal analysis, Methodology, Writing – original draft. Kim Pardilla: Data curation, Methodology. Kasey Coyne: Software, Writing – original draft. Emilie K. Touma: Formal analysis, Methodology, Project administration, Software. Diane Larsen: Data curation, Project administration, Resources. Jane L. Holl: Conceptualization, Data curation, Formal analysis, Resources, Software, Supervision, Visualization, Writing – original draft, Writing – review & editing. Amisha Wallia: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Visualization, Writing – original draft, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: