Validity and reliability of anti-diabetic medication adherence scale among patients with diabetes in Baghdad, Iraq: a pilot study

Abstract

Background

Medication non-adherence is a common problem among patients with diabetes. Patient-reported medication adherence scales are the most commonly used method to assess patient medication adherence, but up to today there is no specific tool for assessing medication adherence among patients with diabetes in Arab countries. This study aimed to develop and validate a new tool for assessment of adherence to antidiabetic medications among Iraqi patients with diabetes.

Methods

The Iraqi Anti-Diabetic Medication Adherence Scale (IADMAS) consists of eight items. The face and content validity of the IADMAS were established via an expert panel. For convergent validity, the IADMAS was compared with the Medication Adherence Questionnaire (MAQ). For concurrent validity, the IADMAS was compared with glycosylated hemoglobin. A total of 84 patients with types 2 diabetes were recruited from a diabetes center in Baghdad, Iraq. Test–retest reliability was measured by readministering the IADMAS to the same patients 4 weeks later.

Results

Only 80 patients completed the study (response rate: 95%). Reliability analysis of the IADMAS showed a Cronbach’s alpha value of 0.712, whereas that of the MAQ was 0.649. All items in the IADMAS showed no significant difference in the test–retest analysis, indicating that the IADMAS has stable reliability. There was no difference in the psychometric properties of the IADMAS and the MAQ. The sensitivity and specificity of the IADMAS were higher than that of the MAQ (100% vs 87.5% and 33.9% vs 29.7%, respectively).

Conclusion

The IADMAS developed in this study is a reliable and valid instrument for assessing antidiabetic medication adherence among Iraqi patients.

Significance of this study.

What is already known about this subject?

• There are many validated tools to assess medication adherence.

What are the new findings?

• The newly developed tool is the first tool developed in Arabic and specific for Iraqi patients with diabetes.

How might these results change the focus of research or clinical practice?

• This tool can be used to assess and indirectly to improve medication adherence among Iraqi patients with diabetes.

• Additionally, the validity and reliability of this tool can be assessed in all Arabic countries.

Introduction

Diabetes mellitus (DM) is a chronic metabolic disorder that has shown a dramatically increased incidence in recent decades.¹ DM can cause many acute and chronic complications that may be associated with significant morbidity and mortality.^{2 3} Glycemic control can effectively reduce diabetic complications and enhance the patient’s quality of life.^{2 4} Glycemic control can be achieved mainly through the utilization of pharmacologic treatment.⁵ Meanwhile, adherence to the pharmacologic treatment is necessary to ensure maximum treatment-related benefits.^{6 7} Unfortunately, lack of adherence is a common problem among patients with type 2 DM.⁸ It was found that non-adherence to antidiabetic medications can be associated with poor glycemic control and with increased costs through increasing use of healthcare resources.^{9 10} Patient-reported tools (scales) are commonly used to assess adherence among patients with DM because they are reliable and practical.^11–13 Most of the currently valid and commonly used tools to assess antidiabetic medication adherence, such as the Medication Adherence Questionnaire (MAQ), the Morisky Medication Adherence Scale - eight items (MMAS-8), the Medication Adherence Rating Scale, and the Adherence to Refills and Medication Scale, are lacking specificity since they were not originally designed for assessing adherence among patients with diabetes.^{11 14} Additionally, all of the above scales were structured and developed to test medication adherence for patients in developed countries. On the other hand, few scales were designed to assess medication adherence among patients with diabetes in developing countries, such as the Malaysian Medication Adherence Scale (MALMAS).¹⁵ It is well known that personal and cultural beliefs can affect patient medication adherence¹⁶; this means that all of these scales may not be suitable for assessing medication adherence among Iraqi patients even after a valid translation process because the culture and patients’ beliefs at which these scales were developed are somewhat different from that in Iraq. Therefore, this study aimed to develop a new tool for the assessment of adherence to antidiabetic medications among Iraqi patients with type 2 DM.

Patients and methods

Development of the Iraqi Anti-Diabetic Medication Adherence Scale

Literature review of self-reported medication adherence scales among patients with diabetes was done through the use of the following sets of keywords. The first set of keywords includes “evaluation of adherence and diabetes mellitus”, while the second set includes “medication adherence scale and diabetes mellitus”. Although many studies were detected, only those with freely available questionnaires were selected, such as the MAQ,¹⁷ Measurement of Adherence to the Treatment,¹⁸ and Medication Compliance Questionnaire.¹⁹

Additional review of the literature was carried out by searching Google Scholar and PubMed to look for articles about the perception of Iraqi patients with diabetes toward diabetes treatment using the following keywords: perception, diabetes treatment and Iraq; besides, the results of a recent qualitative study that was done to assess self-management practices and medication-taking behavior among Iraqi patients with type 2 DM were also taken into account.^20–22 The main reasons for medication non-adherence among Iraqi patients were carelessness, forgetfulness, high medication cost, disappearance of the disease symptoms, medication-related side effects, reduced appetite during sick days and going outside home.

Then all of the selected scales were carefully assessed to find out questions that can fit to evaluate adherence to antidiabetic medications among Iraqi patients. Six items were relevant to the Iraqi patients with DM and thus were adapted from the aforementioned scales. Additionally, two new items were added to the Iraqi Anti-Diabetic Medication Adherence Scale (IADMAS) to increase its sensitivity and specificity in predicting non-adherence among Iraqi patients with diabetes; the first added item was intended to assess the effect of medication cost on medication adherence, while the second one was intended to assess patients’ adherence to the prescribed medication dose during sick days while the appetite is mostly reduced.²³

In total, the IADMAS (online supplementary 1) consists of eight items; three items are used to directly assess medication-taking behavior by giving five responses: (1) always (daily), (2) often, (3) sometimes, (4) rarely and (5) never. The remaining five items are used to measure the determinant of non-adherence by giving a dichotomous response of “Yes” or “No”.

The first item aims to identify the extent of unintentional missing of medication doses; all other questions were directed to identify the extent of intentional medication non-adherence. Two items (1 and 3) were used to identify the extent of non-adherence to the time of medication taking. Four items (2, 6, 7 and 8) were formulated to identify the extent of intentional non-adherence with the prescribed medication dose. Only one item (5) aims to identify the extent of intentional non-adherence through discontinuation of taking DM medication.

Scoring of all items ranged from 0 to 1, 0 for non-adherent answer and 1 for total adherence. All items, except item 4, are inversely calculated. The first three items use a 5-point Likert scale that can take one of five values: 1, 0.75, 0.5, 0.25 or 0.

The total score of the IADMAS ranged from 0 to 8. Medication adherence in the IADMAS was categorized into three levels of adherence: high adherence (total score=8), medium adherence (6 to <8) and low adherence (<6).²⁴

Validation of the IADMAS

Translation

Both the MAQ and the six selected questions from the previously validated scales were translated to Arabic using a forward and backward translation method. Some of the translated questions were modified to ensure maximum understanding by the Iraqi patients. On the other hand, the Arabic questions of the IADMAS (final version) were also translated to English using a forward and backward translation method.

Face and content validity

A research team consisting of two diabetes educators, three clinical pharmacists, one diabetologist and one psychiatrist established the face and content validity of the Arabic and the English versions of the IADMAS; meanwhile, further validations were done only for the Arabic version. A pilot study was conducted on seven Iraqi patients with DM; they were asked to read the Arabic version of the IADMAS and then reply to the main author with their feedback on the clarity of each question in the IADMAS. All patients agreed on the clarity of the IADMAS questions.

Convergent validity

The IADMAS has not been validated and hence should be compared with an already validated questionnaire to assess its convergent validity. To date, there is no validated or gold-standard patient-reported measure of medication adherence in Iraq to be used for comparison. The MAQ was chosen as a comparator because it was already validated to assess adherence among patients with diabetes, in addition to its ease of use.^{17 25 26}

Concurrent validity

For assessing concurrent validity, the glycosylated hemoglobin (HbA1c) value was chosen as a comparator to the IADMAS because it better correlates with the patient’s glycemic control during the previous 2–3 months; besides, it need not be measured in a fasting state.^{27 28} HbA1c was measured using a point of care apparatus (i-chroma, UAE).

Study design

A cross-sectional study was conducted from the end of October 2017 to March 2018 in the National Diabetes Center, Baghdad, Iraq.

Sample size

There is uncertainty about the required sample size in pilot studies for validation of the new questionnaires. According to the rule of thumb, 5–10 participants per item can be considered sufficient, but as the sample size increased the accuracy of results can be also increased.^{29 30} The sample size for this study was set at 80 to achieve 10 participants per item (IADMAS has eight items), with an assumption of 5% for no response. Therefore, the target sample size was 84 patients.

Patient selection

A convenient sample of 84 patients were recruited by the main researcher in the waiting area of the National Diabetes Center, Baghdad, Iraq.

Only Arab patients older than 18 years, having type 2 DM and maintained on antidiabetic medications for at least 3 months were included in this study. However, patients who had renal impairment, cognitive impairment and depression and pregnant women were excluded.

Prior to recruitment, all patients were asked (by the main researcher) two questions specific for initial diagnosis of depression, to exclude patients with suspected but undiagnosed depression³¹ and refer those with abnormal answers to the psychiatrist.

All patients were interviewed by the main researcher to explain the purpose and the study protocol; only those who provided their informed consent were requested to complete a set of self-administered questionnaires that include the IADMAS and the MAQ. The questionnaires were presented to patients with low educational level (13 patients) and to patients with visual impairment (3 patients) via face-to-face interview. The interviews were conducted by the main researcher.

The participants need approximately 5–10 min to complete the two questionnaires.

Only 24 patients who were maintained on the same treatment (the same medication and same dose) were asked to return back after 2 weeks to fill in the IADMAS.

IADMAS diagnostic accuracy

Patients with HbA1c values of less than 7 were considered to have good glycemic control.²⁸ The association between medication adherence and glycemic control was used to measure true positive (TP) results (patients who are non-adherent to their treatment and have poor glycemic control), false positive (FP) results (patients with poor glycemic control despite being adherent to their treatment), true negative (TN) results (patients with good glycemic control and good medication adherence), and false negative (FN) results (patients with good glycemic control despite being non-adherent to their treatment).

The sensitivity of the IADMAS measures its ability to correctly predict poor glycemic control in non-adherent patients, while the specificity of the IADMAS measures its ability to correctly predict good glycemic control in adherent patients. Positive predictive value (also called precision rate) is the proportion of positive test results that are TPs. This value measures how likely non-adherent patients have poor glycemic control; meanwhile, the negative predictive value is the proportion of patients with a negative test result.³² This value measures how likely adherent patients have good glycemic control.

Specificity and sensitivity, as well as positive and negative predictive values, for the IADMAS were measured using the following equations:

Positive predictive value=TP/(TP+FP)×100%

Negative predictive value=TN/(TN+FN)×100%

Sensitivity=TP/(TP+FN)×100%

Specificity=TN/(TN+FP)×100%

Statistical analysis

Data input and analysis were done using the Statistical Package for the Social Sciences (SPSS) V.17. Categorical variables were presented as percentage and frequencies, while mean and SD were used to present continuous variables. χ² test was used to measure the difference among categorical variables. The Kolmogorov-Smirnov test was used to test the normal distribution of continuous variables. Mann-Whitney U test was used to test the mean difference between continuous variables. Spearman’s correlation test was used to determine the correlation between continuous variables. Cronbach’s alpha values were used to determine the internal consistency of the IADMAS and the MAQ. A Cronbach’s alpha value of more than 0.7 was considered optimum. A value of any corrected item–total correlation which is higher than 0.2 is considered acceptable.³³ Test–retest reliability was assessed using Wilcoxon signed-rank test to measure score difference for each item, while Spearman’s correlation coefficient was used to test the correlation of the IADMAS total score before and after retesting. P values <0.05 were considered statistically significant.

Results

A total of 84 participants were recruited in this study; however, only 80 patients completed this study (response rate: 95.2%). Patients’ demographic and clinical data are presented in table 1.

Table 1.

Demographic data of the participants

Parameter	Value
Age (years), mean±SD (range)	55.28±9.02 (33–76)
Gender
Male	48
Female	32
Educational level
None or primary	15
Secondary	17
Diploma or college	44
Postgraduate	4
Duration of diabetes mellitus, mean±SD (range)	9.33±7.02 (4 months–31 years)
Number of prescribed medications, mean±SD (range)	1.62±0.65 (1–5)
Type of medication
Oral	43
Injectable	21
Combination	16
Duration on treatment in years, mean±SD (range)	1.69±2.31 (0.2–12)
Glycosylated hemoglobin, mean±SD (range)	9.04±2.45 (5.0–14.5)
Frequency of drug usage, mean±SD (range)	2.61±0.93 (1–5)

SD, Standard deviation.

Psychometric properties of the IADMAS

Reliability analysis

The internal consistency of the IADMAS and the MAQ was measured using Cronbach’s alpha. The Cronbach’s alpha values for the IADMAS and the MAQ were 0.712 and 0.649, respectively. The Cronbach’s alpha of the IADMAS will be increased to 0.724 if item 8 was excluded, while for the MAQ this value will be increased to 0.706 if item 2 was deleted. The item–total correlations were less than 0.2 for one item (item 3) in the IADMAS (table 2).

Table 2.

Reliability of IADMAS and MAQ

Parameter	IADMAS		MAQ
	Corrected item–total correlation	Cronbach’s alpha if item deleted	Corrected item–total correlation	Cronbach’s alpha if item deleted
Q1	0.396	0.689	0.345	0.644
Q2	0.551	0.651	0.246	0.706
Q3	0.148	0.721	0.605	0.455
Q4	0.378	0.693	0.579	0.486
Q5	0.636	0.629
Q6	0.547	0.647
Q7	0.375	0.690
Q8	0.201	0.724
Overall Cronbach’s alpha score	0.712		0.649

IADMAS, Iraqi Anti-Diabetic Medication Adherence Scale; MALMAS, Malaysian Medication Adherence Scale; MAQ, Medication Adherence Questionnaire.

In table 3, the Spearman’s r for the total score of test–retest reliability was 0.806 (p=0.016). When the individual items in the the IADMAS were analyzed, no any item was significantly different at test–retest (p>0.05).

Table 3.

IADMAS test–retest reliability

Parameter	Z score for test–retest mean difference	P value
Q1	−1.633	0.102
Q2	−1.131	0.258
Q3	0	1
Q4	0	1
Q5	−0.577	0.564
Q6	−1.414	0.157
Q7	0	1
Q8	−1	0.317
Total score correlation	0.806	0.016

IADMAS, Iraqi Anti-Diabetic Medication Adherence Scale.

Convergent and concurrent validity

In table 4, there was a highly significant correlation between the IADMAS and the MAQ (Spearman’s r=0.682, p=0.000). Additionally, the scores of all IADMAS subscales were positively correlated with overall MAQ and IADMAS scores (table 5). Furthermore, there was a non-significant difference between the IADMAS and the MAQ in the number of patients at different adherence levels (table 6).

Table 4.

Correlation of IADMAS with HbA1c and MAQ

Parameter	Parameter	Pearson coefficient	P value
IADMAS	HbA1c	−0.423	0.000
IADMAS	MAQ	0.682	0.000

HbA1c, glycosylated hemoglobin; IADMAS, Iraqi Anti-Diabetic Medication Adherence Scale; MAQ, Medication Adherence Questionnaire.

Table 5.

Correlation of IADMAS subscales with HbA1c, MAQ and IADMAS scores

Parameter (questions)	Correlation coefficient
	HbA1c	IADMAS	MAQ
Intentional non-adherence (Q2–Q8)	−0.370**	0.962**	0.702**
Unintentional non-adherence subscale (Q1)	−0.382**	0.521**	0.290**
Non-adherence to time of medication taking (Q1 and Q3)	−0.390**	0.571**	0.277*
Non-adherence to medication dose (Q2, Q6, Q7 and Q8)	−0.368**	0.880**	0.769**
Intentional stop taking of medications (Q5)	−0.249*	0.586**	0.481**
Newly added questions7 8	−0.445**	0.721**	0.968**

*P<0.05, **p<0.01.

HbA1c, glycosylated hemoglobin; IADMAS, Iraqi Anti-Diabetic Medication Adherence Scale; MAQ, Medication Adherence Questionnaire.

Table 6.

Comparison between IADMAS and MAQ

Parameter	IADMAS	MAQ	P value
Low adherence	18 (22.5%)	16 (20%)	0.532
Moderate adherence	33 (41.25%)	40 (50%)
High adherence	29 (36.25%)	24 (30%)
High (absolute) adherence	29 (36.25%)	24 (30%)	0.401
Non-adherence	51 (63.75%)	56 (70%)
High and moderate adherence	62 (77.5%)	64 (80%)	0.699
Low adherence	18 (22.5%)	16 (20%)

IADMAS, Iraqi Anti-Diabetic Medication Adherence Scale; MAQ, Medication Adherence Questionnaire.

There was a significant inverse correlation between HbA1c values and the total scores of the IADMAS (Spearman’s r=−0.423, p=0.000) (table 4). Additionally, all IADMAS subscales showed a significantly weak negative correlation with the HbA1c values, except for the newly added items which have a moderate inverse correlation with HbA1c (table 5). Furthermore, the mean values of HbA1c were significantly different at different levels of medication adherence (table 7).

Table 7.

Association of HbA1c with level of medication adherence

Adherent patients	HbA1c level for adherent patients	HbA1c for non-adherent patients	P value
All patients (moderate and high adherence)	8.42±2.12	11.16±2.35	0.000
Patients with moderate adherence	8.82±2.22	11.16±2.35	0.004
Patients with high adherence	7.98±1.98	11.16±2.35	0.000

HbA1c, glycosylated hemoglobin.

IADMAS diagnostic accuracy

The association between medication adherence and glycemic control, including TP, TN, FP and FN values for the IADMAS and the MAQ, is shown in table 8.

Table 8.

TP, TN, FP and FN values for IADMAS and MAQ

Parameter	Low adherence, n (%)	Moderate and high adherence, n (%)
IADMAS
Poor glycemic control, HbA1c ≥7%	18 (100) (TP)	41 (66.1) (FP)
Good glycemic control, HbA1c <7%	0 (0) (FN)	21 (33.9) (TN)
MAQ
Poor glycemic control, HbA1c ≥7%	14 (87.5) (TP)	45 (70.3) (FP)
Good glycemic control, HbA1c <7%	2 (12.5) (FN)	19 (29.7) (TN)

FN, false negative;FP, false positive;IADMAS, Iraqi Anti-Diabetic Medication Adherence Scale; MAQ, Medication Adherence Questionnaire; TN, true negative; TP, true positive.

The sensitivity and specificity of the IADMAS were 100% and 33.9%, respectively. The positive and negative predictive values were calculated as 30.5% and 100%, respectively, as shown in table 9.

Table 9.

Sensitivity and specificity for IADMAS compared with MAQ

Parameter	IADMAS (%)	MAQ (%)
Sensitivity	100	87.5
Specificity	33.9	29.7
Positive predictive value	30.5	23.7
Negative predictive value	100	90.5

IADMAS, Iraqi Anti-Diabetic Medication Adherence Scale; MAQ, Medication Adherence Questionnaire.

Discussion

This study showed that the IADMAS has optimum internal consistency, stable reliability, and good concurrent validity, sensitivity and specificity.

The psychometric properties of the IADMAS were not significantly different from that of the validated MAQ. Both scales produced Cronbach’s alpha coefficients greater than 0.6. Meanwhile, only the IADMAS produced Cronbach’s alpha score greater than 0.7. This result was in tune with a previously reported one, where a low Cronbach’s alpha (0.55) was achieved for the MAQ among American patients with diabetes.³⁴ Moreover, most references considered a Cronbach’s alpha value greater than 0.6 as acceptable and a value greater than 0.7 as optimum.³⁵

The Cronbach’s alpha coefficient of the IADMAS was increased slightly if item 8 was excluded from the scale; however, this item was retained in the IADMAS since it was considered an essential variable; besides, the improvement in Cronbach’s alpha by excluding this item is marginal and non-significant.

Seven out of eight items showed good corrected item–total correlations; in this regard, other validated scales such as the MALMAS and MMAS-8 had some items with low corrected item–total correlations.¹⁵

The difference at test–retest was statistically non-significant for all IADMAS items. In addition, a good correlation between the test–retest IADMAS total score was obtained. These findings indicate a stable reliability for the IADMAS.

Both the IADMAS and the MAQ showed good correlation. Additionally, the prevalence of medication non-adherence obtained using the IADMAS was not significantly different from that of the MAQ, confirming the convergent validity of the IADMAS.

This study showed a weak but significant inverse correlation between HbA1c values and the total IADMAS scores of the patients; meanwhile, all subscales of IADMAS had a significant inverse correlation with HbA1c. Furthermore, non-adherent patients had significantly higher mean HbA1c levels than those who were adherent. This finding confirmed the concurrent validity of the IADMAS, and it was in tune with other studies that showed a positive correlation between medication adherence and glycemic control.^{36 37}

This study showed that the IADMAS had excellent sensitivity but with low specificity, and this problem was nearly common with all self-reporting methods for patient assessment.³⁸ The specificity of the IADMAS was slightly higher than that of the MALMAS¹⁵ but less than the specificity of the MAQ¹⁷ and the MMAS-8.³⁹ This may be due to the bias in social desirability which may result from face-to-face interview method of administering the questionnaire to many patients⁴⁰; besides, social desirability was very different among cultures and appears to be higher in developing countries such as Iraq.⁴¹ This explanation was supported by the reported low specificity values of the MAQ (29.7%) obtained in this study as compared with that obtained (44%–72%) in studies performed in developed countries.⁴² Additionally, the low specificity of the IADMAS may be attributed to the small sample size.^{43 44}

The present study also showed that the IADMAS had greater sensitivity and specificity than the MAQ (100% vs 87.5% and 33.9% vs 29.7%, respectively), and this finding may confirm that the IADMAS could be more specific for Iraqi patients with DM since the MAQ was a general tool for assessing adherence.¹²

One of the major limitations of this study was the small sample size recruited from only one diabetes center. In addition, the IADMAS was validated in Arabic language, and hence only Arabic Iraqi patients could be included in the study. Therefore, it is recommended to do further studies to translate and validate the IADMAS in different languages that are common among Iraqis, such as Assyrian, Kurdish, and Turkmen. Furthermore, the IADMAS was only validated for patients with type 2 DM, so further studies are needed to confirm its validity among patients with other types of diabetes.

Conclusion

The study demonstrates that the IADMAS has internal consistency and stable reliability. The psychometric properties of the IADMAS are at least similar or slightly better than that of the validated MAQ. Therefore, the IADMAS is a reliable and valid instrument and can be used for assessing antidiabetic medication adherence among Iraqi patients with type 2 DM.