Abstract
Background
The Ethiopian government, supported by NORAD, the WHO, and other partners, is decentralizing diabetes care to primary health units via a task-shifting approach. Despite substantial investment, there is still a lack of up-to-date information on diabetes screening, diagnosis, treatment, and medication availability in the country.
Objective
This study assessed the effects of the NORAD-WHO intervention on diabetes care services, data quality, and the availability of infrastructure and medical supplies in Ethiopia.
Methods
A quasiexperimental study was conducted across 31 NORAD-WHO project facilities and 62 control facilities in six regions of Ethiopia and Addis Ababa. We used descriptive statistics to assess diabetes screening, diagnosis, treatment services, medication availability, and data quality over 54 months from January 2019 to June 2023. Additionally, we performed a difference-in-differences (DID) regression analysis comparing data from two periods: before the intervention (January to December 2019) and after the intervention (July 2022 to June 2023).
Results
This study revealed a notable increase in diabetes services, with over 82% of facilities offering screening, early diagnosis, and treatment. Written treatment guidelines are present in three quarters of the facilities. The proportion of trained staff increased from 58% in 2019 to 100% in 2023 across all the evaluated facilities. Intervention facilities had significantly more functional glucometers than did control facilities, averaging four (95% CI: 3.4, 4.6) per month in 2023 compared with 2.5 (95% CI: 2.1, 2.9) in 2019. However, hemoglobin A1C testing remains uncommon. Despite improvements in diabetes service data, issues with missing records, overreporting, and timeliness persist, with an average reporting rate of 99.2% and on-time reporting rate of 51.5%. The NORAD-WHO intervention notably increased the average number of fasting blood sugar tests by 17 per month (95% CI: 12.2–21.8, p = 0.014).
Conclusions
This midterm evaluation revealed a significant increase in the availability of fasting blood sugar tests in the intervention facilities. Additionally, the availability of medical equipment, laboratory services, and medications has improved over the years. Intervention facilities, with more trained healthcare professionals and better resources, outperform control facilities in screening, diagnosing, treating, and managing high blood sugar levels. Notably, intervention facilities screened more clients for diabetes and showed that patients receiving follow-up care achieved better glycemic control than did those at control facilities. While there has been progress in diabetes service data availability, addressing issues such as missing data, overreporting, and reporting timeliness is essential for further improving the quality of diabetes services.
Clinical trial number
Not applicable.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12875-024-02650-8.
Keywords: Diabetes screening, Diagnosis and treatment services, Availability of medical supplies for diabetes care, Difference-in-differences analysis, NORA-WHO NCDs midterm project evaluation
Background
Diabetes mellitus (DM) contributes to 4% of noncommunicable disease deaths worldwide [1]. In sub-Saharan Africa (SSA), DM causes a large and growing burden of deaths and disabilities, with a regional diabetes prevalence of 5.1% [2]. A systematic review carried out in Ethiopia reported that the prevalence of DM was 6.5% [3], whereas other studies in the country reported that the prevalence of DM varied between 0.5% and 6.5% [4]. The 2015 national stepwise approach to NCD risk factor surveillance (STEPS) revealed that 5.9% of participants had high blood glucose, with only 0.5% receiving treatment [5]. Moreover, the 2014 Ethiopia Service Provision Assessment Plus (ESPA +), 2016 Service Availability and Readiness Assessment (SARA), and 2018 SARA surveys revealed that only 59%, 22%, and 36% of health facilities in Ethiopia offered diabetes screening, diagnosis, and management services, respectively. Individuals with DM are at increased risk of developing several serious life-threatening complications that result in reduced quality of life, increased medical care costs, and mortality [6, 7].
In response to the increasing burden of noncommunicable diseases (NCDs), the Ethiopian government is implementing a national strategic plan for 2020–2025 [8] aimed at reducing NCD incidence through healthy lifestyle promotion, risk factor reduction, and cost-effective treatment. To put this plan into action, Ethiopia has implemented a system of decentralizing noncommunicable disease care from hospitals to local health centers, called task shifting, in 350 health facilities, with support from the WHO and other partners.
With funding from the Norwegian Agency for Development Cooperation (NORAD), the WHO has directly supported task shifting in 87 health facilities from eight regions distributed over 47 districts. The WHO has also supported the training of 2309 health professionals, including 180 pharmacists in these locations. The NORAD-WHO-NCD intervention in Ethiopia aimed to develop integrated primary healthcare pathways for basic noncommunicable disease (NCD) services, focusing on hypertension, diabetes, and cervical cancer prevention. Starting in June 2021, the intervention supported the Ethiopian Ministry of Health in strengthening NCD services across all levels and supported the adoption of the WHO PEN/HEARTS protocol for improved service delivery. The project adopted a multidisciplinary team approach to improve data quality, service availability, and sustainability. It incorporates pharmacy professionals in NCD care to ensure access to essential medicines and lab services. By training healthcare workers and implementing quality assurance measures such as supervision and mentorship, the initiative aims to increase ownership and commitment. Additionally, it seeks to alleviate hospital burdens by shifting NCD services from hospitals to primary health facilities, improving access for more people [9].
Conducting a midterm evaluation study for this initiative helps measure the project’s progress against its goals and objectives, ensuring that it is on track to achieve the desired outcomes. It can highlight challenges, allowing for timely adjustments to the project strategy. Furthermore, it provides stakeholders with clear insights into effectiveness and resource allocation while identifying successful practices that can be continued beyond the project for lasting benefits in diabetes care. Hence, this study was conducted to assess the effectiveness of the NORAD-WHO NCD intervention for diabetes care services, data quality, and the availability of resources for DM in Ethiopia.
Materials and methods
Study setting and period
This midterm evaluation took place in selected public health facilities across six regions of Ethiopia and Addis Ababa from November 12 to December 31, 2023, and involved 93 health facilities (31 interventions and 62 controls). The evaluation covers the period from the start of the intervention in June 2021 to June 2023, which marks the project’s midpoint. It also incorporates baseline data from the two years prior to the intervention (January 2019–December 2020).
Study design
A multicenter quasiexperimental study was conducted in selected public health facilities.
Study population
The study population included NORAD-WHO project health facilities and nearby districts. For each project facility, two control facilities without NCD-related projects since 2018 were selected. If fewer than two controls were available in a district, we included facilities from the nearest district in the same region. If there were more than two potential controls, we randomly selected two.
Sample size
The sample size was determined considering resource availability and access to facilities at the time of data collection. Accordingly, 36% (n = 31) of NORAD-WHO NCD project implementation facilities and 62 control facilities were selected via simple random sampling and included in the study. In terms of the distribution of health facilities across regions, the Addis Ababa, Oromia, Afar, and Benishangul-Gumuz regions each contributed fifteen health facilities; the Amhara and Somali regions contributed nine and six health facilities, respectively; and central Ethiopia accounted for 18 health facilities.
Data collection guide and procedures
A data abstraction form (Supplementary Material 1), adapted from Service Provision Assessment (SPA) survey tools [10], was used to collect information from multiple departments, such as outpatient services, chronic follow-up clinics, maternal and child health clinics, laboratories, and pharmacies. This tool was then reviewed and enhanced by WHO experts. Data were extracted from different recording and reporting formats, including the diabetes screening tally sheet, intake form, treatment cohort register, follow-up form, drug stock register bin card or stock card, and laboratory registry. The form aggregates the total number of required measures monthly, beginning in January 2019 and ending in June 2023. For some indicators, monthly data were extracted from January 2019 to June 2023 for both NORAD-WHO project facilities and control facilities. For other indicators, this analysis considered only two data points, i.e., January 2019 and June 2023, as these two extremes offer better comparisons. The midterm project evaluation data were collected by 19 experts with bachelor’s degrees in health-related fields who work in zonal health offices and healthcare facilities, as well as seven experienced supervisors with master’s degrees in public health.
Data quality assessment
Furthermore, we evaluated the impact of the NORAD-WHO intervention in improving data quality. Accordingly, data quality was evaluated in terms of correctness, completeness, and timeliness using selected diabetes data elements/indicators. For that purpose, we extracted data from the Ethiopian District Health Information System (DHIS2) for 93 health facilities for the period from January 2019 to June 2023 and evaluated the different data quality dimensions against the facility data collected for the NORAD-WHO project midterm evaluation.
We measured the correctness of the data by looking at the difference between the data collected at the facility and the reported data in DHIS2. We measured the correctness of the indicators “number of individuals screened for diabetes mellitus” and “the number of diabetes patients newly enrolled in care.”
The completeness of reporting indicates whether facilities have reported on the data they are supposed to report on. The calculation of completeness is based on whether an organization unit has entered data and clicks the complete button for a certain period and a certain dataset, regardless of content completeness. The timeliness of reporting indicates whether the facility’s reports were delivered on time within 6 days after the end of the reporting period. The completeness and timeliness reporting summary rate corresponds to the service delivery data element for the period from January 2019–June 2022 and the malaria/NTD/NCD data element from July 2022–June 2023.
Detailed operational definitions and measurements of the variables used to evaluate diabetes care services, data quality, and resource availability in Ethiopia are provided in Table 1.
Table 1.
Operational definitions and measurements of variables used to evaluate diabetes care services, data quality, and resource availability in Ethiopia, 2023
| Variable | Operational definition and measurements |
|---|---|
| NORAD-WHO intervention | The NORAD-WHO initiative supports the Ministry of Health in decentralizing non-communicable disease (NCD) care from hospitals to local health centers through a process known as task-shifting. This support includes providing medical equipment, necessary pharmaceuticals for diabetes care, relevant treatment guidelines, training health workers involved in comprehensive NCD care, supportive supervision, and periodic review meetings for quality assurance. |
| Trained staff | A trained staff member is a healthcare provider who has received training in the programmatic and clinical management of non-communicable diseases (NCDs), including diabetes and is assigned to NCD clinics. |
| Diabetes screening | Diabetes screening includes tests that measure blood glucose levels, such as fasting blood glucose tests or hemoglobin A1c tests in individuals at high risk for diabetes. |
| Diabetes diagnosis | It refers to the process by which healthcare providers assess whether a person has diabetes, using specific criteria such as symptoms, medical history, and laboratory tests. |
| Diabetes treatment | It refers to the various methods used to manage and control diabetes, aimed at maintaining blood glucose levels within a target range. This may include lifestyle modifications such as diet and exercise, medication (such as insulin or oral hypoglycemics), and regular monitoring of blood glucose levels. |
| Availability | Diabetes services or medical supply availability was defined as the percentage of facilities in the sample or percentage of months of observation providing or having the required service or supplies at a given time point. |
| Core diabetic medications | The simultaneous availability of three classes of drugs (metformin, insulin, and sulfonylurea) in a given health facility. |
| Data correctness (no difference) | We considered correct data if the difference between the data collected at that facility and the reported data in DHIS2 were within the tolerable range (10%). If the difference is below 90%, we consider it under reporting, and if it is more than 110%, we consider it over reporting. |
| Percent on reporting rate | percentage of the expected reports that have actually been reported |
| Percent reports on time | Percentage of the expected reports that were reported on time (within 6 days after the end of the reporting period) |
Data management and analysis
The collected data were coded and checked for consistency and completeness in Microsoft Excel and exported to STATA software version 17 for Windows. The descriptive statistics of the numeric variables are presented as either means and standard deviations or medians and interquartile ranges, whereas frequencies and percentages are used to present categorical variables. Additionally, the presence of significant differences between groups was assessed via the chi-square test or Student’s t test on the basis of the type of variables considered. To evaluate the impact of NORAD-WHO interventions, we conducted a difference-in-differences (DID) regression analysis for selected indicators before and after the implementation of the NORAD-WHO intervention. In the DID regression analysis, we concentrated on two specific periods: the first 12 months of the baseline (January to December 2019) and the final 12 months of the intervention (July 2022 to June 2023). We excluded the period in between due to the variability in intervention start times across different facilities.
DID provides a powerful way to evaluate the impact of interventions by leveraging natural variation in intervention exposure over time. We checked whether the trends were the same between the treatment and control groups before the intervention date and whether they would have been the same past this date if there had not been an intervention via a graphical diagnostic for parallel trends and the parallel-trends test.
The model equation for difference-in-differences (DiD) regression can be represented as follows:
 |
where:
Yit is the specified outcome variable for individual ‘i’ at time ‘t’.
α is the intercept.
Postt is a binary variable that equals 1 for the posttreatment period (July 2022–June 2023) and 0 for the pretreatment period (January–December 2019).
Treatmenti is a binary variable that equals 1 if the facility is in the NORAD-WHO NCD group and 0 if it is in the control group.
δ is the coefficient for the interaction term, capturing the treatment effect.
γ represents the coefficients for the covariates Xit, which are additional variables included to control for other factors that may influence the outcome (availability of glucometer, availability of usable strips, facility type, and availability of core diabetes medications).
ϵit is the error term.
Results
Health facility profile and infrastructure availability
A total of 93 health facilities (31 intervention and 62 control facilities) were included in this analysis. Among the ninety-three public health facilities that participated in the survey, 58 were in urban districts, and 35 were in rural districts. In terms of facility type, 47 were public health centers without physicians, 13 (41%) of which were intervention health facilities, and 34 (54.84%) were control health facilities. The number of health facilities with a landline or a mobile telephone increased from 42% in 2019 to 45.2% in 2023. Between 2019 and 2023, the number of health facilities with internet access increased from 50.5 to 53.8%. Both electrical backup and computers slightly increased, from 81.7 to 82.8% and 93.6–94.6%, respectively, but there was no improvement in the availability of electricity between 2019 (93.5%) and 2023 (93.5%) (Table 2).
Table 2.
Available basic infrastructures in the selected public health facilities between 2019 and 2023 (n = 93 health facilities)
| Health facility group (January, 2019) | Health facility group (June, 2023) | ||||||
|---|---|---|---|---|---|---|---|
| Control | Intervention | Total | Control | Intervention | Total | ||
| Count (%) | Count (%) | Count (%) | Count (%) | Count (%) | Count (%) | ||
| Name of region | Addis Ababa | 10 (16) | 5 (16) | 15 (16) | 10 (16) | 5 (16) | 15 (16) |
| Afar | 10 (16) | 5 (16) | 15 (16) | 10 (16) | 5 (16) | 15 (16) | |
| Amhara | 6 (9.7) | 3 (9.7) | 9 (9.7) | 6 (9.7) | 3 (9.7) | 9 (9.7) | |
| Benishangul | 10 (16.1) | 5 (16) | 15 (16) | 10 (16.1) | 5 (16) | 15 (16) | |
| Central Ethiopia | 12 (19.4) | 6 (19.4) | 18 (19.4) | 12 (19.4) | 6 (9.4) | 18 (19.4) | |
| Oromia | 10 (16) | 5 (16) | 15 (16.1) | 10 (16) | 5 (16) | 15 (16.1) | |
| Somali | 4 (6.5) | 2 (6.5) | 6 (6.5) | 4 (6.5) | 2 (6.5) | 6 (6.5) | |
| Facility type | Health Center with physician | 21 (33.9) | 14 (45.2) | 35 (37.6) | 21 (33.9) | 14 (45.2) | 35 (37.6) |
| Health Center without physician | 34 (54.8) | 13 (42) | 47 (50.5) | 34 (54.8) | 13 (42) | 47 (50.5) | |
| Primary Hospital | 7 (11.3) | 4 (12.9) | 11 (11.8) | 7 (11.3) | 4 (12.9) | 11 (11.8) | |
| Location of the facility | Rural | 26 (42) | 9 (29) | 35 (37.6) | 26 (42) | 9 (29) | 35 (37.6) |
| Urban | 36 (58) | 22 (71) | 58 (62.4) | 36 (58) | 22 (71) | 58 (62.4) | |
| Landline or a mobile telephone number | No | 35 (56.5) | 19 (61.3) | 54 (58) | 32 (51.6) | 19 (61.3) | 54 (58) |
| Yes | 27 (43.5) | 12 (38.7) | 39 (42) | 30 (48.4) | 12 (38.7) | 42 (45.2) | |
| Electricity | No | 2 (3.2) | 4 (12.9) | 6 (6.5) | 3 (4.8) | 3 (9.7) | 6 (6.5) |
| Yes | 60 (96.8) | 27 (87.1) | 87 (93.5) | 59 (95.2) | 28 (90.3) | 87 (93.5) | |
| Electricity backup like generator, solar panel | No | 13 (21) | 4 (13) | 17 (18.3) | 13 (21) | 3 (9.7) | 16 (17.2) |
| Yes | 49 (79) | 27 (87) | 76 (81.7) | 49 (79) | 28 (90.3) | 77 (82.8) | |
| Internet connection | No | 30 (48.4) | 16 (51.6) | 46 (49.5) | 28 (45.2) | 15 (48.4) | 43 (46.2) |
| Yes | 32 (51.6) | 15 (48.4) | 47 (50.5) | 34 (54.8) | 16 (51.6) | 50 (53.8) | |
| Computer | No | 4 (6.5) | 2 (6.5) | 6 (6.5) | 3 (4.8) | 2 (6.5) | 5 (5.4) |
| Yes | 58 (93.5) | 29 (93.5) | 87 (93.5) | 59 (85.2) | 29 (93.5) | 88 (94.6) | |
Availability of diabetes services and human resources
The percentage of intervention health facilities (n = 31) providing diabetes services increased from 20 (64.5%) in 2019 to 30 (96.8%) in 2023. The percentage of control health facilities (n = 62) providing diabetes services increased significantly (P < 0.013) from 50 (80.7%) in 2019 to 53 (85.5%) in 2023. Among the facilities studied (93), 82% provide screening, early diagnosis, and treatment services for diabetes.
The number of health facilities with written national or WHO treatment guidelines for diabetes significantly improved from 20 (64.5%) in 2019 to 30 (96.8%) in 2023, compared with 39 (62.9%) to 50 (80.7%) in the control health facilities (P < 0.001). The average number of trained staff to provide diabetes health services was two in both 2019 and 2023. The number of health facilities with trained staff increased significantly (P < 0.001) from 18 (58%) in 2019 to 31 (100%) in 2023 in the intervention facilities.
Diabetes healthcare services
The study revealed that monthly outpatient department visits for all patients increased in the intervention facilities, increasing from 2,101 in 2019 to 2,188 in 2023. In contrast, the number of control facilities increased from 1,380 in 2019 to 2,097 in 2023. The study revealed that intervention facilities screen more clients than control facilities do. In 2019, on average, 73 clients (95% CI 48, 97) were screened per month among the intervention facilities, whereas 57 clients (95% CI 47, 66) were screened per month in the control facilities. Likewise, in 2023, 147 clients (95% CI 118, 177) were screened among intervention facilities per month on average, whereas 78 (95% CI 64, 193) clients were screened among control facilities. Similarly, in 2023, a greater number of diabetic patients who had follow-up at the intervention facilities 78 (95% CI 52,103) tended to control their blood sugar than did their diabetic counterparts who had follow-up at the control facilities 23 (95% CI 19, 28) (Table 3).
Table 3.
Average monthly diabetes mellitus screening, diagnosis, and treatment during 2019 and 2023 in selected public health facilities stratified by their intervention status (n = 93 health facilities)
| Variables | 2019 | 2023 | ||
|---|---|---|---|---|
| Intervention status (months of observation) | Mean (95% CI) | Intervention status (months of observation) | Mean (95% CI) | |
| Number of Out Patient Department visits per month | Intervention (n = 178) | 2101 (1808, 2395) | Intervention (n = 186) | 2188 (1886, 2489) |
| Control (n = 551) | 1380 (1248, 1513) | Control (n = 354) | 2097 (1839, 2354) | |
| Number of diabetes patients screened per month | Intervention (n = 92) | 72.6 (48.1, 97.1) | Intervention (n = 182) | 147.3 (117.9, 176.8) |
| Control (n = 318) | 56.6 (46.9, 66.3) | Control (n = 312) | 78.3 (64.1, 92.5) | |
| Number of newly diagnosed diabetic patients per month | Intervention (n = 92) | 1.4 (0.9, 1.8) | Intervention (n = 172) | 2.0 (1.6, 2.5) |
| Control(n = 337) | 0.7 (0.5, 0.9) | Control (n = 335) | 2.2 (1.3, 3.1) | |
| Number of previously registered diabetes patients per month | Intervention (n = 98) | 9.2 (7.5, 10.9) | Intervention (n = 173) | 95.1 (69.9, 120.3) |
| Control (n = 358) | 8.2 (6.1, 10.3) | Control (n = 356) | 34.7 (29.5, 40.0) | |
| Number of diabetic patients on treatment per month | Intervention (n = 91) | 6.1 (4.6, 7.5) | Intervention (n = 164) | 85.3 (59, 111.6) |
| Control (n = 334) | 7.0 (4.9, 9.1) | Control (n = 332) | 25.4 (20.7, 30.1) | |
| Number of diabetes patients with controlled blood sugar per month | Intervention (n = 66) | 3.7 (2.5, 5.0) | Intervention (n = 149) | 77.5 (51.7, 103.3) |
| Control (n = 315) | 4.1 (3.1, 5.1) | Control (n = 305) | 23.4 (19.3, 27.5) | |
Availability of diagnostic tests and supplies for diabetes
The intervention facilities had a significantly greater number of functional glucometers than did the control facilities, with averages of four (95% CI: 3.4, 4.6) and 2.5 (95% CI: 2.1, 2.9) per month in 2023 and 2019, respectively. An average of nine and 12 hemoglobin A1C tests were available per month in six intervention and four control health facilities, respectively, in 2023. The hemoglobin A1C test was not available in 2019. At the intervention facilities, the average number of FBS tests conducted per month increased from 70.9 (95% CI 59.3–82.5) in 2019 to 120.4 (95% CI 99.9–141.0) in 2023. In contrast, the number of control facilities increased from 73.7 (95% CI: 65.7–81.6) in 2019 to 91.1 (95% CI: 77.3–104.8) in 2023. In addition, the percentage of intervention facilities providing FBS measurements increased from 79.3% in 2019 to 98.4% in 2023, whereas the proportion of control facilities increased from 74.2% in 2019 to 82.8% in 2023 (Table 4).
Table 4.
The number of available and functional diagnostic supplies for diabetes in selected public health facilities stratified by their intervention status (n = 93 health facilities)
| Variables | 2019 | 2023 | ||
|---|---|---|---|---|
| Intervention status (months of observation) | Mean (95% CI)s | Intervention status (months of observation) | Mean (95% CI) | |
| Number of available glucometers per month | Intervention (n = 116) | 2.9 (2.3, 3.5) | Intervention (n = 164) | 5.0 (4.3, 5.8) |
| Control (n = 592) | 2.8 (2.4, 3.2) | Control (n = 360) | 3.4 (3.1, 3.8) | |
| Number of functional glucometers per month | Intervention (n = 116) | 2.5 (2.1, 2.9) | Intervention (n = 164) | 4.0 (3.4, 4.6) |
| Control (n = 580) | 2.8 (2.4, 3.2) | Control (n = 360) | 2.7 (2.4, 3.0) | |
| Number of usable strip per month | Intervention (n = 69) | 284.9 (231.0, 338.8) | Intervention (n = 98) | 687.7 (373.7, 1000.8) |
| Control (n = 422) | 246.0 (219.2, 272.7) | Control (n = 277) | 264.5 (208.5, 320.6) | |
| Number of FBS tests available per month | Intervention (n = 210) | 70.9 (59.3, 82.5) | Intervention (n = 183) | 120.4 (99.9, 141.0) |
| Control (n = 459) | 73.7 (65.7, 81.6) | Control (n = 308) | 91.1 (77.3, 104.8) | |
| Number of HbA1C testes available per month | Intervention (n = 0) | … | Intervention (n = 6) | 9.8 (5.8, 13.7) |
| Control (n = 0) | …. | Control (n = 4) | 11.7 (1.2, 22.2) | |
Medication availability
There was a statistically significant difference in the availability of sulfonylurea, metformin, and insulin between the intervention and control facilities (P < 0.05). Intervention facilities were found to have these medications more often than their control facilities did. In 2019, core diabetic medications were available in only 10% of the months in the intervention facilities, whereas they were available in 15.6% of the months in the control health facilities. In 2023, the availability of core diabetic medications improved to 44% in the intervention facilities and 23% in the control facilities.
Effects of the NORAD-WHO intervention on diabetes screening, diagnosis, and treatment services
Table 5 shows the average NORAD-WHO intervention effect via DID regression analysis for the screening, diagnosis, and treatment of diabetes services in selected public health facilities (n = 93 health facilities). Accordingly, the NORAD-WHO intervention significantly increased the number of fasting blood sugar tests available by 17 tests per month on average (95% CI 12.19, 21.77; p = 0.014).
Table 5.
Difference-in-difference regression analysis for the screening, diagnosis, and treatment of diabetes mellitus services in selected public health facilities (n = 93 health facilities)
| ATET (Intervention vs. Control) | Coefficient | Robust std. err | t | p-value | 95% CI | Parallel-trends test |
|---|---|---|---|---|---|---|
| Number of patients screened per month for DM | 16.23 | 4.29 | 3.78 | 0.165 | (-38.34, 70.82) | 0.065 |
| Number of newly diagnosed DM patients per month | 1.72 | 1.76 | 0.98 | 0.507 | (-20.70, 24.15) | 0.714** |
| Number of patients on treatment for DM | 68.09 | 28.21 | 2.41 | 0.250 | (-290.38, 426.58) | 0.670** |
| Number of patients with controlled blood sugar | 64.15 | 25.43 | 2.52 | 0.240 | (-259.04, 387.35) | 0.551** |
| Number of Fasting Blood Sugar (FBS) tests available | 16.98 | 0.37 | 45.08 | 0.014* | (12.19, 21.77) | 0.104** |
Note ATET estimate adjusted for covariates (availability of glucometer, availability of usable strips, facility type, and availability of core diabetes medications), group effects, and time effects
* statistically significant at the 0.05 level of significance
** Parallel-trends test assumption satisfied
ATET: Average Treatment Effect for treated
Figure 1 shows a graphical diagnostic for the parallel trends and average number of patients having a given diabetes service per month for newly diagnosed diabetes patients (A), diabetes patients on treatment for diabetes B), diabetes patients with controlled blood sugar (C), and FBS tests available (D) for 93 health facilities. Accordingly, the parallel trend assumption for DID regression analysis was fulfilled for the number of newly diagnosed diabetes patients, the number of patients on treatment for diabetes, the number of patients with controlled blood sugar, and the number of fasting blood sugar (FBS) tests available.
Fig. 1.
Graphical diagnostic for the parallel trends and average number of patients having a given diabetes service per month for newly diagnosed diabetes patients (A), diabetes patients on treatment for diabetes (B), diabetes patients with controlled blood sugar (C), FBS tests available (D) for 93 health facilities in Ethiopia, 2023
Evaluation of data correctness on the number of patients screened for diabetes
Table 6 compares diabetes screening data from DHIS2 reports with health facility records over several years. In 2019, most records (97.0% for intervention facilities and 96.1% for control facilities) were missing either from DHIS2 or health facility data, leaving only 11 records for intervention facilities and 29 records for control facilities for comparison. Only a few facilities had accurate matching records (one intervention and six control facilities), with some showing underreporting (4 interventions, 10 controls) or overreporting (6 interventions, 13 controls).
Table 6.
Comparison of data correctness between the DHIS2 report and facility survey data using data elements: “Number of individuals screened for diabetes” and number of newly diagnosed diabetes patients
| Indicator | Year | ||||
|---|---|---|---|---|---|
| 2020, n (%) | 2021, n (%) | 2022, n (%) | 2023, n (%) | ||
| Data correctness on the number of patients screened for diabetes | |||||
| Correct | Intervention | 1 (1.5) | 5 (3.7) | 51 (19.1) | - |
| Control | 21 (13.9) | 30 (13.0) | 54(17.0) | - | |
| Over reporting | Intervention | 23 (34.3) | 53 (39.0) | 101(37.8) | - |
| Control | 54(35.8) | 83 (35.9) | 159 (50.0) | - | |
| Under-reporting | Intervention | 43 (64.2) | 78 (57.4) | 115(43.1) | - |
| Control | 76 (50.3) | 118 (51.1) | 105 (33.0) | - | |
| Total reporting months | Intervention | 67 | 136 | 267 | - |
| Control | 151 | 231 | 318 | - | |
| Data correctness on the number of newly diagnosed diabetes | |||||
| Correct | Intervention | 7 (12.3) | 13 (15.9) | 31(22.5) | 21 (31.8) |
| Control | 15 (11.6) | 33 (21.6) | 45 (22.4) | 27 (25.5) | |
| Over reporting | Intervention | 35 (61.4) | 43 (52.4) | 53 (38.4) | 19 (28.8) |
| Control | 87 (67.4) | 86 (56.2) | 106 (52.7) | 47 (44.3) | |
| Under-reporting | Intervention | 15 (26.3) | 26 (31.7) | 54 (39.1) | 26 (39.4) |
| Control | 27 (20.9) | 34 (22.2) | 50 (24.9) | 32 (30.2) | |
| Total reporting months | Intervention | 57 | 82 | 138 | 66 |
| Control | 129 | 153 | 201 | 106 | |
From 2020 to 2022, the level of missing data decreased in both the intervention (from 81.9 to 28.0%) and control (from 79.7 to 59.3%) facilities. Overreporting persisted, with intervention facilities showing rates ranging from 34.3 to 37.8% and control facilities ranging from 35.8 to 50.0% by 2022. From 2020 to 2022, data correctness for intervention health facilities improved significantly, increasing from 1.5 to 19.1%. The percentage of control health facilities slightly increased from 13.9 to 17.0% over the same period. Overreporting remained higher in both types of facilities, with intervention facilities reporting 34.3–37.8% excess and control facilities reporting 35.8–50.0% excess over the three years.
Evaluation of data correctness on the number of patients with newly diagnosed diabetes
In 2019, nearly all records for newly diagnosed DM data elements were missing from both intervention (98.9%) and control (97.3%) health facilities either in facility records or DHIS2 reports. In the intervention facilities, only four records were available for comparison: one correct, two underreported, and one overreported. For the control facilities, twenty records were available, with three correct, thirteen overreported, and four underreported records.
From 2020 to 2023, the percentage of missing records decreased for intervention facilities (from 84.3 to 64.5%) and slightly for control facilities (from 82.3 to 71.5%). The accuracy of records for the “number of newly diagnosed DM patients” data element improved each year: in intervention facilities, from 12.4% in 2020 to 31.8% in 2023, and in control facilities, from 11.6 to 25.5%.
Overreporting was more common in control facilities than in intervention facilities, although both groups showed a decreasing trend from 2020 to 2023. Underreporting was consistently more evident in the intervention facilities than in the control facilities across these years.
Evaluation of health facility monthly reporting summary rates from January 2019 to June 2023
We examined a total of 54 months for each 93 health facilities, providing a total of 5022 months of observations, to calculate the DHIS2 reporting rate summary for the Service Delivery/Malaria/NTD/NCD data element, of which total of 4980 (99.2%) months of observations had valid reports. The average monthly reporting rate (completeness rate) of facilities was 94.8%. The control health facilities had an average monthly reporting rate of 95.7%, which was slightly higher than the 93.1% reported for the intervention health facilities. However, in 2021, intervention facilities had a lower average reporting rate of 89.5%. In terms of timeliness, the average monthly reporting rate over time was 51.5% for all facilities. The overall average monthly reporting rate over time among the control health facilities was 50.9%, whereas it was 52.7% for the intervention health facilities. The average reporting rate on time was slightly lower (38.1%) in 2022.
Discussion
The findings indicate a modest increase in the availability of telephone communication, internet access, electrical backup, and computer availability over the period from 2019 to 2023. The study showed that control health facilities had better access to phones, computers, and electricity, possibly because the project targeted lower-performing facilities for the interventions. A better availability of communication equipment, lower access to electricity, and functioning computers with internet connectivity were shown in a study conducted in the Sidam Regional State, Ethiopia [11].
Compared with the 2014 Ethiopia Service Provision Assessment Plus (59%), 2016 Service Availability and Readiness Assessment (22%), and 2018 Service Availability and Readiness Assessment (36%) [6], this study revealed a significant improvement in diabetes services, with more than 82% of the surveyed facilities providing screening, early diagnosis, and treatment services for diabetes. Similarly, we reported better diabetes service availability than 28% of the primary health care unit (PHCU) identified as ready to manage diabetes reported in a study conducted in the Sidam Regional State, Ethiopia [11]. The significant increase in health facilities providing diabetes services suggests increased accessibility to diabetes care.
This study revealed that three-fourths of facilities had written treatment guidelines for diabetes, which is much better than the Ethiopian 2015 SPA finding where only 12% of the surveyed facilities had guidelines [12], and a study conducted in the Gamo Gofa Zone, southern Ethiopia [13], revealed the nonexistence of diabetes-specific evidence-based guidelines in the three surveyed facilities. Similarly, only 19% of the facilities had NCD guidelines, as shown in a study conducted at 82 health facilities in the Sidama Region of Ethiopia [11]. Another piece of literature shows the low availability of diabetes guidelines in SSA [14]. The adoption of written national or WHO treatment guidelines in a majority of facilities in this study indicates a standardized approach to diabetes management. This can improve the consistency and quality of care across different facilities, reducing the variability in treatment outcomes.
The increase in trained staff from 58% in 2019 to 100% of the facilities in 2023 demonstrates a commitment to capacity building within health facilities. In contrast, a study conducted in the Sidama Region, Ethiopia, revealed that only 27% of the 82 primary health care units had trained staff for the management of diabetes [11]. Furthermore, a systematic review on the readiness of the health care system in sub-Saharan Africa for diabetes showed that there is little training of the healthcare workforce on diabetes service provision [14].
The study indicates that intervention facilities consistently screen a greater number of clients for diabetes than control facilities do, suggesting a more proactive approach to identifying individuals at risk of diabetes. The low level of diabetes screening services in the control health facilities may be attributed to a shortage of diabetes diagnostic materials and supplies, which implies the need to scale up NORAD-WHO interventions to more health facilities.
Our findings revealed a nonoverlapping confidence interval from 2021 onwards, indicating a statistically significant difference between the intervention and control groups in terms of achieving blood sugar control. This contrasts with a study in the Gamo Gofa Zone [13], where only 19.5% reached blood sugar targets, and another study in a teaching hospital in Ethiopia [15], which reported that 40.6% achieved target HbA1c levels.
The findings indicate that intervention facilities presented a significant increase in functional glucometer availability, with 91% compared with 77.2% in control facilities from 2019 to 2023. This number is higher than that reported in a study in the Sidama Region [11], which reported only 50% availability of blood glucose tests. A systematic review in Africa [16] reported that the pooled availability of glucometers was 49.5%. Our study also revealed the absence of routine hemoglobin A1C testing, which is consistent with findings from the Gamo Gofa Zone [13], and in contrast to [15, 16], who reported 51.2% and 24.6% availability for HbA1c tests. This discrepancy may be due to the greater availability of HbA1c testing in hospitals than in primary health care units.
The findings indicate a notable improvement in the availability of core diabetic medications in intervention facilities (27%) compared with control facilities (19%) over the study period. However, it was far below the WHO 80% target [17]. A systematic review in Africa [16] reported that glibenclamide was available in 55.9% of facilities and that metformin was available in 47% of facilities. In the Sidama region [11], the overall availability of metformin tablets was 36.5%, and that of glibenclamide tablets was 39%. Unlike these findings, a study conducted in Central Ethiopia [18] revealed that two glucose-lowering (glibenclamide 5 mg and metformin 500 mg) medications surpassed the WHO target of 80% availability. This discrepancy may be due to the difference in the number of months of observation. In our study, we considered an average of 54 months of observations, whereas others considered only one-time observation.
The DID analysis indicated that the NORAD-WHO intervention significantly increased the availability of fasting blood sugar tests. This supports continued implementation and potential expansion. Additionally, a time series analysis of a patient empowerment program in an Ethiopian hospital revealed a year-over-year increase in outpatient visits for T2DM patients from January 1, 2018, to August 31, 2021 [19].
The research findings present important insights into the quality of diabetes services in Ethiopia, specifically regarding data management and accuracy within health facilities. A reduction in missing records from 2019 to 2023, especially in intervention facilities, indicates a marked improvement in data recording and reporting practices. The increase in data correctness, particularly in intervention facilities, points to improvements in the accuracy of the recorded data. This could mean that facilities are becoming better at accurately capturing and reporting diabetes-related metrics, which is crucial for effective monitoring and decision-making. Our finding is consistent with a study that reported 0.15% positive outliers for the indicator of diabetes detection and 0.04% for the indicator of diabetes visits [20].
The research findings reveal strong data reporting dynamics for NCD services in Ethiopian health facilities, with 99.2% of observation months having valid reports—exceeding the WHO’s 80% target [17]. This rate is higher than the 54% reported for diabetes screening and the 56% reported for diabetes treatment in the 2019 Ethiopia DHS study [20]. The reporting rates of the control facilities were higher than those of the intervention facilities, likely because the intervention focused on lower-performing sites and the earlier implementation of DHIS2 in the control facilities. The decline in reporting for intervention facilities in 2020/2021 may be linked to the COVID-19 pandemic, whereas the drop in 2021/2022 could be related to internal unrest. However, the high completeness rate must be interpreted cautiously, as it does not reflect content completeness or specify NCD services, as the data also include malaria and NTDs.
The average on-time reporting rate of 51.5% indicates that just over half of the reports are submitted within the required timeframe. Timely reporting is crucial for ensuring actionable data, which directly affects the quality of diabetes care and decision-making. Factors such as human resources for RHIS, infrastructure, budget constraints and the lack of regular evaluations and standardized protocols for NCD data collection impact overall data quality [21, 22].
Strengths and limitations of the study
The strengths of this study include its comparative nature, the use of primary data collected from many health facilities covering more than four years and representing all primary health care facilities in the country, and the use of a robust data analysis technique. However, this study is not without limitations. A key issue is that changes in diabetes care services, data quality, and availability of resources may not be attributable to the NORAD-WHO intervention, even though efforts were made to ensure that the intervention and control facilities were similar at baseline, as some improvements were also observed in the control facilities. Furthermore, there were a significant number of missing data for some data elements; as a result, we could not perform trend analysis. In addition, service availability was not measured using the mean availability of all tracer items that are required to offer diabetes services, which may overestimate our findings.
Conclusions
This midterm evaluation revealed a significant increase in the availability of fasting blood sugar tests in the intervention facilities. Additionally, the availability of medical equipment, laboratory services, and medications has improved over the years. Intervention facilities, with more trained healthcare professionals and better resources, outperform control facilities in screening, diagnosing, treating, and managing high blood sugar levels. Notably, intervention facilities screened more clients for diabetes and showed that patients receiving follow-up care achieved better glycemic control than did those at control facilities. While there has been progress in diabetes service data availability, addressing issues such as missing data, overreporting, and reporting timeliness is essential for further improving the quality of diabetes services.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Acknowledgements
The authors extend their sincere thanks to the study data collectors and regional supervisors for their crucial contributions to this research. Their dedication and efforts ensured high-quality data, essential for the study’s success. We also appreciate the support from WHO Ethiopia and the Ministry of Health-Ethiopia during data collection and for providing necessary information.
Abbreviations
- ATET
Average Treatment Effect on the Treated
- DHIS2
District Health Information System
- DID
Difference-in-Differences
- DM
Diabetes Mellitus
- ESPA +
Ethiopia Service Provision Assessment Plus
- HbA1C
Hemoglobin A1C
- LMICs
Low-and-Middle-Income Countries
- NCDs
Non-communicable Diseases
- NORAD
Norwegian Agency for Development Cooperation (NORAD)
- NTD
Neglected Tropical Diseases
- SARA
Service Availability and Readiness Assessment
- SPA
Service Provision Assessment
- SSA
Sub-Saharan Africa
- STEPS
STEPwise approach to NCD risk factor surveillance
- T1DM
Type 1 Diabetes Mellitus
- T2DM
Type 2 Diabetes Mellitus
- WHO
World Health Organization
Author contributions
YSY, GTA, SG, and ABW were involved in the conception and design of the study. YSY, MGS, ZHG, ZGA, and AKZ were involved in data acquisition and data analysis. YSY, GTA, MGS, ZHG, AKZ, ZGA, SG, MT, and KS were involved in data interpretation. YSY, MGS, ZHG, ZGA, KS and AKZ, were involved in manuscript drafting. GTA, SG, MT, and ABW reviewed and amended the draft manuscript. GTA is the guarantor of this study. All authors read and approved the final manuscript.
Funding
This work was supported by the WHO-NORAD [grant number: NA]. The funder has no role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript.
Data availability
All data generated or analysed during this study are included in this published article’s supplementary information files.
Declarations
Ethical approval
Ethical approval was obtained from the Institutional Review Board of the College of Health Sciences, Addis Ababa University (Protocol Number: 063/23/SPH). The Ministry of Health provided a letter of support to all regions and city administrations to help with data collection. Simultaneously, all the regions issued a letter of support to the selected health facilities and shared all the relevant data sources with the data collectors during the collection period.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Organization WH. Noncommunicable diseases country profiles 2018. 2018.
- 2.Nyirenda MJ. Noncommunicable diseases in sub-saharan Africa: understanding the drivers of the epidemic to inform intervention strategies. Int Health. 2016;8(3):157–8. [DOI] [PubMed] [Google Scholar]
- 3.Zeru MA, et al. Prevalence and risk factors for type-2 diabetes mellitus in Ethiopia: systematic review and meta-analysis. Sci Rep. 2021;11(1):21733. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Norris T et al. Predictors of prediabetes/diabetes and hypertension in Ethiopia: reanalysis of the 2015 NCD STEPS survey using causal path diagrams. Matern Child Nutr. 2022;20(55):e13365. [DOI] [PMC free article] [PubMed]
- 5.Ababa A. Ethiopia steps report on risk factors for chronic non-communicable diseases and prevalence of selected NCDs. 2016.
- 6.Defar A, et al. Health system’s availability and readiness of health facilities for chronic noncommunicable diseases: evidence from the Ethiopian national surveys. PLoS ONE. 2024;19(2):e0297622. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ogurtsova K, et al. IDF Diabetes Atlas: global estimates for the prevalence of diabetes for 2015 and 2040. Diabetes Res Clin Pract. 2017;128:40–50. [DOI] [PubMed] [Google Scholar]
- 8.Non-Communicable O. National strategic action plan (nsap) for prevention & control of noncommunicable diseases in Ethiopia. Google Scholar; 2016.
- 9.WHO Regional Office for Africa. 2024. Transforming chronic disease care in Ethiopia, Accessed on 8/1/2024. https://www.afro.who.int/countries/ethiopia/news/transforming-chronic-disease-care
- 10.Organization WH. Service availability and readiness assessment (SARA): an annual monitoring system for service delivery: reference manual. World Health Organization; 2013.
- 11.Mulugeta TK, Kassa DH. Readiness of the primary health care units and associated factors for the management of hypertension and type II diabetes mellitus in Sidama, Ethiopia. PeerJ. 2022;10:e13797. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Bekele A, et al. Service availability and readiness for diabetes care at health facilities in Ethiopia. Ethiop J Health Dev. 2017;31(2):110–8. [Google Scholar]
- 13.Russo TT, et al. Assessment of quality of care provided to adults with type 2 diabetes mellitus at public hospitals in Gamo Gofa Zone, Southern Ethiopia: facility based cross-sectional study. Volume 5. Endocrinology, Diabetes & Metabolism; 2022. p. e355. 5. [DOI] [PMC free article] [PubMed]
- 14.Nuche-Berenguer B, Kupfer LE. Readiness of Sub-saharan Africa Healthcare Systems for the New Pandemic, diabetes: a systematic review. J Diabetes Res. 2018;2018(1):9262395. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Akale M, Tadesse T, Arega B. Assessment of quality of diabetic care in teaching hospitals in Ethiopia: in comparison to international guidelines. Diabetes Metab Syndr Obes. 2024;2024:17:691–700. [DOI] [PMC free article] [PubMed]
- 16.Kibirige D, et al. Availability and affordability of essential medicines and diagnostic tests for diabetes mellitus in Africa. Tropical Med Int Health. 2022;27(11):942–60. [DOI] [PubMed] [Google Scholar]
- 17.World Health Organization. (2017). Data quality review: module 1: framework and metrics. World Health Organization. https://iris.who.int/handle/10665/259224. Licence: CC BY-NC-SA 3.0 IGO.
- 18.Deressa HD, et al. Access to essential medicines for diabetes care: availability, price, and affordability in central Ethiopia. Global Health Res Policy. 2024;9(1):12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Benoni R, Sartorello A, Uliana M, Solomon H, Bertolino A, Pedot A, et al. Epidemiological factors affecting outpatient department service utilization and hospitalization in patients with diabetes: a time-series analysis from an Ethiopian hospital between 2018 and 2021. J Glob Health. 2022;12:04085. [DOI] [PMC free article] [PubMed]
- 20.Ayele W, et al. Quality of routine health data at the onset of the COVID-19 pandemic in Ethiopia, Haiti, Laos, Nepal, and South Africa. Popul Health Metrics. 2023;21(1):7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Adane A, et al. Exploring data quality and use of the routine health information system in Ethiopia: a mixed-methods study. BMJ open. 2021;11(12):e050356. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Kassa MD, Grace JM. A mixed-method study of quality, availability and timeliness of noncommunicable disease (NCD) related data and its link to NCD prevention: perceptions of health care workers in Ethiopia. Health Inform Manage J. 2019;48(3):135–43. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All data generated or analysed during this study are included in this published article’s supplementary information files.