Abstract.
Since 2010, the WHO has recommended that clinical decision-making for malaria case management be performed based on the results of a parasitological test result. Between 2015 and 2017, the U.S. President’s Malaria Initiative–funded MalariaCare project supported the implementation of this practice in eight sub-Saharan African countries through 5,382 outreach training and supportive supervision visits to 3,563 health facilities. During these visits, trained government supervisors used a 25-point checklist to observe clinicians’ performance in outpatient departments, and then provided structured mentoring and action planning. At baseline, more than 90% of facilities demonstrated a good understanding of WHO recommendations—when tests should be ordered, using test results to develop an accurate final diagnosis, severity assessment, and providing the correct prescription. However, significant deficits were found in history taking, conducting a physical examination, and communicating with patients and their caregivers. After three visits, worker performance demonstrated steady improvement—in particular, with checking for factors associated with increased morbidity and mortality: one sign of severe malaria (72.9–85.5%), pregnancy (81.1–87.4%), and anemia (77.2–86.4%). A regression analysis predicted an overall improvement in clinical performance of 6.3% (P < 0.001) by the third visit. These findings indicate that in most health facilities, there is good baseline knowledge on the processes of quality clinical management, but further training and on-site mentoring are needed to improve the clinical interaction that focuses on second-order decision-making, such as severity of illness, management of non-malarial fever, and completing the patient–provider communication loop.
INTRODUCTION
It is estimated that children in sub-Saharan Africa experience 3–4 episodes of fever per year. Malaria contributes to a large proportion of these fevers, although with expansion of prevention and control measures, the adage “all fever equals malaria” is less true today.1–4 A host of other infections contributes to a growing proportion of causes of fever, and serious bacterial infections in particular contribute to a large proportion of severe morbidity and mortality.5 Key to averting morbidity and mortality from malaria and other febrile illnesses is the skill of health-care workers (HCWs) to make an accurate and timely diagnosis based on history, physical examination, and diagnostic testing to guide proper treatment.
To positively affect morbidity and mortality rates, HCWs must be adepts at quickly identifying severely ill patients for immediate triage and at identifying the underlying causes of their illness early enough to prevent progression to severe disease. A focused history and physical examination as is recommended by the WHO standard of care algorithms for the integrated management of childhood illnesses is critical for differentiating causes, determining the need for specific diagnostic testing, and forming a pretest probability for potential causes. The cardinal symptom of malaria, fever, is highly sensitive but not specific,6 so the availability of malaria rapid diagnostic tests (RDTs) and/or microscopy, their proper application, and adherence to the results are important for the identification or exclusion of one of the primary causes of fever.
MalariaCare was a 5-year U.S. President’s Malaria Initiative (PMI)-funded project that worked in 17 countries to strengthen national malaria control programs’ (NMCPs’) capacity in malaria and febrile case management through implementation of a comprehensive quality assurance (QA) scheme. A key component of the scheme was outreach training and supportive supervision (OTSS) to monitor and improve the performance of health facility staff. Under OTSS, trained supervisors used a standardized checklist to observe clinical staff conducting a febrile consultation, provided individualized feedback on steps carried out correctly and incorrectly, and developed action plans together with facility staff to address broader health facility issues.7 Using MalariaCare program data, we evaluated the impact of OTSS on health facility performance toward meeting clinical competencies during outpatient department consultations.
Clinical supervisors (physicians, mid-level clinical officers, nurses, and pharmacists) were selected by the NMCP or subnational health management teams with assistance from MalariaCare to ensure that qualified staff were selected. They underwent both supervisor training and refresher skills training. Supervisor training took an average of 3 days to complete, was conducted by the project and NMCPs, focused on using the checklist and mentoring techniques, and included both didactic and role-playing sessions and 1 day of practical training at a local facility. Skill-based training lasted 5 days and focused on an overview of the local epidemiology of malaria, a review of national guidelines, and the management of uncomplicated and severe illness, and included both didactic and practical sessions. Practical sessions included simulated patient encounters to practice focused physical examinations and identify reasons for lack of adherence, and to discuss strategies to avoid pitfalls such as treating test-negative cases with antimalarial therapy because of patient expectations. During infield implementation of OTSS, a supervisory team would typically visit one or two facilities per day. Clinical staff were observed performing febrile clinical consultations using a 25-point checklist to indicate if each of the steps was performed correctly by the HCW. We present an analysis of programmatic data from eight countries on the performance of clinical staff, as determined by OTSS supervisors, in conducting a clinical consultation for fever at health facilities across administrative levels of the health-care system. Although these data represent program implementation data and are not generated as part of a controlled study, the large size of the data set under evaluation provides important information and guidance for governments and other large programs that implement malaria clinical case management services across countries and regions.
MATERIALS AND METHODS
Program setting and population.
Eight countries in sub-Saharan Africa were supported by the PMI-funded MalariaCare project to improve malaria case management through implementation of clinical OTSS (the Democratic Republic of the Congo, Ghana, Kenya, Malawi, Mali, Mozambique, Tanzania, and Zambia). In these countries, the Ministry of Health (MOH) selected health facilities (primarily hospitals and health centers) from the public and private sectors within regions and districts mutually agreed on between the ministry and United States Agency for International Development (USAID) mission to receive OTSS visits. Within facilities, health facility staff who performed clinical consultations to manage febrile illness were eligible for observation and feedback. Staff were classified as either medical doctors/medical officers or nonmedical doctors and were asked whether they had received formal training in malaria case management in the 2 years before the observation. The checklist also captured important infrastructural capacity, such as the presence in the facility of malaria case management guidelines and algorithms, and the status of artemisinin combination therapies (ACTs) in stock, the lack of which could limit the ability to dispense recommended treatment.
Program description.
From 2015 to 2017, MOH clinical supervisors, trained in malaria case management and supervision, conducted OTSS visits to hospitals, health centers, and health posts (dispensaries) and observed the clinical performance of eligible staff using a standardized checklist. Depending on resources, some countries received visits to all levels of health facility within target regions, whereas countries with less program support received visits primarily to reference-level hospitals. Some facilities received visits before 2015, but with a different checklist. Supervisors (usually matched with like trained mentees) were instructed to conduct three observations of any staff performing clinical consultations with febrile patients who were available at the time. They were instructed to preferentially choose three different clinicians but could include repeat observations of the same clinician if less than three providers were present. The project recommended that the same supervisor perform follow-up visits; however, this was not systematically monitored. In addition, the follow-up visit observations were not restricted to the same cadre of clinicians and no attempt was made to track progress of individual providers. Each observed clinician was evaluated using a 25-point checklist developed by the MalariaCare clinical team through an iterative process with field testing before this study period.7 Although development of this tool was largely a de novo process, several country-level and WHO malaria case management tools were reviewed in preparation.8–10 The checklist focuses on the most important clinical steps in conducting a clinical assessment as determined by MalariaCare’s technical team in consultation with the NMCPs (Table 1) and is designed to assess an individual clinician’s case management decision-making process, independent of whether all critical supplies were present during the visit. Each item on the checklist was phrased as a question, and observers were prompted to check “yes” or “no” based on whether the HCW made the appropriate choice. Four of the items were “minimum standard” steps that are considered essential for correct case management (outlined in bold).
Table 1.
Steps in performance of clinical consultation evaluated on outreach training and supportive supervision checklist
| Procedure* | Score |
|---|---|
| Determines the age of the patient | ☐Yes ☐No |
| Determines the weight of the patient | ☐Yes ☐No |
| Asks whether the patient is pregnant (count as yes if male or not of reproductive age) | ☐Yes ☐No |
| Asks about current or recent fever | ☐Yes ☐No |
| Asks about diarrhea | ☐Yes ☐No |
| Asks about vomiting | ☐Yes ☐No |
| Asks about coughing | ☐Yes ☐No |
| Asks about any other symptoms | ☐Yes ☐No |
| Asks whether the patient was treated before the visit (at home or another facility) | ☐Yes ☐No |
| Checks for at least one sign of severe malaria (or apparent) | ☐Yes ☐No |
| Checks for evidence of anemia | ☐Yes ☐No |
| Checks for fast breathing or chest indrawing | ☐Yes ☐No |
| Checks the heart rate | ☐Yes ☐No |
| Takes temperature | ☐Yes ☐No |
| Conducts neck examination/checks for stiffness | ☐Yes ☐No |
| Conducts lung examination | ☐Yes ☐No |
| Conducts abdominal examination/checks for abdominal stiffness | ☐Yes ☐No |
| Conducts skin examination/checks for rash or dehydration | ☐Yes ☐No |
| Checks for altered consciousness | ☐Yes ☐No |
| Supervisor agrees with whether a malaria test should be ordered* | ☐Yes ☐No |
| Supervisor agrees with the final diagnosis and severity assessment | ☐Yes ☐No |
| Correct prescription per test result and diagnosis | ☐Yes ☐No |
| Informs caregiver on what is wrong with the patient | ☐Yes ☐No |
| Gives advice on how to take the prescribed medications (outpatients) or informs caregiver of transfer (inpatients) | ☐Yes ☐No |
| Asks the caregiver/patient whether they have any questions | ☐Yes ☐No |
Note: Checklist steps in bold (“minimum standard” steps) are considered more important and are collectively weighted twice as much as the other steps when calculating scores.
* Counted as correct if no test is available.
The score for each observation was weighted so that the four “minimum standard” steps accounted for two-thirds of the score, whereas the remaining 21 steps accounted for one-third. The primary study measure—individual health facility clinical performance scores—comprised an average of one to three HCW observations per health facility. The amount of time between visits for each health facility was typically 3–6 months in length and depended on several factors, including MOH schedules and project budgets.
Analysis of implementation data.
Results gathered during observation of HCWs performing clinical consultations were either captured on a paper-based checklist and subsequently entered into a Microsoft Access database (Microsoft Corporation, Redmond, WA) or entered directly by supervisors into tablets running MalariaCare’s electronic data system application which links to District Health Information System 2 (DHIS2) software (University of Oslo, Oslo, Norway) for data storage and analysis.11 Data from the Access and DHIS2 databases were imported into Stata 14 (StataCorp, 2015. Stata Statistical Software: Release 14.1. College Station, TX: StataCorp LP) for data cleaning and analysis.
Results were analyzed at two levels: at the observation level (for HCW performance on individual checklist steps) and at the facility level (for overall facility improvement on clinical competencies over time). We reported performance on each of the individual 25 steps in the checklist as the proportion of HCWs observed who performed the step correctly during the first, second, and third visit among observations with no steps missing, among facilities with at least three visits and at least one complete observation at each time point. If a HCW was observed more than once during a visit, only the results for the first observation were included. To estimate the impact of OTSS on facility performance by individual visit and over time, HCW observation scores included in the individual step analysis were averaged across each visit to obtain a facility score. We reported average facility scores by country and visit.
Finally, a multi-level mixed-effects linear regression with clustering at the health facility level was conducted on facilities with at least two visits to estimate the independent effect of an OTSS visit versus other facility characteristics on performance scores. Potential factors included in the regression were as follows: number of OTSS visits received, whether the facility received a prior OTSS visit using an older version of the checklist, whether malaria case management guidelines and algorithms were available in the facility, whether all of the observed HCWs were doctors/medical officers, whether all observed HCWs had received formal malaria case management training within the 2 years before the visit, whether the health facility was a hospital, whether there was a stock-out of ACTs lasting more than 7 days in the past 3 months, and country.
RESULTS
At peak implementation, the project covered 53% (4,785 of 8,970) of health facilities within target regions and 13% of all health facilities within these countries; however, the regional coverage variance was considerable from < 1% in the Democratic Republic of Congo, where select provincial-level hospitals were targeted to 98% in Kenya where all public health facilities within eight target counties received mentoring.12 All results were blinded at country level to protect individual country confidentiality. A total of 3,563 health facilities received at least two clinical OTSS visits, with 1,759 (49%) receiving two visits, 1,662 (47%) receiving three visits, and 142 (4%) receiving four or more visits. The proportion of health facilities in each of the eight countries ranged from 1% (n = 24) to 41% (n = 1,459). Of the 3,563 health facilities visited, 3,480 had a complete observation for any visit. To better understand reasons why the clinical checklists were not always completed during a visit, the MalariaCare project introduced a revision of the checklist (typically during the last visit) that allowed capture of likely causes. Among visits with this checklist revision (n = 5,387), 81.7% of clinical observations were fully completed, 17.4% were started but left incomplete without an explanation, and 0.9% were not initiated because of factors beyond the supervisor’s control (a lack of patients with fever, no clinical workers present during the visit, the patient was referred out before starting the checklist, and the patient was referred to the laboratory and did not return).
Observation performance on individual steps.
Of the 1,804 health facilities visited at least three times, 944 (52%) had at least one complete observation at each of the first three visits. A total of 4,134 observations from these facilities were analyzed. Table 2 compares the proportion of HCWs observed at these facilities who performed each checklist step correctly at each visit.
Table 2.
Proportion of health-care workers observed who performed clinical checklist steps correctly: first, second, and third visit (total number of observations: 4,134)
| Clinical observation checklist step | Visit number | Percentage point change in score | ||||
|---|---|---|---|---|---|---|
| First | Second | Third | First to second | Second to third | First to third | |
| Number of observations | 1,352 | 1,357 | 1,395 | – | – | – |
| Determines the age of the patient | 88.5% | 92.5% | 93.3% | +4.0% | +0.8% | +4.8% |
| Determines the weight of the patient | 80.3% | 85.4% | 85.4% | +5.1% | 0.0% | +5.1% |
| Asks whether the patient is pregnant (count as yes if male or not of reproductive age) | 81.1% | 85.4% | 87.4% | +4.3% | +2.0% | +6.3% |
| Asks about current or recent fever | 94.7% | 96.4% | 96.1% | +1.7% | −0.3% | +1.4% |
| Asks about diarrhea | 59.1% | 68.0% | 74.6% | +8.9% | +6.6% | +15.5% |
| Asks about vomiting | 67.7% | 75.8% | 79.4% | +8.1% | +3.6% | +11.7% |
| Asks about coughing | 67.9% | 75.6% | 77.8% | +7.7% | +2.2% | +9.9% |
| Asks about any other symptoms | 38.9% | 49.3% | 49.5% | +10.4% | +0.2% | +10.6% |
| Asks whether the patient was treated before visit (at home or another facility) | 56.3% | 65.1% | 67.0% | +8.8% | +1.9% | +10.7% |
| Checks for at least one sign of severe malaria (or apparent) | 72.9% | 86.2% | 85.5% | +13.3% | −0.7% | +12.6% |
| Checks for evidence of anemia | 77.2% | 84.0% | 86.4% | +6.8% | +2.4% | +9.2% |
| Checks for fast breathing or chest indrawing | 41.9% | 57.1% | 60.0% | +15.2% | +2.9% | +18.1% |
| Checks the heart rate | 30.0% | 46.6% | 47.2% | +16.6% | +0.6% | +17.2% |
| Takes temperature | 73.9% | 83.4% | 83.5% | +9.5% | +0.1% | +9.6% |
| Conducts neck examination/checks for stiffness | 27.9% | 41.1% | 46.4% | +13.2% | +5.3% | +18.5% |
| Conducts lung examination | 28.4% | 42.1% | 44.7% | +13.7% | +2.6% | +16.3% |
| Conducts abdominal examination/checks for abdominal stiffness | 35.2% | 51.9% | 54.3% | +16.7% | +2.4% | +19.1% |
| Conducts skin examination/checks for rash or dehydration | 37.4% | 54.6% | 58.1% | +17.2% | +3.5% | +20.7% |
| Checks for altered consciousness | 37.5% | 48.1% | 54.0% | +10.6% | +5.9% | +16.5% |
| Supervisor agrees with whether a malaria test should be ordered* | 93.4% | 95.4% | 95.7% | +2.0% | +0.3% | +2.3% |
| Supervisor agrees with the final diagnosis and severity assessment | 93.3% | 95.7% | 96.3% | +2.4% | +0.6% | +3.0% |
| Correct prescription per test result and diagnosis | 94.0% | 94.2% | 96.4% | +0.2% | +2.2% | +2.4% |
| Informs the caregiver on what is wrong with the patient | 83.9% | 90.4% | 90.3% | +6.5% | −0.1% | +6.4% |
| Gives advice on how to take the prescribed medications (outpatients) or informs the caregiver of transfer (inpatients) | 79.5% | 86.0% | 87.0% | +6.5% | +1.0% | +7.5% |
| Asks the caregiver/patient whether they have any questions | 33.7% | 46.0% | 45.7% | +12.3% | −0.3% | +12.0% |
Note: Checklist steps in bold (“minimum standard steps”) are considered more important and are collectively weighted twice as much as the other steps when calculating scores.
* Counted as yes if no test is available. The proportion that did not have a rapid diagnostic test or microscopy test available was 7.0%, 2.5%, and 1.2%, at the first, second, and third visits, respectively.
Of the four steps considered by the project to be most important in diagnosing and treating a case of malaria—1) checking for at least one sign of severe malaria, 2) supervisor agreement on whether a malaria test should be ordered, 3) supervisor agreement on the final diagnosis and severity level, and 4) giving the correct prescription per diagnosis—the latter three were performed at a high standard on the initial visit (93.3–94.0%) and were all greater than 95% by the third visit. Checking for at least one sign of severe malaria was conducted by 72.9% of HCWs observed at the initial visit but increased by 12.6 percentage points by the third visit, to 85.5%. Several other steps started below but reached (or nearly reached) the 90% target by the third visit: determining the age of the patient, determining the weight of the patient, asking whether a patient is pregnant, asking about fever, checking for evidence of anemia, informing patient or caregiver about what was wrong, and giving advice on how to take medications or informing patient or caregiver of transfer. In addition, key steps in history taking (asking about diarrhea, vomiting, or any other symptoms), performance of the focused physical examination (checking for rapid breathing, checking heart rate, conducting neck/lung/abdominal/skin examination, looking for evidence of dehydration, and checking for altered mental status), and improving communication (asking patient or caregiver whether they have questions) all improved by more than 10 percentage points. For 15 of the 25 steps, improvement between the second and third visit was less than one percentage point, whereas three steps improved by more than five percentage points: asking about diarrhea, conducting a neck examination, and checking for altered consciousness.
Measuring overall facility performance.
Facility scores were calculated from the 4,134 observations included in the previous analysis by averaging observation scores across each visit. Of the 1,804 facilities that received at least three visits, 944 (52%) had scores for all three visits (thus, the average number of clinical observations per visit was 1.46). Figure 1 shows the average facility performance by country during each visit (two of the eight countries were not included because no facilities received three visits with the revised checklist). Baseline average performance was 79%, and performance during the third visit was 86% (an increase of seven percentage points) and their CIs did not overlap. For four of the six countries, average scores improved steadily. Two countries (one and four) showed declines between the second and third visits, but CIs remained large.
Figure 1.
Average facility performance score over consecutive rounds of outreach training and supportive supervision, by country.
Regression analysis.
Facilities with at least two OTSS visits were considered for further trend analysis. Of the 8,974 clinical visits conducted in 3,563 eligible facilities, 7,276 visits (81%), representing 3,482 health facilities, had at least one complete clinical observation and were included in the unadjusted regression analysis. A subset of 6,642 clinical OTSS visits (74%), representing 3,361 health facilities, also had data for all assessed covariates and was included in the adjusted regression analysis.
Table 3 presents the health facility characteristics of the visit sample included in the adjusted analysis and the mean clinical facility score for each covariate. The range around the mean is indicated by the 25th percentile and 75th percentile scores, respectively. Mean scores by the visit number ranged from 80.3% (25th percentile 71.4, 75th percentile 90.5) to 85.2% (25th percentile 78.6, 75th percentile 93.7).
Table 3.
Health facility characteristics based on OTSS visits included in the adjusted regression analysis, mean facility clinical scores, and scores at the 25th and 75th percentile by facility characteristic (N = 6,642)
| Characteristic | Visits | Clinical score | |||
|---|---|---|---|---|---|
| % | N | Mean | 25th percentile | 75th percentile | |
| Visit number | |||||
| 1 | 36.5 | 2,425 | 80.3 | 71.4 | 90.5 |
| 2 | 40.7 | 2,703 | 85.6 | 79.4 | 95.2 |
| 3 | 22.8 | 1,514 | 85.2 | 78.6 | 93.7 |
| Among all visits | |||||
| Facility received prior OTSS clinical visits | |||||
| No | 38.3 | 2,542 | 83.7 | 77.8 | 93.1 |
| Yes | 61.7 | 4,100 | 83.5 | 75.4 | 93.7 |
| Facility has most recent malaria case management guidelines | |||||
| No | 24.8 | 1,649 | 81.3 | 73.0 | 91.3 |
| Yes | 75.2 | 4,993 | 84.3 | 77.8 | 93.7 |
| Facility has malaria case management algorithms | |||||
| No | 24.4 | 1,620 | 81.2 | 73.0 | 90.5 |
| Yes | 75.6 | 5,022 | 84.3 | 77.0 | 93.7 |
| All HCWs observed are doctors/medical officers | |||||
| No | 83.6 | 5,550 | 83.8 | 76.2 | 93.7 |
| Yes | 16.4 | 1,092 | 82.5 | 73 | 93.7 |
| All HCWs observed have formal training in malaria case management | |||||
| No | 36.9 | 2,449 | 82.1 | 74.6 | 92.1 |
| Yes | 63.1 | 4,193 | 84.4 | 77.8 | 93.7 |
| No stock-out of ACT > 7 days in past 3 months | |||||
| No | 20.2 | 1,345 | 82.6 | 76.2 | 92.1 |
| Yes | 79.8 | 5,297 | 83.8 | 76.2 | 93.7 |
| Facility is a hospital | |||||
| No | 88.8 | 5,897 | 83.5 | 76.2 | 93.7 |
| Yes | 11.2 | 745 | 84.2 | 76.2 | 94.4 |
| Country | |||||
| Country 1 | 36.2 | 2,403 | 84.5 | 76.2 | 94.4 |
| Country 2 | 29.6 | 1,964 | 84.5 | 79.4 | 93.7 |
| Country 3 | 13.2 | 879 | 79.4 | 69.8 | 88.9 |
| Country 4 | 3.9 | 232 | 90.4 | 86.2 | 96.8 |
| Country 5 | 2.3 | 151 | 75.8 | 65.9 | 85.7 |
| Country 6 | 10.8 | 716 | 83.6 | 77.8 | 93.7 |
| Country 7 | 0.5 | 36 | 77.4 | 67.5 | 86.8 |
| Country 8 | 3.9 | 261 | 90.4 | 86.2 | 96.8 |
OTSS = outreach training and supportive supervision; HCW = health-care worker; ACT = artemisinin combination therapy.
Of the 6,642 facility visits evaluated in the adjusted regression analysis, 36.5% were a first visit, 40.7% were a second visit, and 22.8% were a third visit. Most of the visits occurred at facilities that had a previous OTSS visit before the use of the updated checklist (61.7%) and had case management guidelines (75.2%) and algorithms (75.6%) available. Overall, the majority were in nonhospital settings (88.8%) and included observations of nondoctor clinicians (83.6%). A high proportion of the visits was conducted in just two MalariaCare-supported countries (countries one and two combined accounted for 65.9%).
In the unadjusted regression, a facility’s clinical score was estimated as 5.4 percentage points higher during the second OTSS visit (P < 0.001) and 6.5 percentage points higher during the third OTSS visit (P < 0.001) when compared with the first. After adjusting for facility characteristics, a facility’s clinical score was an estimated 5.1 percentage points higher during the second OTSS visit (P < 0.001) and 6.3 percentage points higher during the third OTSS visit (P < 0.001) when compared with the first (Table 4). Thus, the estimated increase in score between the second and third visits is 1.2 percentage points (P = 0.001).
Table 4.
Adjusted regression results for visit characteristics associated with percentage point improvement in scores (n = 6,642)
| Characteristic | Coefficient | 95% CI |
|---|---|---|
| Number of visits (ref: one visit) | ||
| 2 | 5.1 | [4.5, 5.7] |
| 3 | 6.3 | [5.5, 7.0] |
| Had prior outreach training and supportive supervision visits | 1.2 | [−0.3, 2.7] |
| Facility has most recent malaria case management guidelines | 1.7 | [1.0, 2.5] |
| Facility has malaria case management algorithms | 1.9 | [1.1, 2.7] |
| All HCWs observed are doctor/medical officer | 1.5 | [0.5, 2.4] |
| All HCWs observed have formal training in malaria case management | 2.1 | [1.5, 2.7] |
| No stock-out of ACT > 7 days in past 3 months | 0.8 | [−0.02, 1.6] |
| Facility is a hospital | 1.8 | [0.7, 2.9] |
Coefficient = % change in clinical score; HCWs = health-care workers. Note: The regression included a control variable for the country and results are not reported here.
Aside from OTSS visits, given 95% CIs, five factors were associated with facility improvement in clinical performance: having the most recent national clinical guidelines present during the visit (1.7 percentage points, P < 0.001), having malaria case management algorithms during the visit (1.9 percentage points, P < 0.001), all observed clinicians being doctors/medical officers (1.5 percentage points, P = 0.004), all observed clinicians having formal training in malaria case management in the last 2 years (2.1 percentage points, P < 0.001), and being a hospital (1.8 percentage points, P < 0.001). Neither prior OTSS visits before the checklist update nor stock-outs of ACTs for more than 7 days in the last 3 months were found to have a statistically significant impact on the clinical score.
DISCUSSION
A key objective of the MalariaCare project was to improve the clinical performance of outpatient clinical care for patients who present to health facilities with suspected malaria. This implementation study on the use of OTSS in the outpatient department of health facilities in eight sub-Saharan African countries describes rapid improvement in the quality of performance for both individual clinicians and in the health facilities they work in after just three mentoring visits. These improvements are likely due to several important processes instituted by OTSS: a renewal of latent knowledge that accounts for early rapid improvements, a focus on identifying and addressing key knowledge and practice gaps, working with facilities to improve their operational bottlenecks, and providing a steady and sustained external emphasis on improving the quality over time.
At baseline, greater than 90% of the facilities in this study were accurately performing three of four critical clinical performance steps: ordering a malaria test when needed, making the correct diagnosis and determining the correct level of disease severity, and providing the right prescription for malaria when tests were positive. This suggests that HCWs already knew correct protocols for testing and treating malaria in most cases, despite a lack of general clinical internal QA (IQA) measures, and this is a baseline improvement in quality described in prior studies, but consistent with a trend toward improvement with introduction of test-based strategies.13–15 However, key performance deficits were observed in history taking and performance of the physical examination, which are important to early triage of critical illness and evaluation for other non-malarial causes of disease. In addition, weaknesses in communication with the patient and their caregiver also were observed.
In subsequent visits, MalariaCare focused on improving specific gaps identified at each facility. By the second visit, there was dramatic improvement in percentage of HCWs that looked for at least one sign of severe malaria (exceeding 86%)—the fourth critical step in quality case management. The purpose of recognizing at least one sign of severe malaria is to trigger consideration for the critical illness pathway, thus initiating a rapid sequence of triage, evaluation, and treatment (or referral to a higher level facility in the case of lower level facilities without inpatient capacity). This rapid improvement again indicates baseline knowledge that can be rejuvenated through on-site mentoring.
Significant improvements in performance for other targeted history taking and examination skills were also made, with checking for anemia and taking a temperature exceeding 85%. Other steps, such as the neck, lung, and abdominal examinations, showed steady improvements, but were still being performed approximately half of the time, thus requiring continuing attention. These findings are important because getting HCWs to perform focused history and physical examinations continues to be a challenge.16–21 This is not surprising, given the large patient burden and lack of support staff in most countries, but it can contribute to misdiagnosis and poor patient management. Given the time constraints faced by the typical HCW, future work in this area should continue to focus on training HCWs to perform rapid and focused history and physical examinations designed to quickly assess for evidence of severe disease and important secondary, non-malaria diagnoses (such as pneumonia, bacterial sepsis, or a catastrophic abdominal event).
The adjusted regression analysis showed that clinical performance against the checklist improved by an estimated six percentage points by the third visit, holding other relevant factors constant. Other factors benefiting performance at a smaller but statistically significant level—presence of clinical guidelines, presence of clinical algorithms, increased level of clinical training, and formal malaria case management training—indicate where resources are beneficial for future programming. The finding that hospitals—which also started at slightly higher baselines than nonhospitals (81.5 percent versus 79.8 percent)—made greater improvements than nonhospitals may reflect the higher level of training of clinical staff.
Overall, although the lack of a control group prevents definitive attribution of OTSS to these improvements, the robustness of the findings across eight countries and the high quality of data capture enabled by the electronic system lend weight to the argument that these targeted on-site interventions can stimulate rapid improvements in a short period of time.
We envision several risks to progress made in clinical case management. Prolonged stock-outs of key commodities—ACTs, RDTs, and microscopy staining materials—may lead to a regression to reliance on clinical diagnosis and treatment (as opposed to a test-based approach). In addition, a lack of supportive training on case management protocols may cause a deterioration of the background knowledge necessary to maintain quality-assured practices. Another factor that may affect sustainability of QA efforts is the rapid turnover of clinical staff experienced in most of these facilities, thus continuing investment in skills refreshment and improving work in preservice training. On the administrative side, given that institutional leadership is often critical to maintaining a focus on quality, interventions to improve policy and leadership training on quality of care and general management and leadership skills have been shown to make a significant contribution to overall performance improvement in service delivery.22 In addition, by emphasizing adherence to clinical algorithms, there are likely to be improvements in management of both outpatient and inpatient services. Future work could also focus on implementing clinical IQA measures and improving the care received by patients in the inpatient departments.
Limitations of this study are associated with the programmatic nature of the work and the lack of a control group. The checklist tool was designed to assess overall performance improvement at the health facility level over time. Individual provider performance over time was not tracked, thus limiting our ability to rule out the Hawthorne effect. In addition, as the emphasis was on health worker performance to a standard, patient clinical outcomes were not assessed. Finally, as individual supervisors were not systematically assigned to the same facilities for each subsequent visit, differences could be due in part to interobserver variations. Future studies would benefit from controlling for these limitations when considering study design.
CONCLUSION
Targeted observation and mentorship of HCWs implemented during OTSS under MalariaCare indicate that supportive supervision can improve the consistency of quality clinical care for febrile patients in outpatient settings. Progress on most steps can be achieved after only one visit. However, further measurable progress is possible and, where resources allow, a second visit within 3–4 months followed by a third at 12 months could achieve this and lead to these improvements being sustained for longer periods. This would be particularly effective if combined with clinical IQA measures. Lessons learned during these visits can be used to inform programs on improving formal training curricula and highlighting where systemic changes are needed to build sustainable quality care.
Acknowledgments:
We acknowledge the contributions of government and project staff in the Democratic Republic of Congo, Ghana, Kenya, Malawi, Mali, Mozambique, Tanzania, and Zambia, who were the principal implementing actors in the project.
REFERENCES
- 1.Mahende C, Ngasala B, Lusingu J, Butichi A, Lushino P, Lemnge M, Premji Z, 2014. Aetiology of acute febrile episodes in children attending Korogwe district hospital in north-eastern Tanzania. PLoS One 9: e104197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.D’Acremont V, Kilowoko M, Kyungu E, Philipina S, Sangu W, Kahama-Maro J, Lengeler C, Cherpillod P, Kaiser L, Genton B, 2014. Beyond malaria—causes of fever in outpatient Tanzanian children. N Engl J Med 370: 809–817. [DOI] [PubMed] [Google Scholar]
- 3.D’Acremont V, Lengeler C, Genton B, 2010. Reduction in the proportion of fevers associated with Plasmodium falciparum parasitaemia in Africa: a systematic review. Malar J 9: 240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hertz JT, Munishi OM, Sharp JP, Reddy EA, Crump JA, 2013. Comparing actual and perceived causes of fever among community members in a low malaria transmission setting in northern Tanzania. Trop Med Int Heal 18: 1406–1415. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Brent AJ, Ahmed I, Ndiritu M, Lewa P, Ngetsa C, Lowe B, Bauni E, English M, Berkley JA, Scott JAG, 2006. Incidence of clinically significant bacteraemia in children who present to hospital in Kenya: community-based observational study. Lancet 367: 482–488. [DOI] [PubMed] [Google Scholar]
- 6.O’Dempsey TJD, McArdle TF, Laurence BE, Todd JE, Greenwood BM, Lamont AC, 1993. Overlap in the clinical features of pneumonia and malaria in African children. Trans R Soc Trop Med Hyg 87: 662–665. [DOI] [PubMed] [Google Scholar]
- 7.Eliades MJ, Alombah F, Wun J, Burnett S, Martin T, Kutumbakana S, Dena R, Saye R, Lim P, Hamilton P, 2019. Perspectives on implementation consiederations and costs of malaria case management supportive supervision. Am J Trop Med Hyg 100: 861–867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Namagembe A, et al. 2012. Improved clinical and laboratory skills after team-based, malaria case management training of health care professionals in Uganda. Malaria J 11: 44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.WHO , 2012. Training Module on Malaria Control: Case Management. Geneva, Switzerland: World Health Organization; Available at: https://www.who.int/malaria/publications/atoz/9789241503976/en/. Accessed January 10, 2019. [Google Scholar]
- 10.WHO , 2014. IMCI Chart Booklet. Geneva, Switzerland: World Health Organization; Available at: https://www.who.int/maternal_child_adolescent/documents/IMCI_chartbooklet/en/. Accessed January 10, 2019. [Google Scholar]
- 11.Burnett S, Wun J, Evance I, Davis K, Smith G, Lussiana C, Tesha G, Quao A, Robertson M, Hamilton P, 2019. Introduction of an electronic tool for improved data quality and data use during malaria case management supportive supervision. Am J Trop Med Hyg 100: 889–898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.MalariaCare, PATH , 2017. Universal Diagnosis and Treatment to Improve Matenal and Child Health. MalariaCare Project Year 4 Annual Report, October 2015–September 2016. Seattle, WA: USAID; Available at: https://www.pmi.gov/docs/default-source/default-document-library/implementing-partner-reports/malariacare-year-4-annual-report-universal-diagnosis-and-treatment-to-improve-maternal-and-child-health.pdf. Accessed January 10, 2019. [Google Scholar]
- 13.Zurovac D, Rowe AK, 2006. Quality of treatment for febrile illness among children at outpatient facilities in sub-Saharan Africa. Ann Trop Med Parasitol 100: 642. [DOI] [PubMed] [Google Scholar]
- 14.Berendes S, Heywood P, Oliver S, Garner P, 2011. Quality of private and public ambulatory health care in low and middle income countries: systematic review of comparative studies. PLoS Med 8: e1000433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Namuyinga RJ, et al. 2017. Health worker adherence to malaria treatment guidelines at outpatient health facilities in southern Malawi following implementation of universal access to diagnostic testing. Malar J 16: 40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Johansson EW, Selling KE, Nsona H, Mappin B, Gething PW, Petzold M, Peterson SS, Hildenwall H, 2016. Integrated paediatric fever management and antibiotic over-treatment in Malawi health facilities: data mining a national facility census. Malar J 15: 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Baiden F, Owusu-Agyei S, Bawah J, Bruce J, Tivura M, Delmini R, Gyaase S, Amenga-Etego S, Chandramohan D, Webster J, 2011. An evaluation of the clinical assessments of under-five febrile children presenting to primary health facilities in rural Ghana. PLoS One 6: 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Imani P, Jakech B, Kirunda I, Mbonye MK, Naikoba S, Weaver MR, 2015. Effect of integrated infectious disease training and on-site support on the management of childhood illnesses in Uganda: a cluster randomized trial. BMC Pediatr 15: 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Arifeen SE, et al. 2005. Quality of care for under-fives in first-level health facilities in one district of Bangladesh. Bull World Health Organ 83: 260–267. [PMC free article] [PubMed] [Google Scholar]
- 20.Kruk ME, Chukwuma A, Mbaruku G, Leslie H. 2017. Variation in quality of primary-care services in Kenya, Malawi, Namimbia, Rwanda, Senegal, Uganda and the United Republic of Tanzania. Bull World Health Organ 95: 408–418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kruk ME, Gage AD, Mbaruku GM, Leslie HH, 2018. Content of care in 15,000 sickchild consultations in nine lower-income countries. Health Serv Res 53: 2084–2098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Parand A, Dopson S, Renz A, Vincent C, 2014. The role of hospital managers in quality and patient safety: a systematic review. BMJ Open 4: e005055. [DOI] [PMC free article] [PubMed] [Google Scholar]