Improving the quality of care of children in community clinics: an intervention and evaluation in Bangladesh

Abstract

Setting: Community health care providers (CHCPs) in 40 rural community clinics of Comilla district, Bangladesh, were trained using a newly developed case-management job aid based on the World Health Organization Integrated Management of Childhood Illness and a communication guide.

Objectives: To assess 1) the change in knowledge of the CHCPs after training; 2) the absolute quality of care provided by the CHCPs (determined as the proportion of children aged <5 years [under-fives] correctly diagnosed, treated and referred); and 3) the consultation behaviour of the CHCPs.

Design: Change in knowledge was assessed by tests pre-and post-training. The quality of care was determined by reassessments at the clinic exit by a medical officer, without a baseline comparison. Consultation behaviour was assessed through direct observation. The study was performed during 2014–2015.

Results: The mean standard knowledge score of the CH-CPs increased from 19 to 25 (P < 0.001). Of 1490 under-fives examined, 91% were correctly diagnosed, 86% were correctly treated and 99.5% received a correct referral decision. The CHCPs performed well on most of the measures of good communication, although one third did not explain the diagnosis and treatment to patients.

Conclusion: The training was effective in changing knowledge. The CHCPs applied the knowledge gained and provided good quality care. Following these results, the Bangladesh Ministry of Health and Family Welfare has scaled up the training nationwide. The lessons learnt should be useful for other countries.

Globally, the mortality rate for children aged <5 years (under-fives) has halved since 1990.¹ A the mortality rate among under-fives remains unacceptably high, at 52.5 deaths per 1000 live births. The first point of contact with health services for sick children is usually primary care. However, in low-income countries, the quality of primary care is often poor. Additional supply-side factors that contribute to poor outcomes include a lack of information on sources of care, the distance of health facilities from the patients' homes, high access costs and staff discrimination. Demand-side factors responsible for the low rate of utilisation of health care services by the poor include beliefs that the care is of poor quality, cultural and social belief systems and a lack of awareness of the value of the services.^2,3

In Bangladesh, the mortality rate for under-fives remains high, at 46/1000,^4,5 half of which is due to acute respiratory infections, serious infection and diarrhoea. Effective case management through trained health care providers could prevent many of these deaths.⁶ To improve the access, utilisation and equity of care, the Ministry of Health and Family Welfare (MOHFW) has initiated the Revitalization of Community Health Care Initiatives in Bangladesh (RCHCIB) project, which aims to provide community clinics (CCs) with catchments of approximately 6000 people in rural areas to deliver an essential service package for women, children and the poor. So far, about 13 309 CCs have been built.

The CCs are staffed by community health care providers (CHCPs) who are responsible for providing health education, health promotion, treatment for minor ailments, and identifying and referring severe cases to hospital. The CHCPs are given monthly supportive supervision by the sub-assistant community medical officer (SACMO) of the upazilla (subdistrict) in which the CC is located. SACMOs are trained medical professionals who are a level above the CHCPs, with similar clinical skills (i.e., they are not doctors), who provide health care services at the upazilla health complexes (UHCs), the first level of health facility staffed by doctors.

In response to the concerns of the RCHCIB project director and the line director of the Essential Service Delivery Programme about poor quality of care, a programme review was conducted that indicated that although the CHCPs received 12 weeks of training, consisting of 6 weeks of theory in the classroom followed by 6 weeks observing doctors practice, they still lacked practical consultation and communication skills. Based on the assessment of patient records by a medical doctor, a rapid pre-intervention study undertaken at five CCs selected by convenience criteria demonstrated that only 29% of the children seen had received proper diagnosis and care, confirming the concerns of the project director (Table 1). We also estimated that 90% of children who did not need antibiotics nevertheless received them. To address these issues, we developed a diagnostic and case management job aid adapted from the Integrated Management of Childhood Illness (IMCI) guidelines⁷ and communication guidelines, which were pre-tested in Kaliganj, a non-study upazilla, and revised accordingly; we then trained all (standard) CHCPs in the study sites. It was anticipated that this would result in higher quality care, contribute to the improved utilisation of CCs and ultimately help reduce under-five mortality.

TABLE 1.

Rapid assessment of quality of care among children aged <5 years who attended five community clinics in Comilla district, Bangladesh, 2014–2015

The timeframe for this ‘embedded’ research was extremely tight, with a short deadline to develop and pilot the intervention before the Ministry of Health undertook nationwide training. This political reality gave us little opportunity to perform a major assessment of quality of care before the intervention implementation. Furthermore, the existing quality of care was sufficiently poor that we considered it preferable to assess the absolute quality of care to allow the programme managers to judge whether this level of quality was adequate, rather than measuring the improvement in care, as we could have found statistically significant improvements even though the care quality was still inadequate.

The specific objectives of the study were to determine the knowledge and consultation behaviour of the CHCPs after training and to determine the proportion of under-five patients seen by CHCPs who received the correct diagnosis, correct treatment (including rational use of antibiotics) and appropriate referral when necessary.

METHODS

Study design

The study used a cross-sectional approach to assess the quality of care, and pre-post testing to assess changes in the knowledge of the CHCPs. The study population comprised all under-five children who attended the selected CCs from August 2014 to February 2015 and the CHCPs who cared for them.

To assess changes in knowledge following training, the CHCPs were pre- and post-tested (immediately before and after training) using 12 multiple choice questions (MCQs) and three case studies addressing the process and content of service delivery. The MCQs and the case studies were pre-tested among 23 CHCPs in Kaliganj for clarity and relevance and revised accordingly. To assess the diagnosis, treatment and referral of the CHCPs post-intervention, at each selected CC every child aged <5 years examined for 6 consecutive days was re-assessed at exit by a SACMO situated in a separate room within the CC. The SACMOs were not previously known to the CHCPs, although the CHCPs were aware that they would be assessed. To assess the communication of the CHCPs with the patient/carer, on the final day of the assessment a social researcher observed the consultations and completed a checklist.

Setting

The study was carried out in two sub-districts of Comilla, a peri-urban district about 100 km from Dhaka with a population of 3.74 million.⁸ Comilla has 16 upazillas, of which we purposely selected two, Daudkandi and Chandina, to avoid overlap with other RCHCIB non-governmental organisation partner projects and to approximate the national average patient load of 45 patients per day per CC. Daudkandi and Chandina have 23 and 27 functioning CCs, respectively serving approximately 16 000 and 15 000 under-five children annually.^9,10 The CCs were situated 7–25 km from their UHC. Of the 50 CCs, 40 (20 from each upazilla) were non-randomly selected to ensure a range of geographical locations, distances from their UHC, knowledge levels (assessed using pre- and post-testing), and equal numbers of male and female CHCPs. The study CCs were not atypical of CCs nationally, being a mix of rural, peri-urban and very remote CCs (up to 25 km from their UHC). The study CCs may not be representative of certain CCs located in the hill areas of Chittagong and in remote coastal areas, which are difficult to access and possess unique features.

Intervention and training

All the study CHCPs underwent 6 days of refresher training during April–June 2014, which was facilitated by the RCHCIB project and a member of our study team. Eight of the 37 sessions during this training period covered the IMCI guidelines.⁷ The intervention and training package included five new components: 1) a job aid¹¹ based on the IMCI to facilitate the effective management of six common illnesses, including guidance on the appropriate use of antibiotics; 2) training on ‘how to diagnose and treat’; 3) training on how to communicate with the child and the care giver; 4) IMCI user guidelines describing how to use the IMCI job aid; and 5) training modules, including case studies and role-play exercises. The package was developed shortly before the training in January–March 2014 by a national technical working group committee comprising national and international experts in relevant health disciplines, social science and public health. Following the training, the CHCPs were expected to keep the job aid and guide on their desks for ready reference during consultations. A simple referral mechanism was included in the IMCI job aid to improve referral of complicated cases.

Data collection procedures

One field worker was appointed in each upazilla to communicate with the CCs and manage data collection. The field workers received training on data collection tools and procedures. Two SACMOs, one from each UHC, were selected to assess the skills of the CHCPs, with permission from the upazilla authorities. The SACMOs spent 6 consecutive working days at each CC assessing all the consultations of the CHCPs with under-fives. Each child was then re-assessed by the SACMO who was blinded to the CHCP consultation. At the end of each day the SACMO matched his/her own register against the CHCP register and, using a pre-developed reporting form, recorded all consistencies and inconsistencies. On day 6, a researcher observed the CHCP's consultations to assess communication skills and completed a pre-developed consultation observation checklist. At the end of day 6, the SACMO gave feedback to the CHCPs on each consultation. All data were collected using pre-tested structured tools.

Data entry and analysis

All data from the SACMO reporting forms were checked and analysed using SPSS v.23 (IBM Corp, Armonk, NY, USA). The performance of the CHCPs was assessed using outcomes measuring the proportion of the under-fives 1) who received the correct diagnosis, 2) who received the correct treatment, 3) who received appropriate referral services when necessary, and 4) for whom antibiotics were used rationally (prescribed when recommended in the job aid and withheld when not recommended).

The study was powered to estimate 95% confidence intervals (CIs) of at most ±10% for the outcomes of each type of disease, which implied identifying at least 120 cases of severe pneumonia.

A knowledge score was calculated based on the responses in the pre- and post-training tests, and the mean change in scores was assessed using a paired t-test. Simple frequencies and proportions and their 95%CIs, calculated using the Clopper-Pearson exact method,¹² were used to describe the consultation performance and the behaviour of the CHCPs and make inferences about the effectiveness of the training.

Ethics

Ethics approval was obtained from the Bangladesh Medical Research Council (BMRC), Dhaka, Bangladesh, and the University of Leeds, Leeds, UK. Written informed consent was obtained from all the respondents and CHCPs before the interviews.

RESULTS

Of 1501 under-five children who attended the 40 CCs during the study period, 11 had missing data; the analysis was therefore restricted to 1490 (99%) children. One half (51%) of all children were female and the median age was 2 years (interquartile range [IQR] 1–3). The most common presentations were ‘No pneumonia (cough or cold)’ (67% of patients), fever and diarrhoea (Table 2); 17 (1%) children with severe disease required referral to an UHC.

TABLE 2.

Distribution of diseases among children aged <5 years who attended the community clinics in two upazillas of Comilla district, Bangladesh, 2014–2015

Quality of care

Overall, 91% (95%CI 89–92) of the children were correctly diagnosed, 86% (95%CI 84–87) were correctly treated and almost all (99.5%, 95%CI 99–99.8) received a correct referral decision by the CHCPs (Table 3). Only one child who required referral was missed by the CHCP (Table 3). Excluding severe pneumonia, for which there were only two cases (and therefore accurate inference is not possible), the percentage with a correct diagnosis was lowest for pneumonia (68%, 95%CI 53–80), with much higher rates of correct diagnoses for all other diseases (Table 4).

TABLE 3.

Proportion of children aged <5 years who received a correct diagnosis, treatment and referral in two upazillas of Comilla district, Bangladesh, 2014–2015 (N = 1490)

TABLE 4.

Proportion of children aged <5 years correctly diagnosed, treated (in relation to antibiotic use) and referred, by presenting disease, in two upazillas of Comilla district, Bangladesh, 2014–2015

Rational use of antibiotics

Overall, 89% (95%CI 87–91) of consultations resulted in the correct use of antibiotics: 89% (95%CI 81–93) of consultations requiring an antibiotic correctly resulted in their use, and 89% (95%CI 87–91) of consultations not requiring an antibiotic correctly did not result in their use (Table 3). The correct use of antibiotics was lowest for children presenting with ‘No pneumonia (cough or cold)’ (86%, 95%CI 83–88) and pneumonia (88%, 95%CI 76–95), with high levels of correct antibiotic use for all other diseases (Table 4).

Changes in the knowledge of the community health care providers

The mean knowledge score (maximum 30) among the CHCPs before the training was 19 (standard deviation [SD] 16–22), which increased to 25 (SD 23–27) after training (P < 0.001).

Consultation behaviour of the community health care providers

The CHCPs performed well on most of the measures of good communication, although about half interrupted the child's parent or carer while they were talking, around one third did not explain the diagnosis and treatment, and slightly less than one third failed to provide the patients with preventive messages (Table 5).

TABLE 5.

Assessment of consultation behaviours of community health care providers in two upazillas of Comilla district, Bangladesh, 2014–2015 (N = 37)

DISCUSSION

We found that following the training, the CHCPs were able to correctly diagnose, treat and refer 86% of the under-fives they cared for. The pre/post-training evaluation showed a highly significant and clinically meaningful increase in knowledge based on the content of the job aid. These findings indicate that the knowledge the CHCPs gained in training was applied in clinical practice and justify the decision of the MOHFW to roll out the refresher training countrywide.

Some over-prescribing of antibiotics remained—about 11% of children were prescribed antibiotics when they should not have been—but this was less than half the rate found in the pre-intervention study. This is significant given global concerns about antibiotic resistance. There were still some errors in diagnosis (9%) and in treatment (14%), especially among children with respiratory symptoms, which need to be addressed during the supportive supervisory visits and on-the-job training. As arranging classroom-based training for the 14 000 CHCPs is expensive and time consuming, the MOHFW has introduced distance learning and the programme managers have begun to fill the remaining gaps identified in the performance of the CHCPs through electronic distance learning modules. In the next health sector programme, there are plans to arrange 6-day refresher training courses for all the CHCPs in 2017.

An important innovation in this study is the adaptation of the IMCI job aid and training to the Bangladesh context, in line with updated World Health Organization (WHO) guidance and the country's cultural and epidemiological context. The job aid includes an expanded fever section, for example, which recognises that while most of Bangladesh is malaria-free, in the districts bordering Myanmar where there is malaria, a rapid malaria diagnostic test is necessary. The resulting six-page job aid is easy to use and easily replicable and scalable.¹¹ The intervention contributed to a change in national policy and practice, with about 14 000 CHCPs nationwide given the job aid and trained.

A study from Ethiopia¹³ that used a similar training methodology and evaluation found that following training, health extension workers provided correct case management in two thirds of under-five children, but only one third of children with severe illness were correctly managed and referred. The performance of the CHCPs in our study has been substantially better, although the reasons for this difference are unclear.

This is the first study on the performance of the CHCPs in Bangladesh. The intervention and its evaluation had several strengths: the study had a large sample size; the intervention was designed and developed to be replicable, scalable and sustainable within routine services; and there are clear criteria for classifying severe illnesses that need urgent treatment and referral. The CHCPs identified all those children requiring referral, although there were far fewer than expected; this may be because the parents go directly to the UHC when their child appears very ill.

There were some weaknesses in the study. First, there was no systematic baseline assessment in the study sites, due to the rapid programme decision to scale up CHCP training nationwide based on the pilot results. It was widely acknowledged, however, that the baseline quality of care was suboptimal and it was deemed inappropriate to divert scarce resources for baseline assessment. Second, the study was performed in only two subdistricts, and may not be representative of the whole country. Third, the relatively short time interval between the training and knowledge assessment measured short-term recall, and may not represent long-term learning, although the longer period between the training and reassessment by the SACMOs suggests that the knowledge was retained. Fourth, the CHCPs were aware that a SACMO was reassessing the children and that a researcher was observing behaviour during the consultation. The observed quality of care may thus be due in part to the Hawthorne effect,¹⁴ where increased attention during the assessment in itself improves performance. The performance observed in the study may not therefore necessarily reflect that expected in routine practice. Fifth, the re-assessment was conducted by two SACMOs; a panel of clinicians would have increased the rigour of the re-assessment.

In conclusion, we believe this study provides evidence to demonstrate that there are likely to be substantial benefits if countries develop tailored materials and training packages for lower-level health workers. These should be based on the current WHO IMCI guidance.⁷