Assessing Village Health Workers’ Ability to Perform and Interpret Rapid Diagnostic Tests for Malaria 4 Years after Initial Training: A Cross-Sectional Study

Abstract.

Village health workers (VHWs) in Bugoye subcounty, Uganda, provide integrated community case management (iCCM) care to children younger than 5 years for malaria, pneumonia, and diarrhea. We assessed the longevity of VHWs’ skills in performing and reading malaria rapid diagnostic tests (RDTs) 4 years after initial training, comparing VHWs who had completed initial iCCM training 1 year before the study with VHWs who had completed training 4 years before the study. Both groups received quarterly refresher trainings. Trained interviewers observed 36 VHWs reading six mock RDTs each and performing an RDT as part of a larger skills assessment exercise. VHWs read 97% of mock RDTs correctly; of the 36 VHWs, 86% read all six mock RDTs correctly. Most VHWs scored either 12/13 or 13/13 on the RDT checklist (39% and 36%, respectively), with 25% scoring 11/13 or lower. For reading mock RDTs, VHWs in the first group (initial training 4 years before study) read 97% of mock RDTs correctly, whereas those in the second group (initial training 1 year before study) read 96% of mock RDTs correctly; the first group had a mean of 5.83 RDTs read correctly, compared with 5.77 RDTs read correctly in the second group (P = 0.83). For performing an RDT, the first group completed a mean of 12.0 steps correctly, compared with a mean of 12.2 correct steps in the second group (P = 0.60). Overall, VHWs demonstrated proficiency in reading RDTs accurately and performing RDTs according to protocol at least 4 years after initial iCCM training.

INTRODUCTION

Since 2001, Uganda has employed a national cadre of volunteer village health workers (VHWs), locally referred to as “village health teams,” with a broad range of responsibilities focused largely on health promotion and community mobilization.^1,2 In some areas, VHWs also provide integrated community case management (iCCM) care to children younger than 5 years, in keeping with national policy.³ In this model of care, VHWs assess and treat malaria, pneumonia, and diarrhea using a clinical algorithm, which includes performing a rapid diagnostic test (RDT) for malaria on any child presenting with subjective fever and then interpreting the result of the test. Performing and interpreting an RDT correctly underpins VHWs’ ability to provide appropriate care for pediatric malaria. Inaccuracy of either aspect may result in lack of treatment of a potentially fatal disease or, less concerningly, overuse of artemisinin combination therapy. At the time of this study, VHWs in eight villages in Bugoye subcounty, Uganda, were providing iCCM care as part of a pilot program.

Several prior studies have assessed VHWs’ ability to maintain or improve their skills in performing and reading RDTs over time. One study in Zambia documented an improvement in performing RDTs over time, with the percentage of critical steps performed increasing from 87.5% at the 3-month follow-up to 100% at 6- and 12-month follow-ups.⁴ Performance in reading RDTs appeared generally stable over time, with over 90% read correctly at all time points, although in the subset of faintly positive RDTs, correct reading decreased from 89.7% to 76.7% at the 12-month follow-up.⁴ A study in Kenya found that completion of all “safety steps” as well as all steps in performing an RDT did not appear to differ between baseline (after training) and 1-year follow-up.⁵ This study assessed reading of RDTs at 1-year follow-up only and reported a sensitivity of 92% and a specificity of 97%.⁵

A number of other studies have assessed VHWs’ abilities at a single time point, usually shortly after training, overall demonstrating high levels of completion of correct steps in performing RDTs and correct interpretation of RDT results. One prior study in Uganda assessing RDT interpretation immediately after training reported that VHWs read 100% of RDTs correctly.⁶ A study in Malawi assessing health surveillance assistants’ ability to perform RDTs immediately after training found that 97% of RDTs were performed correctly.⁷ Another study in Zambia assessed both performing and reading RDTs and compared groups of VHWs with and without a job aid and formal training; those using a job aid with formal training completed an average of 90% of test steps correctly and read 93% of RDTs correctly.⁸ A study in Kenya used an automated RDT reader to assess both reading of RDTs and a subset of steps in performing RDTs; 92% of RDTs were performed correctly (for the subset of steps assessed) and read correctly.⁹

In this study, we sought to assess the longevity of VHWs’ skills in performing and reading RDTs over a longer time period, comparing a group of VHWs who had completed initial iCCM training 4 years before the study with a group completing initial training 1 year before the study.

MATERIALS AND METHODS

In this setting, VHWs are part-time volunteers who have attended at least some primary school and achieved basic literacy; most work as subsistence farmers. At the time of this study, there were two cohorts of VHWs in Bugoye subcounty: the first group (23 VHWs) had completed initial iCCM training (including training on how to perform an RDT) in 2013 (4 years before this study), whereas the second group (13 VHWs) had completed initial training in 2016 (1 year before this study). Initial iCCM training was similar for both groups and included several hours of training on performing and interpreting RDTs, including supervised practice performing an actual RDT. Both groups of VHWs also received quarterly refresher training sessions and periodic supervisory visits by program staff.

For this study, two trained interviewers visited VHWs at their homes or another agreed-on location to conduct a skills assessment exercise, which included interpretation of mock RDT results and performing an RDT. For the mock RDTs, there were six digitally created RDT results: two positive RDTs, two negative RDTs, and two indeterminate RDTs per VHW. The digitally created RDT results were presented to VHWs on color-printed laminated sheets; each VHW received the RDT results in the same order. Village health workers performed an RDT using a fake hand for the fingerstick sample collection, but otherwise using a real RDT kit (the use of a mock hand obviates the ethical need for the interviewer to interrupt if a critical step is missed). This program uses the SD BIOLINE Malaria Ag P.f/Pan rapid test (Abbott, FL). The exercise also included video and oral case scenarios and a brief qualitative interview; these results will be reported separately. Village health workers were given a small incentive at the end of the exercise.

The interviewer scored VHWs performing an RDT using a 15-point checklist already used by the iCCM program. However, two of the steps on the checklist (recording the patient’s name on the cassette and recording the result in the clinical register) were not included in the skills assessment exercise and thus not included in scoring, so performing the RDT was scored on a 13-point scale. Data were entered into a customized Research Electronic Data Capture database¹⁰ and analyzed in Stata version 15 (StataCorp, College Station, TX).

All VHWs provided written consent before participating in the skills assessment exercise. Ethical approval for this study was granted by the Partners Healthcare Institutional Review Board and the Research Ethics Committee at Mbarara University of Science and Technology.

RESULTS

All VHWs in both groups (a total of 36 VHWs) providing iCCM care in Bugoye subcounty at the time of the study agreed to participate in the skills assessment exercise; 53% are women, and 44% are younger than 40 years (to allow for collection of fully de-identified information, exact ages were not recorded given the small number of participants). Of the 36 VHWs, 31% completed primary school only and 69% completed some or all of secondary school.

Overall, VHWs read 97% of mock RDTs correctly; 100% of positive RDTs, 90% of negative RDTs, and 100% of indeterminate RDTs were read correctly (Table 1). Of the seven negative RDTs read incorrectly, two were read as positive, four were read as indeterminate, and one had no result recorded (reflecting either nonresponse by the VHW for that particular item or a record-keeping error). Of the 36 VHWs, 86% read all six mock RDTs correctly (Table 1).

Table 1.

Village health worker performance reading mock RDTs

Measure	Group 1 (4-year experience)		Group 2 (1-year experience)		Overall
	%	N	%	n	%	N
RDTs read correctly	97	134/138	96	75/78	97	209/216
Positive RDTs read correctly	100	46/46	100	26/26	100	72/72
Negative RDTs read correctly	91	42/46	88	23/26	90	65/72
Indeterminate RDTs read correctly	100	46/46	100	26/26	100	72/72
VHWs reading all six RDTs correctly	87	20/23	85	11/13	86	31/36

RDT = rapid diagnostic test; VHW = village health worker.

Most VHWs scored either 12/13 or 13/13 on the RDT checklist (39% and 36%, respectively), with 25% scoring 11/13 or lower (Table 2). The most commonly missed step was checking the RDT package expiry date, with only 58% of VHWs completing this step. In addition, 88% remembered to put on gloves, meaning that four VHWs neglected this step, and 86% correctly disposed of non-sharps garbage; for all other steps, over 90% of VHWs completed that step correctly (Table 2).

Table 2.

Completion of correct steps in performing an RDT

Measure	% Group 1 (4-year experience) n = 23	% Group 2 (1-year experience) n = 13	% Overall, n = 36
Overall scores (out of 13)
VHWs with score of 13	39	38	39
VHWs with score of 12	30	46	36
VHWs with score of 11	22	15	19
VHWs with score of 9 or 10	9	0	6
VHWs’ completion of each step
Assembles packet, buffer, swab, and lancet	100	100	100
Checks expiry date of test package	61	54	58
Removes contents of test package	100	100	100
Puts on a new pair of gloves	87	92	88
Selects the 4th finger of the left hand and cleans with alcohol	96	100	97
Uses sterile lancet to puncture finger	100	92	97
Discards lancet in sharps bin promptly	96	100	97
Collects blood with an enclosed capillary tube	100	100	100
Uses the capillary tube to blot blood in sample well	96	100	97
Discards capillary tubes/loops in the sharps box	91	92	92
Dispenses buffer into the sample well	91	100	94
Waits 15 minutes before reading results	96	100	97
Disposes of other materials in non-sharps garbage	83	92	86

RDT = rapid diagnostic test; VHW = village health worker.

Performance was similar between the two cohorts of VHWs. For reading mock RDTs, VHWs in the first group (initial training 4 years before study) read 97% of mock RDTs correctly, whereas those in the second group (initial training 1 year before study) read 96% of mock RDTs correctly (Table 1); the first group had a mean of 5.83 RDTs read correctly, compared with 5.77 RDTs read correctly in the second group (P = 0.83; see Table 3). For performing an RDT, the first group completed a mean of 12.0 steps correctly, compared with a mean of 12.2 correct steps in the second group (P = 0.60; see Table 3).

Table 3.

Overall comparison of village health workers with 4 years vs. 1 year of integrated community case management experience

Measure	Group 1 (4-year experience)			Group 2 (1-year experience)			P-value*
	Median	Mean	SE	Median	Mean	SE
Number of mock RDTs read correctly (out of 6)	6	5.83	0.10	6	5.77	0.17	0.83
Number of steps completed correctly in performing RDT (out of 13)	12	12.0	0.23	12	12.2	0.20	0.60

RDT = rapid diagnostic test.

^*By Wilcoxon rank sum test.

DISCUSSION

This study suggests that VHWs can read RDTs with high accuracy and largely adhere to protocol in performing RDTs, in line with prior research in this field.^4–9 The more novel finding of this study is the evidence that VHWs’ performance remains high 4 years after initial iCCM training. Few studies have assessed VHWs’ long-term performance in providing iCCM care, which may be a particular concern for settings like Uganda in which VHWs are part-time volunteers whose long-term motivation may be affected by the many other demands on their time and whose performance over time may be affected by their lower test volume compared with full-time workers. Yet if iCCM is to function as a critical component of care for children younger than 5 years, then ensuring long-term quality of care is crucial. Prior studies of RDT performance and iCCM care more broadly have assessed a single point in time, often just after training, or short-term (e.g., 1 year) follow-up. Although the small study described here provides some reassurance that part-time volunteer VHWs can maintain their skills over time for the specific domain of RDT performance, broader iCCM care quality over time requires further study. Additional research may reveal that VHWs not only maintain but actually improve iCCM care quality over time as they gain experience, but in the absence of data, the prospect of deteriorating quality of care remains a concern. In addition, comparison of iCCM care quality over time among VHWs with different training backgrounds and payment or incentive structures may also help policymakers determine optimal training guidelines and workforce composition.

Of note, this study took place in a program that provides quarterly refresher trainings and periodic individual supervisory visits, so these results may not apply to VHWs who receive initial training without further refresher training and supervision. In addition, this program has maintained an uninterrupted supply of RDTs and medications and exists in a setting in which VHWs regularly evaluate patients with fever and perform RDTs¹¹; thus, these results may also not generalize to setting in which VHWs use these skills more sporadically.

This study has a number of limitations. First, it evaluates a relatively small number of VHWs, and only six RDTs per VHW, which limits the ability to assess modest changes in performance over time. Second, we were unable to assess performance immediately after initial iCCM training as a baseline comparison, so this study does not provide any information about performance in the first year. However, as detailed earlier, several prior studies have assessed this question. Third, this study compares two different cohorts of VHWs, rather than assessing the same VHWs at different time points. Although both groups of VHWs included in this study received similar training and work in similar settings, if baseline performance between the two groups differed, then this study design could fail to detect a change in performance over time. Fourth, although the use of a mock hand, rather than a real patient, for VHWs to perform RDTs obviates the need to interrupt if a critical step is missed, the unfamiliarity of this piece of the assessment could also have affected VHWs’ performance. Fifth, because a real patient was not used, we did not assess the steps related to recording the patient’s name and information. Sixth, the use of mock RDTs for VHWs to read does not fully reflect real-world issues such as faintly positive results, so this study could overestimate VHWs’ abilities in reading RDTs. Finally, because VHWs knew they were being observed during these pre-arranged sessions, the Hawthorne effect may have increased or decreased performance relative to their normal clinical practice.

In conclusion, VHWs appear to maintain their abilities to read RDTs accurately and perform RDTs according to protocol at least 4 years after initial iCCM training. Further research with a larger number of VHWs and in more real-world conditions can help confirm these preliminary findings.