Skip to main content
. 2021 Mar 15;9(Suppl 1):S151–S167. doi: 10.9745/GHSP-D-20-00378

TABLE 5.

Results of Usability Testing of 4 Prototype Supply Chain Solutions for Community Health, 2 Counties in Kenya

Supply Chain Solution Positive Results Negative Results Changes Made in Final Iteration
Paper-based stock record (Paper prototypes)

  1. Most pictures worked well in assisting the CHV to identify the commodity and the symptoms it treats

  2. The circles used in counting the commodities helped those who could not write numbers

  3. When supply chain terms were explained in Swahili, it increased understanding of the action that CHVs needed to do in reporting

  4. If there was a color picture of the actual commodity, this helped CHVs to identify it

  1. Some CHVs had challenges identifying some icons (e.g., family planning method and the avoiding pregnancy icon)

  2. CHVs found it difficult to find where to enter their names

  3. CHVs were familiar with reporting data horizontally (rows) rather than vertically (columns)

  4. Users generally ignored the “units” column and counted based on what they were used to

  5. Some users found the shading of circles difficult, making it difficult to total and some recorded by using tally lines through each circle

  6. CHVs with low literacy could not identify different columns with supply chain terms

  7. Some CHVs entered their age as opposed to the year which the report is being submitted

  1. Changed icons that were confusing to users

  2. Moved CHV names and their details to the left of the document

  3. Changed Month, to Month of Report and Year of Report to clearly define purpose of cell

  4. Included clear visual distinction between commodities such as a gray bar between commodities and the alternate shading of horizontal rows

  5. Used horizontal rows instead of columns for each commodity

  6. Included colloquial or familiar terms for some commodities (e.g., dewormers for albendazole)

  7. Added Swahili translations in brackets under existing supply chain terminology to aid in the translation of often confusing supply chain terminology
Redesigned cStock smartphone application (Paper then clickable prototypes)

  1. The audio was the most appreciated feature by the CHVs as it helped identify the commodities

  2. The icons allowed for CHVs to identify commodities based on look rather than text

  3. The CHVs were also able to identify icons that they had previously seen in the paper recording tool

  4. The back and next arrows were understood by some of the CHVs

  5. They were able to scroll down the months when selecting the period for reporting

  1. CHVs found some of the icons confusing because they were not representative of reality

  2. Some CHVs clicked the icons and not the buttons when selecting which reports to send

  3. Most CHVs had difficulty identifying the fields to enter their data

  4. Some users understood the arrows to mean “next,” but other users did not

  1. Changed the receiving and dispensing icons based on user feedback

  2. Added audio for most icons

  3. Changed the icon for audio from a speaker to a person speaking

  4. Changed commodity icons to reflect the commodities' actual look

  5. Changed icons to match the revised paper tool

  6. Included navigation arrows, as well as buttons with “next and back” to allow multiple paths to navigate the app
CHV supervisor smartphone application (Paper then clickable prototype)

  1. Supervisors appreciated the prototype

  2. The user experience was easy, and they were able to generally follow it with ease

  3. The tech was easy to use, even at various levels of technology literacy

  1. Some supervisors were not able to relate how the dashboard related to the paper-based resupply worksheet

  2. The validation section of the app was confusing due to unclear wording

  3. Most CHVs found the section added for additional CHVs (CHVs that migrate into your area) confusing

  1. Made it clearer that resupply values were the total amounts received from the facility

  2. Changed column heading for the checkboxes in the validation page

  3. Relabeled additional CHV section in dashboard to articulate that those are CHVs who have migrated to the area
Hybrid USSD and SMS reporting system (Paper prototype)

  1. A large number of the CHVs interviewed had a feature phone as their main phone

  1. This prototype was the most difficult for all CHVs, both literate and low literate

  2. CHVs were not able to switch from USSD to SMS. They entered data on the dial pad and not the SMS

  1. Needed to be redesigned to only USSD and significant tests were done to ascertain its usability in these communities

Abbreviations: CHV, community health volunteer; SMS, short message service; USSD, unstructured supplementary service data.