Scoping review of health information technology usability methods leveraged in Africa

Abstract

Objective

The aim of this study was to explore the state of health information technology (HIT) usability evaluation in Africa.

Materials and Methods

We searched three electronic databases: PubMed, Embase, and Association for Computing Machinery. We categorized the stage of evaluations, the type of interactions assessed, and methods applied using Stead’s System Development Life Cycle (SDLC) and Bennett and Shackel’s usability models.

Results

Analysis of 73 of 1002 articles that met inclusion criteria reveals that HIT usability evaluations in Africa have increased in recent years and mainly focused on later SDLC stage (stages 4 and 5) evaluations in sub-Saharan Africa. Forty percent of the articles examined system-user-task-environment (type 4) interactions. Most articles used mixed methods to measure usability. Interviews and surveys were often used at each development stage, while other methods, such as quality-adjusted life year analysis, were only found at stage 5. Sixty percent of articles did not include a theoretical model or framework.

Discussion

The use of multistage evaluation and mixed methods approaches to obtain a comprehensive understanding HIT usability is critical to ensure that HIT meets user needs.

Conclusions

Developing and enhancing usable HIT is critical to promoting equitable health service delivery and high-quality care in Africa. Early-stage evaluations (stages 1 and 2) and interactions (types 0 and 1) should receive special attention to ensure HIT usability prior to implementing HIT in the field.

BACKGROUND AND SIGNIFICANCE

Promoting good health and well-being is a crucial priority for all countries and is a sustainable development goal set by the United Nations.¹ The continent of Africa is an economically diverse geographical area with countries with high gross national income, like Seychelles, and others, such as Burundi, with much lower gross national income.² Health outcomes also vary widely throughout the continent. For example, in terms of maternal mortality, Chad had the highest rate among African countries with 1140 deaths per 100 000 live births in 2017, while Egypt only reported 37 deaths per 100 000 live births.³

Despite the economic diversity within the continent, Africa has the largest proportion of low-income countries with 23 of the 27 poorest countries in the world and 21 of the world’s 55 lower-middle-income countries.² In comparison to other continents, Africa has greater health inequities, worse health outcomes, the lowest average life expectancy, and the highest proportion of under-five mortality and maternal mortality ratios.^2,4,5 Furthermore, a 2018 World Health Organization review found that most countries in Africa were unable to achieve their targets for sustainable development goals of improving health outcomes.⁴ Thus, it is imperative that research and health policies focus on removing barriers to equitable health and improving health outcomes in this region.

Development and utilization of health information technology (HIT) can contribute to improving health outcomes in both high- and low-resource areas. HIT is the “processing, storage, and exchange of health information in an electronic environment”⁶ and has been shown to improve the quality and delivery of care, decrease costs, improve patient safety and health outcomes, assist healthcare providers in efficiently completing tasks, improve patient-provider interactions, and decrease inadvertent errors.^4,6–12

Over the last decade, the growth and use of internet services and mobile devices have rapidly expanded in both urban and rural communities in Africa.⁹ This uptake of mobile devices and information technology has led to the rapid development of HIT and increased utilization of technology by healthcare workers and patients for communication, collaboration, and access to health-related information.^7,9 Previous research on the impact of HIT in low- to middle-income countries has found the technology to be useful in enhancing the efficiency of task completion, which is in line with the positive effects of HIT in high-income countries.¹²

However, HIT must be used deliberately. If the creation, implementation, and use of HIT are not carefully enacted, unintentional consequences may occur.¹³ Tomasi et al¹² also highlighted the importance of conscientious implementation and stakeholder involvement to ensure the optimal impact of system implementation in a specific environment. System usability is critical to ensuring positive influence of HIT on outcomes of interest.¹⁰ The International Organization for Standardization (ISO) defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specific context of use.”⁶ ISO definitions for the usability components of effectiveness, efficiency, and satisfaction are shown in Table 1.

Table 1.

ISO definitions of usability components

Component of usability	Definition
Effectiveness	Accuracy and completeness with which users achieve specified goals
Efficiency	Resources expended in relation to the accuracy and completeness with which users achieve goals
Satisfaction	Freedom from discomfort and positive attitudes toward the use of the product

Note: Definitions from the International Organization for Standardization (2013).

Poor usability can increase the time needed to complete tasks, increase the risk of errors occurring, lead to increased costs related to implementation and use, and discourage users from incorporating the technology into their daily activities, among many other negative consequences.^8,10,14–16 Thus, researchers must perform usability evaluations to ensure that they are cognizant of the preexisting environment and have support from local stakeholders and targeted end-users of the HIT and that sufficient resources exist to promote uptake of the system.¹⁷ Moreover, usability evaluations should occur at multiple stages of system development rather than a single point in time to ensure adequate fit between the technology and task requirements.

We conducted a scoping review to explore the state of HIT usability evaluation in Africa. Specifically, we applied Bennett and Shackel’s usability models^15,18 to characterize the types of usability interactions assessed and methods used and categorized the timing of the evaluation according to the stages of the System Development Life Cycle (SDLC).¹⁶

MATERIALS AND METHODS

This scoping review follows the Joanna Brigg’s scoping review framework.¹⁹ We used the Population-Concept-Context framework²⁰ to inform the inclusion criteria (Supplementary Table S1) and developed an unpublished a priori protocol based on the PROSPERO guidelines (Supplementary Table S2).²¹ We report results according to Stead’s SDLC stages¹⁶ and Bennett’s and Shackel’s usability models.^15,18

Eligibility criteria

We included articles in this scoping review if their main objective was HIT usability evaluation in African countries. We did not include articles of studies that explored general informatics aspects, such as information-seeking behavior or evaluations of personal technology usage, studies that evaluated specific methods, models, or frameworks, or studies focused on information systems for security, bioinformatics, or research purposes not related to health outcomes or the promotion of positive health behaviors (Supplementary Table S2). Our eligibility criteria comprised quantitative, qualitative, and mixed-method studies reported in journals, conference proceedings, and gray literature. While we did not limit the date of publication, our search was limited to the English language. The removal of the non-English articles from the original search results occurred during the title and abstract screening. However, one article had an English abstract but during full-text review, the reviewers found that the full article was not available in English and was thus removed.

Information sources and search strategy

We consulted an informationist from the Columbia University Irving Medical Center library to develop the search strategy for the review. We searched PubMed, Embase, and the Association for Computing Machinery (ACM) (Supplementary Table S3). We also performed an ancestry search of reference lists in the retrieved papers.

Study selection and data extraction

Our literature search retrieved a total of 1088 results comprising articles reporting HIT evaluation findings, study protocols, and conference papers. Two authors (KD and MH) screened, evaluated, and extracted data using Covidence.²² We removed 94 duplicates and added 11 articles from the search of reference lists resulting in 1002 titles and abstracts that were screened for inclusion. We subsequently reviewed 183 full-text articles resulting in 73 that met our inclusion criteria (Figure 1).

Figure 1.

PRISMA flow diagram.

We extracted data about author information, date of publication, study sample and size, study location, type of HIT evaluated, study design, methods used to explore usability, and framework used (if stated). Supplementary Table S4 summarizes study descriptions.

Data extraction from individual sources of evidence

To extract and categorize SDLC stage and types of interactions studied in Africa, we applied Yen and Bakken’s²³ evaluation framework that combined the SDLC with Bennett’s and Shackel’s usability models to determine the when and the what of usability studies in the United States. The when refers to the point along the SDLC the evaluation is occurring. Table 2 presents specific study categorization criteria for each of the SDLC stages.

Table 2.

Criteria for study categorization based on the SDLC stages

SDLC stage	Study categorization criteria
Stage 1	Needs assessments with design methods described
Stage 2	System validation evaluations, such as sensitivity and specificity analyses, ROC curve, and observer variation
Stage 3	Human-computer interaction evaluations focusing on outcome quality, user perception, and user performance in the laboratory setting
Stage 4	Field testing; experimental or quasi-experimental designs with control groups in one setting
Stage 5	Field testing; experimental or quasi-experimental designs with control groups in multiple sites; postimplementation evaluation only; self-control, such as evaluation before and after implementation

Note: Criteria from Yen et al²³.

Abbreviations: ROC: receiver operating characteristic; SDLC: System Development Life Cycle.

Bennett’s and Shackel’s usability models explore the what of usability evaluations, which are the components being explored in the study. These components consist of the task needing to be accomplished, the system or HIT, the users of the system, and the environment where all of this is occurring.^15,18

Figure 2 depicts how these four components interact with one another. The five component interaction types that are included in these models are (type 0) task alone, (type 1) user-task interaction, (type 2) system-task interaction, (type 3) system-user-task interaction, and (type 4) system-user-task-environment.^15,18Table 3 displays the combined evaluation framework used in this literature review.

Figure 2.

Usability interaction diagram.

Table 3.

Usability evaluation frameworks

Stead SDLC	Bennett and Shackel evaluation type
Stage 1: specification	Type 0: task
	Type 1: user-task
Stage 2: component development	Type 2: system-task
	Type 3: system-user-task
Stage 3: combination of components into a system	Type 2: system-task
	Type 3: system-user-task
Stage 4: integration of system into environment	Type 2: system-task
	Type 3: system-user-task
	Type 4: system-user-task-environment
Stage 5: routine use	Type 2: system-task
	Type 3: system-user-task
	Type 4: system-user-task-environment

Note: Integrated usability framework from Yen et al²³.

Abbreviation: SDLC: System Development Life Cycle.

RESULTS

Description of included articles

Of the 73 articles included in this review, 20 (27%) are from the last two years and 31 (42%) from the last 3 to 5 years. The earliest study reviewed was published in 2010 and 2019 had the most publications (n = 14, 19%). Healthcare workers providing direct patient care were the most frequently targeted HIT end-users (n = 48, 66%). Others included health administrators and leadership, patients, caregivers, and content experts, such as HIV treatment experts (see Supplementary Table S4). Sample sizes varied widely among the studies from 5²⁴ to 2327.²⁵ The studies reflected the implementation of HIT in multiple different settings including clinics (n = 16), hospitals (n = 17), communities (n = 19), long-term care facilities (n = 1), academia (n = 1), and health departments (n = 1). In addition, 16 studies described the implementation of HIT in multiple settings.

We describe the details of the 73 articles in this scoping review including the type of HIT, SDLC stage, usability interactions, and theories supporting the studies in the following sections. Some publications describe multistage usability evaluations, so the number of publications at each stage does not match the total number of studies included in the review.

The countries with the most publications were Kenya and Uganda (n = 10, 14% for each) followed by South Africa (n = 9, 12%). The remaining countries had 1–6 publications (Table 4). Two studies occurred in more than one country, one in Kenya and Uganda, and another in Kenya and Mozambique.

Table 4.

Number of articles from each African country

Country	Number of articles
Kenya	10
Uganda	10
South Africa	9
Ethiopia	6
Ghana	6
Malawi	6
Nigeria	6
Tanzania	4
Botswana	3
Mozambique	3
Sierra Leone	3
Madagascar	2
Rwanda	2
Burkina Faso	1
Burundi	1
Central African Republic	1
Gabon	1
Zimbabwe	1

Most articles are from the southeast region of the continent (Figure 3). Thirty-six countries did not have publications that met eligibility criteria, that is, no articles from north or southwest Africa.

Figure 3.

Map of countries in Africa that have studies included in this review.

Types of HIT evaluated

The most common types of HIT evaluated were clinical decision support systems (CDSS) (n = 24, 33%) and clinical information systems (n = 13, 18%), which include electronic health records, information systems, and documentation systems (Table 5). The least common type of HIT was adverse event reporting (n = 1). In terms of the application domain, 21% (n = 15) of HIT evaluated was for HIV care (n = 15, 21%). Very few studies utilized or evaluated preexisting HIT in their work. The exception is Lim et al²⁶ who leveraged two health applications that were previously evaluated for usability to enhance the design of their CDSS. This allowed them to quickly move to stage 3 and 4 usability evaluations rather than duplicate the previous work.

Table 5.

Number of health IT systems evaluated by type

Category	Number
1. Population-based system	9
a. Registry	2
b. Population surveys	2
c. Disease control	5
2. Clinical information system	13
a. Electronic health record	7
b. Information system	5
c. Documentation system	1
3. CDSS	24
a. For information management (eg, information needs)	1
b. For focusing attention (eg, reminder/alerts)	2
c. For providing patient-specific recommendations	21
4. Patient/consumer facing decision support system	6
a. For information management (eg, information needs)	1
b. For focusing attention (eg, reminder/alerts)	5
5. Telehealth system—provider-patient consultation	8
6. Training system	7
a. Provider education	3
b. Patient education	4
7. Adverse event reporting	1
8. Logistics system	5
a. Patient admissions	2
b. Medical supplies	1
c. Pharmaceuticals	2

Note: HIT categories were based on Gremy and Degoulet’s classifications, CDSS classifications described by Musen, AHRQ, and logistic systems are categorized by the item they are tracking.^27–29

Abbreviations: CDSS: clinical decision support system; HIT: health information technology; AHRQ: Agency for Healthcare Research and Quality.

Stages of SDLC

Most publications (n = 48, 66%) described single-stage evaluations. While many methods were similar across stages, some were unique to an individual stage, such as cost-benefit analysis and quality-adjusted life year (QALY) score calculations at stage 5 evaluations (Table 6). Mixed methods was the most common study design (n = 43, 59%), typically including some form of qualitative interviews or focus groups as well as administration of quantitative questionnaires.

Table 6.

Methods used at each SDLC stage

SDLC stage	Methods Used
Stage 1: specify needs and setting	Data collection/data sources Questionnaire/survey Focus group Interview Brainstorming design session
	Data analysis Deductive coding Inductive coding Manual content analysis Logistic regression Logical workflow model creation Mind map creation
Stage 2: system component development	Data collection/data sources CDSS treatment recommendations CDSS reminders Patient clinical notes
	Data analysis Test-retest reliability (repeating clinician diagnostic decisions based on patient notes) Inter-rater reliability (comparison of provider clinical decisions/recommendations using old system and new HIT) Sensitivity and specificity (comparison of pulse oximetry readings reported in patient clinical notes and CDSS reminder creation using old systems and new HITs)
Stage 3: combination of components	Data collection/data sources Think-aloud protocol Heuristic evaluation Low-fidelity prototyping Rapid prototyping Lab simulation with predetermined tasks and patients Hybrid lab-live software evaluation One-to-one theoretical usability workshop Questionnaire/survey Focus group Interview Observation User-experience expert review
	Data analysis Descriptive statistics (eg, time on task, percentage of user errors, severity of errors) Thematic analysis Inductive coding Deductive coding Fisher’s exact test
Stage 4: integrate health IT into a real environment	Data collection/data sources Field usability testing Think-aloud protocol User engagement report Weekly stock report/facility surveillance report Patient health record Questionnaire/survey Focus group Interview Observation Retrospective probing technique
	Data analysis Descriptive statistics (comparison to old system using presence of missing records, errors present, time-on-task, number of IT support request tickets) Inter-rater reliability Sensitivity, specificity, positive predictive value, and negative predictive value Fisher’s exact test Multivariate logistic regression Cause of error analysis Correlation analyses
Stage 5: routine use	Data collection/data sources Questionnaire/survey Focus group Interview Observation Facility surveillance report User engagement report
	Data analysis Descriptive statistics (comparison to old system using frequency of missing records, percentage of errors present) Sensitivity and specificity Test-retest reliability Ranked correlation analysis QALY analysis Cost-benefit analysis Multivariable regression models (to compare success, adherence, and task completion)

Abbreviations: CDSS: clinical decision support system; QALY: quality-adjusted life year; SDLC: System Development Life Cycle; HIT: health information technology.

Stage 1: specify needs and setting

Stage 1 focuses on determining the specific tasks and needs that the technology will support along with user preferences. Evaluations at this stage can occur in the laboratory or the field. A total of six articles (8%) included stage 1 evaluations. To determine specific user needs and knowledge some studies reviewed the current literature.^30,31 To identify task requirements and workflow needs two publications used consultations with domain experts.^32,33 These experts included professionals from implementation science, informatics, and disease-specific treatments, such as HIV/TB. The studies recruited health administrators and officers for their understanding of the health technology infrastructure and policy requirements to create HIT that can be easily implemented into the community.^31,34,35 All stage 1 evaluations focused on the emic perspective, or insider perspective, and recruited healthcare workers and patients as the intended HIT end-users.^30–35 Most studies included qualitative data collection strategies such as interviews, focus groups, and brainstorming design sessions. These studies used inductive and/or deductive coding strategies to analyze their data.^31,32,35,36 Bagayoko et al²⁵ was the only study to use quantitative surveys to elicit targeted end-user knowledge during stage 1 development. They used the survey responses to perform logistic regression to determine which variables are associated with healthcare provider perspectives on a national electronic health system.²⁵

Stage 2: system component development

The purpose of stage 2 evaluations is to determine the validity and reliability of the HIT or its individual components. These evaluations occur in the laboratory, or a controlled environment that is not influenced by real-world conditions. Three articles reported performing stage 2 evaluations^37–39 and calculated sensitivity and specificity, inter-rater reliability, and test-retest reliability to ensure the accuracy and validity of the HIT or its components.^37–39 For example, to ensure validity of their HIT Azfar et al³⁷ performed traditional face-to-face dermatology appointments with participants and made clinical diagnoses and treatment decisions based on these appointments to create a gold standard for comparison with a teledermatology system. Researchers compared the provider decisions in the two appointment modes to determine congruence. To explore test-retest reliability, the researchers provided the healthcare providers with blinded versions of the old cases using the information obtained from the teledermatology visits and asked them to make clinical decisions based on the information collected in the telehealth visits.³⁷ By performing these tests, the authors could ensure reliability and validity of their tool before exploring usability concerns or real-world performance.

Stage 3: combination of components

Stage 3 evaluations occur in the laboratory setting and aim to determine if the HIT can decrease human error and improve the user’s experience performing the task. There were twenty-eight articles (38%) with stage 3 evaluations. To explore users’ perceptions of HIT, researchers used both quantitative and qualitative methods. The most common quantitative method for obtaining user perceptions was usability surveys. Many studies used existing tools, such as International Business Machines Corporation (IBM) usability questionnaires, System Usability Scale, and the Health Information Technology Usability Evaluation Scale.^{26,38,40–43} However, other studies chose to create their own surveys or modify existing surveys.^44–50 Typically, authors used descriptive statistics summarize these subjective usability measures. In addition, 18 of the 28 articles with stage 3 evaluations used focus groups and interviews to gather additional user perceptions. The authors used both inductive and deductive coding to analyze the transcripts from these sessions.

Researchers used several different simulated activities to gain a better understanding of how targeted end-users might complete required tasks using the HIT. These simulated activities include think-aloud protocol, hybrid lab-live software evaluation, low-fidelity prototyping, direct observations, and one-to-one theoretical usability workshops. The most common of these methods was the think-aloud protocol.^{24,26,40,44,47,51} During think-aloud protocols, participants verbalized their thought processes as they completed tasks.⁵² Researchers are also interested in ensuring their HIT not only has high usability with end-users but also meets best practices (ie, heuristics) for user-interface design. Two studies used heuristic evaluations by HIT experts to identify usability violations in their HIT.^43,44

Stage 4: integrate HIT into a real environment

Most articles included stage 4 evaluations (n = 53, 73%). Stage 4 evaluations occur in the field and evaluate how the HIT will perform in real-world conditions. A defining characteristic of stage 4 evaluations is that the HIT is typically still under the control of the individuals who developed it. Most articles describing stage 4 evaluations were pilot studies evaluating the ease of implementation and use of the HIT at one or a handful of sites. To identify barriers to implementation researchers performed cause of error analyses.⁵³ To evaluate user satisfaction researchers used questionnaires (n = 34), focus groups (n = 32), and interviews (n = 28). In addition, chart review, user-engagement reports, frequency of IT support ticket submission, guideline adherence, and outcome comparison to the previously used system were all leveraged to evaluate both efficiency and effectiveness of the HIT during real-world performance. Similar to stage 3 evaluations in the laboratory, time-on-task, counting the number of errors that occurred during HIT use, and asking participants to perform think-aloud protocol during direct observation measured system efficiency.^{26,44,47,48,54} One study compared clinical care decisions made by nurses using a novel telehealth system and traditional face-to-face appointments to determine the reliability of nursing hypothesized diagnoses and feedback.⁵⁵

Stage 5: routine use

The focus of stage 5 evaluations is to determine the long-term impact of the HIT. These evaluations occur in the field, typically when the HIT is no longer under the control of its original developers, and explore the effect of HIT on health outcomes, individuals, and communities.¹⁶ Nineteen (26%) articles had stage 5 evaluations. Data sources included user-engagement reports, weekly stock reports, patient health reports, and observations. For example, Abejirinde et al⁵⁶ used facility inventory checklists to examine the impact the influence of HIT on facilities capacity to provide antenatal care. The follow-up period for these studies ranged from 1 month to 1 year following HIT implementation. Most of the studies applied mixed methods, while four were only quantitative^57–60 and five were only qualitative.^61–65 Overall, 14 articles used qualitative interviews to explore the influence of HIT on patient outcomes, user experiences, and workflow. Additional methods used in stage 5 evaluations included cost-benefit analysis, QALY analysis, and multivariable regression models to evaluate the influence of the HIT on treatment adherence and success.^30,59,66

Bennett and Shackel’s usability interactions

None of the articles included in this review performed a type 0: task evaluation. The remainder of interaction evaluations were: type 1: user-task (n = 2, 3%) during SDLC stage 1; type 2: system-task (n = 13, 18%); type 3: system-user-task (n = 29, 40%); and type 4: system-user-task-environment (n = 29, 40%). Type 4 interactions occur during stages 3–5 evaluations since these studies occur in the field and researchers can explore the environmental component of usability as it relates to the technology.

Theories and frameworks guiding the usability studies

Theoretical models and frameworks guide study activities and support study rationale. Most articles did not include any framework or theory to support their work (n = 44, 60%). Of the 29 articles that used theories, the most common was the Technology Acceptance Model (n = 6, 8%). More than one study used the unified theory of acceptance, the use of technology model, and the consolidated framework for implementation research. The frameworks used in each study are listed in Supplementary Table S4.

DISCUSSION

Overview

This scoping review of HIT usability evaluations conducted in African countries identified and described when evaluations were occurring and what components of usability were evaluated. The review included more articles (n = 73) than a recent review about the presence of HIT in Africa (n = 14)⁶⁷ but fewer than a similar HIT usability literature review focusing on articles in the United States (n = 346).²³ However, this reflects a significant number of such studies in Africa given that Africa only accounts for approximately 1.1% of global investments in research and development.⁶⁸

SDLC stages and Bennett and Shackel’s usability interactions

Most articles in the review consisted of later-stage evaluations where HIT was about to be deployed into the field or was already in real-world use. While several articles provided details on the methods they used to perform early-stage evaluations, many simply alluded to prior research or earlier development without providing any details on the methods used or samples for usability assessments.^{31,32,58,69–71} Review of those articles’ reference lists revealed no additional publications describing the methods in greater detail. Without additional detail, it is difficult to determine the rigor of the methods applied, or the fit of HIT produced to the user or task.

Furthermore, most articles did not describe early-stage evaluations. By ignoring early-stage usability concerns, researchers may develop and deploy HIT that does not perform its required tasks effectively. Additionally, it is more difficult to articulate and address specific usability concerns in later-stage evaluations because the interactions are more complex at later development stages.²³ These findings align with the recent review exploring HIT usability in the United States that also found a predominance of HIT evaluations at stages 4 and 5.²³ This consistency across the two reviews may reflect a broader trend in reporting only later-stage evaluations even if earlier-stage evaluations were conducted. Publication bias may play a role in that there are fewer dissemination venues for formative and typically small sample user-centered studies that may be specific to a single system or site despite the critical role of such evaluations in creating usable HIT.

Methodological characteristics

The majority of the articles in this review applied mixed methods approaches. By leveraging multiple types of data sources, evaluation methods, and analyses, researchers were able to obtain the objective and subjective measures of usability and identify barriers to efficient HIT utilization. Use of mixed methods also enabled the exploration of multiple usability components (eg, efficiency, effectiveness, and satisfaction).⁸

Theoretical frameworks provide rationale for studies, a lens for the interpretation of findings, and a basis for establishing the contribution of findings to advance knowledge generation in the field.⁷² However, fewer than half of the articles in the scoping review explicitly included theoretical frameworks or models. This finding is consistent with Yen and Bakken’s²³ review of HIT usability studies in the United States so does not reflect a trend specific to Africa. To enhance the quality of research related to HIT usability and enable the explication of the research studies findings to advance knowledge generation, researchers should incorporate theoretical frameworks/models into their work. Furthermore, future research should identify theoretical frameworks that are appropriate for evaluations at each SDLC stage.

Gaps in the literature and future research implications

This review found several significant gaps in the literature related to HIT usability evaluations in Africa that are similar to a similar review in the United States.²³ As previously mentioned, much of the literature did not include early-stage or preliminary interaction-type evaluations. Without rigorous evaluations along the SDLC stages, HIT has a limited potential to influence health outcomes and reduce health disparities.¹⁷

Furthermore, few studies evaluated or incorporated preexisting HIT into their work. This gap may be associated with the lack of early-stage evaluations, which would have identified preexisting tools that could answer researchers’ questions or health task needs. The use of preexisting tools can create robust and flexible technology that has proven usability in a wide range of situations. Researchers should perform environmental scans to identify potential solutions that meet the user-task-environment needs. This scan can save researchers time and energy if they can incorporate components of the preexisting HIT or adopt strategies to address common HIT development barriers encountered in prior research. Lim et al²⁶ is the only study that leveraged two health applications that had already undergone usability evaluations. If researchers can incorporate preexisting HIT components that have undergone usability evaluation into their HIT product, they can focus on later-stage usability evaluations exploring the short and long-term influence of the HIT on health outcomes.

Limitations

This review has several limitations. It excluded non-English literature. Since English is not the national or common language in half of the countries in Africa, this may have led to an unintentional exclusion of articles without an English version. This single language selection may explain why the review did not include any North African countries, such as Arabic-speaking countries like Egypt and Morocco, which have more developed healthcare systems and infrastructure and publish the majority of Africa’s scientific research.^73,74 However, of the 54 countries that make up Africa, 27 African countries include English as an official language, so the exclusion of non-English publications is not a major limitation to this review.⁷⁵ Furthermore, the review is based on a search of only three databases, PubMed, Embase, and ACM. However, the use of ACM, an informatic-focused database, along with a thorough reference list review supported the decision to not search additional databases.

CONCLUSION

Developing and enhancing usable HIT is critical to promoting equitable health service delivery and high-quality care in Africa. The trend of more recent usability evaluations aligns with the increased development and utilization of HIT in low-resource settings. The use of mixed methods to obtain a comprehensive understanding of HIT usability is critical and provides researchers with rich results that they can leverage to further tailor their technology. This scoping review aimed to determine when usability evaluations were occurring and what components of usability they were exploring and found gaps in the current literature, and minimal advancement of theoretical approaches for HIT usability evaluation. In future studies, researchers need to incorporate theoretical frameworks to provide a stronger rationale for their work and enhance the rigor of usability evaluation research. The current literature shows that most articles focused on later-stage evaluations right before or at the point of HIT implementation. Early-stage evaluations (stages 1 and 2) and interactions (types 0 and 1) should receive special attention in future research. HIT that is rigorously tested, highly usable, and tailored to the cultural, geographic, and disease-specific needs is critical to overcome the health inequalities found in Africa.^{30,33,38,53,56,57,59–66,76–106}

FUNDING

This work was supported by the Reducing Health Disparities Through Informatics (RHeaDI) Pre- and Postdoctoral Training Program (T32NR007969).

AUTHOR CONTRIBUTIONS

All authors contributed significantly to this work. KD and SB conceptualized the study. KD and MH searched for and retrieved relevant articles, screened, and extracted data from included articles. KD interpreted the data. KD drafted the article, and MH and SB made substantive revisions to the article. All authors gave final approval of and accepted accountability for the article.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

CONFLICT OF INTEREST STATEMENT

None declared.

DATA AVAILABILITY

This article does not include a data set. The list of articles analyzed is available in the references and in Supplementary Table S4.