Artificial intelligence, ChatGPT, and other large language models for social determinants of health: Current state and future directions

Summary

This perspective highlights the importance of addressing social determinants of health (SDOH) in patient health outcomes and health inequity, a global problem exacerbated by the COVID-19 pandemic. We provide a broad discussion on current developments in digital health and artificial intelligence (AI), including large language models (LLMs), as transformative tools in addressing SDOH factors, offering new capabilities for disease surveillance and patient care. Simultaneously, we bring attention to challenges, such as data standardization, infrastructure limitations, digital literacy, and algorithmic bias, that could hinder equitable access to AI benefits. For LLMs, we highlight potential unique challenges and risks including environmental impact, unfair labor practices, inadvertent disinformation or “hallucinations,” proliferation of bias, and infringement of copyrights. We propose the need for a multitiered approach to digital inclusion as an SDOH and the development of ethical and responsible AI practice frameworks globally and provide suggestions on bridging the gap from development to implementation of equitable AI technologies.

Ong et al. discuss opportunities and limitations for AI and large language model (LLM)-based solutions in addressing social determinants of health (SDOH) factors and promoting global health equity. More work is needed to develop responsible AI frameworks and adopt a multitiered approach to digital inclusion.

Introduction

Globally, health equity is a pervasive and prevalent problem. Indicators of health often follow a social gradient: the lower the socioeconomic position, the worse the health indicator.^1,2 Factors contributing to health indicators can be broadly classified into five key domains, genetics, behavior, environmental and physical influences/environmental exposures, medical care, and social,³ of which social and behavioral factors account for up to 60% premature deaths.⁴

The COVID-19 pandemic, which led to lockdowns worldwide, occurred against this backdrop of existing health inequality.⁵ COVID-19 infection and mortality rates were disproportionately higher in socially disadvantaged regions, racial and ethnic minority groups, and marginalized populations across the globe.^6,7 In addition to a higher endemic prevalence of chronic diseases such as diabetes and cardiopulmonary conditions, social factors are thought to be key contributors to this observed phenomenon, e.g., living in crowded spaces, exposure to secondhand smoke, limited access to ambulatory or acute healthcare, and information asymmetry.^8,9 This pandemic has thrown social determinants of health (SDOH) research into the global spotlight: to achieve the goal of global health equity, addressing SDOH is crucial.

SDOH

SDOH is defined by the World Health Organization as non-medical conditions that affect one’s health and impact health-related outcomes.¹⁰ Conventionally, SDOH refers to the environments people reside and work in and encompasses systems and forces that influence one’s daily life such as policies related to the economy, political systems, and social norms. Its importance has led to development of initiatives such as “Healthy People 2023” in the USA, which seeks to remove health disparities. Of note, five key domains related to SDOH were identified, namely economic stability, education access and quality, healthcare access and quality, neighborhood and built environment, and lastly, social and community context.¹¹

A mounting body of evidence has shown that SDOH impacts health outcomes significantly¹²: lead exposure in suboptimal residential areas is associated with poorer cognitive function and physical development in young children,¹³ and air pollution, which is prevalent in disadvantaged regions, is causally related to exacerbations of respiratory diseases¹⁴; children growing up in impoverished communities are more likely to suffer from emotional and mental stressors and subsequently are at higher risk for premature mortality.¹⁵

Overall, despite the wealth of evidence available related to SDOH and health outcomes, there remain significant evidence gaps in the literature, particularly with regard to SDOH with complex health interactions, which include intangible factors such as political, socioeconomic, and cultural constructs that are difficult to quantify.¹⁶ Significant technical challenges lie in the designing of studies, as some factors may not be amenable to conventional studies such as randomized controlled trials.¹⁵ This is also compounded by a long lag time to see the study effects and the need for collection of extensive sociodemographic and economic factors. The advent of artificial-intelligence-related technologies that are able to analyze large volume of healthcare data may aid in allowing better risk stratification of patients and eliciting understanding of the complex interactions between SDOH and health outcomes.

Digital health and SDOH

Digital health tools helped to address major global health challenges through improved access to healthcare services, strengthened health promotion and disease prevention, and enhanced care experiences for professionals and patients. Core digital domains include telehealth and artificial intelligence (AI), supported by other technology domains such as big data analytics, the internet of things (IoT), next-generation networks (e.g., 5G), and privacy-preserving platforms, e.g., blockchain.^17,18 The COVID-19 global health crisis was a major turning point for digital health. Humanitarian and economic needs presented by the pandemic are driving the development and adoption of new digital technologies at an unprecedented scale and speed. Various digital technologies were in clinical implementation during COVID-19. In particular, AI proved to be a powerful emerging technology that improved care during the pandemic in various ways: forecast of infectious disease dynamics and outcomes of public health interventions, disease surveillance and outbreak detection, real-time population monitoring, and timely and accurate diagnosis of infections, as well as prognosis of disease severity.¹⁹

The past months have seen an accelerated development in the field of generative AI with the release of large language models (LLM) such as GPT-4, ChatGPT, and bidirectional encoder representations from Transformers (BERT).²⁰ In brief, Transformer architecture, a type of neural network algorithm, forms the basis of LLMs—also known as “foundation models.” Transformers learn contextual information from sequential data such as written text and audio signals.²¹ LLMs, trained on a large corpus of data from a wide range of sources, contain up to trillions of parameters that enable them to perform a broad variety of tasks without the need for specific training. Their application for the creation of new content has made them a key technology in the field of generative AI.²² The development of ChatGPT is outlined in Figure 1, illustrating the concept of reinforcement learning with human in the loop. In the biomedical space, proposed applications for LLMs are wide ranging^23,24: drug discovery and genome analysis; medical education; searching, analyzing, and interpreting large patient datasets; and generating human-like responses to improve physician-patient communication.

Figure 1.

Reinforcement learning with human in the loop

Despite rapid technological advancements, a concerted effort to address barriers to digital health such as challenges related to leadership and strategic alignment, information and technology governance, data management capacity, and systems integration, effective solution design is still lacking.²⁵ Low digital literacy, unequal access to digital health, and biased AI algorithms have raised mounting concerns over health equity.²⁶ As AI applications and LLM models become pervasive, we seek to understand the potential pitfalls of AI in driving health inequalities and identify key opportunities for AI in SDOH from a global perspective.

Opportunities for AI in SDOH

SDOH data used in AI models

Current AI models incorporating SDOH utilize various levels of data: individual-patient-level data, state- or neighborhood-level data, or country-level data (Figure 2). SDOH data are most collected by public health agencies. These datasets made available to public are often aggregated and anonymized reports or figures. In AI model development, securing patient-level data is often a priority.²⁷ Between high-income countries (HICs) and low- and middle-income countries (LMICs), we see a significant difference in the amount of open-access patient-level data that are availed under each SDOH domain. Table 1 illustrates examples of common SDOH data used in AI algorithms. As discussed in the sections below, a lack of data standardization poses a significant barrier to AI model development and validation across different populations.

Figure 2.

The interconnectedness of SDOH data sources

Individual patient-level data flow into neighborhood- and population-level data sources such as population census reports.

Table 1.

Examples of publicly available SDOH data in HICs and LMICs

WHO SDOH domains	Data collected by HICs28	Data collected by LMICs29
Economic stability	household poverty ratio, type of employment, occupation	type of employment, occupation
Education	highest education level	highest education level, native language
Healthcare services	most recent doctor visit, most recent wellness doctor visit, have a usual care facility, type of usual care facility	any visit to health facility in last 12 months, time spent traveling to healthcare facility
Social and community	household composition, caregiving sources, health insurance, working hours	household composition, health insurance
Neighborhood and built environment	region of residence	source of drinking water, access to toilet, access to vehicular transport, materials used in the construction of home

Under each SDOH domain, a lack of congruence and standardization of variables collected reflects differences in prevailing health priorities of the respective region.

SDOH extraction from electronic health records (EHRs)

Up to 80% medical data are unstructured, and traditional means of extracting information from this text are manual and time consuming. Qualitative information about patients’ lifestyle and social determinants is often embedded within unstructured clinical notes of EHRs. Advances in natural language processing (NLP) offer an efficient and automated approach to identify, collect, and analyze useful SDOH information from existing EHR systems and databases.³⁰ Of note, deep learning (DL) algorithms such as CNNs (convolutional neural networks) and BERT have been applied to SDOH annotation from clinical unstructured text.^31,32 However, the NLP methodology with the best performance for identification of less-well-studied SDOH and longitudinal analyses of SDOH-related data remains a topic of contention. LLMs have also been shown to have superior performance in the extraction of SDOH compared to structured data.³³

Research on SDOH and health outcomes

Several reviews have evaluated the role of AI in assessing social determinants, especially substance abuse, employment status, and socioeconomic status on health outcomes.³⁴ Across these studies, the roles of AI and SDOH were most extensively evaluated in patients with mental health and chronic diseases such as diabetes, particularly with the incorporation of SDOH factors for clinical risk prediction. The relationship between social determinants and health outcomes is often indirect and complex, involving different biopsychosocial processes.¹⁵ Advancements in multilevel modeling have made it possible to combine individual-level and group-level SDOH data to improve disease surveillance and prediction and evaluation of population health interventions.³⁵ In addition, temporal evolution of SDOH factors, geospatial relationships, and their impact on health outcomes have also been poorly characterized by conventional statistical methods. AI models generally outperform parametric models in this respect.³⁶ However, inherent weaknesses such as lack of interpretability with consequent difficulty in deriving mechanistic explanations may limit widespread application of AI models in SDOH assessment.

Improve access to care

AI healthcare applications range broadly from disease diagnosis, patient triage, disease surveillance, and prediction to personalizing treatment, health policy, and planning.³⁷ Applications of AI in LMICs are still fairly limited in scope, most frequently adopted for the screening of non-communicable diseases such as diabetic retinopathy, diagnosis of infectious diseases such as tuberculosis, and augmentation of maternal and child health.^38,39 At a policy level, AI modeling has been used to accurately identify high-risk individuals for more targeted health policy generation.⁴⁰

AI has the potential to democratize specialized care in under-resourced settings and improve care accessibility, an important SDOH. Democratization, used in our context, refers to the endeavor of making healthcare universally accessible. In LMICs, telehealth platforms can be coupled with innovative diagnostic tests and home-based monitoring tools to promote delivery of patient-centric specialist care in a primary care setting. For example, the integration of DL systems with smartphone-based applications has been developed to detect prevalent eye diseases such as glaucoma and diabetic retinopathy in LMICs.⁴¹ Dacal et al. described an end-to-end solution for diagnosis of soil-transmitted helminth infections in Kenya that includes a mobile app for image digitization, a DL algorithm for automatic detection of pathogen, and a telemedicine platform for analysis.⁴² High-risk pregnancy detection in rural areas of Africa is now possible without trained sonographers through the use a low-cost ultrasound probe attached to a smartphone app installed with a real-time DL algorithm.⁴³ While current studies predominantly address the accuracy of various AI tools in different healthcare settings, we acknowledge that a direct correlation between tool accuracy and the enhancement of care quality or accessibility has not been explicitly demonstrated. Empirical studies that holistically examine the integration of AI tools into care systems, their impact on patient outcomes, and broader systems-level outcomes are necessary to solidify the claim that AI can truly democratize care and enhance its accessibility and quality.

The role of AI in health service democratization is evident in high-income settings as well, particularly in the delivery of specialized care in radiology, pathology, and ophthalmology. These countries possess the ability to scale and implement AI tools at local or international levels for healthcare service delivery. AI tools currently used in the screening and detection of debilitating diseases such as diabetic retinopathy (DR) demonstrate high levels of precision and cost effectiveness when implemented country wide.^44,45 Better health equity is then achieved through efficient distribution of specialized person power and resources. In addition to right siting of specialized resources, AI enhances the overall quality of healthcare. AI-powered algorithms have outperformed physicians and traditional screening tools in detecting and diagnosing serious diseases such as skin and breast cancers.^46,47 Detection of these conditions at early stages significantly increases response to traditional lines of treatments that are potentially cheaper and more accessible to the general population.

LLMs in SDOH

LLMs like ChatGPT have the ability to generate and replicate human responses in a highly convincing manner. This linguistic ability, coupled with multilingual capabilities,^48,49 is purported to be capable of enhancing communication between healthcare providers and patients across geographical and language borders. Through the provision of health education and patient enquires, basic medical advice can be administered without the need to engage healthcare professionals directly, which addresses operational challenges in LMICs and remote areas where healthcare professional manpower is in a critical crunch. These models can support time-consuming, administrative activities such as discharge report writing, patient stay summaries, and patient history interrogation.⁵⁰ In addition, LLMs may overcome the infrastructural limitations (discussed in the section below) imposed by semantic and syntactic interoperability in EHR system.⁵¹ However, despite the huge promise of LLMs in transforming healthcare, the deployment of these models needs to be approached with caution. The lack of reliable and robust methods to distinguish between LLM-generated and human-generated content amplifies healthcare professionals’ and patients’ vulnerability to anthropomorphism.^52,53 Inherent risks presented by LLM-based applications include inadvertent disinformation or “hallucinations,” proliferation of bias, and infringement of copyrights, all of which can have far-reaching societal and health equity implications. These technologies in their current form have the potential to incite harm, especially in mental health, where NLP algorithms have shown biases relating to religion, race, gender, nationality, sexuality, and age.²⁰ There is a need for a framework to govern the development, deployment, technology assessment, and eventual regulation of LLMs for medical applications. We discuss some of the challenges and barriers relating to LLMs and SDOH below.

Challenges and barriers

Lack of SDOH data standardization

A lack of a standardized method of data collection and definitions has impeded progress in global SDOH research. Take housing data for example: it can include different definitions, metadata, and measurements. There is a lack of standardized data elements, assessment tools, measurable inputs, and data collection practices in clinical notes for patient-level SDOH information.³⁰ SDOH also presents at individual, neighborhood, and population levels. LMICs face capacity constraints to handle large data. Data sources owned by private sectors may not be available publicly as discussed above. Data are often not harmonized nor interoperable across sectors, making data integration a challenge and subsequent analysis/modeling difficult.²⁹ The lack of standardized terminologies and definitions will also limit semantic interoperability, posing a challenge to the implementation of algorithms. This issue is not specific to LMICs. In Denmark, for example, the healthcare sector is highly digitized, with a long history of health data registry establishment. Robust healthcare research is made possible due to individual-level record linkage between all data sources.⁵⁴ However, important SDOH variables such as lifestyle factors are still not routinely captured in a standardized manner. Furthermore, multicohort validation of AI models rely heavily on standardized frameworks for SDOH data collection.

Initial efforts have been made to improve standardization of SDOH data and integrate data into care delivery. The AMA Integrated Health Model Initiative (IMHI) and Gravity project spearheaded by the American Medicine Academy seeks to develop consensus-driven standards for collection and transfer of SDOH data with subsequent integration into CPT codes.⁵⁵ The NHS England published a consensus common outcomes framework (COF), which recommended the use of 3 primary care codes to standardize recording of social-prescribing activities.⁵⁶ However, the codes lack the level of granularity that is required for the purpose of SDOH research, AI modeling, and implementation. Another widely available tool is the ICD-10 Z-codes (Z55–Z65) that represent SDOH data such as employment status, access to shelter, and social insurance or welfare support. However, despite global adoption of ICD-10, Z-codes are highly underutilized, with less than 2% hospitalized patients having a Z-coded diagnosis.⁵⁷

Health infrastructure limitations and environmental impact

The lack of essential physical facilities, regulatory barriers, and population readiness pose significant challenges to equitable implementation of AI. Variances in internet connectivity, linguistic compatibility, operational skills, and hardware prerequisites may challenge access to benefits conferred by AI models and LLMs. On the level of physical infrastructure, LMICs and remote areas may not possess reliable telecommunication networks, stable internet connections, or sufficient equipment. Recognizing infrastructure limitations is essential for successful implementation of digital tools, e.g., mobile applications, in many cases, need to have the ability to work in both online and offline modes.⁵⁸ Notably, disparities in technological access may further exacerbate global inequalities by disproportionately favoring specific cohorts.⁵⁹ Implementing digital solutions with lower technical requirements and cost is promising. A multidisciplinary approach supported by government, hospital systems, physicians, AI companies, and industry partners is necessary to create a robust local infrastructure of IT systems, clinical services, and digital communications networks.⁶⁰ LMICs should invest in infrastructure for local validation, and model recalibration will also lay the groundwork for eventual contribution of local data to international data repositories.⁶¹ Aside from physical infrastructure, regulatory barriers pose a critical challenge to successful clinical deployment of AI models and digital health solutions. While a plethora of fully validated AI models can be found in published literature, clinical implementation in practice is hampered by the lack of a clear regulatory approval and deployment roadmap.⁶² The development of a regulatory framework for AI models is at its nascent stage. The establishment of national and international regulatory standards could accelerate technology adoption.

Intensive computing processes of AI and LLMs alike give rise to environmental and potential labor impacts. LLMs, in particular, require significant energy demands during the training process and, correspondingly, result in high carbon emissions.⁶³ This can exacerbate existing environmental challenges including rising carbon footprint, water use, and soil pollution of sealing, which has implications for environmental quality and collective influence on SDOH.^52,64 Particular to LMICs, the engagement of low-cost workers to perform annotation of toxic language (including text embedding hate speech, sexual abuse, violence) may contribute to negative psychological health outcomes among these workers.^65,66 It is thus imperative to examine the impact of the aforementioned labor practices to ensure that advancements in AI do not come at the expense of employee well-being.

Digital literacy, inclusivity, and culture of acceptance

Digital literacy and digital inclusivity are important indicators of population readiness. Certain population groups fare better in a digitized world, for example the younger, the better educated, and/or urban dwellers. These groups consistently report higher internet access and digital skills and will benefit the most from digital and AI health technologies. Conversely, vulnerable groups commonly experience high barriers to access, creating a digital health paradox. Singapore is a digitally advanced nation with almost universal digital availability, yet when COVID-19 forced rapid digital adoption, gaps in access by vulnerable groups such as low-income households, the elderly, and migrant workers were found.⁶⁷ Placing emphasis on digital inclusivity as a “super” SDOH could mitigate health inequalities brought about by AI and digital health technologies. Underpinning successful implementation of digital health is innate trust and belief in the benefits of digital technologies at an individual level. Careful, calibrated change management strategies are critical but often neglected by governments and healthcare leaders. This is also critical to the cultural transformation of digital health from traditional health services.

Pertaining to LLMs, we discussed in the above section the potential of fine-tuned or domain-specific LLMs in mitigating healthcare worker burdens through automation of clinical documentation and enhancement of healthcare conversations. However, widespread adoption of such technology is currently impeded by concerns related to accuracy and consistency. Medical document summarization involves more than just noting facts; it also involves creation of an accurate narrative of the patient. Nuances of medical humanities, patient preferences, and complexities of socioeconomic and psychological status have yet to be fully replicated by generative AI.⁶⁸ These current limitations suggest that healthcare professionals may still have to manually review outputs for accuracy that, in turn, may be more time intensive and cognitively fatiguing.

Bias in algorithms and privacy concerns

Inadequate representation in training datasets may jeopardize the effectiveness of AI-based health interventions in vulnerable populations. AI models often generalize poorly in populations outside of its data training and validation cohort. While models may perform well in well-represented regions, disparate performance in less-represented cohorts and minority populations may perpetuate existing health disparities.⁶⁹ Clinical AI models often draw data from EHRs, biobanks, or genome databases, in which groups with irregular or limited access to the healthcare systems, e.g., ethnic minorities and immigrants, will be poorly represented.^70,71 Data shifts and model deterioration may result in missed diagnosis, inaccurate prognosis, and suboptimal treatment recommendations in unrepresented cohorts.^72,73 Along the same vein, implementing algorithms trained using large datasets has been shown to be biased against individuals of different races and socioeconomic status. For instance, chest X-ray classifiers trained using datasets dominated by White patients performed poorly in Medicare patients,⁷⁴ and a risk prediction algorithm trained using large datasets failed to triage African Americans for necessary care.⁷⁵ This will remain a pervasive problem since more than half of the datasets used for clinical AI development originate from either the USA or China.⁶¹ Careful validation of AI algorithms is imperative in ensuring reproducible results before being deployed in other populations. To effectively combat such bias, models should be trained using multi-institutional datasets and surveyed for disparate performance during model deployment.⁷⁶ In addition to a lack of representation in datasets, AI-driven clinical decision support may inadvertently reproduce the biased judgements and decision-making that are entrenched in real-world practice. Biases captured in unstructured clinical notes may be picked up by and replicated in NLP-based AI tools. Along the same vein, bias in pretraining or fine-tuning datasets of LLMs may encode historical and ongoing bias. Prejudiced, stereotyped, and discriminatory outputs reflect empirically observed patterns of judgements from pretraining datasets. In an experiment using commercial LLMs, kidney function estimation queries returned answers based on an outdated and race-biased medical formula.⁷⁷ At this juncture, we caution against unchecked, unaccounted, and untested applications of LLM in clinical decision support. LLMs may inadvertently perpetuate these biases and contribute toward widening health inequities.

In the realm of data and model responsibility, the party that remains liable for the outcomes of AI and LLMs remains unclear, and it is important to note that absence of autonomy and sentience renders a lack of moral agency in AI.⁷⁸ Pertaining to data privacy, patient-related data are entrusted to healthcare institutions and professionals, and the use of big data may lead to potential unauthorized usage of personal data in predictive analyses.⁷⁹ A lack of robust data privacy frameworks may introduce vulnerabilities for penetration and manipulation of sensitive patient information. Reidentification of anonymized medical data is possible with few spatiotemporal datapoints despite deidentification efforts.⁸⁰ Vulnerable populations are in jeopardy when SDOH data carrying sensitive and stigmatizing information are uncovered. Examples of cybersecurity measures include user authentication, audit trails, data encryption, and secure data transmission mechanisms.⁵⁸

Developments in privacy-preserving technologies are gaining traction. Homomorphic encryption is one such solution that permits for analyses to be performed securely on encrypted data without the need for decryption. Generative adversarial networks (GANs) are generative models that learn from the distribution of healthcare data or medical images to create large, realistic synthetic data.⁷⁶ The privacy of patients is protected, as data cannot be attributed to any single individual. In addition, GANs can be used to augment healthcare datasets with class imbalance or insufficient variability. Performance of models augmented by GANs outperforms those trained using traditionally augmented data using a fraction of the original training dataset.⁸¹ GANs present as a novel tool to overcome both privacy concerns and algorithmic bias discussed above. Federated machine learning is a data-private, multisite, collaborative learning approach that distributes model training and aggregates model weights without the need to share individual patient datasets.^82,83 Blockchain technology facilitates secure and auditable exchange of sensitive patient data, safeguarding data integrity and immutability. Blockchain can be coupled with the aforenamed privacy-preserving techniques to augment AI algorithms while providing an additional layer of data security.^84,85

Future directions

Emphasis on digital inclusion

In the recent decades, digital literacy and internet access have been increasingly recognized as “super” SDOHs due to their complex interlink and ability to affect other SODH.⁸⁶ A key barrier to digital literacy lies in the digital divide, which afflicts disadvantaged populations more significantly. For example, approximately one-sixth of low-income household families in the USA do not have broadband internet access.⁸⁷

With expanding applications of AI and digital health, it is imperative to devise strategies to bridge the digital divide in our pursuit of health equity. The concept of digital inclusion encompasses activities that allow for equitable access and use of information communication technologies (https://www.digitalinclusion.org/definitions/). Digital inclusion goes beyond simple possession of internet-enabled devices and internet access; elements such as digital literacy training, availability of technical support, and applications to empower users are equally crucial (https://www.digitalinclusion.org/definitions/). Within the EU, only about half of the population possesses basic digital skills, despite widespread access to the internet (https://digital-strategy.ec.europa.eu/en/library/ict-work-digital-skills-workplace). We propose a multitiered approach to evaluate and address digital inclusion as an SDOH (Figure 3).

Figure 3.

A multitiered approach for digital inclusion using WHO SDOH framework

The framework was adapted from Van Dijk.⁸⁸

Global agenda for ethical and equitable AI practices

The establishment of global guidelines and framework is necessary to guide the development, implementation, and evaluation of AI-based digital health technologies to ensure efficacy sustainability and equitability. Various guidelines and frameworks have been developed to guide the ethical use of AI including guidance from international bodies, e.g., the WHO (https://www.who.int/publications/i/item/9789240029200) and UNI Global Union (https://uniglobalunion.org/report/10-principles-for-ethical-artificial-intelligence/)—however, significant divergences exist in the interpretation and prioritization of specific ethics principles.⁸⁹ In addition, there is a lack of representation from LMICs such as African and South American countries, revealing a power imbalance in this ethical debate. Relevant stakeholders from different global regions should be involved in the development of a global consensus framework while respecting variability in regional priorities, capabilities, and challenges.⁶⁰

Furthermore, on the research front, a global research agenda for AI interventions relevant to addressing SDOH will ensure that tools developed respond to population needs.³⁷ Furthermore, AI research in LMICs can be enhanced by international frameworks to promote model transparency such as the “transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD)” reporting guideline and “developmental and exploratory clinical investigations of decision support systems driven by AI (DECIDE-AI).”^90,91

Bridge development to implementation

Greater efforts are also required to bridge development to implementation for AI-based technologies to maximize gain. Lessons drawn from implementation science suggest a need to map out unique circumstances and processes within each local setting to identify potential interactions and barriers of technology implementation. Active engagement of local stakeholders such as government agencies and non-governmental organizations (NGOs) in the process of workflow design is essential, especially in resource-limited settings.^38,92 For example, DL algorithms developed using computed tomography (CT) images cannot be generalized or implemented in a region where CT scanners are unavailable. Along the same vein, there is a need to conduct economic evaluation of AI-based technologies under different resource settings. For the under-resourced settings, the key is to prevent life-threatening conditions, and thus the referral thresholds by these digital tools will need to be carefully set based on the health economic analysis that is feasible to the specific country. Cost-effective analyses of AI vs. standard of care reveal useful information for policy makers and regulators such as the optimal operating model or whether the new initiative should be adopted at all.^44,93

The “black box” nature of DL- and LLM-based models, compounded by the tendency of LLMs to “hallucinate,” is a key impediment to the development of trustworthy and explainable AI models. Beyond improving accuracy of AI model outputs, significant effort and progress has been made in developing explainability techniques for machine learning and DL models, such as saliency-based explainable AI (XAI).⁹⁴ However, interpretation of such techniques, e.g., SHAP, may not be sufficiently “layman” to facilitate rapid decision-making in clinical practice.⁹⁵ In addition, there is still a paucity of evidence to quantify improved clinician or patient trust in AI model output through incorporation of these explainability techniques. We encourage more research and exploration in examining factors that foster greater user trust in AI models and the design of conceptual models and frameworks⁹⁶ to bridge the gap between model development and effective implementation.

Conclusion

Achieving health equity remains a challenge, and addressing SDOH is progressively becoming a global priority. In a post-pandemic world, the widening digital and health divide has thrown caution to the use of AI and digital health technologies. Significant challenges and barriers to equitable AI implementation remain, especially with regard to overcoming infrastructure limitations and algorithmic bias. Newer techniques and global collaborative efforts have the potential to overcome these barriers and allow disruptive and transformative digital health tools to flourish.