Artificial Intelligence in Healthcare: Balancing Technological Innovation With Health and Care Workforce Priorities

ABSTRACT

Artificial Intelligence (AI) has emerged as a transformative force in healthcare, offering significant potential to address workforce challenges and improve patient outcomes. This perspective article presents a framework for responsible AI innovation, emphasising ethical governance, responsible leadership and a commitment to human‐centred AI. It provides guidance for healthcare organisations to position AI as a strategic enabler, augmenting the health and care workforce and fostering sustainable, patient‐centred advancements in healthcare.

1. Introduction

Artificial Intelligence (AI) has become a transformative force in healthcare, with the capacity to reimagine processes, decision‐making and even workforce dynamics [1, 2, 3]. In an era when the healthcare sector is grappling with a workforce crisis marked by burnout and staffing shortages among health and care workers [4, 5, 6, 7], research suggests that 30%–40% of tasks in healthcare could be automated using AI technologies [8]. Automation holds the promise of alleviating workforce burdens and enabling health and care workers to focus on more complex, patient‐centred care, thus improving retention and solving the crisis. However, its transformative potential must be considered within the broader context of systemic inequities that persist globally in healthcare. While AI presents opportunities to enhance efficiency and precision in high‐resource settings and to address access gaps and workforce shortages in low‐resource environments, its adoption is far from a straightforward solution to the challenges facing healthcare systems [3]. Disparities in data availability, infrastructure and funding within local organizational contexts can constrain its impact, risking the exacerbation of existing inequities. Without deliberate efforts to address these foundational challenges, poorly executed AI integration may ultimately amplify rather than resolve the disparities it aims to overcome [2, 9].

What sets AI apart from any technological innovation in healthcare has previously encountered is its dynamic nature. Unlike traditional technology tools such as imaging devices or electronic medical records, which are static, predefined, and mostly human‐managed, AI systems evolve through self‐learning and often operate in what is referred to as a ‘black box’ [10].

While significant advancements have been made in explainable AI, the inherent opacity of how AI systems arrive at conclusions continues to pose unique challenges, particularly in high‐stakes healthcare environments where trust, accountability and decision‐making transparency are paramount [11].

The dynamic and iterative nature of AI allows it to address complex challenges, such as identifying subtle diagnostic patterns or predicting patient outcomes. For instance, predictive analytics can identify early signs of sepsis [12], and computer vision enhances diagnostic precision in radiology [13]. Yet, this same adaptability introduces risks. An AI system trained on biased datasets may perpetuate inequities, making flawed or harmful decisions for underrepresented populations [14]. These issues underscore the necessity of robust oversight, as AI's self‐learning capabilities are not inherently aligned with human values or ethical principles.

Unlike traditional innovations, which can be integrated into workflows with minimal disruption, AI's reliance on continuous data and its capacity for self‐directed evolution require rethinking the foundational structures of healthcare delivery. AI's transformative potential demands a new approach, one that recognizes the limitations of a black‐box system while leveraging its capabilities responsibly and understanding the importance of the health and care workers. This underscores the need for thoughtful, context‐sensitive approaches to AI adoption that prioritize transparent AI strategy.

2. A Strategy for Balancing Innovation and Responsibility in AI Adoption

To navigate the complexity of balancing innovation and responsibility in AI adoption, we propose a framework for responsible AI innovation in healthcare, grounded in ethical governance, accountable leadership and human‐centred design. Accompanied by a practical checklist (Figure 1), the framework offers guidance for healthcare organisations, emphasising the importance of adapting to local contexts while ensuring the integrity of both the workforce and the patients they serve.

FIGURE 1.

Checklist for responsible AI innovation in healthcare. Source: Authors' own figure.

2.1. Ethical Governance

In 2022, two National Health Services (NHS) hospital trusts in the United Kingdom (UK) incurred losses totalling nearly £15 million (approximately $18 million) due to their investments in Sinsyne Health, an AI startup focused on drug discovery and development. These trusts had entered into agreements with Sinsyne Health, exchanging patient data for company shares. However, the company’s delisting from the London Stock Exchange, UK, led to a significant devaluation of these shares, resulting in substantial financial losses for the trusts [15].

Balancing innovation with ethical AI development requires a thoughtful and holistic approach—one that not only seizes opportunities for technological progress but also steadfastly upholds the values, rights, and needs of the patients it is designed to serve. Achieving this balance requires that every stage of AI development and deployment be grounded in the core principles of equity, transparency and responsibility.

First, equitable access to diverse datasets is vital to ensuring that AI systems are representative and relevant across patient populations, mitigating systemic biases that can stem from skewed or incomplete data. However, the procurement of data for AI innovation must prioritise respect for patient privacy and uphold individual autonomy. Healthcare organisations must adopt transparent and voluntary data collection and sharing practices, adhering to stringent local regulations, such as the General Data Protection Regulation and the Health Insurance Portability and Accountability Act as well as global ethical frameworks like the World Health Organisation's guidelines on AI in health [16]. Patients should be fully informed about how their data will be utilised, with robust anonymisation techniques employed to safeguard against re‐identification. This is particularly critical in low‐resource settings, where the urgent demand for innovation must not lead to the exploitation of communities for AI testing and model development.

Second, when AI models are developed, addressing algorithmic bias must be a core priority. Bias can emerge from imbalances in training data or from structural inequities embedded within healthcare systems. For example, in 2019, DeepMind, an AI subsidiary of Alphabet Inc., collaborated with the U.S. Department of Veterans Affairs (VA) to develop an artificial intelligence system aimed at predicting acute kidney injury in patients up to 48 h before onset. This model was trained on medical records from over 700,000 veterans, a cohort in which female patients comprised only about 6.38%. This underrepresentation led to the model performing less effectively for women, achieving an area under the curve of approximately 71% for female patients with stage 3 acute kidney disease, compared to about 84% for male patients [17]^.

To avoid biases, AI models should be tested with locally relevant datasets and subjected to regular audits to identify and mitigate inequities, ensuring their effectiveness across diverse patient populations, including for women, men, and other genders. Additionally, the development process should involve diverse stakeholders, including clinicians and patient representatives, to align AI systems with organizational values and prioritise patient‐centred care. This collaborative approach ensures that AI solutions reflect ethical standards, support equitable outcomes, and address the lived realities of patients in diverse healthcare contexts.

In addition to governing the data, ethical governance of AI in healthcare must address the critical issue of accountability for AI‐driven decisions, ensuring clear roles and responsibilities across health and care workers, organisations, and AI developers. Central to this governance is the principle that AI should function as a decision‐support tool, not an autonomous decision‐maker. Health and care workers must retain ultimate responsibility for decisions informed by AI, with regulatory frameworks reinforcing their role in verifying and validating AI outputs before acting on them.

To strengthen oversight and support health and care workers, organisations should establish AI Safety Committees tasked with monitoring, evaluating, and guiding the ethical deployment of AI systems. Additionally, creating AI registries at the organizational level could enhance accountability by cataloguing AI systems in use, their intended purposes, and performance metrics [18]. Such registries would enable continuous monitoring of AI systems, ensuring they remain aligned with ethical standards and organizational values. They would also facilitate audits, track outcomes, and provide a centralised mechanism to identify and address potential issues proactively.

2.2. Accountable Leadership

In 2012, the University of Texas MD Anderson Cancer Center, United States, partnered with IBM to develop the Oncology Expert Advisor, a cognitive computing system powered by IBM Watson. The goal was to leverage Watson’s artificial intelligence to advance cancer research and create personalised treatment plans, starting with leukaemia patients. However, by 2017, MD Anderson had discontinued the project after spending over $62 million without deploying the system in clinical practice. An internal audit revealed procurement issues, cost overruns and challenges integrating Watson into the hospital’s operations. The failed AI partnership between MD Anderson Cancer Center and IBM Watson highlights the pivotal role of leadership and strategy plays in implementing transformative technologies in healthcare [19].

The challenges faced by early adopters like MD Anderson implementing AI solutions often highlight critical shortcomings in change leadership, particularly in aligning organizational readiness with the complexities of technological innovation. Ambitious goals for AI systems can sometimes lack a realistic evaluation of the technology's maturity and the institution's capacity to integrate it effectively into clinical workflows. This disconnect can lead to fragmented efforts, operational missteps and unrealistic expectations that ultimately undermine the initiative's success.

Successful AI adoption in healthcare requires accountable leadership that goes beyond technological readiness, addressing the ethical, technical, and operational challenges of integrating AI into complex healthcare systems [20, 21]. At its core, accountable leadership ensures that AI aligns with organizational goals, supports clinical workflows, and respects the broader societal values of equity and inclusivity. This includes a critical assessment of whether AI will genuinely add value to existing healthcare delivery models or inadvertently hinder them, particularly in settings where resources and infrastructure may be limited. For example, Google Health deployed a highly accurate AI system for diabetic retinopathy, demonstrating human‐level accuracy in lab settings. However, when implemented in Thailand, the system generated false positives and led to unnecessary follow‐up visits. These failures were attributed to poor‐quality images and slow internet connections, conditions vastly different from the controlled lab environment where the AI was initially tested [22].

Central to accountable leadership is proactive engagement with all stakeholders—clinicians, frontline workers, patients and administrators—whose insights are essential to designing and deploying AI systems that work in real‐world settings. Exclusion of these key voices often results in systems that fail to address critical needs or integrate effectively into existing workflows. This gap can be further exacerbated by technical challenges, such as interoperability issues with electronic health records, or by deploying AI solutions that duplicate or complicate existing processes, rather than enhancing them.

Building and maintaining trust is a cornerstone of accountable leadership. Transparency about AI's capabilities and limitations is critical to fostering confidence and ensuring realistic expectations. Leaders must openly communicate the potential and boundaries of AI systems, avoiding overpromising while addressing stakeholder concerns. Iterative feedback loops that allow for stakeholder input during deployment and refinement are crucial for ensuring the technology adapts to evolving clinical practices and remains relevant.

Accountable leadership also demands a structured approach to implementation, including thorough evaluations of how AI fits within the healthcare delivery model. Phased and iterative strategies enable organisations to test, refine and scale AI systems in a controlled manner, reducing the risk of failure. Regular audits and reviews ensure that the systems align with organizational priorities and maintain ethical integrity. Leadership must take responsibility not only for the technical performance of AI but also for its impact on the health and care workforce, patients and overall healthcare outcomes.

Finally, accountable leadership at the public policy level can drive the implementation of ethical, gender‐sensitive and inclusive AI strategies through a multi‐layered approach. This ensures that these values are considered at every level—health and care workers, teams, organisations and across the broader healthcare system.

2.3. Human‐Centred AI

The integration of AI into healthcare holds immense promise to alleviate burdens such as repetitive administrative tasks and enhance efficiency hence prioritising workforce well‐being. However, its implementation must prioritise ‘health and care worker in the loop’ to ensure that technological progress supports—not supplants—the professional competencies and capacities of the health and care workforce, like expertise, empathy, and critical thinking. Without this guiding principle, AI risks disrupting the relational and professional foundations of healthcare, particularly for vulnerable and underrepresented groups.

The National Eating Disorders Association (NEDA) in the United States offers a cautionary example of the risks associated with replacing human expertise with AI. The organisation's decision to replace its helpline staff—who had provided support to nearly 70,000 individuals annually—with Tessa, an AI chatbot, was met with significant challenges. The chatbot, intended to enhance service delivery, instead dispensed advice that was not only inappropriate but also potentially harmful, including recommending calorie restriction and weight loss strategies that contradicted established guidelines for eating disorder care [23].

This example highlights the profound risks of displacing human expertise with AI in areas where empathy, contextual understanding and clinical nuance are critical. AI systems that are poorly designed or insufficiently tested risk not only delivering subpar care but also eroding the trust of both professionals and patients. Furthermore, the spectre of AI‐induced job displacement is particularly acute in healthcare, where estimates suggest that 30%–40% of tasks could be automated, notably in diagnostic imaging and interpretation, clinical decision support, administrative workflow automation, personalised medicine, patient monitoring and surgical assistance [8, 24]. In environments constrained by budgetary pressures, the fear of automation displacing traditionally human roles is palpable. Poorly implemented AI systems can exacerbate stress and burnout by introducing inefficiencies, disrupting workflows, or undermining professional standards and autonomy in an already vulnerable sector as healthcare. Furthermore, over‐reliance risks erode critical thinking and diagnostic skills. Maintaining human oversight in decision‐making processes is essential to safeguard patient safety and professional autonomy.

On the other hand, when AI is effective, the democratisation of AI in healthcare is critical to ensuring equitable access, yet many rural and under‐resourced health and care workers—often migrants and women—lack access to high‐quality AI tools, exacerbating existing inequities. These workers, already burdened by workforce shortages and limited infrastructure, face additional challenges when left without the resources or training needed to integrate AI into their workflows effectively. This digital divide risks deepening systemic inequalities, reducing their ability to deliver quality care. To address this, leadership must prioritise the development of affordable, context‐appropriate AI tools, robust training programs and policies that promote equitable access. By ensuring inclusivity and addressing barriers unique to marginalised health and care workers, AI can empower these professionals and improve care delivery in even the most under‐resourced settings.

Targeted upskilling programs are essential to prepare health and care workers for the changing demands of an AI‐augmented environment, providing them with the technical and adaptive skills necessary to thrive. Equally critical is the inclusion of participatory design processes that actively engage health and care workers—particularly those from underrepresented groups—in shaping AI systems. This approach ensures that AI tools reflect the needs and lived experiences of the workforce, fostering trust and alignment.

3. The Need for Robust Policy Support

While ethical governance, accountable leadership and human‐centred approaches are essential for responsible AI innovation, policy at the organizational, professional and health systems levels plays a pivotal role in guiding responsible AI strategies, echoing a need for multi‐level trans‐sectoral approaches to health and care workforce governance that foster implementation [4, 25].

Macro policies like the European Union's AI Act [1], which categorises AI systems by risk and emphasises fairness and safety, and global consensus such as the World Health Organisation's guidance on AI in healthcare [16], which emphasises ethical principles such as transparency, inclusivity and accountability, are crucial. These policies provide a framework for ensuring that AI technologies are developed and deployed in a manner that prioritises patient safety and ethical standards. This should be complemented by national level AI policy frameworks that balance equity considerations with tailored approaches to local workforce challenges, ensuring that benefits are equitably distributed.

At the professional regulatory level, healthcare authorities must establish robust policy frameworks inspired by models such as the Standing Committee of European Doctors (CPME)'s guidelines on AI deployment. (CPME, 2024) These frameworks should ensure the preservation of professional autonomy, the wellbeing of health and care workers, and patient‐centric care. Further, these regulatory policy guidelines should encourage for transparency in algorithmic decision‐making, which is essential for building trust and confidence among health and care workers. This transparency allows for clear accountability, ensuring that unintended consequences can be traced and addressed. It also supports professional autonomy and ethical practice, helping health and care workers confidently integrate AI tools [26].

Within healthcare organisations, organizational level policies are required to foster a responsible AI integration. Policies should guarantee job security for health and care workers by embedding protections against AI‐induced displacement and should offer continuous professional development through upskilling opportunities. Furthermore, organizational‐level policies should actively support the engagement of health and care workers in the co‐design of AI solutions. This collaborative approach ensures that these technologies are not only practical but also seamlessly integrated into clinical workflows, enhancing their overall efficacy and utility.

4. Conclusion

Artificial Intelligence (AI) holds the promise of a transformative impact on healthcare and new solution for the workforce crisis. Nevertheless, its integration requires the guidance of carefully devised strategies that take into account the local organizational and health systems contexts, emphasising ethical governance, accountable leadership and a human‐centred approach. It is essential for health systems and provider organisations to establish trans‐sectorial policy mechanisms that uphold professional values, the wellbeing of health and care workers, and the crucial relational and human aspects of care provision. AI solutions should be developed in a way that they do not prematurely replace the human intelligence and empathy vital for patient care. Engaging health and care workers in a participatory approach to co‐design AI solutions is critical. By doing so, healthcare organisations can leverage AI’s transformative potential while ensuring that innovation aligns with the fundamental mission of healthcare: delivering compassionate, high‐quality care to all patients, ensuring no one is left behind [27] Furthermore, such alignment empowers health systems to prioritise the retention of health and care workers—particularly the 70% who are women [28]—while advancing progress towards achieving the Sustainable Development Goals (SDGs) [29].

Ethics Statement

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

No original data were used in this paper.