A Process Framework for Ethically Deploying Artificial Intelligence in Oncology

Artificial Intelligence (AI) is an emerging technology that uses complex algorithms to arrive at an outcome over a range of circumstances, leveraging the ability of computer systems to perform tasks that would usually require human levels of intelligence.^1-3 The use of AI in cancer care is rapidly expanding: a May 2022 PubMed search of the term cross-referenced with cancer revealed approximately 26,000 citations, with more than 60% published in the past five years. Ethical considerations for AI in oncology include patient equity, privacy, and autonomy; the roles of human- and machine-based judgment; and the patient-oncologist relationship.^3-5 Relative to other parts of medicine, the implications of oncology AI are outsized, and some are idiosyncratic. Oncology AI tools apply to not one but two genomes (germline and somatic); can greatly complicate the existing weight of bias, discrimination, and structural racism in cancer care; and can subtly undermine patient and physician autonomy, leading to cancer care that is algorithmic rather than patient-centered. These diverse concerns, in the context of unreserved enthusiasm for AI, challenge a future where oncology AI is both widely implemented and ethically acceptable. We propose that adapting a process-focused approach for deploying AI in cancer care, such as the accountability for reasonableness framework (A4R),⁶ can address these concerns and realize a future where oncology AI is ethically deployed.

Support and Skepticism of Oncology AI

Public support for implementing AI in cancer care is broad but may be overenthusiastic. This can be seen in lay press articles such as “AI Took a Test to Detect Lung Cancer. It Got an A” in The New York Times⁷ and studies demonstrating that a cancer survival prediction tool is among the most anticipated uses of health care AI.⁸ Patients have high levels of interest and trust in using AI to detect breast⁹ and skin cancer.² Oncologists' support for AI, however, appears less robust. Although a Japanese survey found favorable oncologist perceptions of AI's clinical significance,¹ US-based clinicians are unconvinced,³ with two thirds reporting that AI will have limited or no impact in the coming years.¹⁰ Skepticism may vary by the technology's proximity to clinical decisions: a study of UK physicians found that AI-based imaging diagnostics were acceptable to almost twice as many respondents as AI-based clinical management.¹¹ Suspicions also arise when oncologists recall past enthusiasm for promising treatments¹² that have yet to deliver (eg, cancer vaccination)¹³ or led to adverse outcomes (eg, stem-cell transplantation for breast cancer).¹⁴

Positive results from AI tools have been matched by concerning outcomes, such as data from a study assessing cervical cancer treatment recommendation concordance between oncologists and IBM's Watson for Oncology, a clinical decision-support system.¹⁵ Among 300 Chinese patients, 27% of recommendations were discordant, with most disagreements stemming from patient preference. This highlights a central issue in AI development: as models are trained to maximize specific outcomes, such as overall survival, they may disregard patient preferences, and as a result, their autonomy. Not every patient seeks the same goal.^16,17

Ethical Concerns for Oncology AI

Concerns regarding autonomy in cancer care are inter-related with equity and privacy. All three stem from the increasing divide between patients, how their data are used for developing AI tools, and how tools are reapplied during their care. At present, the lack of defined processes for evaluating and disclosing how a tool's data set represents a given cancer population—and what extrapolation is necessary for deriving an outcome or applying it—can make seemingly small biases lead to discrimination during care delivery.^4,18 Given the many inequities of oncology care, there is reason to believe that prejudicial algorithms may exacerbate them.^17,19,20 Indeed, models built on biased data cannot predict accurately,^5,21 but an environment of hype will engender such models' use.²² Alternatively, if the oncology community develops standards and processes for AI's ethical development and monitoring, it can help remediate care inequities rather than extend them.^5,17,23

The ability to create cancer care AI that minimizes bias while maximizing privacy and autonomy rests upon whose data are intentionally included or excluded, when patients consent to data use, and when that use is disclosed. Data selection presents situations where algorithms maximize an outcome beneficial to the majority but not for minoritized groups.^5,24 This is especially important for a field where there are two inter-related genomes, the germline and somatic, and non-Hispanic Whites are over-represented in oncology biobanks.²⁵ These worries are compounded for AI tools used outside clinical interactions, where patients (and sometimes oncologists) may be less likely to recognize care biases or autonomy infringements. Examples include radiology, pathology, and clinical pathway decision-support tools, where few patients are aware of their existence or impact.²⁶

Beyond the bias of data sets, ethical concerns regarding machine-based judgment center on the trustworthiness of black box AI predictions that cannot be entirely understood by researchers, much less oncologists or patients. This issue is one of inexplicability, where stakeholders cannot evaluate how AI tools reach a prediction, such as if a biopsy is cancerous.^2,18,27-29 Some developers have recognized this issue, including post hoc interpretable models that attempt to illuminate their AI's black box.¹⁸ Nonetheless, such attempts are approximations, and this lack of intrinsic interpretability may explain some of the mistrust oncologists have in AI-based predictions. Beneath the hype, these concerns have been insufficiently addressed through an ethical framework.

A4R-OAI: An Ethical Process Framework for Oncology AI

Process-focused ethical decision frameworks are useful structures for addressing challenging issues in biomedicine when it is difficult to reach agreement regarding acceptable outcomes. They are frequently used for priority-setting during scarce resource allocation—such as chemotherapy shortages³⁰—where there are competing ethically acceptable outcomes such as treating the youngest or sickest first. The most widely used process framework is A4R,⁶ which outlines five principles to establish the legitimacy of decisions for stakeholders. They are (1) relevance: decisions should be based on reasons that fair-minded people can agree are relevant under the circumstances; (2) publicity: decisions and their rationales should be publicly accessible; (3) revision: there should be opportunities to revisit and revise decisions and mechanisms to resolve disagreements; (4) empowerment: power differences should be minimized to ensure effective stakeholder participation; and (5) enforcement: there should be voluntary or public regulation to ensure the other conditions are met.

An A4R-based framework for oncology AI is especially useful now, when AI implementation is beginning and ethical domains for the use of AI in oncology have been outlined. A widely cited outline from Australian and New Zealand establishes ethical AI domains as safety, privacy and protection of data, avoidance of bias, transparency and expandability, application of human values, decision making on diagnosis and treatment, teamwork, responsibility for decisions made, and governance.⁴ An A4R-based approach defines how to apply these domains in real-world oncology practice, recognizing that some will need to be prioritized at the expense of others, and there will be disagreements over which should take precedent.

For instance, most agree that AI tools should be transparent, representative of, and proven effective within their target population^4,17,18; however, at what point does transparency encroach too much on privacy and data protection, what constitutes sufficient representativeness, and where should the balance between representativeness and extrapolation lie? The balancing required for developing, validating, and implementing AI tools that are ethically acceptable to stakeholders necessitates an implementation structure. Adapting A4R to Oncology AI (A4R-OAI; Fig 1)³¹ leverages a broadly acceptable approach for introducing individual AI tools and addressing multiple ethical domains.

FIG 1.

A4R-OAI, an oncology AI-adapted accountability for reasonableness process framework. AI, artificial intelligence; FDA, US Food and Drug Administration.

The A4R principle of Relevance in AI requires deciding on the proposed scope of the tool being developed, how each relevant ethical domain applies, which domains may conflict, and which stakeholder groups should be engaged. Relevance implicitly addresses patient and oncologist AI skepticism by including them in the development process, thereby promoting the patient-oncologist relationship and keeping respect for autonomy at the center of clinical decision making.^5,16,17 Patient autonomy is best respected in AI when individual values are built into the decision-making process.^17,20 Relevance upholds this strategy, which MacDougall has termed value-flexible design.¹⁶ By ensuring that respect for patients and their values remains at the center of care, remediating oncology care disparities through AI may also be possible.^4,16,17

Publicity addresses multiple concerns over transparency in AI: data set accessibility, how data are used in development, and how tools are deployed and monitored. The concrete processes of Publicity in A4R consists of developing formal communications plans regarding the who, what, where, when, and why of decisions, and which communication methods are most appropriate. There is ongoing debate about whether transparency can truly exist for predictions that are unexplainable, and to what extent inexplicability, or computational reliabilism,³² is acceptable. Publicity in oncology AI must balance explicability and reliabilism such that both are considered during development and subject to public acceptance during implementation.

A middle ground that may surface, and is balanced across multiple ethical domains, is defining communication strategies specific to each stakeholder group (eg, data scientists, oncologists, and patients). Understanding that clinical oncologists may not have the expertise to explain an AI tool, one team defined and explained the metrics used in their tool's algorithms.³³ Another developed a post hoc interpretation model where the algorithm computes an outcome, an explanation for how it arrived at that outcome for different stakeholder types, and a confidence interval. This respects transparency and the inherit imperfection of prediction while inviting stakeholders to understand some aspects of the decision process.²⁹ On the larger scale, publicity fosters connections to dissemination and implementation science and promotes the role of data scientists in the communication of comprehensible AI to oncologists and patients.

Revision addresses feedback received from publicity while guarding against concerns over emergent biases and new data that may make AI tools discriminatory or irrelevant over time. This portion of the process framework establishes a formal plan for longitudinal data monitoring and decision review to ensure high quality in both how and where the AI is implemented.^18,20 Revision also allows for a rebalancing of priorities if, for instance, acceptance of computational reliabilism changes over time, or new data prompt a potential expansion of the tool into a new population.

Empowerment facilitates effective participation and training of stakeholders during the development and implementation process. Pertinent to all ethical domains and other A4R principles, empowerment ensures minority opinions are included, publicized, and their concerns monitored for, even if final decisions in development or implementation do not reflect their viewpoints. Empowerment is not restricted to those involved in development. It includes empowering oncologists and patients who will use the tool by ensuring accessible communication tools, training for users, and procedures for voicing concerns about the tool after initial implementation. Finally, empowerment involves explicit characterization of power imbalances among the stakeholders involved in tool development, as well as consideration of how the tool may influence power dynamics in the real world. Technologic advancements can accentuate power imbalances between groups,³⁴ and empowerment requires prospective consideration and planning to minimize this effect.

Finally, enforcement in the context of oncology AI development and implementation centers on providing guidance on appropriate use of the AI, supporting patients and oncologists, and ensuring deployment of the above processes. Guidance may include asking oncologists to input information regarding the clinical situation and display a warning if it is different from those in which the tool was developed; supports include educational materials for each stakeholder group. Enforceable policies on AI development are nascent, such as the Food and Drug Administration Action Plan,³⁵ but they represent promising steps that bolster the other A4R principles.

There is increasing support for the implementation of AI into oncology. Although there are profound ethical considerations, particularly with respect to bias, inexplicability, and autonomy, A4R-OAI provides a process by which to address these challenges. Its flexibility allows for evaluation of numerous ethical domains and their greater or lesser applicability to individual AI tools. Its design fosters norms of inclusion, diversity, and accountability that cancer care continues to strive toward. Moreover, it promotes the necessary expansion of cancer health communication strategies to understand AI technology and assists in maintaining the patient-oncologist relationship, specifically leveraging the role of technology creators. If this occurs, AI may be implemented into oncology both ethically and with stakeholder support.

AUTHOR CONTRIBUTIONS

Conception and design: All authors

Administrative support: Gregory A. Abel

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

A Process Framework for Ethically Deploying Artificial Intelligence in Oncology

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).