Implications of An Evolving Regulatory Landscape on the Development of AI and ML in Medicine

Abstract

The rapid advancement of artificial intelligence and machine learning (AI/ML) technologies in healthcare presents significant opportunities for enhancing patient care through innovative diagnostic tools, monitoring systems, and personalized treatment plans. However, these innovative advancements might result in regulatory challenges given recent Supreme Court decisions that impact the authority of regulatory agencies like the Food and Drug Administration (FDA). This paper explores the implications of regulatory uncertainty for the healthcare industry related to balancing innovation in biotechnology and biocomputing with ensuring regulatory uniformity and patient safety. We examine key Supreme Court cases, including Loper Bright Enterprises v. Raimondo, Relentless, Inc. v. Department of Commerce, and Corner Post, Inc. v. Board of Governors of the Federal Reserve System, and their impact on the Chevron doctrine. We also discuss other relevant cases to highlight shifts in judicial approaches to agency deference and regulatory authority that might affect how science is handled in regulatory spaces, including how biocomputing and other health sciences are governed, how scientific facts are applied in policymaking, and how scientific expertise guides decision making. Through a detailed analysis, we assess the potential impact of regulatory uncertainty in healthcare. Additionally, we provide recommendations for the medical community on navigating these challenges.

1. Introduction

The use of artificial technologies (AI) and machine learning (ML) in healthcare has been continuously expanding,¹ paving the way for more personalized, preventative, and innovative treatments that improve patient outcomes in the future.² Its implementation will likely be widespread in various areas of medicine and can be especially beneficial in prevention, therapeutics, and diagnostics.² The relevance of AI/ML advancements in medicine is evident, as seen in the rise of new innovative treatments and diagnostics, such as the use of AI algorithms for diabetic retinopathy screening,³ smart sensors that assist with more accurately estimating the probability of heart attacks,⁴ and imagining systems that use algorithms for diagnostics information for skin cancer in patients.⁵ In addition, the development of Generative AI (GenAI) tools, which leverage ML, has been skyrocketing since the launch of OpenAI’s ChatGPT in 2022. GenAI can create images, videos, and text, and it is expected to revolutionize healthcare, including providing a more patient-tailored approach.⁶ While exciting, these new developments come alongside various ethical and legal issues, many of which, due to their novelty, have yet to be addressed.⁷ This is especially true regarding the regulatory paths for these new developing technologies.^8–9

Upon the emergence of new AI/ML tools in healthcare, there have been clear efforts by the U.S. Food and Drug Administration (FDA), the regulatory agency for various medical products, including medical devices, to create new pathways and expectations for how to bring such tools to market.^10–13 Recent Supreme Court decisions, such as Loper Bright Enterprises v. Raimondo¹⁴ and Relentless, Inc. v. Department of Commerce,¹⁵ have further complicated the process of determining more concrete paths and expectations for these newly developed technologies. These rulings led to the overruling of the Chevron doctrine, which has, in turn, diminished agency deference and set the stage for more litigation and more stringent criteria for regulatory actions. These decisions might have significant implications for the FDA and its current approach to AI/ML technologies in healthcare. Awareness of recent case law can allow those in the medical community to better anticipate expectations surrounding the development and implementation of new technologies.

This Article first discusses the current regulatory framework for medical AI/ML and some of the FDA’s recent initiatives. It then discusses the Chevron doctrine and relevant Supreme Court decisions before the doctrine’s overruling. Next, it analyzes the two Supreme Court cases that led to Chevron’s fall, Loper Bright Enterprises v. Raimondo and Relentless, Inc. v. Department of Commerce. It also discusses a subsequent and relevant Supreme Court case, namely Corner Post.¹⁶ Finally, this Article analyzes how recent judicial decisions might affect regulatory practices and explores strategies for those looking to develop AI/ML in healthcare. In particular, we show that these uncertainties are likely to result in an increase in litigation, along with a need for more explicit rulemaking to constrain interpretations. However, given the speed at which medical AI/ML has advanced, it is important to keep up to date with recent case law and regulatory developments in the field.

2. The FDA and the Regulation of Medical AI/ML

The FDA protects public health by assuring the safety, efficacy, and security of various medical products, including biological products and medical devices.¹⁷ It draws its authority from various statutes, primarily the Federal Food, Drug, and Cosmetic Act (FDCA).¹⁸ Its authority permits the FDA to create legally binding regulations in cases where Congress delegated such power to the agency. In addition, the FDA also publishes guidance documents that reflect the FDA’s thinking on specific topics and developments. While acting through guidance documents (rather than regulations) seemed to be the FDA’s preferred means in recent years, these are not legally binding. Still, guidance documents play an essential role in practice because they reflect the agency’s current interpretation of the respective topics.

The FDCA covers various products, such as food, drugs, medical devices, and cosmetics. The law, initially enacted in 1938, has been amended over 100 times to address new advancements and emerging challenges. Amendments relevant to the medical field specifically, are the Kefauver-Harris Drug Amendments of 1962¹⁹(which required drug manufacturers to provide proof of efficacy and safety before a drug could be marketed and approved by the FDA) and the Medical Device Amendment of 1976²⁰ (which established a risk-based classification system for medical devices and expanded the regulatory authority of the FDA over medical devices). Ultimately, the FDCA regulates three main pathways for medical devices to obtain marketing authorization: premarket notification or 510(k),²¹ premarket approval or PMA,²² and De Novo Classification request.²³ The FDCA does not, however, allow the FDA to develop new pathways for medical devices unilaterally. This limitation on authority can present challenges as new technologies like AI/ML develop that do not fall under classic categories and might require more innovative regulatory pathways.

Additionally, the FDA operates within the Administrative Procedure Act (APA) framework, which essentially governs the processes by which federal agencies may develop and issue binding regulations.²⁴ The purpose of the APA, enacted in 1946, was to promote transparency, public participation, and accountability in the regulatory process. Under the APA, the FDA must follow specific procedures when creating its regulations, such as publishing notices of proposed rulemaking,²⁵ allowing for public comment,²⁶ and providing detailed explanations of the final rules. The procedures and limitations set forth by APA are to ensure that the FDA’s regulatory acts are both well-informed and open to comments. In contrast, unlike formal actions, the FDA can publish guidance documents much easier and faster because they are legally not binding in nature but still have a considerable impact in practice.

The FDA also regulates medical AI/ML-enabled products so long as they are classified as medical devices under FDCA Section 201(h)(1).²⁷ Recently, the FDA has attempted to advance the regulation of medical Al/ML due to its potential for transforming healthcare delivery and improving patient outcomes. The FDA has been proactively attempting to address the regulatory challenges posed by new AI/ML-enabled products through various guidance documents, publications, regulations, plans, and programs. For example, already back in 2019, the FDA published a discussion paper suggesting a new framework for changes to AI/ML-based Software as a Medical Device (SAMD).¹⁰ This was followed by an Action Plan on AI/ML-based SaMD issued in 2021.¹¹ Just in March of 2024, the FDA also released a paper as a complement to its Action Plan outlining its intent to incorporate collaboration between its different departments to better protect public health while still encouraging innovation through AI/ML.^12,13 While unable to issue new authorization pathways on its own, the FDA may still determine how best to combine the pathways it is authorized to use to fit the needs of new and developing products.

Moreover, there have been more legislative actions attempting to proactively address the potential challenges raised by new medical technology such as AI/ML. For instance, the 21^st Century Cures Act,²⁸ signed into law on December 13, 2016, encouraged the innovation and development of medical devices that could be more effective in treating or diagnosing. However, the 21st Century Cures Act also explicitly excluded several categories of software functions from the FDCA definition of a “device,” which also includes certain clinical decision support software functions.²⁹ The FDA has heavily interpreted the meaning of such statute through guidance documents to state its current thinking on that topic.^30–32

3. Chevron and Relevant Supreme Court Decisions Before Its Overruling

3.1. Chevron U.S.A, Inc. v. Natural Resources Defense Council, Inc. (1984)

In 1984 the Supreme Court decided Chevron U.S.A., Inc. v. Natural Resources Defense Council, Inc.,³³ establishing a key principle in administrative law known as the Chevron doctrine. The case involved a challenge to an Environmental Protection Agency (EPA) regulation, with litigants claiming the EPA regulation was inconsistent with the Clean Air Act. The Court upheld the EPA’s interpretation, establishing judicial deference to agency interpretations of ambiguous statutes. The Chevron doctrine involved a two-step process to resolve a challenge to an agency’s actions. First, the court would determine whether Congress had already directly addressed the matter in question. In other words, the court would decide whether the statute contains ambiguity or is silent on the matter, thereby necessitating any interpretation. If Congressional intent was clear and unambiguous, that intent was to be followed without deviation by the agency. If, however, silence or ambiguity necessitated interpretation of the statute on the matter in question, the court’s second step would be to decide whether the agency’s interpretation was based on a permissible construction of the statute. If the interpretation of delegated authority was reasonable, the agency’s expertise was entitled to deference by the court.^{33 at 843−844} Essentially, the Chevron doctrine was the primary means by which courts would determine whether an administrative agency was acting on or exceeding its congressionally delegated authority.

What became known as “Chevron deference” acknowledged that federal agencies possess relevant expertise (such as scientific or technical expertise) critical for proper interpretation and implementation of statutes that the agency is charged by Congress with administering and that courts often lack this expertise.^{33 at 865} The Chevron doctrine reflected and respected a delicate balance of power between the branches of government: its two-step process ensured that Congress established the initial statutory framework for the federal policy, that administrative agencies implemented the statutory framework using the legislative instructions when available and filling in gaps only when it is permissible to do so; and that judicial oversight would ensure such agency interpretations were necessary and reasonable. The Chevron doctrine was heavily relied upon for 40 years, although pushback and critiques (particularly from those seeking to limit the power of federal agencies) caused judicial application of Chevron deference to evolve over time.

3.2. West Virginia v. Environmental Protection Agency (2022)

In 2022 the Supreme Court decided West Virginia v. EPA.³⁴ West Virginia, backed by several other states and industry groups, challenged the EPA’s authority to regulate greenhouse gas emissions via the Clean Power Plan, arguing the EPA overstepped its authority under the Clean Air Act.^{34 at 715} The Supreme Court agreed and held the agency could not make such significant changes to the nation’s policies unilaterally.^{34 at 733} This ruling foreshadowed more recent decisions further limiting agency power.³⁵ The court sidestepped Chevron precedent in this case. Instead, the court diverted attention to a new doctrine it introduced for the first time: the “Major Questions Doctrine.” The Major Questions Doctrine enabled the court to assume broad powers to invalidate agency actions, positing that agencies cannot act on matters of “economic and political significance” unless there is explicit authority from Congress for the agency to take actions of such importance. Considering the FDA’s scope of authority for regulating health technologies and their sizable economic consequences, this doctrine has the potential for courts to ultimately require more explicit congressional authorization for any regulatory actions of importance.

3.3. Securities and Exchange Commission v. Jarkesy (2024)

In June 2024 the Supreme Court decided Securities and Exchange Commission (SEC) v. Jarkesy,³⁶ making it more difficult for federal agencies to impose monetary penalties using administrative processes overseen by administrative law judges (ALJs). The SEC had brought an enforcement action against George Jarkesy for allegedly misleading investors regarding the hedge funds he managed, and the SEC ALJ found Jarkesy liable and imposed sanctions. Jarkesy successfully challenged the decision before the U.S. Court of Appeals for the Fifth Circuit, arguing that the agency’s use of an administrative process (i.e., the ALJ adjudication) violated his constitutional right to a jury trial under the Seventh Amendment.^{36 at 660,678} The Supreme Court agreed. In reaching its decision, the court examined the “Public Rights Doctrine,”^{36 at 660} a doctrine that acknowledges Congress may delegate adjudicative authority to a federal agency in some situations without infringing upon the Seventh Amendment right to a jury trial. Here, under the facts of this case, the court determined that actions regarding fraud and civil penalties are traditionally handled by courts and cannot be assigned by Congress to an agency.

The case has ramifications beyond the SEC. All agencies, including the FDA, might need to reevaluate the types of cases enforced under its administrative adjudication process. Agencies enforcing health fraud and abuse laws could be on particularly shaky ground now.³⁷ The use of ALJs, subject to more stringent constitutional constraints under the Roberts Court, might make agencies more hesitant to rely upon them. The use of ALJs has been an essential component of federal agencies’ regulatory and enforcement powers. These administrative proceedings, initially created to streamline the regulatory process, might now require more detailed consideration to ensure compliance and avoid litigation.

The FDA’s administrative proceedings, such as hearings and product seizures for noncompliance,³⁸ could be scrutinized as overstepping the constraints set by Jarkesy and the Supreme Court’s interpretation of the Public Rights Doctrine. The FDA’s processes for enforcing compliance with its regulations, including issuing fines or sanctions, might be challenged in a similar fashion to that of Jarkesy. Admittedly, though, the FDA’s enforcement actions have predominantly included informal ones such as warning letters. Thus, Jarkesy might even prompt the FDA to continue its path to use those.³⁹

4. Chevron’s Fall and the Corner Post Case

4.1. Loper Bright Enterprises v. Raimondo and Relentless, Inc. v. Department of Commerce (2024)

The groundbreaking 5–4 Supreme Court decision on June 28^th, 2024, in Loper Bright Enterprises v Raimondo and Relentless v. Dep’t of Commerce (two companion cases collectively referred to as “Loper Bright”¹⁴) expressly overruled Chevron, after 40 years of it functioning—as Justice Kagan described in the dissenting opinion joined by Justices Sotomayor and Jackson—“as a cornerstone of administrative law.”^{14 (dissent at 112)} The case involved challenges to the National Marine Fisheries Service’s interpretation of statutory language for its fishing regulations because the agency’s interpretation required clear congressional authorization, which they did not have. The Court, calling Chevron deference “fundamentally misguided”^{14 at 14} and describing it as incompatible with the APA, held specifically that courts “under the APA may not defer to agency interpretation of the law simply because a statute is ambiguous.”^{14 at 62}

The Majority explained their decision by asserting not only that “…agencies have no special competence in resolving statutory ambiguities” but also that “[c]ourts do.”^{14 at 10} Rather than viewing subject matter expertise (including scientific and technical expertise) relevant to an agency’s scope of authority as reason to defer to the agency interpretation so long as it is reasonable, the Court took a dismissive and even hostile⁴¹ view to agencies, explaining that such deference would be an “abdication” of the court’s responsibilities to use “the traditional tools of statutory construction”^{14 at 46} to “determine the best reading of the statute and resolve the ambiguity.”^{14 at 44−45}

The Court noted that the information provided by litigating parties and others through amici briefs would offer sufficient perspective even on technical details and that a court could consider agency interpretations (e.g., as one persuasive but not conclusive interpretation ^{See 41,42}). The Court also claimed that technical expertise could still be considered under Skidmore deference.^{14 at 11} However, regarding deference under Skidmore, a court is not required to follow an agency’s interpretation of a statute but rather has the choice of determining the amount of deference, if any, to give an agency when considering “the thoroughness evident in its [the agency’s] consideration, the validity of its reasoning, its consistency with earlier and later procurements, and all of those factors which give it power to persuade.”^{42 at 11} While the Court in Loper Bright emphasized that Skidmore deference still permits courts to consider agency interpretations, it does so only if those interpretations are persuasive—a standard that is not governed by a strict rule but left to the courts’ discretion on a case-by-case basis. This level of deference provides significantly less binding authority for federal agencies, shifting more interpretative power to the courts while also limiting agencies’ ability to make decisions with the same level of control that they had previously under Chevron.

As noted in the dissent, Chevron had “formed the backdrop against which Congress, courts, and agencies—as well as regulated parties and the public—all have operated for decades. It has been applied in thousands of judicial decisions. It has become part of the warp and woof of modern government, supporting regulatory efforts of all kinds—to name a few, keeping air and water clean, food and drugs safe, and financial markets honest.”^{14 (dissent at 113)} Its overturning, therefore, could result in regulatory uncertainty, and federal agencies will need to relocate the boundaries of agency authority (even boundaries that previously seemed clear and established). Additionally, the decision to overturn Chevron is likely to slow the efficiency of courts and agencies due to an anticipated flood of litigation.⁴⁰

4.2. Corner Post, Inc. v. Board of Governors of the Federal Reserve System

Just days after overturning Chevron in Loper Bright, the Supreme Court decided Corner Post, Inc. v. Board of Governors of the Federal Reserve System¹⁶ and further molded the regulatory landscape by extending the period during which agency regulations can be challenged. The case involved a challenge from a truck stop business to certain regulations imposed by the Federal Reserve Board regarding limits on fees to merchants for debit card purchases. The regulations were promulgated in 2011, and the truck stop (which was established in 2018) brought the challenge in 2021, a decade after the regulations were issued.

The lower courts dismissed the lawsuit due to the expiration of the APA’s 6-year period within which suits against agencies can be filed,⁴³ but the Supreme Court reversed this decision. The Court’s holding rests on conceptualizing the APA’s period as a “statute of limitations”⁴⁴ (which would begin to run when a specific plaintiff has been injured by the agency action) rather than a “statute of repose”⁴⁵ (which would begin to run as soon as an agency’s action occurred, such as issuance of final regulations). The dissent highlighted that this understanding contradicts statutes within the administrative law context^{16 (dissent at 76,78)} and warned that, under this interpretation, potential litigation against agency regulations would never end.^{16 (dissent at 83)} Rather than an agency having some confidence that regulations issued a decade ago would be safe from legal challenges, this decision ultimately renders them vulnerable indefinitely. This decision opens the door for challenges to older regulations, resulting in a significant extension of the timeframe during which regulations can be contested than was previously understood.

5. Impact of the Recent Supreme Court Decisions on Medical AI/ML

Legal scholars have started unpacking the broad impacts of these recent Supreme Court decisions for healthcare generally^37,41 and medical AI/ML specifically.^35,47 Under the leadership of Chief Justice Roberts, the Supreme Court has issued several decisions that are cause for concern for federal oversight of medical AI/ML. Previous works³⁵ have highlighted cases (e.g., TransUnion v. Ramirez⁴⁸ and Dobbs v. Jackson Women’s Health Org.⁴⁹) that frustrate attempts by the Federal Trade Commission (FTC) and other policymakers to promote algorithmic fairness and responsible data practices in the context of digital health. This trend of the Supreme Court issuing “industry friendly” decisions³⁷ that make it more challenging for federal agencies to engage in effective, adaptive governance has continued with the cases summarized here (i.e., Jarkesy, Loper Bright, and Corner Post). Even prior to the 2024 cases, scholars had remarked that the Supreme Court was, in effect, making America “ungovernable”⁵⁰ by weakening the powers of every governmental branch except its own.⁵¹ Some have gone so far as to dub the Roberts Court as the “anti-innovation Court,”⁵² as its holdings would suppress policy innovations many think are essential to impelling responsible conduct in a rapidly changing, AI-enabled world. The increased polarization of public opinion regarding the Supreme Court is likely expected to continue as policymakers debate President Biden’s proposed court reforms.⁵⁴

The recent Supreme Court decisions have strongly signaled a deregulation era is upon us;⁵⁴ however, the impact of these decisions directly on the FDA’s oversight of medical AI/ML remains to be seen. While it is unlikely that the agency will be immune from these headaches of legal uncertainty, the agency’s habit of choosing to govern mainly by informal, non-binding guidance documents (as opposed to formal rulemaking) means that Loper Bright and Corner Post are not directly applicable to them. Governance by guidance, a subject garnering its own distinct criticisms,⁵⁵ means in practical terms that there might rarely be regulatory interpretations of statutes ripe for legal challenges against the FDA.

The FDA has so far mainly published discussion papers and non-binding guidance documents related to AI (as highlighted in Section 2 above).⁵⁶ These informal actions are intended to offer agency flexibility, allowing the FDA to quickly change its current thinking on a topic to keep pace with innovation. Guidance documents can be released through a more direct and efficient process than proposed rules.²⁵ These AI-related documents over the last few years have certainly provided helpful information to manufacturers and other stakeholders on the new regulatory challenges raised by AI, the FDA’s thinking and initiatives on this topic, and likely expectations during premarket reviews of AI/ML-based medical devices.

Even if Loper Bright is not directly applicable to guidance documents, the holding still has important indirect implications for the regulation of medical AI/ML. In particular, the holding might encourage the FDA to continue using primarily informal actions like guidance documents rather than creating legally binding regulations, which are more time-consuming and costly for the agency.⁴⁷ On the flip side, with the Supreme Court’s holding in Loper Bright, this approach will likely receive even more skepticism from courts in the future and might be seen as a potential bypass of formal rulemaking.⁴⁷

But even when creating legally binding regulations regarding AI/ML, the FDA would need to ensure that such regulations would be explicitly defined and authorized by Congress, as the agency’s flexibility to interpret ambiguous statutes has been reduced.⁴⁷ When ambiguities arise, courts can now take the primary role in deciding outcomes. This could be difficult given the in-depth scientific and technological details specific to the areas of AI/ML. The courts can now rule in these highly specialized areas and attempt to piece together the relevant information to make decisions where Congress has not provided unambiguous language (which is often the case). Additionally, decisions on ambiguities by courts might carry the weight of stare decisis. In the jurisdictions bound by these holdings, agencies will not be allowed to enforce regulations that are inconsistent with the interpretation provided by those courts. Moreover, the courts’ need for a comprehensive understanding in anticipation of these rulings—each likely to be in different industries with different facts—will likely slow both the courts and the implementation of new regulations while legal challenges await adjudication.

Moreover, Corner Post poses serious implications for the FDA, FTC, and other federal agencies, with the indefinite timeframe for challenges of agency regulations under the APA, complicating even those areas of regulatory compliance that might have been considered settled, well-established frameworks. Agencies might need to anticipate extended periods of litigation and AI/ML developers might experience ongoing regulatory uncertainty, which could impact market strategies, encourage litigious strategies, and delay innovations. Regulatory uncertainty has the potential to create spaces in which corporations might, for example, be tempted to engage in cost-cutting measures to increase profits but also result in compromised safety and efficacy.³⁷ These conditions also might enable some biotech/biomedical corporations to engage in questionable data practices and increased litigious practices to dominate the market, stifle competition, limit access to alternatives, and increase prices to consumers,³⁷ necessitating more antitrust measures by the FTC.

Congress might also begin to face pressure to revise statutes to make them less ambiguous to ensure their intent is reflected correctly and will likely need to draft and pass more detailed legislation for future issues. This might very well be the case for legislation specific to AI/ML technologies—including those developed for medicine,⁴⁷ in light of AI in clinical applications presumptively considered “rights impacting” and “safety impacting”⁵⁷ and could result in waiting for specific and comprehensive laws governing these technologies.

With existing legal mechanisms available for use in the federal government’s oversight of medical AI/ML in question and new legislation slow to pass, it is incumbent upon the AI/ML community to be more proactive in their commitments to responsible development and use of these technologies. Innovative, integrated policy research is needed, with one clear example being the opportunity for AI/ML developers to pilot the CAITE model.⁵⁸ Moreover, the integrated, holistic training of the medical AI/ML workforce is needed so that researchers and clinical practitioners can anticipate the evolving expectations and constraints in regulations across the AI/ML product (or system) lifecycle. This need for interdisciplinary training—such as that offered within Penn State’s Law, Policy, and Engineering initiative; Arizona State University’s School for the Future of Innovation in Society; or MIT’s Institute for Data, Systems, and Society, for example—is growing more urgent with the tremendous challenges and opportunities on the horizon for digital twins, as noted recently by the National Academies of Science, Engineering, and Medicine.⁵⁹

6. Conclusion

This paper has examined the regulatory challenges and implications of recent Supreme Court decisions on the FDA’s oversight of AI/ML technologies in healthcare. The dual challenge of promoting innovation while safeguarding public health underscores the importance of a balanced and nuanced regulatory framework. Ethical practices, risk management, and proactive compliance might become essential in navigating these uncertainties and ensuring the successful integration of AI/ML technologies into clinical practice. A collaborative approach involving regulators, industry stakeholders, and the biomedical community might become necessary to develop effective strategies for balancing innovation with patient safety and public health protection.