Opportunities and challenges of artificial intelligence in the medical field: current application, emerging problems, and problem-solving strategies

Abstract

Recent advancements in the field of artificial intelligence have demonstrated success in a variety of clinical tasks secondary to the development and application of big data, supercomputing, sensor networks, brain science, and other technologies. However, no projects can yet be used on a large scale in real clinical practice because of the lack of standardized processes, lack of ethical and legal supervision, and other issues. We analyzed the existing problems in the field of artificial intelligence and herein propose possible solutions. We call for the establishment of a process framework to ensure the safety and orderly development of artificial intelligence in the medical industry. This will facilitate the design and implementation of artificial intelligence products, promote better management via regulatory authorities, and ensure that reliable and safe artificial intelligence products are selected for application.

Introduction

Artificial intelligence (AI) originated in the United States in 1956,¹ at which time its essence was an algorithm established by analyzing existing data and self-learning. After decades of development, AI has gradually been integrated into daily medical practice and has made considerable progress in medical image processing,^2–7 medical process optimization,^8,9 medical education, and other applications.^6,7,10,11 However, several problems have emerged along with the rapid development of AI technology; therefore, it is not yet widely applied in clinical practice and public health. We searched the PubMed database to identify articles on the topic of AI medical applications, laws, and ethics in the most recent 5 years. In this narrative review article, we discuss the current application of AI in medicine and summarize the existing obstacles preventing AI from universal acceptance. We also explore and discuss potential problem-solving strategies.

Application of AI in the COVID-19 epidemic

In the COVID-19 epidemic at the beginning of 2020, AI algorithms combined with chest computed tomography findings, clinical symptoms, exposure history, and laboratory tests were able to quickly diagnose COVID-19-positive patients. AI systems also improved the reverse-transcriptase polymerase chain reaction viral detection ability in COVID-19-positive patients with normal computed tomography findings.^12–14 Through AI system screening, we could quickly focus on drugs with anti-COVID-19 effects^15,16 and predict the possible sequence of next-generation viruses.¹⁷ Thermal scanners with body and face detection techniques were used to screen people passing through crowded places and identify high temperatures that may be related to COVID-19. Based on individuals’ self-reported health status, travel history, contact history, and other parameters, “health quick response codes” were established by AI analysis to distinguish diagnosed patients, patients with suspected disease, close contacts, and normal residents, thus guiding anti-epidemic policies in various places; this endeavor has effectively slowed the spread of the epidemic.¹⁸ An AI strategy based on an improved susceptible-infected model and self-organizing feature map could summarize the propagation law close to the actual situation and predict the development of the epidemic.^19,20 The use of a low-cost blockchain and AI-coupled self-detection and tracking system helped resource-limited areas carry out effective disease surveillance.²¹

Problems and challenges

The inherent challenges of machine learning, the imperfection of ethics and laws, and the poor acceptance by society have all hindered the development of AI.

Ethical issues

Data ethics

Data ethics is the foundation of AI, and its key areas include informed consent, privacy and data protection, ownership, objectivity, and transparency.²² Can our personal health data be measured by money? Unfortunately, such data transactions are not uncommon; an example is the data exchange cooperation between DeepMind and the Royal Free London Foundation Trust. Who has ownership of such a massive amount of personal health data? Regulations in Canada admit that healthcare providers are the “information custodians” of patients’ private health data, and their ownership belongs to patients. This “guardianship” reflects the reality that there are interests in patients’ medical records, and these interests are protected by law.²³ Therefore, as the owners of data, patients have the right to know how and to what extent their personal health data are recorded and used.²⁴

Fairness

Unfairness caused by bias in data sources is the most common ethical issue. Any data set will be biased to a certain extent based on gender, sexual orientation, race, or sociologic, environmental, or economic factors.²⁴ AI programs are developed based on existing data to learn and give corresponding conclusions. Historical data also capture patterns of inequality in healthcare, and machine learning models trained on these data may perpetuate these inequalities.²⁵ For example, a study conducted in the United States showed that clinicians might have neglected African Americans’ positive results because they assumed that the model’s positive predictive value for African Americans was low. Actually, the low positive rate was caused by the small number of African Americans who participated in the initial experiment, and false-positive results were more likely.²⁶ In addition, injustices may be encountered in the design, deployment, and evaluation of models.²⁵ When providing medical services, clinicians always stop treatment for patients with certain findings (such as extreme preterm birth or brain injury). These differences in personal tendencies are learned by AI and will lead to serious ethical problems that may be fatal for patients.²⁷ In terms of resource allocation, residents in poor communities with long hospital stays due to poverty or long distances will be abandoned by the model. The model may disproportionately allocate case management resources to patients from wealthy communities that are predominantly white.²⁸ The occurrence of automation deviations will also hinder the practical application of AI. This situation is exacerbated by decisions made by AI based on subtle features that humans cannot perceive. In under-resourced populations, the risk of automation bias may be magnified because there is no local expert to veto these results.²⁴

However, it may be difficult for us to know in advance whether an AI system or improvement plan will harm or benefit a particular group.

Ethics of clinical practice

The introduction of AI to healthcare practice brings new challenges to doctors.²⁹ Although a strictly rule-oriented robot may initially seem more reliable, an ethical person is more trustworthy in situations where complex clinical practice decisions are required.³⁰ AI tends to amplify biased findings regardless of the specific clinical interactions. In a practical example, AI classified patients with pneumonia alone as high-risk but erroneously classified patients with pneumonia and asthma as low-risk despite comorbid asthma complicating pneumonia.³¹

Some scholars have considered that when the participants of an algorithm no longer have the ability to predict the machine’s future behavior, they cannot assume moral responsibility.³² This will aggravate the contradiction between doctors and patients, which is not feasible in the field of healthcare. In AI ethics joint statements in Canada, Europe, and North America, doctors played the role of the “guardian of the machine,” serving as an active operator rather than a passive user. Therefore, doctors were responsible for the outcome of the patient’s diagnostic strategy regardless of whether the strategy was partially or fully assisted by the AI system.^23,24,33 In addition, doctors now face new ethical challenges because AI-generated models tend to be difficult to explain, and many doctors lack an understanding of the internal working principles of algorithms.^32,33

Legal issues arising in AI

Healthcare workers undergo strict assessments before they are employed, and they should abide by a series of codes of conduct in daily practice. No globally unified laws or regulations regarding the application of AI in medicine are currently in place to standardize the behavior of practitioners.³⁴ If AI is used by criminals, AI-crime (a new and destructive crime) may occur.³⁵ Therefore, the formulation of broad and detailed AI laws is urgently needed. However, several issues must be considered. First, legal experts alone will not work to formulate such laws. We need the participation of stakeholders who are involved in the construction or development of AI-based medical solutions.³⁶ Second, when encountering AI-related infringement, we should clarify whether the responsibility belongs to the AI manufacturer, user, or maintainer. Where is the boundary of responsibility of each stakeholder? When a complicated case arises, what proportion of responsibility needs to be distributed instead of simply asking doctors to bear all the risks of AI medical treatment?²⁹ Finally, we must continuously upgrade the laws that have been formulated. Studies have shown that health-related data have far exceeded the original expectations of the original privacy protection laws (such as the HIPAA Act of the US Congress in 1996).^37–39 Fortunately, many new laws have been introduced to regulate AI data protection, responsibility determination, and supervision.^37,40,41

Even if a legal basis is established, no clear AI regulatory agency or accountability path is currently available to better regulate AI. For example, the NHS 111 powered by Babylon, a children’s intelligent inquiry app, was recognized as a medical device by the Medicines and Healthcare products Regulatory Agency despite the absence of rigorous clinical verification and sufficient evidence.⁴²

Security

Security is the most important issue in the application of AI to the medical industry, and it also requires the most rigorous review.

Hardware security

All AI products currently require a series of electronic products to perform their functions, such as computers, mobile phones, and bracelets. Three key issues regarding the security of such hardware must be noted. First, even the best physically unclonable functions will be affected by factors such as cost, temperature variations, and electromagnetic interference.⁴³ Second, the complexity and professionalism of medical knowledge and information technology makes it difficult for doctors or engineers to use AI that integrates multiple technologies. On one hand, engineers must be retrained to access and process medical system data, which may disrupt the medical workflow and cause data leakage.⁴⁴ On the other hand, doctors may have a poor understanding of the principles and usage methods of AI products in real practice, causing problems such as reduced efficiency and increased errors. Third, the issue of AI network security must be addressed. A global cascading reaction may occur if key nodes are attacked or fail in the complex network transmission process.^45,46

Software security

Even algorithm programs with powerful functions are very vulnerable under design attacks.^47–50 The performance of the AI system is often unsatisfactory in a targeted design confrontation despite its performance being excellent in the initial design inspection. In fact, all stages of the AI algorithm formation process will be attacked, assuming that the attacker knows everything related to the trained neural network model (training data, model architecture, hyperparameters, number of layers, activation function, and model weights).⁵⁰ A false-positive attack can be used to generate a negative sample or a false-negative attack can be used to generate a positive sample, causing confusion in the system classification. Attacks can even be made without awareness of the structure and parameters of the target model or the training data set.^50,51 Errors will also occur in the system without external interference. The original algorithm will gradually deviate from the correct direction because of changes in disease patterns,⁵² missing data,⁵³ and autonomous update errors.⁵⁴

Human factors

Today’s AI is realized by software codes. When dealing with thousands of codes, engineers will inevitably make mistakes. An AI system can be improved through subsequent patches and updates. However, in the AI programs used in the medical field, such errors may directly endanger the health of patients. The end point to which developers pay attention is often the effectiveness of the AI system, not the security.⁵⁵ Doctors can also make mistakes, and when the usually predictable AI system is consistent with their diagnosis, busy clinicians no longer consider other alternatives.^56,57

Protection of human genetic resources

All DNA sequences of human chromosomes have been sequenced to complete the mapping of the human genome, and a shared information system has been established. This information system focuses on the relationship between function and genes, especially the relationship between genes and diseases.⁵⁸ Scientists have mapped the intact human genome during the past three decades, facilitating much more detailed research of clinical diseases at the genetic and cellular levels. On one hand, human genetic resources are of great help in medical diagnosis and treatment⁵⁹; on the other hand, however, such resources have catastrophic potential for all human beings if illegally applied. The Ministry of National Defense of the People’s Republic of China has reported a biologic weapon called the “gene weapon,” which consists of bacteria, insects, or microorganisms containing disease-causing genes through genetic engineering modifications with devastating effects in warfare. Genetic mutations in specific ethnic groups could be designed through the study of genetic characteristics; therefore, the protection of human genome resources is of vital importance. Additionally, with the development of the Internet and gene testing technology, increasing numbers of patients are undergoing genetic testing to assist in diagnosis and treatment, and the testing results are being recorded in hospitals or testing organizations. This introduces a risk of data exposure among inappropriate personnel and may also induce discrimination in insurance or employment. Thus, the legal provisions protecting the privacy of patients in DNA collection, transmission, and storage should be improved.

Societal acceptance

Although most patients show a willingness to believe in an AI-based diagnosis, they tend to more frequently trust doctors when the AI-based diagnosis differs from that of doctors. One survey showed that resident physicians and medical students desired AI-related training but that only a few had opportunities to participate in personal-level data science or machine learning courses.⁶⁰ Moreover, medical workers in less developed areas have great concerns about whether they will be replaced by AI in the future.⁶¹

Possible solutions

AI-based ecological network

The progression from the initial acquisition of complex clinical data or clinical phenomena to the birth of AI programs for clinical application is complex. We herein propose an AI-based ecological network and connect the entire network through a public big data sample database. The framework describes in detail the various links generated by AI and the issues needing attention. It can be divided into the following three steps.

Step 1. Preclinical (Figure 1)

Figure 1.

Artificial intelligence before clinical application

The first step is to identify valuable problems from clinical work or the public big data sample database. A preliminary algorithm is then formed through data collection and technology development, and a specially designed attack program is used to test the algorithm. Those algorithms that survive the test will be simulated and applied in the public big data sample library to screen out truly safe and stable AI programs.

Step 2. Clinical application (Figure 2)

Figure 2.

Application of artificial intelligence in clinical practice

Small-scale clinical application can be applied after a series of review procedures. The problems and experience in the process of use and optimization of the algorithm are summarized, and large-scale clinical application is subsequently conducted. Finally, an AI program suitable for clinical application in the real world is formed.

Step 3. Construction of public big data sample database (Figure 3)

Figure 3.

Maintenance and application of database

We use blockchains, big data, and other technologies to integrate high-quality data sets and store them in secure data storage software. The big data sample database requires daily maintenance (including maintenance of its data storage status, data security, and other features) and regular updates. The AI program is verified again through the updated database. If the program fails to meet the corresponding standards or is problematic, it needs to be re-debugged. Additionally, a special supervisory department is needed to supervise the whole process and provide corresponding complaint channels to solve problems in a timely manner.

Ethical adoption of AI in medicine

The development of AI requires ethicists to participate in the supervision of the whole process to solve ethical issues in data, resource allocation, and practice. First, we must ensure the independence of ethics committee.⁶² Next, with respect to the issue of data ethics, we can refer to the “land” policy, which states that patients have ownership of medical data just as landowners have surface rights. However, the right to access the data for the purpose of improving healthcare can be considered to belong to other parties, such as healthcare providers or the government.²³ To ensure fairness, we must fully consider marginalized populations and the fairness of distribution in specific clinical and organizational environments. We must ensure the “three fairs”: equal outcomes, equal performance, and equal allocation.²⁵ In terms of practice, we believe that it is necessary for ethics committees to formulate uniform rules, standards, and codes of conduct that must be agreed upon and continuously updated to ensure that the development of AI in medical care does not violate ethics.

Establishment and perfection of the legal system

A complete legal system must be developed. Based on the “no harm” principle, strict and prudent rules are formulated for every step of AI from the laboratory to clinical application. The legal system must be flexible; i.e., it should not restrict the development of AI. Instead, the development of AI must be guided by established regulations. Laws must keep pace with the era and cannot be set in stone. Laws and regulations based on the level of AI technology at former stages are bound to fail to meet the needs of supervision with the advancement of technology. Thus, we need to re-study the current regulations, eliminate any emerging legal ambiguities or inadequacies in a timely manner, and improve the legal terms.

Notably, however, it is doctors who are responsible for medical decisions. Doctors should not blindly accept the information given by AI but should instead remain skeptical and formulate the best plan for patients based on the actual situation.

Improvement in feasibility of AI in the real world

Individuals and communities must be provided more powerful AI products with convenient usage flow, reliable results, and stable performance. Production can then be improved according to the shortcomings reflected by the users. Hence, people’s exposure to AI should be increased, such as through offline experience stores in conjunction with social media, live broadcast platforms, and other methods.

Above all, patients and medical staff can truly benefit from AI products in terms of decreasing expenses, saving time, and reducing errors and conflicts. We believe that after a period of hard work, increasingly more people will accept AI products in the medical industry.

Conclusion

AI will not replace doctors. Just as biochemical analyzers do not replace laboratory scientists, the application of AI is not a threat to doctors. On the contrary, it will promote reshaping of the doctor’s role. AI research should not be limited to the accuracy and sensitivity of the report but should focus on the nature of diseases, such as their etiology and pathogenesis, and should enrich our understanding and knowledge of biology.⁶³ Interpretable algorithms will be recognized by more people and will bring AI-based medical treatment to people’s lives. We must improve the research on the relevant ethics, laws, and supervision of AI as soon as possible. Additionally, we need to build a large public database containing information such as human genome data in conjunction with strict security protection measures, regular upgrades, and maintenance.

The era of AI has arrived, and all walks of life are stepping up to integrate with it; medical treatment is no exception. In the near future, AI-assisted medicine will experience a qualitative leap, and people’s understanding and acceptance of medical AI will increase.