To the Editor,
The rapid advancement of artificial intelligence (AI) is transforming healthcare, with large language models (LLMs) emerging as one of the most promising tools [1]. One of the most influential classes of LLMs is the generative pretrained transformer (GPT) models, such as ChatGPT, which have demonstrated immense potential in various medical scenarios [2]. In May 2024, OpenAI released its latest model, GPT-4o, which boasts more powerful real-time voice interaction capabilities compared to its predecessors. After receiving voice signals, GPT-4o can perform end-to-end voice output with a response latency similar to that of humans, providing new opportunities for seamless human–computer interaction, particularly in the field of clinical immunology and allergy.
The real-time voice interaction function of GPT-4o is expected to enhance the quality and efficiency of medical services in numerous ways. For example, GPT-4o can engage in natural dialogue with patients even before they enter the consultation room. Before meeting the healthcare professional, the AI can take a full allergy history, explore patient ideas, concerns, or expectations, and even offer preliminary counseling. The system can also process all obtained data into a structured format, highlighting key points and offering individualized recommendations for physicians to review. This process can streamline diagnosis and treatment decisions, as GPT-3.5 and GPT-4 have already demonstrated exceptional performance in extracting structured data from clinical records [3]. Furthermore, as previous LLMs have already demonstrated the ability to detect emergency scenarios via voice input [4, 5], GPT-4o could potentially identify critical situations early (such as anaphylaxis, asthma exacerbations, or severe cutaneous adverse reactions) and promptly alert medical personnel even before patients have reached the emergency department, to ensure timely intervention.
AI-based telemedicine has been proven to help improve adherence and disease monitoring [6]. By maintaining regular voice communication, GPT-4o can provide personalized assessments, guidance, and support tailored to their specific conditions. The AI may also be able to infer information from verbal cues or monitor disease based on voice characteristics. For example, one study has shown that machine-learning techniques can accurately predict lung function by recognizing voice features in asthma patients, enabling remote monitoring of asthma [7]. Research has also demonstrated that ChatGPT performs exceptionally well in systematic evaluations of clinical decision-support tasks in allergy and immune diseases [8, 9]. Particularly for patients residing in rural areas or with lower health literacy, voice interaction enables them to more easily articulate their symptoms and concerns to GPT-4o. Moreover, the voice capabilities of GPT-4o bring exceptional promise in regions (such as the Asia Pacific) with a shortage of allergists, providing specialist care for patients with previously limited access to essential allergy services.
However, the clinical application of GPT-4o also faces significant challenges. First, GPT-4o is trained on existing data and may therefore generate inaccurate or outdated information. This could potentially lead to serious consequences in medical decision-making and undermine patient trust. Therefore, the information generated by GPT-4o still requires confirmation by physicians and cannot completely replace human decision-making. Second, the voice interaction between GPT-4o and patients involves voice characteristics and personal privacy, necessitating robust measures for data security and privacy protection. Hence, it is crucial to use GPT-4o appropriately and moderately, ensuring that it can enhance rather than diminish doctor–patient communication. Randomized controlled clinical trials are necessary to comprehensively assess the clinical efficacy, safety, and real-world impact of GPT-4o in everyday practice.
In conclusion, the advent of GPT-4o marks a significant milestone in the evolution of AI-assisted healthcare. With its advanced real-time voice interaction capabilities, GPT-4o has the potential to revolutionize various aspects of clinical practice. As we continue to refine and adapt this powerful tool, it is essential to remain mindful of the ethical, social, and technical challenges involved in its deployment. By proactively addressing these considerations and engaging in multidisciplinary collaboration, we can harness the transformative potential of GPT-4o to improve patient outcomes, expand access to healthcare, and support healthcare providers. Ultimately, the success of GPT-4o will depend on our ability to strike a balance between innovation and responsibility, always keeping the best interests of patients at the forefront and ensuring the highest standards of patient care in allergy and immunology practice.
Conflicts of interest
The authors have no financial conflicts of interest.
Author contributions
Conception of the work, acquisition, analysis, interpretation of data: Qiang Li. Drafting and revision: Qiang Li and Philip H. Li.
References
- 1.Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, Cui C, Corrado G, Thrun S, Dean J. A guide to deep learning in healthcare. Nat Med. 2019;25:24-29. [DOI] [PubMed] [Google Scholar]
- 2.Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29:1930-1940. [DOI] [PubMed] [Google Scholar]
- 3.Huang J, Yang DM, Rong R, Nezafati K, Treager C, Chi Z, Wang S, Cheng X, Guo Y, Klesse LJ, Xiao G, Peterson ED, Zhan X, Xie Y. A critical assessment of using ChatGPT for extracting structured data from clinical notes. NPJ Digit Med. 2024;7:106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Blomberg SN, Christensen HC, Lippert F, Ersbøll AK, Torp-Petersen C, Sayre MR, Kudenchuk PJ, Folke F. Effect of machine learning on dispatcher recognition of out-of-hospital cardiac arrest during calls to emergency medical services: a randomized clinical trial. JAMA Netw Open. 2021;4:e2032320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wenstrup J, Havtorn JD, Borgholt L, Blomberg SN, Maaloe L, Sayre MR, Christensen H, Kruuse C, Christensen HC. A retrospective study on machine learning-assisted stroke recognition for medical helpline calls. NPJ Digit Med. 2023;6:235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ramsey A, Wu AC, Bender BG, Portnoy J. Teleallergy: where have we been and where are we going? J Allergy Clin Immunol Pract. 2023;11:126-131. [DOI] [PubMed] [Google Scholar]
- 7.Alam MZ, Simonetti A, Brillantino R, Tayler N, Grainge C, Siribaddana P, Nouraei SAR, Batchelor J, Rahman MS, Mancuzo EV, Holloway JW, Holloway JA, Rezwan FI. Predicting pulmonary function from the analysis of voice: a machine learning approach. Front Digit Health. 2022;4:750226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Shu L, He Q, Yan B, Wu D, Wang M, Wang C, Zhang L. Human-in-the-loop: human involvement in enhancing medical inquiry performance in large language models. Allergy. 2024;79:1348-1351. [DOI] [PubMed] [Google Scholar]
- 9.Yoshiyasu Y, Wu F, Dhanda AK, Gorelik D, Takashima M, Ahmed OG. GPT-4 accuracy and completeness against international consensus statement on allergy and rhinology: rhinosinusitis. Int Forum Allergy Rhinol. 2023;13:2231-2234. [DOI] [PubMed] [Google Scholar]