Skip to main content
NCBI home page
Future Healthcare Journal logoLink to Future Healthcare Journal
. 2024 Sep 19;11(3):100178. doi: 10.1016/j.fhj.2024.100178

The FHJ debate: Will artificial intelligence replace clinical decision making within our lifetimes?

Joshua Hatherley a,1,⁎, Anne Kinderlerer b,1, Jens Christian Bjerring a, Lauritz Aastrup Munch a, Lynsey Threlfall c
PMCID: PMC11452837  PMID: 39371529

Proposition: AI systems will replace clinical decision making within our lifetimes

Joshua Hatherley, Jens Christian Bjerring and Lauritz Aastrup Munch

AI systems could replace clinical decision making in two ways. First, from the ‘top down’, by way of hospitals and healthcare organisations. Second, from the ‘bottom up’. by way of patients and patient-facing AI systems. In what follows, we argue that AI systems are likely to replace clinical decision making within our lifetimes – in both a top-down and bottom-up fashion – for several institutional, economic and psychological reasons.

Top-down

Top-down replacement occurs when hospitals and healthcare organisations reduce the degree of human involvement, authority or discretion in clinical decision making tasks to the extent that human judgement becomes subordinate to, or replaced with, AI-driven reasoning. It is likely to occur within our lifetimes for three reasons.

First, hospitals and healthcare organisations are facing increasing pressure to increase productivity and economic efficiency to address both national and global health challenges, including population growth and increasing rates of chronic illness. Purchasing and maintaining AI systems will often be cheaper than ongoing employment for human professionals since AI systems can process clinical cases far more swiftly and efficiently than human clinicians,1 and they can do so without becoming fatigued or requiring breaks for rest and relaxation. Moreover, AI systems cannot unionise or demand better working conditions. In a time where hospitals are increasingly squeezed for time and money, many are therefore likely to prioritise investing in technology over investing in humans.

Second, as AI becomes increasingly accepted in medicine, top-down replacement provides hospitals, healthcare organisations and AI organisations with a compelling opportunity to maximise their return on investment. Many AI systems have demonstrated impressive performance in a wide range of clinical tasks.2,3 In some cases, AI systems even appear to match or exceed the performance of human clinicians.4,5 However, the quality of data contained in repositories used to train medical AI systems – such as electronic health records – is highly inconsistent.6 Primarily, this is due to errors, inconsistencies and oversights in clinicians’ data-recording practices. By reducing human involvement and authority over the clinical decision-making process, healthcare organisations and AI developers can improve the quality of these datasets by standardising data collection processes, thereby boosting the overall performance of medical AI systems.

Third, AI and healthcare organisations are strongly incentivised to expand their power and prestige,7,8 and top-down replacement is a compelling strategy for achieving this. For instance, by reducing human involvement in clinical decision making, AI organisations can increase their capacity to monitor, manipulate and control patients’ behaviour for economic gain, otherwise known as their ‘instrumentarian’ power.9 Moreover, new medical technologies increase the power and prestige of institutions that choose to adopt them,10,11 and medical AI systems are currently subject to an enormous degree of hype and attention. Pundits anticipate that medical AI will deliver a range of benefits that include improved patient health and wellbeing, enhanced health equity, and increased patient safety.3,12,13. Regulatory approvals of medical AI systems in the USA have also rapidly accelerated since around 201814, and the McKinsey Institute has even estimated that AI technologies will generate between US$200–$300 billion of value in the healthcare sector.15

Of course, professional and regulatory bodies are likely to persist in their recent attempts to restrict AI systems to a merely supportive role in medicine.16 Ultimately, however, this project is unlikely to succeed, given that AI systems are likely to increasingly outperform human clinicians in the near to mid-term future. Moreover, effectively combining human reasoning with AI reasoning is notoriously difficult due to a broad range of human factors challenges including automation bias, algorithmic aversion and automation complacency.17, 18, 19 Eventually, therefore, hospitals and healthcare organisations are likely to pursue top-down replacement to overcome or avoid such challenges.

Bottom-up

Bottom-up replacement occurs when patients themselves minimise the degree of human involvement in clinical decision making by turning increasingly to AI systems, rather than human clinicians, for medical advice. It is likely to occur during our lifetimes for three reasons.

First, healthcare is becoming increasingly expensive and unaffordable, particularly for patients from low- and middle-income backgrounds, and patients in countries without universal health coverage. Citizens worldwide are also facing increasing financial pressures due to inflation, stagnant wages and the economic fallout of COVID-19. By seeking out medical advice from AI systems rather than human clinicians, patients can substantially reduce the financial pressures associated with healthcare. Indeed, many patients already report self-diagnosing with large language models such as ChatGPT.20,21

Second, patients continue to be disempowered and disconnected from their own medical care and treatment due to imbalanced power dynamics in clinician–patient relationships.22 However, bottom-up replacement provides patients with an opportunity for greater empowerment and control in healthcare. AI systems, in combination with other technologies such as wearables, are likely to enable patients to manage many aspects of their health independently of human clinicians and reduce patients’ reliance on frequent visits to human clinicians for routine check-ups and in-person consultations.23 Indeed, the trend towards self-monitoring with wearables is already strong, and it is likely to be accelerated by AI.24 Additionally, the data collected by wearables can be shared with other AI systems – as well as human practitioners, of course – which is likely to improve the quality of AI-assisted care and decision making over time. Relatedly, since AI systems can analyse and remember vast amounts of data, the level of personalisation that these systems can achieve is likely to exceed what busy human clinicians can provide in the near- to mid-term future.

Third, many patients will likely even come to prefer receiving healthcare services from AI systems rather than human clinicians. AI systems can simulate care, and can in some ways do so better than human clinicians.25 Empirical studies have already shown that users are more comfortable revealing sensitive or potentially embarrassing information to machines rather than human professionals.26,27 Patients have also been found to interpret text that is generated by large language models as more empathetic and caring than text written by human clinicians.28 While medical AI systems cannot actually care for or empathise with patients – at least, not for the foreseeable future – the strong tendency for humans to anthropomorphise AI systems suggests that this does not matter. As we have already seen with therapeutic robots like Paro, patients need not actually be cared for in order to feel cared for.29

Objections

Our opponents raise several objections against the preceding arguments. First, they suggest that AI systems are unlikely to achieve the degree of safety or flexibility needed to replace all parts of the clinical decision-making process. One key reason for this is that these systems cannot incorporate contextual information or information about patients’ values and preferences into their reasoning processes. Another reason is that clinical decision making goes beyond simple pattern recognition; it involves genuinely engaging with patients in a collaborative decision-making process that AI systems cannot currently replicate.

However, many of these limitations are likely to be addressed as the technology improves. Indeed, multimodal systems are already advancing to learn from facial expressions, tones of voice, and more.30 AI systems are increasingly being designed to incorporate patient-specific data, including medical history, genetic information and lifestyle factors, in their reasoning.31 Patient interactions with AI systems could be structured to allow for input of preferences and values, which can then be factored into the AI's decision-making process.32 Moreover, the objection relies on an overly stringent definition of ‘replacement’ in which human clinicians no longer play any role in clinical decision making. We agree that humans may continue to play a role in clinical decision making, but we suggest that their authority and responsibilities will diminish with the progression of AI to the point that human clinicians are at least subordinated to these systems. Despite any continued involvement of human clinicians, therefore, it will ultimately be the AI systems that are in command.

Second, our opponents highlight that human–AI teams appear to perform better than AI systems or humans alone. Relatedly, they argue that recent performance estimates of AI systems are likely exaggerated in practice due to human factors challenges such as human error, algorithmic bias and automation complacency. We agree that clinician–AI teams may reliably outperform medical AI systems operating alone. We also agree that current performance estimates for medical AI systems may be inflated due to several human factors challenges. As noted above, however, medical AI systems need not be completely autonomous to effectively replace human clinicians in clinical decision making, and we suggest that the balance of authority in clinician–AI teams will increasingly shift toward AI systems. Indeed, one reason that hospitals and healthcare organisations will be incentivised to pursue a minotaur-based approach to clinician–AI teams is precisely to avoid these human factors challenges, as we also emphasised earlier.

Finally, our opponents argue that top-down and bottom-up replacement are unlikely to occur because clinicians and patients are unlikely to trust AI systems that are known to contain biases, generate inexplicable predictions, generate ‘bullshit’ outputs and hallucinate information. Yet, these weaknesses in medical AI systems are gradually being addressed through technical methods (eg explainability methods, improved data curation and algorithmic auditing), and their associated risks are likely to diminish as the technology improves. The integration of AI into clinical decision making will not result in overnight replacement, but is rather a gradual, iterative process. As AI systems demonstrate increased accuracy and reliability over time, both clinicians and patients will likely build more trust in these technologies.

Ultimately, therefore, we find these objections serious, but unconvincing.

Conclusion

AI systems are likely to replace clinical decision making within our lifetimes, both from the top down and the bottom up. This is not an inevitable future, nor is it necessarily a desirable one. At present, however, without serious and radical change to our health systems and technological governance and regulations, it is likely to materialise in response to a variety of institutional, economic and psychological challenges that currently affect patients, hospitals and healthcare systems at large.

Opposition: AI will not replace clinical decision making in our lifetime

Anne Kinderlerer and Lynsey Threlfall

To answer the question, we will first define clinical decision making. Is it merely the process by which the clinician sifts available data and arrives at a diagnosis and a treatment which they communicate to the patient? Given the exponential growth in medical data – the NHS alone generates billions of clinical data points a week33 – it is easy to argue that at some point humans will be unable make decisions, as the human brain will be unable to assimilate the volume of data.34

We argue, however, that the task of clinical decision making is not merely pattern recognition based on available data, but that it encompasses the process in which patient and clinician arrive at a shared model of pathology and a treatment plan.

This is a fundamentally human method of care, as central to the clinician’s art as history taking, physical examination and patient education.35 Computer-based clinical decision support systems (CDSSs) cannot yet follow the multi-step process to collect, contextualise and integrate the clinical data and the biopsychosocial context of the individual patient so central to beneficial outcomes.36

CDSSs are ‘active knowledge systems which use two or more items of patient data to generate case-specific advice in more or less real time’.37 The proposers argue that CDSSs are expected to improve and ultimately replace clinical decision making by making it faster, cheaper and less prone to error. Certainly, current algorithms can process large volume data more swiftly than humans, but several studies have demonstrated that the most effective results come from AI performing the initial screen with humans reviewing the more complex cases.38,39 AI has been shown to have impressive results matching or improving upon clinicians in areas where there are clear guidelines and widely accepted evidence.40, 41, 42 However, Hager et al suggest that large language models (LLMs) cannot apply that knowledge to more realistic clinical scenarios encompassing the complexity of patients in the real world,43 contrasting with the performance on medical exams cited by the proposers. In an increasingly ageing population with multiple long-term conditions, it is difficult to see AI replacing humans.

There is reason to think that the difference in the way that AI models and human beings ‘think’ raises questions about the ability of such systems to operate autonomously. Data-driven CDSSs using big data sets generate conclusions that are not easy to explain.44 And if, as the proposers celebrate, we develop ‘AI that begets AI’, then the algorithms will become increasingly complex, further worsening the problem of explainability.45 There are already concerns about the black-box nature of existing decision support systems.46,47 This lack of explainability affects the trust of both clinicians and patients in clinical decisions. Lack of trust affects the likelihood of the patient accepting decisions about treatment, and potentially infringes their autonomy to make choices.48 Furthermore, this lack of explainability means that it is more difficult to identify bias in the outcomes of AI decision making. Bias is arguably more and not less likely in these systems.49 Just as our concepts of disease and decisions on treatment are value-laden and culturally specific,50 data used to train LLM encode individual and societal values, which may differ from those of the physician and patient in the room and which will not necessarily be transparent.51

As LLMs interact with the world primarily through language, as humans do, the hope is that they are poised to be the point of access to the AI solutions of the future. However, there are moral and practical complexities with the way they function. It is worth reflecting on the relationship of the output of an LLM, such as ChatGPT, to truth. They are designed with no reference to truth, but only to the plausibility of the next word in the sequence: to be plausible rather than correct. There is no difference in model function between ‘truthful’ outputs of the LLM and times when it ‘hallucinates’. In this sense the overall activity of LLM, is better understood as bullshit in the sense explored by Frankfurt.52 It is therefore not surprising that current LLMs are hasty and unsafe clinical decision makers.43

Wider society is likely to reject potentially unsafe algorithms where bias is both baked in and undetectable, and such systems are unlikely to add to the prestige of organisations that deploy them.

AI is a disruptive technology and, as with all disruptive technologies, we agree with the proposers that it will democratise healthcare and give patients options to manage their healthcare better and more cheaply.53 Wearable technologies are improving continuously, and AI algorithms can already diagnose hypertension, left ventricular disfunction and sleep apnoea using data from our smart watches.54,55 This will change the conversations that patients are able to have with their human clinicians. Studies indicate that most patients are willing to use AI for self-diagnosis,56 as they already do with search engines. But as we have argued, AI cannot integrate patient preferences, laboratory data, response to treatment and guidelines into a personalised treatment plan.

Even if, as the proposers of the motion argue, it were possible to get beyond these fundamental problems in design that limit the building of an autonomous CDSS, experience of implementing technology into complex health systems does not suggest that it will be possible to build widespread safe systems in our lifetime.57 This will limit the ability of healthcare organisations to impose such systems in order to save costs. Roll out of AI in place of humans in a ‘top-down’ replacement strategy requires significant ‘fixing’ of the underlying technological infrastructure. Health data to train algorithms and to deploy risk-based algorithms are siloed and poorly curated.58 Autonomous CDSS will need to be integrated into existing digital systems, which currently exist as multiple subsystems, increasing the difficulty of successful implementation with workable processes.59 Such implementation will require large upfront costs, not only to fix the technological issues but also in the human expertise required for deployment. Large-scale transformative digital projects have a history of failure, particularly in healthcare.60 Indeed, its complex nature makes the implementation of rule-based algorithms without human oversight even more difficult.61

The proposers argue for significant benefits of AI, that it is faster, cheaper, more accurate, better at learning and preferred by patients. However, to achieve these outcomes the technology must be deployed correctly. The idealised performance promised by the advocates of innovation doesn’t always marry up with the complex socio-technical interactions seen in the real world,62,63 as we have seen in the well-publicised failure of the autonomous computer system designed to operate without the possibility of human correction in the Post Office in the UK.64 We can already see that well-validated AI predictive models that appear accurate in development are increasingly found to be ineffective or unsafe when deployed in clinical pathways in real healthcare systems.65

In a future where clinical decision making is replaced by AI, who is creating and inputting the data? If humans are involved, there will still be workarounds, whether these are clinician or patient generated, and these will compromise the data integrity.66,67 The consequences of low-quality data in a sophisticated algorithm, when a human is merely observing the loop and not sense-checking, is that whole populations may be misdiagnosed or mistreated. Workarounds are prevalent in all industries wherever a human interacts with a machine; lack of high-quality training is often felt to play a part alongside a lack of understanding about the capability of the software.68 Within areas of high social deprivation, for whom the proposers argue there may be the greatest benefits, we often have adult populations with a reading age of 8–9 years old, populations for whom English may not be their first language, or who have additional learning needs. How can we ensure that these populations receive adequate training and education to interact in the desired way with AIs? Given that these programmes will be trained on data predominately from developed English-speaking countries, there is the risk of widening global health inequalities rather than, as the proposers argue, reducing them. There is significant risk of creating new hazards in systems without controls and checks able to identify the risk.

Conclusions

Healthcare and its delivery are a ‘wicked’ problem – difficult to resolve because they are difficult to define – and it is easy to oversimplify such issues in the quest for ideal solutions.69 The proposers argue that the takeover of AI is inevitable, if not necessarily desirable, driven by social, economic and psychological imperatives.

We should not blindly walk toward a future where AI supersedes humans. Modern history is littered with examples of grandiose IT schemes that ultimately fail to deliver benefits. The autonomy of patients and clinicians is precious; let us not carelessly cast it aside. Fundamentally, do you want a ‘bullshit machine’ to oversee your healthcare?

What do you think? Vote at https://forms.office.com/e/eeHsZfa6vz until 18 November 2024. Do you like the debate feature? Send us your thoughts to fhj@rcp.ac.uk.

CRediT authorship contribution statement

Joshua Hatherley: Conceptualization, Writing – original draft, Writing – review & editing. Anne Kinderlerer: Writing – original draft, Conceptualization. Jens Christian Bjerring: Conceptualization, Funding acquisition, Writing – review & editing. Lauritz Aastrup Munch: Conceptualization. Lynsey Threlfall: Writing – review & editing.

Declaration of competing interest

All authors declare no competing interests.

Acknowledgements

JH, JCB, and LAM were supported a Carlsberg Foundation Young Researcher Fellowship (grant number CF20-0257).

Footnotes

This article reflects the opinions of the author(s) and should not be taken to represent the policy of the Royal College of Physicians unless specifically stated.

References

  • 1.Mainz J., Aastrup Munch L., Bjerring J.C., Godtfredsen S. Why algorithmic speed can be more important than algorithmic accuracy. Clin Ethic. 2023;18(2):161–164. [Google Scholar]
  • 2.Esteva, Andre A.R., Ramsundar B., et al. A guide to deep learning in healthcare. Nat Med. 2019;25(1):24–29. doi: 10.1038/s41591-018-0316-z. [DOI] [PubMed] [Google Scholar]
  • 3.Rajpurkar, Pranav E.C., Banerjee O., Topol E.J. AI in health and medicine. Nat Med. 2022;28(1):31–38. doi: 10.1038/s41591-021-01614-0. [DOI] [PubMed] [Google Scholar]
  • 4.Liu X., Faes L., Kale A.U., et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. 2019;1(6):e271–e297. doi: 10.1016/S2589-7500(19)30123-2. [DOI] [PubMed] [Google Scholar]
  • 5.Agarwal, Nikhil R.H., Moehring A., Rajpurkar P., Salz T., Yu F. Comparative advantage of humans versus AI in the long tail. AEA Paper Proceed. 2024;114:618–622. [Google Scholar]
  • 6.Cios, Krzysztof J., William Moore G. Uniqueness of medical data mining. Artifi Intelli Med. 2002;26(1–2):1–24. doi: 10.1016/s0933-3657(02)00049-0. [DOI] [PubMed] [Google Scholar]
  • 7.Cua, Santino S.M.-B., White S. Reputation and the best hospital rankings: what does it really mean? Am J Med Qual. 2017;32(6):632–637. doi: 10.1177/1062860617691843. [DOI] [PubMed] [Google Scholar]
  • 8.Morales-Burton V., Lopez-Ramirez S.A. Rethinking healthcare quality and prestige: is this a manager's number one problem? Front Public Health. 2022;10(863):383. doi: 10.3389/fpubh.2022.863383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Zuboff S. Profile Books; London UK: 2019. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. [Google Scholar]
  • 10.Leff B., Finucane T.E. Gizmo idolatry. JAMA. 2008;299(15):1830–1832. doi: 10.1001/jama.299.15.1830. [DOI] [PubMed] [Google Scholar]
  • 11.Sharkey N., Sharkey A. Robotic surgery: on the cutting edge of ethics. Computer. 2012;46(1):56–64. [Google Scholar]
  • 12.Topol, Eric J. Basic Books; New York NY: 2019. Deep Medicine: How Artificial Intelligence can Make Healthcare Human Again. [Google Scholar]
  • 13.Sparrow R., Hatherley J. The promise and perils of AI in medicine. Int J Chin Comparat Philosoph Med. 2019;17(2):79–109. [Google Scholar]
  • 14.Lyell, David E.C., Chen J., Shah P., Magrabi F. How machine learning is embedded to support clinician decision making: an analysis of FDA-approved medical devices. BMJ Health Care Informat. 2021;28(1) doi: 10.1136/bmjhci-2020-100301. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Chui M., Manyika J., Miremadi M., Henke N., Chung R., Nel P., Malhotra S. McKinsey Global Institute; New York NY: 2018. Notes from the AI Frontier: Insights from Hundreds of Use Cases. [Google Scholar]
  • 16.American Medical Association . American Medical Association; Chicago IL: 2024. Future of Health: The Emerging Landscape of Augmented Intelligence in Medicine. [Google Scholar]
  • 17.Burton J.W., Stein M.-K., Jensen T.B. A systematic review of algorithm aversion in augmented decision making. J Behav Decisi Mak. 2020;33(2):220–239. [Google Scholar]
  • 18.Gerke S., Babic B., Evgeniou T., Glenn Cohen I. The need for a system view to regulate artificial intelligence/machine learning-based software as medical device. npj Digit Med. 2020;3(1):53. doi: 10.1038/s41746-020-0262-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Sujan, Mark D.F., Grundy K., et al. Human factors challenges for the safe use of artificial intelligence in patient care. BMJ Health Care Informat. 2019;26(1) doi: 10.1136/bmjhci-2019-100081. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Fan, Xiangmin D.C., Zhang Z., Wang D., Li X., Tian F. Utilization of self-diagnosis health chatbots in real-world settings: case study. J Med Internet Res. 2021;23(1):e19928. doi: 10.2196/19928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Shahsavar Y., Choudhury A. User intentions to use ChatGPT for selfdiagnosis and health-related purposes: cross-sectional survey study. JMIR Hum Fact. 2023;10(1):e47564. doi: 10.2196/47564. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Scholz, Brett J.B., Platania-Phung C., Banfield M., Happell B. Not an afterthought”: power imbalances in systemic partnerships between health service providers and consumers in a hospital setting. Health Policy. 2018;122(8):922–928. doi: 10.1016/j.healthpol.2018.06.007. Scholz, B., J. Bocking, C. Platania-Phung, M. Banfield, and B. Happell. 2018. ‘Not an afterthought’: power imbalances in systemic partnerships between health service providers and consumers in a hospital setting. Health Policy 122(8): 922–928. [DOI] [PubMed] [Google Scholar]
  • 23.Kang, Singh H., Exworthy M. Wearing the future—wearables to empower users to take greater responsibility for their health and care: scoping review. JMIR mHealth uHealth. 2022;10(7):e35684. doi: 10.2196/35684. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Heyen N.B. From self-tracking to self-expertise: the production of self-related knowledge by doing personal science. Public Understand Sci. 2020;29(2):124–138. doi: 10.1177/0963662519888757. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Inzlicht M., Cameron C.D., D'Cruz J., Bloom P. In praise of empathic AI. Trend Cognit Sci. 2023;28(2):89–91. doi: 10.1016/j.tics.2023.12.003. [DOI] [PubMed] [Google Scholar]
  • 26.Lucas, Gale M., Gratch J., King A., Morency L.-P. It's only a computer: virtual humans increase willingness to disclose. Comput Hum Behav. 2014;37:94–100. [Google Scholar]
  • 27.Ho A., Hancock J., Miner A.S. Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. J Commun. 2018;68(4):712–733. doi: 10.1093/joc/jqy026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Ayers J.W., Poliak A., Dredze M., et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Internal Med. 2023;183(6):589–596. doi: 10.1001/jamainternmed.2023.1838. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Wang, Xinxia J.S., Chen Q. How PARO can help older people in elderly care facilities: a systematic review of RCT. Int J Nurs Knowl. 2022;33(1):29–39. doi: 10.1111/2047-3095.12327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Turaev, Sherzod S.A.-D., Babu A., et al. Review and analysis of patients’ body language from an artificial intelligence perspective. IEEE Access. 2023;11:173. 62,140–62. [Google Scholar]
  • 31.Ahmed Z., Mohamed K., Zeeshan S., Dong X.Q. Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine. Database. 2020;2020:baaa010. doi: 10.1093/database/baaa010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Singh A., Schooley B., Floyd S.B., Pill S.G., Brooks J.M. Patient preferences as human factors for health data recommender systems and shared decision making in orthopaedic practice. Front Digit Health. 2023;5(1):137. doi: 10.3389/fdgth.2023.1137066. 066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Delivering healthcare's ‘triple aim’: electronic health records and the health research participant in the UK National Health Service - Wyatt - 2020 - Sociology of Health & Illness - Wiley Online Library, (n.d.). https://onlinelibrary.wiley.com/doi/10.1111/1467-9566.13101 (accessed July 9, 2024). [DOI] [PubMed]
  • 34.Heckman G.A., Hirdes J.P., McKelvie R.S. The role of physicians in the era of big data. Can J Cardiol. 2020;36:19–21. doi: 10.1016/j.cjca.2019.09.018. [DOI] [PubMed] [Google Scholar]
  • 35.van Baalen S., Boon M. An epistemological shift: from evidence-based medicine to epistemological responsibility. J Eval Clin Pract. 2015;21:433–439. doi: 10.1111/jep.12282. [DOI] [PubMed] [Google Scholar]
  • 36.S. van Baalen, M. Boon, P. Verhoef, From clinical decision support to clinical reasoning support systems, (n.d.). 10.1111/jep.13541. [DOI] [PMC free article] [PubMed]
  • 37.Wyatt J., Spiegelhalter D. Field trials of medical decision-aids: potential problems and solutions. Proc Annu Symp Comput Appl Med Care. 1991:3–7. [PMC free article] [PubMed] [Google Scholar]
  • 38.Mittermaier M., Raza M., Kvedar J.C. Collaborative strategies for deploying AI-based physician decision support systems: challenges and deployment approaches. Npj Digit Med. 2023;6:1–2. doi: 10.1038/s41746-023-00889-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Xie Y., Nguyen Q.D., Hamzah H., et al. Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study. Lancet Digit Health. 2020;2:e240–e249. doi: 10.1016/S2589-7500(20)30060-1. [DOI] [PubMed] [Google Scholar]
  • 40.Ting D.S.W., Pasquale L.R., Peng L., et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. 2019;103:167–175. doi: 10.1136/bjophthalmol-2018-313173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Lin C.-S., Liu W.-T., Tsai D.-J., et al. AI-enabled electrocardiography alert intervention and all-cause mortality: a pragmatic randomized clinical trial. Nat Med. 2024;30:1461–1470. doi: 10.1038/s41591-024-02961-4. [DOI] [PubMed] [Google Scholar]
  • 42.Doudesis D., Lee K.K., Boeddinghaus J., et al. Machine learning for diagnosis of myocardial infarction using cardiac troponin concentrations. Nat Med. 2023;29:1201–1210. doi: 10.1038/s41591-023-02325-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Hager P., Jungmann F., Holland R., et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nat Med. 2024:1–10. doi: 10.1038/s41591-024-03097-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Amann J., Blasimme A., Vayena E., Frey D., Madai V.I. the Precise4Q consortium, Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20:310. doi: 10.1186/s12911-020-01332-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Jiang F., Jiang Y., Zhi H., et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2:230–243. doi: 10.1136/svn-2017-000101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.How AI is changing warfare, (n.d.). https://www.economist.com/briefing/2024/06/20/how-ai-is-changing-warfare? (accessed July 9, 2024).
  • 47.Uddin S., Khan A., Hossain M.E., Moni M.A. Comparing different supervised machine learning algorithms for disease prediction. BMC Med Inform Decis Mak. 2019;19:281. doi: 10.1186/s12911-019-1004-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Kundu S. AI in medicine must be explainable. Nat Med. 2021;27 doi: 10.1038/s41591-021-01461-z. 1328–1328. [DOI] [PubMed] [Google Scholar]
  • 49.Albahri A.S., Duhaim A.M., Fadhel M.A., et al. A systematic review of trustworthy and explainable artificial intelligence in healthcare: Assessment of quality, bias risk, and data fusion. Inf Fusion. 2023;96:156–191. doi: 10.1016/j.inffus.2023.03.008. [DOI] [Google Scholar]
  • 50.Thorell A. Health and Disease: Between Naturalism and Normativism. Philos Sci. 2024;91:449–467. doi: 10.1017/psa.2023.113. [DOI] [Google Scholar]
  • 51.Omiye J.A., Lester J.C., Spichak S., Rotemberg V., Daneshjou R. Large language models propagate race-based medicine. Npj Digit Med. 2023;6:1–4. doi: 10.1038/s41746-023-00939-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Hicks M.T., Humphries J., Slater J. ChatGPT is bullshit. Ethics Inf Technol. 2024;26:38. doi: 10.1007/s10676-024-09775-5. [DOI] [Google Scholar]
  • 53.Will Disruptive Innovations Cure Health Care?, (n.d.). https://hbr.org/2000/09/will-disruptive-innovations-cure-health-care (accessed July 12, 2024).
  • 54.Baptista P.M., Martin F., Ross H., O'Connor Reina C., Plaza G., Casale M. A systematic review of smartphone applications and devices for obstructive sleep apnea. Braz J Otorhinolaryngol. 2022;88:S188–S197. doi: 10.1016/j.bjorl.2022.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Attia Z.I., Harmon D.M., Dugan J., et al. Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction. Nat Med. 2022;28:2497–2503. doi: 10.1038/s41591-022-02053-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Shahsavar Y., Choudhury A. User intentions to use ChatGPT for self-diagnosis and health-related purposes: cross-sectional survey study. JMIR Hum Factors. 2023;10:e47564. doi: 10.2196/47564. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Sirkin H., Keenan P., Jackson A. The hard side of change management. Harv Bus Rev. 2005;83:108–118. doi: 10.1109/EMR.2014.6966953. 158. [DOI] [PubMed] [Google Scholar]
  • 58.Morley J., Kinsey L., Elhalal A., Garcia F., Ziosi M., Floridi L. Operationalising AI ethics: barriers, enablers and next steps. AI Soc. 2023;38:411–423. doi: 10.1007/s00146-021-01308-8. [DOI] [Google Scholar]
  • 59.J. Morley, On designing an algorithmically enhanced nhs: towards a conceptual model for the successful implementation of algorithmic clinical decision support software in the national health service, [PhD thesis]. University of Oxford. (n.d.). https://ora.ox.ac.uk/objects/uuid:0f58b2c4-ffa0-412d-afc3-aedc2eaf46d4 (accessed July 15, 2024).
  • 60.Clarke A., Watt I., Sheard L., Wright J., Adamson J. Implementing electronic records in NHS secondary care organizations in England: policy and progress since 1998. Br Med Bull. 2017;121:95–106. doi: 10.1093/bmb/ldw055. [DOI] [PubMed] [Google Scholar]
  • 61.Grimson J., Grimson W., Hasselbring W. The SI challenge in health care. Commun ACM. 2000;43 doi: 10.1145/336460.336474. [DOI] [Google Scholar]
  • 62.Davison R.M., Wong L.H.M., Ou C.X.J., Alter S. The coordination of workarounds: Insights from responses to misfits between local realities and a mandated global enterprise system. Inf Manage. 2021;58 doi: 10.1016/j.im.2021.103530. [DOI] [Google Scholar]
  • 63.Alter S. Theory of Workarounds. Commun Assoc Inf Syst. 2014:34. doi: 10.17705/1CAIS.03455. [DOI] [Google Scholar]
  • 64.Post Office Horizon scandal explained: everything you need to know | Computer Weekly, ComputerWeeklyCom (n.d.). https://www.computerweekly.com/feature/Post-Office-Horizon-scandal-explained-everything-you-need-to-know (accessed July 15, 2024).
  • 65.Oddy C., Zhang J., Morley J., Ashrafian H. Promising algorithms to perilous applications: a systematic review of risk stratification tools for predicting healthcare utilisation. BMJ Health Care Inform. 2024;31 doi: 10.1136/bmjhci-2024-101065. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Roman L.C., Ancker J.S., Johnson S.B., Senathirajah Y. Navigation in the electronic health record: a review of the safety and usability literature. J Biomed Inform. 2017;67:69–79. doi: 10.1016/j.jbi.2017.01.005. [DOI] [PubMed] [Google Scholar]
  • 67.Ignatiadis I., Nandhakumar J. The effect of ERP system workarounds on organizational control: an interpretivist case study. Scand J Inf Syst. 2009:21. https://aisel.aisnet.org/sjis/vol21/iss2/3 [Google Scholar]
  • 68.Ash J.S., Berg M., Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc JAMIA. 2004;11:104–112. doi: 10.1197/jamia.M1471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.McLeod J., Childs S. A strategic approach to making sense of the “wicked” problem of ERM. Rec Manag J. 2013;23:104–135. doi: 10.1108/RMJ-04-2013-0009. [DOI] [Google Scholar]

Articles from Future Healthcare Journal are provided here courtesy of Royal College of Physicians

RESOURCES