An artificial intelligence-based chatbot for prostate cancer education: Design and patient evaluation study

Abstract

Introduction

Artificial intelligence (AI) is increasingly used in healthcare. AI-based chatbots can act as automated conversational agents, capable of promoting health and providing education at any time. The objective of this study was to develop and evaluate a user-friendly medical chatbot (prostate cancer communication assistant (PROSCA)) for provisioning patient information about early detection of prostate cancer (PC).

Methods

The chatbot was developed to provide information on prostate diseases, diagnostic tests for PC detection, stages, and treatment options. Ten men aged 49 to 81 years with suspicion of PC were enrolled in this study. Nine of ten patients used the chatbot during the evaluation period and filled out the questionnaires on usage and usability, perceived benefits, and potential for improvement.

Results

The chatbot was straightforward to use, with 78% of users not needing any assistance during usage. In total, 89% of the chatbot users in the study experienced a clear to moderate increase in knowledge about PC through the chatbot. All study participants who tested the chatbot would like to re-use a medical chatbot in the future and support the use of chatbots in the clinical routine.

Conclusions

Through the introduction of the chatbot PROSCA, we created and evaluated an innovative evidence-based health information tool in the field of PC, allowing targeted support for doctor–patient communication and offering great potential in raising awareness, patient education, and support. Our study revealed that a medical chatbot in the field of early PC detection is readily accepted and benefits patients as an additional informative tool.

Introduction

Prostate cancer (PC) ranks second as the most frequent malignancy among men with an incident rate of 27% of all new cancer diagnoses in males. ¹ Diagnosis of PC at an early stage using screening has the benefits of a high possibility of cure, less aggressive treatment options, reduced disease progression to advanced or metastatic stages, and improved quality of life. ² Early detection of PC is feasible using the biomarker prostate-specific antigen (PSA) or the digital rectal examination (DRE) as a screening method. Multiparametric magnetic resonance imaging (mpMRI) and fusion biopsy of the prostate improve the accuracy of PC detection.^3,4 Within the European Randomised Study of Screening for PC, PSA screening was shown to reduce PC death and metastasis in men aged 55 to 69 years by 29% and 30% to 42%, respectively.^5,6 If PC is diagnosed at an early or regional stage, 5-year relative survival rates are about 100%. ⁷

Reasons for not getting screened for PC include obstacles to access, lack of knowledge, fear of cancer, embarrassment, and low perceived risk. Information about PC and PC screening is often received through lay media, friends, or family members. ⁸ According to data from the National Cancer Institute Health Information National Trends Survey (HINTS), ⁹ patients are increasingly turning to online resources to get medical information. HINTS data show that 45.8% of patients specifically looked up advice or information about cancer at least every few months. However, despite this, patient trust in online resources is low, with 80% of participants reporting a trust level of some, a little, or not at all. It is promising that patients are taking an active role in the medical decision-making process by utilizing available online resources to educate themselves. However, there is a clear risk that shared information on the internet may be inaccurate. This is in line with a recent study supporting that social media content about PC is generally of low-to-moderate quality and poorly actionable for health consumers. ¹⁰

The ease, speed, and convenience of using chatbots provide their users with instant and consistent answers to obtain information or assistance in a timely and efficient manner. Chatbots as automatic conversational agents that run on artificial intelligence (AI) interaction between users and machines are often referred to as the most promising form of human–machine interactions. ¹¹ Particularly, medical chatbots as a virtual doctor harbor the potential to improve the accessibility of medical knowledge, reduce the burden of healthcare costs, and empower patients in their medical decision-making process. ¹² Chatbots can present large amounts of detailed information in a concise, personalized form anytime and anywhere. It allows patients to retrieve validated information in a timely manner on their own devices, such as their smartphones, and from home.

With PC communication assistant (PROSCA), we developed an extendable chatbot about PC that provides men at risk of PC with high-quality, doctor-validated information in a situation-adapted manner. The purpose of the chatbot was to assist men by offering targeted information about PC, invasive diagnostics, potential complications as well as treatment options to reduce decisional conflict, enhance patient knowledge, and promote shared decision-making.

Materials and methods

Chatbot development

The chatbot was developed in cooperation between the Urology Clinic of Heidelberg University Hospital, the German Cancer Research Center (DKFZ), and SAP SE.

The chatbot was trained to answer questions about PC, for example, about prostate diseases, diagnostic procedures, and treatment options for PC. The SAP Conversational AI (CAI) Platform was selected for this use case. SAP CAI is a native low-code chatbot platform that uses natural language processing (NLP) models to analyze user input and provide appropriate answers. SAP CAI uses a hybrid approach of pre-built NLP models and customizable machine learning models to understand and interpret natural language input from users. The NLP core models are used for tasks such as Named Entity Recognition, Intent Recognition, Sentiment Analysis, and Language Detection. These models are mainly based on deep learning algorithms and are trained on large datasets to accurately extract information from text inputs. The custom machine learning models can be trained using context-related data. This allows for a chatbot that is tailored to its domain and can understand and respond to, for example, PC-specific terminology and language. The NLP core is then used by the chatbot instance to supply information to a rule-based framework to handle specific scenarios or phrases, referred to as answer-response flow. The answer-response flow is structured as followed ¹³ : the user provides an input, a sentence, or part of a sentence (expression), which may contain trained keywords (entities) providing parts of the expression in a structured way such as use-case specific vocabulary, for example, biopsy. Entities were either part of the general set of SAP CAI or were generated to specify PC-related expressions such as “biopsy,” “DRE,” or “PSA,” which had interferant presence in many intents. The patient's input is then analyzed in the NLP model in several steps, including language detection, preprocessing, act detection, classification, sentiment analysis, and entity recognition. Based on the analysis, the model identifies the proper intent (a collection of expressions with the same meaning intended to achieve a specific goal). Intent recognition executes the matching skill (defined dialog sequence) and provides a response back to the patient ¹⁴ (Table 1). After identifying the main purpose and objectives of PROSCA, we designed the target conversation flows to structure the intents in a hierarchical order. Based on those, the initial dataset containing PC-related expressions, intents, and entities was built. The basic NLP model of the SAP CAI Platform, which is pretrained and can recognize several entities by default, was enhanced with the collected training data.

Table 1.

Examples of trained intents and entities in the chatbot PROSCA.

Patient's question	Matching intent	Matching entities	Operated skill
“Why do I need a biopsy?”	“i-03-03-biopsy“	Pronoun “I,” number “a/one,” “biopsy”	Provide general information on the indication and significance of prostate biopsy
“How long could I see blood in the urine after the biopsy?”	“i-07-10-biopsy-complications”	Pronoun “I,” “biopsy”	Provide information on the frequency, duration, and severity of side effects after prostate biopsy
“What is the PSA level?”	“i-03-02-psa”	“PSA”	Provide information about the enzyme PSA, its significance, measurement, and threshold values
“I am worried.”	“fear”	Pronoun “I”	Advice to take the concerns seriously and to discuss them with his doctor as well as provision of contact information for immediate help
“What do I need to carry with me on the day of surgery?”	“i-07-01-instructions-before-biopsy“	Pronoun “I,” “Date/Time”	Provision of a list of items to bring along to the hospital stay

PROSCA: prostate cancer communication assistant; PSA: prostate-specific antigen.

To give a descriptive example, the patient might ask “why do I need a biopsy?” Thus, NLP recognizes intent “i-03-03-biopsy” (retrieve general information about biopsy) with 91% confidence. Two general entities (pronoun “I”, number “a/one”) and one specific entity “biopsy” are found (Table 1). The matching skill “sk-03-03-biopsy” (provide general information about biopsy) is executed and the chatbot responds with the predefined answer. The chatbot's answer will include the indications when patients require prostate biopsies, as well as threshold values and contraindications for a prostate biopsy such as high patient age. Besides giving short answers to the patient, the chatbot also provides a selection of additional topics on which the user can click on to receive additional information.

Various fallback options were integrated into the chatbot’s skill set in case the chatbot does not recognize the patient's question. When the chatbot can’t match any corresponding intent, it encourages the user to try another wording or to click through several topics to find the right answer. The patient is also invited to ask to write “help” in the chatbot to get an overview of the thematic fields covered by the chatbot. When the chatbot identifies more than one intent, it can provide the three most probably matching answers to choose from for the patient. In addition, in case the chatbot did not recognize the intent or selected the wrong intent, the chatbot can be further trained by adding additional expressions.

The responses given by the chatbot were formulated, examined, and validated by two senior urologists certified by the clinics (senior physician MG and medical director MH). The chatbot's responses are based on published highest-quality scientific evidence and state-of-the-art standard procedures. They follow the recommendations of the guidelines of the European Association of Urology, which were formulated by more than 300 international experts. ¹⁵ Seven urologists and urology residents were asked to test the chatbot and to apply common questions to PC patients. Monitoring this communication allowed us to check if the chatbot understood its users correctly and led to updates on the chatbot’s intents and the addition of new capabilities to the chatbot.

Recruitment of participants

Ten patients with suspicion of PC were recruited from December 2021 to January 2022 at Heidelberg University Hospital. Suspicion of PC was raised by an elevation of the biomarker PSA or a suspicious DRE. All patients received a mpMRI of the prostate and a fusion-guided prostate biopsy. The chatbot was provided to the study patients via a website starting with the first visit to the Heidelberg University Hospital until several weeks after the discussion of the histopathological result of the prostate biopsy with the patient. Study inclusion criteria were the regular use of a laptop, computer, smartphone, or tablet and sufficient knowledge of the German language. Patients were asked to fill out a first questionnaire upon study inclusion containing demographic characteristics. A second questionnaire was given to be filled out at the end of the chatbot's use with closed-ended questions about usability, usage data, and perceived benefits, and open-ended questions about the strengths and weaknesses of the chatbot. The chatbot was provided as an alternative source of information in the study. Standard education and information by physicians of Heidelberg University Hospital were maintained for the participants of the study.

Ethical, legal, and organizational framework

Data were collected prospectively, and the study was approved by the ethical committee of the University of Heidelberg (Approval No. S-005/2021). All study participants gave their written informed consent. Patients were informed that the use of the chatbot cannot replace a personal, medical consultation, especially in case of acute complaints. With participation in the study, patients agreed to the processing, storage, and evaluation of their data, including the chat messages. Patients had to acknowledge that they will not disclose their identity in chat messages, especially when combined with questions about their health condition. The chatbot was made accessible through a custom, responsive website embedding the SAP CAI Web Client User Interface (UI) hosted on Heidelberg University Hospital infrastructure (Figure 1). As a result, SAP SE, as the provider of the chatbot technology, did not store any internet protocol addresses of its users, prohibiting the identification of patients. The Web Client UI manages conversations between the patient and the chatbot backend, which is hosted on the SAP Business Technology Platform (BTP). ¹⁶ The underlying Cloud Foundry environment on SAP BTP provides a cloud-based platform as a Service capabilities ¹⁷ and a private instance (Enterprise Edition) of the SAP CAI platform with the trained NLP model to ensure secure data isolation. ¹⁸

Figure 1.

Access to the chatbot PROSCA. The chatbot was made accessible to the patients through a custom, responsive website embedding the SAP Conversational AI Web Client User Interface hosted on Heidelberg University Hospital infrastructure. The chatbot backend was hosted on the SAP Business Technology Platform.

PROSCA: prostate cancer communication assistant; AI: artificial intelligence.

Results

Chatbot content

The chatbot built in this study contained around 70 trained intents with more than 3000 expressions and around 70 corresponding skills, as well as around 30 entities. During development, the initial dataset was continuously improved, for example, if the input was matched to the wrong intent, or if it was not recognized properly. The precision of the chatbot was monitored throughout all stages of development, peaking at 78.7% on average in a final benchmark test with a variety of known and unknown expressions.

The chatbot was developed to inform the patient comprehensively about PC, while at the same time providing short, targeted information (Figure 2). In the chatbot, patients can either enter their specific questions or view the provided topic areas and click through to increasingly detailed topics.

Figure 2.

Structure, design, and functionality of the chatbot PROSCA. The welcome page of the chatbot and the AI-based answering of two sample questions are presented. Besides giving short answers to the patient, the chatbot also provides a selection of additional topics on which the user can click on to receive additional information (e.g. MRI results). If the patient does not have any specific questions, he can get an overview of the chatbot's topics via the “Help” function. The chatbot pages have been translated into English, the original chatbot pages can be found in the Supplemental material.

AI: artificial intelligence; MRI: magnetic resonance imaging; PROSCA: prostate cancer communication assistant; PSA: prostate-specific antigen.

The chatbot was designed to provide anatomical and functional information about prostate and prostate diseases such as benign prostate enlargement and PC including their frequency and their symptoms. It informs about the opportunities of early PC detection and explains the diagnostic examinations of PSA, DRE, and mpMRI. It provides detailed information on the prostate biopsy to confirm or exclude the suspicion of PC, from the procedure to possible side effects of the biopsy, and the possible results of the biopsy. Patients are informed that different therapy options are available depending on the tumor stage. All guideline-compliant forms of therapy of PC, including possible complications, are explained in detail and it is described which therapy is most suitable for individual (patient- or tumor-specific) factors (Figure 3). Furthermore, organizational advice is given via the chatbot (e.g. what to pay attention to before and after prostate biopsy) and further support is distributed in the form of links (e.g. to psycho-oncological support).

Figure 3.

Topics covered by the chatbot PROSCA after training. To create a conversational flow, the intents of the chatbot were structured in a hierarchical order. The chatbot was designed to provide anatomical and functional information about the prostate as well as prostate diseases. It explains the diagnostic examinations of early prostate cancer detection and provides detailed information on the prostate biopsy (from the procedure to side effects to the possible results of the biopsy). Patients are informed that different therapy options are available depending on the tumor stage. Furthermore, organizational advice and support are given via the chatbot.

MRI: magnetic resonance imaging; PSA: prostate-specific antigen.

Participant characteristics

Ten men were recruited to test the newly developed PC chatbot for functionality, suitability, and utility from a patient perspective. The median age of the study cohort was 68 years old, the youngest study participant was 49, and the oldest was 81 years old. All participants regularly used a laptop or computer in everyday life and 90% of them used a smartphone in addition. The median PSA of the study participants was 4.6 ng/mL. In total, 90% of the study participants expressed a high need for information about PC (Table 2).

Table 2.

Patient characteristics.

	Study cohort
Men included in the study, n	10
Age, year, median (IQR)	68 (60–71)
PSA level, ng/ml, median (IQR)	4.6 (3.7–6.2)
Regular use of a laptop/computer, n (%)	10 (100%)
Regular use of a smartphone, n (%)	9 (90%)
Regular use of a tablet, n (%)	4 (40%)
High need for information on PC	9 (90%)

IQR: interquartile range; PC: prostate cancer; PSA: prostate-specific antigen.

Patient evaluation

All recruited patients completed the study questionnaires. Nine out of 10 patients used the chatbot and their feedback was included in the evaluation. The patient who did not use the chatbot was the oldest study participant with 81 years. None of the patients had used a medical chatbot before (Figure 4). Patients most frequently accessed the chatbot via laptop/computer (78%), and second most frequently via smartphone (22%). The chatbot was most frequently used in the days after the biopsy (67%), around the time of the discussion of the histopathological result. The evaluation showed that the patients had no concerns about the security of their own data when using the technology of a chatbot, as none of the study participants felt unsafe when using the chatbot. Ease of use of the chatbot was confirmed by the patients, with 78% of study participants needing no assistance in using the chatbot. In total, 89% of patients fully or partially agree that they have gained significant information about PC from the chatbot. All study participants would like to be able to use a chatbot more often in healthcare in the future (67% agree, 33% partly agree, and none disagree). Similarly, patients suggest implementing medical chatbots in clinical routine (67% agree, 33% partially agree, and none disagree). In the free-text evaluation, patients suggest a further thematic expansion of the chatbot and detailed coverage of additional topics. Another feedback expressed by patients on the chatbot was the desire to receive individual answers and therapy recommendations tailored to their own situation.

Figure 4.

Evaluation of patient satisfaction with the chatbot PROSCA. The evaluation of the chatbot was obtained after use via structured questionnaires. Nine out of 10 recruited patients used the chatbot and their feedback was included in the evaluation. The participants’ answers are given as a percentage.

Discussion

At all stages of decision-making and treatment in PC, patients report a general lack of readily available information on diagnosis, prognosis, medical procedures, treatment options, side effects, and the logistics of care.^19,20 Resources that are designed specifically to help patients understand medical tests and make a treatment decision are perceived as central for decision-making, because they provide high-quality evidence that patients can understand and trust, preparing them to ask relevant questions based on the knowledge they obtained. ¹⁹ Here, chatbots hold great potential in healthcare, as a low-threshold offer to inform patients about diseases before a visit to the doctor, as a second opinion tool in addition to the doctor–patient conversation, and as a companion at home during and after therapies. Chatbots can contribute to the development of healthcare to a discipline that is more predictive, personalized, preventive, and participatory, ²¹ while at the same time relieving medical staff of the need to repetitively provide standard information to the patients, ensuring that they can spend their time on individual engagements with patients. Therefore, chatbots have the potential to create benefits for patients, physicians, and hospitals, as well as for public health.

In this study, we designed and evaluated a user-friendly chatbot with evidence-based content for men at risk for PC, accompanying them during diagnosis and treatment decisions.

Our analysis revealed several major results. First, the development and use of a medical chatbot to provide information about cancer are both highly innovative and desirable for patients. None of the study patients had used a medical chatbot before, however, the study participants clearly expressed the recommendation for the increased use of chatbots in healthcare.

Second, our study results confirmed that even in older generations (median age of the cohort 68 years), who are regularly confronted with a cancer diagnosis, the usability of the chatbot was feasible and without much need for assistance. The evaluation results also revealed that patients had no privacy or security concerns about accessing the digital technology of a medical chatbot. This is in line with a previous study showing that most internet users would be receptive to using AI-led health chatbots. ²²

Eight out of nine patients declare that they have gained benefits concerning their knowledge about PC via the chatbot. The patient who did not receive a clear gain in the information stated that he was already well informed about PC via the doctor consultations. The aim of this proof-of-concept study was to evaluate in the first step the chatbot's usability and the patients’ willingness to use a chatbot on PC. Therefore, the chatbot was made available to the study participants as an additional tool without replacing any standard doctor–patient conversation.

With PROSCA, we developed and assessed for the first time an AI-based chatbot to inform patients about PC early detection. Compared to a medical pamphlet or a hospital information website, the knowledge offered via the chatbot is much more usable for the patient in his current situation as the information provided is focused as a result of the patient's questions, and the information fed into the chatbot is carefully curated for the very use case of the patient. This is in contrast to an exhaustive list of all available information in, for example, a medical pamphlet to work through on his own. The chatbot offers the possibility of sharing detailed health information from, for example, the size of a large encyclopedia in a very intuitive, fast, and short text-based way. In addition, the broad application, potential reach, and independence of the chatbot distinguish it from medical pamphlets and hospital information sites.

The potential benefits of chatbots for patients and the healthcare system are in sharp contrast with the scarcity of clinical trials evaluating chatbots in healthcare and in particular in oncology. ²³ One blind, noninferiority randomized controlled trial compared the information given by a chatbot with that given by a group of physicians to patients with breast cancer. In this study, the assessment scores from the chatbot were noninferior to the scores of the physicians. ²⁴ In other previous studies, chatbots have been applied to improve access to mental healthcare services, being used for therapy, training, education, counseling, and screening in mental health. ²⁵ Another field of chatbot use was to improve post-operative care delivery for patients and to address common concerns, for example, post-ureteroscopy. ²⁶ The occurrence of the pandemics with novel coronavirus led to the rise of health chatbots to promote preventive measures, virus updates, communicate vaccine-related messages, and reduce psychological damage caused by isolation.^11,27 These research results demonstrate that the use of chatbots to support patients is on the rise to be incorporated into medical practice. In line with our study results, chatbots can be tools to improve the quality of patient care and can have sustained value. The COVID-19 pandemic has also led to the wider acceptance of new strategies such as telemedicine to appear as a pragmatic approach and practical alternative to doctor's consultation.^28,29

Chatbots have the potential to support a variety of health-related activities, including behavior change, treatment support, health monitoring, training, triage, and screening support. Automation of these tasks could free clinicians to focus on more complex work and increase the accessibility to healthcare services. ³⁰ Chatbots are cost-effective to run and can automate repetitive tasks, allowing physicians to provide higher quality, personalized, and empathetic care to their patients. Chatbots are able to run 24 h a day, be tailored to specific populations or health conditions, and communicate in multiple languages. ³¹ Chatbots might even save patients with minor health concerns from a consultation with the doctor, potentially saving a significant amount of money and resources. This could allow clinicians to spend more time treating patients who need a consultation. However, the quality of chatbots needs to be rigorously assessed, so that they are able to actually detect the difference between minor and major symptoms. ²³ In addition, there is little evidence concerning the evaluation of the costs of chatbots as well as their security and interoperability. ³⁰

One of the main criticisms of chatbots is that they are not capable of empathy, recognizing users’ emotional states, and tailoring responses reflecting these emotions. On the other hand, chatbots allow anonymous, convenient, and fast access to relevant information, enabling users to discuss their intimate and perhaps embarrassing health issues. ²²

For the emerging technology of chatbots to become more widespread, physicians need to adopt to the use of chatbots. Providing physicians with evidence-based research on chatbots will help to inform them on the most appropriate way to complement their practice rather than impede their work. ³¹ Likewise, patients can show hesitancy regarding the new technology of a chatbot with concerns about accuracy. Thus, the user experience needs to be optimized to achieve the best uptake and utilization. While the passage of time is necessary for any innovation to be adopted, certain characteristics of social systems such as governmental endorsement are likely to positively influence potential adoption. ²²

Going further, it is straightforward to envision several dimensions in which to expand our chatbot in subsequent projects. For future models, a direct connection to the hospital's Electronic Medical Records system could be envisioned, where the respective information of the patient's medical background is readily available. However, this will require high levels of trust in the correct identification of the patient's identity in order to preserve data privacy and security. The chatbot can also be extended thematically, in particular, to explain and accompany metastatic PC and the possible options of systemic therapy lines in detail. With primary and secondary prevention increasingly becoming the focus of attention, ³² chatbots cannot only be applied at the first contact with the urologist, but also as a low-threshold information tool to create awareness for early detection of PC among the general population. In addition, the concept of the chatbot is not meant to be restricted to PC or urology; training the chatbot about health promotion in general, other tumor entities, and various medical diseases is a viable option. The technology of a chatbot allows opportunities for patient empowerment, in particular, in remote areas with poor medical care, as the chatbot and its content can be accessed globally. Healthcare systems can implement digital tools in disadvantaged communities to reach underserved patients and combat health disparities. ³³

Limitations of our study include the challenge of applying validated criteria for the evaluation of chatbots. Future directions may include the design of validated instruments to evaluate the vast diversity of eHealth formats.

The use of the internet is not ubiquitous in older age groups. However, PC screening is recommended starting from 40 to 50 years ¹⁵ and our data reveal the willingness to use the chatbot across different age groups, with the median age of our study population being 68 years old.

Furthermore, our analysis of PROSCA was limited to German-speaking men in Germany who already had an appointment for a urology consultation. Our results may not be completely transferable to men in various societies worldwide.

Strengths of our study include the design and patient assessment of a chatbot with detailed content about PC. This has great healthcare relevance because PC is the second most common cancer in men, ¹ and the use of online health information is expanding rapidly. Our novel findings highlight the value of a chatbot as a reliable, AI-based source of information and support for patients during PC diagnosis.

Conclusions

Through the introduction of the chatbot PROSCA, we created and evaluated an innovative evidence-based health information tool in the field of PC, allowing targeted support for doctor–patient communication and offering great potential in raising awareness, patient education, and support. Our study reveals that a medical chatbot in the field of early PC detection is readily accepted and benefits patients as an additional informative tool.