kBot: Knowledge-enabled Personalized Chatbot for Asthma Self-Management

Abstract

There is a well-recognized need for a shift to proactive asthma care given the impact asthma has on overall healthcare costs. The demand for continuous monitoring of patient’s adherence to the medication care plan, assessment of environmental triggers, and management of asthma can be challenging in traditional clinical settings and taxing on clinical professionals. Recent years have seen a robust growth of general purpose conversational systems. However, they lack the capabilities to support applications such an individual’s health, which requires the ability to contextualize, learn interactively, and provide the proper hyper-personalization needed to hold meaningful conversations. In this paper, we present kBot, a knowledge-enabled personalized chatbot system designed for health applications and adapted to help pediatric asthmatic patients (age 8 to 15) to better control their asthma. Its core functionalities include continuous monitoring of the patient’s medication adherence and tracking of relevant health signals and environment data. kBot takes the form of an Android application with a frontend chat interface capable of conversing in both text and voice, and a backend cloud-based server application that handles data collection, processing, and dialogue management. It achieves contextualization by piecing together domain knowledge from online sources and inputs from our clinical partners. The personalization aspect is derived from patient answering questionnaires and day-to-day conversations. kBOT’s preliminary evaluation focused on chatbot quality, technology acceptance, and system usability involved eight asthma clinicians and eight researchers. For both groups, kBot achieved an overall technology acceptance value of greater than 8 on the 11-point Likert scale and a mean System Usability Score (SUS) greater than 80.

I. INTRODUCTION

Asthma is a chronic lung inflammatory disease that has impacted more than 25 million people in the United States, out of which 6 million are children [1]. Though incurable, asthma can be controlled and managed by strict adherence to a asthma care plan and by avoiding the triggers [2]. However, due to a lack of consistent adherence to the care plan, and inadequate information about the patient’s environment, asthma management can be challenging [3]. These issues are further compounded by its multifactorial nature, where every patient is sensitive to different triggers, and react differently even when exposed to the same trigger [4]. This demands personalized care beyond the regular hospital setup to which the clinical professionals are limited. With the advent of digital health monitoring using Internet of Things (IoT), it is possible to monitor medication adherence, exposure to environmental triggers, and asthma control level in real-time using low-cost sensors and mobile devices [5]. By leveraging such technology prevalence, our NIH funded kHealth Asthma project involves an ongoing study that is in collaboration with Dayton Children’s Hospital (DCH), approved by the DCH Institutional Review Board (IRB) with a targeted cohort size of 200 (120 deployments already completed) asthmatic children between 5 and 17 years.

kHealth Asthma is a digital framework that personalizes the long-term assessment and management of asthma in children [6]. It consists of a kit given to each consented patient for one or three months. The kit comprises of a mobile health application with a broad range of low-cost wearable and sensors (Fitbit for sleep measurement, Foobot for indoor air quality monitoring, and peak flow meter for measuring lung function). We have achieved a compliance rate of 63% so far across 107 completed deployments. These health data signals, along with anonymized data extracted from respective clinical notes, and publicly available outdoor environmental data are then aggregated, analyzed, and transformed into actionable information. Our findings involving clinical collaboration have revealed that continuous monitoring of asthmatic patients in their everyday life can indeed give more useful insights into their asthma health as opposed to episodic clinic visits. Patients with poor medication compliance reflected poor asthma control [7], and different patients reacted to different environmental triggers with varying intensity [8]. Our results were in concordance to existing studies which have concluded that medication nonadherence continues to be a frequent problem among asthma patients [9]. Such issues, along with the assessment of environmental triggers and asthma control could benefit from a lighter-weight, more interactive, and knowledge-driven conversational approach [10].

Recent years have seen immense maturity in Artificial Intelligence (AI) research which has in part proliferated the growth of intelligent conversational systems, also known as chatbots. They are increasingly popular due to their capability of simulating human-like conversations with a user through speech, text, smart display, and multimodal communication¹. Contrary to static applications, it can understand user intents and choices through interactions and communicate accordingly. Such technology is rapidly gaining traction in the healthcare domain where professional care is limited². Chatbots are capable of delivering a more convenient and accessible approach through cost-effective mediums. Nonetheless, they are limited in their inherent ability to contextualize, learn interactively, and provide the proper hyper-personalization needed to hold a meaningful conversation.

In this paper, we propose kBot, a knowledge-enabled chatbot, that is intended to replace our use of the mobile app in the above-reported study. It is capable of interacting with a patient in a contextualized and personalized manner. It curates its asthma domain knowledge from different online sources and inputs from our clinical collaborators. It then aggregates this knowledge with patients’ data such as symptoms and medication intake to deliver a personalized conversation experience. The first version of kBot reported here takes on a design approach that centers around addressing medication nonadherence issue in pediatric asthma management and assessing environmental triggers at an individual level. The ultimate goal is to bridge and simplify long-term real-time monitoring of asthma condition, alert on potential environmental triggers, and educate the patients on various asthma self-management skills. Our current implementation is limited to the Android ecosystem but can be easily adapted to iOS.

II. RELATED WORK

Various chatbot applications have been developed to address issues of healthcare management but in isolation. There are chatbot technologies that encourage users towards improved quality of life. Fadhil et. al. [11] designed a chatbot that encourages adult population towards healthy eating habits to prevent weight gain. HealthTap³ and Your.MD⁴ are examples of health assistants that provide online health counseling. Some chatbots focus on streamlining and simplifying the existing healthcare system. MANDY [12] is a primary care chatbot that assists healthcare staffs by automating the patient intake process by interacting with patients and collecting initial reports on them.

Other forms of chatbots are also available on various popular messaging platforms, such as Facebook messenger and Slack, and work to help patients to adhere to their regular care plan and better manage their health. FLORENCE⁵ Bot is an example of a Facebook messenger bot that reminds patients regularly to take their pills and also tracks their health factors like body weight and moods. In addition, it also helps patients to find a doctor or pharmacy location. Similarly, SMOKEY⁶, another Facebook messenger bot, alerts patients when the air quality is unhealthy.

While contemporary chatbots focus on solving individual issues of healthcare, our holistic approach to chronic, multifactorial, asthma seeks to address its self-management issues. kBot, as a multi-functionality personal assistant, distinguishes itself by offering a personalized approach to track patients health and their medication intake, remind and encourage them towards medication adherence, alert them about asthma triggers, and collectively help them self-manage their asthma to achieve an overall improved health outcome.

III. METHODS

A. System Architecture

kBot follows a client-server architecture (Figure 1) where the client is a lightweight frontend chat interface, and the backend server is a standalone web application hosted in the cloud. The frontend interface communicates (in both text and voice) with patients at least twice a day and logs the conversation history in the server in JSON⁷, a lightweight data-interchange format. The data of interest (refer to Section III-B) are then extracted from these conversation logs. Concurrently, the server continuously monitors and collects environmental data, at different frequencies, through third-party weather Application Programming Interface (API) for the zipcodes specific to each patient (more on Section III-A2). These data are stored in Elasticsearch ⁸, a full-text search engine with a NoSQL database. Contrary to traditional SQL databases, the fast query engine of Elasticsearch allows kBot to use the stored data (conversation logs and environmental data) to generate dynamic prompts and personalized response seamlessly.

Fig. 1. System architecture and workflow of kBot:

The pipeline starts with the patient conversing with the front-end Android application (client) and data being propagated to the backend server that manages the dialogues. A client communicates with the server through an SSL encrypted socket layer. Patient data captured from the conversation is stored in Elasticsearch, a NoSQL database, in JSON format. A service layer in the server facilitates clients with services like notification, email, data storage, and weather reports. kBot separately stores raw conversation logs with each user in a file for human review.

1). Client:

kBot clients are Google Android applications. We choose the Android platform for its widespread popularity and availability on a wide range of devices and operating systems. Clients are lightweight applications serving as a frontend chat interface that sends raw user inputs to the kBot server. All the computational and data processing tasks are server-centric, and clients receive and display the processed data from the server. The chatroom activity in the client application is the primary interface that patients are provided to converse with kBot. This activity supports text messages as well as rich media contents like images, videos, and hyperlinks. kBot supports text and voice-based interaction between the user and the client. For speech-based communication, the voice input is first captured and then converted to textual data by the client using Android SpeechRecognizer service⁹ prior sending to the server. Upon receiving a server response, a Google Text-To-Speech (TTS) API¹⁰ is used to generate the speech output in a natural-sounding human utterance.

Aside from its system functionality and usability, we recognize the vital role user experience plays in the success of a chatbot application. Hence, kBot user interface is designed according to the best practices laid by chatbot UI/UX guidelines¹¹ to deliver a rich and seamless user experience. Additionally, kBot is iteratively tested against standard (simulated) test case scenarios¹² to identify and eliminate any bugs or idiosyncratic interactions that may arise in real patient scenarios.

2). Server:

On the server side, kBot has a standalone Python web application instantiating and serving multiple kBot clients. Flask, a micro web framework, is used to build the web application¹³. A service layer is defined internally to the server to provide services such as database operation, push notification, email alert, weather report, and file log. Some of these services use third-party APIs to perform the task. Push notification uses Firebase Cloud Messaging (FCM), a Google cloud messaging platform, to send notifications and alerts to the clients. Weather service uses third-party weather APIs like openWeatherMap, Pollen.com, and EPA AirNow to collect weather data. Email service uses Google SMTP protocol to send out email alerts.

The server connects with clients through a standard web-socket which enables real-time data exchange. Web-Socket is a standard communication protocol that offers a full-duplex communication channel over a single TCP connection¹⁴. A secure HTTPS connection is established between server and client using SSL certificates for encrypted data exchange.

B. Creating Better Conversations with Contextualization and Personalization

Contextualization refers to data interpretation in terms of domain knowledge, in this case, the asthma context. It determines the data type and value, and then situates it in relation to other domain concepts, thus developing a meaningful interpretation of results. Whereas, personalization in the asthma context refers to the determination of a treatment plan based on the severity of disease, the prevalence of triggers, and vulnerabilities based on the use of past and current health data. Together, they create better conversations that are meaningful and relevant to the patient which are essential to captivate long-term interest in kBot. For brevity, we describe an instance of kBot conversation (Figure 2) that demonstrates the value of contextualized and personalized knowledge in the chatbot conversation.

Fig. 2.

(a) Generic chatbot without contextualization and personalization (b) kBot with contextualization and personalization

A conversation without contextualization and personalization provides a response to the user question like a question answering system. While in kBot, it warns about high pollen as a potential trigger (contextualization) to patient’s asthma in addition to answering the question (personalized response). kBot uses its past knowledge on the patient to identify pollen as a trigger, checks the pollen level in the patient’s environment, and warns the user if pollen is in harmful range (more on Section III-B3).

1). Contextualization with Knowledge Base:

Asthma knowledge is manually extracted from different online sources such as Asthma and Allergy Foundation of America (AAFA)¹⁵, verywell health¹⁶, Mayo clinic¹⁷, and webMD¹⁸ as well as inputs from our clinical pulmonologist. This information is curated to best represent the domain knowledge (context) and stored in kBot cloud server as a knowledge base in the form of NoSQL database as well as preloaded to DialogFlow as entities (more on Section 3.2.2). Patients can, therefore, ask kBot on asthma-relevant questions to learn more about asthma zones, symptoms, various triggers, medications (their usage and side-effects), and self-management skills.

Apart from asthma domain knowledge, kBot uses rich media contents such as images and videos to deliver and present information more effectively. Images of different asthma medicines and inhalers are used to help patients to quickly identify the various types of medicines, and video contents are used to educate on skills like how to use Metered Dose Inhaler. The image contents are available from A Guide To Aerosol Delivery Devices by American Association for Respiratory Care (AARC) [13], and video contents on inhaler techniques are sourced from Use Inhalers - interactive guidance and training [14]. All the information and knowledge including the media contents are consulted and revised with our clinical collaborators to validate their authenticity and quality.

2). Dialogue Processing, Context and User Intent Analysis:

DialogFlow¹⁹, Google’s developer platform for Natural Language Processing (NLP) and machine learning, is leveraged for dialogue parsing and processing. A list of asthma-related concepts and vocabularies such as symptoms, medication types, and activity limitation types are preloaded into DialogFlow as the knowledge base. This allows DialogFlow to better identify asthma-relevant entities. For example in Figure 3, when a patient mentions cough or wheezing, they are captured and contextualized as report_symptom, and the user intent is then represented as collect symptoms indicating the patient is having asthma symptoms. kBot will then use this knowledge to trigger the appropriate course of actions (see III-B3) such as communicating with the kBot server to retrieve the patient’s data required to personalize the responses appropriate for the current context state. Understanding both context and user intent are essential to maintaining the state of a conversation. The dialog fulfillment task is handled by Firebase Cloud Functions linked to DialogFlow.

Fig. 3.

Data extraction from the conversation in Dialog Flow

A shallow and dull conversation discourages users’ interest in conversing with kBot daily. Therefore, to improve engagement, kBot script is designed and written according to the standard UX design guidelines^20,21. We also work in tandem with clinical collaborators and cognitive psychologist to ensure that the dialogues are encouraging asthma self-management.

3). Personalization in Response Generation:

Before kBot client is provided to the patient, the patient first consents on data privacy. A user profile is then initialized based on the patient’s existing medical record. The patient data in the user profile is anonymized using a pseudonym to comply with HIPAA. These information are stored as a separate patient knowledge base and is continuously updated with data captured in patient-kBot conversations (Section III-B2). As the patient continues to converse with kBot, past captured data is referenced to generate a more personalized and palatable experience. Next, we describe how personalization is implemented in the context of (i) tracking medication adherence and (ii) identifying patient-specific triggers.

a). Medication Adherence:

Once context and user intent are established, for example when the patient reports having asthma symptoms (Figure 2), kBot will check for the medication history of the patient. If there is poor compliance, kBot informs the patient that his/her asthma symptoms might be a result of poor compliance with controller medicine. In another case, kBot may suggest contacting the doctor if rescue medication is used more than four times a day for two days.

b). Patient-specific Triggers:

Each asthma patient reacts differently to environmental triggers. When a patient reports a symptom or asks for weather-related queries, kBot checks the patient’s surrounding environment (as per zip code) for the list of triggers specific to the patient (initialized with anonymized clinical information) that fall in the unhealthy range. These triggers are then ranked based on its co-occurrence score with the patient-reported symptom. Co-occurrence score is a binary score that indicates if a particular trigger was present or absent during a time window of the symptom. The higher the score, the higher the chance of being a potential trigger. A personalized alert is then generated to the patient whenever these triggers are in an unhealthy range^22,23.

IV. EVALUATION

A. Method

A preliminary evaluation to assess technical viability and effectiveness is conducted on kBot. The evaluation criteria are chatbot quality, technology acceptance, and system usability (Table I). Chatbot quality is grouped into three categories: naturalness, information delivery, and interpretability. These assessment criteria are designed in consultation with a cognitive psychologist. The evaluation involved eight domain expert (clinicians) and eight non-domain experts (researchers). Each evaluator is provided with the kBot client application and asked to interact with it separately using a random patient scenario. Patient scenario simulates clinicians experience with real patients. Future studies will evaluate the clinical effectiveness of kBot in children with asthma.

TABLE I.

kBOT Evaluation Metrics

Metrics		Questions
Quality of chatbot	Naturalness	• kBot uses simple and understandable vocabulary. • kBot dialogues were unambiguous. • kBot dialogues were natural.
	Information delivery	• kBot provides patients with the right information at right time. • Information provided by kBot helps an asthma patient manage their asthma better.
	Interpretability	kBot properly understood what a patient intended to say during the conversation. The patients will be able to express their current asthma condition and medication usage accurately through the conversation.
Technology acceptance		• The information kBot is trying to collect through the conversation adequately conveys a patient’s asthma condition. • I recommend this technology to monitor and manage a patient’s daily asthma condition. • Overall, I am very satisfied with this technology.

Illustrative patient scenario:

Patient A is a 12-year-old male with a history of moderate persistent asthma. On a Sunday morning, he started experiencing vigorous coughing with wheezing and shortness of breath. He took a dose of Albuterol inhaler followed by another dose and the symptoms subsided. He reported this episode to kBot. Based on its domain knowledge, kBot suggested the patient to stay on controller therapy and contact doctor if albuterol was taken more than four times a day for two consecutive days.

An online survey form was presented to the evaluators at the end to assess kBot based on their user experience. The survey consists of two sets of questionnaires. The first set to access the technology acceptance and quality of kBot on an 11-point Likert scale ranging from 0 to 10, zero being strongly disagree and ten being strongly agree. The second set as the System Usability Scale (SUS)²⁴ on a 5-point Likert scale ranging from 1 to 5, one being strongly disagree and five being strongly agree.

B. Result

The evaluation responses from both clinicians and researchers for each set of the questionnaires are aggregated and averaged. In the first questionnaire, clinicians rated kBot with a mean score better than 8 on all four metrics (naturalness, information delivery, interpretability, and technology acceptance). For the same metrics, kBot received a mean score better than 8.4 from researchers. A response better than 7.5 on 11 point Likert scale is equivalent to a score better than 4 on a 5 point Likert scale [15]. The detailed score on each metric is shown in Figure 4. In the second set of the questionnaire (SUS), both clinicians and researchers rated kBot with an average SUS score of 83.13 and 82.81 respectively.

Fig. 4.

Mean score and standard deviation for each evaluation metric of kBot (Higher mean score is better).

As described earlier, this evaluation sought to measure technology acceptance. The mean score of technology acceptance obtained from both groups are above 8 out of 10. In addition, we also analyzed the significant difference of this metric within the two distinct groups using t-test. This is performed for hypothesis testing. However, with the p-value of 0.88 (>0.05) (refer Tabel II), we cannot reject the null hypothesis stating that there is no significant difference between the two groups in terms of technology acceptance. Hence, we can conclude that both the group finds this technology useful for self-management of asthma.

V. DISCUSSION AND CONCLUSION

With the digitization of healthcare, a massive amount of patient-relevant data is generated every day. Converting this data into meaningful and profound insights about patient’s health would enhance the healthcare approach. Healthcare professionals are limited by resources to monitor the patients and get insights personally. In such a scenario, technology such as chatbot can closely interact with patients and monitor them engaging personally in their everyday life. Hence, building chatbots for healthcare that collects comprehensive data related to patient and their environment is demanding. However, a generic chatbot system without domain knowledge or patient’s history is improbable to bring any changes into patients health. Chatbots can be far more effective by using patient history and domain knowledge to generate personalized responses. kBot, as such a system, is aware of patients history and has in-depth domain knowledge which enables it to generate a response that is more meaningful and contextually relevant to the patient.

We have successfully prototyped a chatbot system that is capable of interacting with asthma patients with in-depth context knowledge and personalization to monitor their asthma relevant data and help them self-manage their asthma. Nonetheless, kBot is not a medical diagnosis or a decision-making application. The preliminary evaluation shows great acceptance of kBot among domain and non-domain experts as a system for asthma self-management. The next step is to conduct a pilot study on a group of pediatric asthma patients.

VI. FUTURE WORK

As future work, kBot could improve upon a few things. First, different asthma symptoms may reflect varying disease severity level with symptoms such as shortness of breath at rest requires immediate medical attention. Thus, we need to classify the severity level for each symptom and handle patient-reported symptoms separately. Second, to be able to deliver a more human-like conversation, we can design a custom language model and train it on real-life patient-doctor conversation data. In addition to environmental data, we can incorporate various IoT sensors to collect data such as indoor air quality and sleep activity to better understand the patient’s surrounding environment for a more likely prognosis of triggers contributing to the worsening of asthma condition. The current kBot knowledge base can be further enriched with various asthma concepts to answer a broader and wider variety of questions. A demo of our kBot application is available at this URL: https://bit.ly/kBotdemo.

TABLE II.

T-TEST WITH THE P-VALUE FOR TECHNOLOGY ACCEPTANCE COMPARING RESPONSES FROM TWO DISTINCT GROUPS

Difference	0.0833	T Ratio	0.159982
Std Err Dif	0.5209	DF	12.29055
Upper CL Dif	1.2153	Prob >|t|	0.8755*
Lower CL Dif	−1.0486	Prob >t	0.4377
Confidence	0.95	Prob <t	0.5623