How Will the Internet of Things Enable Augmented Personalized Health?

Abstract

The Internet of Things refers to network-enabled technologies, including mobile and wearable devices, which are capable of sensing and actuation as well as interaction and communication with other similar devices over the Internet. The IoT is profoundly redefining the way we create, consume, and share information. Ordinary citizens increasingly use these technologies to track their sleep, food intake, activity, vital signs, and other physiological statuses. This activity is complemented by IoT systems that continuously collect and process environment-related data that has a bearing on human health. This synergy has created an opportunity for a new generation of healthcare solutions.

The paradigm shift from reactive medicine to proactive and preventive medicine is primarily motivated by economic imperatives such as the rising cost of healthcare, as well as continued improvements on quality of life and longevity. According to the Centers for Medicare and Medicaid Services (CMS), in 2016 the cost of healthcare in the US reached $3.6 trillion and is expected to increase to $5.5 trillion by 2025 (www.advisory.com/daily-briefing/2017/02/16/spending-growth). On the other hand, the global smart healthcare industry is expected to reach $169.30 billion by 2020.

It’s also projected that by 2019, 87 percent of US healthcare organizations will have adopted Internet of Things (IoT) technology (www.i-scoop.eu/internet-of-things-guide/internet-things-healthcare), of which 73 percent will be used to reduce cost, and 64 percent will be for patient monitoring.

IoT data itself isn’t adequate to understand an individual’s health and associated aspects of well-being and fitness; it’s usually necessary to look at that individual’s clinical record and behavioral information, as well as social and environmental information affecting that individual. Interpreting how well a patient is doing also requires looking at his adherence to respective health objectives, application of relevant clinical knowledge and desired outcomes, such as the patient’s preference for quality of life versus longevity and expert knowledge.

Augmented Personalized Healthcare (APH) is a vision (http://wiki.knoesis.org/index.php/Augmented_Personalized_Health:_How_Smart_Data_with_IoTs_and_AI_is_about_to_Change_Healthcare)¹ for exploiting the extensive variety of relevant data and medical knowledge using artificial intelligence (AI) techniques to extend and enhance human health and well-being. It anticipates the use of physical, cyber, and social data obtained from wearables and IoT devices; clinical information including electronic medical records (EMRs); mobile applications supporting targeted interactions and engagement with the patients; and web-based information including web services (such as those providing health-relevant data on allergens and air quality), social media (such as posts by patients with similar concerns and conditions), and extensive online knowledge bases of clinical practice and medicine. Data can be collected at the personal, public, and population levels, and be combined with knowledge that affects human health. Augmentation refers to aggregating this data and converting into actionable information that can improve health-related outcomes through better and more timely decisions. This embodiment of APH is an entirely new approach to human healthcare in comparison with the current episodic system of periodic care primarily centered around healthcare establishments (such as clinics, hospitals, and labs).

APH involves continuous monitoring, engagement, and health management in which, instead of treating a patient for a disease, the focus shifts to involving the patient in preventing disease, predicting possible adverse outcomes and intervening to mitigate or eliminate them through proactive measures, and trying to keep citizens healthy and fit with continuous lifestyle changes. Rather than only focusing on the management of chronic conditions, APH proposes a holistic approach for improving the overall quality of life.

Patient-generated health data (PGHD) is the heart of APH. It’s primarily generated by IoT devices and captures the digital footprint representative of patients’ health over time with finer details that are distinct from the data generated in clinical settings through EMRs and personal health records (PHRs). The two main IoT categories for patient health monitoring are wearable sensors and environmental sensors. Wearable sensors are portable sensors that patients wear most, if not all, of the time. These close-vicinity wearable sensors monitor patients’ physiological markers, such as heart rate, breathing rate, and blood pressure. They’re designed to integrate into patients’ daily routines to enable passive and continuous sensing and monitoring for timely interventions. Environmental sensors, on the other hand, are sensors that collect environmental data relevant to patients. These sensors are normally not portable, but can sometimes provide critical information for health management. For example, weather data, such as humidity, pollen index, and air quality, are important for managing asthma. However, this data provides a coarse population-level measure, and wouldn’t account for the differences of individual patients. To mitigate this, different sensors can be utilized as a complementarity. For example, Foobot (https://foobot.io) monitors indoor air quality and reflects a closer overview of a patient’s environment. Hence, these sensors enable personalization and allow both physicians and patients to monitor asthma at a finer level.

STAGES OF TECHNOLOGY-ENABLED HEALTH AUGMENTATION

In this section, we review various stages of augmented health management strategies using APH technology.

Self-Monitoring

Currently, doctors see patients infrequently or as needed for new conditions; or, for people with chronic conditions and disease, they are at well-defined time intervals (monthly, quarterly, etc.), depending on the established medical protocols and severity of the medical condition. A doctor’s understanding of a patient’s condition often comes primarily from the patient’s self-description (self-reporting) in addition to the observations gathered by the clinician during the visit. This has limitations, of course, as sometimes not all significant events or issues are recalled at the time of the visit. In addition, the exact timing, location, and reasons for the triggering event might not be available. With continuous monitoring using IoT-enabled sensor devices, wearables, and a periodically administered contextually relevant questionnaire, we can better capture relevant aspects of a patient’s surroundings, diet, activities, and other factors related to health. All of these aspects, when analyzed, can help to determine possible and precise contributing factors of patients’ conditions or level of well-being. PGHD plays an important role in supplementing existing avenues for collecting clinical data and filling in information gaps on a routine basis, thus generating a more comprehensive picture of long-term patient health (www.healthit.gov/policy-researchers-implementers/patient-generated-health-data).

Self-Appraisal

Self-appraisal describes the patient’s ability to evaluate the relevance of a variety of data and observations within the context of his or her general health objectives or specific health concerns. Wearable devices are used to keep track of patients’ day-to-day activities. However, there’s a big gap between simply having the access to relevant data generated from self-monitoring and being able to analyze and interpret the data in a useful way. Patients are interested in understanding if they are keeping up with progress toward their health goals. Consider, for example, using a Fitbit² to measure the number of steps taken each day and the quality and duration of sleep. Is this data helping the patient fulfill their desired objective or do they need to do something more? What is the distinction between expending 1,700 calories and 2,200 calories per day vis-à-vis the objective to shed 5 pounds in next 3 months to improve the management of diabetes? What about the existence and impact of any abnormal behavior on body activity? For example, for patients taking asthma meds, if their Fitbit shows a heart rate of 100+ while asleep, is that a serious enough condition to require clinical consultation?

Self-Management

Self-management refers to the patient’s decisions and behaviors that impact management of their chronic conditions. Generally, the impact that a patient may seek is getting back in line with the prescribed medical care plan or agreed upon health objective. Patients empowered with IoT-generated PGHD have a better sense of their health condition and can make informed decisions about care as opposed to episodic clinical visits in which patients aren’t aware of their state until diagnosed. An APH technology that intends to support self-management is expected to identify actionable information, such as increasing weight-bearing exercise or reducing consumption of energy-dense food. An APH technology can aid patients by providing alerts about potential triggers (such as high pollen counts) or feedback on adherence (such as unexpected weight gain or not meeting activity targets), which can be used to keep patients on course. This improves the effective use of IoT for both data collection and relevant data/analysis/alert access by the patient. It can then provide alternatives to the patient to take steps to better adhere to the physician-specified care plan to reduce adverse impact due to deviation from the plan or improve the outcome of the objective (for example, an APH technology used to promote self-management for patients who are obese can use IoT capabilities, such as activity monitoring and fluid consumption to also measure increases in activity level and targeted water intake if weight gain continues after use of oral steroid has ended).

Intervention

The next step up in health management is clinical intervention, which includes a change in the care plan prescribed by the clinician. An APH technology can use the data it gathers to help clinicians provide PGHD, environmental, and other data as well as corresponding analysis and interpretation to help evaluate and adjust a patient’s clinical plan. The timely analysis of IoT data can yield insights for early intervention before a patient’s situation deteriorates. In the case of kHealth Asthma, which developed an APH technology for managing asthma in children, the observed deterioration of asthma symptoms through PGHD can suggest change in medication or its dosage, develop trigger-avoidance plans, and so on. The IoT data collected from individuals can help clinicians develop a personalized, patient-centric recommendations for use in the healthcare system with implicit feedback and support adherence to physician-prescribed protocols.

Disease Progression Tracking and Prediction

Going beyond immediate and short-term management of health concerns, it would be highly rewarding on the individual and public health levels if the longitudinal collection of personalized health data including PGHD and environmental data could facilitate tracking how a disease is progressing, predicting significant changes in health status, and identifying and taking remedial actions. For example, for a patient who is pre-diabetic and has an A1C score higher than 6, it would be highly valuable to be able to track the score’s changes and issue an alert when it has reached diabetic status (A1C ≥ 6.5), thus predicting the high probability that a patient becomes diabetic and requires insulin treatment. For an asthmatic patient who is overweight and is on long-term steroid medication—which may lead to a number of adverse situations including higher energy intake—it would be important to track associated weight gain and compute the probability of worsening of asthma severity. A more straightforward strategy would be for the clinician to periodically review the patient’s data and make an educated judgment on the disease progression. A more advanced strategy would be to use the personalized health data and analyze it vis-à-vis published clinical studies and longitudinal data collected with relevant cohort population.

The objective is to devise more proactive interventions and incorporate more nonmedical solutions, such as lifestyle changes that are often very effective but take a longer time to show benefit. Individual and public health will greatly improve based on what we learn from such strategies, which will become evidence-based enhancements of widely accepted clinical pathways and protocols.

APH SHOWCASE AND APPLICATION SCENARIO

The knowledge-enabled healthcare (kHealth) initiative at Kno.e.sis is an example of an APH framework to enhance decision making and improve health, fitness, and well-being (http://bit.ly/kAsthma). The early prototyping and testing involved kHealth–ADHF, a mobile app and a sensor kit designed to reduce readmission in patients with Active Decompensated Heart Failure (ADHF). kHealth–ADHF involved continuous monitoring using targeted questions driven by application-specific (cardiovascular) knowledge as well as sensors to record blood pressure, heart rate, and body weight. These measurements provided observational data via Bluetooth to the mobile app, which also asked the patient pertinent questions and analyzed answers and collected data and then generated alerts.

A follow-on application, kHealth–asthma, designed to better control asthma in children, extends physical data collection using cyber and social data. Figure 1 shows an instance of the kHealth–Asthma application collecting multimodal data to monitor pediatric asthma, which is a multifactorial and multifaceted disease. We are running a trial with 200 patients with asthma, collecting possibly the broadest modality of data, with an average 124 readings collected per day (2 tablet readings per day, 24 Fitbit readings per day, 2 Peak flow readings, 96 Foobot readings per day), for a duration of 1, 3, and 6 months. This vast amount of multimodal and multisensory data poses a big data challenge (due to the data variety and associated challenges with integration) in comparison to other mHealth studies like Google Verily, IBM, and Swiss startup Docdok.health project, as well as the Stanford wearable study that deals with fewer modalities and a smaller sample size. To address the aforementioned problem, kHealth supports contextual (condition-specific) annotation, integration, and interpretation of sensor data using Semantic Sensor Network (SSN) ontology. Furthermore, kHealth supports contextualized actionable feature selection in PGHD to generate Smart Data using SSN and domain-specific knowledge sources. Utilization of Smart Data provides timely medical intervention and remediation measures. In a broad sense, a knowledge graph is a knowledge base that provides semantic annotation using its characteristic functionalities like fact extraction, named entity recognition, relationship identification, locale-specific information, event extraction, and intent identification to enrich information.

Figure 1.

kHealth, an mHealth application that gathers patient-generated health data (PGHD) through contextually relevant questions (tablet), sensors (Foobot, peak flow meter), wearable (Fitbit), and from external data sources for contextual (such as location-specific) environmental data.

We describe the kHealth APH approach with an example. Sara is a 10-year-old girl. With the help of our kHealth kit (Figure 1), she is able to monitor her daily activities, helping her and her clinician to intervene and update her treatment plan accordingly. Self-monitoring refers to the data collection using the mobile devices and sensors. Sara completes her mHealth application questionnaire twice a day, collects her daily activity level and sleep pattern using Fitbit² (http://bit.ly/1VdkW3I) and places an indoor air quality monitor to measure her indoor environment at home. Self-monitoring in many cases, won’t be helpful if collected data aren’t acted upon. In general, we don’t want a technology to make a clinical decision and change the care plan, but to enable adherence to a care plan specified by the patient’s physician. Self-appraisal refers to the process of self-monitoring and self-reconciliation of the observed data. Self-management helps a patient to make better judgment or action within the scope of the care plan. For instance, a patient might observe that every time he goes out and has allergic symptoms, the APH systems has found a strong correlation with high pollen. The intervention involves looping in the clinician for monitoring the severity level of patient and alter the care plan. Activities involved in the intervention may include the addition of new medication, changing the course of intake of medication or suggesting preventive medication considering the historical observations of the patient by involving the clinical and support services in care and health monitoring process. Assessment of post-intervention processes is crucial for reclassification of patient’s disease. For instance, the collected evidences can help the physician to reclassify the asthma from mild persistent to moderate persistent and adjust the care plan with modification in the medication.

CHALLENGES IN CONVERTING BIG DATA INTO SMART DATA

A variety of studies involving PGHD and other health-relevant data using a broad variety of IoT are ongoing for developing personalized digital care solutions for a variety of health related objectives, as shown in Figure 2. These systems need to deal with a host of data related challenges such as accessing, storing, querying, and managing large volumes of highly dynamic data, and systems related challenges such as interoperability and integration, security, privacy, trust, scalability, and reliability.³ We characterized the ongoing efforts along two critical dimensions: abstractions (making the data meaningful and interpretable with respect to an individual’s health) and actionable information (supporting decision making and actions informed from the data). These involve addressing challenges in data analysis including semantic data modeling, annotation, knowledge representation (for example, modeling for constrained environments, complexity issues, and time/location dependency of data), and so on. While statistical analysis of the data collected helps one identify correlations, it’s widely observed that a correlation doesn’t necessarily imply a causation. Working with domain experts (that is, clinicians for the health applications) for understanding correlations between observations from different modalities is the key in associating meaning to the variations in observations that can then support derivation of causations. Another challenge is that the clinicians, health practitioners, and patients cannot keep up with an enormous amount of data being generated. Patients can’t interpret the data in the context of health conditions and objectives and clinicians don’t have time to look at it. There’s an urgent need to convert the raw data into Smart Data. By making sense out of big data (http://j.mp/SmData), Smart Data provides value from harnessing the challenges posed by volume, velocity, variety, and veracity of big data, in turn providing actionable information and improving the decision-making process. Smart Data is focused on the actionable value achieved by human involvement in data creation, processing, and consumption phases for improving the human experience (http://bit.ly/HumanExperience). We propose the following evidence-based semantic perception approaches: (a) contextualization, (b) abstraction, and (c) personalization.

Figure 2.

Example efforts involving PGHD and other health-relevant data. Converting PGHD data into actionable information. [References for kHealth Bariatrics,⁵ kHealth ADHF,⁶ kHealth Dementia,⁸ TIHM⁷ provide further details.]

Contextualization

Contextualization refers to data interpretation in terms of knowledge (context). PGHD consists of demographic and medical information from EMRs and time-series data collected from various environmental sensors, physiological sensors, and public web resources. Contextualization supports ranking a patient’s diagnosis and patient similarity based on demographics and PGHD. It deals with low-level fine-grain data covering various facets by determining the data type and value, and then situates it in relation to other domain concepts, thus developing a meaningful interpretation of results. A large body of existing research on ontologies and semantic web techniques and technologies can be leveraged for this purpose.⁴ However, relying solely on description logics or formal knowledge representation alone is often not sufficient to understand the complex nature of many health conditions. Probabilistic graph models from representing knowledge graphs, combined with machine learning and NLP on relevant data is an alternative some recent approaches have used.

Abstraction

Abstraction is a computational technique that maps and associates raw data to action-related information, taking into account personal details but ignoring inessential differences to provide an integrated view of proper remediation measures. For example, high activity translates to different workout durations based on age, weight, current health, weather, and sport; or a low risk of heart problems depends on demographic and genetic information, as well as diet. In some cases, abstraction can be embedded on the device: for example, question-answering systems in mobile health applications as a way of indirectly supervising personalization of healthcare. However, one of the challenges is the need to formalize normalcy and detect an anomaly. Anomaly detection is nontrivial because the notion of normalcy itself is intrinsically dynamic, based on spatiotemporal and personal context. It also requires personalization and the ability to uncover various correlations among multimodal data streams and discovering medically relevant abstract interpretations and the factors that influence them. The challenge itself can be overcome if sufficient patient data can be obtained through large-scale clinical studies, followed by identification of correlations, and then analyzed and explained by those with domain knowledge and expertise to derive causations.

Personalization

Personalization in healthcare refers to the determination of a treatment plan based on severity of disease, the prevalence of triggers, and vulnerabilities vis-à-vis the use of past and current health data. For example, a low-dose SABA (Short-Acting Beta Agonists) might help someone keep asthma symptoms in check during the fall season, but it might not work for another patient who needs a higher dosage due to more severe asthma and a greater prevalence and intensity of triggers during the spring season. IoT data provides an opportunity for personalization of future course of action and treatment plans by taking into account the contextual factors such as patient’s health history, physical characteristics, environmental factors, activity, and lifestyle.

With contextualization, abstraction, and personalization in place, the next problem is how to synthesize a personalized vulnerability score for a patient’s given medical condition or disease with respect to relevant health management objectives to better establish a control level, and to quantify and express the effectiveness of remedial measures in a manner readily accessible to both patient and clinician.

CONCLUSION

In terms of IoT and health, most current efforts are focused on data collection and improving understanding what the data implies at a basic level. Collected data needs to be analyzed and validated with EMRs that capture patient-care objectives, plans, and information self-reported by patients. The key aspect is generating actionable information that will be acceptable, easy to use and integrate into clinical pathways used by other systems, and can be given at the right time with the right modality to the end beneficiaries (clinicians and patients) with appropriate information governance procedures and privacy/security measures. The privacy issues in healthcare data can be dealt with by homomorphic encryption schemes, differential privacy, and data perturbation. Depending on the privacy level needed, additional cryptographic services can be introduced into the framework.

Transitioning from cohort-based treatment to more personalized treatment, basic statistical computing with causality and machine learning algorithms won’t suffice. There’s a need to combine and integrate machine learning and data analytics with reasoning engines and knowledge bases, thus propelling us into the realm of augmented personalized healthcare management and well-being applications and services.

Biographies

Amit Sheth is the LexisNexis Ohio Eminent Scholar and the executive director of Kno.e.sis, the Ohio Center of Excellence in Knowledge-enabled Computing and BioHealth Innovations at Wright State University. His research interests include physical-cyber-social big data and computing for human experience, supported by knowledge-empowered artificial intelligence (AI) techniques. Sheth received his PhD from Ohio State University. He is an IEEE and an AAAI Fellow. Contact him at http://knoesis.org/amit.

Utkarshani Jaimini is a PhD candidate in computer science at Kno.e.sis Center at Wright State University. Her research interests include applied machine learning to solve sustainability and healthcare problems, predictive analytics, information extraction, augmented personalized healthcare, and the semantic web. Jaimini received her bachelor’s degree in computer science engineering from The LNMIIT University. Contact her at utkarshani@knoesis.org.

Hong Yung (Joey) Yip is a PhD student at Kno.e.sis Center at Wright State University. His research interests include conversational artificial intelligence and semantic web and their applications to augmented personalized health. Yip received his MS in bioinformatics from University of Malaya. Contact him at joey@knoesis.org.