Using Smart City Technology to Make Healthcare Smarter

Abstract

Smart cities use information and communication technologies (ICT) to scale services include utilities and transportation to a growing population. In this article we discuss how smart city ICT can also improve healthcare effectiveness and lower healthcare cost for smart city residents. We survey current literature and introduce original research to offer an overview of how smart city infrastructure supports strategic healthcare using both mobile and ambient sensors combined with machine learning. Finally, we consider challenges that will be faced as healthcare providers make use of these opportunities.

I. Introduction

Ongoing population growth and urbanization are sparking a renewed desire to integrate technology into the design of city services, thus creating the essence of “smart cities”. This renewed focus has resulted in the use of information and communication technologies (ICT) to scale up critical urban support for larger communities including transportation [1]–[3], energy systems [4], [5], crime-sourcing [6], [7], and emergency response [8].

Smart cities rely heavily on sensors to perceive parameters such as temperature, humidity, allergens, pollution, traffic conditions, and power grid status. The values of these parameters provide a context that helps a system to understand the state of a citizen at any given time [9]. Strategically responding to sensed data helps heathcare be smarter. By gaining real-time access to this information, city services can respond promptly to urgent health needs and make decisions to avoid unhealthy situations.

The maturation and adoption of computing technologies have dramatically changed the face of healthcare. Figure 1 illustrates these changes. We can describe each of these approaches to healthcare based on three characteristics: the size of the group that is analyzed, the use of ICT, and the nature of the data.

Fig. 1.

A timeline illustrating the influence of ICT and community in healthcare.

Traditional medicine consists of a physician examining an individual patient to generate a diagnosis and recommend a treatment. Instead of using computing technologies to do this, the doctor relies on previous training and experience.

Electronic health records (EHRs) and personal health records (PHRs) made an appearance in the early 2000s and even influenced government decisions on where to invest healthcare funds [10]. While doctors do not typically analyze real-time streaming e-health data, data mining these historical records allows physicians to examine conditions that are common across entire subpopulations and to understand health trends [11], [12]. An estimated 55% of physicians now make use of EHR and PHR resources [13].

In 2006, Istepanian et al. [14] predicted the potential impact of mobility on healthcare services (m-health). With the introduction of mobile devices and body area networks, mobile device owners self-monitor their physiological variables in real time using mobile sensors and ICT. Additionally, care providers use this information to overcome geographic and temporal barriers and thus more effectively prescribe medical treatments and behavioral changes [15].

As an alternative to embedding sensors on personal devices, sensors are now also embedded into physical environments. In the past few years, continuously-collected data in ambient intelligent environments (a-health) has been used to look for changes in health status and design in-the-home health interventions [16]. These ambient assistive environments do not require any interaction or wearables on the part of the user but do have to overcome the possible challenges of monitoring multiple people at once [17].

This ICT foundation paved the way to consider smart city-based healthcare (c-health). ICT infrastructure throughout a city can offer a more global view of the health status of community residents and insights on the relationship between city services and health provisioning.

In this article, we look at the technologies that can form a foundation for smarter healthcare using smart city ICT. We examine techniques that analyze data collected from mobile and ambient sensors for health assessment and intervention and survey representative work in these fields. We then look at the current state of the art in c-health and discuss challenges for ongoing research and development in this area.

II. Mobile Sensor Data Collection and Analysis

Smart cities can pull information from many sources. These include the information sources listed in the previous section such as mobile device sensor data and ambient sensor data. Additionally, data can be tapped from city-wide sites such as power grid status, transportation grid status, vehicular networks, locations of emergency service providers, and size of crowds in locations throughout the region. Here we begin by describing the data that is collected from mobile devices and how it can be used for personal healthcare as well as healthcare of an entire smart city.

A. Mobile Sensors and Features

Smartphones and watches come equipped with many sensors. A number of apps such as the AL activity learner [18], illustrated in Figure 2, are available to gather this information. As Figure 2 shows, sensors that are common to these devices include accelerometers to measure movement in three axes relative to the device and gyroscopes to measure rotation about those axes. These devices commonly also collect location information (latitude, longitude, and altitude) using a combination of GPS, Wifi, and GSM sources, depending on whether the device is inside or outside of a building. Related information includes the compass heading of the device (course) and the current speed. The compass heading is determined by magnetometers while barometers generate altitude values.

Fig. 2.

AL activity learning app collecting real-time sensor data.

Depending upon the actual device that is used, additional information can be collected. Many devices have cameras and microphones that provide a dense source of data indicative of the state of the user and environment. Use of other apps on the device, including phone calls and texting, can be captured. Table I lists sensors that are commonly found on mobile devices and wearable devices.

TABLE I.

Mobile / Body Sensors

Sensor	Measurement
Accelerometer	acceleration in x/y/z directions
Gyroscope	rotational velocity
Location	latitude, longitude, altitude
Camera	surrounding image / video
Compass	orientation
Microphone	surrounding audio
App status	usage of apps, phone, text
Photodiodes	heart rate
Glucometer	blood sugar
Barometer	atmospheric pressure
Carbon dioxide (CO2)	CO2 concentration
Electrocardiography (ECG)	cardiac activity
Electroencephalogram (EEG)	brain activity
Electromyography (EMG)	muscle activity
Electroculography (EOG)	eye movement
Force	screen touch pressure
Light	ambient light level
Proximity	nearness to external object
Pulse oximetry	blood oxygen saturation
Galvanic Skin Response (GSR)	perspiration
Thermal	temperature

While these sensors are standard for most mobile devices, other health-related insights can be suggested by specialized devices and mobile device attachments. As an example, smart watches offer LEDs and photodiodes that utilize light to monitor heart rate by detecting correlated changes in blood flow, while glucose meters can plug into phones to monitor blood sugar.

When sensors are placed on, in, or around a physical body then data can be collected using a Body Area Network (BAN) [19]. Generally, body area networks are wireless personal area networks that act as gateways working together with small sensors and control units to collect data [20]. While smart watches can be integrated into a BAN and placed on the body, BANs can also utilize implanted sensors as well as sensors that are near to the body but do not touch it. A popular type of implanted sensor is the electrochemical glucose sensor that is used for the management of diabetes [21], but there exists a wealth of sensors that have been implanted for monitoring conditions such as rheumatoid arthritis, sleep apnea, heart arrhythmias, and cranial pressure [22].

The first step in analyzing mobile sensor data is to extract features from the raw data. Features not only create descriptive statistics but they contribute a context for the setting in which the data was produced. Most sensors generate updated readings at constant time increments (e.g., 30 times a second) and features are extracted from a fixed-length sequence of raw sensor values.

Table II summarizes features that are commonly extracted from mobile sensor values. In addition to standard signal processing features, higher-level information about the sequence of data as a whole represents valuable context, including time and date features and trajectory features [23]. The features can be used to analyze behavior patterns for device users.

TABLE II.

Standard Mobile Data Features

Type	Features
Time	date, day of week, weekday / weekend, days past January 1, time of day, hours / minutes / seconds past midnight
Statistical	max, min, sum, mean, median, standard / mean absolution / median absolute deviation, zero / mean crossings, interquartile range, coefficient of variation, skewness, kurtosis, signal energy, log signal energy, power
Relational	correlation (between axes / variables), autocorrelation
Trajectory	heading change rate, stop rate, sequence overall trajectory, normalized distance to user mean location
Phone / App	currently in use, use / call time for current day, number of bouts / calls for current day, elapsed time since most recent use / call
Physiological	pulse, respiration, blood glucose, blood pressure

Additionally, machine learning algorithms can be used to map the vector of features onto activity labels [24], [25]. These labels then create a vocabulary to express routine behaviors and changes in these behaviors. An activity recognition algorithm learns a mapping from a sequence of sensor readings to a corresponding activity label. More formally, let A = {a₁, a₂, …, a_T} be the set of T activities, where a_i corresponds to the i^th activity class. Given a sequence of n observed sensor readings, <r₁ r₂ ‥ r_n>, a feature vector X is extracted from the sequence. In order to learn a model of activities, individuals need to use an app such as AL to answer occasional queries about the activity they are currently performing. The user-specified label and corresponding sensor data represent training data that can be used to learn the class of activities. The extracted feature vector and user-provided label are input to a machine learning classifier, which learns a function h that maps the feature vector onto an activity label, h:X→A. Note that activity learning can be considered as part of the feature extraction process because the generated activity labels become a component of a feature vector that describes a person’s behavior over time and can be mapped to the person’s health status.

In addition, it is not necessary to use only one sensor platform at a time to monitor behavior and provide smarter health assessment and intervention. Different sensors provide different types of insight: accelerometers may indicate the type of movement the user is performing and at the same time, light sensors explicate the surrounding environment conditions. Particular sensors may also be chosen based on their battery consumption profiles and ability to extend battery life through energy harvesting [26]. Varying information sources can be combined using data fusion to learn a more robust model [27]. Alternatively, models learned using one sensor platform can be mapped to another sensor platform using a technique called transfer learning [28], which can reduce or eliminate the need to train models for each new type of sensor device or collection parameters.

B. ICT-Driven Healthcare at a Personal Level

Mobile ICT can support health monitoring and intervention at multiple scales ranging from personal data collection to an entire city and beyond. At the individual level, mobile devices have become a mainstay for personal healthcare. Recent statistics report that 52% of smartphone users gather health-related information on their phones and 61% of users have downloaded an mHealth app [29]. Most commonly, people search for insights on a medical or insurance problem, but users also look for hints on nutrition, fitness, drugs, and doctor choices.

In addition to investigating specific medical issues, another popular personal use for mobile and wearable ICT is step counting, which provides a foundation for many fitness apps. Mobile devices and apps infer step counts from the 3D accelerometer signals. While there can be a lack of uniformity among alternative step counting devices, most of the disagreement is due to the wearing site of the tracker rather than the embedded signal processing algorithm that calculates steps from the accelerometer data. Studies have shown that these devices perform quite similarly and are reliable for normal conditions [30], although they do experience performance degradation when the person moves together with an accessory (e.g., walker, shopping cart) or performs vigorous non-walking activity near the mobile device tracking site.

An advantage of mobile ICT-driven healthcare is that continuous monitoring of behavioral patterns facilitates detection of subtle disease symptoms that are otherwise difficult to observe and associate with diagnoses. As an example, older adults may experience cognitive decline but because they still retain a high degree of autonomy this change may be difficult to catch and treat. However, early stages of dementia are associated with frequent bouts of spatial and temporal disorientation and an increased likelihood of not finishing important daily tasks [31]. These changes translate into abnormal mobility patterns. The SIMPATIC project [32] analyzes these mobility patterns to communicate detected abnormalities to patients and care providers. As shown in Figure 3, warnings can be transmitted to caregivers based on individualized rules or based on movement patterns that are unusual for the user. In the case of wandering, mobile guidance also guides the individual back to their home.

Fig. 3.

An example of m-Health through the SIMPATIC project [32]. Here warnings are provided for no movement or unusual speeds, depending on the patient current location.

Another disease with subtle manifestations is depression. In some cases, symptoms are too faint for a person to note. With ICT-based psychiatry, changes in behavioral patterns such as lower activity levels, degrading sleep, decreased phone conversations, and even mobility patterns may point to a possible diagnosis of depression [33], [34]. Mobile healthcare intervenes when these changes are detected by recommending that the user contact a health care professional. ICT-based assistance can further extend from diagnosis to intervention by monitoring treatment compliance and medication effects [35].

C. ICT-Driven Healthcare at a Community Level

Moving from ICT-based individual monitoring to community-level monitoring, mobile devices again play a large role in this effort. Doctors keep hospital and office note records to assist with patient diagnosis. Electronic health records allow physicians to access additional records not only for the individual but for a population of individuals with similar health conditions. However, with the advent of mobile devices, each person has enhanced access to more rich, real time, granular data about themselves and, if this information is made public, about others in the community.

Citizen sensing and crowdsourcing allows diagnosis to become a community effort and intervention to be boosted by a community support system. People with serious and chronic illnesses turn to social media to share their illness experiences as well as to seek and offer support [36], [37]. For some of these individuals, physically attending support groups is not practical but they find a sense of community in online settings.

Social media outlets can play an even more central role in smarter citywide healthcare. Researchers have detected influenza epidemics based both on individuals posting symptoms [38] and querying about symptoms [39]. Similarly, the wording and content of Twitter posts have been used to infer heart disease mortality at a county level [40] and obesity at a country level [41].

A study by De Choudhury et al. [42] examined social media usage over an entire year and used the data to identify individuals that were vulnerable to depression. Ground truth labels were obtained from individuals in the cohort who were diagnosed with depression at some point during the year-long data collection period. By comparing individuals diagnosed with depression and those without this diagnosis the researchers discovered differences in behavior. Differences include lowered social activity, more indicators of negative emotions, high attention on themselves and increased concerns about sickness and relationships. Figure 4 shows an example of one difference that was discovered between the two groups: participants in the non-depression group do not use social media extensively late at night and increase their use of social media throughout the day. In contrast, individuals in the depression group maximize their social media usage late at night and do not use it as much during daytime hours.

Fig. 4.

Number of Twitter posts made hourly throughout a day (mean plots and least squares trend fit) for individuals in two classes: depression and non-depression [42].

In addition to comparing behavior between the two participant groups, De Choudhury et al. also used a Support Vector Machine (SVM) classifier to predict individuals who would be diagnosed at a future time with depression. The feature vector input to the classifier included the following Twitter usage statistics:

• Mean of usage frequency X over N days.

• Variance of X over the observed N days.

• Mean momentum, which compares each M=7 day time period to the previous time period using Equation 1, where t represents data for one day.

  $$(1/N){\sum }_{t}X(t)-(1/(t-M)){\sum }_{(M\le k\le t-1)}X(k))$$ (1)
• Entropy, which computes the uncertainty in the sequence of usage frequencies based on Equation 2.

  $$-{\sum }_{t}X(t)\text{log}(X(t))$$ (2)

This method yielded an average predictive accuracy of 70% and a precision of 0.75. Early detection of problems such as depression, influenza outbreaks, and obesity will allow communities to take steps to prevent and treat these pervasive health issues.

In Louisville, Kentucky, mobile ICT combined with citizen sensing helped the city to respond to asthma triggers and thus circumvent possible long-term chronic conditions for its residents. In 2014 Louisville was ranked the 16^th most challenging city for people with asthma [43]. To identify where asthma triggers might be located throughout the region, sensor-enabled inhalers were distributed to asthma sufferers throughout Jefferson County. When the inhaler is used, the use is recorded on a smartphone app and the sensor monitors the nearby air for particulates that might be triggering the episode.

Figure 5 illustrates the asthma hotspots that were located throughout the county using this mobile ICT. The sensor data spotlighted one particular road where inhaler use was three times as high as throughout the rest of the city. The city was able to respond to this ICT-provided information and planted trees that separate the congested road from residential neighborhoods. The result was a 60% decrease in particulate matter, addressing a major contributor to breathing problems in the geographic area.

Fig. 5.

Map of asthma hotspots collected by Propeller inhaler sensor and smartphone app [43].

D. Mobile Sensor-Based Analysis of the Relationship between City Planning and Walkability

In this section, we describe our study that highlights the impact that city planning can have on behavior and health. Specifically, we investigate the relationship between a community’s “walkability” and a resident’s behavioral routine. The impact of the built environment on lifestyle choices and resulting health is increasingly noticeable [44]. One quantifiable measure for urban design is neighborhood walkability [45]. Commercial tools compute the walkability index for a location based on distance to nearby amenities including schools, parks, stores, community centers, and restaurants; then it is adjusted to account for pedestrian friendliness and bicycle friendliness. The impact of walkability on lifestyle and health has been discussed in detail [46], [47]. While early studies often rely on self-report, mobile technology allows smart city planners to analyze the impact of design choices using sensor data and automated activity monitoring.

In this study, we utilize information from smartphones to monitor daily routines in combination with neighborhood walkability for a population sample. Machine learning-based activity models label the captured sensor data with activity names and data mining techniques are used to analyze the relationship between walkability, activity, and demographics. Study participants are drawn from the Washington State Twin Registry and include 187 female pairs and 77 male pairs. The participation of twins in this study allows us to control for genetic background by comparing behaviors and health between twins. Twin participants supplied demographic information, height, weight, body mass index (BMI), and home address. The walkability score for each home was generated via walkscore.com and are close to uniformly distributed between 5 and 100.

To generate continuous monitoring information, participants wore GPS devices for 1–2 weeks while performing normal daily routines. Reverse geocoding was applied using the OpenStreetMap Nominatum tool [48] to categorize each visited location as either house, place of worship, recreation, restaurant, road, school, service, store, work, or other.

As shown in Figure 6, the 3,509,133 collected GPS events are mapped to activity labels using the AL activity learner [49]. Previous evaluation of AL indicates that activities are labeled with an accuracy of 89% based on 3-fold cross validation. AL results are based on a feature vector of dimensionality 328 which includes the time, statistical, relational, and trajectory features shown in Table II. The modeled activities include drive, eat, errands, exercise, hobby, housework, sleep, and work. This method outperforms random guess which would yield an expected accuracy of 12.5%.

Fig. 6.

Walkability data mining process.

The participant demographics, activity-labeled GPS events, and neighborhood walkability scores are collected. These are combined with the fraction of a participant’s time spent on each activity and location type, the average size of the community region visited by the participant on a daily basis, the time spent near home (proximal) and the time spent far from home (distal).

Analyzing activity routines and home-based walkability scores, relationships between these parameters does become apparent. In particular, there is a Cohen’s moderate-size correlation between the walkability of a resident’s home location and the time they spend exercising (r=.40) as well as the time they spend working (r=.36). While both of these activities have an established connection with health and overall well-being, the small-size direct correlation between home walkability and a participant’s BMI (r=.28) yields additional evidence of the effect of city design on health.

We separately consider genetic influences on lifestyle and BMI. To do this, we introduce the notion of twin-sample permutation tests, which represent a variation of the permutation tests defined by Ojala and Garriaga [50]. This test allows us to determine whether the aforementioned results are obtained because of chance. With this permutation-based evaluation, we calculate a p-value to test a null hypothesis about the relationship between the features and the class labels. This p-value is calculated as a fraction of times that the correlation obtained by shuffling (permuting) the class labels exceeded the correlation on the original dataset. The permutation test determines whether we found a correlation that truly exists in the data by randomly shuffling class labels among the data points while maintaining relative label frequencies.

We adapt this test to our twin participants by first shuffling the walkability scores from one twin to the other for a randomly-chosen 50% of the twin pairs. We then re-calculate correlations, repeating the entire process 100 times. A t-test is used to determine whether the difference in correlation values using the original data and using the permuted data is statistically significant.

The null hypothesis (H) is that there exists no relationship between the data features and the class labels. Based on the statistically significant p-value (p<0.05) we fail to reject the null hypothesis H. Thus, we conclude that there does exist a relationship between the detected activities (including exercise and work) and BMI and that the correlations are not obtained by chance.

The results of this study offer insights that city planners can use to generate urban analytics and modify city design to improve the health of city residents. Similar ICT-based methods can be utilized for additional analytics including transportation routing and provision of resources such as electricity and water.

Mobile ICT plays a major role in today’s healthcare. This technology also connects healthcare with other smart city services. This connection can improve the ability for city services to respond to health needs. Smartphones enable information gathering without spending city dollars on additional facilities and infrastructure. With mobile citizen sensing, city residents can inform service providers of time-critical events [51]. This may refer to a broken-down subway car or a power outage. However, a person may also use their device to report hearing gun shots or coming across an injured person who is lying on a city sidewalk. Such a report allows cities to deploy emergency services to those sites in a more timely manner.

III. Ambient Sensor Data Collection and Analysis

In many ways, mobile sensing and crowd sourcing represent the lifeblood of smart cities. Mobile technologies do require active involvement on the part of city residents, however. A second, rich source of information relevant to smart cities and health care comes from sensors embedded in the environment, or ambient sensors. While mobile health monitoring allows a user to potentially gather critical data anywhere, at any time, there is a presumption with that the user is physically, cognitively, and emotionally equipped to operate the devices. In practical situations, limitation arise because people forget to charge or put on the device, they operate the device inaccurately, or they position the device incorrectly. In contrast, ambient physiological monitoring devices can be considered zero-effort technologies (ZETs) [52]. This means that the technology frees the user from the effort needed to operate the technology correctly.

Remote monitoring is beneficial for many types of chronic health conditions such as dementia and cardiovascular disease. Behavior monitoring is important for these types of diseases because they are so ubiquitous. The number of people with dementia was estimated at 47 million globally in 2017 and is projected to increase to 132 million by 2050 [53]. The cost of caring for individuals with Alzheimer’s disease in the US alone was estimated at $818 billion [53]. Cardiovascular disease also strikes a large percentage of the population. It is the leading cause of morbidity and mortality globally across all age groups [54]. In the US, an estimated 84 million people have cardiovascular disease including coronary heart disease, high blood pressure, and diabetes [54].

A. Ambient Sensors and Features

Ambient smart environments typically consist of sensors that measure the quantity of interest (e.g., room temperature), transceivers for communicating the collected information, and a computer to collect, timestamp, and store (or transmit to another computer for storage) the data. Figure 7 shows an example smart home. Here, disks are attached to the ceiling that monitor heat-based movement with passive infrared (PIR) sensors as well as ambient light levels. Similarly, magnetic sensors are attached to doors and door frames to monitor their open/closure status. In these settings, a wide range of additional sensors can be used. Some, such as radio-frequency identification (RFID) or Bluetooth low energy transmitters, require that a sensor be attached to an item and that a separate device be positioned nearby (a reader) to communicate with the sensor. Proximity between the sensor and reader can be used to determine the location of objects and thus whether the resident is moving and possibly using the object.

Fig. 7.

Common ambient sensors monitor (top) infrared-based motion, ambient light, ambient temperature and (bottom) door open/close status. Arrows indicate a sensor location.

Unlike many of the mobile sensors, ambient sensors are typically designed to generate a reading only when there is a change in their state. For example, PIR sensors indicate when there is movement in their field of view and when the movement has stopped. Similarly, magnetic door sensors indicate when the door state changes from closed to open and vice versa. However, many additional sensors can be included in these settings that generate the same continuous or periodic readings found on mobile devices. Table III characterizes sensors that are typically found in smart environments.

TABLE III.

Ambient Sensors

Type	Measurement	Data Format	Data Rate
PIR	motion	state	low
RFID / Bluetooth	object interaction	state	low
Pressure	pressure on mat, chair, object	state	low
Magnetic switch	door/cabinet open/close	state	low
Camera	video, still image	numeric vector	very high
Microphone	audio	numeric vector	very high
CO2	CO2 concentration	numeric	high
Power	electricity consumption	numeric	high
Water	water consumption	numeric	high
Temperature	ambient temperature	numeric	high
Light	ambient light level	numeric	high

Because many of the sensors that are used in smart environments are discrete event and thus only report a change in state, the features that are listed in Table II are well suited to these settings. Table IV lists features that are often used when analyzing data collected from ambient sensors. We again assume that features are extracted from a fixed-size window of sensor events. The extracted feature vector can be used to recognize activities and to analyze behavior patterns that are helpful for health assessment and intervention. In the same way that AL is used to learn a mapping from mobile device sensor sensors to activity labels, so machine learning techniques can be used to learn a mapping from ambient device sensor features to activity labels. These labels can be used to provide real-time information on the types of behavior that are automatically observed in a home or other physical environment.

TABLE IV.

Standard Ambient Data Features

Type	Features
Time	date, day of week, weekday / weekend, days past January 1, time of day, hours / minutes / seconds past midnight, duration of window
Sensor	number of events generated by each sensor, elapsed time since most recent event for each sensor
Location	location in space of first and last motion sensor in window, fraction of motion sensor events in window, change in fraction of motion sensor events between first and second half of window
Complexity	entropy of window to measure diversity of sensors generating events

B. Ambient Intelligence: Connecting Personal Health with Smart Cities

In Section II, we showed that medical technology is already making its way into the home and throughout the smart city using mobile devices and apps. However, ambient sensors allow smart cities to take this one step further with integrated systems that can support vulnerable and non tech-savvy patients.

Plextek recently conducted a survey of future health needs [55] and found that 76% of the population is worried about their elderly relatives living alone. They mention specific concerns about their relatives experiencing an emergency such as a heart attack or fall and about these relatives not telling anyone if they feel ill. For much of the at-risk population, the likelihood of visiting doctors in person is limited by lack of transportation, possible embarrassment about symptoms, and discouragement when facing long wait times.

Given that by 2020 the average home will have more than 500 connected devices [55], smart cities can employ embedded technologies to meet the health needs of their citizens and face the coming “age wave” despite limited availability of care providers.

As a first step in this effort, researchers have been focusing efforts on performing automated health assessment and intervention in everyday settings such as smart homes, using ICT technology including ambient sensors, networking, and machine learning. Continuously collecting sensor data in everyday environments sheds light on behavior changes that are consistent with changes on cognitive and physical health. Some of these changes might be too difficult to observe without ICT assistance because they are too gradual or too subtle. For example, scientists have found behavioral markers in early-stage dementia patients that include high daily variability in walking speed/trajectory [56], changes in time spent on key activities of daily living [57], interrupted sleep patterns [58]–[60], and changes in patterns of using a computer mouse at home [61].

Smart home technology can not only aid in determining if there are changes in a resident’s health but it can also offer insights on the effects of known conditions in daily life. Austin et al., for example, found that cognitive performance is affected when individuals do not regularly follow prescribed medication treatments in certain populations [62]. Similarly, continuous monitoring of gait patterns using ambient sensors can be used to generate predictive models of falls [63]. Ambient sensors can also detect the presence of visitors in the home, time the resident spends out of the home, and time the person spends on the phone and using social media, all of which impact not only loneliness and mood but also cognitive health [64], [65].

If smart cities make use of ambient intelligence technology they may not only improve quality of life for its residents but also save the city substantial costs of patient treatment. ICT-based smart health allows patients to spend less time in hospitals and more time in their own homes by using monitoring sensors, telemedicine, and reasoning techniques that transmit alerts [66], [67]. These technologies also enable health providers to quantify rehabilitation at both hospital and home settings [68], making the transition more seamless and recovery more constant.

Fortunately, the role of ICT health technologies in smart cities does not end with assessment. Many automated health interventions have been introduced in recent years. Some focus on guiding patients through physical therapy and exercise [69]. Others use web interfaces to strengthen cognitive performance and promote social interaction [70].

Furthermore, smart homes can use the activity learning methods described in Section IIIA to automated interventions. If activities can be recognized, then their future occurrences can also be predicted [71]. Instead of learning a mapping from the extracted feature vector to an activity label with a classifier, a regression algorithm is used to learn a mapping from the extracted feature vector to a number indicating the amount of time that will elapse until a particular activity will occur again. Training the regressor requires activity-labeled sensor data. At each point in time within the sequence of training data, the actual elapsed time until an activity next occurs is calculated and becomes the target output for the regressor. Given an established baseline routine, the activity prediction algorithm is used to prompt smart home residents to initiate critical activities at the appropriate time and context, without any need for preprogrammed reminder rules [71]. For example, the home can sense when the resident is eating dinner and at that time remind them to also take their medicine. Employing such assessment and intervention technologies means that existing health providers can scale their services to a growing city.

C. Detecting Health Events at Home Using Ambient Sensors

Most health assessment technologies look at long-term changes in behavior to alert individuals of possible physical or mental health issues. As mentioned in the previous section, a large portion of the population is concerned about detection of sudden health events. While research has concentrated on fall detection in everyday city settings, most methods require the incorporation of wearable devices or customized detection environments [72].

Here we introduce one specific new approach to automatically detect health events in smart environments using ambient sensors. The method compares two or more time periods, or windows, of activity-labeled data using an algorithm called Behavior Change Detection (BCD). If the two time windows contain significantly different activity information this may indicate a significant behavior change. The size of the windows governs how sudden (or short term) the change is. If the windows are drawn from two different homes then the method can be used to actually compare behavior patterns between different subpopulations within a smart city or between different cities.

Let X denote a sample of time series (ambient sensor) data where each day’s data are expressed by extracted activity features, X = {x₁, x₂, ‥} and let W be a window of n days such that W ⊆ X. In this particular study, activity features consist of the amount of time spent on each activity and the sensor density (number of sensor events) for one day. BCD then compares two windows of data, W_i and W_j, within time series X. The examples we present here consider windows that are one week in length (n=7) and that compare a baseline window (i=1) with each subsequent window (j=2,3,‥). In our experiments, baseline windows are manually selected. In future work, they can instead represent a distribution over a large number of observed weeks.

In order to generate activity labels, we employ the CASAS-AR activity recognition algorithm [73]. CASAS-AR learns a mapping from the sensor feature vector to a set of activity labels using a random forest classifier. Training data for CASAS-AR annotators who look at one month of data and utilize both the house floorplan and resident information to generate corresponding ground truth activity labels. The data input to CASAS-AR utilizes the feature types described in Table IV and has a dimensionality of 99. The labeled activities are hygiene, sleep, bed-toilet, eat, drink, enter home, leave home, relax, and work. For these activities, CASAS-AR achieved a recognition accuracy (computed as ratio of correctly-labeled sensor events to the total number of sensor events) of 98% using 3-fold cross validation on a total of 8,735,293 data points collected from 30 homes.

The time series literature offers strategies to perform change detection. Relative unconstrained Least-Squares Importance Fitting (RuLSIF) [74] is one such approach that estimates the ratio of probability distributions in the two windows using the Pearson divergence dissimilarity window. We then introduce a significance test in which the magnitude of inter-window change should exceed the day-to-day variability within each window [75]. We also introduce our approach called small-window Permutation-based Change Detection in Activity Routine (sw-PCAR) which breaks each day unit into smaller time intervals and uses Kullback-Leibler (KL) divergence as a distance measure of the two compared windows. To perform significance testing, sw-PCAR randomly shuffles the time intervals and computes the KL distance for the permuted pair. The algorithm repeats the shuffling procedure multiple times to form an empirical distribution of the possible permutations for the two windows; then boxplot-based outlier detection can be used to report statistical significance.

The third method we report in this study utilizes a binary classifier to not only detect behavior change but to explain it using human-interpretable rules. This type of virtual classifier for change analysis was proposed by Hido et al. [76]. The classifier labels data points from one window as positive and from the second window as negative, then trains a decision tree to learn the boundary between the two classes. If a significant change exists between the two windows, then the average classification accuracy should be higher than 0.5. This threshold represents the critical value at which n Bernoulli trials are expected to exceed the accuracy of random guess at α=0.05 significance.

BCD has been used to detect health events in six smart homes. Detected health events include cancer diagnosis and treatment, insomnia, in-home falls, hypoxia, and restless leg syndrome. Figure 8 shows the results of analyzing one of these homes with an 86 year old resident. This resident was diagnosed with lung cancer during Week 4 of the analysis and underwent radiation treatment starting in the middle of Week 10. The hypothesis is that treatment has an observable, quantifiable, and explainable impact on daily behavior.

Fig. 8.

(left) BCD change scores plotted using RuLSIF, sw-PCAR, and VC comparing each week with the baseline week with values above the red line showing significant changes. (right) Density maps for the Sleep, Enter home, and Eat/Drink activities for each week, where darker colors indicate more time spent on the activity. (bottom right) A VC-discovered rule explaining the change during Week 11.

Figure 8 illustrates results of applying the three change detection methods on this activity-based smart home data. All points above the horizontal red line indicate a statistically significant change detected by the corresponding method. Comparing Week 1 and Week 11 (the first full week of treatment), RuLSIF finds a change of 2.298, sw-PCAR finds a change of 0.091, and the virtual classifier finds a change of 1.000. All three are significant results and are much greater than the corresponding changes of −0.017, 0.001, and 0.500 that were computed when comparing Weeks 1 and 2.

Figure 8 also shows associated activity density maps to visualize the amount of time spent on a particular activity over a one-week period. In these maps, each row represents an hour of the day and each column represents a single week. Cells represent data aggregated over a one week period. The darker the color is for a cell, the more time was spent on the activity during that particular hour of the day in the corresponding week. Finally, the figure shows a rule that was learned by a virtual classifier to explain the difference between Weeks 1 and 11. As the density maps show and the rule indicates, the resident’s level of sleep decreased once treatment started and the number of visitors who came to the home, likely to help the resident, increased. Another observed impact of the treatment is the increased number of trips to the kitchen. These more frequent kitchen trips are consistent with a known effect radiation treatment has on a feeling of thirst which results in a patient drinking more liquids throughout the day [77].

To illustrate a health event that is observed by the smart home, Figure 9 shows another sensor-detectable health event in a smart home with a resident who has Parkinson’s disease. Going into the kitchen at night to get a drink is a frequent activity for this resident. The kitchen visit typically lasts one minute and generates 30–40 sensor events. On one evening, however, the resident fell while in the kitchen. As can be seen in the figure, the ambient sensors do not provide information about gait or acceleration that indicates a falling motion. However, the length of time spent near the stove was much longer than normal and the person did not even reach the sink. Given the time of day, the movement pattern, and the unlikely lingering in one area of the home, this can be detected as a health event. If the resident or a clinician suggests a reason for this behavior change, then this type of health event can be more readily recognized in future cases and can prompt service providers to respond quickly.

Fig. 9.

(top) Smart home where fall occurred and (bottom) corresponding sensor events. Red area highlights likely time of fall.

This approach holds promise for detecting behavioral changes that may indicate health events. Advantages to this approach are that it incorporates activity information into reasoning about routine behavior and changes can be identified in an unsupervised manner. Additionally, the changes can be explained by using a virtual classifier to generate rule-based explanations of the changes. There are areas for ongoing work with behavioral change detection as well. The examples illustrated in this paper look at one week of behavior at a time. Automated tools should examine varying time scales to detect changes at different granularities. In addition, because some of the changes may not be health critical (e.g., changes may be detected when a family member comes for a visit or the resident leaves for a vacation), a future version of BCD might use weakly supervised learning with a clinician expert to identify classes of behavior changes that carry clinical relevance.

IV. Ongoing Challenges for Smart City Healthcare

The discussion in this paper highlights work that uses smart city ICT to provide smart healthcare and that can in turn more effectively channel city resources to help residents promptly as the need arises. While researchers and city designers have taken important first steps towards integrating smart ICT and healthcare, here we discuss how these solutions can scale further by becoming more fully connected.

A. Privacy and Security

Certainly the advances in pervasive computing and machine learning open up possibilities for smarter healthcare using smart city technology. At the same time, these advances rely on data being both collected and shared. Privacy issues often deter data sharing. In fact, security and privacy are the issues that are most often headlined in discussions about barriers to incorporating technology into healthcare [78], [79].

Concerns about privacy and security are not unfounded. Smart homes offer tremendous benefits for health monitoring and intervention, yet reports are frequently released about ways for intruders to hack into these homes [80]. This new breed of burglars may obtain details about resident living patterns in order to more effectively break into a house, putting both the resident’s belongings and their safety at risk. Other forms of security risk may rise even when there is no malicious intent. For example, health monitoring devices that do not follow prescribed software standards can endanger lives by not providing critical information at the needed time.

Sharing mobile app information can also be dangerous, because the collected information may not only identify the user but also track their current location and predict future locations. In some countries, releasing location information to third parties is prohibited by law [81]. In practice, however, Zang et al. tested 110 popular apps found that 73% of Android apps and 47% of iOS apps shared location and other personal information with third parties [82].

While data is assumed to be encrypted before transmission and storage, the power of the data lies in sharing the data in order to analyze patterns over entire populations. While the first line of defense that is employed by researchers before sharing data is to replace names with randomized identifiers (de-identifying the data), this is not sufficient. A well-known investigation into anonymized Massachusetts health and voter records revealed that 87% of the population was identifiable by the quasi-identifiers of zip code, gender, and date of birth. Similarly, an analysis of newspaper stories about hospital visits led to identifying the matching health record in 43% of the cases [83].

While researchers such as Bhuyan et al. [84] and Kotz et al. [85] try to educate policymakers about the potential privacy and security risks of smart city-based healthcare, there are advances that are being made on the technology side as well. In the area of privacy-preserving data mining, data analysis techniques guarantee certain levels of privacy while attempting to maximize the utility of the data [86]. This is referred to as the privacy-utility tradeoff.

Three types of privacy-ensuring data mining techniques are being investigated that may offer assurance for smart city residents. First, data can be “camouflaged” as it is collected by perturbing the data. Collected data can be perturbed by adding (or multiplying) noise to the original data, as shown in Equation 3. The noise has a known statistical distribution, Y, and the result, Z, of perturbing using Y can be shared publicly.

$$Z=X+Y$$ (3)

$$X=Z-Y$$ (4)

Using Equation 4, the original data distribution, X, can be reconstructed for data mining but the original values cannot. While perturbation techniques remain an area of ongoing investigation [87], achieving optimal data privacy while not shrinking data utility is an NP-hard task.

Second, in cases where the original collected data will be released to third parties, a goal of privacy-preserving data mining is to ensure the k-anonymity of the data [86]. This refers to the assurance that the identifiable attributes for any given user are undistinguishable from at least k-1 other users. K-anonymity can be achieved through techniques that include removing sensitive attributes, increasing diversity of sensitive attributes, or adding synthetic data to obfuscate the real values, thus allowing sensitive data to “hide in the crowd”. In the case of mobile data, an alternative to assigning a single constant identifier for each user is to change identifiers periodically. This change makes tracking users over time and space difficult. An ideal time to change the identifier is when a user enters a space with at least k-1 other users so that old and new IDs cannot be easily linked.

Third, the output of data mining algorithms can be modified to avoid leaking sensitive information. For example, the effectiveness of a classification algorithm can be downgraded enough so that it meets performance thresholds but minimizes the risk of being used to identify individuals. Whichever combination of techniques is employed, it is important that all parties including city designers, policy makers, and tech providers agree on privacy guarantees and describe to city residents the potentially sensitive data sharing that may occur with smart city technologies.

B. Accessibility and Usability

Healthcare is a keyword that appears in many smart city efforts. However, cost is still a barrier for widespread adoption of technologies that can be employed at an individual and community level [88]. In fact, Mehl and Labrique [89] place financial coverage of technologies at the topic of their cascading model of priorities in ensure universal health coverage. In the long term, integrating technology into city-wide healthcare can reduce costs for the city and its citizens. In the short term, however, the cost of the technology itself may prevent communities from adopting the technology.

Cities can take a similar approach as insurance companies to not only offset the short-term cost of health technology but also address another barrier of healthcare technology, which is taking the time to learn and adopt the technology [90]. In recent years, some insurance companies have offered discounts to customers who agree to install telematics in their vehicles that monitor driving patterns [91]. The potential boon for deciding to install the devices, in addition to lowering premium rates, is that the information can improve road safety because the insurance app can inform drivers when they are driving too recklessly. Incentives have also been offered over the last decade from other service providers such as utility companies [92], [93]. Similar incentives may be offered in the future by health insurance companies to install ICT in homes. The technology will support quality healthcare in a cost-effective manner to citizens when and where they need it.

C. Connection with other Smart City Services

Smart cities utilize information and communications technologies in multiple ways, as shown in Figure 10. As Jin et al. state, the goal of smart city platforms is to facilitate plug-and-play smart objects that can be deployed anywhere with an ability to blend in to their surroundings [94]. The objects should support not only health monitoring but structure monitoring, environment monitoring, security, and intelligent transportation. As an example, while many city citizens would look at the icon in the middle of the figure and see a streetlamp, in some smart cities such as Spokane, Washington, streetlights already contain sensors to capture a variety of air quality conditions around the town [95]. Indeed, by the year 2020 10% of the smart cities will also likely use streetlamps as the backbone for citywide wireless networks [96]. Weaving sensors into existing city features represents an example of the ever-increasing connectivity of data that can help researchers understand the connection between city design and health. By monitoring air quality as well as behavior, we can see the impact of our behavioral patterns as well as design choices on air quality. Additionally, we can monitor the impact of air quality on health and we can design interventions such as changing city design or providing real-time information to residents to stay indoors during times of poor air quality.

Fig. 10.

The connection between healthcare and other smart city services.

Biographies

Diane J. Cook is a Huie-Rogers Chair Professor at Washington State University. She received her Ph.D. in computer science from the University of Illinois and her research interests include machine learning, pervasive computing, and designed of automated strategies for health monitoring and intervention. She is an IEEE Fellow and a Fellow of the National Academy of Inventors.

Glen Duncan is a Professor in the Elson S. Floyd College of Medicine and the Chair of the Department of Nutrition and Exercise Physiology at Washington State University. He is also the Director of the Washington State Twin Registry. He received the PhD in Education with a concentration in Exercise Physiology from the University of Tennessee at Knoxville. Over a scientific career spanning two decades, he has studied the effects of physical activity and nutrition on community health. His conceptual approach situates these phenomena in the context of social and built environments.

Gina Sprint received the B.S. degree in computer science from Eastern Washington University in 2012 and the Ph.D. degree in computer science from Washington State University in 2016. She was a National Science Foundation Fellow with the IGERT Integrative Training Program in Health-Assistive Smart Environments. Currently she is an Assistant Professor of computer science at Gonzaga University. Her research interests include wearable computing, machine learning, technology applications for healthcare, and computer science education. Dr. Sprint is a Member of the IEEE and the Association for Computing Machinery.

Roschelle L. Fritz, PhD, RN, is an assistant professor at Washington State University College of Nursing, Vancouver, WA. Dr. Fritz received her Ph.D. in nursing from Washington State University in 1990. Dr. Fritz’s research focuses on the application of technology in the delivery of healthcare and human-computer interactions.