A Survey on Ambient Intelligence in Health Care

Abstract

Ambient Intelligence (AmI) is a new paradigm in information technology aimed at empowering people’s capabilities by the means of digital environments that are sensitive, adaptive, and responsive to human needs, habits, gestures, and emotions. This futuristic vision of daily environment will enable innovative human-machine interactions characterized by pervasive, unobtrusive and anticipatory communications. Such innovative interaction paradigms make ambient intelligence technology a suitable candidate for developing various real life solutions, including in the health care domain. This survey will discuss the emergence of ambient intelligence (AmI) techniques in the health care domain, in order to provide the research community with the necessary background. We will examine the infrastructure and technology required for achieving the vision of ambient intelligence, such as smart environments and wearable medical devices. We will summarize of the state of the art artificial intelligence methodologies used for developing AmI system in the health care domain, including various learning techniques (for learning from user interaction), reasoning techniques (for reasoning about users’ goals and intensions) and planning techniques (for planning activities and interactions). We will also discuss how AmI technology might support people affected by various physical or mental disabilities or chronic disease. Finally, we will point to some of the successful case studies in the area and we will look at the current and future challenges to draw upon the possible future research paths.

I. INTRODUCTION

A. What is Ambient Intelligence?

Imagine a day when a small tricoder-like ¹ device monitors your health status in a continuous manner, diagnoses any possible health conditions, has a conversation with you to persuade you to change your lifestyle for maintaining better health, and communicates with your doctor, if needed. The device might even be embedded into your regular clothing fibers in the form of very tiny sensors and it might communicate with other devices around you, including the variety of sensors embedded into your home to monitor your lifestyle. For example, you might be alarmed about the lack of a healthy diet based on the items present in your fridge and based on what you are eating outside regularly. This might seem like science fiction for now, but many respecters in the field of “Ambient Intelligence” expect such scenarios to be part of our daily life in not so far future.

The Ambient Intelligence (AmI) paradigm represents the future vision of intelligent computing where environments support the people inhabiting them [1], [2], [3]. In this new computing paradigm, the conventional input and output media no longer exist, rather the sensors and processors will be integrated into everyday objects, working together in harmony in order to support the inhabitants [4]. By relying on various artificial intelligence techniques, AmI promises the successful interpretation of the wealth of contextual information obtained from such embedded sensors, and will adapt the environment to the user needs in a transparent and anticipatory manner. An AmI system is particularly identified by several characteristics:

• Context Aware: It exploits the contextual and situational information.

• Personalized: It is personalized and tailored to the needs of each individual.

• Anticipatory: It can anticipate the needs of an individual without the conscious mediation of the individual.

• Adaptive: It adapts to the changing needs of individuals.

• Ubiquity: It is embedded and is integrated into our everyday environments.

• Transparency: It recedes into the background of our daily life in an unobtrusive way.

Besides characteristics such as transparency and ubiquity, an important characteristic of ambient intelligence is the intelligence aspect. By drawing from advances in artificial intelligence (AI), AmI systems can be even more sensitive, responsive, adaptive, and ubiquitous. While ambient intelligence draws from the field of AI, it is not synonymous with AI [5]. In addition to the AI sub-areas such as reasoning, activity recognition, decision making and spatio-temporal logic, an ambient intelligence system has to rely upon advances in variety of other fields. Some example areas include “sensor networks” to facilitate the data collection, “robotics” to build actuators and assistive robots, and “human computer interaction” to build more natural interfaces.

We have already embarked on the path of achieving such a vision. Nowadays, we are surrounded by various computing devices such as personal computers, smart phones, GPS, tablets, various sensors such as RFID tags, infrared motion sensors, as well as biometric identification sensors. The widespread presence of such devices and sensors and accompanying services such as location service has already sparked the realization of ambient intelligence. In addition, recent computational and electronics advancements have made it possible for researchers to work on ambitious concepts such as smart homes, and to bring us one step closer to the full realization of ambient intelligence in our daily environments.

B. Health Care Challenges

These days, the majority of industrialized nations are facing significant complications regarding the quality and cost of various healthcare and wellbeing services. These difficulties will exacerbate even more due to an increasing aging population, which translates into a multitude of chronic diseases and tremendous demand for various health care services. As a result, the cost of the healthcare sector might not be sustainable and therefore industrialized countries need to find and plan policies and strategies to use the limited economical resources more efficiently and effectively. This need for sustainable healthcare systems translates into a range of challenges in science and technology which if solved, ultimately could benefit our global society and economy. In particular, the exploitation of information and communication technology for implementing autonomous and pro-active healthcare services will be extremely beneficial. In the past decades, consumer-driven healthcare in conjunction with web-based platforms and electronic health records have led to an array of improved health care solutions. In recent years, we also have witnessed the emergence of many smart phone apps that are becoming readily available for physiological status monitoring [6], [7], [8]. However, despite being an important step towards personalized medicine, these solutions often suffer from scalability, security and privacy issues. Furthermore, such solutions are only able to provide a snapshot of physiological conditions rather than a continuous view of the overall health over the course of many years.

With recent advances in sensor networks research we are already embarking on the path of revolutionary low cost health care monitoring systems embedded within the home and living environments [9] [10]. In particular, AmI systems have the potential to enhance the health care domain dramatically. For example, ambient intelligence technology can be used to monitor the health status of older adults or people with chronic diseases, and it can provide assistive care for individuals with physical or mental limitations. It can be used for developing persuasive services to motivate people to lead a healthier lifestyle. It also can be used in rehabilitation settings or in general in enhancing the wellbeing of individuals. Ultimately, it can support the health care professionals in terms of providing innovative communication and monitoring tools. These systems will provide health monitoring in a transparent and unobtrusive way. Figure 1 depicts how AmI systems might be used as cohesive services integrated into different environments and devices.

Fig. 1.

Interconnected world of AmI health services.

In this survey, we will explore how different scientific and technological methodologies can be used for supporting the development of AmI-based solutions for health care and we will review a multitude of applications that support such health care solutions.

II. SUPPORTING INFRASTRUCTURE AND TECHNOLOGY

This section will introduce and describe the supporting infrastructure and technologies used in AmI systems in the context of health care domain. In particular, we will explain Body Area Networks (BANs) and Dense/Mesh Sensor Networks in Smart Homes, and we point to some recent trends in sensor technology, such as epidermal electronics and MEMS sensors, among others.

A. BANs: Body Area Networks

The widespread use of wireless networks and the constant miniaturization of electrical devices has empowered the development of Body Area Networks (BANs) [11]. In a BAN, various sensors are attached on clothing or on the body or even implanted under the skin [12]. This new communication approach offers numerous new, practical and innovative applications for improving human health and the quality of life by continuously monitoring health features such as heartbeat, body temperature, physical activity, blood pressure, ECG (electrocardiogram), EEG (electroencephalography) and EMG (electromyography). BANs provide a technological infrastructure for remotely streaming sensored data to a medical doctor’s site for a real time diagnosis, to a medical database for record keeping, or to a corresponding technological equipment that, pro-actively and autonomously, can issue an emergency alert or intelligently manage this information for taking suitable actions and improving the quality of human life [13].

There are several benefits of using wireless BANs in health care applications; mainly communication efficiency and cost-effectiveness. Indeed, physiological signals obtained by body sensors can be effectively processed to obtain reliable and accurate physiological estimations. At the same time, the ultra-low power consumption provision of such sensors makes their batteries long-lasting. Moreover, with the increasing demand of body sensors in the consumer electronics market, more sensors will be mass-produced at a relatively low cost, especially for medical purposes. Another important benefit of BAN is their scalability and integration with other network infrastructure. BANs may interface with Wireless Sensor Networks (WSNs), radio frequency identification tags (RFID) [14], [15], Bluetooth, Bluetooth Low Energy (BLE, previously called WiBree) [16], video surveillance systems, wireless personal area network (WPAN), wireless local area networks (WLAN), internet, and cellular networks. All of these important benefits are opening and expanding new marketing opportunities for advanced consumer electronics in the field of ubiquitous computing for health care applications.

Figure 2 better depicts BANs’ communication architecture in terms of three different layers: Tier-1-Intra BAN, Tier-2-Inter BAN, and Tier-3- beyond-BAN communications. These architectural layers cover multiple aspects of communication that range from low-level to high-level design issues, and facilitate the creation of a component-based, efficient BAN system for a wide range of applications.

Fig. 2.

A three-tier architecture of BAN communication system.

The term “intra-BAN communications” refers to radio communications of about 2 meters around the human body, which can be further sub-categorized as: (1) communications between body sensors, and (2) communications between body sensors and a portable Personal Server (PS) device (i.e., a PDA), as shown in in Figure 2. Due to the direct relationship with body sensors and BANs, the design of intra-BAN communications is very critical. Furthermore, the intrinsically battery-operated and low bit-rate features of existing body sensor devices make it a challenging issue to design an energy-efficient MAC protocol with QoS provisioning.

The “inter-BAN communications” enables the communications between the body sensors and one or more access points (APs). The APs can be deployed as part of the infrastructure, or be strategically placed in a dynamic environment for handling emergency situations. Similarly, the functionality of a tier-2-network (as shown in Figure 2) is used to interconnect BANs with various networks that are easy to access in daily life, such as the Internet and cellular networks. We divide the paradigms of inter-BAN communications into two categories, infrastructure-based architecture and ad hoc-based architecture. While the infrastructure-based architecture provides larger bandwidth with centralized control and flexibility, the ad hoc-based architecture facilitates fast deployment when encountering a dynamic environment, such as medical emergency care response, or at a disaster site (e.g., AID-N system [17]).

Most BAN applications use infrastructure-based, inter- BAN communications that assumes an environment with limited space, e.g., a waiting room in a hospital, home and office, etc. Compared to its ad-hoc networks counterpart, infrastructure-based networks offer the advantage of centralized management and security control. Due to this centralized structure, the AP also works as the database server in some applications, such as SMART [18], or CareNet [19].

Tier 3 is intended for streaming body sensor data to metropolitan areas. Sensor data are moved from inter-BAN network to beyond-BAN network by using a gateway device; for instance a PDA could be employed to create a wireless link between these two networks, transfer body information between geographical networks and, consequently, enhance the application and coverage range of health care systems by enabling authorized healthcare personnel (e.g., doctor or nurse) to remotely access a patient’s medical information through cellular network or the Internet. A database is also an important component of the “beyond-BAN” tier in the scenario of health care. This database maintains the user’s profile and medical history. According to a user’s service priority and/or doctor’s availability, the doctor may access the user’s information as needed. At the same time, automated notifications can be issued to his/her relatives based on this data via various means of telecommunications. The design of beyond-BAN communication is application-specific, and should adapt to the requirements of user-specific services. For example, if any abnormalities are found based on the up-to-date body signal transmitted to the database, an alarm can be notified to the patient or the doctor through email or short message service (SMS). In fact, it might be possible for the doctor to remotely diagnose a problem by relying on both video communications with the patient and the patient’s physiological data information stored in the database or retrieved by a BAN worn by the patient.

B. Dense/Mesh Sensor Networks for Smart Living Environments

Besides BAN, sensors can be embedded into our environments, resulting in intelligent and pro-active living environments capable of supporting and enhancing daily life, especially in case of elderly or individuals suffering from mental or motor deficiencies. In particular, Wireless Mesh Sensor Networks(WMSNs) could be used for designing unobtrusive, interconnected, adaptable, dynamic and intelligent environments where processors and sensors are embedded in everyday objects (clothes, household devices, furniture and so on) [20]. The sensors embedded into daily environments are usually called “ambient sensors” (as opposed to body sensors). The ambient sensors will collect various type of data to deduce the activities of inhabitants and to anticipate their needs in order to maximize their comfort and quality of life [21]. WMSNs are based on mesh networking topology, a type of networking where each node must not only capture and disseminate its own data, but also serve as a relay for other nodes. In other words, each sensor must collaborate to propagate the data in the network. The main benefits of WMSNs is their capability to be dynamically self-organized and self-configured, with the network automatically establishing and maintaining mesh connectivity among sensors [22]. WMSNs do not require centralized access points to mediate the wireless communication and they are particularly suitable to be used in complex and dynamic environments such as the living spaces [23].

The general architecture of WMSNs, described in [24], is composed of three distinct wireless network elements:

1. Network Gateways;

2. Access Points;

3. Mobile and Stationary Nodes.

These elements are usually referred to as mesh nodes (MNs). In WMSNs, each node acts not only as a client but also as a router. Unlike WiFi hotspots, which need a direct connection to the Internet, mesh networks pass a data request until a network connection is found. The architecture of WMSNs can be classified into three classes: Infrastructure/BackBone WMNs, Client WMSNs and Hybrid WMSNs. In Infrastructure WMSNs, mesh routers form an infrastructure for clients; in Client WMSNs, all client nodes constitute the actual network to perform routing and configuration functionalities; Hybrid networks are a combination of the former two; as a result, mesh clients can perform mesh functions with other ones as well as access the network.

The innovative WMSNs networking platform allows smart environments to offer new solutions that provide high reliability and power efficiency. WMSNs also enable high adaptability and scalability, since low-profile mesh modules can be easily embedded and integrated with existing sensing devices throughout a building to form seamless networks. In general, WMSNs enable intelligent environments [23] to be characterized by:

• Faster Retrofitting: One of the main reasons of increasing costs and time-delays in retrofitting office space is caused by the labor-intensive movement of utility wiring to conform to the new wall organization. By means of WMSNs, systems’ designers can relocate sensors quickly and conveniently without intrusive, disruptive, and costly rewiring efforts [25].

• Simplified maintenance: Low maintenance costs are a key concern in designing a sensor network. The self configuring and self healing capabilities of WMSNs combined with its low power usage yield an effective solution to the maintenance issue.

• Reduced life-cycle costs: WMSNs continuously lead to economic benefits because they are easy to maintain, move, or replace resulting in a distributed system with life-cycle costs that are significantly less than traditional wired installations.

• Seamless upgrades, transitions: With the convergence and coordination between principal standard communication corporations, such as ZigBee Alliance[15] and the ASHRAE BACnet Committee[26], the transition to a wireless solution is not an all-or-nothing proposition. In this way WMSNs can be phased in easily - one room, area, floor, or building at a time.

• Flexibility: Free from wiring problems, systems’ designers can install a WMSN by placing wireless controllers virtually anywhere. This approach results in easily reconfigurable systems to create adaptable workspaces or less intrusively retrofit the existing network infrastructures while saving time and reducing costs.

Some examples of WMSNs for intelligent living environments have been provided by the Siemens APOGEE project[27] and by the HomeMesh project [28]. Both projects highlight that, starting from WMSNs features, it will be possible to design living spaces particularly suitable for supporting the capabilities of elderly or individuals with disabilities in order to enhance their quality of life.

C. Sensor Technology

Both BANs and WMSNs can be viewed as a collection of interconnected wireless sensors based on a particular processing and communication technology. In general, a wireless sensor is characterized by its small size and its capability of sensing environmental (in the case of ambient sensors) or physiological information (in the case of body sensors).

1) Ambient Sensor Architecture

Ambient sensors typically consist of transducers for measuring the quantity of interest (e.g. room temperature), and transceivers for communicating the collected information.

Different approaches can be taken for designing the transducer hardware. The most common and scalable approach is based on development of transducer boards that can be attached to the main processor board by the means of an expansion bus. A typical transducer board can provide light, temperature, microphone, sounder, tone detector, 2 axis accelerometer and 2 axis magnetometer devices. Alternatives include economical versions that provide a reduced set of transducers or more expensive versions that boast GPS, for instance. Special boards are also available that carry no transducers, but provide I/O connectors that custom developers can use to connect their own devices. The alternative design approach puts transducers directly on the micro-controller board. Transducers are soldered or can be mounted if needed but the available options are very limited and generality and expandability is affected. However, the on-board transducers design can cut production costs and provides more robustness than stand-alone transducer boards which may detach from the micro-controller board in harsh environments.

Through a transceiver circuitry, a sensor device communicates the sensed information to nearby units using a physical layer based on RF communication. Over the physical layer, different protocols have been implemented for allowing sensors to communicate among themselves [29]. The higher number of supported protocols makes it easier for a BAN to be integrated with other applications. Bluetooth is a popular wireless protocol for short range communications, but BANs need protocols that support low energy consumption and the self-organizing feature seen in ad-hoc networks. Even though Bluetooth has a very good communications mechanism over a short range, it is not a very feasible solution for BANs. To overcome these problems, most of the BAN applications use the ZigBee protocol. A key component of the ZigBee protocol is its ability to support mesh networks. ZigBee is used nowadays for communications among sensors in a network. Some of the advantages of using ZigBee are: (1) it incurs low energy consumption for communications between the nodes, (2) it has a low duty cycle that enables it to provide longer battery life, (3) its communications primitives enable low-latency communications, (4) and it supports 128-bit security [30]. In addition, it has all the basic features required for communications between the sensors in wireless nodes. ZigBee also enables broad- based deployment of sensor networks in a cost-effective manner.

Some of the most widely used ambient sensors are summarized in Table I.

TABLE I.

Ambient sensors used in smart environments.

Sensor	Measurement	Data Format
PIR1	Motion	Categorical
Active Infrared	Motion/Identification	Categorical
RFID2	Object Information	Categorical
Pressure	Pressure on Mat, Chair, etc.	Numeric
Smart Tiles	Pressure on Floor	Numeric
Magnetic Switches	Door/Cabinet Openning/Closoing	Categorical
Ultrasonic	Motion	Numeric
Camera	Activity	Image
Microphone	Activity	Sound

¹Passive Infrared Motion Sensor

²Radio Frequency Identification

2) BANs: Hardware and Devices

A body sensor node mainly consists of two parts: the physiological signal sensor and the radio platform to which multiple body sensors can be connected in order to create a complex communication network. The general functionality of body sensors is to collect analog signals that correspond to human’s physiological activities or body actions. The analog signal is later digitized by an Analog to Digital converter (A/D), and is forwarded to the network to be analyzed.

Different body sensors for measuring physiological signs are summarized in Table II where depending on the captured physiological signal, high or low data sampling rate might be needed.

TABLE II.

Body sensors.

Sensor	Measurement	Data Rate
Accelerometer	Direction	High
Gyroscope	Orientation	High
Image/video	Activity	Very high
Glucometer	Blood Sugar	High
Blood Pressure	Oscillometric	Low
CO2 gas sensor	CO2 Concentration	Very low
ECG1	Cardiac Activity	High
EEG2	Brain Activity	High
EMG3	Muscle Activity	Very high
EOG4	Eye Movement	High
Pulse oximetry	Blood Oxygen Saturation	Low
GSR (Galvanic Skin Response)	Perspiration	Very low
Thermal	Body Temperature	Very Low

¹Electrocardiography

²Electroencephalography

³Electromyography

⁴Electrooculography

More specifically, some of the most important body sensors include:

• Accelerometer/Gyroscope: Accelerometers are used in the field of health care for recognizing body postures (e.g. sitting, kneeling, crawling, laying, standing, walking, running and so on). The accelerometer-based posture monitoring for BANs typically consists of 3-axis accelerometers (or tri-axial accelerometers) positioned on well-defined locations on a human body. They can also be used to measure the vibration or acceleration due to the gravity, useful for recognizing, for example, elderly falls. Gyroscopes are used for measuring orientation, based on the principle of conservation of angular momentum. Gyroscopes are typically used together with accelerometers for physical movement monitoring.

• Blood glucose: Glucose, also referred to as blood sugar, refers to the amount of glucose circulating in the blood. Traditionally, glucose measurements are done by pricking a finger and extracting a drop of blood, which is applied to a test strip composed of chemicals sensitive to the glucose in the blood sample [31]. An optical meter (glucometer) is used to analyze the blood sample and gives a numerical glucose reading. Recently, noninvasive glucose monitoring is available through infrared technology and optical sensing.

• Blood pressure: The blood pressure sensor is a noninvasive sensor designed to measure systolic and diastolic human blood pressure, utilizing the oscillometric technique.

• CO₂ gas sensor. This sensor measures gaseous carbon dioxide levels to monitor changes in CO ₂ levels, as well as to monitor oxygen concentration during human respiration.

• ECG sensor. ECG is a graphic record of the heart’s electrical activity. Healthcare providers use it to help diagnose a heart disease. They can also use it to monitor how well different heart medications are working. In order to obtain an ECG signal, several electrodes are attached at specific sites on the skin (e.g., arms, and chest), and the potential differences between these electrodes are measured.

• EEG sensor. This sensor measures electrical activity within the brain by attaching small electrodes to the human’s scalp at multiple locations. Then, information of the brain’s electrical activities sensed by the electrodes is forwarded to an amplifier for producing a pattern of tracings. Synchronous electrical activities in different brain regions are generally assumed to imply functional relationships between these regions. In a hospital, the patient may be asked to breathe deeply or to look at a flashing light during the recording of EEG.

• EMG sensor. EMG measures electrical signals produced by muscles during contractions or at rest. Nerve conduction studies are often done together while measuring the electrical activity in muscles, since nerves control the muscles in the body by electrical signals (impulses), and these impulses make the muscles react in specific ways. Nerve and muscle disorders cause the muscles to react in abnormal ways.

• Pulse Oximetry: This sensor measures oxygen saturation using a non-invasive probe. A small clip with a sensor is attached to the person’s finger, earlobe, or toe. The sensor gives off a light signal that passes through the skin. According to the light absorption of oxygenated hemoglobin and total hemoglobin in arterial blood, the measurement is expressed as a ratio of oxygenated hemoglobin to the total amount of hemoglobin.

• Humidity and temperature sensors: They are used for measuring the temperature of the human body and/or the humidity of the immediate environment around a person. An alarm signal can be issued if a certain amount of changes are measured.

3) Recent Trends in Sensor Technology

Since body sensors are in direct contact with body tissue or might even be implanted, their size and physical compatibility with human tissues are crucial. This motivates the research and synthesis of novel materials and technologies, such as the Micro-Electro Mechanical Systems (MEMS) [32]. MEMS is an innovative technology for sensors design based on miniaturized mechanical and electro-mechanical elements (i.e., devices and structures) that are made using the techniques of micro-fabrication. The physical dimensions of MEMS devices can vary from well below one micron on the lower end of the dimensional spectrum, all the way to several millimeters. As a consequence, they open up new scenarios for ubiquitous health care applications. Recently MEMS technology has been used for design of different kinds of sensors such as accelerometer, blood glucose, blood pressure, carbon dioxide (CO₂) gas sensor, ECG, EEG, EMG, gyroscope, pulse oximetry, as well as some sensors typically used in WSNs. For example, in case of ECG bedside monitoring, disposable electrodes are traditionally made of silver chloride (AgCl). However, long-term usage of these types of electrodes may cause failure of electrical contacts, as well as skin irritation problems. MEMS technology can alleviate this problem by using textile structured electrodes which are embedded into clothes fabrics. These textile-structure electrodes, possibly woven into clothes, will not cause any skin irritation and thus are comfortable and suitable for long-term monitoring. Compared to the conventional electrodes, they are also much more flexible, since their shape can be adapted to human motion.

Other research directions are also considering the possibility of innovative and noninvasive sensors. For example, MIT researchers have designed a scalable electronic sensate skin by using a collection of flexibly interconnected small (1” × 1”) rigid circuit boards 3. Each board contains an embedded processor together with a suite of thirteen sensors, providing dense, multimodal capture of proximate and contact phenomena. Other important results have been obtained in the field of computer vision, thanks to the advancement of charge-coupled devices (CCD) and complementary metal-oxide-semiconductor (CMOS) active-pixel sensors [33]. The recent advancements are allowing cameras to be made so small as to be embedded into eye glasses, as a consequence enhancing the capabilities of BANs with vision features. The captured images can be mapped to audible outputs in order to assist people who have eyesight problems. The images can even be translated to other kinds of formats, e.g., gentle electrical impulses on the tongue. Together with a lollipop-sized electrode array in their mouths, blind people can also be trained to regain “vision”.

III. ALGORITHMS AND METHODS

In this section we introduce the set of computational methodologies that, combined with technologies presented in section II, enable systems’ designers to develop enhanced AmI health care applications, as summarized in Table IV.

TABLE IV.

Ambient Intelligence Applications in Health Care.

Class of Applications	Goals	Environmental Sensors1	Body Sensors1	Methodologies2
Continuous Health Monitoring	Using sensor networks for monitoring physiological measures (ECG, EEC, etc.)	•	◦	Activity Recognition
Continuous Behavioral Monitoring	Using sensor networks for monitoring human behav- iors (Watching TV, Sit- ting, etc.)	•	•	Activity Recognition
Monitoring for Emergency Detection	Using sensor networks for detecting hazards, falls, etc.	◦	•	Activity Recognition
Assisted Living	Creating smart environ- ments for supporting pa- tients and elderly during their daily activities	•	×	Activity Recognition, Decision Support
Therapy and Rehabilitation	Supporting people who re- quire rehabilitation ser- vices with remote and au- tonomous systems	◦	•	Activity Recognition, Decision Support
Persuasive Well-Being	Systems aimed at chang- ing persons attitudes in or- der to motivate them to lead a healthier life style	•	×	Activity Recognition, Decision Support
Emotional Well-Being	Ubiquitous systems based on neurological and psy- chological insights to an- alyze emotions and im- prove well-being	•	•	Activity Recognition
Smart Hospitals	Improving communication among hospital stakeholders trough ubiquitous technology	•	×	Decision Support

¹•: Mandatory - ◦: Optional - × : Not required (e.g. they could increase the intrusiveness of the system without additional benefits)

²All application classes use Anonymization and Privacy Preserving Techniques for ensuring personal data hiding

A. Activity Recognition

Ambient intelligence systems focus on the needs of a human and therefore require information about the activities being performed by the human [34]. At the core of such technologies is activity recognition, which is a challenging and well researched problem. The goal of activity recognition is to identify activities as they occur based on data collected by sensors. There exist a number of approaches to activity recognition [35], [36] that vary depending on the underlying sensor technologies that are used to monitor activities, the alternative machine learning algorithms that are used to model the activities and the complexity of the activities that are being modeled.

1) Sensor Modalities

Advances in pervasive computing and sensor networks have resulted in the development of a wide variety of sensor modalities that are useful for gathering information about human activities. In the case of wearable sensors, sensors are attached to the body [37] or woven into garments [38], [39]. When 3-axis accelerometers are distributed over an individual’s body then each sensor can provide information about the orientation and movement of the corresponding body part. Researchers commonly use these inertial measurement units to recognize ambulatory movements (e.g., walking, running, sitting, climbing, and falling) [40], [41], posture [42] and gestures [43], [44], [45], [46].

Environment sensors such as infrared motion detectors, magnetic door sensors, break-beam sensors, and pressure mats [47], [48] have been used to gather information about more complex activities such as cooking, sleeping, and eating. These sensors are adept in performing location-based activity recognition in indoor environments [49], [50] and their long battery life supports long-term data collection [51]. Because this approach embeds sensors within environments, it is well suited to creating ambient intelligent applications such as smart environments and has been widely adopted for health monitoring and ambient assisted living [52].

Some activities such as washing dishes, taking medicine, and using the phone are characterized by interacting with unique objects. In response, researchers have explored the usage of RFID tags [53], [54] and accelerometers or shake sensors [55] for tagging these objects and using the data for activity recognition. The challenge with this modality is deciding which objects to tag with sensors. One approach that has been investigated [56], [57] is to mine web page description of activities to determine which objects are instrumental to the activity and help differentiate the activity from others.

Other modalities that have been researched for activity recognition include video cameras [58], [59], [60], [61], microphones [62], [63], and GPS locators [64], [65]. Each of these does face a unique challenge for use in health care applications. Cameras and microphones need to be carefully positioned and robust in the presence of occlusion. Furthermore, these technologies are not always well accepted because of privacy concerns. Smart phones are increasing in popularity for activity recognition [66], [67] because sensors in the phone collect all of the gyro, accelerometer, GPS, acoustic, and video data found the other methods, as long as they are on the individual while they perform activities.

2) Activity Models

The methods that are used to model and recognize activities are as varied as the sensor modalities used to observe activities. Existing methods can be broadly categorized into template matching / transductive techniques, generative, and discriminative approaches. Template matching techniques employ a nearest-neighbor classifier based on Euclidean distance or dynamic time warping [68], [69]. Generative approaches such as naïve Bayes classifiers where activity samples are modeled using Gaussian mixtures have yielded promising results for batch learning [58], [70], [71]. Generative probabilistic graphical models such as hidden Markov models [72], [70], [73], [74] and dynamic Bayesian networks [64], [48] have been used to model activity sequences and to smooth recognition results of an ensemble classifier [75]. Decision trees as well as bagging and boosting methods have been tested [40]. Discriminative approaches, including support vector machines [58] and conditional random fields [76], [77], [52], [78] which attempt to maximally separate activity clusters, have also been effective.

3) Activity Complexity

Activity complexity. Many of these methods analyze pre-segmented activity sequences that have been collected in controlled settings. More recently, attempts have been made to perform automatic segmentation of the data into sensor events that belong to the same activity class [79], [56], [80]. Still others have focused on recognizing activities in real time from continuous sensor streams [80]. In addition, researchers have also investigated methods of leveraging information or models in one setting to boost activity recognition for a new sensor network [81], a new environmental setting [82], [83], [49], or new activity labels [84]. Another level of complexity for activity recognition is analyzing data for interwoven activities [85], [86] or concurrent activities [87]. Humans often make efficient use of time by performing a step for one activity while still in the middle of another activity, causing the sensor streams to interweave. Concurrent activities may occur if a single sensor event contributes to more than one activity. This situation may also indicate that multiple residents are in the space, which can be a challenge for activity recognition algorithms [88], [89].

B. Behavioral Pattern Discovery

Ambient intelligence systems focus on the needs of a human and therefore require information about the activities being performed While recognizing predefined activities often relies on supervised learning techniques, unsupervised learning is valuable for its ability to discover recurring sequences of unlabeled sensor activities that may comprise activities of interest. Methods for activity discovery build on a rich history of discovery research, including methods for mining frequent sequences [1,17], mining frequent patterns using regular expressions [47], constraint-based mining [90], and frequent-periodic pattern mining [91], [92].

More recent work extends these early approaches to look for more complex patterns. Ruotsalainen et al. [93] design the Gais genetic algorithm to detect interleaved patterns in an unsupervised learning fashion. Other approaches have been proposed to mine more complex discontinuous patterns [36], [90], from streaming data over time [80], in different types of sequence datasets and to allow variations in occurrences of the patterns [90]. Discovered behavioral patterns are valuable to interpret sensor data, and models can be constructed from the discovered patterns to recognize instances of the patterns when they occur in the future.

C. Anomaly Detection

While it is value to characterize and recognize common normal activities that account for the majority of the sensor events that are generated, for health applications we are also very interested in abnormal events. These abnormal events may indicate a crisis or an abrupt change in a regimen that is associated with health difficulties.

Abnormal activity detection, or anomaly detection, is also important in security monitoring where suspicious activities need to be flagged and handled. Anomaly detection is most accurate when it is based on behaviors that are frequent and predictable. There are common statistical methods to automatically detect and analyze anomalies including the box plot, the chart, and the CUSUM chart [94]. Anomalies can be captured at different population scales. For example, while most of the population may perform activity A, one person carries out activity B, which pinpoints a condition needing further investigation [95]. Anomalies may also be discovered at different temporal scales, including single events, days, or weeks [96].

Little attention has been devoted to anomaly detection in ambient environments. This is partly because the notion of an anomaly is somewhat ill-defined. Many possible interpretations of anomalies have been offered and use cases have even been generated for ambient intelligence environments [97]. Some algorithmic approaches have been suggested that build on the notion of expected temporal relationships between events and activities [98]. Others tag events as anomalies if the occur rarely and they are not anticipated for the current context [99].

D. Planning and Scheduling

Automatic planning and scheduling can be useful in many AmI applications. Automatic planning techniques achieve a goal state by starting from an initial known state and choosing among possible actions at each state. Planning can be useful in a number of different AmI care related scenarios. For example, planning can be used to schedule daily activities in a flexible manner for reminding dementia patients about their daily activities. It also can be used in order to detect any possible deficiencies in task execution, and to help dementia patients to complete those steps. Another use of planning is in automating daily routines, in order to allow users with physical limitations to live a more independent lifestyle.

In the past, many planning techniques have been proposed. Some techniques include decision-theoretic techniques (e.g. Markov Decision Processes [100]), search methods (e.g. forward and backward search [101]), graph-based techniques (e.g. GraphPlan [102]), hierarchal techniques (e.g. O-Plan [103]), and reactive planning techniques (e.g. [104]). For example, graph-based planning techniques represent search space of possible actions in form of a graph, hierarchal planning techniques use hierarchies to predefine groups of actions, and reactive planning techniques adjust the plan based on sensed information.

AmI applications pose many new challenges to the classical planning techniques. For example, the planner has to be functional in a dynamic environment where the outcome of the actions and their duration is not deterministic. Also, the availability of resources might change due to user mobility or other factors. Therefore, more advanced planning techniques have been proposed by extending classical planning techniques [105]. One example is the distributed hierarchal task network (D-HTN) technique [106] which extends the hierarchal task network (HTN). It uses a centralized approach to manage the distributed capabilities provided by the distributed devices. The distributed devices might be available in a permanent or transient manner. D-HTN has been studied in the context of care for diabetic patients at home, where different home devices communicate and coordinate plans with each other in a distributed manner. For example data from monitoring devices might require actions such as adjusting the room temperature, suggesting insulin injection, or contacting medical help.

Several AmI systems have been reported in the literature which use automated planning and scheduling, especially to help dementia patients. COACH is one such system which provides task guidance to Alzheimer’s disease patients [107]. It uses a hand-coded representation of detailed steps of handwashing, and relies on vision techniques to recognize user steps. If user is unable to complete a particular step, detailed instructions are provided. Another example is PEAT which also provides task guidance to the user [108]. It maintains a detailed model of the daily plan in terms of hierarchal events, and tracks their execution. PEAT has the capability of rescheduling activities in case of unexpected events, however it lacks any real sensory information from the world, except for user feedback. Autominder by Pollack et al. [109] is another system which provides users with reminders about their daily activities by reasoning about any disparities between what the client is supposed to do and what she is doing, and makes decisions about whether and when to issue reminders.

E. Decision Support

Decision Support Systems (DSS) [110], [111], [112] have been widely used in the field of heath care for assisting physicians and other health care professionals with decision making tasks, for example for analyzing patient data [113], [114], [115], [116], [117], [118], [119]. DSS systems are mainly based on two mainstream approaches: knowledge-based and non knowledge-based.

The knowledge-based DSS consists of two principal components: the knowledge database and the inference engine. The knowledge database contains the rules and associations of compiled data which often take the form of IF-THEN rules, whereas the inference engine combines the rules from the knowledge database with the real patients’ data in order to generate new knowledge and to propose a set of suitable actions. Different methodologies have been proposed for designing health care knowledge databases and inference engines, such as the ontological representation of information [120].

The non knowledge-based DSS have no direct clinical knowledge about a particular health care process, however they learn clinical rules from past experiences and by finding patterns in clinical data. For example, various machine learning algorithms such as decision trees represent methodologies for learning health care and clinical knowledge.

Both of these approaches could be used in conjunction with AmI technologies. Indeed, the sensitive, adaptive, and unobtrusive nature of AmI is particularly suitable for designing decision support systems capable of supporting medical staffs in critical decisions. In particular, AmI technology enables the design of the third generation of telecare systems. The first generation was the panic-alarms gadgets, often worn as pendants or around the wrist to allow a person to summon help in the case of a fall or other kinds of health emergency. The second generation of telecare systems uses sensors to automatically detect situations where assistance or medical decisions are needed. Finally, the third generation represents AmI-based systems which move away from the simple reactive approach and adopt a proactive strategy capable of anticipating emergency situations. As a result, DSSs could be used with multimodal sensing and wearable computing technologies for constantly monitoring all vital signs of a patient and for analyzing such data in order to take real-time decisions and opportunely support that people.

Finally, DSSs are jointly used with the AmI paradigm for enhancing communications among health personnel such as doctors and nurses. For example, Anya et al. have introduced a DSS system based on context aware knowledge modeling aimed at facilitating the communication and improving the capability to take decisions among healthcare personal located in different geographical sites [121].

F. Anonymization and Privacy Preserving Techniques

As ambient intelligent systems become more ubiquitous, more information will be collected about individuals and their lives. While the information is intended to promote the well being of individuals, it may be considered an invasion of privacy and, if intercepted by other parties, could be used for malicious purposes.

While some privacy concerns focus on the perception of intrusive monitoring [122], many heavily-deployed Internet gadgets and current ambient intelligent systems are nearly devoid of security against adversaries, and many others employ only crude methods for securing the system from internal or external attacks. The definition of privacy will continue to evolve as ambient intelligent systems mature [123]. This is highlighted by the fact that even if personal information is not directly obtained by an unwanted party, much of the information can be inferred even from aggregated data. For this reason, a number of approaches are being developed to ensure that important information cannot be gleaned from mined patterns [124], [125].

IV. APPLICATIONS

Different kinds of AmI applications for heath care have been developed in academia and industry, as summarized in Table IV. This section discusses about each application class by presenting both scientific and real-world frameworks and highlights the benefits provided to patients, elderly, and so on.

A. Continuous Monitoring

1) Continuous Health Monitoring

In the past decade, a variety of noninvasive sensors have been developed for measuring and monitoring various physiological parameters such as ECG, EEG, EDA, respiration, and even biochemical processes such as wound healing. Some of those sensors are in form of wearable devices such as wrist bands, while others are embedded into textile, known as E-textile or smart fabrics. The majority of these sensors allow for noninvasive monitoring of physiological signs, though some physiological measurements such as EEG still require the use of invasive devices and sensors (e.g. measuring EEG requires the use of electrodes). Regardless of the form of the sensors, such sensors allow the patients with chronic diseases to be in control of their health condition by benefiting from continuous monitoring and anomalous situation detection. Achieving continuous monitoring is almost impossible in conventional health care settings, where typical measures are taken only during occasional doctor visits. The use of such sensors will also allow the healthy adults to keep track of their health status and to take the necessary steps for enhancing their lifestyle.

Gouaux et al. [126] describe a wearable personal ECG monitoring device (PEM) for early detection of cardiac events, which detects and reports anomalies by generating different alarm levels. Another example is AMON which is in the form of a wristband and measures various physiological signals [127]. Nowadays, there are several commercially available health monitoring devices, such as HealthBuddy by Bosch [128], TeleStation by Philips [129], HealthGuide by Intel [130], and Genesis by Honeywell [131]. A number of academic projects also have tried to integrate monitoring devices with clothing fabrics, including the WEALTHY project [132], BIOTEX project [133], and MagIC project [134]. For example, BIOTEX monitors sore conditions based on pH changes and inflammatory proteins concentration [133]. Other projects have tried to provide a variety of accessible medical implants, for example the “Healthy Aims” project focuses on developing a range of medical implants to help the aging population [135]. Developing completely noninvasive methods for health monitoring is another active research area. For example, Masuda et al. [136] measure physiological signs such as respiration rate and heart beat by measuring perturbations in the pressure of an air-filled mattress and relying on the low frequency characteristics of heart and respiration. Similarly, Andoh et al. have developed a sleep monitoring mattress to analyze respiration rate, heart rate, snoring and body movement [137]. The SELF smart home project also monitors various factors such as posture, body movement, breathing, oxygen in the blood, airflow at mouth and nose and apnea, using pressure sensor arrays, cameras, and microphones [138].

2) Continuous Behavioral Monitoring

In addition to monitoring physiological measures, another potential monitoring application is behavioral monitoring. Behavioral monitoring especially can be useful in assisted living settings and monitoring of individuals with mental disabilities. Such systems can assess mental health and cognitive status of inhabitants in a continual and naturalistic manner. They can also provide automated assistance and can decrease the caregiver burden. In some cases a single activity is monitored, for example Nambu et al. [139] monitor watching TV for diagnosing health conditions. The majority of research projects monitor a subset of daily tasks. For example, the CASAS project [91] monitors a subset of daily tasks to identify consistency and completeness in daily activities of dementia patients. The IMMED project monitors instrumented activities of daily living (IADL) in dementia patients by using a wearable camera to monitor the loss of motor or cognitive capabilities [140]. Other researchers have worked on recognizing social activity, especially in nursing homes [141], [142]. Identifying any changes in activities might be an indicator of cognitive or physical decline. For example, indicators such as changes in movement patterns, walking speed, number of outgoings, and sleep rhythm have been identified as early signs of dementia [143], [144], [145].

3) Monitoring for Emergency Detection

There also have been some projects to monitor emergency situations. In the UK, British Telecom (BT) and Liverpool City Council have developed a project on telecare technology which monitors residents using a variety of sensors such as PIR sensors [146]. In case of any detected hazards, the system asks the residents if they are OK, otherwise the selected personnel are notified. Another important area of emergency detection is fall detection, which can be especially useful for the elderly, as falls contribute a high rate of morbidity and mortality in elderly. Fall detection techniques rely on several technologies: wearable devices, ambient sensors, and cameras [147]. Wearable fall detection systems measure posture and motion using sensors such as accelerometer and gyroscope and by measuring orientation and acceleration [148], [149]. Ambient fall detection systems use ambient sensors such as passive infrared (PIR) sensors and pressure sensors to detect falls. They also rely on techniques such as floor vibration detection and ambient audio analysis to detect possible fall [150], [151]. Finally, vision based fall detection systems extract video features such as 3D motion, shape, and inactivity to detect falls [152], [153]. There are also some preventive fall detection tools, such as the smart cane developed by Wu et al. which classifies cane usage and walking patterns, and informs the elderly in case of high risk of falling [154].

It should be noted that there is a huge potential for combining and fusing data from various sensors such as physiological sensors with electronic health records (EHR) or daily activity information [155]. This will allow the health care to shift from cure to prevention by early detection of diseases using continuous monitoring, as well as to reduce the need for institutional care by shifting the care to a personalized level.

B. Assisted Living

AmI technology can allow individuals with disabilities to maintain a more independent lifestyle using home automation, it can offer them continuous cognitive and physical monitoring, and can provide them with realtime assistance, if needed. Those services especially can be useful for the older adults who are suffering from physical and cognitive decline [156].

We already have discussed how behavioral monitoring and fall detection methods can be useful for the elderly. Medication management is another area which can provide great benefit to the elderly [157], [158], [159]. The majority of older adults take many different medications and they usually forget medication dosage and timing due to cognitive decline. Using appropriate contextual information obtained from various sensors, medication reminders can be delivered in a context aware and flexible manner. Care personnel can be contacted, if non-compliance is detected. For example, John will be reminded about his medications right after finishing his breakfast, but he will not be reminded if he is watching his favorite program on TV or if he is talking on the phone. If John forgets to take his medication more than a certain number of times (depending on the medication), his doctor will be automatically contacted. Current medication management systems are not yet fully context aware, though there has been some great progress. For example, iMat is a user friendly medication management system [160]. An iMat user has no need to understand the directions of her/his medications, rather iMAT enables the pharmacist of each user to extract a machine readable medication schedule specification from the users prescriptions or over the counter descriptions. Once loaded into an iMAT dispenser or schedule manager, the tool automatically generates a medication schedule. Other medication management tools also have been proposed by researchers, such as the “Magic Medicine Cabinet” which can provide reminder and can interact with healthcare professionals [161], or the “Smart Medicine Cabinet” which uses RFID tags to monitor medication usage and can communicate with a cell phone [162].

Besides medication management, other cognitive orthotics tools can be quite useful for people with mental disabilities, especially older adults suffering from dementia. COACH is a cognitive orthotics tool which relies on planning and vision techniques to guide a user through hand washing task [107]. Other cognitive orthotics tools such as PEAT [108] and Autominder [109] also use automated planning to provide generic reminders about daily activities. They can adjust their schedules in case of any changes in the observed activities. Cognitive orthotics tools also can be used for cognitive rehabilitation. SenseCam is a small wearable camera developed by Microsoft, which captures a digital record of the wearer’s day in terms of images in addition and a log of sensor data [163]. It has been shown to help dementia patients to recollect aspects of earlier experiences that have subsequently been forgotten, thereby acting as a retrospective memory aid. Hoey et al. [164] also describe the development of a cognitive rehabilitation tool to assist art therapists working with older adults with dementia.

AmI tools also can be useful for preventing wandering behavior of older adults who suffer from dementia. There are a number of outdoor wandering prevention tools. KopAL [165] and OutCare [166] support issues related to disorientation by contacting the caregiver in case of leaving predefined routes or deviating from daily signature routes. A number of tools have also been developed for preventing indoor wandering. For example, Lin et al. [167] use RFID technology to detect if people prone to disorientation (e.g. children or elderly) have approached a dangerous area, and Crombag [168] proposes using virtual indoor fencing. Some commercially available products for wandering prevention include safeDoor and SafetyBed [169], for example safeDoor raise alarm if a person walks out a door without opening it, to prevent nighttime wandering. Navigation assistance tools also have been developed to help elderly suffering from early signs of dementia. “Opportunity Knocks” is a mobile application which provides public transit system guidance by learning user’s routes [170].

A number of AmI projects try to provide comprehensive assistance through a variety of services. “RoboCare” is an assisted living project providing assistance to people with disabilities using a combination of software, robots, intelligent sensors and humans [171]. It uses a tracking system for tracking people and robots by exploiting vision techniques to determine various 3D positions. It also relies on a task execution and monitoring component to recognize current situation and to compare it with the expected schedule. The “Aware Home Research Initiative” (AHRI) at Georgia Tech includes a number of different projects focused on providing assistance to elderly, such as the “Independent LifeStyle Assistant” project which monitors the behavior of elderly in a passive manner and alerts caregivers in case of emergency (e.g. fall) [172]. The “ Technology Coach” is another AHRI project which watches the use of home medical devices by the elderly and provides appropriate feedback and guidance for better use [173]. Smart home projects such as CASAS also try to provide comprehensive monitoring and assistance services in a noninvasive manner by relying on various machine learning and data mining techniques to make sense of sensor data.

AmI systems also can provide great help to visually impaired people. A number of different systems have been proposed for blind navigation, relying on various sensors such as RFID tags, infrared sensors and GPS technology. Chumkamon et al. [174] used RFID tags to develop a tracking system for indoor guidance of blind persons. Jinying et al. [175] embed RFID tags in the tiles of a blind path for better navigation. Some systems also use audio interface to communicate the name of important locations to the user, e.g. the SAWN system [176]. There are also applications to facilitate daily tasks such as shopping, e.g. the ShopTalk project [177].

Finally, several AmI-based assisted living environments have been designed by using decision support methodologies. For example, the ALARM-NET project [178] is an assisted living and residential monitoring network for pervasive healthcare developed at the University of Virginia. It integrates environmental and physiological sensors in a scalable, heterogeneous architecture to support real-time data collection and processing. The ALARM-NET network creates a continuous medical history while preserving resident comfort and privacy by using unobtrusive ambient sensors combined with wearable interactive devices [179], [180]. The project CAALYX (Complete Ambient Assisted Living Experiment) [181] is another project for increasing elderly autonomy and self confidence by developing a wearable light device capable of measuring specific vital signs and detecting falls, and for communicating in real time with care providers in case of an emergency. MyHeart [182] is an integrated project for developing smart electronic and textile systems and services that empower the users to take control of their own health status [183]. The system uses wearable technology and smart fabrics to monitor patients vital body signs in order to provide proper wellbeing recommendations to the user. The SAPHIRE [184] project develops an intelligent healthcare monitoring and decision support system by integrating the wireless medical sensor data with hospital information systems [185]. In the SAPHIRE project the patient monitoring will be achieved by using agent technology complemented with intelligent decision support systems based on clinical practice guidelines. The observations received from wireless medical sensors together with the patient medical history will be used in the reasoning process. The patients history stored in medical information systems will be accessed through semantically enriched web services.

C. Therapy and Rehabilitation

According to the Disability and Rehabilitation Team at the World Health Organization (WHO), the estimated number of people who require rehabilitation services is continuously growing (1.5% of the entire world population) [186]. Nevertheless, the current health care solutions and technologies are not nearly sufficient to fulfill the rehabilitation needs. In such scenarios, AmI can shape innovative rehabilitative approaches that support individuals to have access to rehabilitation resources. This can be achieved by developing ad-hoc rehabilitation systems based on sensor networks and other technological approaches such as robotics and brain-computer interfaces (BCI).

Sensor networks have the potential to greatly impact many aspects of medical care, including rehabilitation [187]. For example, Jarochowski et al. [188] propose the implementation of a system, the Ubiquitous Rehabilitation Center, which integrates a Zigbee-based wireless network with sensors that monitor patients and rehabilitation machines. These sensors interface with Zigbee motes which in turn interface with a server application that manages all aspects of the rehabilitation center and allows rehabilitation specialists to assign prescriptions to patients. Another systems proposed by Piotrowicz et al. [189] describes the requirements of a system for cardiac tele-rehabilitation at home, and in particular it discusses the different components controlling a physical exercise training session, which needs to recognize and identify critical patient states through a continuous monitoring (based on AmI technology) and react accordingly. As a side-effect, the health related data gathered during the tele-rehabilitation session are used for providing cardiologists with useful information for patient care. The rehabilitation systems proposed by Helmer et al. [190] improves the quality of life for patients suffering from chronic obstructive pulmonary disease (COPD). The system includes a component for monitoring the rehabilitation training and automatically. As a consequence, it controls the target load for the exercise on the basis of his or her vital data.

Moreover, by equipping patients with wireless, wearable or environmental vital sign sensors, collecting detailed real-time data on physiological status can enable innovative activities as autonomous rehabilitation and therapy [191][192][193]. The Stroke Rehab Exerciser by Philips Research [194] guides the patient through a sequence of exercises for motor retraining, which are prescribed by the physiotherapist and uploaded to a patient unit. The system lies on a wireless inertial sensor system aimed at recording the patient’s movements, analyzes the data for deviations from a personal movement target and provides feedback to the patient and the therapist [195]. The Stroke Rehab Exerciser coaches the patient through a sequence of exercises for motor retraining, which are prescribed by the physiotherapist and uploaded to a patient unit. A wireless inertial sensor system records the patient’s movements, analyzes the data for deviations from a personal movement target and provides feedback to the patient and the therapist. The Hocoma AG Valedo system [196] (see Fig. 4) is a medical back training device, which improves patient’s compliance and allows one to achieve increased motivation by real time Augmented Feedback based on trunk movements. It transfers trunk movements from two wireless sensors into a motivating game environment and guides the patient through exercises specifically designed for low back pain therapy. In order to challenge the patient and to achieve more efficient training, the exercises can be adjusted according to the patient’s specific needs. Finally, GE Healthcare [197] is developing a wireless medical monitoring system that is expected to allow one to gather physiological and movement data thus facilitating rehabilitation interventions in the home setting. Several other systems are currently under research and development. As an example, Jovanov et al. [198] have developed a computer-assisted physical rehabilitation applications and ambulatory monitoring based on a Wireless Body Area Network (WBAN). This system performs real-time analysis of sensors’ data, providing guidance and feedback to the user in different therapy fields such as stroke rehabilitation, physical rehabilitation after hip or knee surgeries, myocardial infarction rehabilitation, and traumatic brain injury rehabilitation. A practical application example is given by the Tril [199] project that, by means of its subcomponent named BASE [200], provides a home-based interactive technology solution to deliver and validate the correctness of a personalized, physiotherapist prescribed exercise program to older adults. BASE uses a sensor networks to gather data necessary to deliver the exercise program and it exploits computer vision algorithms for validating the correctness of these rehabilitation experiences. One of the main aims of the Active Care [201] project is related to the support of at-risk elders [202]. This project exploits two environmental cameras for extracting human silhouettes and investigate about the human gait by analyzing shoulder level, spinal incline, and silhouette centroid. This analysis could be precious for remotely or autonomously aiding elder or impaired people [203]. Other interesting work based on sensors networks are related to the design of rehabilitation systems for degenerative pathologies such as Parkinson’s disease [204]. The authors presents the results of a pilot study to assess the feasibility of using accelerometer data to estimate the severity of symptoms and motor complications in patients with Parkinson’s disease. This system is based on a support vector machine (SVM) classifier used for estimating the severity of tremor, bradykinesia and dyskinesia from accelerometer data features and, as a consequence, optimizing the patient therapy. Bachlin et al. [205] also introduce a wearable assistant for Parkinson’s disease patients with the freezing of gait (FOG) symptom. This wearable system uses on-body acceleration sensors to measure the patients’ movements to detect FOG and automatically provide a rhythmic auditory signal that stimulates the patient to resume walking. In the future, by using the wearable sensor networks together with haptic hardware, it will be possible to design medical training systems based on augmented reality frameworks for improving medical staff capabilities to support elderly or patients during their rehabilitation [206].

Fig. 4.

The Hocoma AG Valedo at work [196].

The combination of sensor network technology and robots is also a very recent development in the field of rehabilitation systems [207][193]. Interest in this approach originates from the observation that subjects with chronic conditions (such as hemiparesis following a stroke) could benefit from therapeutic interventions that can be facilitated by robotic systems and enhanced by wearable technology [208]. Indeed, these integrated systems could be used in a variety of health care scenarios. A concrete application of these concepts is the human-friendly assistive home environment, Intelligent Sweet Home (ISH) developed at KAIST, South Korea [209][210]. The system considers the residents’ lifestyle by continuously checking their intention or health status, the home itself is seen as an intelligent robot supporting actively the with appropriate services people with disabilities. Kubota et al. [211] also propose a similar hybrid AmI-Robotic systems for aiding disabled people with quadriplegia.

Recently there has been some attempt to further improve the sensor networks rehabilitation capabilities by sensing electroencephalography (EEG) signals directly using BCI technology. BCI systems represent a natural extension for AmI environments. Indeed, they are envisioned to be typically used for allowing smart environments habitants to deal with their surrounding space in a transparent way. This effortlessly interaction approach is particularly suitable for enhanced rehabilitation systems. The ASPICE [212] and DAT [213] projects are examples of this kind of technology which allow the temporary or permanent neuro-motor disabled persons to improve or recover their mobility (directly or by emulation), as well as their communication skills.

D. Persuasive Well-being applications

Persuasive technology [214], represents computing systems, devices, or applications intentionally designed to change a person’s attitude or behavior in a predetermined way in order to motivate people to lead a healthier life style by mediating prevention and treatment [215]. Although the field of persuasive technologies has lately attracted lots of attention, only recently the notion of ambient persuasive technology was introduced [216][217][120]. Ambient persuasive technology constitutes a radically new category of relationships between human beings and technological artifacts by blending insights from the behavioral sciences with computing technology [218]. One of the first examples of computerized persuasion system for healthy living is the Persuasive Mirror [219]. This system uses ubiquitous sensors for continuously gathering information about human behaviors and provides user with continuous visual and non-intrusive feedback matching the psychological strategy (see Figure 5). Other applications of the ambient persuasive mirror are introduced in [220].

Fig. 5.

The Persuasive Mirror project at works [219].

Another seminal application of ambient persuasive technology is provided by the HygieneGuard projector [221]. This environmental persuasive system is used in restaurant and hospitals to motivate employees or workers to wash their hand before leaving the restrooms. The equipment is installed in the restrooms and every employee is required to wear a badge. Whenever the employee goes to the restroom, she has to use the sink for a period of time. De Carolis et al. [216] presents an approach to ambient persuasion based on a combination of pervasive and distributed computation to motivate users in a fitness center. The user is surrounded by several connected devices that cooperate during persuasion process. Another interesting AmI application based on persuasive technologies is PerCues [222]. Different from the previous applications, PerCues is oriented to achieve a collective human-well being by persuading users to reach a common goal like decreasing environmental pollution. The project perFrame [223] implements a persuasive interface in form of an interactive picture frame which integrates unobtrusively into the working environment and provides affective feedback in order to persuade employees for better healthy habits while working with a computer.

Etiobe [224] is another project devoted to treat child obesity. Its architecture merges ubiquitous, intelligent and persuasive features for implementing a cyber-therapy approach. It is based on virtual and augmented reality, and attempts to persuade children to avoid poor eating habits. The systems uses a collection of environmental sensors for capturing important information such as contextual, physiological and psychological data.

Lastly, some game-based ambient persuasive systems for well-being have been introduce or they are under development [225]. For an example, the project Dance Dance Revolution connects a sensor-enabled dance floor with a video interface and provides stimulating exercise as dance competition [226]. A recent trend is the use of motion sensing controllers such as WiiMote or Kinect sensor, allows the individuals to naturally manipulate digital worlds in persuasive games. Taken together, this body of work demonstrates that games and social competition can be used to establish long-term commitments. For example, such games can be used by the elders or individuals with physical impairments during their rehabilitation sessions.

E. Emotional Well-being

Recent advances in neurology and psychology have demonstrated the importance of emotions in various aspects of our lives and, in particular, in the field of healthcare and well-being. Indeed, it has been demonstrated that negative emotions have adverse effects on the immune system of a person [227]. Emotions are typically communicated through three channels: audio (speech), face and body gestures (visual), and internal physiological changes such as blood pressure, heart beat rate, or respiration.

AmI sensor-based infrastructures may represent a suitable tool for recognizing and managing emotions, as well as for improving well-being. McNaney et al. [228] have designed a Wearable Acoustic Monitor (WAM) device, which provides support in various aspects of social and emotional wellbeing by inferring level of social interaction and vocal features of emotionality. It can monitor and evaluate the level of wearer’s voice by identifying vocal features such as amplitude, pitch, rate of speech and pause length in order to provide insight into the emotionality of the wearer at a given time. This feature allows the individual to reflect on the contexts or situations which prove particularly stressful or pleasurable, and may affect future behaviors. Another interesting application of environmental sensors and AmI to emotional well-being is AffectAura [229] project. This system continuously predicts users’ valence, arousal and engagement based on information gathered from webcam, kinect sensor, microphone, electrodermal activity sensor, GPS, file activity sensor, and calendar scraper. The users were allowed to leverage cues from AffectAura to construct stories about their days, even after they had forgotten particular incidents or their related emotional tones. Another project, EmoSoNet [230], introduces an emotion-aware social network for the purpose of increasing emotional wellbeing. The framework uses sensors and behavior analysis methods in order to infer users’ stress level automatically with minimal user effort. It also uses audio, animation and vibro-tactile feedback for enhanced engagement. Another system named MONARCA [231] develops and validates solutions for multi-parametric, long term monitoring of behavioral and physiological information relevant to bipolar disorder. In particular, the system consists of a sensor enabled mobile phone, a wrist worn activity monitor, a physiological sensor (GSR, pulse), a stationary EEG system for periodic measurements, and a home gateway. Combining the sensor information with patients’ medical records and established psychiatric knowledge, a prediction of depressive and manic episodes is given.

The Emo&Pain project [232] is an intelligent system that enables ubiquitous monitoring and assessment of patients pain-related mood and movements. Specifically, this system aims to develop a collection of methods for automatically recognizing audiovisual cues related to pain, behavioral patterns typical of low back pain, and affective states influencing pain. Aziz et al. [233] also propose an animated conversational agent providing emotional support and companionship in order to promote the emotional wellbeing of patients and enhance patient care and outcomes during a hospital stay.

F. Smart Hospitals

AmI technology can be also useful for other stakeholders such as nurses, doctors, and other health care personnel, especially for facilitating communication among them. Sanchez et al. have developed the iHospital project which provides context aware communication based on activity recognition [234]. Various pieces of contextual information are collected and used, including location, time, the roles of the people present, and RFID-tagged artifacts, in order to help in decision making and communication.

There have been some efforts to create AmI based middleware for healthcare. Rodriguez et al [235] describe development of SALSA, an agent based middleware to facilitate responding to the particular demands of patients and hospitals personnel. SALSA takes into account the distributed access nature of doctors which is a result of their high mobility. A doctor has to access patients’ clinical records, access medical devices distributed throughout the premises, and communicate with colleagues spread throughout the hospital. In order to track the location of people, Rodriguez et al. use radio frequency signal strength between mobile devices and access points and build a signal propagation model to estimate the distance.

Favela et al. [236] describe several possible scenarios for using AmI in hospitals and build their frameworks around those scenarios. For example, Dr. Garcia is checking the patient in bed 234, when he is alerted that a new message has arrived. His handheld device displays a hospital floor map informing him that the X-ray results of another patient are available. He approaches the nearest public display that detects his presence and provides him with a personalized view. Dr. Garcia selects the message on bed 225, which opens windows displaying the patient’s medical record and the X-ray image recently taken. Aware of the context of the situation, the system automatically opens a window with the hospital’s medical guide that relates to the patient’s current diagnosis, and an additional one with previous similar cases to support the physicians’ analysis. While Dr. Garcia is analyzing the X-ray image, he notices on the map that a resident physician is nearby and calls him up to show him this interesting clinical case.

Kofod et al. [237] also describe using AmI technology for supporting health workers cooperating in patient diagnosis and treatment by using context information, goal recognition and case based reasoning. GerAmi (Geriatric Ambient Intelligence) is another AmI hospital project which helps doctors and nurses to monitor patients and to better manage theirs tasks [238]. For example, it keeps track of the patients’ locations using RFID technology and generates alarms, if needed. It also allocates tasks to nurses based on various contextual information such as the availability of nearby nurses and their profile information.

V. Ongoing Challenges and Future Research Directions

A. Artificial Intelligence

Since the birth of modern computing, around the fifties, scientists and doctors alike were always captivated by the potential that artificial intelligence (AI) methodologies might have had in medicine and health care applications [239]. Indeed, conventional AI methods such as machine learning, expert systems, knowledge representation techniques have been strongly exploited for designing and implementing medical applications belonging to the following areas: diagnose, prognosis, medical training and so on. Expert or knowledge-based systems are the most common type of AI application in medicine. They contain medical knowledge, usually about a very specifically defined task, and are able to reason with ‘data’ from individual patients to come up with reasoned conclusions by using, typically, a set of rules based on some logic inference. When there is not enough knowledge for designing a medical expert systems, machine learning approaches can be used for analyzing a collection of clinical cases producing a systematic description of those clinical knowledge that characterize the clinical conditions of a give patient or disease. Moreover, techniques for knowledge representation (e.g. ontologies) have been used for formally gather medical knowledge and implement tutoring systems for young doctors or nurses. These only some samples of AI applied to health care domain but, in general, of all of AI health care systems have to do with the manipulations and transformation of data and knowledge. In this scenario, approaches based on pervasive or ubiquitous hardware will be very useful for enhancing the current state of the art of AI in health care [240], [241], [242], [243], [244], [245], [246], [247], [248]. Indeed AmI features could allow systems’ designers to develop complex software architectures capable of analyzing the knowledge spread on everyplace in surrounding environments and, as a consequence, learning distributed expert systems classifying diseases or other health disorders depending upon the environmental content and yielding performances better than conventional data mining approaches. Thanks to these innovative features, in the future it will be possible to design and develop smart environments characterized by an intelligence similar to HAL 9000 ². More specifically, these environments will be able to not only instantly recognize who he was interacting with, but also whether he could lip read, judge aesthetics of visual sketches, recognize emotions subtly expressed by people and respond to these emotions in an adaptive personalized way. Consequently, they will increase the human health by providing therapy, health support without intervention of medical staff.

B. Design and Human Factors

The next generation of AmI systems will try to improve the quality of human life by means of increasingly innovative applications in the health care domain or, in an implicit way, by assuring high comfort together with an intelligent usage of scarce resources. This benefit will be mainly achieved by designing intelligent environments completely pervaded by different kind of sensors whose wireless interconnection could result in a larger human exposure to radio frequency electromagnetic fields. Although the electromagnetic fields from wireless sensors are weak, a long time exposition can raise concerns about possible health hazards. Indeed, some recent studies indicated a direct link between exposure to radiation from wireless devices and an increased risk of acoustic neuronal, a cancer of the nerve connecting the ear to the brain. For this reason, the next generation of AmI systems must be designed by taking into account additional factors such as, sensors positioning, sensors mobility, sensors radiation and so on. By following these design guidelines, future AmI frameworks will be fully suitable for enhancing human health without providing dangerous side effects. This challenge can only be tackled with the joint efforts of computer scientists, architects, doctors, physicists and telecommunication engineers. Moreover, government regulatory authorities need to direct policy decisions intelligently to maintain a useful and safe future of humans [249].

C. Security and Infrastructure

The wealth of data that is collected in AmI systems can be beneficial in many aspects. However it also opens up many security issues. Particularly for health care systems, privacy and security are already very complex issues, and the addition of a large number of sensors and devices will result in additional challenges. Wright et al. [250] discuss several AmI security issues and call for the need to develop safeguards. They describes various dark scenario where AmI might result in serious security breaches. For example, burglars might obtain various details about an older adult living alone and his habits for leaving home at a particular hour, and use such information for breaking into the smart home system, potentially even endangering his life. Lack of interpretability and overly restricted access might also pose some issues. For example, out of date health monitoring devices might result in fatal misdiagnosis by paramedics in an accident, or they might not allow the paramedics to access such information in case of incompatible devices used in different geographical regions.

Bohn et al. [251], as well as Van Heerde [252] discuss the potential security dangers of AmI due to the ability of such systems to collects large amounts of data about each individual. They also warn against the future possibility of an invisible and comprehensive surveillance network. Friedewald et al. [253] note that the unidirectional flow of information in AmI systems might result in an asymmetric data distribution. For example, heath providers have access to every detail of daily life of an individual, but the individuals might not have enough information to decide and choose among their providers, or even if they have access, they might lack the sophisticated means necessary to process such complex web of information.

Sensors which are used in AmI systems might also be a source of concern. For example, RFID tags have already been subject of many security discussions. The fact that an object with an RFID tag can be uniquely identified and tracked back to its user, might bring up many privacy issues. Also, many AmI sensors and devices will rely on wireless protocols for communication and as wireless communications are easier to intercept, all communications should be encrypted and made secure. Personal monitoring devices should authenticate the identity of their users using unobtrusive biometrics or possibly key physiological signs to avoid any data tampering (owner-aware devices).

In general, Kotz et al. [254] identify three particular features of remote home health care that have implications for privacy: (1) more medical data will be collected about a patient over extended periods, (2) broader health data will be collected about the patient (physiological data, information about an individuals lifestyle, activities, etc.), and (3) A broader range of applications will use the collected data. To allow for provision of health care in such system and at the same time to maintain privacy, a “Privacy by Design” (PbD) approach can be taken [255]. PbD advocates full privacy provisions during design in a proactive manner. In other words, once the users privacy requirements have been determined, then the design of the sensor system itself can be completed.

D. Social and Ethical Issues

The initial European commission (ISTAG) report [256] identified several characteristics necessary for social acceptance of AmI system in general: facilitate human contact; be oriented towards community and cultural enhancement; inspire trust and confidence; be controllable by ordinary peoplethere should be an off-switch within reach.

Overreliance on AmI systems might have its own dangers for the individuals with health needs and might result in the early loss of the ability and confidence to manage their life. Care must be taken to ensure that AmI technology is not limited to the affluent individuals, because less privileged individuals can also benefit from the benefits of AmI.

Fear of decreased communication and patient isolation is another ethical issue that many researchers have brought up. Identifying where the problem lies in a misdiagnosis will become more and more difficult in such complex system and will result in many ethical and legal discussion.

VI. CONCLUSION

AmI paradigm represents the vision of the next wave of computing. By relying on various computing and networking techniques, AmI promises the successful acquisition and interpretation of contextual information, and offers an unobtrusive and intuitive user interface. Thanks to these features, AmI systems have the potential to enhance our everyday life in many different aspects and, in particular, one of the areas which promises the widespread use of this innovative paradigm, is the health care domain.

In this survey, we explored the application of ambient intelligence to the health care from various perspectives. We discussed the use of AmI in health care based on individuals medical conditions, such as physical or mental disabilities, chronic disease or rehabilitation situations. From a different perspective, we discussed current technology and infrastructure, such as smart environments, assistive robots, wearable sensors, and smart fabrics. More importantly, we provided a high level description of various AmI methodologies in the health care domain, such as automated decision making, planning techniques, activity recognition and other numerous techniques.

However, we are aware that the goals set up for AmI in health care are not easily reachable and there are still many challenges to face and, consequently, this research field is getting more and more impetus. Researchers with different backgrounds are enhancing the current state of the art of AmI in health care by addressing fundamental problems related to human factors, intelligence design and implementation, and security, social and ethical issues. As a result, we are confident that this synergic approach will materialize the complete vision of AmI and its full application to health care and human well-being.

Fig. 3.

The scalable electronic sensate skin from MIT.

TABLE III.

Ambient sensors used in smart environments.

Component	Example Techniques	Example Applications
Activity Recognition	Graphical models	Automation
Behavior Discovery	Sequence mining	Behavior monitoring
Anomaly Detection	Statistical methods	Emergency detection
Planning	D-HTN	Prompting
Decision Support	Knowledge based	Enhancing communication among health care personnel
Anonymization	K-Anonymity	Privacy preservation

Biographies

Giovanni Acampora received the Laurea (cum laude) and Ph.D. degree in computer science from the University of Salerno, Salerno, Italy, in 2003 and 2007, respectively. Currently, he is Assistant Professor in Process Intelligence at School of Industrial Engineering, Information Systems, at Eindhoven University of Technology (TU/e). Previously, he has been a Research Associate at Department of Computer Science, University of Salerno. He was also a Member of the Multi-Agent Laboratory at the University of Salerno and scientific co-responsible of the CORISA Research Centre. From September 2003 to June 2007, he was also involved in CRDC-ICT Domotic project, where he was engaged in the research on multi-agent systems and artificial intelligence applied to ambient intelligence environments. In this context, he designed and developed the Fuzzy Markup Language, an XML-based environment for modeling transparent fuzzy systems. Currently, FML is under consideration by IEEE Standard Association to become the first standard in computational intelligence. He has written some seminal papers on ambient intelligence and, in particular, his work about fuzzy computation in smart environments is one of the most cited paper of IEEE Transactions on Industrial Informatics. Since 2010, he serves as Secretary and Treasurer of Italian Chapter of IEEE Computational Intelligence Society. Dr. Acampora serves as reviewer and guest editor for several international journals and conferences. Dr. Acampora is the chair of the IEEE Computational Intelligence Society Standards Committee. He also served as Chair of Task Force on Taxonomy and Terminology and as the Vice-Chair of Task Force on New Standard Proposal.

Diane J. Cook is the director of the CASAS Smart Home project at Washington State University (WSU). She has conducted research in the areas of machine learning, data mining, and smart environments for over 15 years and has over 320 peer-reviewed publications in these areas. She led the efforts that supported the creation of the MavHome smart environment. She is currently serving as the director of several programs centered on the theme of health-assistive smart environments. Cook is an IEEE Fellow and has received numerous awards from IEEE, Lockheed Martin, UTA, and Washington State University. She serves on the editorial board of six international journals and has experience leading and collaborating with large multidisciplinary teams working on various projects on smart environments and assistive technologies.

Parisa Rashidi is an assistant professor at Northwestern University. Her research interests include ambient intelligence, ambient assisted living systems and applying data mining and machine learning techniques to various health care problems. She has organized workshops and tutorial sessions regarding context aware systems and assisted living technology and she has served on the technical program committee of several conferences including ACM International Conference on Knowledge and Data Discovery (KDD), the IEEE International Conference on Data Mining (ICDM,) IEEE International Conference on Tools with Artificial Intelligence (ICTAI) and the IEEE International Smart Environments to Enhance Health Care (SmartE). She has been a reviewer of numerous journals, such as IEEE Transactions on Knowledge and Data Engineering (TKDE), IEEE Transactions on Systems, Cybernetics, Man (TSCMA), ACM Transaction on Intelligent Systems (TIST), among others.

Thanos Vasilakos is currently Professor at the University of Western Macedonia, Greece and a visiting Professor at the National Technical University of Athens (NTUA). He has authored or co-authored over 200 technical papers in major international journals and conferences. He is author/coauthor of five books and 20 book chapters in the areas of communications. He has edited some important books in the field of Ambient Intelligence. Prof. Vasilakos has served as the general chair and the technical program committee Chair for many international conferences. He served as an editor or/and Guest Editor for many technical journals, such as the IEEE Transactions on Network and Services Management, IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, the IEEE Transactions on Information Technology in Biomedicine, ACM Transactions on Autonomous and Adaptive Systems, IEEE Communications Magazine, ACM/Springer Wireless Networks(WINET), ACM/Springer Mobile Networks and Applications(MONET). He is the founding Editor-in-Chief of the International Journal of Adaptive and Autonomous Communications Systems, and the International Journal of Arts and Technology. He is the general chair of the council of computing of the European alliances for innovation.