A Conceptual Framework for Clinicians Working with Artificial Intelligence and Health-Assistive Smart Homes

Abstract

The smart home designed to extend older adults independence is emerging as a clinical solution to the growing aging population. Nurses will, and should play a key role in the development and application of Smart Home technology. Accordingly, conceptual frameworks are needed for nurse scientists who are collaborating with multidisciplinary research teams in developing an intelligent Smart Home that assists with managing older adults’ health. We present a conceptual framework that is grounded in critical realism and pragmatism, informing a unique mixed methodological approach to generating, analyzing and contextualizing sensor data for clinician-based machine learning. This framework can guide nurse scientists in knowledge construction as they participate in multidisciplinary health-assistive Smart Home and artificial intelligence research. In this paper, we review philosophical underpinnings and explicate how this framework can guide nurse scientists collaborating with engineers to develop intelligent health-assistive Smart Homes. It is critical that clinical nursing knowledge is integrated into Smart Home and artificial intelligence features. A conceptual framework and practical method will provide needed structure for knowledge construction by nurse scientists.

Introduction

The rapid growth of the aging population is a global concern (United Nations, 2015). A change in the world’s population by age from 2010–2015 includes a 188% increase in people 65 and older, a 351% increase of people 85 and older and a 1004% increase of those 100 years and older (United Nations, 2010). Older people are living longer (Healthy People 2020, 2018). However, not all people experience healthy aging. Longer life expectancy is also associated with higher prevalence of chronic disease (Avendano, Glymour, Banks, & Mackenbach, 2009; Prince et al., 2015). Commonly, older adults with deteriorated physical or mental health and function utilize more health care (Agency for Healthcare Research and Quality, 2017; Healthy People 2020, 2018; Prince et al., 2015). The current practice of housing and caring for older people in care centers is unsustainable without innovative collaboration across industries (World Health Organization, 2018), and interdisciplinary research teams are called upon to develop innovative solutions that include technology to promote healthy aging (National Institute on Aging, 2017; National Science Foundation, 2017). Further, caregivers who want to support their loved ones’ to age-in-place, need a system that can assist with providing for their loved one’s health and safety (Healthy People 2020, 2018).

Aging-in-place with health-assistive smart home technology (hereafter referred to as Smart Home), may be an alternative to assisted living or nursing home admission (National Institute on Aging, 2017). Smart Homes use motion sensors to track activities of the aged, and harnesses the power of artificial intelligence to identify and predict changes in health states (Cook, Krishnan, & Rashidi, 2013b). The Smart Home is designed to assist the aging person to remain in their home and extend independence (Rantz et al., 2014; 2015a; 2015b). Additionally healthcare system leaders are interested in decreasing hospital utilization and the associated costs (American Hospital Association, 2018; Center for Medicare and Medicaid Services, 2017). As the integration and refinement of Smart Homes evolves, nurse researchers will need to be “in-the-loop” (described below) to make sense of Smart Home-generated data, and provide clinical ground truth to engineers to improve efficacy of Smart Home design and utilization by integrating theoretically based knowledge development and clinical knowledge application. Accordingly, conceptual frameworks are needed to guide nurse researchers involved in Smart Home research. Further, to concurrently evolve and evaluate technology for use in patient care, pragmatic approaches and agile methodologies for conducting clinical Smart Home research are critical (Kaye et al., 2011).

In this methodological paper, we introduce a conceptual framework for nurse researchers working with Smart Home data for training artificial intelligence (AI) algorithms (i.e., machine learning algorithms). We discuss our epistemological and ontological leanings, which informed the mixed methodology and neoteric Fritz Method that nurse researchers can use to integrate clinical knowledge into AI for clinician-based machine learning. This article is organized in the following manner: (a) we describe the Smart Home and the nature of the data. (b) We introduce the Clinician-in-the-Loop-AI (CIL-AI) conceptual framework and discuss how knowledge is constructed within the framework, which includes an innovative data collection and analytic method, the Fritz Method; and (c) we briefly introduce the Fritz Method and describe how it generates data, guides analysis, and facilitates provision of ground truth for Smart Home AI.

The Health-Assistive Smart Home

‘Smart Home’ is a general term primarily used in two ways: when referring to consumer-driven in-home ‘smart’ products (e.g., Amazon’s® Alexa), or when referring to technology that assists with in-home delivery of healthcare aging-in-place technologies such as Life Alert®, AngelSense®, GPS Smart Sole®). In this article, we refer to ‘Smart Home’ as the Washington State University Center for Advanced Studies in Adaptive Systems (CASAS) Smart Home in a Box (SHiB) (Cook, Crandall, Thomas, & Krishnan, 2013a). This Smart Home is designed to extend independence by assisting with health management in older adults. It is a home that perceives its environment through sensors and can act upon the environment using actuators. Actuators are small devices that can receive remote digital instructions and activate or de-activate another device to which they are attached. For example, an actuator could turn off a stove that has been left on accidentaly, after receiving digital instructions that sensor movement in the stove area for a certain length of time is absent while the temperature is increasing (Cook & Das, 2005). The Smart Home consists of hardware (e.g., sensors and relays) installed in existing residences. Sensor types are infrared motion (direct and area), door use (contact), temperature, humidity, and light. Sensors are placed in specific locations in every room to track patterns of motion that represent the activities of daily living of the residents living within.

Nature of smart home data.

Smart Home data consist of a date, time stamp, sensor labels (e.g., Bedroom, a door), sensor states (e.g., ON/OFF), and activity labels (e.g., sleeping, grooming) (Figure 1). A single sensor may vacillate ON/OFF, or multiple sensors may vacillate ON/OFF, multiple times per second. At other times, all sensors may remain quiet (inactive), indicating the resident is out of the home or sleeping. Such data results in thousands of lines of text amounting to thousands of data bytes per day and hundreds of megabytes per person each year. Though a single person’s annual data is not large (think storing 3–4 pictures or streaming 15 minutes of music), it is too much data for a human to process in a meaningful way. Further, if hundreds or thousands of people were monitored in this fashion, the compiled data would be considered ‘big data.’ Big data are extremely large sets of data the can be analyzed computationally and which can reveal trends and associations that regard human behaviors and patterns (“Big Data Analytics,” 2018).

Figure 1:

Smart Home sensor data (date, time, sensor, state of sensor). The data represents a resident in their living room recliner chair on February 19, 2018 just after 5:00 A.M.

Data are low fidelity (showing only a partial picture of a comprehensive situation) and lack meaning without the appropriate application of AI tools, which assist with interpreting the meaning of the data. Data alone do not contain personal health information (PHI) and are not secured with Health Insurance Portability and Accountability Act (HIPAA) requirements. Once the machine (a computer running software that includes AI algorithms capable of monitoring, labeling, and assessing all incoming data) labels the data with participants’ daily activities (e.g., eating, grooming, sleeping), the fidelity is heightened but it remains low. It is plausible that in the future Smart Homes will be capable of using sensor data to identify and predict health events. It is also plausible that higher fidelity data (data providing a clearer picture of the patient enabling identification by detecting unique and precise patterns) will need HIPAA protections, which are not currently in place.

Conceptual Framework

Assumptions and Definitions of Key Concepts

The extant literature indicates that philosophical paradigms contribute to the theoretical position of researchers, which in turn influences the methods used to conduct research (Klein & Holly, 1990). While there are differences in the methodological perspectives among nurse researchers (quantitative positivist stance versus qualitative interpretive stance), methodological debates are not isolated to the discipline of nursing (Baker, Norton, Young, & Ward, 1998; Frost et al., 2010). Today, nurses are less constrained by the dominant discourse of the past—the “paradigm wars”—that demanded an alignment with one of the philosophical paradigms and single methodology for conducting research. Modern methodological pragmatists propose that mixing research methodologies is acceptable if the method is informed by the phenomenon, and is preferably grounded in theory (Landry & Banville, 1992). We embrace Maxwell and Mitapalli (2015), who indicate that while research methods need not be tied to a specific methodological paradigm, the philosophical assumptions should significantly inform research methods (Maxwell & Mittapalli, 2015). Our epistemology and ontology iteratively informs our perspective and the dynamic relationship with technology which significantly contributed to the conceptualization of our framework. See Table 1 for key definitions of key concepts.

Table 1:

Definitions of Key Concepts

Epistemology:	How we know what we know
Ontology:	Inquiry into the nature of things; how things are
Pragmatism:	The nature of the phenomenon determines the method; practical needs to inform the development of research frameworks
[Author] Method:	A pragmatic method that generates data, guides analysis and knowledge contextualization to explain and predict for machine learning
Contextualization:	Participants share their own experience of health behavior change, and how the message was understood is provided by clinicians through feedback
Mixed Methodology:	Integrated quantitative and qualitative participatory design
Clinician-Based Machine Learning:	A process whereby expert nurse clinicians assist in training a health-assistive AI agent by contextualizing sensor data with situational health-event information
Artificial Intelligence:	A computer system (machine made of hardware and software) capable of performing tasks that normally require human intelligence; such as speech or activity recognition, or decision-making, and more
Ground Truth:	Data patterns that are labeled by human experts to provide meaningful context; these data are classified by computer scientists and fed to the AI machine to train it

Because of the nature of sensor data (quantitative) and the nature of contextualized clinical aspects that inform sensor data (qualitative), an integrated approach to expand knowledge construction in Smart Home research was developed. A main goal of Smart Home research is to extend independence and facilitate aging-in-place by assisting with management of chronic conditions and health maintenance. Researchers are using AI to accomplish this goal. The AI agent can automatically identify and predict clinically relevant changes in older adults’ health states (Sprint, Cook, Fritz, & Schmitter-Edgecombe, 2016), and can take an action on behalf of the resident (e.g., identifying a fall and notifying someone). For the AI agent to work as intended, it must first be “trained” to do so. Training is achieved using machine learning tools and techniques (e.g., supervised learning) and by ‘feeding’ data of individually unique and complex movement patterns to the machine (Cook & Das, 2005) (See ‘Ground Truth’ in Table 1.). When a negative health event occurs, or is about to occur, older adults’ routine patterns are altered (Kaye et al., 2011; Sprint, Cook, & Schmitter-Edgecombe, 2016). For example, a congestive heart failure patient experiencing shortness of breath may spend the night in his recliner chair instead of his bed, so that the head is elevated and he can breathe better. The Smart Home can be trained to detect this change in health behavior as a clinically relevant anomaly. To accurately train the Smart Home, ground truth labels are needed. Nursing knowledge is essential and integral to these ground truth labels. Consistent application of a framework will enhance ground truth because the application of nursing knowledge to large sensor data sets is an unexplored area of research. A well-defined philosophical approach is a necessary component of explicating this framework because of the growing gray area between human and artificial intelligence. Understanding the ‘stance’ informs the purpose, meaning, and importance of infusing the voice of the nurse researcher with clinical nursing expertise into the AI agent.

Clinician-in-the-Loop – Artificial Intelligence (CIL-AI) Framework

Critical realism and pragmatism are the foundation to the CIL-AI framework, and the post-phenomenological thought about the relationship between technologies and a person’s experience informed our mixed methodology and participatory (see Participatory section below) Smart Home research design. These tenets also informed the development of the “Fritz Method,” which uses an integrated quantitative and qualitative participatory design to contextualize Smart Home data so causal mechanisms can be understood leading to clinician-based machine learning (The Fritz Method is briefly introduced below).

Critical Realism.

Critical realism is a post-positivist philosophy based on the writings of Bhaskar (1978; 1989) that emerged four decades ago in response to the inadequacies of both positivism and interpretivism. It is considered a meta-theoretical philosophical position that has been likened to family resemblances and commonalities that naturally intertwine (Critical Realism: Essential Readings, 1998; Gorski, 2013). In critical realist philosophy, the causal processes are viewed as intertwined with a person’s beliefs and meanings. With a focus on socially constructed knowledge (Lincoln & Guba, 1985), critical realism embraces interpretation, and includes contextual aspects of human existence (i.e., psychological, social, behavioral). Critical realism does not negate the usefulness of a quantitative paradigm, but emphasizes the need to truly understand the phenomenon and consider the causal processes of social situations that people are in, including the aspects of beliefs and meaning and how people are socially constructed (Gorski, 2013; Maxwell & Mittapalli, 2015). Critical realism has informed empirical investigations across a variety of social sciences and other fields, including nursing (McEvoy & Richards, 2006). The concept of human agency where the mind is viewed as an emergent power of both matter and reason leading to actions (Critical Realism: Essential Readings, 1998) is exhibited in the social interactions of older people and healthcare providers seeking to make changes in the realities that older adults encounter as they age. Social construction and contextual aspects of human existence—as it is known by older adults and people in their social planes—inadvertently affect the idea of how and where older adults should live as they age, and how technology could influence aging-in-place.

Post-phenomenology.

The idea that technologies and artificial intelligence can be used to recognize and intervene in changes in older adults’ health, solidifies the reality of technology mediating the daily lives of older adults. The influence of technology on people is of interest to researchers. Phenomenology explores the lived experience (Heidegger, 1996). Post-phenomenology, influenced by both descriptive and hermeneutic phenomenology (Aagaard, 2017), is a philosophy of technology with the primary tenant of technologies actively mediating a person’s every-day being (Idhe, 1990). Technologies are said to have social and cultural roles in that technologies do not only function for their specific purpose, but also have the capacity to transform the experiences of people. In other words, how a person experiences their daily life can be transformed by technologies (Verbeek, 2005). This is important because Smart Home technologies could transform how and where older people choose to age and therefore influence how older people experience aging. These realities, along with the existence of nuanced experiences of older adults residing in a Smart Home, need to be carefully considered and methodically captured. While the main objective of Post-phenomenology is to explore the mediation of technology and explain the relationship between the technology and the experience of the person, we apply it in providing a blueprint for nurse researchers conducting Smart Home research.

Artificial Intelligent agents learn complex human motion patterns based on the nature of the sensor data, and the interactive relationships between older adults and the Smart Home, and researchers (Fritz, Corbett, Vandermause, & Cook, 2016). The nature of sensor data and AI functionality are integral to our epistemology, and ultimately led to the participatory aspect of research design. Ontologically, we acknowledge the existence of a relationship between humans and AI, and technology (Idhe, 1990). We assert there is a complex interplay of realities between older adults’ needs and desires, and clinical researchers’ professional roles and the AI agent designed to assist both. We embrace Maxwell and Mittapalli’s (2015) proposal that creating a dialogue between people with different perspectives is important to deepen the understanding of a phenomenon. We acknowledge the need for deep understanding for providing ground truth. We believe a dialogue, based on deep understanding of context, will improve the AI agent by facilitating the use of well-chosen, clinically relevant data sets. We recognize that multiple realities exist when clinicians seek to understand and interpret a participant’s reality and experience. We also know that engineers and computer scientists rely on our presentation of this reality for ground truth. Our goal is to provide sensor data that informs the Smart Home’s understanding of the human response to health and illness because we recognize that the human’s reality of their own experience is not directly measurable using sensors (Figure 2).

Figure 2.

The relationship between epistemology, ontology, methodology and Fritz Method for CIL-AI knowledge construction in health-assistive smart home research.

Naturally, a mixed methodological approach to generate data and guide analysis in Smart Home research cannot be viewed as situated at polar ends of the spectrum, but needs to be successfully integrated to complete the picture (Fritz & Vandermause, 2017). We take a pragmatic approach to engaging with the multidisciplinary Smart Home team. We consider the key stakeholders to be older adults, engineers, computer scientists, psychologists, nurses, and the AI agent itself. We include older adults’ voices via our participatory design. We avoid long narratives, and methodically construct our dialogue with the multidisciplinary team so that clinical information can be clearly understood. To improve clarity, we use a pragmatic and concise dialogic style familiar to engineers (e.g., spreadsheets with definitions and descriptors). For psychologists, we provide some narrative highlighting neurocognitive symptoms and diagnosis. For the older adult, we verbalize our goals, plans, and our findings; and we reiterate as needed. We build and share our knowledge within this critical realist, pragmatic, participatory paradigm. The shared knowledge leads to an AI agent that is based in clinical knowledge derived from participants’ experiential and physical states of being.

Participatory Design.

In using the participatory approach, our goal is to develop accurate clinical ground truth, which is important for clinician-based machine learning. Using a participatory research approach fosters greater input of the perspectives of older adults (Muller & Kuhn, 1993). Central to Smart Home development are older adults who prefer to age-in-place, their social support system (family, friends, caregivers), and senior living corporations (e.g., continuing care retirement communities, assisted living facilities), and their clinicians (nurses and doctors). All of these stakeholders are critical partners, without which, a good end-user experience is not possible. A primary goal of the participatory research design is to directly include the targeted end-user and all stakeholders in development of the Smart Home. We purposefully recruit older adult participants with chronic health conditions and engage community stakeholders interested in emerging aging-in-place options. Using a participatory approach encourages future end-users (i.e., older adults and caregiving agencies) to express their needs, wishes and insights (Levasseur et al., 2016; Muller & Kuhn, 1993).

Older adults are autonomous beings who have the right to decide on their own destiny. Accordingly, we place emphasis on a person’s right to self-determination and right to autonomy (American Nurses Association, 2015). The principles of ‘right to self-determination’ and ‘right to autonomy’ inform the partnership with the older adult end-user. For example, during periods of data collection we periodically show participants their data and explain how the data contributes to building a better Smart Home with a future goal of aging in place. Similarly, we elicit healthcare team feedback, which is critical to visual analytic design of Smart Home dashboards used by the healthcare team for intervention planning.

Multidisciplinary Collaboration in Smart-Home Research

Smart Home research draws on multiple disciplines including engineering, computer science, nursing, and psychology (Rantz et al., 2015a; Sprint et al., 2016), and an emphasis is placed on shared meaning between the disciplines (Morgan, 2007). Each discipline has discipline-specific goals and research methodologies that contribute to the end goal – aging in place. Accordingly, the integration of research methodologies is both natural and necessary (Teddlie & Tashakkori, 2009). For example, engineers and computer scientists focus on quantifiable aspects and technical results while nurses focus on experiential aspects and pragmatic solutions. This well-choreographed collaboration between the disciplines, and the mixing of methodologies, illuminates the naturally emerging relationship between “hard” data (sensor) and “soft” data (human context). The ability for engineers and computer scientists to validate and enhance analytics through the nurse researcher’s (clinician-in-the-loop) clinical vignette of social context and human reality contributes to the development of the Smart Home (Figure 3).

Figure 3.

Clinician-in-the-loop; shows the important role of nurses as the clinician-in-the loop providing an expert guided approach to machine learning.

To provide these contextual vignettes of human reality to the engineering team the nurse researcher conducts weekly telehealth and monthly in-home nursing assessments and records health events reported by participants (e.g., a fall on Tuesday night or flu-like symptoms on Friday). If a fall is reported, the nurse researcher collects information on the time and location, how long the patient was on the floor, how the participant was able to get up, or how help was called, and activities occurring just before and after the fall. Once health events are identified, the sensor data is labeled to indicate when the health event occurred. Additionally, the nurse researcher provides contextual information on the participant’s: (a) diagnosis, (b) symptoms (chronic and acute – at time of event), (c) diurnal rhythm, (d) normal daily routines, (e) normal sensor data patterns for the associated time of day, (f) abnormal sensor data noted at time of event, (g) beginning and ending lines of data encompassing the event. After the data are annotated and the contextual information has been recorded, the nurse researcher checks back with the participant (i.e., during the next telehealth or home visit) and verifies that recorded information aligns with the participant’s recollection of the event. If discrepancies arise, the nurse adds a comment to the report, quoting verbatim the participant’s description of the event.

Other roles of the nurse researcher are to: (a) communicate which data visuals (graphs showing trends) are meaningful and easy to read, (b) assist with maintaining a focus on real-world context, (c) repeatedly assess participants’ comfort level with the smart home. Additionally, the nurse supports older adults right to self-determination and autonomy by iteratively moving multidisciplinary conversations toward developing smart home features that are clinically pragmatic, and that older adults deem relevant and necessary.

Fritz Method: A Brief Introduction

The “Fritz” Method integrates activity recognition, discovery of clinically relevant sensor data, and clinical interpretation (contextualization) of sensor data. This dynamic, practical method is part of the conceptual framework and used for CIL-AI knowledge construction. The Fritz Method uses a qualitative approach where the Smart Home resident’s perspectives and experiences are explored, and included in the process of analyzing their quantitative sensor data. The participants’ health context, social context, the lived experience, beliefs and meaning, and their rights are considered. These components guide concurrent data generation and analysis of quantifiable sensor data and qualitative contextualization of that data. The Fritz Method enables the nurse scientist to strengthen the knowledge of causal mechanisms, thereby significantly aiding clinician-based machine learning. Importantly, the Fritz Method provides situational knowledge so accurate ground truth labels are provided to the multidisciplinary team.

Conclusion

This paper makes valuable contributions to nursing’s growing involvement in Smart Home research. We proposed an overall conceptual framework for knowledge construction that may be useful to nurse researchers as they embark on smart home research with the multidisciplinary team. We briefly introduced the Fritz Method, a unique approach to concurrent data collection and analysis that includes contextualized health information. We highlighted the important “clinician-in-the-loop” role of nurse researchers who directly inform the development of a health-assistive smart home that uses an AI agent to identify and predict changes in health states.

Although aged care homes are well-meaning solutions to the growing aging population, the voices of older adults have solidified understandings that for many, aging in an institutional setting is often less desirable than aging in place (IFAIA, 2011). A home represents more than “bricks and mortar” to the older person. Older people have an attachment to their home and the community in which the home is situated (Pipher, 1999). However, while a move from the home may not be viewed as entirely negative, and perhaps necessary due to decline in function or other health concerns, not being able to age-in-place may be a stressful, unwanted event for some (Walker, Curry, & Hogstel, 2007; Wiles et al., 2009). Additionally, the enormous costs associated with aging in aged care homes, may soon make aging-in-place in one’s own home with health-assistive Smart Home technology a preferable alternative.

More research is needed to discover if older people would choose to adopt health-assistive Smart Home technology and remain aging-in-place, if this was a viable option. However, nurse researchers with clinical expert knowledge need to be involved in the research and development of the Smart Home. There is a paucity of information on how (and if) nurse-researchers are collaborating with engineers and computer scientists to infuse clinical knowledge into the AI agent used in health-assistive Smart Homes. Nurse researchers may be unaware of the important role and potential for knowledge contribution in Smart Home research. This article draws attention to the role of nurse researchers working with Smart Homes and AI and provides a framework to guide nurses entering this field.