Abstract
Background:
Dementia impairs spatial orientation and route planning, thus often affecting the patient’s ability to move outdoors and maintain social activities. Situation-aware deliberative assistive technology devices (ATD) can substitute impaired cognitive function in order to maintain one’s level of social activity. To build such a system, one needs domain knowledge about the patient’s situation and needs. We call this collection of knowledge situation model.
Objective:
To construct a situation model for the outdoor mobility of people with dementia (PwD). The model serves two purposes: 1) as a knowledge base from which to build an ATD describing the mobility of PwD; and 2) as a codebook for the annotation of the recorded behavior.
Methods:
We perform systematic knowledge elicitation to obtain the relevant knowledge. The OBO Edit tool is used for implementing and validating the situation model. The model is evaluated by using it as a codebook for annotating the behavior of PwD during a mobility study and interrater agreement is computed. In addition, clinical experts perform manual evaluation and curation of the model.
Results:
The situation model consists of 101 concepts with 11 relation types between them. The results from the annotation showed substantial overlapping between two annotators (Cohen’s kappa of 0.61).
Conclusion:
The situation model is a first attempt to systematically collect and organize information related to the outdoor mobility of PwD for the purposes of situation-aware assistance. The model is the base for building an ATD able to provide situation-aware assistance and to potentially improve the quality of life of PwD.
Keywords: Alzheimer’s disease, assistance, data collection, dementia, knowledge base, mobility limitation, situationawareness
INTRODUCTION
People with dementia have difficulties in performing everyday activities [1] and assistive technology devices (ATDs) have the potential to help them maintain their independent social life by supporting their everyday mobility. On the one hand, such systems should only intervene if needed so that the user still relies on his or her own cognitive resources. On the other hand, they should be able to also recognize normal behavior and to potentially reinforce positive behaviors. Such systems have been termed situation aware deliberative ATDs [2]. To be operational, such a device needs to track the user’s actions, recognize the goal, detect the errors in behavior, and decide about the best way of assisting the user [3]. To achieve that the ATD needs domain knowledge in machine understandable format, which describes the actions a user is able to execute, the errors in behavior that may occur, and the reasons for these errors as well as the situation in which the person is. We call this collection of knowledge a situation model. To our knowledge there are no existing situation models for ATDs supporting the outdoor mobility of people with dementia. To address this problem, in this work we present our attempt at building such a situation model for people with dementia. The situation model and the ATD are aimed at assisting people with mild dementia who can still independently interact in society but who sometimes exhibit cognitive problems and disorientation.
In the following, we discuss in more detail the problems associated with mobility in dementia and the potential of ATDs to improve this mobility. Later, we discuss the idea of situation-aware assistance and the concept of situation models for situation-aware assistance.
Mobility in dementia
An active and socially integrated lifestyle protects against cognitive decline in aging [4]. People with mild cognitive impairments have a reduced risk of conversion to dementia when engaged in outdoor and social activities [5, 6]. Furthermore, it allows people with mild cognitive impairments to continue their independent social life and to retain their dignity and self-esteem. In dementia, cognitive rehabilitation shows promising results in delaying further cognitive decline [7]. Outdoor mobility gains importance in this context, both as a prerequisite for an active lifestyle and also as a cognitively demanding activity in itself. Decreased mobility in dementia results from progressive cognitive decline. The complexity of highly dynamic outdoor environments poses an additional obstacle for cognitively impaired people. Indeed, people with dementia report decreased access to socially salient public places and socially stimulating activities [8] due to lack of motivation, previous experiences of errors in complex situations, and feeling of insecurity. The interplay between cognitive decline and decreasing mobility forms a vicious circle: decreased cognitive abilities reduce social activities which in turn impairs cognitive abilities, already affected by the disease process. In this perspective, ATDs that respect the remaining cognitive abilities and provide help for the user when needed might have the potential to break this vicious circle and preserve the autonomy of persons with dementia.
Such an assistive system needs to be able to reason about the user’s situation, as the situation influences the intervention strategy the ATD will follow[9, 10]. To be able to provide situation awareness, however, the system should have access to the knowledge base that represents the various situations related to outdoor mobility. Additionally, a group of individuals with dementia is typically heterogeneous with respect to declined mental abilities, and above that they are very heterogeneous with respect to premorbid characteristics as environmental exposure, desired activities, preferred range and variation of activities, and subjective meaning of particular activities. Thus, an ATD that can make use of situation knowledge has the advantage of providing a standard platform/instrument to quickly develop an individual prediction model or activity profile that is maximally useful for the individual user.
Situation models for situation-aware ATDs
Situation-aware assistance means that the ATD plays the role of a mediator between the person and the environment by observing the person through sensors, predicting his goals, and, only if needed, providing the missing information required to achieve the goal [3, 11, 12]. In other words, the assistance is not limiting the person’s outdoor activities unless necessary. A core element of a situation-aware assistance system is the situation model. It defines the domain knowledge needed to reason about the person’s situation. This knowledge could include personal preferences, social background, external and internal factors influencing the behavior of the person with dementia, elements of the environment he is acting in, etc. [13]. The situation model together with sensor observations is then used to reason about the person’s current situation and goals as well as about possible assistance strategies [3, 14]. It is also used as a codebook for annotating the observed behavior.
Situation models can be obtained in two ways, either through specification-based approaches, or through learning-based approaches [15]. Specifica-tion-based approaches rely on manually incorporating expert knowledge into logic rules that allow reasoning about the situation [15]. On the other hand, learning-based methods can rely on the sensor data to learn the situation [16] or on textual sources to extract the situation related information and its semantic structure [17–20]. In this work, we use specification-based approach to build the situation model. The situation model is often encoded in the form of ontologies. In this work, we consider a situation model to be a special case of ontology that incorporates the domain knowledge required by an assistive system to reason about the user behavior and the possible assistance strategies.
Objective
The objective of this paper is to provide a situation model for situation-aware assistance of people with dementia during their outdoor mobility. This model was developed as part of the SiNDeM project (Funded by the German Federal Ministry of Education and Research (BMBF, reference number: 16SV7091), aiming to enable technology support for outdoor mobility of people with dementia. The model has two purposes: 1) to provide the knowledge base in a situation-aware assistive technology device, namely a device that recognizes situations of disorientation and provides individually tailored guidance in real time; and 2) to be used as a codebook for annotating the data, collected during the outdoor mobility of people with dementia.
As a result, this work provides the basic structure of a situation model of outdoor mobility behavior and the associated cognitive factors that form the basis of knowledge-based situation-aware assistance and that can be used to code (transcribe) behavior traces. The situation model is implemented in the form of an OWL ontology.
METHODS
Development process for situation models
To build the situation model, we followed a development process based on the general ontology process proposed in [21]. It consists of four phases; Fig. 1 shows the process. The proposed process consists of domain analysis, conceptualization, implementation, and maintenance. The domain analysis involves analysis of different scenarios, discussion of relevant problems, observation of use cases, analysis of the requirements of the system, and review of existing solutions. The second step of the process is the conceptualization. It involves the conceptualization of the model and the integration of existing ontologies or relevant information. The last step in the development process is the model implementation. It deals with implementing the formal model in a concrete representation language. In addition, it has the explicit phase “Maintenance” which involves adapting and extending the developedmodel. In parallel to these phases, there are two additional processes. These are the evaluation of the results of each phase and their documentation.
Fig.1.
The proposed development process for situation models. Figure adapted from [21]. The process consists of domain analysis, conceptualization, implementation, and maintenance. In parallel to these phases, there are two additional processes. These are the evaluation of the results of each phase and their documentation.
Domain analysis
System requirements examination
System examination was performed by first reviewing the literature with respect to the problems associated with outdoor mobility in people with dementia. Moreover, we reviewed the requirements identified in existing works addressing the problem of assisting people with cognitive impairments during their outdoor mobility. The literature review followed the principles proposed in [22, 23]. We focused on high-quality peer-reviewed articles and conference papers based on recognized ranking systems [e.g., the Core conference ranking (http://www.core.edu.au/index.php/conference-portal), Scientific Journal Rankings (http://www.scimagojr.com/)]. Furthermore, as the topic of assisting mobility of people with dementia is an emerging one, we also explored scholarly databases (e.g., Elsevier, ACM, IEEE, Google Scholar, PubMed). Our search string was defined using a brainstorming process with the ATD designers and medical experts to elaborate a keyword matrix. The matrix with the search strings contained the following strings: “assistance”, “dementia”, “outdoor mobility”, “assistive system”, “assistive device”, “Alzheimer”, “problems”.
Review of existing ontologies
To identify existing ontologies of interest, a literature review was conducted. We followed the same procedure as above. Brainstorming with experts, we categorized the ontologies into four categories: ontologies describing the behavior of people with dementia, ontologies describing outdoor mobility, ontologies describing the body motion, and ontologies for activity recognition. The following reasoning was behind our choice. Ontologies and studies describing the behavior of people with dementia represent our target group. Outdoor mobility ontologies represent the use case, namely outdoor mobility. Movement ontologies describe the body position and movement. We addressed these ontologies as the body position and motion can be an indicator for a person’s disorientation. Finally, we decided that ontologies for activity recognition are worth looking into, as ATD relies on activity recognition to identify the person’s situation. Furthermore, we looked into the ontology repositories, OBO Foundry [24] and NCBO BioPortal [25]. We used the same strings as during the literature review only omitting the strings “situation model” and “ontology”.
Outdoor mobility study
People with dementia were observed during their outdoor mobility. Ethical approval was granted by the ethics commission at the medical faculty of the University of Rostock (Approval #A2014-0154). The study had two purposes: 1) to collect sensor data that is used for training a model of human behavior during outdoor mobility; and 2) to collect behavior traces for the construction of a situation model. During the initial study, the psychologist followed the participants in 20-min walks, observed their behavior, and later discussed the observed problems with the ontology developer. Finally, the psychologist, together with a computer scientist responsible for the later situation model and ATD development, analyzed the video log of the walks to identify the observed behavior.
Additionally, a life-space assessment was conducted (see Fig. 2). There, the study participants took part in an at least 4-week long assessment of their outdoor mobility. It was recorded through sensors and mobility diaries. The data from the long-term assessment has not been fully analyzed. However, initial analysis provided some insights about the behavior patterns and user preferences.
Fig.2.
Life-space levels as proposed in [29]. Life-space of 0 is limited to the bedroom; Life-space of 1 is limited to the home of the person; Life-space of 2 is limited to the immediate outside of the home; Life-space of 3 is limited to the neighborhood; Life-space of 4 is limited to the town where the person is living; Life-space of 5 is unlimited.
Study participants. Our study participants were patients from the local memory clinic who were diagnosed with amnestic mild cognitive impairment or clinically probable Alzheimer’s disease dementia, according to the national guidelines [26]. Cognitive testing had been performed using the CERAD cognitive battery [27]. Severity of dementia was graded using the Mini-Mental State Exam (MMSE) [28]. We only included people with an MMSE score of 18 and above (mild to moderate dementia). Participants were interviewed about their current level of mobility. Patients who reported to have independently reached a Life-space level [29] of at least three (Level 3 indicates that the person has been outside the house, visited places in the neighborhood) within the last month (see Fig. 2 for details on the levels of Life-space) were eligible to further participate in the mobility assessment. Patients were only included after oral and written information on the study procedures, and after they had given their written informed consent. The study had been approved by the local institutional review board of the University Medicine Rostock.
The mobility assessment study comprises two separate setups, a guided walk and a long-term assessment.
Guided walk. The fifteen participants that answered the above criteria were invited to a 20-min guided walk from which thirteen completed the walk. The sensor system consisted of sensors to measure electrodermal activity, electrocardiography, GPS, and accelerometric data. Data about electrodermal activity and electrocardiography were intended to infer the stress level and possible states of disorientation during the walk. For later analysis and annotation, the walk was videotaped and events were protocoled by a trained psychologist. The motion sensors were attached to the left ankle, the chest, and the left wrist. The sensor data was annotated according to the schema identified in the situation model and it was shown that the data conforms with the behavioral findings in the situation model. Detailed information about the sensors and their placement can be found in Supplementary Material B.
Prior to the recorded and videotaped part of the walk, the participant and the psychologist met at a defined public location (see Fig. 3). The psychologist guided the participant to a tram station approximately one kilometer away. The route was designed to be cognitively challenging for patients with dementia. Aside from the usually crowded tram station, the route contained a four-way crossing of two main streets, as well as narrow streets in a residential neighborhood. At the tram station, the participant was asked to walk back on his/her own. On the way back, the psychologist stayed in the background, observed and protocoled the behavior, and only interacted with the participant in case of unresolvable disorientation or potentially endangering situations. The study contained two potential risks for the participants: increased stress level and risk associated with the surrounding traffic. To address the first risk, the participants were briefed before the study and the route was clearly explained. It was also made clear that the psychologist will be following the participants at all times, so that they will not be left alone. This also addresses the second problem, where to avoid any risks from the traffic, the psychologist stayed near the participants at all times, so that in case of risk she could physically or verbally intervene.
Fig.3.
The guided walk. The person with dementia is guided by a psychologist to a tram station, which is 1 km away. The person is then asked to find the way back to the start point on their own.
Life-space assessment. The guided walk was followed by a long-term assessment over at least 4 weeks, called life-space assessment. Here, daily outdoor mobility activities of 11 out of the initial 15 study participants were recorded. The sensor system for the long-term assessment was only GPS andaccelerometer. In addition, the participant kept a mobility diary. It was developed based on experts’ brainstorming. After the first prototype was developed, it was tested on two of the study participants and optimized before being used in the study itself. We asked the participants to wear a bandage containing GPS and accelerometer and to add an entry to their mobility diary every time they left their home. The diary was filled in twice for each outing, once for the way to their goal, and once for the way back home. Each entry in the diary had a standardized form and contained date, time, destination, stopovers, means of transportation, incidents, and eventually company (e.g., relatives, friends, or a dog).
Interviews with people with dementia
To collect information about behavior patterns, preferences, and social and medical background, a psychologist conducted structured interviews with people with dementia. Furthermore, the participants were asked questions aiming at identifying the affinity of the target user group to technology, and any individual system requirements. The interviews followed a semi-structured form and were conducted with 14 participants (6 of which were women), age from 58 to 86 years old. Cognitive status of participants ranged from mild cognitive impairment to middle stages of Alzheimer’s disease dementia (the average MMSE value of the participants was 24.8 with a minimum of 17 and a maximum of 28). The interviews were digitally recorded, transcribed, and evaluated based on the Mayring method for content analysis [30]. The procedure was performed by the same psychologist that conducted the interviews.
The design of the interviews was based on the technology acceptance model (TAM) [31]. The choice of TAM was based on the assumption that in order to provide assistance to people with dementia, they have to be able to accept such ATD. The interviews consisted of questions divided into the following five categories.
-
•
General questions and social background: these included education, current or former occupation, social status.
-
•
General questions concerning the usage of technology: these included usage of technology at home and work, frequency of usage of technology, problems during usage of technology, and readiness to use technology.
-
•
Questions regarding the need of assistance during outdoor mobility: these included types of outdoor activities, escorts during outdoor activities, frequency of outdoor activities, importance of independence during outdoor mobility, type of transportation mode, radius of the outdoor activities according to the life-space levels; problems during outdoor mobility, support during mobility, changes in outdoor activities based on the disease.
-
•
Questions regarding devices and assistance through navigation: these included the usage of mobile phones, types of functions used from the mobile phone, problems during usage of mobile phones, experience with navigation systems, problems during usage of navigation systems.
-
•
Questions regarding the concept of an ATD for outdoor mobility: these included dimensions and functions of such device.
Conceptualization
Interaction unit analysis
During the conceptualization phase, the identified relevant knowledge was conceptualized through interaction unit analysis (IUA) as proposed by Hoey et al. [32] All relevant information gathered during the domain analysis was categorized. 17 categories were identified during the requirements analysis and literature review (the complete list of categories can be found in Table 1). The list of possible behaviors and their relations to the rest of the elements in the different categories were identified (e.g., the weather influences the type of transportation a person will take). For each type of challenging behavior also the possible causes for observing it were identified [i.e., possible types of errors that occurred (for more details see Table 4, Errors due to disorientation)]. The result of this process was a list of possible behaviors with their challenging counterparts and the possible causes of challenging behavior. An example of the interaction unit analysis can be seen in Table 2.
Table 1.
Results from the literature review with respect to the identified system requirements. Black box indicates the requirement was met in the corresponding work
| Concepts from system | Concepts present in existing ontologies | |||||||||||||||||||
| requirements analysis | Behavior of People | Mobility | Motion | Activity | ||||||||||||||||
| with Dementia | Recognition | |||||||||||||||||||
| [39] | [40] | [41] | [42] | [43] | [44] | [45] | [46] | [47] | [48] | [49] | [50] | [51] | [52] | [53] | [54] | [55] | [16] | [56] | [57] | |
| Interaction with Environment | ||||||||||||||||||||
| Action | + | + | + | + | + | + | – | – | + | – | – | + | + | – | + | + | + | + | + | + |
| Interaction | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | + | + | + | + | + |
| Communication | – | – | – | – | – | – | – | – | + | – | – | – | – | – | – | – | – | – | – | – |
| Extrinsic Factors | ||||||||||||||||||||
| Area | – | – | – | – | – | – | + | + | + | + | + | – | – | – | – | – | – | – | – | – |
| Transportation mode | – | – | – | – | – | – | + | + | – | + | + | – | – | – | – | – | – | – | – | – |
| Location | + | + | – | – | – | – | – | + | + | + | – | – | – | – | – | + | + | + | + | + |
| Local conditions | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| Objects | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | + | + | + | + | + |
| Intrinsic Factors | ||||||||||||||||||||
| Motion | + | – | – | – | – | – | – | – | – | – | – | + | + | + | + | + | + | – | + | – |
| Physiological state | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| Intention / Goal | + | – | – | – | + | – | – | – | – | – | – | – | – | – | – | + | – | + | – | + |
| Long-term Intrinsic Factors | ||||||||||||||||||||
| Routines | + | – | – | – | – | – | – | – | – | – | – | – | – | – | – | + | + | – | – | – |
| Transportation preferences | – | + | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| Social activities | – | + | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| Social background | – | + | – | – | – | – | – | – | – | – | – | – | – | – | – | – | + | – | – | – |
| Medical history | – | + | – | – | – | – | + | – | – | – | – | – | – | – | – | + | – | – | – | – |
| Technology affinity | – | + | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
+ concept included –concept not included.
Table 4.
Identified concepts and subconcepts based on the system requirements, literature review, mobility study, and interviews
| Concept | Example | Source | |||
| Interaction with Environment | Challenging behavior | See Table 3 | Attempting to orient based on signs. | Mobility study, IUA, [58] | |
| Action | See Table 3 | Crossing a street. | Mobility study, IUA, [16, 39, 40, 43, 44, 47, 54–57, 59] | ||
| Interaction / Manipulation | See Table 3 | Touching an object. | Mobility study, IUA, [16, 57] | ||
| Communication | See Table 3 | Speaking to a person. | Mobility study, IUA, [47] | ||
| Extrinsic Factors | Area | Rural | The person is in a field. | [45, 46], Interviews | |
| Urban | The person is in the city. | [45, 46], Interviews | |||
| Mode of Transportation | Non-motorized | Walking | Walking on foot. | [46, 49], Interviews | |
| Cycling | Using bicycle. | [46, 49], Interviews | |||
| Motorized | Public transportation | The person takes a bus. | [45, 46, 49], Interviews | ||
| Private transportation | The person drives a car. | [45, 46, 49], Interviews | |||
| Location | Building | The person is in a public building. | [40, 46, 47] | ||
| Street | The person drives on the street. | [46, 47] | |||
| Off-road | The person walks in the forest. | Mobility study | |||
| Crossing | The person waits at the crossing. | [46, 47] | |||
| Sidewalk | The person walks on the sidewalk. | [46, 47] | |||
| Local conditions | Meteorological state | It is raining. | [2], Interviews | ||
| Public locations state | Opening hours of public buildings. | [2], Interviews | |||
| Transportation state | Working hours of public transportation. | [2, 9], Interviews | |||
| Objects | Animate | Human | Human other than the user. | Mobility study, [57], Interviews | |
| Animal | A dog accompanying the user. | Mobility study, [57], Interviews | |||
| Inanimate | Signs, doors, bags, etc. | [16, 54–57] Interviews | |||
| Intrinsic Factors | Motion | Rotational motion | The person turns around. | [39, 50–56, 59, 60] | |
| Translation motion | The person walks forward. | [50–56, 59, 60] | |||
| Errors due to disorientation | Initiation error | The person does not leave the start position. | [41–44] | ||
| Organization error | The person does not use landmarks to orient in the environment | [41–44] | |||
| Realization error | The person is unable to make an appropriate decision at an intersection. | [41–44] | |||
| Sequence error | The person starts walking before planning the route. | [41–44] | |||
| Judgment error | The person is unable to recognize potential errors and safety issues during task execution. | [41–44] | |||
| Completion error | The person crosses the street without attention to traffic. | [41–44] | |||
| Physiological state | The person’s heart-rate is very high. | [2] | |||
| Intention / Goal | See Table 3 | The person wants to visit a particular place. | [9, 16, 39 43, 55, 57] | ||
| Long -term | Routines | The person goes to the bakery every morning. | [2, 39, 40, 54, 55], Interviews | ||
| Intrinsic Factors | Transportation preferences | The person usually travels with the tram. | [2, 9], Interviews | ||
| (User Profile) | Social activities | The person meets with other elderly people once a week. | [2, 40], Interviews | ||
| Social background | The person worked as an engineer before retiring. | [2, 40], Interviews | |||
| Medical history | The person has a cardio-vascular disease. | [2, 40, 45, 54], Interviews | |||
| Technical affinity | Type 1 users | Technical understanding and interest in technology based on professional experiences. | Interviews | ||
| Type 2 users | Technical understanding and interest in technology based on personal interests and hobbies. | Interviews | |||
| Type 3 users | None or small interest in modern technology. | Interviews | |||
“Concept” contains the concepts and their subconcepts, “Example” gives an example of the concept, and “Source” gives the source from which the concept was identified.
Table 2.
Example output from the IUA for the action walk. It shows the normal action, its challenging counterpart, the possible causes behind the challenging behavior, and an example of this behavior
| Action | Challenging | Cause | Example |
| action | |||
| walk | wander | initiation error | The person cannot remember the destination. The person does not start walking. |
| organization error | The person cannot remember the way to the destination. | ||
| realization error | The person cannot realize that walking in a certain direction is necessary for reaching the destination. The person walks without using landmarks for orientation. | ||
| sequence error | The person knows the destination but cannot perform the necessary steps for reaching it. The person starts walking before he is oriented. | ||
| judgment error | The person knows the destination and the way to it but is unaware of obstacles or dangers on the street. | ||
| completion error | The person does not realize that he has reached the destination. |
Integration of existing models
As no suitable existing models to integrate into the situation model were found, we concentrated on including the relevant knowledge we have found from existing ontologies and models (from Step Review of existing ontologies from the domain analysis).
Implementation
During the implementation phase, the Web Ontology Language (OWL) [33] and OBO Edit (http://oboedit.org/) were used to encode the ontology. OWL was chosen, as it is a semantic web language that can represent rich and complex knowledge about things, groups of things, and the relations between things. Moreover, it is a widely-used language that allows the use of the ontology across different applications. That also allows the easy conversion of the ontology in another targeted language.
Evaluation
To validate the situation model, the OBO Edit tool and its reasoner were used:
-
•
Verification tests on the situation model in OBO Edit were run;
-
•
The OBO Edit reasoner was used to identify redundant links in the ontology.
Furthermore, domain experts were asked to revise the situation model. The experts’ revision of the semantic structure is considered to be genuine evaluation for ontologies [34] and as the situation model is a special case of ontology, we also use the experts’ revision for evaluation. Apart from that, two annotators were asked to annotate the video logs from the guided walk in the mobility study using the same codebook extracted from the situation model. The annotation serves as a ground truth for evaluating the performance of the ATD when recognizing the user behavior. In this case, it was also used to evaluate whether the concepts, identified in the situation model, are recognizable in the video log of the mobility study. This provides information about the objectiveness of the situation model. In order to measure this, the interrater reliability in terms of Cohen’s kappa between the two annotators was calculated. Cohen’s kappa between 0.00–0.20 indicates slight agreements between annotators, between 0.21–0.40 fair agreement, between 0.41–0.60 moderate agreement, between 0.61–0.80 substantial agreement, and between 0.81–1.00 almost perfect agreement[35, 36]. We have to mention that there are no established benchmarks for this methodology. Landis and Koch [35] addresses the annotation of dance movement, while [36] comes from the field of radiology. Regardless, to our knowledge, existing benchmarks from different fields use the same scales to measure the annotation reliability with Cohen’s kappa. As the aim was to evaluate whether the concepts in the situation model can be recognized in the observed behavior by different annotators, a training phase was not conducted with the annotators. This is opposed to other approaches aiming at training the annotators to always code a given behavior with the same label. In the latter case, the name of the label could be irrelevant, as the annotators are already trained to associate the label with the corresponding behavior, regardless of the fact that the label could trigger other behavior associations in untrained annotators.
RESULTS
Model structure and contents
The system requirements evaluation and the review of existing ontologies yielded a knowledge base divided into four main groups describing the factors influencing the behavior of people with dementia during their outdoor mobility. These are “interaction with the environment”, “extrinsic factors”, “intrinsic factors”, and “long-term intrinsic factors”. The results from the requirements evaluation and the literature review can be found in Table 1. More details about the rationale behind the obtained results can be found in Supplementary Material A.
During the outdoor mobility study, four challenging behaviors, associated with disorientation during outdoor mobility were identified. These are “spatial reasoning”, “topological reasoning”, “communication”, and “wandering”. Apart from them, different normal actions were identified. The list of identified behaviors can be found in Table 3. The outdoor mobility study also produced a set of goals the participants were following as well as different transportation modes. These are listed in Table 4 under “Intention / Goal” and “Transportation preferences”. More details about the outdoor mobility study can be found in Supplementary Material B.
Table 3.
Identified interaction concepts during the outdoor mobility study and the IUA as well as their subconcepts
| Challenging behavior | Rational (compensatory behavior) | Communication action |
| Spatial action | ||
| Topological action | ||
| Irrational | Wandering | |
| Action | Walk | – |
| Evade obstacle | – | |
| Cross street | – | |
| Wait | – | |
| Drive a vehicle | – | |
| Take public transportation | – | |
| Enter a building | – | |
| Interaction / Manipulation | Haptic interaction | Buy an object |
| Take an object | ||
| Put an object | ||
| Manipulate an object | ||
| Open object | ||
| Close object | ||
| Touch a person / object | ||
| Visual interaction | Read a sign | |
| Search for landmarks | ||
| Communication | Direct communication | Make eye contact with a person |
| Speak to a person | ||
| Remote communication | Use device to communicate | |
| Goal | Do sport | – |
| Visit the doctor | – | |
| Visit the garden | – | |
| Do shopping | – | |
| Take a walk | – | |
| Visit friends / family | – |
The interviews with people with dementia showed three different types of technology users: Type 1: people that have technical understanding and interest in technology based on their professional experiences; Type 2: people that have technical understanding and interest in technology based on their personal interests and hobbies; Type 3: none or small interest in modern technology. The type of users affects the way in which the ATD can provide assistance. More details about the outdoor mobility study can be found in Supplementary Material B.
The knowledge extracted from the domain analysis and IUA was divided into different sub-categories and the relations between the different elements were identified. This resulted in 101 concepts categorized in a hierarchical structure where the uppermost categories are those identified from the system requirements examination, while the middle and bottom layers are based on the literature review, the interviews, the mobility study, and the interaction unit analysis. The concepts and their hierarchical structure can be seen in Table 4. More details about the definition of each concept can be found in Supplementary Table 1.
Apart from the concepts, 11 relation types were identified, divided into five categories. These are “relations describing the abstraction hierarchy of concepts”, “relations describing the interaction of the user (person) with the environment”, “relations describing locational dependencies”, and “relations describing the causal dependencies between concepts”. The complete list of relations can be seen in Table 5. Some additional details and examples about the relations can be found in Supplementary Material C.
Table 5.
Relations identified between the different concepts (left) and their meanings (right). For more details and examples of the relations, see Supplementary Material C
| Relations describing the abstraction hierarchy of concepts | |
| is a | indicates that the given concept is a sub-concept of another concept |
| consists of | indicates that a given concept is divided into several sub-concepts and that only the collection of all these concepts represents the upper concept |
| has | indicates that a given concept has certain properties |
| Relations describing the interaction of the user (person) with other concepts | |
| uses | indicates that a given concept (i.e., the user) makes use of other concept |
| executes | indicates that a given concept (i.e., the user) can execute another concept |
| is applied on | indicates that one concept can be applied to another |
| Relations describing locational dependencies | |
| is at | indicates that a given concept is located at another concept that describes a type of location |
| is executed at | indicates that a given concept is executed at a given other concept that describes a type of location |
| Relations describing the causal dependencies between concepts | |
| affects | indicates that a given concept is influenced by another concept |
| causes | indicates that the presence of a given concept causes the presence of another concept |
| is achieved through | indicates that a given concept is the effect of another concept |
Figure 4 shows a conceptualization of the situation model with the corresponding relations between the different concepts. The situation model implemented in OWL is publicly available for download at the Library of the University of Rostock [37]. The concepts’ definitions can be found in Supplementary Table 1.
Fig.4.
The identified concepts and the relations between them. Orange indicates the person with dementia, green the interaction the person can execute. The behavior can be normal or challenging (both in dark grey). The rest of the concepts are in light grey, while their subconcepts or concepts identified during the mobility study and interviews are in white. The full list of concepts can be found in the Supplementary Material C and Supplementary Table 1. The implemented situation model can be found in [37] (https://doi.org/10.18453/rosdok_id00000150).
Evaluation
Validation of the situation model
A verification test was run on the situation model. It produced 14 non-critical warnings. All of them addressed misspelling of some concepts and were appropriately corrected.
Furthermore, the reasoner detected 5 redundant relations in the situation model. The redundant relations and the explanation for their redundancy can be found in Table 6. They were also removed from the final situation model. The resulting model can be downloaded from [37]. The list of concepts and their hierarchical structure can be found in Table 4.
Table 6.
Redundant relations in the situation model
| Child name | Parent name | Derived by the reasoner |
| Abnormal behavior | Interaction with environment | from link Abnormal behavior → is_a → Normal behavior. Normal behavior → is_a → Interaction with environment |
| Communication action | Abnormal behavior | from link Communication action → is_a → Rational (compensatory behavior). Rational (compensatory behavior) → is_a → Abnormal behavior |
| Spatial action | Abnormal behavior | from link Spatial action → is_a → Rational (compensatory behavior). Rational (compensatory behavior) → is_a → Abnormal behavior |
| Topological action | Abnormal behavior | from link Topological action → is_a → Rational (compensatory behavior). Rational (compensatory behavior) → is_a → Abnormal behavior |
| Wandering | Abnormal behavior | from link Wandering → is_a → Irrational. Irrational → is_a → Abnormal behavior |
“Child name” and “Parent name” indicate the manually modeled “is-a” relations between the two concepts. “Derived by reasoner” shows the alternative way through which the relations were derived, making the manually modeled ones redundant. The relations were discovered by validating the model with the OBO Edit tool. They were later removed from the final version of the model.
Evaluation by experts
A team of clinician experts in the field (also coauthors in the manuscript) revised the situation model and added certain clinically relevant concepts and relations to the model, increasing its pragmatic usability. The new concepts and relations mainly addressed factors describing the disease and the associated errors in behavior. Table 4 contains the errors in behavior (errors due to disorientation).
Data annotation
The video logs from the guided walk were annotated by two annotators. The annotators concentrated on annotating the normal and challenging behavior of the study participants. The annotation of disorientation showed an interrater reliability of k = 0.61 where k indicates Cohen’s kappa. k of 0.61 indicates a substantial overlapping between the two annotations [35, 36]. We consider the overlapping of the annotation to be satisfactory, as the two annotators were not trained to recognize the observed behavior. This means that they were able to assign the same concept from the situation model to the video log without previously learning the mapping between a given concept and the observed behavior. The difference in the annotation of the two annotators is explained by the fact that the first annotator more often interpreted the behavior of the study participants as compensatory behavior rather than as normal behavior. Interestingly enough, it was observed that the overlapping of the annotation of the directly observed physical behavior was lower than that of the indirectly observed cognitive processes (e.g., topological reasoning). This indicates that the annotators were able to agree on the cognitive processes but had different interpretations of the physical actions.
Nevertheless, the results from the annotation showed that both annotators were able to observe the selected from the situation model concepts in the video log. This stands to show that the selected concepts are suitable for annotating the behavior of the targeted group. Furthermore, it shows that non-trained annotators are able to label the observed behavior with the same labels, which shows that the selected labels are identifiable also for untrainedpersons.
DISCUSSION
The ability to be mobile outdoors allows people with dementia to continue their social life. Therefore, people with dementia can benefit from systems that non-invasively assist their outdoor mobility. To realize such system, however, one needs the knowledge base necessary for recognizing the user’s actions, goals, and needs.
In this work, we proposed a situation model for the outdoor mobility of people with dementia. We first investigated the requirements for the knowledge base based on the user needs and system requirements. We then conducted an ontology analysis based on relevant state of the art ontologies. Furthermore, we analyzed the behavior of people with dementia during their outdoor mobility. We also conducted semi-structured interviews with people with dementia to identify behavior patterns, preferences, and problems during outdoor mobility. We then refined the knowledge base through interaction unit analysis and realized it in the form of OWL ontology. The resulting situation model serves two purposes: 1) to provide the code raster for annotating the outdoor mobility of people with dementia; and 2) to serve as a knowledge base in an ATD aiming to assist people with dementia during their outdoor mobility. To validate the model, two annotators independently used selected concepts from the situation model to annotate the video log collected during a mobility study. The results showed substantial overlapping between the two annotations (Cohen’s kappaof 0.61).
Below we discuss the advantages and limitations of the proposed model as well as the clinicalimplications.
Advantages
The situation model provides the code raster needed for annotating the user behavior. It also provides the semantic meaning behind the annotated behavior, such as reasons for the observed behavior and relation to other annotated behaviors. The situation model is also essential for detecting the user situation and reasoning about the appropriate intervention in case of abnormal behavior. It is the first systematically developed situation model for assistance during outdoor mobility for people with dementia. Existing situation models concentrate on assisting people with dementia in indoorsettings.
Our work is also the first to conduct a mobility study for people with dementia in order to obtain the sensor data and the context information necessary for developing assistive systems for outdoor mobility for people with dementia. It is also specifically tailored to address the needs of people with dementia. For that reason, it contains detailed information about errors in behavior during outdoor mobility and the corresponding causes of these errors.
The annotation of the guided walk also showed that using the code raster from our situation model produces substantial [35] agreement between two different annotators (Cohen’s kappa of 0.61). This stands to show that the proposed model is not only a subjective interpretation but that it also reflects the perception of people who did not take part in the model development. Finally, the situation model incorporates the knowledge from the literature, from personal experiences of people with dementia, and from the requirements needed to develop an ATD for outdoor mobility. This makes it a good starting point for any assistive system that aims at outdoor mobility for people with dementia.
Limitations
Apart from the advantages listed above, there are some limitations associated with the proposed model.
The model does not integrate the knowledge from the long-term assessment and the mobility diary as at the time of development they were not analyzed. We were, however, able to conduct an initial analysis of the long-term assessment study and to integrate our first impressions in the situation model. In the future, we plan to annotate the long-term study with the code raster from the situation model, which will provide additional evaluation of the model’s functionality.
Clinical implications
The situation model for outdoor mobility for people with dementia has two clinical application fields.
1) It tests hypotheses on disoriented behavior in real world settings with sensor based behaviorassessment.
The phenomenology of functional decline in dementia is poorly described. The majority of studies use rating instruments by proxy or self-rating instruments to determine impairments in (instrumental) activities of daily living including way-finding, e.g., the Bayer ADL scale. Such scales are usefully employed to determine the effect on aggregated functional outcomes in clinical trials, but even here they have been criticized for limited sensitivity to functional change and limited validity in respect to underlying behavior features [38]. Another approach is to assess the degree of impairment of cognitive functions that are involved in the maintenance of navigation abilities using psychometric scales. This approach cannot directly assess the effect of cognitive change of everyday function in a given patient, but is based on inference from a stochastic model on the association between cognitive ability and everyday function in groups of patients. In acknowledgment of this fact, major stakeholders, such as the Alzheimer’s Association, the Food and Drug Administration, pharmaceutical industry, and the European Research Council (ERC) have adopted the concept of real world evidence. In 2015, the ERC asked for the “collection and analysis of high quality real world evidence (RWE)”, stating that “many factors prevent the reliable prediction of the natural course of [Alzheimer’s disease] AD in real-world settings... [Including]... lack of clarity on how to best model the natural history of the disease using real world data, particularly as disease stages are difficult to define outside of highly qualified assessment centers or research cohorts.” (http://www.imi.europa.eu/sites/default/files/uploads/documents/IMI2Call6/IMI2_Call6_FINAL.pdf, p. 29). Sensor based automated assessment of disorientation using an adequate situation model can be regarded a major patient-powered real world endpoint to directly and objectively assess the effect of disease progression or interventions on patient function.
2) An adequate model of disorientation in dementia can serve to inform a situation-aware ATD to select an intervention that meets the actual need of a person in a given phase of disorientation. The inferred type of orientation error would drive the selection of the intervention allowing for a user-tailored set of intervention. This in turn provides the user with as little information as necessary to find his/her way back again rather than a generic maximal intervention, such as calling the caregiver every time the user leaves a predefined route, irrespective of potentially underlying decisions or needs of the user.
In the future, we plan to integrate the situation model into a situation detection system. We will evaluate its performance by applying it to the sensor data collected during the mobility study. This will provide us with evidence of whether the system is able to detect cognitive problems and disorientation based on state of the art sensors.
Later, a new study with people with dementia will be conducted where the aim is to test the system’s ability to provide adequate intervention strategies and assistance cues.
Supplementary Material
ACKNOWLEDGMENTS
We acknowledge the efforts of our annotators who spent hours annotating the video log. The SiNDeM Project was funded by the German Federal Ministry of Education and Research (Project number 16SV7091).
Authors’ disclosures available online (http://j-alz.com/manuscript-disclosures/17-0105r2).
SUPPLEMENTARY MATERIAL
The supplementary material is available in the electronic version of this article: http://dx.doi.org/10.3233/jad-170105.
REFERENCES
- [1]. Gill TM, Kurland B (2003) The burden and patterns of disability in activities of daily living among community-living older persons. J Gerontol A Biol Sci Med Sci 58A, M70–M75. [DOI] [PubMed] [Google Scholar]
- [2]. Teipel S, Babiloni C, Hoey J, Kaye J, Kirste T, Burmeister OK (2016) Information and communication technology solutions for outdoor navigation in dementia. Alzheimers Dement 12, 695–707. [DOI] [PubMed] [Google Scholar]
- [3]. Hoey J, Poupart P, von Bertoldi A, Craig T, Boutilier C, Mihailidis A (2010) Automated handwashing assistance for persons with dementia using video and a partially observable markov decision process. Comput Vis Image Underst 114, 503–519. [Google Scholar]
- [4]. Ertel KA, Glymour MM, Berkman LF (2008) Effects of social integration on preserving memory function in a nationally representative US elderly population. Am J Public Health 98, 1215–1220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5]. Hughes TF, Flatt JD, Fu B, Chang CH, Ganguli M (2013) Engagement in social activities and progression from mild to severe cognitive impairment: The myhat study. Int Psychogeriatr 25, 587–595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6]. Kaye J, Matteck N, Hayes T, Austin D, Dodge H (2012) Community life withdrawal with MCI progression. Alzheimers Dement 8, P483. [Google Scholar]
- [7]. Kasper E, Ochmann S, Hoffmann W, Schneider W, Cavedo E, Hampel H, Teipel S (2015) Cognitive rehabilitation in Alzheimer’s disease - a conceptual and methodological review. J Prev Alzheimers Dis 2, 142–152. [DOI] [PubMed] [Google Scholar]
- [8]. Brorsson A (2012) Access to everyday activities in public space: Views of people with dementia (PhD thesis). Karolinska Institutet, Stockholm and Sweden.
- [9]. Patterson DJ, Liao L, Gajos K, Collier M, Livic N, Olson K, Wang S, Fox D, Kautz H (2004) Opportunity knocks: A system to provide cognitive assistance with transportation services In UbiComp 2004: Ubiquitous Computing, Davies N, Mynatt ED, Siio I, eds. Springer, Berlin, Heidelberg, pp. 433–450. [Google Scholar]
- [10]. Liao L, Patterson DJ, Fox D, Kautz H (2007) Learning and inferring transportation routines. Artif Intell 171, 311–331. [Google Scholar]
- [11]. Kirste T (2001) A reference model for situation-aware assistance Workshop Mensch-Computer-Interaktion in allgegenwärtigen Informationssystemen, Mensch und Computer 2001, Bad Honnef. [Google Scholar]
- [12]. Krüger F, Yordanova K, Burghardt C, Kirste T (2012) Towards creating assistive software by employing human behavior models. J Ambient Intell Smart Environ 4, 209–226. [Google Scholar]
- [13]. Yordanova K, Bader S, Heine C, Teipel S, Kirste T (2016) Towards a situation model for assessing challenging behaviour of people with dementia Proc Int Workshop Sensor-based Activity Recognition and Interaction. ACM, Rostock, Germany. [Google Scholar]
- [14]. Yordanova K, Kirste T (2015) A process for systematic development of symbolic models for activity recognition. ACM Trans Interact Intell Syst 5, 20:1–20:35. [Google Scholar]
- [15]. Ye J, Dobson S, McKeever S (2012) Situation identification techniques in pervasive computing: A review. Perv Mob Comp 8, 36–66. [Google Scholar]
- [16]. Ye J, Stevenson G, Dobson S (2012) USMART: An unsupervised semantic mining activity recognition technique. ACM Trans Interact Intell Syst 4, 16:1–16:27. [Google Scholar]
- [17]. Yordanova K, Kirste T (2016) Learning models of human behaviour from textual instructions Proc Int Conf Agents Artificial Intell, Rome, Italy, pp. 415–422. [Google Scholar]
- [18]. Yordanova K (2016) From textual instructions to sensor-based recognition of user behaviour In Comp Pub Int Conf Intelligent User Interfaces. ACM, New York, pp. 67–73. [Google Scholar]
- [19]. Yordanova K (2017) TextToHBM: A generalised approach to learning models of human behaviour for activity recognition from textual instructions. AAAI Proc AAAI Workshops. San Francisco, CA. [Google Scholar]
- [20]. Yordanova K (2017) Automatic generation of situation models for plan recognition problems Proc Recent Advances in Natural Language Processing, Varna, Bulgaria. [Google Scholar]
- [21]. Simperl EPB, Tempich C (2006) Ontology engineering: A reality check In On the Move to Meaningful Internet Systems 2006: CoopIS, DOA, GADA, and ODBASE, Lecture Notes in Computer Science, vol 4275, Meersman R, Tari Z, eds. Springer; Berlin Heidelberg, pp. 836–854. [Google Scholar]
- [22]. Webster J, Watson RT (2002) Analyzing the past to prepare for the future: Writing a literature review. MIS Q 26, xiii–xxiii. [Google Scholar]
- [23]. vom Brocke J, Simons A, Niehaves B, Riemer K, Plattfaut R, Cleven A (2009) Reconstructing the giant: On the importance of rigour in documenting the literature search process. Verona, Italy, pp. Proc European Conf Inform Syst 2206–2217. [Google Scholar]
- [24]. Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W, Goldberg LJ, Eilbeck K, Ireland A, Mungall CJ, Leontis N, Rocca-Serra P, Ruttenberg A, Sansone S, Scheuermann RH, Shah N, Whetzel PL, Lewis S (2007) The obo foundry: Coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol 25, 1251–1255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [25]. The National Center for Biomedical Ontology. NCBO bioportal. https://bioportal.bioontology.org/. Accessed on October 07, 2016.
- [26]. Deuschl G, Maier W (2009) S3-Leitlinie “Demenzen": Kurzversion.
- [27]. Morris JC, Heyman A, Mohs RC, Hughes JP, van Belle G, Fillenbaum GDME, Mellits ED, Clark C (1989) The consortium to establish a registry for Alzheimer’s disease (CERAD): I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology 39, 1159–1165. [DOI] [PubMed] [Google Scholar]
- [28]. Folstein MF, Folstein SE, McHugh PR (1975) “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12, 189–198. [DOI] [PubMed] [Google Scholar]
- [29]. Peel C, Baker PS, Roth DL, Brown CJ, Bodner EV, Allman RM (2005) Assessing mobility in older adults: The UAB Study of Aging Life-Space Assessment. Phys Ther 85, 1008–1019. [PubMed] [Google Scholar]
- [30]. Mayring P (2000) Qualitative content analysis, Forum Qual Soc Res, 1, Art. 20 http://nbn-resolving.de/urn:nbn:de:0114-fqs0002204. [Google Scholar]
- [31]. Venkatesh V, Davis FD (2000) A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manage Sci 46, 186–204. [Google Scholar]
- [32]. Hoey J, Plötz T, Jackson D, Monk A, Pham C, Olivier P (2011) Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive Mob Comput 7, 299–318. [Google Scholar]
- [33]. McGuinness DL, van Harmelen F (2004) OWL web ontology language overview. W3C Recommendation 10, 2004. [Google Scholar]
- [34]. Obrst L, Ceusters W, Mani I, Ray S, Smith B (2007) The evaluation of ontologies In Semantic Web, Baker CJO, Cheung KH, eds. Springer US, Boston, MA, pp. 139–158. [Google Scholar]
- [35]. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33, 159–174. [PubMed] [Google Scholar]
- [36]. Kundel HL, Polansky M (2003) Measurement of observer agreement. Radiology 228, 303–308. [DOI] [PubMed] [Google Scholar]
- [37]. Yordanova K, Koldrack P, Heine C, Henkel R, Martin M, Teipel S, Kirste T (2017) Ontology for the outdoor mobility of people with dementia. University Library, University of Rostock. https://doi.org/10.18453/rosdok_id00000150
- [38]. Sheehan B (2012) Assessment scales in dementia. Ther Adv Neurol Disord 5, 349–358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [39]. Fook VFS, Tay SC, Jayachandran M, Biswas J, Zhang D (2006) An ontology-based context model in monitoring and handling agitation behavior for persons with dementia. PerCom Workshops, IEEE. [Google Scholar]
- [40]. Skillen K, Chen L, Nugent CD, Donnelly MP, Solheim I (2012) A user profile ontology based approach for assisting people with dementia in mobile environments. Eng Med Biolog Soc IEEE, 6390–6393. [DOI] [PubMed] [Google Scholar]
- [41]. Baum C, Edwards DF (1993) Cognitive performance in senile dementia of the Alzheimer’s type: The kitchen task assessment. Am J Occup Ther 47, 431–436. [DOI] [PubMed] [Google Scholar]
- [42]. Bouchard B, Giroux S, Bouzouane A (2007) A keyhole plan recognition model for Alzheimer’s patients: First results. Appl Artif Intell 21, 623–658. [Google Scholar]
- [43]. Serna A, Pigot H, Rialle V (2007) Modeling the progression of Alzheimer’s disease for cognitive assistance in smart homes. User Model User-adapt Interact 17, 415–438. [Google Scholar]
- [44]. Bettcher BM, Giovannetti T, Macmullen L, Libon DJ (2008) Error detection and correction patterns in dementia: A breakdown of error monitoring processes and their neuropsychological correlates. J Neurol Neurosurg Psychiatry 14, 199–208. [DOI] [PubMed] [Google Scholar]
- [45]. Mattson JW, Upper Great Plains Transportation Institute. (2012) Travel Behavior and Mobility of Transportation- disadvantaged Populations: Evidence from the National Household Travel Survey. UGPTI publication; Upper Great Plains Transportation Institute. [Google Scholar]
- [46]. Berdier C (2011) An ontology for urban mobility In Ontologies in Urban Development Projects, Advanced Information and Knowledge Processing, Springer, London, pp. 189–196. [Google Scholar]
- [47]. Schlenoff C, Messina E (2005) A robot ontology for urban search and rescue Proc ACM Workshop KRAS, ACM, New York, pp. 27–34. [Google Scholar]
- [48]. Armand A, Filliat D, Ibanez-Guzman J (2014) Ontology-based context awareness for driving assistance systems. Intelligent Vehicles Symposium, IEEE, pp. 227–233. [Google Scholar]
- [49]. Berdier C, Roussey C (2007) Urban Ontologies: The Towntology Prototype towards case studies In: Ontologies for Urban Development, Teller J, Lee JR, Roussey C, eds. Springer-Verlag, Berlin Heidelberg, pp. 143–155. [Google Scholar]
- [50]. Saad S, De Beul D, Mahmoudi S, Manneback P (2012) An ontology for video human movement representation based on Benesh notation In 2012 International Conference on Multimedia Computing and Systems (ICMCS). IEEE, pp. 77–82. [Google Scholar]
- [51]. El Raheb K, Ioannidis Y (2012) A labanotation based ontology for representing dance movement In Gesture and Sign Language in Human-Computer Interaction and Embodied Communication, Efthimiou E, Kouroupetroglou G, Fotinea SE, eds. Springer, Berlin Heidelberg, pp. 106–117. [Google Scholar]
- [52]. Chung H, Kim JM, Byun YC, Byun SY (2005) Retrieving and exploring ontology-based human motion sequences Comput Sci Applications, Springer, pp. 788–797. [Google Scholar]
- [53]. Bertini M, Cucchiara R, Del Bimbo A, Torniai C (2005) Video annotation with pictorially enriched ontologies Int Conf Multimedia and Expo, IEEE, pp. 1428–1431. [Google Scholar]
- [54]. Zhang S, McCullagh P, Nugent C, Zheng H (2011) An ontology-based context-aware approach for behaviour analysis In: Activity Recognition in Pervasive Intelligent Environments, Chen L, Nugent C, Biswas J, Hoey J, eds. Atlantis Press, pp. 127–148. [Google Scholar]
- [55]. Okeyo G, Chen L, Wang H, Sterritt R (2011) Ontology-based learning framework for activity assistance in an adaptive smart home In Activity Recognition in Pervasive Intelligent Environments, Chen L, Nugent C, Biswas J, Hoey J, eds. Atlantis Press, pp. 237–263. [Google Scholar]
- [56]. Wongpatikaseree K, Ikeda M, Buranarach M, Supnithi T, Lim AO, Tan Y (2012) Activity recognition using context-aware infrastructure ontology in smart home domain In International Conference on Knowledge, Information and Creativity Support Systems, Melbourne, pp. 50–57. [Google Scholar]
- [57]. Krüger F, Nyolt M, Yordanova K, Hein A, Kirste T (2014) Computational state space models for activity and intention recognition. A feasibility study. PLoS One 9, e109381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [58]. Kiefer P, Scheider S, Giannopoulos I, Weiser P (2015) A wayfinding grammar based on reference system transformations In Spatial Information Theory—12th International Conference, COSIT 2015, Fabrikant SI, Raubal M, Bertolotto M, Davies C, Freundschuh SM, Bell S, eds. Santa Fe, NM, pp. 447–467. [Google Scholar]
- [59]. Guest AH (2005) Labanotation: The system of analyzing and recording movement, Psychology Press. [Google Scholar]
- [60]. Benesh R, Benesh J (1977) Reading dance: The birth of choreology, International Specialized Book Services. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.