Accuracy of Wearable Cameras to Track Social Interactions in Stroke Survivors

Abstract

Background

Social isolation after a stroke is related to poor outcomes. However, a full study of social networks on stroke outcomes is limited by the current metrics available. Typical measures of social networks rely on self-report, which is vulnerable to response bias and measurement error. We aimed to test the accuracy of an objective measure—wearable cameras—to capture face-to-face social interactions in stroke survivors. If accurate and usable in real-world settings, this technology would allow improved examination of social factors on stroke outcomes.

Methods

In this prospective study, 10 stroke survivors each wore 2 wearable cameras: Autographer (OMG Life Limited, Oxford, United Kingdom) and Narrative Clip (Narrative, Linköping, Sweden). Each camera automatically took a picture every 20–30 seconds. Patients mingled with healthy controls for 5 minutes of 1-on-1 interactions followed by 5 minutes of no interaction for 2 hours. After the event, 2 blinded judges assessed whether photograph sequences identified interactions or noninteractions. Diagnostic accuracy statistics were calculated.

Results

A total of 8776 photographs were taken and adjudicated. In distinguishing interactions, the Autographer’s sensitivity was 1.00 and specificity was .98. The Narrative Clip’s sensitivity was .58 and specificity was 1.00. The receiver operating characteristic curves of the 2 devices were statistically different (Z = 8.26, P < .001).

Conclusions

Wearable cameras can accurately detect social interactions of stroke survivors. Likely because of its large field of view, the Autographer was more sensitive than the Narrative Clip for this purpose.

Introduction

Social networks impact health outcomes at the same level as traditional risk factors.¹ In stroke survivors, social isolation is associated with poor outcomes.² Therefore, stroke rehabilitation, unlike acute interventions, requires attention and engagement with social-behavioral patterns to optimize success.³ However, understanding and leveraging social networks to aid recovery requires accurate and precise metrics to inform interventions.

Typical measures of social networks rely on self-report, which may be valid for core network characterization, but is vulnerable to response bias and measurement error for more expansive mapping.⁴ Social sensors (e.g., mobile phones, proximity sensors) offer more objective and thorough measurement of network typology and dynamics.⁵ Wearable cameras have been used to measure lifestyle behaviors such as eating and sedentary behavior.⁶ We aimed to quantify the accuracy of wearable cameras as a social sensor in stroke survivors.

Methods

This prospective study of stroke survivors quantified the accuracy of wearable cameras to detect social interactions. Ten stroke survivors and 10 healthy adults participated. Stroke survivors had a history of ischemic stroke and mild to moderate deficits (National Institutes of Health Stroke Scale < 21 and language sub-score < 1). The institutional review board of the Washington University in St. Louis approved the study. All participants signed an informed consent to participate in the study.

Each stroke survivor wore 2 life-logging cameras: Autographer (OMG Limited, Oxford, United Kingdom) and Narrative Clip (Narrative, Linköping, Sweden). Each camera took a picture every 20–30 seconds. Stroke survivors mingled with healthy controls for 5 minutes of 1-on-1 interactions alternating with 5 minutes of no interactions for 2 hours total. In this alternating design, patients with stroke could interact with different individuals and rest in between conversations. During noninteraction periods, stroke survivors watched television or had non-engaging individuals walking in front of them. Twelve interactions and 12 noninteractions per participant served as the reference standard. Healthy controls did not wear cameras.

Statistical Analysis

After the staged event, 2 blinded judges were trained to identify interactions and noninteractions in practice photographs based on prespecified rules (e.g., a person faces camera, close, and present in at least 3 consecutive pictures). Each judge assessed photographs from 1 device for all participants. Then, they assessed 20% of photographs from the opposite device for inter-rater reliability. We estimated the overall diagnostic accuracy, defined as the proportion of correctly classified cases among all cases.⁷ We also calculated DeLong’s test for 2 correlated receiver operating characteristic curves and Cohen’s kappa for 2 raters.

Results

The cameras recorded 8776 photographs during 1 staged event in September 2015. Stroke survivors were 60% women, had a median (SD) age of 68 (13.3), and were 50% black and 50% white. The stroke location was 50% right hemisphere and 50% left hemisphere, National Institutes of Health Stroke Scale median (SD) was 1 (1.2), and Modified Rankin Scale score median (SD) was 2 (.7).

Figure 1 shows photographs from the 2 devices of a stroke survivor’s interaction. The field of view was 136 degrees for the Autographer and 70 degrees for the Narrative Clip. Figure 2 shows the diagnostic accuracy and confidence intervals for the devices. The Autographer’s sensitivity was 1.00 and specificity was .98. The Narrative Clip’s sensitivity was .58 and specificity was 1.00. The receiver operative characteristic curves of the 2 devices were statistically different (Z = 8.26, P < .001). The judges had strong inter-rater reliability (kappa = .98, P < .001).

Figure 1.

Photographs of the same interaction taken by 2 wearable cameras.

Figure 2.

Overall diagnostic accuracy, defined as the proportion of true positives and true negatives among all cases, for the 2 devices.

Discussion

Wearable cameras can reliably detect social interactions of stroke survivors. Likely because of its large field of view, the Autographer more accurately distinguished human interactions from noninteraction scenarios than did the Narrative Clip.

This study makes at least 2 contributions to the literature. It is the first to our knowledge to test and compare the Narrative Clip and the Autographer as social sensors. The study is also 1 of the first to be conducted in a clinical population, albeit with mild deficits. Typically, social sensing is performed using smartphones in healthy volunteers.⁸ There are a number of advantages for using wearable cameras for public health assessment. Comparable with, and potentially more accurate than, accelerometers, wearable cameras allow real-time and accurate measurement of sedentary behavior, active travel, and eating.^9,10 The unobtrusive and passive recording requires little attention or intervention from the subjects or researchers to obtain objective information. Intriguingly, the images may also be used as a memory prosthetic to recall daily events for patients with memory disorders.¹¹ Methodologically, these devices could complement traditional surveys or diary interventions to overcome recall and social-desirability biases.⁶ Conceivably, wearable cameras, accelerometers, and smart phones may be part of a multimodal passive sensing package to monitor and encourage lifestyle and behavior changes for patients with stroke.

Challenges to adoption of wearable cameras include ethical and privacy concerns. Inappropriate or unwanted images, confidentiality, and capture of unconsented third parties are issues that face researchers and interested participants. An ethical framework providing best practice guidelines and language for regulatory bodies is available.¹² Greater use of such devices in public and integration with smart phones may alleviate some concerns. Other issues include the time-consuming analysis for researchers,⁶ which may be improved with machine-learning face-detection algorithms.¹³ The current study was limited by being conducted in a controlled setting with a small number of stroke survivors with mild to moderate deficits.

Conclusion

This study supports the use of wearable cameras with a wide field of view to study and encourage social interactions of stroke survivors. The technology offers an accurate and objective means to track social life, which is important for stroke recovery.³ Further studies in other contexts could assess the use of cameras in everyday life with attention to privacy and ethical concerns. Similar to other wearable devices, body cameras have the potential to catalyze health behavior change, particularly in the social domain.