Perceived Usefulness of a Social Robot Augmented Telehealth Platform by Therapists in the United States

Abstract

With the shortage of rehabilitation clinicians in rural areas and the ongoing COVID-19 pandemic, remote rehabilitation (telerehab) fills an important gap in access to rehabilitation, especially for the treatment of adults and children experiencing upper arm disability due to stroke and cerebral palsy. We propose the use of a socially assistive robot with arms, a torso, and a face to play games with and guide patients, coupled with a telepresence platform, to maintain the patient-clinician interaction, and a computer vision system, to aid in automated objective assessments, as a tool for achieving more effective telerehab. In this paper, we outline the design of such a system, Lil’Flo, and present a uniquely large perceived usefulness evaluation of the Lil’Flo platform with 351 practicing therapists in the United States. We analyzed responses to the question of general interest and 5 questions on Lil’Flo’s perceived usefulness. Therapists believe that Lil’Flo would significantly improve communication, motivation, and compliance during telerehab interactions when compared to traditional telepresence. 27% of therapists reported that they were interested in using Lil’Flo. Therapists interested in using Lil’Flo perceived it as having significantly higher usefulness across all measured dimensions than those who were not interested in using it.

I. INTRODUCTION

THERE is an increasing need for the development and adoption of telehealth methods and systems: the remote application of healthcare using telecommunications, often using two-way video and audio [1]. Telehealth allows patients to receive healthcare remotely, avoiding potential barriers to care such as travel or scheduling, enabling more interactions with clinicians and thus improving patient outcomes. While the need for telehealth is often illustrated in the context of rural communities and other resource-scarce areas, where access to local clinicians is limited, the recent pandemic caused by the infectious COVID-19 virus has highlighted telehealth’s potential in all healthcare environments [2].

Telehealth can be extended to remote rehabilitation (telerehab), where frequent exercise and stretching are important, and regular assessments of function are necessary to ensure the best treatment for patients living with impairments due to disease or injury [3]. Frequent contact with clinicians may increase overall patient adherence and motivation to complete prescribed exercises at home [4]. With the shortage of rehabilitation clinicians in rural areas [5], [6] as well as the ongoing COVID-19 pandemic, telerehab can fill an important gap in access to rehabilitation. Two key use cases are in the treatment of adults and children experiencing upper arm disability due to a stroke or cerebral palsy (CP). Stroke is a leading cause of serious long-term disability with an estimated 7 million cases in the US [7], predicted to increase to approximately 3.88% of the population over 18 years old by 2030 [8]. CP is the leading cause of serious long-term disability in children, with reports suggesting that 3.2 of every 1000 children aged 3–17 in the US have CP [9]. These diseases result in varying levels of motor, sensory, and cognitive impairment due to brain injury, affecting the person’s ability to complete activities of daily living and fully participate in society. These groups often need rehabilitation services at home and in their community, settings beyond traditional inpatient and outpatient care.

We propose the use of a socially assistive robot with a telepresence platform, coupled with a computer vision system, as a tool for achieving more effective telerehab. In this paper, we present the design of such a system, Lil’Flo. The Lil’Flo robotic system is conceived to address limitations of telehealth by creating a physically present social entity to interact with patients. We have developed two generations of this robotic system [10], [11], and in this paper we present our second generation system, Lil’Flo [12]. We first present the motivations of the system and provide a brief overview of the design requirements and methodologies of the Lil’Flo robotic system. We then present results of a virtual usefulness survey with 351 therapists across the United States, representing a cross-section of therapists actively treating patients in a variety of rehabilitation care settings. Therapists were asked to evaluate their perceived usefulness of the Lil’Flo robotic system compared to traditional telepresence for telerehab.

II. Related Works

A. Telepresence Systems

Telepresence systems may contain only a screen, camera, and Internet connection via a cellphone, tablet, or computer, equipment which patients and providers may already possess. Some systems include a mobile robotic base which the operator can control remotely, such as the commercialized systems from Double Robotics and VGo Communications. Others also have robotic appendages to demonstrate the operator’s intent [13]. Some actuate the display screen to face the direction the remote operator is looking [13], [14].

There have been reported successes in telerehabilitation. For example, Dodakian et al. presented a custom-designed tabletop game system attached to a computer for rehabilitation of stroke patients. By prompting the patient to play physical games while monitoring movements over telepresence, patient compliance and motivation was increased [4]. In addition, evidence suggests that patients’ range of motion can be effectively assessed over telepresence, and some patients may in fact prefer telehealth appointments for certain rehabilitation tasks, such as range of motion assessments and wound tracking [15]. Prvu Bettger et al. tested a tele-physical therapy program for therapy post total knee arthroplasty. 143 participants in the telerehab cohort were compared to 144 in a traditional in-person cohort for 12 weeks. The telerehab program had lower costs, lower rates of rehospitalization, and was non-inferior in measures of rehab outcomes when compared to the traditional program [16].

There are reported limitations to using telepresence for routine rehabilitation care. Interactions over telepresence may not be as fulfilling as in-person therapy, which may decrease patient motivation and compliance. If, as a result, patients fail to comply with the instructions during a remote therapy session, the clinician may not see the movements required for a proper assessment of the patient’s current function and progress. Additionally, limitations associated with this technology, including field of view of the operator (clinician), network latency, display screen resolution, and projection of three dimensional interactions into two dimensions, lessen the perception of the presence of the remote operator and reduce spatial reasoning for both users (clinician and patient) [14], [17]. The resulting lack of physical presence, coupled with unclear instructions for movements over telepresence, may decrease patients’ compliance and motivation to perform required motor assessment tasks and, as a result, make each interaction less effective overall. This highlights a need to develop platforms that have a physical presence and can perform both assistive and social functions. With the current pandemic, the need and call for telerehab systems has grown [18], however there are few systems that can meet the need. The use of a humanoid robot in addition to the traditional telepresence platform may improve telerehab care.

B. Socially Assistive Robots (SARs)

SARs [19] combine both assistive robots, which support users with disabilities, and social robots, which are designed to interact and communicate with humans. By interacting socially with users with disabilities, SARs may facilitate more effective communication, leading to greater progress in rehabilitation and motor assessment activities. Mann et al. compared responses to a physical robot with responses to a remote tablet during an interaction and found that subjects engaged more and responded more positively to the physical robot, followed the robot’s instructions better, and found the robot more likeable and trustworthy [20]. Bainbridge et al. showed that having a physical presence for interactions is critical for trust and motivation of the user, especially for tasks that cause discomfort [21]. Additionally, Kiesler et al. showed that subjects interacting physically with a robot are more engaged and comply better with instructions compared to interacting with a virtual robot [22].

Given the benefits shown by robotic presence in a variety of interaction types, several social robots have been developed for upper extremity rehabilitation. For example, the Nao-Therapist project developed several generations of social robots [23], [24] to play games with patients by demonstrating motions and asking patients to mirror them for motor exercise. In a study performed with the Nao-Therapist, it was demonstrated that patients, their caretakers, and clinicians found the system useful and expressed a desire to continue using it for future rehabilitation interactions [24]. Several other Nao-based SARs for rehabilitation have been developed, notably the Zora Robot, a commercialized system shown to improve movement and communication skills in children with severe physical disabilities [25] and the RAC CP Fun platform, designed for young children with cerebral palsy [26]. The RAC CP Fun interacts with users by singing songs, changing positions, and providing verbal feedback while playing games that help improve patients’ motor functions and activities of daily living; the system was found to exhibit high levels of interaction in children with CP [26]. When compared to a video of the robot, the physical system promoted significantly higher engagement and compliance [27].

Other studies have examined the perceived usefulness of SARs with clinicians and therapists. In a survey of 15 clinicians on their perceptions of a Nao-based SAR applied to treadmill-based cardiovascular rehabilitation, 80% of the clinicians found the robot useful [28]. Winkle et al. interviewed 21 therapists about how SARs can be useful in rehabilitation, how engagement can be measured, and how SARs can be tailored to specific patients [29]. The interviewed therapists believed that the presence of an SAR could help motivate patients both in interactions with a therapist and interactions with other technology and that during those interactions the SAR would be able to improve compliance.

III. The Design of Lil’Flo

The Lil’Flo robot system has been through two evolutions. It is composed of two distinct components: a humanoid robot and a mobile base. The humanoid has arms, a face, and a torso. The arms allow the humanoid to gesture and play games with patients, such as Simon Says and hand target touch, helpful for upper extremity rehabilitation. The face allows the robot to express emotion [30]. The base presents the remote clinician and allows the system to be driven remotely. The humanoid is not meant to replace the remote clinician, but instead operates as an independent social entity and rehab buddy for the patient.

The first version of the system was comprised of two individual components: a NAO humanoid robot mounted on a VGo telepresence system. The VGo has an integrated camera, microphones, and video display on a remotely operated mobile platform. The NAO is an easily programmable humanoid. The NAO/VGo combination was expensive, with the NAO costing $9000 and the VGo costing $4000. Even though the NAO is easy to use and program, it lacks flexibility in configuration and has a static face with limited facial expressiveness. Operating the system was challenging: the components did not connect to an integrated interface and were hard to modify and maintain. Our primary takeaways from initial demonstrations and surveys with this prototype were that the clinicians viewed the robot as a social entity but did not find it as useful as we had hoped due to its lack of modularity, difficulty in setup, and high cost [10], [11].

In developing our current design, we leveraged data from surveys with clinicians [11]. Our primary design requirements were to build a social robot that was low-cost, flexible and adaptable to testing, expressive, and easily programmable. Figure 1 shows the current version of the system.

Figure 1.

Lil’Flo: Second generation socially assistive robot with telepresence.

A. Hardware

The custom exoskeleton shell for Lil’Flo is designed to minimize weight and assembly steps, and to allow easy maintenance and experimentation on the robot’s form. To improve performance, the motors are placed as high up in the kinematic chain as possible. The motors are fully encased and pinch points are minimized. The system of motors is controlled via custom software, exposing it to a web dashboard that allows clinicians to adjust the position of the arms easily. The current design supports shoulder flexion/extension, shoulder adduction/abduction, shoulder internal/external rotation, and elbow flexion/extension.

The base of the robot is comprised of three custom sections built on top of a commercially available Kobuki differential drive unit. The first custom section houses the computer, USB hub, battery, and excess wiring. The middle section mounts permanently to the first and holds the humanoid, the screen which displays the remote operator (clinician), and one of the cameras. The top section is screwed to the middle section and holds two cameras, which allow for a better field of view.

Because the face is often the center of attention for human-human interactions, we want the flexibility to alter the facial expression in real-time. We designed a head with a dark translucent face and variable brightness LED matrices behind the surface. By using a dark face, we provide good contrast to the head with clean lines and good transmission of light from the internal LEDs, leading to good facial feature visibility.

B. System Architecture/Software

The entire system runs on an Intel NUC microPC. Color and depth video is captured by two Intel RealSense D415 cameras and color video by a 180 degree fish-eye camera by ELP. The system also includes a speaker, microphone, and touchscreen panel connected to the NUC.

The software stack is built to interface with the Robot Operating System which has become the de-facto standard for robot integration [31]. Our system primarily uses a plays and scripts style of operating under traded control [32]. Full autonomy remains out of reach for reliable and safe human-robot interaction. The human-in-loop system allows a clinician to provide the perception and reasoning that allow a complex application like ours to be feasible.

The robot is easily programmable by clinicians through a single interface. The entire system is designed to be controlled remotely through a custom web interface written in typescript using React with WebRTC for video and audio transmission.

All relevant data is recorded in rosbags on the robot. A long term goal of the project is to collect assessment information to build a pipeline to assess patient state. Leveraging depth information from the RealSense cameras we will compute the reachable workspace, range of motion, and velocity and jerk profiles for a patient’s upper extremities. This type of automated pipeline will not only save time for clinicians but also allow them to continuously and objectively re-evaluate and update treatment for the patient given their current performance in rehabilitation tasks.

IV. Methods

A. Survey Methodology

To evaluate the effects of COVID-19 on patient care, the current use and need for telerehab practice, the feasibility of using Lil’Flo for telerehab, and the sentiments of clinicians towards robots in general, we designed and distributed a survey to the rehabilitation community. The survey takes approximately 15 minutes to complete and was implemented using REDCap [33], [34]. The survey and study was approved by the University of Pennsylvania Institutional Review Board.

1). Survey Distribution and Data Collection:

Distribution was done through multiple avenues: state occupational therapy (OT), physical therapy (PT), and speech and language pathology (SLP) professional societies, Facebook groups, blogs, direct emails to connections, and direct emails to OTs and PTs in Ohio (where the state licensing board provides member emails). Subjects who opted in and provided their email address were entered into a drawing for a $20 gift card, given to one in every 20 respondents.

We collected a convenience sample of 423 complete rehabilitation clinician’s responses between June 30th, 2020 and September 15th, 2020. The vast majority of responses came from therapists in the United States of America. For the purpose of this analysis, we analyzed only responses from therapists in the United States with one or more years of experience who had no prior knowledge of Lil’Flo (n=351).

2). Survey Questions:

The survey asked a number of questions about demographics and the typical patient population treated by respondents, followed by questions to determine the prior experience of the clinician with performing telerehabilitation. Subjects were then asked to watch a short video overview of Lil’Flo (https://youtu.be/OHybatsjzog), which depicted the system being driven by a remote operator and the humanoid robot introducing itself to the viewer and demonstrating its capabilities including talking, moving the two upper limbs, and instructing the viewer in an exercise. After watching the video overview of Lil’Flo and answering questions on their prior knowledge of Lil’Flo, subjects were asked about their general interest in the system: “How interested would you be in using the Lil’Flo system?” and responded by placing a continuous slider on a scale of 0–100 with 0 labeled as “Not At All Interested” and 100 labeled as “Very Interested”. To better understand perceived usefulness of Lil’Flo, subjects were also asked: “How do you believe that adding a social robot as a companion for your patients during video+audio telepresence interactions (such as the LIl’Flo system) would change the following when compared with traditional video+audio telepresence based rehab?” They were presented with 5 categories to rate along a slider scale (table I). For all of the sliders, the text range labels were displayed in the appropriate positions, but numerical values were not displayed. The default value for the sliders was 50. Subjects were required to click in the slider area to continue, but not required to change the value from 50. The complete survey can be found in the attached multimedia materials.

Table I.

Questions Comparing Lil’Flo to Traditional Video and Audio Telerehab

	Scale values (0–100)
	0	50	100
COMMUNICATION during the interaction	Decrease communication	No change	Help communication
Patient MOTIVATION during the interaction	Decrease motivation	No change	Increase motivation
Your ability to ASSESS your patients from telepresence interactions	Impair assessment	Same	Improve assessment
How well your patients COMPLY with instructions DURING the telepresence interaction	Reduce compliance	Same	Improve compliance
How well your patients ADHERE to the treatment plan AFTER a telepresence interaction	Reduce adherence	Same	Improve adherence

B. Hypotheses

We hypothesized that clinicians would perceive that Lil’Flo would increase communication (H1a), patient motivation (H1b), patient compliance (H1c), patient adherence (H1d), and the ability of clinicians to assess patients (H1e). From prior experience, we expected that therapists’ perceptions of the usefulness of Lil’Flo would be affected by years of experience (H2a) and type of therapist (OT, PT, SLP) (H2b). We also hypothesized that prior experience using telerehab would affect perceptions of the system (H2c). We also explored whether therapists who expressed interest in using the system had different responses than those who did not, expecting that therapists who rated their interest as less than 50/100 would perceive different levels of usefulness than those who rated their interest as greater than 50/100 (H2d).

C. Statistical Analysis

All statistical analysis was performed in R [35] and plots generated with ggplot2 [36]. The complete analysis can be found in the attached multimedia materials. The data were first cleaned by coding responses by therapist type: Occupational Therapist, Physical Therapist, Speech and Language Pathologist, and other using responses to an open ended question on therapist type. One data record which was marked complete was removed due to missing data.

We evaluated therapists’ interest in using Lil’Flo using descriptive statistics.

We used a one-sample Wilcoxon signed rank test with alpha set at 0.05, for each hypothesis H1a-e to evaluate whether Lil’Flo is perceived to improve communication, motivation, assessment, compliance, adherence, and ability to assess patients when compared to video and audio alone for telerehab. The null hypothesis was that the median was less than or equal to 50, with the alternative that it was greater than 50. In addition, how many respondents changed from the default value of 50 was calculated, as a proxy for level of opinion/interest in the respective question.

To test H2a-c, whether a variety of demographic features affect therapists’ perceived usefulness of Lil’Flo, responses were grouped by years of experience practicing therapy, therapist type, and prior telepresence experience. Years of experience practicing was defined by breaking therapists into three categories: 1–4 years (n=96), 5–15 years (n=105) and 16+ years of experience (n=150). Therapist type was defined as physical therapists (n=131), occupational therapists (n=165), and speech an language pathologists (n=41). Prior telepresence experience, was defined as three categories: experience with video based telepresence pre-COVID-19 (n=50), experience with video based telepresence during COVID-19 but not before (n=106), and no prior video based telepresence experience (n=195). The responses in each group were compared, for each question, to determine if any factor had an effect on perceived usefulness. When possible ANOVA was used for the comparisons. When the assumptions for ANOVA were not met, a Kruskal-Wallis rank sum test was used. Given the group size, ANOVA is somewhat robust against non-normality. However, it is sensitive to platykurtosis in the group distributions and differences between the variance of the groups. For groups where the variances at each level were not significantly different (p-value on Fligner-Killeen test not less than 0.05) and for which the kurtosis at each level was between 1 and 5, an ANOVA was used.

To test H2d, whether being interested or not interested in using Lil’Flo changed therapists’ perceived usefulness of Lil’Flo, responses to the usefulness questions were separated into a group for therapists who responded to the interest question with less than 50/100 (n=224) and a group which responded greater than 50/100 (n=95). A Wilcoxon rank-sum test was used to compare responses from the two groups.

P-values for each question, treating the factors as multiple comparisons (four comparisons per question), were adjusted using the Benjamini & Hochberg method.

V. Results

A. Sample Demographics

Respondents average 40.4 ± 12.5 (μ ± sd) years old with 14.8 ± 11.9 years of experience. There were 165 occupational therapists, 131 physical therapists, 41 speech language pathologists and 14 other types of therapists. There were 317 women and 34 men. The respondents worked in a wide variety of clinical settings (table II). 149 of the respondents worked in an urban setting, 199 in a suburban setting, and 65 in a rural setting (subjects could select multiple classes of locations). We collected responses from 26 states. The largest number came from Ohio, with 186 responses. Illinois had 47 responses, Pennsylvania 30, New York 16, and South Carolina 13. All other states had fewer than 10 responses.

Table II.

Clinical Settings Respondents Work In (Multiple Choice)

Location Types	n		Location Types	n
Community Center	1		Outpatient Facility	87
Elder Care Home	45		Patient Home	34
Elder Care Hospital	8		Private Practice	35
General Hospital	78		Rehab Center	68
Hospital for Children	8		School	51
Inpatient Facility	48		Other	27

B. Interest in Using Lil’Flo

The level of interest in using Lil’Flo among therapists is shown in fig. 2. The mean of the responses is 34.5 with standard deviation 25.7. The median is 30, first quartile 13, and third quartile 55. 9.7% of the respondents indicated 0 (not at all interested in using Lil’Flo) while 9.1% of the respondents indicated 50, the default value. 63.8% of respondents rated their interest as less than 50 (including 0) and 27.1% rated their interest as greater than 50.

Figure 2.

Responses to: “How interested would you be in using the Lil’Flo system?” on a continuous scale from 0 (Not At All Interested) to 100 (Very Interested), as a boxplot with mean at the diamond, violin plot, and dot plot.

C. H1:Perceived Usefulness of Lil’Flo

The answers to the five usefulness questions can be seen in fig. 3 with summary statistics shown in table III. Table III also shows the count of therapists who were positive (> 50), neutral (= 50) or negative (< 50) in their responses to the usefulness of the system for each category measured. on visual inspection, the data is non-normal, especially so with the peak at 50, which was confirmed visually on quantile-quantile plots (not shown).

Figure 3.

Responses to the questions comparing the perceived usefulness of Lil’Flo to traditional video based telepresence. On the left for each question, a boxplot showing the inter quartile range (IQR) with whiskers extending to the most extreme value within 1.5 IQR. Outliers beyond 1.5 IQR shown as dots. The mean is shown as a diamond. On the right for each question, violin plots [37] showing the estimated kernel density with overlaid dot plots showing actual responses. Distributions with median significantly above 50 (by Wilcoxon signed rank test) are annotated with a star on their label.

Table III.

Summary of responses to Lil’Flo usefulness questions by all subjects, subjects who rated interest in using Lil’Flo < 50, and > 50. Questions with median significantly above 50 (Wilcoxon signed rank test) indicated with star.

	All								interest < 50			interest > 50
	mean	sd	Q1	med	Q3	n(< 50)	n(= 50)	n(> 50)	mean	sd	med	mean	sd	med
Communication *	53.3	22.3	40.5	56	69.0	102	62	187	47.5	23.0	50.0	65.8	16.1	67
Motivation *	58.3	21.3	50.0	63	71.0	68	52	231	53.1	22.4	58.5	68.9	14.7	69
Assessment	50.2	21.9	39.0	50	65.0	103	98	150	44.5	22.7	50.0	61.8	16.2	61
Compliance *	53.5	20.6	50.0	56	67.0	81	84	186	48.0	21.4	50.0	65.7	14.1	65
Adherence	48.8	17.7	50.0	50	57.0	86	158	107	44.9	18.5	50.0	57.2	14.6	53

The median responses for communication (H1a, median = 56, p = 1.0 × 10⁻³), motivation (H1b, median = 63, p = 1.1 × 10⁻¹²), and compliance (H1c, median = 56, p = 2.3 × 10⁻⁴) were significantly above 50 (Lil’Flo perceived as improving those categories) as determined by Wilcoxon signed rank tests. The median responses for adherence (H1d, median = 50, p = 0.7) and assessment (H1e, median = 50, p = 0.289) were not found to be significantly above 50 as determined by Wilcoxon signed rank tests.

D. H2: Effect of Demographic Factors on Perceived Usefulness

To determine if demographic factors had a significant effect on perceived usefulness, Kruskal-Wallis rank sum tests were used for prior telepresence experience on the motivation and compliance questions and for all comparisons on the adherence question due to the assumptions for ANOVA being violated for those groups; ANOVA were used for all other comparisons in H2a-c. Wilcoxon rank sum tests were used to evaluate the effect of interest in using LIl’Flo on perceived usefulness. The resulting p-values can be seen in table IV. None of the three independent demographic variables — years of experience (H2a), therapist type (H2b), or prior telepresence experience (H2c) — showed significant results. A significant difference on all perceived usefulness measures was found between therapists rating interest in using Lil’Flo greater than 50 and those rating interest less than 50 (H2d).

Table IV.

P-values for Effects Analysis on Questions (Adjusted for Multiple Comparisons)

	Communication	Motivation	Assessment	Compliance	Adherence
Years of Experience	0.61	0.32	0.093	0.11	0.72
PT vs OT vs SLP	0.19	0.91	0.35	0.36	0.96
Prior Telepresence Experience	0.68	0.71	0.73	0.88	0.96
Level of Interest	5.9 × 10−11	8.7 × 10−9	1.3 × 10−9	4.7 × 10−12	6.7 × 10−8

VI. Discussion

The US Bureau of Labor Statistics reports that in the USA there are 136K occupational therapists (88% women), 304K physical therapists (67.9% women) and 180K speech and language pathologists (95.8% women) [38]. We successfully conducted a large study to evaluate the perceived usefulness of Lil’Flo, compared with traditional telepresence, with 351 therapists in the USA of which 47% were occupational therapists, 37% were physical therapists, 12% speech language pathologists, and 90% were women. This study represents one of the largest queries regarding a socially assistive robot for rehabilitation conducted in the United States.

The hypothesis underlying the design of Lil’Flo is that adding a social robot which can gesture, speak, play games, and in general interact with patients during telerehab interactions will improve the quality of those interactions. We have focused on improving 5 metrics: communication, motivation, assessment, compliance, and adherence. We surveyed therapists from across the United States with a variety of experiences and backgrounds to understand what usefulness they believe Lil’Flo would provide to telerehab interactions. We found that overall therapists perceived that Lil’Flo would help communication with their patients, increase patient motivation, and improve patient compliance during interactions. The results did not show that therapists expected Lil’Flo to improve patient assessments or improve patient adherence after interactions. Results do not appear to deviate across therapists with different disciplines, years of experience, or levels of telepresence experience. Therapists who are interested in using Lil’Flo perceived its usefulness significantly higher across all measured usefulness metrics.

Our results are in agreement with results by Winkle et al. who also found that therapists believed a SAR could help motivate patients and drive compliance during rehabilitation interactions [29]. The perceptions of therapists in our study also align with the results from Fridin et al. showing a physically present robot motivated patients and drove higher compliance when compared to a virtually present one [27]. Our results further agree with a number of studies which have measured patient engagement with SARs for rehab like interactions and shown positive results, including usefulness [28] and motivation [24]-[26].

From our prior experience with Lil’Flo and from the literature, we had expected to see all usefulness questions show values greater than 50 (Lil’Flo improving each metric over traditional video and audio based telerehab). Confirmation of our hypotheses for three of the five questions is a positive outcome. Although we had hoped to see therapists viewing a social robot as being more useful for assessment and at driving adherence with rehab plans than traditional telepresence, our failure to do so is instructive. First, it may indicate a knowledge gap in therapists’ understanding of the power of AI or the power of the robot companion, or a failure on our part to adequately design to those needs. Second, assessments in rehabilitation drive reimbursement, and therapists may be hesitant to “try” new systems especially if the system’s accuracy is not clear. Third, it may highlight the limitations of what can be demonstrated over video, especially with regard to the perception system designed to assess upper limb function. Therapists’ skepticism in the areas of adherence and assessment highlights a need to focus future study on providing evidence of social robots with telepresence impacting assessments and adherence.

A large number of the therapists expressed interest in using Lil’Flo — 95 respondents rated their interest as greater than 50. An even greater number, 258, rated their interest as 50 or less. Given that none of the respondents had prior knowledge of the system and they were only exposed to it through a short video, we view this outcome as being positive. Unsurprisingly, therapists who are interested in using the system viewed it as having greater usefulness than those who are not.

A. Limitations of the Study

Our sampling method was not random and there may be bias in the sample. For example, the sample likely skews towards therapists who are active on social media or read emails from their state professional society, which could represent only a subset of therapists. For the state of Ohio, we sent emails to the entire population of physical and occupational therapists, leading to a broader sample from that state. From our demographic questions, we know that we have been able to survey therapists with a wide range of experience levels, ages, clinical settings and locations, and patient types. Because of our large sample size and diversity of respondents, we believe that the results we present here are strong enough to significantly contribute to the space.

There were a large number of responses at the 50 level on all questions. It is likely that those responses represent some combination of respondents feeling that the Lil’Flo system would be equivalent to traditional telepresence and some who were simply picking the default as a non-response. A survey design that prevented a default response may have provided stronger results. The variability in responses at 50 across the five questions may provide more insight as to the intent of respondents. The motivation question had far fewer responses at 50 than the adherence question (fig. 3), perhaps suggesting that the adherence question was not as well understood.

The use of the video may have limited our ability to communicate certain design features. For example, one assumption of our design is that a social robot will be able to motivate and instruct a patient during a remote assessment and that this will improve the assessment quality. In addition, we propose the use of machine learning algorithms to enable extraction of patient kinematics to provide objective assessment of movement. Therapists were not convinced that Lil’Flo would improve their ability to do assessments with many feeling Lil’Flo would be worse at the measured metrics. These responses may be attributed to a failure of the video to communicate this design feature well.

Some of the groups in H2a-c were unbalanced, which lowers the statistical power when compared to equal sample sizes. This provides a source of concern for the validity of tests which resulted in marginal p-values (table IV).

VII. Future Work

Lil’Flo will be tested for usability in the clinic, and clinicians operating the system will be surveyed to determine if the system improves telerehab interactions and further clarify perceived usefulness by clinicians. There are plans to further automate the system to allow clinicians to focus on the aspects of interactions they find most important.

VIII. Conclusions

The data presented in this paper suggest that therapists believe that Lil’Flo would benefit care for their patients and provides motivation for further exploration of the system and others like it. There are many other technologies which could potentially improve telerehab interactions using virtual reality, augmented reality, etc. However, previous studies [20]-[22] have shown the importance of physical presence, which a social robot can provide. This study supports those findings, showing that therapists believe an SAR would be useful for augmenting telepresence for rehab. Some systems have taken the approach of full autonomy with a social robot. We believe full autonomy has a powerful place to fulfil in helping increase frequency of care. However, by bringing expert clinicians along with the robot, remotely, the best of both worlds can be achieved, human expertise, compassion, and empathy, along with the social robot to demonstrate and motivate.