The effect of robot interventions on sleep in adults: a systematic review and network meta-analysis

Abstract

Study Objectives:

Robotic pets or companion robots have demonstrated positive effects on several emotional and physiological factors in humans. Robots could constitute a complementary or alternative method to treat sleep problems, but individual studies on robots’ effectiveness regarding sleep show mixed results. The aim of the current study was to compare the effects of robots, plush toys, and treatment as usual on sleep in adults.

Methods:

The current study is a systematic review and frequentist network meta-analysis of all randomized and cluster randomized controlled trials comparing the effects of robots, plush toys, and treatment as usual on total sleep time in adults.

Results:

Four studies were included in the analysis. Three studies were considered to have a high risk of bias, whereas one was rated with some concerns. The studies comprised 381 participants. These participants were older adults, with or without dementia, living in nursing homes. The total sleep time was the only common sleep measure included in all 4 studies. The network meta-analysis showed no statistically significant differences between the 3 experimental groups.

Conclusions:

The robot interventions were not found to have positive effects on total sleep time in older adults compared with plush toys or treatment as usual. Future studies should use robots especially made to target sleep, include a thorough screening of the participants, and exclude people with adequate sleep, select appropriate sleep measures, and report the results appropriately for future meta-analyses.

Citation:

Støre SJ, Beckman L, Jakobsson N. The effect of robot interventions on sleep in adults: a systematic review and network meta-analysis. J Clin Sleep Med. 2022;18(7):1877–1884.

BRIEF SUMMARY

Current Knowledge/Study Rationale: Companion robots have shown to have positive effects on several emotional and physiological factors, especially in older adults. Robots represent a potential line of complementary or alternative treatments for troubled sleep in adults, but the few conducted randomized controlled trials show mixed results. There are no previous meta-analyses of robots’ effects on nighttime sleep.

Study Impact: Taken together, the randomized and cluster randomized controlled trials conducted up until today studying the effects of robots on total sleep time in adults, compared with plush toys and treatment as usual, are flawed and do not comprise any statistically significant differences between the groups. The insufficient empirical evidence implies that robot interventions should not be recommended for the time being.

INTRODUCTION

Sleep is essential to human health. Among many things, it supports a person’s cognitive capacity, helps regulate emotions, and is intertwined with the body’s metabolism and immune system.¹ Everyone sleeps poorly from time to time. When the sleep problem occurs frequently and causes significant impairment, the problem may meet the criteria of a sleep disorder. Insomnia is one of the most common sleep disorders in adults.² The first-line treatments for insomnia are cognitive behavioral therapy and medications.³ The first-line treatments of sleep disorders are not always effective or appropriate for all. Complementary and alternative medicine methods are attractive to people with sleep problems,^3,4 but are often not recommended due to insufficient empirical evidence,³ hence the need for more research on credible complementary and alternative medicine.

The recommended sleep duration for young adults (18–25 years) and for adults (26–64 years) is 7 to 9 hours. However, less than 6 and more than 11 or 10 hours, respectively, are not recommended; everything in between may be appropriate. For older adults (aged 65+ years), the recommended sleep duration is 7 to 8 hours, but everything between 5 and 9 hours may be appropriate, whereas less than 5 hours or more than 9 hours are not recommended.⁵ Self-reported sleep problems decline with age,⁶ in contrast to objective sleep problems that generally increase with age.⁷ Sleep problems in older adults are often associated with medical conditions, polypharmacy use, lifestyle changes (eg, retirement, transition to nursing home), and age itself.⁸ This may lead to the underdiagnosis and insufficient treatment of concurrent clinical sleep problems in this age group.

Several robots have been produced in recent years to assist humans. Robots can be divided into 2 main types: industrial and service robots.^9,10 The latter term refers to robots that can serve humans in useful ways and can be further divided into robots for either professional or personal use.¹⁰ So-called social robots can resemble humans or animals. These robots are social in that their functions enable certain communication and interaction with humans.^11,12 Robotic pets or companion robots, alternatively therapeutic robots, social commitment robots, and social assistive robots,¹³ mimic animal-assisted therapies, which, for instance, have been shown to have positive effects on emotional and physiological factors in older adults in nursing homes.¹⁴ Many companion robots are in a familiar animal shape, such as a dog or a cat, but the oldest and most studied robotic pet—the PARO (Personal Assistance RobOt)—resembles a baby harp seal.^12,15

The PARO can see, hear, keep its balance, and feel touch, that is, use 4 senses.¹⁶ It can move its head, flippers, and tail; blink its eyes; and make authentic seal sounds.¹⁶ In a systematic review and meta-analysis of all randomized controlled trials on the effectiveness of social robots for older adults, PARO was used in 8 out of 11 studies.¹² The control groups in the included studies consisted of interventions with a plush toy like PARO,¹⁷ with live dog visits,^18,19 with reading activities,²⁰ and, most commonly, treatment as usual.¹² Pu et al’s¹² review focused on 1) agitation, neuropsychiatric symptoms, and anxiety, 2) depressive symptoms and apathy, 3) cognitive level, and 4) quality of life, that is, no sleep outcome measures. Another systematic review and meta-analysis by Saragih et al²¹ studied all randomized and cluster randomized controlled trials of robotic interventions in people with dementia. Their meta-analysis included separate analyses for 1) agitation, 2) anxiety, 3) cognitive function, 4) depression, 5) neuropsychiatric symptoms, 6) total hours of sleep during daytime, and 7) quality of life. Regarding total hours of sleep during daytime, 2 studies were reportedly included in the analysis, according to the results section.^22,23 However, the forest plot included both daytime and nighttime sleep measures from Moyle et al.,²³ while the title of the figure still referred to daytime sleep only.²¹ The authors concluded that there was a significant difference in total hours asleep during the daytime when comparing the robot and control groups. There are no previous meta-analyses on robots’ effects on nighttime sleep.

The aim of the current study was to assess the effects of robots on total sleep time in adults, compared with plush toys or no intervention, by combining the results from all relevant randomized and cluster randomized controlled trials in a network meta-analysis. The research question was whether robots have a larger positive effect on total sleep time at nighttime in adults compared with plush toys and treatment as usual.

METHODS

The current study is a systematic review and network meta-analysis of randomized and cluster randomized controlled trials comparing the effects of robots, plush toys, and treatment as usual on total sleep time in adults. The study has been registered in the International Prospective Register of Systematic Reviews (PROSPERO).

Search strategy

The electronic databases PubMed, PsychINFO, Scopus, and Web of Science were searched for relevant studies. The International Standard Randomized Controlled Trial Number (ISRCTN) registry, the International Clinical Trials Registry Platform (ICTRP), and ClinicalTrials.gov were also searched for ongoing trials. The keywords used in all searches were (“robot” OR “plush toy” OR “soft toy” OR “cuddly toy” OR “stuffed animal” OR “teddy bear”) AND (“sleep”) AND (“randomized controlled trial”). We then sought to augment the initial list of studies in 2 ways. For each study in the initial list, all studies they reference were examined. Additionally, Google Scholar was used to identify other studies that reference studies in our initial list. The search was conducted on October 15, 2021.

Criteria for inclusion

The Population, Intervention, Comparison, Outcome and Study Design (PICOS) method was used to determine the criteria of eligibility.²⁴ The criteria for inclusion were 1) adult participants (18+), 2) a comparison of a robot and/or a plush toy intervention, and/or a treatment as usual control group, 3) an outcome of nighttime sleep, 4) a randomized or cluster randomized controlled trial design, and 5) publication in an English language peer-review journal. The rationale for including plush toy interventions in the analysis was partly to enable the inclusion of more studies than what was included in a previous meta-analysis of robots’ effects on sleep²¹ (ie, studies that include comparisons of robots and plush toys, without a treatment as usual control group) and partly to actually compare the effects of robots and plush toy interventions, to assess whether the latter is sensible to use as a placebo condition in future randomized controlled trials on robots’ effects on sleep.

Study selection and extraction

The first author, SJS, conducted the search. SJS was also responsible for the first screening of the titles and abstracts to assess the articles’ relevance according to the eligibility criteria stated above. SJS and NJ read and assessed all relevant full-text articles for their eligibility. The standard extraction sheet, the “Cochrane’s data extraction form for randomized controlled trial studies,” was used to extract the data.

Quality assessment

The assessment of the studies’ quality in terms of risk of bias was based on the Cochrane’s Risk of Bias 2 tools.²⁵ The risk of bias assessments were conducted independently by SJS and NJ before ratings were compared, and potential discrepancies were discussed and solved. The 6 quality criteria assessed with this tool were 1) the randomization process, 2) deviations from intended interventions, 3) missing outcome data, 4) measurement of the outcome, 5) selection of the reported results, and 6) overall bias. The 6 criteria were rated to have either low risk, some concerns, or high risk of bias.²⁵

Statistical synthesis and analysis

A network meta-analysis is a specific technique for “comparing three or more interventions simultaneously in a single analysis by combining both direct and indirect evidence across a network of studies”²⁶. It entails an estimation of the relative effects of 2 interventions that are not directly compared within the same study.²⁶ In the current study, it means direct and indirect comparisons of robots vs plush toys vs treatment as usual, and a multivariate random-effects meta-regression was used. This enabled the inclusion of studies that had at least 2 but not necessarily the same 2 or all 3 of these conditions. Since all studies measured sleep in the same way, mean differences (MDs) in total sleep time and standard deviations of change scores with associated P values and 95% confidence intervals (CIs) were extracted from the studies. If the network is consistent, meaning that the indirect evidence agrees with the direct evidence, a consistency model can be used; if not, an inconsistency model is preferred. We estimated consistency and inconsistency models as multivariate random-effects meta-regressions, with the mean differences in total sleep time as the between-treatment contrasts, using a Wald test to assess consistency. We used the augmented data format, where all treatments were compared with the robot treatment (the reference treatment). The analyses were performed in Stata/MP 17.0 using the suite of programs described by White.²⁷

RESULTS

Study selection

The first screening of the articles’ titles and abstracts for their relevance according to the inclusion criteria led to 59 potential studies. Seventeen studies were duplicates and therefore removed. Additionally, 38 studies were removed, either because the outcome or the study as a whole was not relevant to the research question (eg, review or study design other than a randomized controlled trial). This narrowed the search down to 4 relevant studies. No additional relevant studies were found by searching the 4 articles’ reference lists or by searching for articles that had cited the 4 studies according to Google Scholar. The 4 full-text articles were read and were all deemed relevant for inclusion. See Figure 1 for a flow chart of the selection process.

Figure 1. Flow chart of the selection process.

ICTRP = International Clinical Trials Registry Platform, ISRCTN = International Standard Randomised Controlled Trial Number.

Study characteristics

All 4 studies had samples of older people living in nursing homes, either with (n = 3) or without (n = 1) dementia. The 4 studies comprised 381 participants. The intervention sessions lasted between 10 and 30 minutes and were given 2 to 5 days a week for 6 to 12 weeks (see Table 1 for detailed descriptions about each study). The outcome measure selected for the data analysis was total sleep time or the equivalents referred to as sleep hours nighttime/total sleep hours/sleep duration, as this was the only outcome measure all 4 studies had in common. The assessment points immediately preintervention and directly postintervention were selected for the data analysis. For 1 of the studies,²³ we had to choose between using the assessment at 10 weeks (precisely after the intervention) or at 15 weeks (what the authors termed postassessment), and we picked the first. Total sleep time was measured with actigraphy (or the equivalent SenseWear in 2 studies^22,23) that is, “a device that uses an accelerometer to measure limb activity associated with movement during sleep for physiologic applications”²⁸.

Table 1.

Studies included in the review.

Author (Year)	Sample	Intervention and Control Groups	Outcome Measures	Assessment Points	Main Finding
Jøranson et al (2021)29	Residents with dementia from Norwegian special care units (n = 34).	30-min group sessions twice a week for 12 weeks: Robot (n = 16) vs Treatment as usual (n = 18).	Wrist actigraphy: Total sleep time (TST)	Pre-post intervention.	A statistically significant difference between the groups, in favor of the robot intervention.
Moyle et al (2018)23	Residents with dementia from Australian special care units (nighttime analyses n = 280).	15-min individual sessions 3 times a week for 10 weeks: Robot (n = 98) vs Plush toy (n = 95) vs Treatment as usual (n = 87).	SenseWear: Sleep hours nighttime.	Pre-post intervention (week 10).	No statistically significant difference between the groups.
Pu et al (2021)22	Residents with dementia from Australian nursing homes (n = 41).	30-min individual sessions Mon-Fri for 6 weeks: Robot (n = 21) vs Treatment as usual (n = 20).	SenseWear: Total sleep hours.	Pre-post intervention.	No statistically significant difference between the groups.
Thodberg et al (2016)19	Residents with/without dementia from Danish nursing homes (n = 26)	10-min biweekly individual sessions for 6 weeks: Robot (n = 14) vs Plush toy (n = 12).	Wrist actigraphy: Sleep duration.	Pre-post intervention.	No statistically significant differences between the groups.

In the first study, the sex and age of the participants with both baseline and post data were not separately reported (n = 34 out of 54 participants, where 15 had baseline data only and 5 had post data only).²⁹ The authors state, “[A] convenience sample of 60 participants was recruited (65.4% female, age range: 62–95 years),” and “inclusion criteria were aged over 65 years” to illustrate the initial composition of the sample.²⁹ In the second study, 98 participants were in the robot group (78% female, age: mean 85, standard deviation [SD] 8.3), 95 participants were in the plush toy group (80% female, age: mean 87 years, SD 7.0), and 87 participants were in the treatment-as-usual group (67% female, age: mean 85 years, SD 7.1).²³ In the third study, 21 participants were in the robot condition (85.7% female, age: mean 86.48 years, SD 8.81) and 20 in the no intervention control group (55% female, age: mean 85.50 years, SD 6.02).²² In the fourth study, the sexes and ages of the participants were not presented by experimental groups, but rather the total population (n = 100 over 3 conditions, of which only 2 studies and 26 participants were included in the current analysis): 69% female, age: mean 85.5 years, age range 79–90 years.¹⁹

Regarding the baseline total sleep time, the robot group in Jøranson et al²⁹ slept an average of 5.15 hours (SD 1.27), and the control group slept an average of 5.39 hours (SD 1.08) at baseline. In Moyle et al, ²³ the participants in the robot group slept an average of 7.05 hours at nighttime at baseline (SD 2.77), compared with 6.66 hours (SD 3.39) for the plush toy group and 7.07 hours (SD 2.75) for the treatment as usual control group. In Pu et al, ²² the robot group slept an average of 5.55 hours (SD 4.31) at baseline, whereas the usual care group slept 3.97 hours (SD 3.73). In Thodberg et al, ¹⁹ sleep duration is presented in minutes. The robot group slept an average of 481 minutes (or 8.02 hours) at baseline (SD 148), while the plush toy group slept an average of 531 minutes (or 8.85 hours) at baseline (SD 146).¹⁹ The baseline information about the participants’ total sleep time is relevant to judging whether robots have a chance to be effective (ie, a small chance if the participants have adequate sleep hours to begin with). See Table 1 for the extracted data.

Risk of bias

Concerning the risk of bias, 3 studies were considered to have a high risk of bias, whereas 1 was rated with some concerns.²² When comparing the individual assessments according to the Cochrane’s Risk of Bias 2 tool, SJS and NJ agreed on all 24 ratings (6 criteria applied to 4 studies). Two studies used cluster randomization and were judged to have a high risk of overall bias related to their design.^23,29 Other limitations with the studies were related to known group allocations during the interventions (all 4 studies) and missing outcome data,¹⁹ among other things. See Table 2 for the risk of bias assessments of the included studies.

Table 2.

Risk of bias assessment of the studies included in the review.

Study	Randomization Process	Deviations from Intended Interventions	Missing Outcome Data	Measurement of the Outcome	Selection of the Reported Results	Overall Bias
Jøranson et al (2021)29	High risk	Some concerns	Low risk	Some concerns	Low risk	High risk
Moyle et al (2018)23	High risk	Some concerns	Low risk	Some concerns	Some concerns	High risk
Pu et al (2021)22	Some concerns	Some concerns	Low risk	Some concerns	Some concerns	Some concerns
Thodberg et al (2016)19	Low risk	Some concerns	High risk	Some concerns	Some concerns	High risk

Synthesis of results

The MD in total nighttime sleep (after the baseline) and the SDs of the MDs were requested from the corresponding authors of all 4 studies. This information was provided for 3 studies. For the last study’s missing data, the MDs were calculated using the means reported in the article.²³ The SDs of the MDs were imputed assuming a correlation coefficient of 0.5, as shown in Higgins et al.²⁶ For all studies, we reported the MDs and SDs in minutes. A frequentist network meta-analysis was conducted, using multivariate random-effects meta-regression to simultaneously compare the effects of 3 treatments on total sleep time. Mean deviations and 95% CIs were computed. Three studies directly compared the robot intervention to treatment as usual; 2 studies directly compared the robot intervention to the plush toys. One study compared treatment as usual to the plush toy. Table 3 presents the MDs and SDs of the included studies’ experimental conditions in minutes.

Table 3.

Data extracted from the four included studies.

Study	MD (minutes)	SD	n	Treatment
Jøranson et al (2021)29	25.12	61.24	16	Robot
Jøranson et al (2021)29	−11.08	43.58	18	Treatment as usual
Moyle et al (2018)23	−31.20	185.40	98	Robot
Moyle et al (2018)23	3.60	153.60	87	Treatment as usual
Moyle et al (2018)23	−4.20	195.60	95	Plush toy
Pu et al (2021)22	−35.95	179.49	21	Robot
Pu et al (2021)22	35.50	226.95	20	Treatment as usual
Thodberg et al (2016)19	−8.71	73.28	14	Robot
Thodberg et al (2016)19	−22.50	93.18	12	Plush toy

Mean difference (MD) is the change in total sleep time (after – baseline). A positive value implies that total sleep time has increased during treatment; a negative value, that it has decreased. n = number of participants, SD = standard deviation of the mean differences.

Findings

According to a Wald test, the null hypothesis of overall consistency could not be rejected (χ² = 0.28, P = .87). Thus, we fit a consistency model in our network meta-analysis. The robot intervention was not found to change total sleep time more than plush toys (MD = 1.18, 95% CI [−57.66 60.03]). Moreover, treatment as usual was not found to change total sleep time more than plush toys (MD = 2.15, 95% CI [−64.46 68.75]), and treatment as usual was not found to change total sleep time more than robots (MD = 3.33, 95% CI [−48.36 55.02]. These results are presented in the network forest plot (Figure 2), which shows the results of the network meta-analysis for all 3 interventions pairs. Boxes are proportional to the weight of each study in the random-effects network meta-analysis, and show the estimated mean difference for each study, lines are 95% CIs. Diamonds presents pooled estimates and 95% confidence intervals from the network meta-analysis, including both direct and indirect comparisons.

Figure 2. Network forest plot.

Boxes are proportional to the weight of each study in the random-effects network meta-analysis, and shows the estimated mean difference for each study, lines are 95% confidence intervals. Diamonds presents pooled estimates and 95% confidence intervals from the network meta-analysis, including both direct and indirect comparisons.

DISCUSSION

Research on robots’ effects on sleep in humans is in its infancy. The current network meta-analysis is the first meta-analysis on robots’ effects on nighttime sleep in adults. The current study finds no statistically significant differences between robot interventions, plush toys, and treatment as usual concerning total sleep time. However, there are many possible reasons for this finding that are important to discuss. Of the 4 included studies, 3 were considered to have a high risk of bias. The studies’ methodological limitations affect both the internal and external validity of the current results. More large, rigorous studies are needed to help answer the research question of whether robots affect nighttime sleep in adults. As Kulpa et al³⁰ write, we need consensus on the assessment tools and outcome measures for such larger studies, in this case sleep measures, for the results of individual studies to be translatable. In sleep research, it is generally wise to use both objective and subjective sleep measures (when participants can complete questionnaires or sleep diaries, that is), as subjective and objective measures often show discrepancies and are complementary.³¹ Different sleep parameters measured with actigraphy (such as total sleep time) vary a great deal within the same individual, which is why several nights should be recorded.³² Seven days are generally accepted as the minimum. When 4 of 7 nights are missing, the entire week should be considered missing.³³ One of the included studies in the current network analysis did not measure total sleep time over several days.²³ Future randomized controlled trials should also follow reporting guidelines and report change scores and their SDs for the outcome measures in the studies.

One strength of the current network meta-analysis is that it is the first to focus on (nighttime) sleep. It was open to include studies with participants of all ages (although there were only older participants in the eligible studies). The inclusion of plush toy interventions in a network meta-analysis enabled the inclusion of studies that have not been included in previous meta-analyses (ie, individual studies without a treatment as usual control group), with the added benefit of learning about the potential effect of plush toys on sleep in adults, as plush toys have been and can be used as a placebo condition in future studies. Another strength of the current analysis is the similar functionalities of the robots used in the included studies, as PARO was used in all 4 studies. Three out of four studies assessed individual interventions, which can be more personalized to meet each individual’s need and which has been found to be the preferred format by the participants themselves, according to previous research.¹⁶

One limitation of the current network meta-analysis was the very few studies in the review, thereby restricting our analyses of potential sources of heterogeneity and bias. Given that 3 of 4 included studies were deemed to have a high risk of bias, this is even more problematic. More high-quality studies would have increased the bias assessment’s accuracy. A second limitation concerns the small samples of the included studies. A third limitation was the fact that all the included studies were conducted on older people, as the results cannot be generalized to younger adults, which is why it is of the essence to conduct randomized controlled trials of robots’ effects on nighttime sleep in different age groups. Furthermore, older people with lower cognitive capacity have been found to respond less positively to the PARO compared with people with higher capabilities.³⁴ This is relevant because most participants in the 4 included studies were diagnosed with dementia. Fourth, all the included studies were conducted in Western countries, which is why it is at risk of a cultural bias. On the same note, literature in languages other than English was not searched. A fifth limitation is the varied interventions (time and dose) used in the included studies. The interventions were also relatively short, with sessions between 10 and 30 minutes, 2 to 5 days a week, for 6 to 12 weeks, which perhaps were not enough to reach an effect, why more extensive interventions should be explored in the future. Previous studies on the PARO have shown that there were mixed responses to the robot (some liked it and others did not interact much with it³⁵). Unfortunately, the included studies have not measured adherence to the interventions. There was not much information about what “no intervention” or “treatment as usual” looked like in the studies, possibly masking a ceiling effect depending on what activities the nursing homes offered and these activities’ direct or indirect effects on the participants’ sleep.

Perhaps the most important limitation with the included studies was that the participants on average had adequate sleep hours according to their age, which is why it was unsurprising that the robot interventions did not have any significant effects on the participants’ total sleep hours compared with the other conditions. Additionally, total sleep time has been shown to be less sensitive to other sleep interventions compared with, for instance, sleep onset latency and wake time after sleep onset in people with insomnia,³⁶ which is why it is of utmost importance to choose the sleep outcome measures wisely in future randomized controlled trials. The PARO used in all the included studies has not been created to target nighttime sleep, which is something future studies on robots’ effects on sleep should focus on. One relevant ongoing trial is a randomized controlled trial on the effectiveness of the Somnox sleep robot in adults with insomnia.³⁷

Taken together, the randomized and cluster randomized controlled trials conducted up until today studying the effects of robots on total sleep time in adults, compared with plush toys and treatment as usual, do not comprise any statistically significant differences between the groups. More large, rigorous studies are needed, and the current study suggests critical aspects to consider when planning and conducting such studies, from a sleep researcheŕs, a public health researcher’s, and a statistician’s point of view.

ABBREVIATIONS

CI

confidence interval

MD

mean difference

PARO

Personal Assistance RobOt

SD

standard deviation

DISCLOSURE STATEMENT

All authors have seen and approved this manuscript. Work for this study was performed at Karlstad University, Sweden. The authors report no conflicts of interest.

DATA AVAILABILITY

The data analyzed in this study are available upon reasonable request to the corresponding author.