Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2020 Nov 4;15(11):e0240622. doi: 10.1371/journal.pone.0240622

Two is better than one: Social rewards from two agents enhance offline improvements in motor skills more than single agent

Masahiro Shiomi 1,*,#, Soto Okumura 1,2,#, Mitsuhiko Kimoto 1,3,#, Takamasa Iio 1,4,5,#, Katsunori Shimohara 2,#
Editor: Grzegorz Pochwatko6
PMCID: PMC7641341  PMID: 33147230

Abstract

Social rewards as praise from others enhance offline improvements in human motor skills. Does praise from artificial beings, e.g., computer-graphics-based agents (displayed agents) and robots (collocated agents), also enhance offline improvements in motor skills as effectively as praise from humans? This paper answers this question via two subsequent days’ experiment. We investigated the effect of the number of agents and their sense of presence toward offline improvement in motor skills because they are essential factors to change social effects and people’s behaviors in human-agent and human-robot interaction. Our 96 participants performed a finger-tapping task. Our results showed that those who received praise from two agents showed significantly better offline motor skill improvement than people who were praised by just one agent and those who received no praise. However, we identified no significant effects related to the sense of presence.

Introduction

Social rewards as praise from others not only promote positive emotional status but also physical changes. For example, past studies reported that praise provides various positive effects for humans: increased self-efficacy [1], improved motivation in children [2, 3], increased academic self-concept [4], greater pleasurable emotional reactions [5], and improved academic performances [6, 7]. Praise also positively influences physical changes, e.g., online improvement of task performances [8, 9]. Another previous study investigated praise effects on skill consolidation (i.e., offline improvement) in the offline consolidation of implicit sequence learning [9]. These results suggest that praise from others is a fundamental social reward [10].

Does praise from such artificial beings as computer-graphics-based agents (displayed agents) and robots (collocated agents) also positively affect humans? Past studies investigated the effect of praise from such agents. People perceived friendlier relationships with agents [11, 12], experienced improved motivation [13], and increased task online performances [14, 15]. A possible reason for such an effect of praise from such agents is that people regard them as social beings [16–18]. However, even if studies identified the effectiveness of praise, it remains unknown whether the praise from such agents has a similar effect on an offline consolidation, which is a critical learning process that is related to the amount of time spent on the learning [19, 20], as praise from humans. We hypothesize that praise from agents also directly influences the skill consolidation process. Our first research question: 1) Does praise from an agent influence the offline improvements of motor skills?

We are also interested in the factors that influence social rewards. Positive social feedback, which is an important factor in learning, improves offline learning. To increase the effects of such social feedback, the power of numbers is effective due to past human-human interaction literatures. For example, human behaviors and performance change consciously/unconsciously due to an increase in the number of people, such as social facilitation/loafing [21–25]. The power of the number effect is also observed in human-agent and human-robot interactions [26–28]. We hypothesize that the number of agents influences the effects of praise. Our second research question: 2) Does the number of agents increase the effects of the praise of offline improvements in motor skills?

Finally, we focused on the sense of the presence of the agents because researchers in human-robot interaction fields often experimentally investigated the advantages of collocated agents over displayed agents. Several studies claimed that collocated agents are perceived as more persuasive, lifelike, and positive [29–31]. Revealing the effectiveness of a sense of presence on praise effects would be useful knowledge for understanding how the characteristics of artificial agents influence human behaviors. We hypothesize that the sense of presence influences the effects of praise. Our third research question: 3) Does a sense of presence increase the effects of the praise of offline improvements on motor skills?

We verified these hypotheses on consecutive days in a two-day experiment that manipulated the timing of the praise, its number, the number of agents, and their sense of presence. Participants experienced training and praise (based on the conditions) on the first day and performed identical training on the second day without any information about the retest to minimize the possibility that they could practice the training task (Fig 1). These procedures enabled us to investigate the effects of praise on offline motor skill improvements.

Fig 1. Experiment design of study.

Fig 1

Open in a new tab

This study is an extended version of a previous work [32] and contains additional conditions with more participants, analysis, and discussions. Moreover, this study design follows a past study that investigated praise effects toward the offline improvement from people [9]. We changed the form of the praise, i.e., agents praised participants instead of the human experimenter’s movie.

Material and method

Participants

All 96 participants (48 women and 48 men who averaged 21.2 years of age, S.D: 1.37) gave informed consent for taking part in our two-day experiment. They were recruited by campus flyers at several universities in Japan and paid 500 Japanese yen (about $5). The ethics committee at the Advanced Telecommunication Research Institute approved this paper’s methodology. None of the participants had any history of neurological, psychiatric, or sleep disorders.

Environment

We conducted our experiment in a laboratory room at Doshisha University. Based on the condition, we installed either one or two displayed/collocated agents and a PC on a desk.

Task design

We measured their offline motor skill improvements with a sequential, finger-tapping task, which has commonly been used in past studies, to focus on the effects of praise and offline improvements on motor tasks [9]. The participants repeatedly tapped four keys (1, 2, 3, 4) on a computer keyboard as quickly and as accurately as possible. Each trial period was 30 seconds. The sequence was identical for all participants and trials: 4, 1, 3, 2, and 4.

Robot

In this study, we compared a collocated agent and a displayed agent. For the collocated agent, we used Sota, which was developed by VSTONE Inc. It has three degree of freedoms (DOFs) in its head, one DOF for each shoulder, one DOF for each elbow, and one for its base. It is 28 cm tall and weighs 763 g. It faced the participants and maintained a slight idling behavior to move its arms. For the displayed agent, we used Virtual-Sota, which is a computer-graphics-based agent that simulates Sota’s motion and speech. Virtual-Sota expresses the same motions and sounds as Sota.

Conditions

Our experiment used a 2 x 3 between-between-participant design, where participants were randomly assigned to one of two different senses of presence (displayed and collocated agents) and one of three different numbers of praise agents (no-praise, one-agent, and two-agent) by maintaining identical gender ratios among them. Thus, the participants were divided into six groups (Fig 2): “No-praise with one displayed agent,” “Praise with one displayed agent,” “Praise with two displayed agents,” “No-praise with one collocated agent,” “Praise with one collocated agent,” and “Praise with two collocated agents.”

Fig 2. Experiment design: In no-praise group, one displayed/collocated agent explains task’s progress without any praise.

Fig 2

Open in a new tab

In one-agent group, one displayed/collocated agent gives praise during task. All praise contents include two sentences. In two-agent group, each agent gives one sentence of praise. Praise amounts are identical to one-agent group.

Presence factor

We compared the task performance under two different presence conditions: displayed or collocated agent.

Number factor

We compared the task performance under three different number conditions. In the no-praise condition, we used one agent that always made neutral comments after each trial. In the one-agent condition, we also used one agent whose praise consisted of two sentences. In the two-agent condition, two agents used praise comments based on the rules; all the comments were separately used by two displayed/collocated agents.

Praise comments

By following the praise manipulation of past studies [9, 32], the agents provided comments to the participants after each trial. We prepared three kinds of comments to provide greater variety: pre-defined praise, neutral, and praise based on the task performance (Fig 3). Identical praise was not used on the same day.

Fig 3. Praise comments.

Fig 3

Open in a new tab

The pre-defined praise included seven comments. Each comment consisted of two sentences, e.g., “your performance is very good, especially if this is your first trial” and “your typing speed seems faster than in the previous trial.”

We had 12 neutral comments. We prepared ten comments about the current trial number and the remaining trials, e.g., “this completes the seventh trial, so you have five trials left.” We also prepared two comments that consisted of the current trial number and the average performance, e.g., “this completes your third trial, and your performance resembles that of others.”

For praise based on task performance, we implemented a simple rule-based algorithm to select praise comments (Fig 4) with three features: correct typed numbers, incorrect typed numbers, and total typed numbers. We prepared two comments for six categories: increase of correctly typed numbers, error-free typing, correct typing, decrease of typing errors, maintaining performance, and training attitude. The agents made these comments during the experiment to avoid skepticism based on overuse of the pre-defined praise comments, i.e., after the fourth and sixth trials. When the same category was selected by the algorithm for both timings, different comments were used. The order of the praise categories is based on the priority of the algorithms (Fig 3B).

Fig 4. Pseudo-code for praise comment control.

Fig 4

Open in a new tab

Increase of correctly typed numbers

When the amount exceeds the maximum number at this point, the agents praised the number of correctly typed words: “since you correctly typed more numbers, your typing speed is increasing.”

Error-free typing

When a participant did not make any mistakes, the agents praised her accuracy: “since you made no mistakes, your typing is really accurate.”

Correct typing

When the participant’s typing number exceeded a threshold (150 types in this study), the agents praised the typing amount: “your typing record is good since your number of correctly typed numbers is above average.”

Decrease of tying errors

When the amount of typing errors fell below a minimum number at this point, the agents praised: “since you’ve decreased your typing errors, it has become more accurate.”

Maintaining performance

When none of the above praise categories were selected and the total amount of typed numbers exceeded the last task, the agents praised the training task for maintaining its performance. Thus, the person’s concentration was praised regardless of the performance: “you are maintaining your performance and concentrating.”

Training attitude

When the algorithm could not select any of the above categories, i.e., when the participant failed to maintain her performance or her amount of typing, the agents praised her attitude: “since your typing performance seems to have stabilized, your performance will probably increase.”

Procedure

All the participants came to our laboratory room on back-to-back days. On day 1, they did 12 trials of sequential finger-tapping tasks. The participants were told that the displayed/collocated agents would explain their roles and that the experimenter would explain the start/end timing of the tasks. After the experimenter left the room, the agents greeted the participants and explained their MC roles. The agents asked the participants to start the training task and explained that the experiment would end after all the trials were completed. The agents provided comments based on the conditions after each trial. The participants were given a 20-second break after each trial. At the end of day 1, the experimenter explained the next day’s schedule (with a different task) to minimize the possibility that they would practice the training sequence before the evaluation of day 2.

On day 2, the participants experienced the same finger-tapping task in five trials (Fig 1A, right), different from the explanations of the last day. On day 2, the agents didn’t praise; they just offered neutral comments.

Measurement

We measured both objective and subjective items to evaluate the effects of praise. As objective items, we measured the performance of the last three training trials on day 1 and the first three retest trials on day 2. As subjective items, we gathered two questionnaire items to investigate the feeling of the perceived happiness to the agents’ speech contents (“I felt happy after listening to the agent speeches”), and the degree of perceived praise (“I think that the agents praised me”). For these questionnaires, we used a response format on a seven-point-scale that described the options from 1 (strongly disagree) to 7 (strongly agree) [33].

Results

Performance of trained sequences

Fig 5 and Table 1 show the performance of the last three training trials on day 1 and the first three retest trials on day 2. First, we investigated the performances of each group at the end of day 1 as a manipulation check. A Levene’s test indicated that the equality of the error variances was assumed at a significant level of 0.05 (p = 0.840), and a Kolmogorov-Smirnov test indicated that the data are consistent with a normal distribution (p = 0.200). A two-way analysis of variance (ANOVA) showed that the performance at the end of the training on day 1 did not show a significant difference for all the factors: the presence factor (F(1, 90) = 1.197, p = .277, partial η2 = .013), the number factor (F(2, 90) = 1.214, p = .302, partial η2 = .026), and the interaction effect (F(2, 90) = 0.174, p = .840, partial η2 = .004). Therefore, the mean performances of the last three training trials on day 1 were not significantly different among the conditions.

Fig 5. Graph of mean performance of trained sequence (average and S.E.).

Fig 5

Open in a new tab

Table 1. Mean performance of trained sequence (average, S.D, and S.E.).

Day 1 Day 2 Day 1 Day 2 Day 1 Day 2
Average S.D. S.E
No-praise 155.0 160.7 8.6 10.0 2.1 2.5
Collocated One-agent 160.4 172.9 5.9 6.3 1.5 1.6
Two-agents 153.8 176.3 6.9 8.3 1.7 2.1
No-praise 145.8 150.0 8.0 7.4 2.0 1.9
Displayed One-agent 158.9 168.1 7.5 8.1 1.9 2.0
Two-agents 145.3 162.2 5.8 5.0 1.4 1.2
Open in a new tab

Fig 6 shows the offline improvement of each condition, which is the percentage of increase from the mean performance of days 1 to 2. A Levene’s test indicated that the equality of the error variances was assumed at a significant level of 0.05 (p = 0.392), and a Kolmogorov-Smirnov test indicated that the data are consistent with a normal distribution (p = 0.200). A two-way ANOVA showed a significant main effect in the number factor (F(2, 90) = 29.4485, p < .001, partial η2 = .396). No significance was found in the presence factor (F(1, 90) = 1.685, p = .198, partial η2 = .018) or the interaction effect (F(2, 90) = 0.472, p = .625, partial η2 = .010). Multiple comparisons with the Bonferroni method revealed a significant difference: two > no-praise (p < .001), two > one (p < .001), and one > no-praise (p = .021). Therefore, praise from the displayed/collocated agents influenced the offline improvement of the motor skills. The number of agents significantly affected the offline improvement of the motor skills. On the other hand, even though a trend seemed to suggest the presence factor impacted the improvement, as shown in the graph, it did not significantly affect their offline improvements.

Fig 6. Rate of offline improvement: Vertical axis indicates percentage of increase from mean performance of day 1 to 2.

Fig 6

Open in a new tab

Subjective impressions toward praise: Happiness rating and degree of perceived praise

At the end of day 1, we subjectively assessed the participant’s happiness and degree of perceived praise by a questionnaire on a seven-point-scale (where 1 is most negative, and 7 is most positive). For the happiness rating (Fig 7), a Levene’s test indicated that the equality of the error variances was assumed at a significant level of 0.05 (p = 0.208), but a Kolmogorov-Smirnov test did not indicate that the data are consistent with the normal distribution (p<0.001). Therefore, we performed a Box-Cox transformation for the questionnaire results. A two-way ANOVA showed a significant main effect in the number factor (F(2, 90) = 25.312, p < .001, partial η2 = .360). No significance was found in the type factor (F(1, 90) = 0.435, p = .511, partial η2 = .005) or the interaction effect (F(2, 90) = 0.199, p = .820 partial η2 = .004). Multiple comparisons with the Bonferroni method revealed a significant difference: two > no-praise (p < .001) and one > no-praise (p < .001). We found no significant differences between two and one (p = 1.000).

Fig 7. Happiness rating.

Fig 7

Open in a new tab

Concerning the degree of perceived praise (Fig 8), a Levene’s test indicated that the equality of the error variances was assumed at a significant level of 0.05 (p = 0.029), and a Kolmogorov-Smirnov test did not indicate that the data are consistent with the normal distribution (p<0.001). Therefore, we performed a Box-Cox transformation for the questionnaire results. A two-way ANOVA showed a significant main effect in the number factor (F(2, 90) = 168.085, p < .001, partial η2 = .789). No significance was found in the type factor (F(1, 90) = 0.584, p = .447, partial η2 = .006) or the interaction effect (F(2, 90) = 0.578, p = .563, partial η2 = .013). Multiple comparisons with the Bonferroni method revealed a significant difference: two > no-praise (p < .001) and one > no-praise (p < .001). We found no significant differences between two and one (p = 0.175). Therefore, the results indicate successful manipulation of praise.

Fig 8. Degree of perceived praise.

Fig 8

Open in a new tab

Discussion

Design implications

In this study, we have three research questions, and our results showed support for R1 and R2, but not for R3. All the praised groups showed more offline motor skill improvements between days 1 and 2 than for the no-praise groups; the results showed the effectiveness of praise from artificial beings and humans [9, 19]. The number of praise agents enhanced the offline motor skill improvements, i.e., where praise from two agents was significantly better than praise from just one, even if the amount of praise was identical between the conditions.

Offline motor skill improvement would be useful for the education support of children, e.g., learning such basic computer skills as inputting characters by a keyboard. Moreover, if the effects of praise from multiple agents improve both offline motor skills and motivation, the knowledge will be helpful for designing educational support systems because such recent systems often use displaced/collocated physical agents to support learning. In fact, several studies investigated the effectiveness for such learning support agent systems [34, 35]. Since they reported some positive effects of praise from a single collocated robot, investigating praise effects on other education topics is interesting future work.

On the other hand, even though our experiment results suggest that the presence factor impacted the improvement, it did not significantly affect their offline improvements, i.e., the presence effects of agents is limited in our experimental contexts. A possible reason is the simplicity of the conversational interaction in our study. Past studies identified the effectiveness of such physical interaction as touching and hugging to change people’s behaviors and their perceived feelings [36]; if collocated agents physically interact with participants, the praise effects might be changed. Of course the absence of evidence is not proof of an absence. Our results do not downplay the advantages of collocated agents compared to displayed agents.

Our experimental results pose another research question: does praise from two people enhance offline motor improvements more than praise from just one person? Since we focused on the effects of artificial beings, we did not directly evaluate the effects of praise from multiple persons. Yet people regard displayed/collocated agents as social beings [16–18]. In fact, past studies and our study showed that the effect of praise from artificial agents resembles that from humans [21–25]. Thus, we assume that praise from two people would be more effective than praise from just one.

In summary, our study showed that praise from artificial beings improved skill consolidation in humans in a manner that resembles praise from human beings. The number of agents provides a basic effect for improving skill consolidation. Unlike past studies, our experimental results showed no significant effect of a sense of presence in the context of offline motor skill improvement.

Effects of sense of presence, number, and environment

We compared the collocated and displayed agents to investigate the effects of praise from them. However, we are interacting with different types of agents, such as unembodied agents, voice-only agents, and simple-appearance collocated agents like a smart-speaker. Our experiment results did not show any significant effects in the sense of presence, although comparing these simpler agents would provide rich knowledge about praise effects.

In the context of the sense of presence, the number of agents is another essential factor to influence people’s behaviors. Although this study showed that praise from two agents is better than one agent, we did not investigate the effects of three or more robots. The number of agents will increase such social influences and a saturation point will be identified. For example, past studies reported that the number of others influences social facilitation [37], social loafing [38], and peer pressure effects [39]. A past study on peer pressure effects from social robots reported that the number of robots changes peer pressure effects, but they also concluded that their total power is relatively weak compared to humans [40]. The praise effects, which have also been influenced by the number of robots, might be weaker than human participants. Investigating such effects is another interesting future work.

From another perspective, the differences of environments, i.e., physical and virtual reality, might be an important research topic. Several past studies investigated people’s similar reactions to agents in VR/AR settings compared to collocated agents [41–43]. Our experiment results showed that the displayed agents showed relatively weaker effects than the collocated robots without any significant differences between them. But based on these related studies, immersive virtual reality or augmented reality will increase the effects of displayed agents. Therefore, investigating the praise effects from displayed agents between physical/virtual environments is another interesting future work.

Conclusion

We investigated the effects of social rewards from agents for offline improvements in motor skills by considering the sense of presence (i.e., displayed and collocated agents) and the number of agents (one or two). We used a finger-tapping task and followed previous similar studies to investigate offline improvements and prepare praise comments for agents. Our experiment results showed that participants who received praise from two agents showed significantly better offline motor skill improvement than people who were praised by just one agent and those who received no praise. On the other hand, we identified no significant effects related to the sense of presence, unlike past studies that investigated its effects with displayed and collocated agents.

Supporting information

S1 File. Anonymized data set.

(XLSX)

Click here for additional data file. (69.2KB, xlsx)
S2 File

(PDF)

Click here for additional data file. (1MB, pdf)

Data Availability

All relevant data are within the manuscript and its Supporting Information files.

Funding Statement

M.S. S.O. and K.S: JST CREST Grant, Number JPMJCR18A1, Japan M.K: JSPS KAKENHI Grant Numbers JP19J01290 T.I: JST PRESTO Grant Number JPMJPR1851, Japan, and JSPS KAKENHI Grant Numbers JP18H03311.

References

  • 1.Bandura A., “Self-efficacy: toward a unifying theory of behavioral change,” Psychological review, vol. 84, no. 2, pp. 191, 1977. 10.1037//0033-295x.84.2.191 [DOI] [PubMed] [Google Scholar]
  • 2.Ames C., “Motivation: What teachers need to know,” Teachers college record, vol. 91, no. 3, pp. 409–421, 1990. [Google Scholar]
  • 3.Kohls G., Peltzer J., Herpertz-Dahlmann B., and Konrad K., “Differential effects of social and non-social reward on response inhibition in children and adolescents,” Developmental science, vol. 12, no. 4, pp. 614–625, 2009. 10.1111/j.1467-7687.2009.00816.x [DOI] [PubMed] [Google Scholar]
  • 4.Chalk K., and Bizo L. A., “Specific praise improves on-task behaviour and numeracy enjoyment: A study of year four pupils engaged in the numeracy hour,” Educational Psychology in Practice, vol. 20, no. 4, pp. 335–351, 2004. [Google Scholar]
  • 5.Aliyev R., Karakus M., and Ulus H., “Teachers’ Verbal Cues That Cause Students to Feel Various Emotions,” Anthropologist, vol. 16, no. 1–2, pp. 263–272, 2013. [Google Scholar]
  • 6.Henderlong J., and Lepper M. R., “The effects of praise on children’s intrinsic motivation: a review and synthesis,” Psychological bulletin, vol. 128, no. 5, pp. 774, 2002. 10.1037/0033-2909.128.5.774 [DOI] [PubMed] [Google Scholar]
  • 7.Kulkarni S., and Mitra S., “Management of Remote Mediation for Children’s Education over the Internet,” in Global Learn, pp. 2044–2049, 2010. [Google Scholar]
  • 8.Catano V. M., “Relation of improved performance through verbal praise to source of praise,” Perceptual and Motor Skills, vol. 41, no. 1, pp. 71–74, 1975. [Google Scholar]
  • 9.Sugawara S. K., Tanaka S., Okazaki S., Watanabe K., and Sadato N., “Social rewards enhance offline improvements in motor skill,” PLoS ONE, vol. 7, no. 11, pp. e48174, 2012. 10.1371/journal.pone.0048174 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Buss A. H., Social Behavior and Personality (Psychology Revivals): Routledge, 2014. [Google Scholar]
  • 11.C. Nass, J. Steuer, and E. R. Tauber, “Computers are social actors,” in Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 72–78, 1994.
  • 12.Johnson D., Gardner J., and Wiles J., “Experience as a moderator of the media equation: the impact of flattery and praise,” International Journal of Human-Computer Studies, vol. 61, no. 3, pp. 237–258, 2004. [Google Scholar]
  • 13.Mumm J., and Mutlu B., “Designing motivational agents: The role of praise, social comparison, and embodiment in computer feedback,” Computers in Human Behavior, vol. 27, no. 5, pp. 1643–1650, 2011. [Google Scholar]
  • 14.D. Leyzberg, S. Spaulding, and B. Scassellati, “Personalizing robot tutors to individuals’ learning differences,” in Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pp. 423–430, 2014.
  • 15.Warren Z. E., Zheng Z., Swanson A. R., Bekele E., Zhang L., Crittendon J. A., et al. , “Can robotic interaction improve joint attention skills?,” Journal of autism and developmental disorders, vol. 45, no. 11, pp. 3726–3734, 2015. 10.1007/s10803-013-1918-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.B. Reeves, and C. Nass, “How people treat computers, television, and new media like real people and places,” CSLI Publications and Cambridge, 1996.
  • 17.Kahn P. H. Jr, Kanda T., Ishiguro H., Freier N. G., Severson R. L., Gill B. T., et al. , ““Robovie, you’ll have to go into the closet now”: Children’s social and moral relationships with a humanoid robot,” Developmental psychology, vol. 48, no. 2, pp. 303, 2012. 10.1037/a0027033 [DOI] [PubMed] [Google Scholar]
  • 18.Hayashi K., Shiomi M., Kanda T., and Hagita N., “Are Robots Appropriate for Troublesome and Communicative Tasks in a City Environment?,” IEEE Transactions on Autonomous Mental Development, vol. 4, no. 2, pp. 150–160, 2012. [Google Scholar]
  • 19.Robertson E. M., Pascual-Leone A., and Miall R. C., “Current concepts in procedural consolidation,” Nature reviews. Neuroscience, vol. 5, no. 7, pp. 576, 2004. 10.1038/nrn1426 [DOI] [PubMed] [Google Scholar]
  • 20.Walker M. P., “A refined model of sleep and the time course of memory formation,” Behavioral and brain sciences, vol. 28, no. 1, pp. 51–64, 2005. 10.1017/s0140525x05000026 [DOI] [PubMed] [Google Scholar]
  • 21.R. B. Zajonc, Social facilitation: Research Center for Group Dynamics, Institute for Social Research, University of Michigan Ann Arbor, 1965.
  • 22.S. J. Karau, and K. D. Williams, "Social loafing: A meta-analytic review and theoretical integration," American Psychological Association, 1993.
  • 23.B. W. Jorgensen, “Group size: its effects on group performance and subsequent individual performance,” 1973.
  • 24.Thornburg T. H., “Group size & member diversity influence on creative performance,” The Journal of creative behavior, vol. 25, no. 4, pp. 324–333, 1991. [Google Scholar]
  • 25.Dennis A. R., and Williams M. L., “A Meta-Analysis of Group Size Effects in Electronic. Brainstorming,” Emerging E-collaboration Concepts and Applications, vol. 1, pp. 250. [Google Scholar]
  • 26.Sakamoto D., Hayashi K., Kanda T., Shiomi M., Koizumi S., Ishiguro H., et al. , “Humanoid Robots as a Broadcasting Communication Medium in Open Public Spaces,” International Journal of Social Robotics, vol. 1, no. 2, pp. 157–169, 2009. [Google Scholar]
  • 27.M. Shiomi, and N. Hagita, “Do Synchronized Multiple Robots Exert Peer Pressure?,” in Proceedings of the Fourth International Conference on Human Agent Interaction, Biopolis, Singapore, pp. 27–33, 2016.
  • 28.T. Iio, Y. Yoshikawa, and H. Ishiguro, “Pre-scheduled Turn-Taking between Robots to Make Conversation Coherent,” in Proceedings of the Fourth International Conference on Human Agent Interaction, Biopolis, Singapore, pp. 19–25, 2016.
  • 29.A. Powers, S. Kiesler, S. Fussell, and C. Torrey, “Comparing a computer agent with a humanoid robot,” in Human-Robot Interaction (HRI), 2007 2nd ACM/IEEE International Conference on, pp. 145–152, 2007.
  • 30.Bainbridge W. A., Hart J. W., Kim E. S., and Scassellati B., J. I. J. o. S. R., “The benefits of interactions with physically present robots over video-displayed agents,” vol. 3, no. 1, pp. 41–52, 2011. [Google Scholar]
  • 31.Li J., “The benefit of being physically present: A survey of experimental works comparing copresent robots, telepresent robots and virtual agents,” International Journal of Human-Computer Studies, vol. 77, pp. 23–37, 2015. [Google Scholar]
  • 32.S. Okumura, M. Kimoto, M. Shiomi, T. Iio, K. Shimohara, and N. Hagita, “Do Social Rewards from Robots Enhance Offline Improvements in Motor Skills?,” in International Conference on Social Robotics, pp. 32–41, 2017.
  • 33.Carifio J., and Perla R. J., “Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes,” Journal of Social Sciences, vol. 3, no. 3, pp. 106–116, 2007. [Google Scholar]
  • 34.Senft E., Lemaignan S., Baxter P. E., Bartlett M., and Belpaeme T., “Teaching robots social autonomy from in situ human guidance,” Science Robotics, vol. 4, no. 35, pp. eaat1186, 2019. [DOI] [PubMed] [Google Scholar]
  • 35.Belpaeme T., Kennedy J., Ramachandran A., Scassellati B., and Tanaka F., “Social robots for education: A review,” Science robotics, vol. 3, no. 21, pp. eaat5954, 2018. [DOI] [PubMed] [Google Scholar]
  • 36.Willemse C. J., and van Erp J. B., “Social Touch in Human–Robot Interaction: Robot-Initiated Touches can Induce Positive Responses without Extensive Prior Bonding,” International Journal of Social Robotics, vol. 11, no. 2, pp. 285–304, 2019. [Google Scholar]
  • 37.Zajonc R. B., “Social facilitation,” Science, vol. 149, no. 3681, pp. 269–274, 1965. 10.1126/science.149.3681.269 [DOI] [PubMed] [Google Scholar]
  • 38.Karau S. J., and Williams K. D., “Social loafing: A meta-analytic review and theoretical integration,” Journal of personality and social psychology, vol. 65, no. 4, pp. 681, 1993. [Google Scholar]
  • 39.Asch S. E., “Opinions and Social Pressure,” Scientific American, vol. 193, pp. 17–26, 1955. [Google Scholar]
  • 40.Shiomi M., and Hagita N., “Do the number of robots and the participant’s gender influence conformity effect from multiple robots?,” Advanced Robotics, vol. 33, no. 15–16, pp. 756–763, 2019. [Google Scholar]
  • 41.Pan X., Gillies M., Barker C., Clark D. M., and Slater M., “Socially anxious and confident men interact with a forward virtual woman: an experimental study,” PloS one, vol. 7, no. 4, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Llobera J., Spanlang B., Ruffini G., and Slater M., “Proxemics with multiple dynamic characters in an immersive virtual environment,” ACM Trans. Appl. Percept., vol. 8, no. 1, pp. Article 3, 2010. [Google Scholar]
  • 43.K. Kim, L. Boelling, S. Haesler, J. Bailenson, G. Bruder, and G. F. Welch, “Does a digital assistant need a body? The influence of visual embodiment and social behavior on the perception of intelligent virtual agents in AR,” in 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 105–114, 2018.

Decision Letter 0

Grzegorz Pochwatko

6 May 2020

PONE-D-20-08172

Two is Better than One: Social Rewards from Two Agents Enhance Offline Improvements in Motor Skills More than Single Agent

PLOS ONE

Dear Dr. Shiomi,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Jun 20 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Grzegorz Pochwatko, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include in your methods section a short description of how participants were recruited.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere. Please clarify whether this [conference proceeding or publication] was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Additional Editor Comments (if provided):

The paper is sound and interesting, however it has to be improved for better clarity. Both reviewers have done excelent job in specifying what has to be done to reach perfect result. I strongly encourage You to introduce their suggestions and proceed

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The article introduces a study to show that positive feedback from virtual agents can improve offline learning. The study is timely and relevant for the field, although it suffers from a number of shortcommings that should be addressed. I detail them below.

English should be improved. For example, in the abstract:

Such social rewards as praise from others enhance offline improvements in our motor skills. Does praise from artificial beings, e.g., computer-graphics-based agents (virtual

should read:

Social rewards such as praise from others…

Also, in the introduction:

Thus, the following is our first research question:

should be rephrased.

Also:

The effects of enhancing offline improvement in motor skills resemble a kind of social influence from others. In the context of social influence, one leading factor in human-human interaction is the power of numbers; human behaviors and performance change consciously/unconsciously due to an increase of the number of people, such as social facilitation/loafing [19-23]. The power of the number effect is also observed in human-agent and humanrobot

interactions [24-26]. We hypothesize that the number of agents influences the effects of praise. Thus,

the following is our second research question:

This paragraph should be rewritten almost entirely to read better. For example:

The effects of enhancing offline improvement in motor skills resemble a kind of social influence from others.

You probably mean:

Positive Social feedback is an important factor in learning, and it improves offline learning.

In the disucssoin:

Still our study only…

This is again a strange construction.

I can find similar constructions all across the article, and would recommend addressing them by trying to put simpler grammatical structures, or have a native speaker thoroughly review the article.

Regarding references:

A key concept of the paper stated in the introduction is

offline consolidations (a critical process for learning)

but has no reference to point to. Although this is a basic concept, since this kind of article might also be read by software engineers with no background in learning science I would advise to add relevant references for this.

There are a striking number of similarities of this work with a work they cited (reference 9, which can be found here:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0048174

Authors should better clarify inthe introduction that the designed is very similar (task, variable manipulated), but changing the way the praise is given (in one case a movie, in another a robot), and that the knowledge that the agent is not a real human (in the movie participants were told these were real people giving live feedback) does not change the main challenge.

Regarding experimental design:

The experiment is well designed, and it avoids a number of possible confounding factors.

however, in the no-physical condition actually rendered a character of similar size and position than the physical robot. It is possible to expect that an agent that is not-embodied (i.e., a voice with no body) would produce a different effect. I would therefore avoid the term non-physical, and rather opose a physical robot to a digital character.

Results.

I believe this could be improved.

Figure 2.a shows the average number of sequences completed in 30 seconds, but does not report on inter-subject variability which, I would argue, is quite important for a between group design. I would recomment including this.

Figure 2.b shows there is an effect of the number of agents. It also shows a trend in the impact of the physicality of the agent (a physical robot or a robot rendered on a screen).

However, the interpretation states clearly

authors should also explain what the vertical axis stands for in the caption of the figure.

authors should also state clearly that there seems to be a trend towards the fact that the physicality of the agent impacted the improvement, but the analysis method chosen (anova with bonferroni correction) does not reflect that.

It is entirely possible that the election of the analysis method is affecting this conclusion. I have not seen anywhere whether the measure shown in figure 2.b is normally distributed, which should be tested before performing an ANOVA analysis.

A linear regression, a generalized linear model or a bayesian analysis might be more suited than an ANOVA for this purpose.

In figure 2.c

rating questionnaires are ranked variables, and therefore not suited for anova analysis. Authors should resort to non-parametric tests, or do another kind of analysis.

I would also recommend providing the anonymized dataset together with the article submission in order to facilitate a re-analysis by other teams. If they provided the code

Discussion

I see a number of elements missing in the discussion that should be nuanced

Authors state: our experiment results did not show significant differences in physicality in the context of offline motor skill improvements

this can be misleading given the previous comments on the results of figure 2.b

-Immersive virtual reality (and recently, Immersive Augmented reality) has shown that the reaction to digital characters can be quite similar to physical agents, even with low quality of rendering. This is probably caused by the feeling of having a shared physical space with a digital agent, and therefore a social interaction much closer to real social interaction.

see, for example

https://dl.acm.org/doi/10.1145/1857893.1857896

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0032931

https://vhil.stanford.edu/pubs/2018/does-a-digital-assistant-need-a-body/

I recommend the authors to discuss this fact in the context of their experiment.

In addition to the nuancing of the impact of the physicality of the agent (and eventually, a re-analysis of that result), I would also recommend nuancing the statements regarding the number of agents. It is unlikely this effect would scale linearly. For example, whether having 4 robots, or 8 robots would improve the feedback (keeping the feedback sentences stable).

Structure.

In the results I can read this:

Each group had 16 participants, all of whom came to our laboratory

on back-to-back days. They were trained on a sequential finger-tapping task for which offline

improvement has previously been described [9, 30-32]. Their performance was defined by the number of

correctly tapped sequences per 30-second trials.

I believe this should be in the methods section.

I also can read this:

Note that the amount of praise is identical between groups; in the “Praise with one virtual agent” and “Praise with one physical agent” groups, the virtual/physical agents provided two sentences of praise. In the “Praise with two virtual agents” and “Praise with two physical agents” groups, each virtual/physical agent made just one sentence of praise.

This kind of clarification could probably be avoided if the methods section was before the results.

also, in the methods sectoin I found:

By following a past study’s praise manipulation

I would recommend that authors reference the study mentioned

Reviewer #2: PONE-D-20-08172

Two is Better than One: Social Rewards from Two Agents Enhance Offline Improvements in Motor Skills More than Single Agent

The paper presents one study aimed to determine whether the offline motor skills of a human partner could be improved by (1) the praise from a synthetic agent; (2) the number of agents that praise; (3) the type of physicality of the agent (virtual vs. physical).

The paper has many positive aspects (i.e., the proposed methodology is sound; the statistical treatments are adequate; Fig 1 and 2 are clear and very helpful to understand quickly the design and to picture the experiment done). The topic is relevant, and the paper could contribute significantly to the body of literature on Human Agent/Robot Interaction in general and specifically in an educational context.

However, there are some limitations (that can be fairly easily addressed) which should be addressed before publication.

Although the theoretical part is short, it is direct and explicit.

MISCELLANEOUS - MINOR ISSUES

The Fs, ps and other indicators should be in italics.

A table with the means and the standard deviations should be presented.

Some typos need to be corrected: ex: “popele would be effective moreat than praise from one person”

DOF = degree of freedom? Make it clear for the reader who is not used to with robotics.

MAJOR ISSUES

Although I do understand the initial intention of the authors, I would suggest a reorganization of the paper. Indeed, the present organization does not help to easily understand the experiment. For example, information about the experimental design, the procedure, or the material are either repeated in different sections (sometimes three times) or explained too “late” to have a big picture of the experiment that would help to understand the results (e.g., the reader has to wait the Results section to know that Happiness and Perceived Praise have been measured, as well as the way they were measured).

Consequently, I would suggest a more traditional organization to avoid repetitions: theoretical part, method with sample description, material description, procedure etc.. and then the results section and the discussion.

Moreover, how happiness and perceived praise were measured is not described. It is only said that it was measured “by a questionnaire on a seven-point-scale”. Instruments should be described as well as their reliability if applicable (e.g. Cronbach alpha).

Practical implications of the influence of praise on offline motor skills in educational context (for example) should be explored in the discussion.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Joan Llobera

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Nov 4;15(11):e0240622. doi: 10.1371/journal.pone.0240622.r002

Author response to Decision Letter 0

16 Jun 2020

We are grateful to the reviewers and editors for their detailed comments and specific feedback that enabled us to improve our paper. We considered all comments and tried to address them. We hope that the changes are satisfactory. In the attached document("Response to Reviewers" at the bottom of the PDF file), the comments from the reviewers are underlined.

Attachment

Submitted filename: ReviewReply_PlosOne_Two_ver006.pdf

Click here for additional data file. (223.2KB, pdf)

Decision Letter 1

Grzegorz Pochwatko

30 Sep 2020

Two is Better than One: Social Rewards from Two Agents Enhance Offline Improvements in Motor Skills More than Single Agent

PONE-D-20-08172R1

Dear Dr. Shiomi,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Grzegorz Pochwatko, Ph.D.

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The comments have been addressed. Some English writing has been improved. The statistical analysis is quite more clear, now.

I think this work should be published

Reviewer #2: Authors have modified the main text according recommendations of both reviewers and responded to all the questions.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Joan Llobera

Reviewer #2: No

Acceptance letter

Grzegorz Pochwatko

8 Oct 2020

PONE-D-20-08172R1

Two is Better than One: Social Rewards from Two Agents Enhance Offline Improvements in Motor Skills More than Single Agent

Dear Dr. Shiomi:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Grzegorz Pochwatko

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File. Anonymized data set.

    (XLSX)

    Click here for additional data file. (69.2KB, xlsx)
    S2 File

    (PDF)

    Click here for additional data file. (1MB, pdf)
    Attachment

    Submitted filename: ReviewReply_PlosOne_Two_ver006.pdf

    Click here for additional data file. (223.2KB, pdf)

    Data Availability Statement

    All relevant data are within the manuscript and its Supporting Information files.

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES