Evaluating the Effects of Technology-Based Self-Monitoring on Positive Staff–Consumer Interactions in Group Homes

Abstract

The quality and frequency of positive interactions between staff and consumers are related to reductions in consumer problem behavior and increases in other desired outcomes, such as leisure and self-help skills. Unfortunately, the frequency with which group home staff positively interact with consumers is often low and regularly requires intervention. We evaluated the effects of technology-based self-monitoring on positive interactions between staff and consumers during consumer leisure time. Participant data were collected off-site through video recordings from cameras already present in the group homes. During baseline, participant interactions were low. Upon introduction of an intervention containing self-monitoring completed via a tablet device, staff interactions increased and maintained when the intervention was in effect. Supplemental feedback via text message was provided to two of the three participants to reach criterion. These findings demonstrate the utility of technology-based self-monitoring for some individuals to increase positive staff–consumer interactions in group homes.

Of the estimated 7.38 million individuals with intellectual and developmental disabilities (IDD) in the United States, about 1.28 million receive long-term supports; 296,097 of whom receive services in group homes (Larson et al., 2020). Nearly 473,000 individuals are on waitlists for home and community-based services (United Cerebral Palsy & ANCOR Foundation, 2020). The approximately 1.3 million direct support professionals who work in service agencies (President’s Committee for People with Intellectual Disabilities, 2017) are not sufficient to meet the growing demand for services. Thus, it is essential that interventions and other services within group homes are effective, efficient, and produce meaningful outcomes for consumers.

Ensuring positive interactions between staff and consumers occurs is one approach to achieving meaningful outcomes for consumers (e.g., Burg et al., 1979; Baldwin & Hattersley, 1984; Felce & Emerson, 2001; Manente et al., 2010; Zoder-Martell et al., 2014). The importance of positive interactions has been demonstrated through the active treatment literature (Medicaid, n.d.; Parsons et al., 1989). For example, Parsons et al. (1989) showed positive interactions as part of active treatment correlated with increased consumer engagement and self-help, leisure, social, and community living skills. Research also shows higher consumer compliance, on-task behavior, and indices of happiness when consumers interact with familiar staff with whom a relationship exists than with unfamiliar staff (Parsons et al., 2016). Taken together, these findings suggest it is worthwhile to ensure positive interactions between staff and consumers occur.

Unfortunately, positive interactions occur at disappointingly low levels. Chan and Yau (2002) observed staff–consumer interactions in an institutional care setting. Results showed staff–consumer interactions occurred in only 37% of intervals. In addition, multiple studies conducted in various settings have shown interactions do not occur frequently during baseline (e.g., Burg et al., 1979; Parsons et al., 1989; Zoder-Martell et al., 2014). Based on these patterns of results, it appears there is a need for interventions that increase and maintain positive staff–consumer interactions.

Researchers have evaluated ways to increase positive interactions between staff and consumers across a small number of studies (i.e., Burg et al., 1979; Burgio et al., 1983; Baldwin & Hattersley, 1984; Doerner et al., 1989; Kamana et al., 2021; Montegar et al., 1977; Mowery et al., 2010; Zoder-Martel et al., 2014). For example, Kamana et al. (2021) evaluated the effects of behavioral skills training (i.e., instructions, model, rehearsal, and feedback) and on-the-job feedback on several recommended practices for staff behavior, including positive interactions, in group homes. Results showed staff–consumer positive interactions increased and were more stable during intervention compared to baseline. In another study, Zoder-Martell et al. (2014) evaluated the use of real-time prompts delivered via an in-ear headphone on the rate of positive verbal interactions initiated by participating direct support professionals. During baseline, participants were uninformed of the dependent variable and did not receive training. Upon implementation of direct training using prompts, all participants increased their positive interactions with consumers, which were maintained for at least five sessions. Results maintained during follow-up probes conducted 2 and 6 weeks after maintenance. These findings suggest common staff training procedures can increase and maintain positive staff–consumer interactions. Although promising, both studies required significant supervisory resources to achieve desired outcomes. Thus, additional research should evaluate interventions that are resource-sensitive and do not require the presence of a supervisor.

One intervention worthy of investigation is self-monitoring, which has the potential to reduce resources and mitigate supervisor presence. Self-monitoring is a procedure where an individual records the occurrence or nonoccurrence of their own behavior (Olson & Winchester, 2008). Self-monitoring has been implemented across various workplace settings (e.g., Rinaldi-Miles et al., 2019; Rodriguez et al., 2006; Rose & Ludwig, 2009) and with a range of treatment packages that include prompts (Petscher & Bailey, 2006), feedback (Normand, 2008), and goal setting (Calpin et al., 1988). Workplace applications of self-monitoring suggest it is useful for schedule adherence (Richman et al., 1988), safety (Heckman & Geller, 2003), and treatment integrity (Belfiore et al., 2008; Mouzakitis et al., 2015). Taken together, these findings suggest self-monitoring can be flexibly used to achieve desired staff performance.

Self-monitoring has been implemented with various types of employees (e.g., Hillman et al., 2021; Mouzakitis et al., 2015), including employees working in human-service settings. For example, Petscher and Bailey (2006) used a multiple baseline design across participants and behaviors to evaluate a treatment package consisting of self-monitoring, a tactile prompt, and feedback on token economy implementation by teachers working with students with disabilities. The self-monitoring package effectively increased accurate token economy implementation. In addition, video self-monitoring has been researched in human-service settings (Aherne & Beaulieu, 2018; Belfiore et al., 2008; Pelletier et al., 2010). Belfiore et al. (2008) successfully used video self-monitoring to improve the accuracy with which staff implemented discrete trial instruction with children with autism. The effects maintained for up to 4 weeks.

Researchers have targeted positive interactions between staff and consumers using intervention packages containing self-monitoring(e.g., Baldwin & Hattersely, 1984; Burg et al., 1979; Burgio et al., 1983; Doerner et al., 1989; Parsons et al., 1989). Burgio et al. (1983) evaluated staff’s use of self-monitoring, goal setting, self-evaluation, and self-reinforcement procedures on positive interactions. Others have shown improvements in positive interactions with self-monitoring in addition to supervision (Burg et al., 1979) and group meetings (Baldwin & Hattersley, 1984). Given the packaged interventions in these studies, it is unknown the extent to which self-monitoring contributed to improvements. Evaluating the effects of self-monitoring in isolation and incorporating additional performance management interventions, such as feedback, only if necessary, addresses this limitation and has the potential to aid organizational leaders in adopting resource-efficient procedures.

Researchers have shown self-monitoring’s effectiveness using a range of modalities such as golf clickers (Doerner et al., 1989), wrist-worn response counters (Burgio et al., 1983), clipboards with removeable stickers (Burg et al., 1979; Baldwin & Hattersley, 1984), paper and pen (Plavnick et al., 2010), and technology (Sigurdsson et al., 2011). Advances in technology in recent decades necessitate additional research investigating the efficacy of technology-based self-monitoring interventions on the behavior of staff working in group homes. For example, many organizations have made the transition to electronic data recording rather than using paper data sheets (e.g., Sleeper et al., 2017). Leveraging the technology already available in organizations makes a technology-based self-monitoring intervention potentially more feasible.

Given the literature documenting the beneficial outcomes associated with staff positively interacting with consumers living in group homes (e.g., Burg et al., 1979; Burgio et al., 1983) and repeatedly observed low levels of interactions (e.g., Chan & Yau, 2002), additional research evaluating ways to foster positive interactions is worthwhile. Studies measuring positive interactions commonly evaluate self-monitoring as a packaged performance management intervention. However, packaged interventions are not always ideal for human-service settings given the need to develop effective interventions with as few resources as possible (President’s Committee for People with Intellectual Disabilities, 2017). Moreover, group home self-monitoring research has not included advances in technology as other aspects of the job of group home staff have evolved over time (e.g., electronic data collection). Therefore, the purpose of the current study is to evaluate a technology-based self-monitoring intervention on positive staff–consumer interactions in group homes.

Method

Participants and Setting

Approval from the university’s Institutional Review Board and the organization’s Human Rights Committee was obtained before starting the experiment. Participants included three staff who worked in different group homes for adults with IDD. Sierra was a 49-year-old woman with an associate’s degree who worked at the organization as a home supervisor for almost 1 year. Billie was a 27-year-old woman with some college education who worked at the organization as a direct support professional for almost 2 years. Carson was a 23-year-old man with some college education who worked at the organization as a direct support professional for almost 1 year. The setting consisted of three different three-bed, one-bath townhomes with a kitchen, dining room, living room, hallway, and fenced-in outdoor area. Homes had built-in cameras with continuous video and audio recording capabilities in the public areas of the homes. Recordings could be viewed at the organization’s office suites.

Upon hire, participants experienced a 5-day preservice workshop where they were informed of job responsibilities, including the organization’s expectations for how to implement various practices to prevent problem behavior and enhance the quality-of-life of consumers. An hour-long presentation focused on describing and modeling several research-supported practices to help participants meet this organization expectation. One of these practices included initiating positive interactions with consumers. In this training, participants received information about the rationale for positive interactions (i.e., positive interactions promote healthy relationships, decrease problem behavior, and increase appropriate behavior) and that they should interact at least once every 5 min per consumer. Although the organization communicated this standard to its employees, its leaders recognized that service-delivery responsibilities may compete with this standard and were satisfied if employee interacted with consumers at least once approximately every 5 min per consumer.

Supervisors were asked to nominate staff who could benefit from participating in a project that targeted positive interactions between staff and consumers. The first author contacted nominated staff through text message on their work phone and provided a brief explanation of the study. Staff who tentatively agreed to participate in the study met with the experimenter to sign the consent form and review job responsibilities pertaining to interacting with consumers. Participants were informed that observations would begin on the day of consent, would occur during evening leisure time, and would be conducted remotely via the cameras in the home. Because observations required participants to be in the home and consumer and staff schedules varied, observations depended on availability and participant assent1 prior to each session. At most, one observation (i.e., session) was possible each day participants worked. Upon completion of the study, participants were paid $50.00 via a prepaid debit card.

Materials

Participants received a tablet (i.e., Samsung Tab A6), charger, and laminated written instructions. The Samsung Tab A6 had two preinstalled applications, Countee^© (Peić & Hernández, 2021) and an interval timer. Laminated instructions included the tablet password, the experimenter’s contact information, examples of positive interactions, and a task analysis for using the applications.

Data Collection and Response Measurement

All observations were recorded via in-home cameras and microphones and were saved to a secure network hosted by the group-home organization. Observations occurred during evening leisure activities and were not conducted during meals or medication administration lasting longer than 10 min.

Primary Dependent Variable

The primary dependent variable was the percentage of 5-min intervals with a positive interaction. We used 5-min partial interval recording during each 30-min session. No more than one session was scheduled per day. Organizational leaders desired employees to engage in positive interactions with consumers approximately once every 5 min; thus, the measurement system adopted in this experiment reflected that expectation. Definitions for positive interactions were identical to Kamana (2019) and are shown in Table 1. A positive interaction included several behaviors such as a compliment, conversation, greeting, appropriate physical interaction, expression of care, or praise. A positive interaction did not include participants teaching individualized programs, implementing procedures from a behavior plan, or giving redirection statements. If a participant was absent for three or more 1-min intervals within a 5-min interval, the data from that interval were excluded for that session. The percentage of 5-min intervals with a positive interaction was calculated by dividing the total number of intervals with a positive interaction by the total number of intervals, multiplied by 100.

Table 1.

Definition and Examples of Positive Interactions (Kamana, 2019)

Interaction	Definition	Example
Compliment	Saying something favorable about the consumer to the consumer.	“You look nice today!”
Conversation	Talking about topics that consumers may prefer or commenting on an activity in which consumers were engaged.	“Did you enjoy the movie?”
Greet consumer	A salutation to the consumer.	“Hi, great to see you!”
Appropriate physical interaction	Making physical contact that is appropriate for adults.	High fives or a pat on the back
Expression of care	Acknowledging when consumers appeared sad, tired, upset, or need help.	“It looks like you are sad. Is everything okay?”
Praise	Acknowledging appropriate consumer behavior.	“Excellent job (specify behavior).”

Secondary Dependent Variables

We also measured two secondary dependent variables. First, we determined whether participants showed scalloped patterns of interacting with consumers by analyzing how they allocated their interactions throughout the 5-min intervals. To conduct this analysis, we summed the total number of 1-min intervals with an interaction2 for each minute of every 5-min interval (i.e., 1st, 2nd, 3rd, 4th, and 5th min) for the entire 30-min session. Thus, the maximum possible value is six (30-min session /5-min intervals = 6 opportunities for each 1-min interval to be scored) for each minute.

Second, we measured participant recording accuracy to determine if participants correctly recorded their behavior. To determine accuracy, experimenter observation data were compared to participant output data from the Countee^© app. An accurate response occurred when the experimenter and participant recorded the same response within the last 30 s (+15 s) of each 5-min interval. A +15 s buffer period was included because participants were instructed to do their best recording interactions every 5 min, but that other household and clinical needs may take priority temporarily. Thus, this brief window recognized that participants might not record at the exact end of the interval. An error of omission occurred when the experimenter recorded that a positive interaction occurred, but the participant recorded that a positive interaction did not occur or did not record any interaction. An error of commission occurred when the experimenter recorded a positive interaction did not occur, but the participant recorded a positive interaction occurred. The percentage of accurate responses was calculated by dividing the total number of accurate responses by the total number of 5-min intervals, multiplied by 100. The percentage of omission errors was calculated by dividing the total number of omission errors by the total number of 5-min intervals, multiplied by 100. The percentage of commission errors was calculated by dividing the total number of commission errors by the total number of 5-min intervals, multiplied by 100.

Interobserver Agreement (IOA)

A second observer recorded data on participants’ positive interactions for a minimum of 30% of sessions for each phase and participant to calculate interobserver agreement. For an agreement to occur, both observers must have recorded that an interaction did or did not occur in each 1-min interval.3 Agreement was calculated using the interval-by-interval method by dividing the number of intervals with agreement by the total number of intervals, multiplied by 100. Mean agreement for Sierra was 91.9% (range: 86.7%–100%), for Billie was 89.6% (range: 80%–100%), and for Carson was 89.6% (range: 80%–96.7%).

Procedure

An ABAB design was used to evaluate the effects of self-monitoring on positive staff–consumer interactions for Sierra. Although we had intended to withdraw the intervention for Billie and Carson, participant attrition prevented that analysis. As a result, we used a nonconcurrent multiple baseline design across participants for Billie and Carson to show experimental control.

Baseline and Withdrawal

During baseline and withdrawal, participants performed their job responsibilities as they typically would. When they provided informed consent, participants were reminded that interactions with consumers should occur at least once every 5 min. They did not receive experimenter feedback about their performance or have access to the tablet. Both phases continued until participant behavior data were stable or decreasing based upon visual inspection.

Self-Monitoring

Self-monitoring training

We trained participants to self-monitor using an evidence-based training procedure, behavioral skills training (Parsons et al., 2012), including instructions, modeling, rehearsal, and feedback. Training was conducted in one session lasting no more than 40 min. Participants received both written and vocal instructions for a positive interaction, which included exemplars and nonexemplars. Participants also received written and vocal instructions on how to use the tablet, start the applications, and send the data to the researcher via encrypted email. Instructions included information about the procedures. In particular, participants were instructed to start the applications, and self-monitor their interactions with consumers as described below. To prompt participants to self-monitor, an interval timer was used. The interval timer beeped twice: after 4 min, 30 sec of each 5-min interval and at the end of the 5-min interval. Participants were instructed to record the occurrence or nonoccurrence of an interaction during the last 30 sec of the 5-min interval (i.e., between the two beeps). After providing instructions, a model was provided on how to use the tablet, start the applications, and send the data. Participants then had the opportunity to imitate the model and ask questions. Participants received corrective feedback (e.g., “Remember to press start on the interval timer.”) and/or praise (e.g., “Great job double-checking the session name.”) until they independently set up a self-monitoring session on the tablet. Once participants started a session independently, they rehearsed two 5-min intervals and recorded the occurrence or nonoccurrence of positive interactions. The experimenter provided corrective feedback if participants made an error and praise for accurate self-monitoring.

During training, researchers instructed participants to start a self-monitoring session by opening both the interval timer and Countee^© apps. Participants used Countee^© to record their interactions (i.e., to self-monitor). Participants started a new session by clicking a button labeled “start new session” and labeled the session with the date. They would then switch to the interval timer app and press the play button to begin the 30-min timer. The interval timer had three audible beeps that sounded before the interval began. During the three audible beeps, participants would return to Countee^© and press the “start” button. After 30 min, the interval timer stopped and Countee^© closed. Data were automatically saved within the Countee^© app. Participants then sent their data to the experimenter through encrypted email by pressing the “share data” icon (i.e., airplane icon in the top right corner of the Countee^© app), selecting that day’s data set, typing in the correct email address, and pressing send.

Self-monitoring

The purpose of this phase was to evaluate the effects of self-monitoring on the percentage of 5-min intervals participants engaged in positive interactions with consumers. Once training was complete and the self-monitoring phase began, participants were contacted each workday morning via text message to confirm participants were available to self-monitor. Approximately 5 min before the start of each session, a text message was sent informing participants they could start the session. Participants used both the interval timer and Countee^© apps to self-monitor. During this phase, participants did not receive performance feedback. This phase continued until participant data maintained at or above a criterion of 80% or higher across several sessions, or participant data were stable or decreasing upon visual inspection (in the latter situation, we introduced the next phase).

Self-Monitoring and Feedback

The purpose of this phase was to evaluate the effects of both self-monitoring and feedback on the percentage of intervals participants positively interacted with consumers. We introduced this phase if participants did not meet criterion with self-monitoring alone based on visual inspection. Experimenter feedback included a text message that contained the participant’s percentage of intervals with an interaction for the previous session. In addition, if participants reached or surpassed 80%, the text message contained a praise statement. If participants did not reach 80%, they were reminded of the goal and were asked if they needed additional information or help. For example, participants would receive a text message containing the following information if they surpassed the 80% goal: “Good morning, are you able to self-monitor this afternoon? In your last session, you interacted with consumers in six out of six intervals, so 100%. Fantastic job interacting with consumers!” If participants did not meet the 80% goal, they received a text message containing the following information: “Good morning, are you able to self-monitor this afternoon? In your last session, you interacted with consumers in four out of six intervals, so 67%. Remember it would be great to have a positive interaction in at least 80% of intervals. Please let me know if you have further questions or need any help.”

Participants did not ask follow-up questions or request additional information. Feedback on their last performance was delivered before the next scheduled session, which occurred one day to one week following the previous session. This phase continued until participant data maintained at or above 80% for three consecutive sessions.

Training Integrity

An independent observer reviewed 100% of the self-monitoring training sessions at the organization’s office suite and collected data on experimenter training integrity. To measure training integrity, the observer completed a checklist consisting of training activities that included instructions, modeling, rehearsal, and feedback for each self-monitoring step. Training integrity was calculated by dividing the number of correctly implemented steps by the total number of steps, multiplied by 100. Training integrity ranged from 96% to 100%.

Social Validity

Upon completion of the study, participants had the opportunity to complete an anonymous social validity questionnaire that could be returned via the postal service. The questionnaire was a modified version of the Intervention Rating Profile-15(Martens et al., 1985) that consisted of 11 items asking participants to rate the acceptability of the self-monitoring training and self-monitoring intervention on a 6-point Likert-type scale where 1 indicated “strongly disagree” and 6 indicated “strongly agree.” Higher scores represent higher acceptability. Other questions included one yes or no question that asked if participants used self-monitoring outside of the research study and one open-ended question that asked for additional comments.

Results

Figure 1 depicts the percentage of 5-min intervals with a positive interaction for each participant (left y-axis) and when participants interacted within each 5-min interval (right y-axis). The percentages for all participants were variable, low, and decreasing during baseline. In particular, the data for Sierra (M = 40%; range: 16.7%–66.7%), Billie (M = 22.2%; range: 5%–50%), and Carson (M = 38.6%; range: 0%–100%) were below the criterion of 80% (i.e., the organization’s expectation of approximately one interaction every 5 min).

Fig. 1.

Time-Series Data of 5-min Intervals and Heat Map of 1-min Intervals

Upon the introduction of self-monitoring, Sierra’s behavior immediately increased and maintained above the 80% criterion for 8 of 10 consecutive sessions (M = 84.5%; range: 50%–100%). When self-monitoring was discontinued, the percentage of intervals in which Sierra engaged in a positive interaction decreased to baseline levels by the end of this phase (M = 60.7%; range: 16.7%–100%). High levels of responding were observed upon reintroduction of self-monitoring (M = 87.8%; range: 80%–100%).

Billie and Carson showed initial increases in their percentages at the start of self-monitoring, but performance did not maintain. Both Billie’s (M = 67.3%; range: 33.3%–100%) and Carson’s (M = 65%; range: 25%–100%) performance was well below the organization’s expectation. With the addition of feedback, performance increased to criterion levels (Billie M = 83.3%; range: 66.7%–100%; Carson M = 95%; range: 80%–100%).

The heat map, also shown in Figure 1 (right y-axis), depicts when positive interactions occurred by minute. The purpose of this analysis was to determine how participants allocated their interactions for each minute of every 5-min interval across a 30-min session. Darker shading indicates a higher number of 1-min intervals with positive interactions. Results showed no clear scalloping pattern; that is, interactions occurred across each minute of the 5-min interval for most sessions. However, there are minutes in which interactions never occurred.

Accuracy

Accuracy varied by participant as did errors of omission and commission. Sierra’s accuracy and omission errors averaged 49% with few commission errors (1%). Billie’s accuracy averaged 21% with a high percentage of omission errors (71%) and relatively few commission errors (8%). Carson’s accuracy was higher than other participants averaging 68% with relatively few omission errors (10%) and commission errors averaging 22%. We conducted a supplemental analysis to determine if omission errors were due to self-monitoring errors or if participants failed to self-monitor altogether. The analysis revealed that participants were more likely to fail to self-monitor than to incorrectly self-monitor. That is, Sierra, Billie, and Carson failed to self-monitor in 76%, 97%, and 88% of omission errors, respectively.

Social Validity

Of the three participants, only one returned a completed questionnaire. Due to the lack of responding, we are unable to make valid conclusions about the social validity of the self-monitoring procedure.

Discussion

The purpose of the present study was to evaluate the effects of technology-based self-monitoring on positive staff–consumer interactions in group homes. Results show technology-based self-monitoring (alone or with feedback) produced increases in the percentage of intervals with positive interactions between three participating staff and consumers. These data are similar to the findings of previous studies and support the use of self-monitoring, or self-monitoring as part of a packaged intervention, to achieve and maintain desired employee performance (Baldwin & Hattersley, 1984; Burg et al., 1979; Burgio et al., 1983; Richman et al., 1988). In addition, during the most effective intervention for each participant (i.e., self-monitoring for Sierra; self-monitoring and feedback for Billie and Carson), participants tended to have more than one interaction with consumers in the 5-min interval. These results are promising as participants improved performance during intervention relative to baseline during which they had few, if any, positive interactions. Because self-monitoring was effective as a standalone intervention for only one participant, these findings offer the most support for the use of self-monitoring as part of an intervention package

Previous research has documented that human-service staff demonstrate scalloped patterns of responding for some job tasks (e.g., conducting observations; Reed et al., 2010). Although research is lacking indicating a scalloped pattern of positive interactions with consumers is problematic, ideally staff interactions would occur at various times and as natural social opportunities present themselves. In the current study, participants varied the minute in which they interacted throughout the 5-min interval. Previous self-monitoring research has not assessed temporal patterns of responding, so our data are novel and show that the current participants engaged in interactions throughout a 5-min interval rather than demonstrating a scalloped pattern of responding (i.e., waiting until the end of the interval to interact). Participants were not instructed to avoid scalloping, and these data suggest it may be unnecessary to instruct some staff to do so.

The findings from the present study also showed participants had low self-monitoring accuracy. Previous studies demonstrated socially significant improvements in target behaviors when participants implemented self-monitoring with higher accuracy (Gravina et al., 2013; Sasson & Austin, 2004). It may not always be feasible for staff to implement self-monitoring with high levels of accuracy in human-service settings given competing responsibilities, such as implementing teaching programs, behavior intervention plans, or assisting with self-care skills. Thus, determining that the intervention may produce changes in behavior despite lower self-monitoring accuracy is important. The present finding is preliminary, so additional research is necessary before firm conclusions for the applied setting can be made.

These findings contribute to the literature on increasing positive staff–consumer interactions in group homes in several ways. Although research has shown that the quality and frequency of positive interactions influence consumer outcomes (e.g., Burg et al., 1979; Burgio et al., 1983; Felce & Emerson, 2001; Manente et al., 2010), only a handful of studies have addressed how to increase positive interactions (e.g., Kamana et al., 2021; Zoder-Martel et al., 2014). The present findings contribute to this small body of research and support the beneficial effects of self-monitoring as part of a packaged intervention.

The results of this study also contribute to the technology-based self-monitoring literature. This study adopted technology to assist participants with self-monitoring, which reflects recent trends of using technology in human-service settings (LeBlanc et al., 2020). Previous self-monitoring research in group homes relied on golf clickers (Doerner et al., 1989) and clipboards with removable stickers (Burg et al., 1979; Baldwin & Hattersley, 1984), which were effective but may not reflect current data collection methods in human-service settings. Embedding the intervention into currently used technology-based methods could streamline procedures and possibly remove barriers to self-monitoring. Because incorporating technology into service-delivery settings is gaining popularity (DiGennaro Reed & Reed, 2013), evaluating technology-based interventions, such as the current intervention, is a worthwhile endeavor. Future research should continue the evaluation of technology-based interventions in service-delivery settings.

An additional contribution to the self-monitoring literature includes the evaluation of self-monitoring as a single intervention. Previous research used interventions containing two to four components and did not evaluate the effects of self-monitoring alone (e.g., Burg et al., 1979; Burgio et al., 1983; Calpin et al., 1988; Doerner et al., 1989; Mowery et al., 2010; Parsons et al., 1989). Evaluating packaged interventions makes it difficult to determine if all components are necessary without conducting a component analysis. Some components may be unnecessary to produce desired behavior change and, as a result, may waste resources. The current study evaluated the effects of self-monitoring alone and incorporated additional components (experimenter feedback delivered via text message) only if needed. This sequential approach has precedence in the literature (e.g., Erath et al., 2020; Howard & DiGennaro Reed, 2014) and was intentionally designed to ensure careful use of resources. One participant met criterion with self-monitoring alone. Two participants showed improved performance with self-monitoring but required feedback to reach criterion levels. These data suggest that, under certain circumstances and for some individuals, self-monitoring alone may yield desired outcomes. Future research should investigate these variables further to aid in the design of resource-efficient interventions.

This study also contributes to the feedback literature. Feedback is a common performance management intervention as it can be relatively inexpensive to administer and is effective (e.g., Sleiman et al., 2020). Technology-based feedback has been delivered in a variety of formats such as emails (Barton & Wolery, 2007; Brown & Woods, 2011; Hemmeter et al., 2011), telecommunication technologies (Marturana & Woods, 2012; Zhu et al., 2020), and text messages (Warrilow et al., 2020). Despite its popularity, we are unaware of any research that has evaluated text-message feedback on the performance of staff working in group homes. Feedback via text message was selected because of its potential advantages. First, if adopted by the organization, text-message feedback could be cost-effective for supervisors to implement. The organization in this study had continuous audio and video recording capabilities, so supervisors could observe and provide text-message feedback at a time that was convenient for them. This method reduces the amount of time traveling to homes, thus increasing valuable personnel time and reducing mileage reimbursement. Second, the delivery of feedback via text message could help reduce the response effort associated with giving in-person feedback or delivering feedback via video or telephone conferencing. These latter forms of feedback require both the supervisor and supervisee to be available at the same time. Third, text-message feedback did not require additional training and was quick to deliver. It is presently unknown the extent to which staff become distracted by other alerts and apps on their personal phones when receiving text-messaged feedback. One way to potentially mitigate the phone’s distraction is to deliver the feedback using a text-messaging app on a work tablet that has preapproved apps downloaded.

A final contribution to the feedback literature is the finding that feedback was effective even though it was delayed relative to when the participant performed the behavior. Some research recommends supervisors should provide feedback immediately (i.e., within 60 s) after a target behavior is performed (Goomas & Ludwig, 2007; Luke & Alavosius, 2011; Sleiman et al., 2020). However, other research suggests providing feedback immediately before a task can improve performance more than feedback delivered following a task (Aljadeff-Abergel et al., 2017; Bechtel et al., 2015). Henley and DiGennaro Reed (2015) did not find significant differences in the timing of feedback when comparing pre- and post-session feedback. In the current study, we provided feedback 1 day to 1 week following a session but several hours before the next session. Although we did not systematically compare pre- and post-session feedback, the timing of our feedback suggests pre-session feedback can increase interactions.

The results of this study suggest technology-based self-monitoring may have utility in human-service settings, but it contains limitations to be addressed in future research. For instance, an assessment (e.g., Performance Diagnostic Checklist—Human Services; Carr et al., 2013) was not administered to assist in developing an intervention to increase interactions. Future studies addressing staff–consumer interactions may benefit from using this type of assessment to develop interventions that are unique to an organization or individual.

Due to time constraints and participant attrition, components of the intervention were not systematically withdrawn. Further evaluation of the effects of self-monitoring components, such as the beep that prompted participants to self-monitor or feedback that was provided after each session, were not evaluated. It would be interesting for future studies to evaluate the effects of fading strategies, such as the removal of the beep or intermittent feedback, on positive interactions. Future research regarding the outcome of fading could make the intervention more manageable for staff and supervisors.

Limitations regarding the intervention’s maintenance, generalization, and social validity should also be addressed. Future research should measure long-term maintenance of performance as well as generalization of the effects of the intervention to other times of the day. An important area to address is the intervention’s effectiveness when group-home supervisors or other relevant employees deliver performance feedback as opposed to researchers. The intervention’s social validity should also be a topic in future research as only one participant returned the social validity questionnaire. The lack of responses may indicate participants did not find the intervention acceptable or that self-monitoring was not easy to implement.

Additional limitations include the extent to which self-monitoring alone increases interactions and the time required to observe. Results show only one participant reached criterion using self-monitoring alone. Although Billie’s and Carson’s percentages increased when self-monitoring was implemented, their performance did not meet criterion. Thus, if an individual or organization were to implement self-monitoring as a single intervention, we recommend observing performance to ensure desired outcomes. Because observations (and possibly feedback) are necessary, adopting self-monitoring does not absolve supervisors of their responsibilities to conduct observations and deliver timely feedback.

The results of the present study provide some support for technology-based self-monitoring and self-monitoring plus feedback in group homes as a potentially resource-efficient way to increase interactions between staff and consumers. It is important for research to continue evaluating interventions that are resource-sensitive and use technology in human-service settings because resources are scarce (President’s Committee for People with Intellectual Disabilities, 2017) and consumer outcomes may be affected (Burg et al., 1979; Burgio et al., 1983). Using technology-based self-monitoring with and without feedback can be one way to embrace such organizational challenges and still deliver quality services.

Declarations

Conflict of Interest

The first author does not have a conflict of interest. The second author serves on the Board of Directors for the organization where the research was conducted. The second author did not interact directly with participants.

Ethical Approval

The study was approved both by the University IRB and the organization’s Human Rights Committee where the research was conducted.

Informed Consent

All participants provided written informed consent. Moreover, participants could assent or not assent to a research session being conducted on any given day.