Guided and Directed Cues: Developing a Standardized Coding Scheme for Clinical Practice

Abstract

Guided cues facilitate discovery of problems and strategies. Directed cues are instructional statements and commands. Both types of cues are used by therapists to promote learning; yet, little is known about the frequency and the effects of these cues in clinical practice. We developed a standardized coding scheme for characterizing training cues. We recorded 27 sessions among 10 participants engaged in inpatient rehabilitation after stroke. Two raters coded type and number of cues on 33% of sessions. We resolved discrepancies between raters and refined the standardized coding scheme, achieving excellent inter-rater reliability (ICC=.950, guided cues; ICC=.995, directed cues). We applied the final coding scheme to all 27 sessions (10 participants). Average cues per minute ranged from 1 to 13. Guided cues were less frequent than directed cues. This study sets the stage for future, larger studies designed to examine the significance of the number and types of cues in usual practice.

Individuals with cognitive impairments after stroke experience greater disability and obtain less benefit from rehabilitation than those without cognitive impairments (Heruti et al., 2002; Rabadi, Rabadi, Edelstein, & Peterson, 2008; Skidmore et al., 2010). A closer look at the training that individuals with stroke-related cognitive impairments receive in rehabilitation may be warranted to improve rehabilitation outcomes for these individuals.

The therapist-client interaction is a key component of rehabilitation training. An important part of therapist-client interaction is the delivery of cues, which may be in the form of instruction, guidance, and feedback. There is very limited evidence regarding the delivery of cues, but the available evidence comes from a body of motor learning literature that focuses specifically on cues provided during feedback. Research suggests that progressive reduction in frequency of feedback cues leads to improved retention of learning (Winstein & Schmidt 1990). It may be reasonable to conclude that the frequency of other types of cues (i.e. instruction, guidance) also play a vital role in learning. However, the types of cues applied in usual clinical practice vary greatly and little is known about the effectiveness of different types of cues for individuals with cognitive impairments (Skidmore et al., 2014; Swanson, 2001). Furthermore, the majority of what we understand about cueing has been evaluated in a controlled, experimental environment (Winstein & Schmidt 1990). Before we can examine the significance of the types and frequency of cues, we require a standardized coding scheme for assessing current clinical practice.

Cues used in inpatient rehabilitation can be classified into two observable methods of training: guided training and directed training. Guided training is comprised of cueing that is used to enable a client to discover a strategy or plan to solve a problem. The cueing can include open-ended questions and open-ended statements used to facilitate a patient's independent planning and problem solving (Swanson, 2001). Additionally, guided gestures or tactile cues may be used by the therapist during training to achieve the goal. For example, a therapist use of a sweeping gesture to cue the client to scan the full environment before starting a task. Directed training is a method where the therapist identifies and solves the problem and instructs the client how to improve performance (Swanson, 2001). Directed verbal training can be formulated as an instructional statement or a command that is used by the therapist during training as a means to elicit a specific, desired behavior. For example, a therapist may point to a specific item they want the client to attend to, or they would demonstrate how to perform a task. Evidence suggests that guided training may be better than directed training for promoting generalization and transfer of selected skills for individuals with cognitive impairments (Skidmore et al., 2014; Swanson, 2001); however, it remains unclear.

To better understand the relative merits of types of training cues, we require methods to reliably characterize the types and number of cues that are used during a given therapy session. The purpose of this report was to develop a standardized coding scheme for characterizing training cues in inpatient rehabilitation, and to assess the reliability of that coding scheme. We chose to focus on characterizing guided and directed cues in this report.

Methods

We developed a coding scheme. We then examined the reliability of using the coding scheme for assessing training in inpatient rehabilitation for 27 sessions among 10 participants. Characteristics of these participants are provided in Table 1.

Table 1. Participant Characteristics.

Sex, Male, n (%)	5 (50%)
Age, Years, M (SD)	61.9 (13.9)
Race, White, n (%)	10 (100%)
Hemisphere, Left, n (%)	5 (80%)
Stroke Type, Ischemic, n (%)	8 (80%)
Quick Executive Interview, M (SD)	11.6 (5.4)
Functional Independence Measure, M (SD)	57.5 (15.0)

In the initial development phase, we used 3 recorded videos of an occupational therapy session, a physical therapy session, and a speech language pathology session for a single participant engaged in inpatient rehabilitation after stroke (Figure 1). The first author reviewed these videos to observe therapist-client interactions. From these observations, we defined distinctions between assessment segments and training segments, indicated by start and stop times for each activity. Segmentation ensured that multiple raters examine the same segments when coding cues. It also provides the potential opportunity to examine distributions of cues across activities and sessions. Next, we focused on defining guided and directed cues during training segments. From these observations and definitions, we created a scheme to classify the type and count the number of cues during training [See Appendix for final coding scheme. The final manual with definitions and examples is available from the senior author].

Figure 1. Procedural Flow Diagram.

The first author then trained an independent evaluator in the categories and definitions. Both coders independently applied the coding scheme to three additional recorded sessions from among 2 participants (inter-rater Time 1) (Figure 1). Raters compared their findings, and discussed them with the senior author. Based on observed discrepancies and the subsequent discussion, the team clarified definitions of guided and directed training cues, developing a manual with examples for each category. The two raters then applied the revised coding scheme to 6 additional recorded sessions among 4 different participants (inter-rater Time 2).

Once inter-rater reliability was established, the first author applied the final coding scheme to 27 recorded therapy sessions (10 participants) to characterize the type and number of cues (Figure 1). These 27 inpatient rehabilitation sessions represented one occupational therapy session and one physical therapy session for each of 10 participants, and one speech language pathology session for 7 of the 10 participants who received speech language pathology services during inpatient rehabilitation. All participants were enrolled in inpatient rehabilitation and all demonstrated cognitive impairments after a stroke (Quick Executive Interview Scale Scores greater than or equal to 3) (Royall, Mahurin, & Gray, 1992; Larson & Heinemann, 2010). Nine occupational therapists, 8 physical therapists, and 6 speech language pathologists were recorded across the 27 sessions. In each case, the therapist and the participant were told that the video would be used to examine usual practice, and the first author remained unobtrusive during the sessions to prevent alteration of therapist or participant behavior and interactions. In all cases, therapy was delivered in a one-to-one paradigm.

Data Analysis

We examined the type and number of cues using descriptive statistics. We computed intra-class correlation coefficients to perform inter-rater reliability after the first and second inter-rater reliability exercises.

Results

The initial coding scheme identified assessment and training segments. Standardized definitions and examples were developed for each of these categories. Assessment was a defined portion of the therapy session where the therapist formally or informally gathered information from a participant to determine physical and cognitive abilities or health status. Training was defined as a therapy activity that was designed to practice a skill. Time at the start and end of each assessment and training activity was also recorded to examine the frequency of cues given during active therapy time.

Inter-rater reliability analyses at time 1 revealed intra-class correlation coefficients of 0.832 (p=0.002) for guided cues and 0.988 (p<0.001) for directed cues. Discussion among raters revealed variation in interpretations of the coding scheme and standardized definitions. This discussion was used to create consensus. At time 2, using the revised coding scheme and standardized defections, intra-class correlation coefficients were 0.950 (p<0.001) for guided cues and 0.995 (p<0.001) for directed cues.

The number of cues varied across all disciplines and ranged from 1 cue per minute to 13 cues per minute (Figure 2). Guided cues were less frequent than directed cues across participants, therapy activities, and therapeutic disciplines. Specifically, 98% of occupational therapy, 97% of physical therapy, and nearly 99% of cues for speech language pathology were a combination of directed verbal and gestural/tactile cues (Figure 3). Three percent of 2% of occupational therapy, physical therapy, and 1% of speech language pathology were guided verbal cues. Among directed cues, verbal cues were more used more frequently (75% in occupational therapy, 74% in physical therapy, and 78% in speech and language pathology). All guided cues were verbal; no guided gestures were observed.

Figure 2. Average Number of Cues Per Minute, Per Discipline by Participant.

Figure 3. Proportions of Guided and Directed Cues.

On average, coding video recordings of therapy sessions required 3.0 to 4.0 hours per 1.0 hour therapy session.

Discussion

The present study suggests that it is feasible to reliably assess training in inpatient rehabilitation using a standardized coding scheme. We identified assessment and training as distinct elements of therapy, and identified and defined guided and directed training cues. Guided and directed cues proved to be observable and measurable with excellent reliability. Initial exploration of the coding scheme with selected sessions from a small sample of rehabilitation participants suggests that there is variability in the number of cues within a given session, but overall, the majority of clinical practice sessions were comprised of directed cues, regardless of discipline. The frequency of cues provided also varied greatly across sessions and disciplines, but it is possible that the frequency of cues provided are influenced by the different types of training activities. Interestingly, no guided gesture cues were observed in any sessions, although there were direct gesture cues observed. A guided gesture may include the therapist's use of a sweeping gesture to cue the client to scan the full environment before starting a task, while the use of a directed gesture would include the therapist directing the client's attention to the item of interest by pointing. Guided gestures were not apparent in this sampling, but the reason for this is unclear.

These findings are only preliminary, and not generalizable. Nonetheless, there were two findings that bear further study. First, the volume of cues per minute in the reviewed sessions was relatively high. Given that these sessions were designed for individuals with cognitive impairments after stroke, one may expect that the number of cues provided may be greater compared to the number of cues provided to individuals without cognitive impairments. However, too many cues may impede learning (Winstein & Schmidt, 1990), particularly for individuals with cognitive impairments. In the absence of empirical evidence defining the appropriate amount of cues for a given patient population or training setting, a more direct interpretation of the significance of this finding is difficult. This underscores the importance of developing a reliable coding scheme to characterize and systematically study the effects of types and number of cues used in clinical practice and how these cues affect occupational therapy outcomes.

Second, there were variations in the number of cues used among disciplines and across sessions. The meaning of these variations is unclear. The methods we used to validate this coding scheme were insufficient to conclude that the variations are truly reflective of usual practice, and therefore we hesitate to draw conclusions from these results. Rather, we propose that comparisons of this nature (e.g., across training activities, sessions, disciplines, and or patient populations) are precisely the types of appropriate and meaningful applications of validated standardized coding scheme in future studies. Future studies seeking to examine these questions should include methods for systematic sampling that account for multiple potential confounding factors (e.g., patient functional status, therapist experience levels, timing of sessions in the duration of the therapeutic episode) that would influence interpretations of the findings.

Third, there was a high frequency of directed cues, relative to guided cues, in this sample. Preliminary evidence suggests that guided cues may be more effective than directed cues for individuals with cognitive impairments after stroke (Skidmore et al., 2014; Swanson, 2001). This may be because guided cues incorporate guided discovery, a critical ingredient in interventions designed to improve self-awareness of performance deficits and meta-cognitive problem solving (Wales, Nardi, & Stager, 1986), two critical needs that impede reduction of disability in individuals with cognitive impairments (Skidmore et al., 2010). However, further investigation is warranted. While these findings are based on a small sampling of clinical rehabilitation sessions, and are therefore not necessarily representative of usual care, these findings underscore the importance of studies examining therapist-client interactions during clinical rehabilitation. The proposed coding scheme provides a method for assessment of this interaction that would be useful to further study the effectiveness of interventions in clinical practice.

Despite the promise of the coding scheme that we developed, there were some difficulties that need to be addressed in future studies development and validation studies. Coding cues from a recorded therapy session was a time consuming process that required thorough attention to detail. The current method for reviewing the recorded videos of a full therapy session may not be an efficient or practical method for assessing the effects of cueing. Randomly sampling ten minute increments from the recorded therapy session may remedy this challenge. Furthermore, the use of selected software programs to help organize and reliably code data are available, and can be customized to coding schemes, such as the one we developed in this study.

The video recordings were not representative of variations in practice across the inpatient rehabilitation length of stay, but instead reflected only a single therapy session per discipline for each participant. Future studies seeking to validate this tool in representative samples should randomly select multiple sessions at various time points during inpatient rehabilitation providing a more comprehensive validation of the tool. Furthermore, variations in practice across inpatient rehabilitation facilities have not yet been explored. Additional validation incorporating multiple sites is needed before we can ensure adequate validity and reliability of the tool.

Conclusion

The impact of therapeutic cues on outcomes is considered important, but very few studies have actually examined cues in clinical settings. This is largely due to a lack of standardized methods for characterizing the type and number of cues used in clinical practice. We developed a standardized coding scheme for reliably assessing the number and types of cues used during training in inpatient rehabilitation. Additional validation is required with more efficient procedures in larger samples, at various points in the inpatient rehabilitation length of stay, and in multiple locations. We anticipate that a valid and reliable standardized coding scheme will be very useful in future studies examining the significance of the type and number of cues used in clinical practice.