Use of the Quasi-experimental Sequential Cohort Design in the Study of Patient-Nurse Effectiveness with Assisted Communication Strategies (SPEACS)

Introduction

There are many challenges in designing and conducting psychosocial-behavioral interventions in the dynamic real-life clinical setting of the intensive care unit (ICU). Decisions must be made about whether to target the intervention toward the organizational unit (i.e., hospital, ICU) or individuals within the unit (i.e., clinician and/or patient). Researchers must also consider the separation of intervention and control groups to prevent diffusion or contamination of the intervention, selection of outcome measures that are sensitive to the intervention yet clinically meaningful, and methods for minimizing threats to validity. Objective measurement of the behavioral impact of an intervention may require evaluation of some observational record, such as audio or videorecordings. Observational measurement is particularly relevant in the study of interventions to impact clinical communications.

NIH has recently invited work on designs to improve causal inference from non-experimental and quasi-experimental research (PA-05-090) (http://grants1.nih.gov/grants/guide/pa-files/PA-05-090.html - accessed 2-14-07). Although considered optimal for determining whether an intervention has an important or significant effect, randomized control trial (RCT) designs may not be feasible, may be unnecessary, and may even be unethical for the study of many clinical interventions and care problems. The risk of contamination or crossover between treatments is particularly troublesome in clinical health services settings [1, 2]. A comparative review of behavioral health studies showed that non-randomized controlled studies of high quality may generate outcomes equivalent to those found in RCTs suggesting that “trial quality may have a greater impact on treatment effect size than randomization alone” [3]. The purpose of this paper is to describe a quasi-experimental three-phase, sequential cohort research design employed in SPEACS, a current clinical trial testing two interventions to improve nurse-patient communication in the ICU. The practical and scientific considerations in constructing, and implementing a nurse-patient communication study and intervention trial in the ICU clinical setting are also discussed.

Background and Study Purpose

Many patients in the ICU are unable to speak because of artificial airways required to facilitate mechanical ventilation (i.e., artificial respiration) [4]. Additional communication barriers include fatigue, delirium, or neurologic disease. Patients report anxiety, panic, frustration and distress with the inability to communicate during mechanical ventilation in the ICU [5–7]. The literature suggests that provision of assistive and alternative communication (AAC) strategies as well as trained communication partners might improve patients’ daily interactions and, by extension, clinical outcomes [7–16]. Critical care nurses, however, typically receive little or no training in the interpretation of nonvocal communication or in the assessment and application of AAC techniques [17, 18]. Although the problems associated with the inability to speak during critical illness have been clearly established in the literature, few solutions have been offered or systematically tested with ICU patients [4].

A Mexican study showed significantly increased positive nurse behaviors and decreased negative nurse behaviors after implementation of an intensive 8-week nurse training program that emphasized attention, touch, and empathic skills. [19] The researchers used ratings of video recorded nurse-patient interactions to measure target behaviors. [19] The study did not report an equal number of observations per participant; thus, the extent to which multiple observations of nurses with high or low ratings may have influenced the group outcomes is unknown. Nevertheless, the study demonstrated that nurse training can affect communication behaviors and provided justification for behavioral measurement using video recordings in the ICU.

To date there have been no large scale, prospective investigations of the impact of systematically implemented assistive communication interventions with nonspeaking ICU patients and their nurse caregivers using control group comparison. The SPEACS study measures nurse-patient communication performance with nonspeaking ICU patients over multiple equal-length time points. The purpose of our clinical trial is to measure the impact of two experimental interventions on the communication performance between nurses and nonspeaking ICU patients when compared with a usual care (control) group. The interventions are: (1) basic communication skills training (BCST) for nurses and (2) BCST + AAC techniques and education for nurses with individualized speech language pathologist consultation (BCST + AAC-SLP). The study was approved by the University Institutional Review Board.

Research Questions

The following Research Questions (RQ) were proposed:

RQ #1 Does nurse participation in basic communication skills training (BCST) improve communication performance (ease, quality, frequency, successfulness of communications) between nurses and temporarily nonspeaking ICU patients when compared to a nonintervention (control) cohort after controlling for level of consciousness, physical restraint use, and severity of illness?

RQ#2 Does nurse participation in AAC training and individualized intervention consultation (AAC-SLP) improve communication performance (ease, quality, frequency, successfulness of communications) between nurses and temporarily nonspeaking ICU patients when compared with nonintervention (control) and BCST cohorts after controlling for level of consciousness, physical restraint use, and severity of illness?

RQ #3 What are the differential effects of the BCST versus AAC-SLP intervention on improve communication performance (ease, quality, frequency, successfulness of communications) in nurse-patient communication interactions?

RQ #4 What are nurses’ perceptions of the ease and success of communication interactions with nonspeaking ICU patients at baseline (control), and after participating in BCST and AAC-SLP interventions?

Methods

Design

In this quasi-experimental sequential cohort design, 10 nurses are randomly selected to participate in each of 3 nonconcurrent phases: control (usual care), BCST, AAC-SLP (See Table 1). Three nonspeaking ICU patients are assigned to each study nurse forming 30 nurse-patient dyads in each phase (total n = 90 dyads). Conditions are implemented in sequential order (control, BCST, AAC-SLP) to prevent contamination from other intervention conditions and to systematically investigate the effect of the intervention components [20, 21]. This design has been used in previous intervention studies in critical care and long-term care settings [22–24].

Table 1.

Nonconcurrent Cohort Design with Repeated Measures

Phase 1	Phase 2	Phase 3
Control	BCST	AAC-SLP
T1 T2 T3 T4	X1T1 T2 T3 T4	X2T1 T2 T3 T4
10 RNs*	10 RNs*	10 RNs*
30 patients**	30 patients**	30 patients**

Legend: X = an intervention condition; BCST = basic communication skills training for nurse participants. AAC-SLP = enhanced communication training for nurses, electronic AAC device options + SLP individualized assessment and consultation.

T = data collection time points: T1 = morning day 1; T2 = afternoon day 1; T3 = morning day 2, T4 = afternoon day 2.

^*RNs = 5 MICU + 5 CT-ICU in each cohort or phase

^**patients = 15 MICU/15 CT-ICU or 3 patients per study nurse in each cohort or phase

The use of nonconcurrent groups is functional because it is reasonable to assume that the first cohort will differ in only minor ways from its two contiguous cohorts [21]. That is, the demographic characteristics (severity of illness, age, gender, level of consciousness, physical restraint) of ICU patients are expected to remain relatively stable. Characteristics of the nurse participant pool are also predicted to remain similar across a three-year period. The assumption of quasi-comparability between treatment and nontreatment groups is the essential feature required to draw casual inferences in cohort designs [21]. In the SPEACS study, long (12-month) accrual periods for each cohort, control any seasonal effects in admission of ICU patients. Differential selection, a common threat to validity, will be tested post hoc to identify whether the three patient and nurse cohorts differ in any systematic way on variables that might be related to intervention outcomes (e.g., level of nurse experience, patients’ level of consciousness, age, severity of illness).

The use of two different ICUs (medical ICU and cardiothoracic ICU) is intended to dampen any historical effects experienced within a single unit. Field notes and intervention logs recorded by researchers are used to identify diffusion of the intervention or other social system factors that may have influenced or altered nurse-patient communication over time.

Setting

The study is conducted in the 32-bed medical intensive care unit (MICU) and 22-bed cardiovascular-thoracic intensive care unit (CT-ICU) of a large academic medical center. All rooms in the study are private.

Sample

Patient participants

Patient participants meet the following entry criteria: (1) age ≥ 18 years; (2) are nonspeaking due to oral endotracheal tube or tracheostomy; (3) likely to remain intubated for 48 hours after study enrollment based on clinician judgment; (4) able to understand English; (5) score 13 or above on Glasgow Coma Scale [25]. Individuals who are reported by family to have a diagnosed hearing, speech or language disability that significantly interfered with communication prior to hospitalization are excluded from this study. This is confirmed by a score of ≤ 3 on the National Outcomes Measurement Survey (NOMS), a speech-language clinical evaluation tool [26]. Patients who meet study eligibility criteria are selected based on availability of a study nurse as determined from the daily staffing schedule. When more than one eligible patient are identified, selection is determined by (1) patient’s decisional capacity for research participation to understand and complete consenting process with family present if possible; (2) if neither patient is capable of making decision, priority is given to family who visits first. To maintain independence of the cohorts, previously enrolled subjects who are readmitted to the ICU are excluded from the subject pool for subsequent cohorts.

Nurse participants

Ten nurses are randomly selected from a sampling pool prior to the beginning of each treatment phase. All staff RNs who meet the following inclusion criteria are included in the sampling pool: (1) At least 1 year of critical care nursing experience (2) permanent staff in MICU or CT-ICU working at least 2 consecutive week-day shifts on a regular basis, (3) English-speaking. Nurses with a diagnosed hearing or speech impairment are excluded. To maintain independence in the sample, a nurse can not return to the selection pool after participation. If a selected nurse is unable or refuses to participate, he/she is deleted from the list and replaced with the nurse whose name appeared next. Those who withdraw during the course of the study are replaced with an RN of similar educational and work experience.

Sample size

The final data sample will consist of 90 ICU patients and 30 RNs equally distributed across the two ICUs and three conditions. See Table 1. Given the demands of 24-hour coverage and shift scheduling, 5 RNs per ICU (n=10 total) in each cohort was the minimum number to assure that a “study nurse” would be available for assignment to patient subjects. Our past intervention research involving the use of AAC in ICU settings suggested large clinically meaningful effect sizes of d=.80 (f=.40) from a behavioral science perspective when measuring the effect of AAC on patient assessed ease of communication [14,15]. The sample size for this study was estimated to have sufficient power (.80) to detect group by time effects as small as f=.375 when investigating the effect of the BST and AAC/SLP interventions relative to usual care (control) cohort on the longitudinally assessed, continuous-type measures of communication performance (ease, quality, frequency, successfulness of communications) (Research Questions #1, #2, and #3), while taking into account the average patient cluster size and the anticipated degree of clustering with a maximum intra-cluster correlation of about 0.1. A maximum intra-cluster correlation of 0.10 is anticipated based on the intra-cluster correlations observed in hospital-based patient outcomes research. Additionally, intra-class correlations on the order of 0.1 are often reported in educational research where clustering effects are expected [27]Although the BCST and AAC/SLP interventions are anticipated to be beneficial, negative effects may be possible. Hence, two-sided hypothesis testing at a significance level of .05 will be used [28] when comparing the intervention strategies to usual care.

Interventions

The Basic Communication Skills Training (BCST) program, delivered in Phase 2, is a 4-hour interactive educational session delivered primarily by a speech-language pathologist specializing in AAC (KG). Program content emphasizes assessment of the ICU patient’s cognitive and motor function to communicate, basic interactive communication strategies, the use of “low tech” communication strategies, and the provision of materials (e.g., alphabet and picture communication boards, writing tools, etc) accessible to ICU staff in a designated “communication cart.” In Phase 3, nurses receive the original 4-hour BCST educational program and an additional 2-hour educational session on the use of electronic AAC devices in the medical setting delivered by specially-trained SLPs. When a patient is enrolled in the study, the SLP provides individual assessment and develops and posts a communication plan for each study patient in consultation with the nurse. This communication plan may include nurse interaction tips, instructions to use low tech communication strategies, or placement of electronic speech generating devices that are matched to individual patient’s abilities, needs, and preference.

Procedures and Data Collection

Each weekday, the research coordinator or designee reviews the nurse-patient assignment with the charge nurse to assign a study nurse to the daytime (7a-3p or 7a-7p) care of a study patient for a 2-day study period. Following procurement of written informed consent and study nurse assignment, patient demographic data are obtained directly from the patient or decisional surrogate. Table 2 displays data collection elements and timing. Data consist of demographics, surveys and clinical measures, videotaped observation of nurse-patient communication, open-ended de-briefing questions, and interventionist notes and logs.

Table 2.

Summary and Description of Data Collection and Measurement

Concept/Variable	Measure	Time Points
Dependent		SE	NT	T1	T2	T3	T4	EP
Ease	Ease of Communication Patient Self-Rating	X			Xa		Xa
	Observer Rating			X	X	X	X
Quality	Observer Rating			X	X	X	X
Frequency	Observer Rating			X	X	X	X
Successfulness	Observer Rating			X	X	X	X
Potential Covariates
Level of Consciousness	Confusion Assessment Method [38]	X		X	X	X	X
	Richmond Agitation Sedation Scale[39]	X		X	X	X	X
Severity of Illness	Acute Physiology and Chronic Health Evaluation III score[40]	X			Xa		Xa
Premorbid Speech-Language Function	National Outcomes Measurement System (NOMS)[23]	X
Current Communication Function	Lowenstein Communication Scale (LCS)[41]	X		X		X
Motor Function	Mobility Subscale of LCS	X
Nurse Contact Time	Computerized Patient Record							X
Time Elapsed since Training	Nurse Demographic Data Sheet			Xb	Xb	Xb	Xb
Descriptive Data
AAC-Related Skills	Lowenstein Communication Scale (LCS)	X			Xa		Xa
Patient Characteristics	Patient Demographic Data Sheet	X
Physical Restraint	Observation Record	X		X	X	X	X
Nurse Characteristics	Nurse Demographic Data Sheet	X	Xb					X
Perceptions of Communication	Nurse Communication Survey	X	Xb					X

Key:

X_a = Daily

X_b = Phase II & III Cohorts only

SE = Study Enrollment

NT = Nurse Training

EP = End of study participation

T₁ = Day one, morning

T₂ = Day one, afternoon

T₃ = Day two, morning

T₄ = Day two, afternoon

Communication Observation

Data on communication performance between the nurse and patient are obtained by videotaping nurse-patient communication twice daily over two consecutive days for a total of four, videotape sessions per nurse-patient dyad. Trained data collectors collect the observational data two times per day (morning and afternoon/evening) across a designated two-day study period while the nurse is assigned to the care of the nonspeaking ICU patient enrolled in the study. A minimum of 3 minutes of nurse-patient interaction are recorded, either continuously or in additive increments, to ensure adequate and equal opportunities for communication interaction. This time unit was selected because the literature suggests that typical nurse-patient interactions in the ICU are 1–5 minutes in length and three minutes was determined, after viewing videotapes from prior research of gestural communication with nonspeaking ICU patients, to be a functional time frame for observation of sending, receiving, confirmation, and follow-up of messages [17, 29–32].

Two data collectors are positioned outside of the target participants’ ICU rooms during designated data collection time periods. When a call light is activated and/or a nurse participant enters the patient’s room, the observer(s) immediately follow and stand in an unobtrusive location in the room. They begin recording the nurse-patient interaction using a small handheld digital video recorder (data collector 1) and semi-structured observational field note (data collector 2). Videography permits the researchers to quantify the ease, quality, frequency and successfulness of communication exchanges in an efficient, unobtrusive manner. Ethical considerations, privacy protections, and storage methods for this study were discussed in a previous publication [33]. To minimize the “Hawthorne effect,” one “practice” or sham session is videotaped for each dyad. To supplement observational recordings, the field notes document salient events pertaining to the setting, patient, nurse, hospital environment or routine, reliability and availability of AAC equipment, interruptions, etc.

Measurement

Outcomes

Communication performance between the nurse-patient dyad is measured by rating each of the four, three-minute videotaped communication interactions per nurse-patient dyad for performance indicators of success, quality, frequency, and ease. Trained coders first transcribe each communication act within a communication exchange. An exchange is composed of an initiating communication act and subsequent acts required to convey a single idea. Communication acts are coded for initiation, function, and specific communication strategy use. Ratings are then applied for success, quality, and ease. This rating system represents a revision of Garrett’s [34] and Garrett and Huth’s [35] communication performance coding systems for use in the ICU setting and is consistent with other coding systems applied to nurse-patient interaction in trauma and ICU settings [19, 36]. Coders maintain inter-rater reliability above 0.80 [19, 36–38].

Ease of communication was measured in two ways. First, patients completed a simplified single item self-rating (on a scale of 1 to 5) after each observation session to report their communication difficulty during the nurse-patient interaction. A second, objective rating of communication ease is conducted by trained coders using communication breakdowns and communication complexity as defining criteria applied to each 3-minute video session.

Successfulness. Coders apply a categorical rating for success to each communication exchange judging how much of the intended message had been understood by the recipient ((1) no communication response; (2) message attempted but not conveyed or abandoned; (3) message partially conveyed; (4) message conveyed with adequate partner response indicating comprehension; and (5) message conveyed with elaborated partner response). The number of exchanges for each of the success rating categories are summed and divided by the total number of exchanges to obtain average proportion of success ratings across sessions and phases.

Frequency is measured by the number of communication exchanges per session. Other indicators of frequency will include the ratio of patient to nurse initiated exchanges, the number of acts in each exchange, and the number of exchanges per topic in each 3- minute nurse-patient interaction session.

Quality is measured by composites of 10 positive nurse communication behaviors, 10 negative communication behaviors, and 6 behaviors facilitating AAC use. These communication behaviors may occur more than once per session or not at all. Proportions of these positive and negative communication behaviors within the total number of nurse communication acts are calculated for each 3-minute session.

Covariates

Severity of Illness is measured using the Acute Physiology Age and Chronic Health Evaluation (APACHE) III score, a valid and reliable measure of illness severity and mortality prediction in the ICU [39, 40]. APACHE III values are obtained by review of the electronic medical record. Scores are independently calculated by two trained research assistants to ensure accuracy.

Level of Consciousness, comprised of level of alertness, mentation, and potential for meaningful communicative interaction, is assessed with two instruments:

1. Confusion Assessment Method-ICU (CAM-ICU) Designed for use with non-speaking, mechanically ventilated ICU patients, the CAM-ICU [41] is an adaptation of the CAM, the most widely used instrument for diagnosing delirium by non-psychiatric clinicians [42]. The instrument is easily administered and shows high inter-rater reliability (kappa=0.79–96). When compared with a reference standard (psychiatrist) diagnosis of delirium, the CAM-ICU had sensitivities of 93–100% and specificities of 89–100%.[41, 43]

2. Richmond Agitation and Sedation Scale (RASS), is a 10-point scale for assessing mental status with excellent inter-rater reliability and validity among a wide range of critically ill patients. Patients are assigned a single score ranging from +4 (“Combative”) to -5 (“Unarousable”), with 0 indicating “Alert and calm.” [44, 45]

Premorbid Speech-Language Function, the patient’s functionality in verbal and non-verbal communication prior to the current hospitalization, is measured with the American Speech-Language and Hearing Association’s National Outcome Measurement System (NOMS) [26]. Seven subscales assess the production and comprehension of spoken, non-spoken, and written language; cognitive communication, and hearing sensitivity. Individuals are assigned a single score for each subscale, based on a 7-point interval where higher scores reflect greater functionality. Ratings are recorded by the data collector during interview with family members or significant others during study enrollment.

Current Communication Function is assessed using the Loewenstein Communication Scale (LCS) [46]. The LCS measures 5 behavioral functions (Mobility, Respiration, Visual Responsiveness, Auditory Comprehension, and Linguistic Skills) by rating 5 items per category on a 5-point scale. Total possible LCS scores range from 0–100, with higher scores indicating greater communication ability. Inter-rater reliability for the total scale is reported as k=0.90. The LCS also demonstrated good predictive ability in distinguishing between brain injured patients in a vegetative state and those who had rehabilitation potential [46].

Contact Hours with Patient is the study nurse’s total time (hours) assigned to bedside nursing care for a study patient prior to and during the study data collection period as measured by direct count on the computerized nursing documentation record.

Time Since Training is the amount of time (days) elapsing between the nurse’s intervention training and observation of that nurse with a study patient in order to monitor the potential effect of time on communication skill retention and application.

Demographics and Clinical Descriptive Data

In addition to standard patient demographic information (e.g., age, gender, ethnicity, educational level, diagnosis/surgical procedure, intubation history, medication use, hospital and ICU admission dates), general items pertaining to communication (e.g., handedness, vision/hearing acuity, prior computer use) and hospitalization (discharge disposition, duration of intubation/mechanical ventilation (days), and lengths of ICU and hospital stay) are collected using a demographic data form standardized by the Center for Research and Evaluation at the University of Pittsburgh School of Nursing. Data collectors also document the presence or absence of physical restraint devices, which restrict free movement of the patient’s upper extremities, at each data collection time point.

Nurses’ Perceptions and Attitudes About Communication with Nonspeaking Patients are assessed with the Nurse Communication Survey (NCS), an investigator-developed survey derived from the literature [47–50]. This 36-item self-report survey contains 6 nurse demographic questions (including professional credentials/certifications and years in practice); twelve 5-point Likert-style statements (Strongly Agree - Strongly Disagree) about communication difficulty and the nurse’s own perceived ability to interpret the communication efforts of non-speaking patients. Face validity of the NCS was confirmed by members of the Society of Head and Neck Cancer Nurses experienced in caring for seriously ill, non-speaking adult patients. Psychometrics testing of the tool involved surveys completed by 139 critical care nurses [51]. Factor analysis of the Likert-scaled items revealed three factors explaining almost 50% of the total item variance. Data from the NCS will be used to describe diffusion of the intervention and any historical change in knowledge and beliefs about communicating with non-speaking ICU patients.

Data Analysis

To address the first three research questions posed, regression analysis methodologies will be used to model the target communication outcomes (ease, quality, frequency, successfulness of communications) as a function of study group (usual care control, BCST, and AAC/SLP), nurse, possibly unit, and selected fixed and time-varying covariates (i.e., severity of illness, physical restraint use, level of consciousness, AAC-related skills, the length of the nurse’s shift, and time elapsed since training). Given the design of the study, the data collected pose substantial challenges to providing valid statistical analyses including: (1) possibly non-normal distributions of the dependent variables, (2) hierarchical sampling frames, and (3) possibly incomplete or variably sampled data. To help address these analytical challenges, we plan to use hierarchical generalized linear models (HGLM) [52]. An extension of generalized linear models, HGLM also allows for the multilevel modeling of dependent variables having error structures other than normal, such as binomial and Poisson which may be encountered in this study as the observational data from the videotaped sessions are quantified. The data from this study arise from hierarchical levels of sampling, with the full sampling hierarchy consisting of four levels: unit, nurse with unit, patient with nurse, and the repeated assessments with a patient. Four outcome assessments (morning and afternoon observations on each of two consecutive days) will be made with each patient to explore the temporal stability (reliability) of communication outcome and to yield a more stable outcome estimate. Since patients and their nurses can participate in only one group, the study groups are independent. Lastly, HGLM can accommodate unbalanced designs, either by design or due to attrition or missing data, and can handle missing data that are missing completely at random or missing at random.

Each of the HGLMs that are to be estimated will include fixed, between-subjects factors for group (usual care control, BCST, and AAC/SLP) and possibly unit (MICU and CT-ICU). Nurse effects will be treated as random. Potential time-varying covariates measured at the patient level will include physical restraint use and level of consciousness, evaluated at each assessment; and severity of illness, length of the nurse’s shift, and patient communication ability, evaluated daily. The need for and the functional form of these covariates will be assessed in the initial stages of modeling. The primary outcomes of interest are the observer-rated ease, quality, frequency, successfulness, and perception of communication. F-statistics will be used to test each of the factors at a significance level of .05 (two-tailed). Linear contrasts will be formed to for the specific group comparisons of interest. Point and interval estimates (95% confidence intervals) will be obtained to summarize the effects of factors and contrasts.

Each regression model will be checked for both systematic and isolated departures from the presumed model [53–55]. Systematic departures will be diagnosed using residual-versus-fitted plots to assess for possible model misspecification (e.g., incorrect link function, an incorrect variance function, omitted covariates, or correlated error). Data points detected as isolated departures will be reviewed for their validity. If outliers are deemed valid, sensitivity analyses will be conducted to assess the influence of these observations, with particular attention given to outliers previously found during the preliminary analyses.

Finally, to address RQ#4, nurses perceptions of communication with nonspeaking ICU patients as measured by the NCS items and summary scores will be described as appropriate (i.e., frequencies and percentages for items; measns/medians, and standard deviations/ranges for subscale scores) for each group at each assessment time point.

Discussion

The quasi-experimental sequential cohort design offers particular strengths for clinical trials in settings or circumstances where randomization is not practical or feasible. Several possible designs were considered. Table 3 summarizes the advantages and disadvantages of alternative design choices. In this particular situation, an educational intervention served as the independent variable. Therefore, it was not possible to randomly assign study nurses or patients to one of the three conditions while simultaneously preventing crossover from nurses who had received training or general diffusion of the interventions into the research settings across time. Use of a sequential cohort design provided some control over this potential, and highly probable, source of experimental contamination. In addition, the research plan provides for equal measurement of each nurse and patient subject in all cohorts – a feature that is lacking in other experimental studies of nurse-patient interaction in ICU [19].

Table 3.

Clinical Research Design Advantages and Disadvantages

Design	Advantages	Disadvantages	Other considerations
Randomized control trial	Strongest causal inference	High potential for contamination/diffusion of intervention if conducted on single ICUs or in a setting where nurses rotate among different ICUs. Difficulty in achieving equality/comparability between ICUs (groups).	Could include a larger number of ICUs and/or multiple hospitals. Training a larger group of nurses (all unit staff) presents problems in intervention fidelity monitoring and potential variation in “dose”. Expensive
One Group Repeated Treatment design	Each patient is used as his/her own control. Sample size (nurses and patients) required may be smaller.	Changes can be attributed to patients’ improved conditions or acclimation over time. Patients may not remain intubated and nonspeaking long enough for all conditions to be implemented. High risk for diffusion of the intervention.
One Group Pretest-Posttest	Simple, efficient when access to experimental population is limited	Usually used to test a single intervention. Unable to examine stability of behaviors within individuals and/or group. Threats to validity include history, maturation.	Pre and post-test samples are not independent.
Sequential cohort design	Can use two or three independent groups; Intervention and nonintervention groups remain separate.	History is the main threat to validity;	Total study time and time to comparative analysis may be lengthy.

However, several potential threats to external validity limit the generalizability of study findings when using this design. History is the primary threat to validity in quasi-experimental cohort designs where some change or changes in the study population over time may affect the target outcomes. In this study, history effects are addressed in the following ways: the use of two different ICUs, careful research field notes of changes in practice or unit design that could potentially affect study outcomes, and analysis of nurse and patient characteristics with comparison between treatment groups. This monitoring prompted us to adjust the simple sampling framework for nurse selection to a stratified sampling on age/years of experience after the first cohort when we noted the potential risk for imbalance between study cohorts because of differences in nurse age and experience, particularly among those with 10 or more years of experience. Monitoring also showed that baseline cohort was truly naïve to the intervention - none had previous SLP-AAC training. One nurse in each of the intervention groups has had minimal prior exposure to AAC strategies (i.e., sign language training, AAC use in a setting with disabled children).

The Hawthorne effect is an important threat to validity that should be considered in most clinical research in which participants are aware that they are being studied and therefore potentially alter their behavior [56]. Similarly, reactivity, the concept that study participants will act differently in response to being observed, is a particular threat to external validity in studies using video cameras to record observations [57]. In the present study, the use of three independent groups may minimize the Hawthorne effect and the effect of reactivity as these effects should be experienced similarly in all three groups and not differentially impact the test of the intervention. We can assume that the study will measure nurses’ “best” communication skills in each condition. While this “best performance” may somewhat limit generalizability to the unobserved population of nurses and ICU patients, it provides a reasonable and pragmatic test of the intervention in the natural setting. Allowing time for participants to acclimate to the observer and the observational situation is an additional strategy to minimize the Hawthorne effect and the effect of reactivity in observational research. In this study, the first video-recorded observation is treated as a “sham” or practice observation to allow for participant acclimation and is not included in the database for analysis of outcomes [57]. Prolonged engagement by the researcher in the study settings both before and during the research period also helps to normalize the presence of research staff.

Although qualified patients were generally available on the study ICUs, it was more complicated to pair them with study nurses than we initially anticipated due to fluctuations in staff scheduling, changes in patient condition or discharge, and the dynamic complexities of ICU patient care needs and the assignment of nurses to meet those needs. An alternative is to apply an educational intervention to an entire unit, and then assess dyadic (nurse and patient) interactions measuring each dyad with single observations [30,31]. However, more pairs would need to be enrolled and researchers would be unable to investigate the stability of interaction within a given nurse and/or nurse-patient dyad (see Table 3).

The SPEACS study design illustrates that, with careful attention to minimizing the effects of history and reactivity, the quasi-experimental sequential cohort design provides an innovative, and rational alternative to a RCT for testing the impact of educational and behavioral interventions in the unpredictable and uncontrollable real-life clinical setting of the ICU.