Instrument Development Measuring Critical Care Nurses’ Attitudes and Behaviors with End-of-Life Care

Abstract

Background

Although critical care nurses are expected to focus on providing life-sustaining measures, many intensive care patients actually receive end-of-life care.

Objectives

To develop an instrument to measure nursing attitudes and behaviors with end-of-life care.

Method

Phase I was focused on item development from a content analysis of the literature and qualitative interviews of critical care nurses. Phase II consisted of content validity assessment and pilot testing. Phase III included field testing, factor analysis, and reliability estimation.

Results

The Values of Intensive Care Nurses for End-of-Life (INTEL-Values; n = 695) was found to have four factors: Self-appraisal, Appraisal of Others, Emotional Strain, and Moral Distress. Reliability estimates (alpha) were acceptable at .59–.78, but the interitem (.12–.78) range was wider than desirable. Test-retest reliability was deemed adequate based on Pearson’s correlations (.68–.81) and intraclass correlation coefficients (.65–.79) but less so when considering kappa (.05–.30). The Behaviors of Intensive Care Nurses for End-of-Life (INTEL-Behaviors; n = 682) was found to have two factors: Communication and Nursing Tasks. Reliability estimates were adequate when considering internal consistency (alpha .67 and .78, respectively), item total correlations (.30–.61) and test-retest as judged by Pearson’s and ICCs (.77–.81), but not when Kappa was considered (.02–.40). The interitem correlations (.20–.35) were also lower than desirable.

Discussion

Both the INTEL-Values and the INTEL-Behaviors were found to have conceptually linked factors and acceptable internal consistency estimates (alpha). However, test-retest estimates were inconsistent, suggesting further work needs to be done on the stability of these instruments.

Although the intensive care unit (ICU) is viewed typically as an intensive life saving area, 20% of all hospital deaths occur in this setting (Halcomb, Daly, Jackson, & Davidson, 2004). The ICU is not an ideal place to die as patients are often isolated from their families in this highly technical and “sterile” environment (Kirchhoff et al., 2000). Although nurses report a desire to provide a dignified end-of-life experience, they are not achieving this goal currently (Back et al., 2009; Brumley, Enguidanos, & Hillary, 2003; Daly, 2001). Inadequate pain and symptom management, miscommunication, fragmentation of care, less than desirable access to medical care, lack of adherence to medical directives, and failure to respect the values and decisions of the patient and family are cited as the most common problems in the critical care setting (Carson, Fitch, & Vachon, 2000; Ciccarello, 2003; Curtis et al., 2001; Daly, 2001; Donaldson & Field, 1998; Sulmasy & McIlvane, 2002).

Nurses are in a pivotal position to improve care for dying patients and their families by redefining the perspective of the ICU and challenging current end-of-life care practices. However, critical care nurses report a lack of preparation when dealing with end-of-life care and suggest interventions focused on improving education, nursing attitudes, and nursing behaviors as ways to improve care delivery (Kirchhoff, Beckstrand, & Anumandla, 2003; Kirchhoff et al., 2000; Mallory, 2003). Since the role of the intensive care nurse is focused on saving lives, the transition to end-of-life care can cause a division between what nurses routinely do in the ICU setting and what they are now expected to do. With increases in the numbers of patients dying in the ICU, it is important to understand the attitudes and behaviors of critical care nurses in providing this care as a basis for developing educational interventions and other resources to help them provide quality end-of-life care in the ICU.

Personal, cultural, and professional experiences with end-of-life care may influence nursing attitudes towards the dying experience. Previous end-of-life experiences have been shown to shape nurses’ expectations of the current delivery of care (Beuks et al., 2006; Chen & McMurray, 2001; Dunn, Otten, & Stephens, 2005; Heyland, Rocker, O'Callaghan, Dodek, & Cook, 2003). These experiences may impact the types of behaviors provided by the nurse.

Nurses cite their lack of training in end-of-life care and their personal commitment to saving lives as a significant cause of stress and discomfort while providing care to dying patients and their families (Beckstrand & Kirchhoff, 2005). The need to minimize pain and discomfort, promote dignity, diminish false hopes, discontinue futile treatment, and resolve communication problems with physicians contribute towards nursing attitudes regarding end-of-life care (Halcomb et al., 2004). Critical care nurses often find themselves conflicted regarding their roles and responsibilities when care transitions from critical to end-of-life care (Brown, 2003; Elpern, Covert, & Kleinpell, 2005). It was reported that about half (47%) of healthcare providers in five hospital settings stated they had acted against their own values when caring for critically ill patients (Solomon et al., 1993). Efforts to improve the delivery of care at the end of life have included various interventions, with little significant benefit (Chan & Webster, 2010; Rubenfeld & Curtis, 2001; Shaw, Clifford, Thomas, & Meehan, 2010).

Despite the emphasis on improving the care of dying patients and their families in the hospital, in general, and the ICU, specifically, no instrument could be located to measure the quality of nursing care delivered in the ICU. Instruments examining nurses' attitudes with and behaviors when providing end-of-life care in the ICU could aid in improving the dying experience in the ICU. Therefore, the aim of this study was to develop an instrument to assess nursing attitudes and behaviors when providing end-of-life care in the ICU.

Methods

This instrument development study consisted of three phases. Phase I was a content analysis of the literature and qualitative interviews of adult critical care nurses to identify the domains and subdomains needed to generate items for the instrument. In Phase II, what became two instruments were assessed for content validity and pilot tested. Phase III consisted of field testing the instruments and exploring their psychometric properties. With the exception of the content validity assessment, all relevant data collection was approved by the University’s Institutional Review Board.

Domain Identification

When developing an instrument, DeVellis (2003) and Lynn (1995) recommend the identification of the concept or domain through literature reviews and qualitative interviews. Thus, a thorough literature review on quality nursing behaviors with end-of-life care and qualitative interviews was the first step when developing the desired instrument. The results of the literature review are combined with results from the qualitative interviews to determine the domains to be measured.

The literature review included peer-reviewed studies of the role of the ICU nurse in providing end-of-life care. The Cumulative Index to Nursing and Allied Health, PubMed, and Dissertation Abstracts International databases were searched for relevant publications between 1996 and 2006. Search terms included palliative care, end-of-life care, intensive care unit, nursing, barriers, values, attitudes, behaviors, quality, and expert behavior. The concepts identified in the literature were extrapolated to identify key components of quality end-of-life care in the ICU setting.

Sample

Following the literature review, nine nurses working in adult critical care units were interviewed and asked to describe what constitutes optimum end-of-life care. Participants ranged in age from 26 to 56 years, with a mean time of 10.3 years (SD = 8.6) of clinical experience in adult ICU settings. They were employed full-time at an academic medical center in the Southeastern United States and worked in the burn center (n = 1), medical ICU (n = 3), surgical ICU (n = 1), coronary care unit (n = 2), and the cardio-thoracic ICU (n = 2). Twenty-two percent of the nurses were male, and 11% were African American, which approximates the demographics of practicing critical care nurses (American Association of Critical-Care Nurses [AACN], 2008).

Procedures

Each interview began with a single question: “How would you define optimum end-of-life care in the ICU?” The interviews were semistructured to guide the participants to describe both positive and negative experiences with providing end-of-life care in the ICU. The interviews were audiotaped and lasted from 35 to 70 minutes. Independent, complete phrases were identified from the transcripts, which were read multiple times in order to ensure that all relevant phrases were identified. Each interview was examined for distinct aspects of nursing attitudes and behaviors with end-of-life care in the ICU, which, ultimately, were the basis for the item generation for instrument.

After the content analysis of the literature and analysis of the transcripts from the qualitative interviews, a total of 163 potential items were identified. After deleting duplicate items (56), additional items were eliminated based on criteria outlined in DeVellis (2003), which includes eliminating items of great length, items containing multiple negatives, and items that would elicit only a yes or no response. After employing these criteria, the item pool was reduced to 74 items.

After reviewing the item pool with an expert in instrument development, a clear distinction existed between the concepts of nursing attitudes and behaviors. Thus, two instruments were created (one measuring nursing attitudes and values and the other examining nursing behaviors). The instruments were subjected separately to all subsequent assessments and administrations. The Values of Intensive Care Nurses for End of Life (INTEL-Values) instrument initially consisted of 44 items and was used to assess general nursing attitudes toward providing end-of-life care in the ICU. The Behaviors of Intensive Care Nurses for End of Life (INTEL-Behaviors) contained 30 items, assessing specific nursing behaviors performed when providing end-of-life care in the ICU.

Content Validity

Sample

In Phase II, the instruments were assessed for content validity by a convenience sample of eight critical care nurses actively practicing in adult ICUs or participating in end-of-life care research. Participants were identified by the researcher as having expertise in the field of critical care nursing and end-of-life care based on experience level, advanced degrees, and publications on the topic of end-of-life care and critical care nursing (critical care nurse clinicians and doctorally prepared nurse researchers). Participants ranged in age from 29 to 54 years (M = 42.8 years; SD = 8.47) and had worked an average of 15 years in the critical care setting (SD = 7.83). All content validity experts reported being comfortable with providing end-of-life care to patients and their families in the ICU and were recruited from academic medical centers, university-owned community hospitals, and urban and rural community hospitals from various parts of the country to provide a heterogeneous sample of experts.

Procedures

Participants were mailed the instruments and instructions for completing the content validity assessment. They were to review each item and determine if the item was an appropriate aspect of nursing attitudes and values or behaviors when providing end-of-life care as well as to evaluate the comprehensiveness of the entire collection of items (separately for each instrument). Using the technique recommended by Lynn (1986), the content validity index (CVI) was calculated for each item (participants rated each item using a 4-point scale; 1 = not relevant, 2 = unable to assess or in need of so much revision that it would no longer be relevant, 3 = relevant but needs minor revision, and 4 = very relevant and succinct).

The CVI was calculated as the proportion of experts who rated each item a 3 or 4, and the CVI for the total instrument was calculated as the proportion of total items judged a 3 or 4. Items were eliminated or revised if they did not have a CVI above the recommended .80 agreement (Lynn, 1986). Additionally, items were reviewed for their clarity and conciseness, and experts were asked if any aspect of quality end-of-life care in the ICU was missing. The total CVI for the INTEL-Values instrument was 0.88. Eight items were found to have a CVI < .80, the a priori minimum cut-off for items with eight judges (Lynn, 1986, 1995). Of these 8 items, 2 were deleted and the remaining 6 were revised to clarify them. The experts offered no suggestions about 26 of the items, and the remaining 10 items were revised to provide clarity regarding the concept of nursing attitudes at the end-of-life. The experts suggested adding three questions related to pain management, advance care planning, and decision making.

For the INTEL-Behaviors instrument, the total CVI was 0.96 and only one item did not meet the recommended CVI value (≥.80). This item was revised based on suggestions from the experts. Twelve items did not require any changes and the remaining items had minor grammatical changes. The experts made suggestions for two additional items and requested more emphasis on advance care planning and communication.

Pilot Testing

Procedures

Following content validity testing, both instruments were formatted using a 5-point Likert response format (strongly disagree to strongly agree) and then pilot tested with three groups (n = 12) of critical care nurses who had cared for dying patients and their families recruited from two academic medical centers and one community hospital. Group sessions were used for this phase of the instrument development process because a collective (versus individual evaluation) encourages brainstorming about the items. The focus of the sessions was the evaluation of the adequacy and clarity of the directions, item clarity and formatting, and overall usability of the instrument.

Sample

Pilot participants were primarily female with an average age of 40.6 years (SD =11.4) and an average of 9.15 years (SD = 8.1) practicing in adult ICUs. The sample demographic is comparable to that of ICU nurses’ practices nationally (AACN, 2008). Participants completed consent forms, demographic sheets, and the proposed instruments before being asked to provide feedback regarding the instructions and items. Participants were asked specifically about the instruments’ directions, item structure, and response options. This group session was audiotaped to insure accuracy.

The INTEL-Values instrument was reported to have clear directions and the response format was noted to be consistent with the items. No items were eliminated based on the pilot testing but 18 items were revised for clarity. The order of the items was altered based on the pilot participant’s feedback. Five questions related to evaluating other health care providers’ competence with end-of-life care, fear of death, and debriefing of the nurse after the death were added based on participants’ feedback.

Regarding the INTEL-Behaviors instrument, no items were eliminated, but 10 items were revised for clarity. Like the INTEL-Values instrument, pilot participants suggested a change in the order of some items. Items involving advance care planning and advance directives were suggested to come earlier in the INTEL-Behaviors instrument as they thought this provided better structure and flow to the instrument. Four additional items were added to address family spirituality, communication, debriefing, and family dynamics.

The response format for the INTEL-Behaviors instrument was changed based on the feedback of the pilot participants. Originally formatted with five response options, the majority of pilot participants indicated this response format was problematic. Based on their feedback, the directions were revised and participants were asked to respond by indicating how often they performed each behavior. Thus, the response format was changed to five options--never, rarely, sometimes, usually, or always. Five neuroscience critical care nurses then evaluated the original and revised response formats and expressed unanimous support for the new response format and options. After completion of Phase II, the two nursing instruments (INTEL-Values and INTEL-Behaviors) were formatted for online administration using Survey Monkey (SurveyMonkey.com, Menlo Park, CA).

Field Testing and Factor Analysis

Procedures

Phase III consisted of field testing the instruments with ICU nurses across the US. Participants were RNs who practiced actively in an ICU and had cared for at least one dying patient and his or her family. Participants were recruited from the AACN national database (AACN is the largest professional organization for critical care nurses, with approximately 400,000 members with a variety of nursing backgrounds). An e-mail message, which provided participants with information about the study as well as a hyperlink to access the consent document and the instruments, was sent to all AACN members through the electronic newsletter. It is not possible to determine the true accessible population and response rate as some AACN members may not have an e-mail address, the e-mail might have not been delivered to all with e-mail addresses, or could have been filtered by spam software. Members of AACN who accessed this internet link were provided with a consent form and asked to complete each of the two instruments. They were asked to provide a unique identifier that was used to link responses for those who later completed the retest. The opening screen contained a consent form which was followed by a radio button that, when selected, indicated their agreement to participate in the study.

Sample

A total of 857 critical care nurses completed some portion of the two instruments, with 716 fully completing both instruments. Eleven people opened the consent form but did not proceed to the instruments. Of the 716 nurses completing both instruments, 684 also completed all or part of the demographic questionnaire. Participants were primarily female with an average age of 44.1 years (SD = 10.7) and an average of 14.71 years (SD = 10.3) practicing in an ICU. Participants were from every state in the US except South Dakota and Wyoming, as well as Canada (n = 4), Belgium (n = 1), Puerto Rico (n = 1), and Australia (n = 1), which is consistent with demographics reported by the AACN (2008).

Test-Retest Reliability

To assess test-retest reliability, a second email was delivered 2 weeks after the first completion of the instruments using the earlier described process. As with the initial administration, the response rate cannot be determined. The purpose of the test-retest was described in the e-mail message and individuals were asked to complete the instruments a second time. A 2-week interval between administration times was selected because it limits the recall of responses provided on the first administration but does not, generally, allow enough time for the respondents to have altered their attitudes or behaviors (DeVellis, 2003). Again, the e-mail message included the website link to use to the second consent form and access to the instruments. Thirty ICU nurses from the original sample completed the instruments a second time. The average age of these participants was 45.37 years (SD = 10.0) with an average of 15.9 years experience in as an ICU nurse (SD = 10.7).

Results

Field Testing

Using the international sample of respondents, the data were examined and the extent of missing data identified. Respondents were dropped from the analysis if they had more than 10% of their responses missing (DeVellis, 2003). Analysis of the field test data focused the structure and reliability parameters of each instrument. Principal axis factoring method (PAF) was chosen as the exploratory method of extraction to avoid overestimating the number of factors and the item loadings on factors commonly found with principal components analysis. Oblique and orthogonal rotations were examined to identify the most interpretable structure. The number of factors to be rotated was determined by the elbow of the Scree plot, with one less and one more factor also examined to pinpoint the most salient structure. The minimum factor loading was set at .35 for an item to be deemed as belonging to a factor, and items were not included if they contained double loadings less than .15 between the highest and lowest loading (Tabachnick & Fidell, 2001). The factors were named by reviewing items, in order, from the highest to the lowest loaded and identifying the label that best encompassed most of the items on the factor. Cronbach’s alpha was calculated for each factor and alpha values greater than 0.7 were considered acceptable.

The data from the test-retest sample were analyzed for stability across the 2-week period. Both Pearson’s correlations and Kappa statistics were used to examine the stability of the factors; with stability considered sufficient with a Kappa greater than 0.5 or a Pearson’s correlation greater than 0.7 (DeVellis, 2003).

Exploratory Factor Analysis: INTEL-Values

A total of 716 individuals completed the INTEL-Values and 21 had more than 10% missing data so were dropped from the analysis. Thus, 695 respondents were included in this exploratory factor analysis (EFA) for the instrument. The statistical tests preceding the EFA gave a mixed reading on the factorability of the data. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was .75, which would be considered middling by Kaiser (1974), suggesting that clear, distinct factors may not be easy to identify. In contrast, the Bartlett’s test of sphericity (5725.93, p < .001) provided support for the factorability of the data.

The elbow of the Scree plot was determined to be at three factors so solutions between two and four factors were examined. After examination of the orthogonal and oblique rotations, the four-factor solution (oblique rotation) was determined to be the best solution because the majority of the loadings on the factors were high (>.35), there were fewer double loadings than the other solutions, and it was conceptually consistent with the qualitative analysis. These factors were labeled Self-appraisal, Appraisal of Others, Emotional Strain, and Moral Distress. The total variance explained by the four factors was 25.8%.

Factor 1 (Self-appraisal, 13 items) had an alpha of .78, with factor loadings ranging .35-.54, and interitem correlations ranging .12–41. No items were deleted following examination of the alpha values if item deleted. Factor 2 (Appraisal of Others, 6 items) had an alpha of .71 with factor loadings ranging .36-.83. The interitem correlations ranged .18-.78. Two items in Factor 2 were highly correlated (r = .78), indicating that one of the items was not needed and thus one should be deleted. However, upon further analysis it was determined that the alpha of the factor would drop substantially (.64 and .65, respectively), so neither item was deleted. Initially, Factor 3 (Emotional Strain, 4 items) had an alpha of .59 with factor loadings ranging .40-.66. The decision was made to delete the item: “Find that some patients’ deaths are more difficult because they make me think of the death of someone I cherish.” The decision to remove the item was based on the fit of the items after reading all the items in the factor. The alpha increased to .62 once this item was deleted. The interitem correlations ranged .15-.73. Two items were highly correlated (r = .73), suggesting these items are redundant and should be deleted; however, they were not deleted, as the alpha would have decreased to .42. Factor 4 (Moral Distress, 4 items) had an overall alpha of .60 with factor loadings ranging .37-.60. No items were deleted from this factor and interitem correlations ranged .16-.34.

Of the original 51 items, 24 items were deleted due to double loadings or loadings less than 0.35 on their respective (highest loaded) factor. Two items were deleted due to loadings on more than one factor. These items addressed religious beliefs and comfort with emotions. Twenty-two items were deleted for insufficient loadings on the factor (<.35) and reflected concepts surrounding Do Not Resuscitate orders, communication, conflict resolution, and debriefing.

Exploratory Factor Analysis: INTEL-Behaviors

Of the 687 individuals who completed the INTEL-Behaviors, five had more than 10% missing data and were eliminated from the analysis. A total of 682 respondents were included in the EFA with the KMO measure of sampling adequacy (.88, considered meritorious by Kaiser, 1974) and the Bartlett’s test of sphericity (3947.98, p < .001) supporting the factorability of the data. Again, PAF was used for this analysis. The elbow of the Scree plot appeared to be at 2 factors, which was the starting point in examining the number of factors. Using similar evaluation criteria for determining the number of factors of the INTEL-Behavior, two factors were deemed the best solution. These factors were labeled Communication and Nursing Tasks. The total variance explained by the factor analysis was 24.9%.

Factor 1 (Communication, 12 items) had an initial alpha of .78. The decision was made to delete one item based on the fit of the items after reading all the items in the factor. This item possessed the lowest loading on the factor (.36) and the lowest interitem loadings (.07); and the alpha increased to .79 once this item was deleted. Interitem correlations were .14-.43. Factor 2 (Nursing Tasks, 7 items) had an alpha of .67. No items were deleted in this analysis as it appeared that they correlated well with each other and the factor did not benefit from having items removed. The interitem correlations ranged .10-.35.

Of the original 34 items, 15 items were deleted due to double loadings or loadings less than 0.35. These items reflected concepts surrounding debriefing, family conflict, care after the death, pain medication, and utilizing resources. No items in the two-factor analysis possessed double loadings.

Test-Retest for INTEL-Values and INTEL-Behaviors

The same sample was used for the assessment of the test-retest of the INTEL-Behaviors and INTEL-Values. Pearson’s correlation, Kappa, and the intra-class correlation (ICC) were used to assess stability of the instruments across the 2-week period. Test-retest estimates for the INTEL-Values yielded low Kappa values (.05-.30) for all four factors. Pearson’s correlations (.68-.81) and ICC coefficients (.65-.79) across the 2 weeks were high. The INTEL-Behaviors also produced low Kappa values (.02-.40) on both factors; however, Pearson’s correlations with the factors across time were high (.77-.81). These values are presented in Table 1.

Table 1.

Test-Retest Estimates for INTEL-Values and INTEL-Behaviors

Instrument Factors	Pearson’s Correlation	Kappa	ICC
INTEL-Values
Self-appraisal	.68	.13	.65
Appraisal of Others	.73	.05	.70
Emotional Strain	.79	.30	.78
Moral Distress	.81	.20	.80
INTEL-Behaviors
Communication & Decision Making	.77	.02	.77
Nursing Tasks Ensuring a Peaceful Death	.81	.40	.82

Notes. INTEL-Values = Values of Intensive Care Nurses for End-of-Life, INTEL-Behaviors = Values of Intensive Care Nurses for End-of-Life

Discussion

Factor Loadings for INTEL-Values

The lower than desired interitem correlations on the factors for the INTEL-Values may be due to the challenge noted when measuring attitudes. This difficulty has been noted previously and is not exclusive to nursing (Halloran, 1976; Mueller, 1986; Wealleans, 2003). The ICU nurses in the pilot testing phase said they liked the format of the instrument but found it challenging to think about how they “truly felt” about the subject of “end-of-life care” in the ICU. This is common among critical care nurses as their culture is focused on intensive and technological procedures, and critical care nurses have limited time to reflect on their values, attitudes, feelings, or roles when providing this care (Halcomb et al., 2004; Puntillo et al., 2001). The difficulty that the critical care nurse respondents may have had when asked to assess, identify, and reflect on their individual attitudes, values, beliefs, or preferences may have influenced their responses, which in turn affected the factorability and reliability of this instrument.

Although the items on Factor 1 (Self-Appraisal) appeared to relate to each other and to the concept of self-appraisal, they possessed low interitem correlations, indicating that there was little commonality among the items and may be due to the respondents’ difficulty in appraising their own attitudes, values, or self-perceptions. In an environment where death is seen as a failure, ICU nurses report that self-appraisal of their ability to provide this care is not a major focus or concern (Dobratz, 2005; Puntillo et al., 2001). Despite the lower than desirable correlations, the alpha for the factor was high, implying that the items seem to relate to each other (Pett, Lackey, & Sullivan, 2003).

The higher correlations and alpha value for Factor 2 (Appraisal of Others) indicate that these items relate to each other more strongly than those in Factor 1, which may be because it is easier to evaluate another individual than to self-evaluate. This factor is important since nurses, especially novices, model the attitudes and behaviors of their colleagues (Benner, 1984; Puntillo et al., 2001).

The lower than acceptable alpha value and interitem correlations for Factor 3 (Emotional Strain) reflects the difficulty of assessing nursing values when providing end-of-life care in the ICU. In a recent survey of critical care nursing needs at the end of life, 49% of respondents indicated that they never had time to debrief after a death (Puntillo et al., 2001).

The low reliabilities, correlations, and item loadings on Factor 4 (Moral Distress) exhibit evidence related to the struggle with identifying nursing attitudes and values with end-of-life care in the ICU, which may have contributed to the low psychometric properties of this factor.

Factor Loadings for INTEL-Behaviors

The Cronbach’s alpha for the INTEL-Behaviors factor was higher than that of the INTEL-Values instrument, and the factor solutions were clearer and easier to identify. This is most likely due to the more concrete identification of behaviors compared to attitudes and values. Although the items on Factor 1 (Communication) appeared to relate to each other and to the concept of communication, the items had low interitem correlations, indicating that there was acceptable commonality among some of the items. Despite the lower than desirable correlations, the alpha for the factor was high, implying that the items belong together. These results may be due to the complex role of communication in nursing; although communication is an important concept, it is possible that it is not easy to encapsulate.

The low reliabilities, correlations, and item loadings for Factor 2 (Nursing Tasks) exhibit evidence related to the struggle identifying what tasks nurses actually perform when providing end-of-life care in the ICU. Despite a lack of training in end-of-life care, ICU nurses are often given the responsibility of caring for the dying patient and their family, and receive little respite from the stress of caring for these individuals (Beckstrand & Kirchhoff, 2005).

Test-Retest for INTEL-Values and INTEL-Behaviors

Comparing the two instruments, the Pearson’s correlation and ICC for the INTEL-Values instrument were considered relatively high, but both instruments’ Kappa coefficients were low. This discrepancy is most likely due to the sensitivity of Kappa. Pearson’s correlation indicates whether the factor scores systematically covary while Kappa only indicates identical responses over time periods (Streiner & Norman, 2003). In terms of stability and consistency, the high correlations indicate that the total scores on the factors were systematically consistent across time, but the Kappa results offer another interpretation. The low Kappa coefficient could be due to an instrumentation effect (Cook & Campbell, 1979). It is possible that a change in attitudes did occur during the 2-week period as a result of taking the INTEL-Values instrument. Simply allowing the respondents to think about their own attitudes and values at Time 1 could have influenced their responses at Time 2. The first administration of the INTEL-Values could have heightened their awareness and sensitivity about nursing attitudes and values while providing end-of-life care in the ICU, thus changing their responses during the 2-week period. This change could have been subtle, but any difference between the first and second administration would have resulted in a low Kappa value.

For the INTEL-Behaviors, consistency and stability of the items over time was not expected, as the use of this instrument was to measure specific behaviors when providing end-of-life care to a particular dying patient and family. The field testing participants were asked to consider the last dying patient they had cared for when completing the instrument. It is possible that the patients considered at Time 1 and Time 2 were different, causing the scores to vary considerably over time, thereby influencing the stability. Thus, lower Kappa coefficients would be expected. Although respondents for the INTEL-Behaviors were relying on their memory of the patient when completing this instrument, the Kappa coefficient on Factor 2 (Nursing Tasks) was the highest value. Two explanations can be provided for this result, that respondents actually remembered the patient they provided care for consistently over a 2-week period, or that nurses do not vary considerably in the care they provide for patients. The second explanation is more likely, as without an intervention to change care delivery, it is unlikely that nurses would vary in behavior.

Conclusions

The purpose of this instrument development was to evaluate nursing attitudes and behaviors when providing end-of-life care in the ICU. The scope of this study did not include specific assessments of construct validity, which will be done in future research. At least a part of the assessment of the construct validity of these instruments will include relating scores to an assessment of the end-of-life care by family members. Satisfaction with end-of-life care by family members should relate positively to nurses’ scores on the INTEL-Behaviors and INTEL-Values instruments. Criterion-related validity assessment will be considered also by examining the relationship between the INTEL-Values and INTEL-Behaviors scores and nurses’ levels of education and experience to see if scores covary with different levels of education and experience (hospice nurse vs. critical care nurse, new graduate vs. experienced ICU nurse). Following criterion-related testing, the instruments can be used in intervention research or for educational purposes with nursing students, continuing education participants, and practicing intensive care nurses. Future work will examine if these instruments can be used as pre- and postintervention assessments to determine if an educational intervention is effective. Finally, additional stability testing is needed.

The INTEL-Values and INTEL-Behaviors instruments offer an opportunity for research on nursing attitudes and behaviors and how these concepts can impact care. It is important to continue the focus on improving end-of-life care in the critical care setting, but this focus of quality improvement will not be successful if the values and behaviors of the nurse providing this care are not addressed. The instruments are administered easily via the online method and may be a useful addition to the current instruments examining quality end-of-life care from the patient and family perspective. By combining the views of the patient, family, and critical care nurses, it is possible to identify concepts necessary to create and implement effective interventions.