Psychometric testing of the caring assessment tool: Administration (CAT-Adm^©)

Abstract

Objectives:

The overall purpose of this study was to evaluate the validity and reliability of the Caring Assessment Tool-Administration survey. Three specific aims were to (1) evaluate construct validity of the Caring Assessment Tool-Administration survey by testing the hypothesized eight-factor structure of staff nurses’ perceptions of nurse manager caring behaviors, (2) estimate the internal consistency, and (3) conduct item reduction analysis.

Methods:

A 94-item Caring Assessment Tool-Administration designed to assess nurse manager caring behaviors appeared in the literature but lacked robust psychometric testing. Using a foundational theory and a cross-sectional descriptive design, the Caring Assessment Tool-Administration was evaluated for reliability and construct validity. Using convenience sampling, 1143 registered nurses were recruited from acute care hospitals in three states located in the Midwestern, Mid-Atlantic, and Southern Regions of the United States.

Results:

Psychometric testing of the Caring Assessment Tool-Administration was conducted using confirmatory analysis to determine the dimensionality of the construct, nurse manager caring behavior. The null hypothesis was an eight-factor solution fitting the theoretical model being tested. The null hypothesis was rejected because none of the measures examined for goodness of fit indicated the model fit the data. Confirmatory factor analysis did not support the hypothesized structure; however, exploratory factor analysis supported a one-factor solution that was conceptually labeled caring behaviors. To decrease subject burden, the 94-item survey was reduced to 25 items using item reduction analysis including assessing minimum factor loadings of ≥0.60 and evaluating survey item-total correlation and alpha. The Cronbach’s alpha of the new 25-item survey was 0.98.

Conclusion:

The new 25-item Caring Assessment Tool-Administration survey provides hospital administrators, nurse managers, and researchers with a sound, less burdensome instrument to collect valuable information about nurse manager caring behaviors.

Introduction

The current acute care health systems are complex, chaotic, and rapidly changing, generating unprecedented uncertainty.^1,2 This uncertainty stems from the nearly complete restructuring of American healthcare, which has placed immense pressure on leaders of acute care organizations to make changes that affect employees at all levels.³ Among the many challenges presented to hospital administrators at all levels, work must be done to keep their institutions financially viable, which often requires difficult and unpopular decisions related to reorganization, consolidation and elimination of services, and workforce reduction. Balancing the demands of fiscal management and, at the same time, providing effective leadership to employees is difficult in times of uncertainty.⁴ More often, priority is given to the financial needs of the institution over the needs of employees.

Inattention to employees’ needs adds additional strain to an already stressed organization plagued by change and growing complexity that adversely affects the work environment. The nature of nurse work has also changed in recent years because nurses have been asked to simultaneously care for sicker patients with fewer resources, implement new “smart” technologies, launch electronic medical record programs, and deal with value-based reimbursement models. These changes, coupled with administrative pressure to focus on fiscal cutbacks, can frustrate nurses and lead to job dissatisfaction, burnout, fatigue, anxiety, and inability to provide the quality of care expected⁵ and desired. The consequences of work-related frustration contribute to unhealthy work environments that are associated with excessive absences and the intention to resign, which further adds to an unstable workforce.^6–10

There is considerable agreement in the literature about the relationship between a healthy work environment and staff nurses’ satisfaction, professional empowerment, and retention.^{5,7,8,11–18} Cara et al.¹⁹ suggested that a supportive and caring work environment could be enhanced by caring relationships between nurse managers and staff nurses. Expanding the relational role of the nurse manager is a viable way to create, enhance, and sustain a healthy work environment.^6,14,20,21

Caring relationships between nurse managers and staff nurses support the idea that nurses who feel cared for by their managers are more motivated to develop caring relationships with patients.¹⁹ According to Duffy’s²² Quality-Caring Model^® (QCM), caring relationships are instrumental and expressive behaviors that when used in the context of shared professional interactions facilitate understanding, learning, comfort, human dignity, security, self-confidence, hope, and encouragement. Examples of behaviors used by nurse managers that demonstrate caring are listening, being accessible, being encouraging, maintaining safe working environments, and offering support. When staff nurses feel cared for, positive patient, nurse, and system outcomes are theoretically proposed to occur.²²

The measurement of staff nurses’ perceptions of nurse managers’ caring behaviors has not been extensively studied. The lack of a psychometrically tested tool has contributed to the lack of evidence supporting more relational approaches to leadership. For example, the Caring Assessment Tool-Administration (CAT-Adm^©) survey originally developed by Duffy²³ had not undergone comprehensive psychometric testing,^22,23 and no other instruments were found that adequately measured this phenomenon.

Thus, the overall purpose of this study was to evaluate the validity and reliability of the CAT-Adm survey. Three specific aims were to (1) evaluate construct validity of the CAT-Adm survey by testing the hypothesized eight-factor structure of the measure of staff nurses’ perceptions of nurse manager caring behaviors, (2) estimate the internal consistency, and (3) conduct item reduction analysis to reduce administrative and participant burden.

Theoretical framework

Duffy’s²² QCM served as the theoretical framework for this study because it informs development of the items on the CAT-Adm Survey and specifically identifies the critical nature of nurse managers’ caring behaviors. The QCM is a middle-range theory originally described as a blended model with major concepts related to quality and caring.²⁴ The model was revised in 2013 to include the influence of complex adaptive systems.²² Relationship-centered professional encounter is a major tenet of the QCM and is described as a process between individuals that when conducted in a mutually reciprocal manner promotes “feeling cared for” in the recipients which then leads to progression or advancement. Relationship-centered professional encounters can be expressed through the attitudes, behaviors, and skills of professional nurses and their managers. Duffy originally proposed that eight factors comprised the concept: mutual problem-solving, attentive reassurance, human respect, encouraging manner, appreciation of unique meanings, healing environment, affiliation needs, and basic human needs. This study hypothesized that an eight-factor structure comprised this aspect of the QCM.

Methodology

Design

A cross-sectional, descriptive design was used to evaluate the specific aims of this study.

Setting/sample

To maximize the potential for recruiting the study sample, hospitals and hospital systems with greater than 500 beds were selected as possible recruitment locations. Hospital administrators from 10 hospital/hospital systems were approached for inclusion as recruitment sites. Of the 10 hospital/hospital systems, five hospital administrators chose to participate in the study. These five hospital/hospital system administrators represented the seven hospitals from which data were collected (see Table 1). Staff nurses were recruited from these seven hospitals located in three states in the Midwestern, Mid-Atlantic, and Southern Regions of the United States (see Table 1). These seven hospitals included suburban community, urban academic/teaching, and rural institutions and included both magnet- and non-magnet-certified facilities. Permission from chief nurse executives (CNEs) and approval from the university and each site’s institutional review board were obtained prior to beginning the study.

Table 1.

Demographics and sample characteristics^a.

Variable	Description	Frequency (%)
Hospital (# participants/hospital)	1 Hospital (system A)	2 (0.28)
	2 Hospital (system A)	134 (19.06)
	3 Hospital (system A)	17 (2.42)
	4 Hospital	182 (25.89)
	5 Hospital (system B)	102 (14.51)
	6 Hospital (system B)	70 (9.96)
	7 Hospital	196 (27.88)
Age (years)	<25	64 (9.10)
	25–34	176 (25.04)
	35–44	156 (22.19)
	45–54	157 (22.33)
	55–64	136 (19.35)
	65+	14 (1.99)
Gender	Female	644 (91.61)
	Male	59 (8.39)
Ethnicity	Not Hispanic/Latino	689 (98.01)
	Hispanic/Latino	14 (1.99)
Race	American Indian or Alaska Native	9 (1.28)
	Asian	20 (2.84)
	Black or African American	30 (4.27)
	Native Hawaiian or Other Pacific Islander	1 (0.14)
	White	658 (93.6)
Education degree (level of education)	RN diploma	27 (3.84)
	Associate degree nursing	200 (28.45)
	Bachelor’s degree nursing	351 (49.93)
	Bachelor’s degree non-nursing	41 (5.83)
	Master’s degree nursing	45 (6.40)
	Master’s degree non-nursing	19 (2.70)
	Doctorate degree	6 (0.85)
	Some graduate classes	14 (1.99)
Length of time (duration) on current unit	6 months–1 year	85 (12.09)
	>1–3 years)	195 (27.74)
	>3–5 years	109 (15.50)
	>5–10 years	150 (21.34)
	>10–15 years	70 (9.96)
	>15–20 years	46 (6.54)
	>20–25 years	13 (1.85)
	>25 years	35 (4.98)
Number of years as RN	6 months–1 year	48 (6.83)
	>1–3 years	86 (12.23)
	>3–5 years	88 (12.52)
	>5–10 years	144 (20.48)
	>10–15 years	75 (10.67)
	>15–20 years	65 (9.25)
	>20–25 years	51 (7.25)
	>25 years	146 (20.77)
Nursing certification	Yes	477 (67.85)
	No	226 (32.15)
PPM	Yes	272 (38.69)
	No	12 (1.71)
	Unsure	419 (59.60)
Type of hospital	Community hospital	315 (44.81)
	Academic hospital (teaching)	374 (53.20)
	Rural hospital/critical assess	14 (1.99)
Type of population cared for	Newborn	41 (5.83)
	Pediatric	37 (5.26)
	Adult	625 (88.90)
Type unit worked on	Medical	42 (5.97)
	Surgical	22 (3.13)
	Medical/surgical	134 (19.06)
	Step-down	58 (8.25)
	Progressive care	37 (5.26)
	Critical care	84 (11.95)
	Transplant	12 (1.71)
	Pediatrics	18 (2.56)
	Intensive care	80 (11.38)
	Perioperative	40 (5.69)
	Maternal–child (women’s health)	54 (7.68)
	Emergency department	47 (6.69)
	Psychiatric/mental health	9 (1.28)
	Other	66 (9.39)

RN: registered nurse; PPM: professional practice model.

^aSample characteristics were based on 703 staff nurses who met inclusion criteria and completed all survey items.

DeVellis²⁵ suggested using a ratio of 5–10 participants per survey item as a basis for determining sample size. Therefore, a sample of 940 participants were sought. Larger sample sizes are needed in factor analysis to determine the factor structure particularly depending on the number of items to be factored.²⁶ A convenience sample of 1143 acute care hospital staff nurses (registered nurses (RNs)) were recruited. To be included in this study, staff nurses must have been permanent staff members for a minimum of 6 months on the unit in which they were employed. Six months duration of unit-specific employment was chosen because this allowed the participants time to establish and reflect upon relationships that had developed with the nurse manager. Staff nurses with less than 6 months of experience working on the unit or were temporary or traveling nurses were excluded.

Instruments

Study data were collected using two instruments. The first instrument, Staff Nurse Demographic Questionnaire, was developed specifically for this study to collect demographic data about the participants. The second instrument was CAT-Adm, a 94-item survey developed in 1997, which was designed to measure staff nurses’ perceptions of nurse manager caring behaviors.^22,27 The CAT-Adm (1997) was adapted from the Caring Assessment Tool Survey (CAT Survey)^© (1994) which was developed to measure patients’ perceptions of nurse caring behaviors. Both CAT-Adm and CAT Survey are similar in that the theoretical framework used in their development was the QCM.²²

The original 94-item CAT-Adm survey was written in English at a fourth-grade reading comprehension level and was administered using paper and pencil. The respondents were asked to circle how often each activity occurred during their work period. Responses were arranged on a 5-point Likert-type scale with anchors 1 (never) to 5 (always) and 20 items were worded negatively to minimize error.²⁷ The range of scores for the 94 items was 94–470 with lower scores indicating less caring. The original 94-item questionnaire data collected in 1997 were evaluated using exploratory factor analysis (EFA). Three factors were found to explain 63.44% of the variance. Internal consistency and reliability for the total instrument was 0.942 using Cronbach’s alpha. Although the number of participants was satisfactory (N = 1850 RNs) and internal consistency reliability for the total instrument was acceptable, three major concerns impacted the quality of the study; it was conducted in one region of the country using a convenience sample and confirmatory analysis was not completed.

This study was conducted to determine the dimensionality of the concept, nurse manager caring behaviors. Full psychometric testing of the original 94-item CAT-Adm was performed. This included confirmatory factor analysis (CFA) to evaluate whether the eight-factor structure represents the concept nurse manager caring behaviors.

Procedure

Study data were collected and managed using Research Electronic Data Capture (Indiana University (REDCap))²⁸ electronic data capture tools hosted at Indiana University. REDCap is a secure, web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry, (2) audit trails for tracking data manipulation and export procedures, (3) automated export procedures for seamless data downloads to common statistical packages, and (4) procedures for importing data from external sources.

The CAT-Adm was administered through an electronic device within the hospital setting. The CNE from each of the participating hospitals provided one information technology contact person to assist in creating a link to the web-based (REDCap) CAT-Adm survey and provided a connection with the staff nurses’ hospital email addresses. Once the link was operational, an administrative contact person was alerted to begin distribution of Staff Nurse Demographic Questionnaire and CAT-Adm survey to the identified pool of prospective study participants through their hospital-secured email accounts. The hospital-secured email accounts of the study participants provided additional confidentiality protection. The data were collected over a 6-month time period.

Data analysis

The software program for conducting statistical analysis for this study was SAS^® (Statistical Analysis Software), version 9.4 of the SAS System.²⁹ Data from Staff Nurse Demographic Questionnaire and CAT-Adm were exported from REDCap in a form directly usable for SAS.

Data were checked for accuracy prior to conducting statistical analysis. The data cleaning process revealed that the 94-item survey uploaded into REDCap inadvertently deleted three of the survey items. This omission was determined to not pose a significant risk to the integrity of the analysis since the three omitted questions were spread out across the instrument, and an adequate number of items representing the same concept were available for analysis.

Demographics and sample characteristics

There were 1143 staff nurses available for recruitment, and of those, 1080 agreed to participate. Of those who agreed to participate, 995 met inclusion criteria and were enrolled. Of the 995 who were enrolled, 703 completed the questionnaire and comprised the study sample. Sample characteristics are displayed in Table 1.

Of the seven participating hospitals, five were members of two systems. One hospital administrator oversaw three hospitals (system); all the others administered one hospital each. Two hospitals were part of a system, but each had their own hospital administrator.

Demographic data collected from this study are similar to national statistics for age, gender, education, race, and ethnicity.^30–32

Most of the participants were white, not Hispanic/Latino, and female and were between the ages of 25 and 64 years. Approximately 50% had a bachelor of science in nursing (BSN) degree and 67.85% were certified in a specialty practice area. The duration of employment on the current unit was >1–3 years (27.74%) followed by those who worked on their units >5–10 years (21.34%). The two largest categories in “years of experience as an RN” were between >5 and 10 years (20.48%) and >25 years (20.77%). Most staff nurse respondents worked with adult patients in either community (44.8%) or academic (53.2%) hospitals. RN certification type was collected; however, small cell sizes precluded use in analyses.

Results

The first aim was to evaluate the construct validity of CAT-Adm survey by testing the factor structure underlying variance in staff nurses’ perceptions of nurse manager caring behaviors. More specifically, the intent was to determine whether the hypothesized factor model fit the data. The null hypothesis was an eight-factor solution based on eight caring factors described in the theoretical framework QCM. To address this aim, CFA was used.

The null hypothesis was rejected since none of the four measures examined for goodness of fit indicated adequate model fit to the data ((chi-square statistic) chi = 14732.7, degree of freedom (df) 3976 (p < 0.0001), standardized root mean square residual (SRMR = 0.047), root mean square of approximation root mean square error approximation (RMSEA = 0.062), and Bentler comparative fit index (0.856)).^33,34 Since the model was rejected using CFA, the next logical step was to perform EFA to identify a plausible factor structure given the observed data.³⁵

EFA included assessment of sampling adequacy, determining the number of factors to extract and retain, estimating factor structure (item loadings) and factor rotation, and interpretation of the factors. Prior to factor analysis, a Kaiser–Meyer–Olkin (KMO) test for sampling adequacy of 0.99 was found, supporting factorability of the correlation matrix.³⁶

Parallel analysis (PA) was then performed to determine the number of factors to retain. Factor retention was determined by comparing the eigenvalues of the observed data set produced by EFA with random data sets that parallel the observed data. Eigenvalues were extracted from random data sets that parallel the actual data set (an observed data set of this study) in terms of the number of cases and variables. Two factors were retained using this methodology.^37,38 The variance that could be explained by each of the factors is displayed in Table 2. As shown, Factor 1 explained 55.67 (61.17%) of the variance and Factor 2 contained 3.89 (4.27%) of the variance. Combined, the two factors explained 59.56 (65.45%) of the total variance.

Table 2.

Factor variance.

Factor number	Factor analysis (using PCA)	PA (using PCA)	Components to retain (using PA—PCA
	Eigenvalue (actual)	Eigenvalue (mean) (random)	Retain (+) Do not retain (−)
1	55.67	1.80	+
2	3.89	1.75	+
3	1.64	1.71	−
4	1.24	1.67	−
5	1.14	1.64	−
6	1.01	1.61	−
7	0.97	1.59	−
8	0.90	1.56	−

Only the first seven components are listed out of 91 variables in this table. Only first two components were retained.

PCA: principal component analysis; PA: parallel analysis.

Orthogonal and oblique factor rotations were then completed to make the loadings more easily interpretable.^34,35 Oblique (PROMAX) rotation of the data resulted in an inter-factor correlation of 0.709 indicating that there were substantive correlations between the two factors. Use of oblique rotation (PROMAX) was selected for further interpretation of the data because of high inter-factor correlations.

To assist in factor interpretation, factor loadings were examined using a priori factor loading limits set at 0.60 or higher to designate factor membership (relationship of the variables to the factors). As can be seen in Table 3, Factor 1 had 77 of 91 (84.6%) items loading 0.60 or higher and 73 of 91 (80%) survey items loaded 0.60 or higher on Factor 2. Negatively worded items loaded higher on Factor 2 (12/20, 60%) than on Factor 1 (8/20, 40%). Items loading at minimum 0.60 or higher on both Factor 1 and Factor 2 were 64 of 91 (70%). Factor 2 had 27 of 64 (42%) items loading 0.60 or higher and the factor structure loadings were greater on Factor 2 than on Factor 1. Of those 27 items that had higher correlation with Factor 2 (belonged more to Factor 2, but not exclusively), 16 were negatively worded items. All but one negatively worded item (item number 88) loaded more strongly on Factor 1.

Table 3.

Factor structure correlations.

Correlation between each item and the factor		Factor 1	Factor 2
q61	Helps me explore questions	0.87119	0.54992
q57	Asks me think nursing/health care1	0.86121	0.55647
q54	Helps me understand how I think about work	0.89562	0.61985
q60	Teaches me nursing/health care	0.85171	0.56113
q29	Helps me with my bad feelings1	0.86049	0.57371
q71	Spends times with me	0.88651	0.61525
q66	Makes sure my co workers know my needs1	0.84619	0.57051
q92	Helps me to cope with job stress	0.91016	0.66849
q28	Ask me how I like to do my work1	0.85933	0.60409
q91	Acknowledges my inner feelings1	0.90410	0.67319
q63	Checks with me to make sure I understand workplace1	0.87215	0.63777
q58	Provides me literature about work1	0.75645	0.47833
q34	Checks on me	0.85332	0.61906
q80	Helps me feel less worried	0.89161	0.67398
q55	Asks me how I think work is going1	0.87919	0.66222
q77	Helps me feel special1	0.91518	0.71586
q90	Knows what is important to me1	0.89317	0.68833
q73	Allows my family to be involved in work decisions	0.74332	0.47796
q64	Makes me feel stress free as possible	0.87048	0.65774
q53	Helps me deal with difficult situations	0.88798	0.69835
q50	Helps me find solutions to work problems1	0.87811	0.68807
q89	Is concerned about how I view things1	0.86467	0.67072
q52	Helps me with all work problems	0.86951	0.67908
q23	Anticipates my needs	0.88296	0.70225
q30	Shares personal information with me	0.74880	0.51631
q87	Understands my unique situation1	0.86768	0.68413
q49	Helps set performance goals	0.82220	0.62803
q26	Shows concern for family	0.84703	0.66675
q45	Is aware of my feelings1	0.75674	0.53980
q78	Keeps me challenged1	0.79660	0.59994
q75	Makes sure I get breaks I need	0.76640	0.56166
q72	Makes me feels safe1	0.87444	0.71599
q83	Helps me to achieve goals1	0.86992	0.71206
q93	Respect for things having meaning to me	0.87688	0.72888
q39	Encourages me to talk about my mind	0.85523	0.69991
q33	Initiates conversations	0.87006	0.72304
q15	Helps me believe in myself	0.88258	0.75010
q6	Includes me	0.84008	0.70369
q25	Shows concern for me1	0.89047	0.77670
q31	Expresses emotions with me1	0.79942	0.65618
q13	Seems interested in me	0.87172	0.76670
q21	Helps me see good in situation	0.86501	0.76030
q69	Protects me from harmful situations	0.73142	0.57275
q86	Respects my need for rest relaxation	0.81441	0.70002
q18	Encourages me to care for self	0.81197	0.70516
q76	Monitors my activities	0.39848	0.12374
q22	Encourages me to advance career	0.75581	0.63043
q47	Allows me to talk truth no risk to job	0.80948	0.70828
q24	Allows me to choose time talk1	0.82004	0.73653
q9	Pays attention to me	0.84911	0.78260
q20	Encourages questions	0.83312	0.77110
q41	Interested information I have about work1	0.83312	0.77255
q10	Enjoys working with me	0.82021	0.75805
q19	Supports my beliefs	0.83965	0.78916
q12	Available to me	0.80057	0.74942
q40	Patient with me even if I am difficult1	0.78945	0.73843
q43	Accepts what I say even if negative1	0.72328	0.65134
q16	Keeps me informed1	0.78363	0.73658
q48	Questions me about my work	0.33070	0.10318
q81	Allows me times off to be with family friend1	0.68522	0.61412
q88	Has no idea how my job affects my life	−0.60097	−0.54656
q3	Treats me kindly	0.71666	0.84783
q37	Pays attention to me when I talk	0.71316	0.81099
q2	Accepts me	0.72213	0.80019
q7	Respects me	0.78146	0.83625
q5	Answers my questions	0.73702	0.78831
q35	Looks me in eye when talking	0.64548	0.71624
q1	Listens	0.78084	0.79416
q32	Responds honestly questions	0.77240	0.78609
q67	Knows what to do in emergency	0.63753	0.68766
q65	Respect my need for privacy	0.67412	0.69941
q11	Uses my name	0.65345	0.65693
q27	Never shows emotion	−0.53471	−0.57424
q68	Never asks what I need	−0.68645	−0.69024
q94	Out of touch with my work world	−0.65247	−0.68099
q51	Deals with work problems impractical to me	−0.51675	−0.64982
q84	Does not care whether I take care of myself	−0.62812	−0.73225
q14	Has no time for me	−0.68310	−0.77632
q42	Talks about me openly in front of others	−0.06255	−0.35556
q8	More interested in own problems	−0.63980	−0.77069
q4	Ignores me	−0.63176	−0.77077
q17	Fails to keep promises	−0.48736	−0.67026
q44	Seems annoyed if I speak my true feelings	−0.58479	−0.74339
q79	Makes me wait a long time for appointment	−0.44075	−0.64613
q59	Uses terms I do not understand	−0.24588	−0.52016
q36	Refuses to tell me aspects of my work	−0.43466	−0.65625
q74	Interferes with my basic routine practice	−0.33132	−0.60245
q46	Does not want to talk to me	−0.62433	−0.81174
q38	Acts as if disapproves of me	−0.57545	−0.79294
q62	Discourages me ask questions	−0.56147	−0.80711
q82	Discourages me from interacting with others	−0.44794	−0.75488
Factor loadings minimum 0.60 or higher		77/91 (84.6%)	73/91 (80%)
Number loadings negative worded items Minimum loadings 0.60 or higher		8/20 (40%)	12/20 (60%)
Number of items loading 0.60 or higher on both factors		64/91 (70%)

Selected items for inclusion in the 25-item Caring Assessment Tool—Adm questionnaire are marked with superscript¹ and bold italic.

Given (1) the large amount of shared variance (50.3%) between the two factors, (2) the negatively worded items loaded more heavily on Factor 2, and (3) Factor 1 eigenvalue (55.7) was 14 times larger than Factor 2 eigenvalue (3.9), Factor 2 was determined to be redundant with Factor 1 and was eliminated, resulting in a one-factor solution. Only items that loaded both more substantively and stronger on Factor 1 were retained, resulting in a reduction of items on Factor 1 to 59.

After reviewing the items remaining on Factor 1, it was conceptually labeled caring behaviors. These items are congruent with the QCM major tenet of relationship-centered professional encounters, are central to the nurse manager–staff nurse caring relationship, and logically describe the multiple caring behaviors simultaneously displayed by nurse managers during their interactions with staff nurses. For example, a nurse manager’s behavior may be perceived by a staff nurse as “showing concern for me” when the manager was also “checking with me to make sure I understand what is going on in the workplace.” Furthermore, the term caring behaviors may be easier to conceptualize and better recognized by nurses in the work setting.

The second aim was to estimate internal consistency. The Cronbach’s alpha coefficient obtained for the retained Factor 1 was 0.99, which is considered good.²⁵ According to DeVellis²⁵ and Streiner,³⁹ alphas greater than 0.90 suggest the need to shorten the instrument due to redundancy of items.

The third aim, item reduction analysis, was evaluated through a series of procedural steps to reduce the number of items while at the same time maintaining an acceptable Cronbach’s alpha. The goal of reducing the number of survey items (item reduction) was to decrease administrative and participant burden. The procedural steps for item reduction analysis included assessing minimum factor loadings of 0.60 or higher, evaluating survey item-total correlation and alpha.^25,34

In the prior steps, 32 survey items were removed, resulting in 59 survey items remaining. Recall from the interpretation of the factors, one-factor was retained and had high internal consistency indicating that all items reflected a relatively homogeneous construct (caring behaviors). In addition, many of the nurses (n = 377, 35%) who agreed to participate in the study failed to complete the entire survey leaving the last items without responses which indicates survey fatigue or time constraints. Therefore, reducing the survey to 25 items was amendable to reduce participant burden in completing the CAT-Adm survey.²⁵ The decision to reduce the survey instrument was jointly made by the authors, with a target of 25 items for the final scale.⁴⁰ To reduce the number of items to 25, 59 items were analyzed using PROC CORR, which generated a randomly chosen subset of items.⁴¹ Randomly selecting items eliminated the potential for investigator bias and ensured that a representative sample of items was obtained. Cronbach’s alpha was then performed on the reduced set of 25 items (α = 0.98), which was determined to be good. Examples of the retained survey items were as follows: keeps me informed, openly shows concern for me, is patient even when I am difficult, and knows what is important to me. See Table 3 for the 25 items that are marked with bold italic and superscript.

Discussion

The purpose of this study was to evaluate the dimensionality of the CAT-Adm survey, estimate internal consistency and reliability, and reduce the number of items on the survey to decrease participant and administrative burden. Using CFA, the hypothesized eight-factor solution was rejected. A follow-up EFA suggested a single factor be retained. Review of the items that loaded ≥0.60 on Factor 1 led to the conceptual label, caring behaviors. These behaviors are congruent with the QCM major tenet of relationship-centered professional encounters and also are central to the nurse manager–staff nurse caring relationship.²² The one-factor solution representing caring behaviors seemed logical since, in the work environment, nurse managers frequently use multiple behaviors simultaneously. Furthermore, the term caring behaviors may be easier to comprehend and eventually be actualized in the work setting.

Recognizing the absence of a psychometrically sound instrument to measure nurse managers’ caring behaviors and the industry need for such a measure, the CAT-Adm survey needed further development. In 2008, Duffy conducted EFA to begin the validation process of CAT-Adm survey and found a three-factor solution, whereas in this study a one-factor solution was found. Consistent with Duffy’s prior study in 2008 of the 94-item survey, which produced a coefficient alpha of 0.94, the 25-item CAT-Adm survey’s Cronbach’s alpha was 0.98.²⁷ This new 25-item survey measures nurse manager caring behaviors as perceived by staff nurses and is consistent with the theoretical basis of the QCM and captures the essence of Duffy’s QCM relationship professional encounters. Congruent with Duffy’s QCM, caring behaviors affect relationship-centered professional encounters. This study builds on prior research and provides a valid instrument to advance the research in the field.

Study limitations

One limitation of this study is the extent to which findings may be generalized, since participants were recruited from three states in the Midwestern, Mid-Atlantic, and Southern Regions of the United States. Conducting this study in other regions of the United States would improve generalizability. This is also true with the demographics of this study which were somewhat comparable to the national statistics for RNs overall. Conducting this study in other regions of the United States may increase the number of participants by gender (more males), and diversity related to race and ethnicity other than White. Another limitation of this study was the use of convenience sampling. This sampling method may not yield a representative sample of the population of interest and therefore limits a more general application of the results. Designing a study using random selection would improve the generalizability of the findings.

Other possible limitations are that the participants’ cultural perception of caring, assessing caring behaviors through an electronic format versus face to face, and the influence of factors in the work environment could influence participants’ responses.

Future research/applications

Having a valid and reliable tool (CAT-Adm survey) to measure staff nurses’ perceptions of nurse manager caring behaviors can provide healthcare administrators with important information. For example, findings may point to differences among departments and identify those nurse managers who might benefit from caring behavior educational offerings. In addition, correlation between nurse manager caring behaviors and patient, nurse, and system outcomes could be explicated. Although not evaluated in this study, data were easily collected and analyzed using an electronic platform. Further research evaluating this approach to data capture may lead to less burdensome administration.

Healthcare administrators and nurse leaders who desire to develop a culture of caring within their institutions or who want to evaluate application of a relationship-centered professional practice model may find the reduced 25-item CAT-Adm survey useful in a preliminary examination of relationships between nurse managers and staff nurses before creating or adopting interventions. Designing interventions focused on caring behaviors can then be embedded into on-line learning modules, leadership workshops, simulation laboratory scenarios, and interactive groups. Through these types of activities, nurse managers would be given the opportunity to be engaged and to incorporate their newly learned caring behaviors into their managerial practice. This would ultimately heighten positive work environments that may positively impact nurse retention, engagement, job satisfaction, and patient outcome indicators such as patient experiences. The results can also be evaluated using individual survey items to design specific educational and leadership development programs to create a more caring culture. Together, healthcare administrators and nurse managers can use the results of CAT-Adm to increase awareness of nurse manager caring behaviors and guide leadership development. Awareness of nurse manager caring behaviors may contribute to ongoing self-development of nurse managers and ultimately positively influence staff nurses’ decisions to remain employed and contribute to the quality of the institution.

The 25-item CAT-Adm survey can be used by educators and human resource personnel involved in leadership development programs in the healthcare environment, academic institutions, and proprietary organizations. The goal would be to develop and measure program outcomes when assisting nurse managers with new employee orientation, in improving relationships with employees, and during adoption and implementation of caring-based professional practice models. In addition, adding caring content to nursing administration programs and linking the findings back to initiatives such as healthy work environments would benefit students enrolled in nursing administration programs.

Conclusion

In conclusion, this study was designed to evaluate the psychometric properties of the CAT-Adm. Although the CFA did not demonstrate an eight-factor solution as postulated, the EFA supported a one-factor solution, labeled caring behaviors. Nurse manager caring behaviors provides more clarity for the theoretical concept of relationship-centered professional encounters as documented in the QCM. Based on study findings, there is now evidence supporting acceptable validity and reliability of CAT-Adm. Furthermore, the revised CAT-Adm was reduced to 25 items while maintaining adequate internal consistency, reducing survey fatigue and administrative burden. As a result of this study, healthcare administrators, managers, educators, and researchers now have a shorter, reliable, and valid tool to evaluate nurse manager caring behaviors, an important component of healthy work environments.