Resident Perceptions of the Impact of Work Hour Limitations

Abstract

BACKGROUND

Mandatory work hour limitations for residents began in July 2003. There has been little evaluation of the impact of the new limitations on Internal Medicine residency training.

OBJECTIVE

To assess Internal Medicine residents’ perceptions of the impact of work hour limitations on clinical experiences, patient care, resident education, and well-being, and their compliance with the limitations.

DESIGN AND PARTICIPANTS

Cross-sectional survey administered to Internal Medicine residents at 1 large U.S. teaching hospital.

MEASUREMENTS

Resident perceptions using 5-point Likert scales, and self-reported compliance. Exploratory factor analysis was used to identify underlying domains and develop scales.

RESULTS

The survey response rate was 85%. Five domains were identified by factor analysis: 1) clinical experience, 2) patient care and safety, 3) communication, 4) satisfaction with training, and 5) work–rest balance. Residents perceived work hour limitations to have a negative impact on clinical experience (mean scale score 1.84, 1 = negative, 5 = positive), patient care and safety (2.64), and communication domains (1.98). Effects on satisfaction (3.12) and work–rest balance domains (2.95) were more positive. Senior residents perceived more negative effects of work hour limitations than interns. Compliance was difficult; 94% interns and 70% residents reported violating work hour limits. Patient care and teaching duties were the main reasons for work hour violations.

CONCLUSIONS

This study suggests that the current work hour limitations may be having unintended negative consequences on residency training. Ongoing monitoring to evaluate the impact of program changes as a result of work hour regulation is crucial to improving residency training.

INTRODUCTION

For the last 2 decades, the merits and dangers of the long hours required for residency training in the United States have been debated by educators, lawmakers, public safety groups, and residents themselves. The discussion has focused on the need for adequate training to ensure physician competency versus the risk to patients cared for by fatigued house officers. National concern for patient safety increased after the publication of the Institute of Medicine’s report, To Err is Human,¹ and work hour reduction was proposed as a mechanism to decrease medical errors thought to be partly caused by resident fatigue. As a result, starting in July 2003, the Accreditation Council for Graduate Medical Education (ACGME) work hour limitations went into effect for all residency programs in the United States. The limitations include a maximum of 80 hours of duty per week, 30 consecutive hours of work (24 hours of direct patient care plus 6 hours for transfer of care), a minimum of 10 hours off between shifts, and at least 1 day off in 7, averaged over 4 weeks.²

To comply with ACGME regulations, many residency programs initiated changes to their resident coverage schedules. There is controversy about the potential effects of these changes. Previous studies have found both positive and negative effects from program changes designed to reduce resident work hours, ranging from a reduction in errors in the ICU³ and a decrease in resident burnout⁴ to delays in patient care⁵ and decreased patient satisfaction.⁶ A recently published systematic review concerning effects of resident work hours found mixed results on education and job satisfaction,⁷ although the majority of studies examining Internal Medicine programs were published before work hour limitations became mandatory. Therefore, more data are needed to fully assess the effects of current work hour limitations on Internal Medicine programs.

Designing a training system that balances resident well-being with adequate training requires ongoing evaluation of residency programs changes. Evaluation at 1 year after universal implementation of work hour limitations provides a unique opportunity to survey residents who have trained under both systems. Our preliminary formative research suggested that work hour limitations had mixed effects on residency training and patient care.⁸ To further delineate the consequences of work hour limitations, we designed a survey to measure house staff attitudes and behaviors regarding work hour limitations, and their perceptions of the effects of work hour limitations on their training experience.

METHODS

We conducted a survey of Internal Medicine house staff at 1 teaching hospital in St. Louis, Missouri. The Washington University Medical Center Human Studies Committee approved the study.

Study site Barnes-Jewish Hospital is a tertiary care teaching hospital licensed for 1,385 beds, with 450 beds covered by the medicine service. Inpatient medical care is provided by a combination of private physicians, Hospitalists, and house staff. During the 2003–2004 academic year, there were 16 house staff teams covering the medicine service. Each team comprised of 1 attending physician, 1 resident, 2 interns, and 1 or 2 medical students. All house staff were: (1) limited to 80 hours of duty per week, (2) limited to 30 consecutive hours of duty, (3) required to have 10 hours off between shifts, and (4) required to have 1 day off per week. To preserve team continuity of care, interns and residents had different duty hour schedules during call and postcall days, with interns working 30 consecutive hours and residents working 2 shifts with a 10-hour break in between (Fig. 1B). Call was every fourth night, and a night float system was in place. Previous to July 2003, all house staff were scheduled for less than 80 hours of duty per week (but this was not enforced), had 1 day off per week, and took overnight call every fourth night; a similar night float system was in place (Fig. 1A).

Figure 1.

Resident Coverage System. Panel A: Call coverage system before July 1, 2003. The on-call team took overnight call and left the next day when their work was finished. There was no consecutive work hour limit. Night float interns handled overnight admissions and cross-cover for the other teams. Night float residents supervised the night float interns. Panel B: Call coverage system after July 1, 2003. The on-call resident was on duty from 7:30 a.m. until 9:30 p.m., then signed out to a night float resident. The call resident returned at 7:30 a.m., and stayed until around 5 p.m. On-call interns worked from 7:30 a.m. on the call day until 1:30 p.m.on the postcall day, a total of 30 consecutive hours. Night float residents and interns worked from 9:00 p.m. until 8:00 a.m.

Participants We recruited participants from the house staff in the Internal Medicine residency program. All house staff were eligible to participate, except for 1 involved in the study. In this report, the following definitions are used: “house staff” are all respondents, “interns” are house staff in their first year of residency (postgraduate year [PGY]-1), and “residents” are house staff in the second (PGY-2) and third (PGY-3) years of residency.

Survey Instrument Development From February through April 2004, the authors conducted focus groups of Internal Medicine house staff at Barnes-Jewish Hospital to assess the impact of work hour limitations. Focus group data were analyzed by 3 authors (GAL, DCB, JMG) for themes pertaining to residents’ perceptions of the impact of work hour limitations.⁸ Survey items and response scales were developed to assess those themes and were pilot tested on 2 faculty members and 2 obstetrics and gynecology residents to ensure clarity and item comprehension. Respondents used Likert scales to indicate their agreement with attitudinal statements (1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree or 1 = negatively, 2 = somewhat negatively, 3 = not at all, 4 = somewhat positively, 5 = positively) and frequency of activities (1 = frequently, 2 = occasionally, 3 = rarely, 4 = never). To assess compliance, respondents reported the number of times they violated each work hour rule applicable to them during their last call month (0 to >4 for working >80 hours per week; 0 to >7 for working more than 30 consecutive hours [PGY-1] or having less than 10 hours off between shifts [PGY-2, 3]; 0 to 4 for having 1 day off in 7), and also specified reasons for noncompliance. Demographic questions assessed respondents’ age, gender, year of training, and race. The final instrument contained 47 items and took approximately 20 minutes to complete.

Survey Administration The self-administered survey was distributed at a house staff conference in June 2004. Participants signed consent forms attached to the surveys, which were then separated from the survey to make responses anonymous. Nonresponders received reminder e-mails, and additional copies of the survey were placed in their mail boxes. Surveys were collected until August 2004.

Factor Analysis and Scale Development Exploratory factor analysis is a method for uncovering an underlying structure to a group of variables. We applied this method to the 25 questions in the survey pertaining to the effects of work hour limitations to uncover attitudinal domains (factors); questions about compliance were excluded. The underlying latent domains were not specified a priori. We used iterated maximum likelihood factor analysis with promax (oblique) rotation to extract the factors.⁹ An item was said to load on a given domain if the factor loading was at least 0.4. Items that loaded on multiple factors were placed on the factor that had the highest loading. Items whose highest factor loading was <0.4 were dropped. Determination of the factors to retain in the final solution was based on comparison of item loading tables for the cleanest factor structure, which we defined as item loadings above 0.4 and no or few item crossloadings. We then assessed internal consistency reliability for the factor-based scales with item–scale correlation coefficients and Cronbach’s alpha coefficients.There was less than 1% missing data for each scale; for missing data, scale scores were calculated as the mean of the nonmissing items. For consistency, all questions asked with responses on a 4-point scale were normalized to an equivalent 5-point scale with a simple algebraic conversion. Negatively worded questions were scored in reverse, so that for all questions, a score of 1 = most negative response, 3 = neutral response, and 5 = most positive response.

Analysis All data analyses were conducted using STATA 9.0. Scale scores are reported as means (SD). Analysis of variance was used to compare scores by resident training level. For comparisons with nonresponders, the chi-squared test or Fisher’s Exact test was used to compare proportions. Compliance data are reported as percentages. A 2-tailed P ≤ 0.05 was used to establish statistical significance.

RESULTS

Respondents

The survey was completed by 139 (85%) of 163 eligible residents. Demographic information is provided in Table 1. Respondents did not differ from nonrespondents by training level, age, or gender (data not shown). The most recent 4-week call block for 115 (84%) of the respondents was within 3 months of the survey; 68 (49%) of those who had their last call block within 1 month of the survey.

Table 1.

Characteristics of Study Participants

	Total* (n = 139)
Gender
Male	87 (67%)
Program level
PGY-1	52 (39%)
PGY-2	41 (31%)
PGY-3	39 (30%)
Mean age	28.4 years (SD = 2 years)
Most recent call block before survey
1 month ago	63 (48%)
2 months ago	30 (23%)
3 months ago	16 (12%)
≥3 months ago	22 (17%)

*Some participants did not supply complete demographic data.

Scales

Nineteen of the 25 items included in the factor analysis were retained. From these items, 5 domains were identified: (1) clinical experience (6 items), (2) patient care and safety (3 items), (3) communication (3 items), (4) satisfaction with training (5 items), and (5) work–rest balance (2 items). Table 2 lists the individual items for each scale, along with their factor loadings. The high Cronbach’s alphas indicate overall good internal consistency reliability of the scales. Mean scores (SD) for each scale, stratified by program year, are shown in Table 3. Higher scores indicated a more positive response to the work hour limitations. Raw scores for each individual item are presented in Appendix.

Table 2.

Factor Analysis Results*

Factors analyzed	Factor loading
Clinical experience (Cronbach’s alpha = 0.79)
Compliance with the work hour rules resulted in delays in reviewing my patient’s test results†	0.8799
Compliance with the work hour rules limited the thoroughness of my patient workups†	0.7247
Compliance with the work hour rules limited the time I had to teach other team members†	0.5409
Too much attention is directed at complying with the work hour rules†	0.5022
I am able to provide a level of care to my patients that satisfied me.‡	0.4716
How often did delays in patient care occur due to multiple physician handoffs?§	0.4603
Patient care and safety (Cronbach’s alpha = 0.81)
How does restricting resident work hours affect patient safety?§	0.8428
How does restricting resident work hours affect patient care?§	0.8034
Doing what is best for my patients and complying with the work hour rules often conflict.†	0.4376
Communication (Cronbach’s alpha = 0.66)
How often did consult services have difficulty locating the primary team?¶	0.6866
How often did nurses have difficulty loading the appropriate member of the primary team?¶	0.6860
How often did work hour limitations impair communication between physicians and patients/families?¶	0.5393
Satisfaction with Training (Cronbach’s alpha = 0.77)
How does restricting resident work hours affect your job satisfaction?‡	0.7481
Compliance with the work hour rules has limited my ability to learn what I need to know to be an excellent physician.†	0.7023
Compliance with the work hour rules limited my attending physician’s ability to teach my team.†	0.4510
Restricting resident work hours is a good idea.‡	0.4404
How does restricting resident work hours affect resident education?‡	0.4177
Work-Rest Balance (Cronbach’s alpha = 0.61)
I have adequate time away from the hospital.‡	1.0253
I am well rested during my call months on the wards.‡	0.4179

*Factor loadings >0.4 were included. Higher factor loadings indicate a stronger correlation with the underlying domain.

^†Scored “Strongly agree” = 1, “Agree” = 2, “Disagree” = 3, “Strongly Disagree” = 4

^‡Scored “Strongly disagree” = 1, “Disagree” = 2, “Agree” = 3, “Strongly Agree” = 4

^§Scored “Negatively” = 1, “Somewhat negatively” = 2, “Not at all” = 3, “Somewhat positively” = 4, “Positively” = 5

^¶Scored “Frequently” = 1, “Occasionally” = 2, “Rarely” = 3, “Never” = 4

Table 3.

Mean* (SD) Scale Scores by Year of Training

Scale	Overall	PGY-1 (N = 52)	PGY-2 (N = 41)	PGY-3 (N = 39)	P value†
Clinical experience	1.84 (0.55)	2.03 (0.46)	1.81 (0.56)	1.61 (0.50)	<.001
Patient care and safety	2.64 (0.88)	3.04 (0.89)	2.43 (0.83)	2.24 (0.61)	<.001
Communication	1.98 (0.54)	1.96 (0.40)	1.84 (0.62)	2.15 (0.56)	.03
Satisfaction with training	3.12 (0.74)	3.38 (0.64)	3.07 (0.80)	2.76 (0.62)	<.001
Work–rest balance	2.95 (0.80)	3.43 (0.73)	2.70 (0.60)	2.57 (0.66)	<.001

*Mean scores on a 5-point Likert scale: 1 = most negative response; 3 = neutral response; 5 = most positive response.

^†Comparison between interns (PGY-1) and residents (PGY-2 and PGY-3)

Clinical Experience

The mean score for this scale was 1.84 (0.55), indicating negative effects of compliance with work hour limitations on clinical experience. House staff felt that the limitations led to delays in care, limited workups, and limited teaching time. Residents had a significantly more negative view of the impact of the limitations on their clinical experience than interns (Table 3).

Patient Care and Safety

House staff perceived negative effects on patient care and safety, with a mean score of 2.64 (0.88). The negative effects included feeling conflict between providing optimal patient care and complying with the work hour limitations. There was a significant difference based on the level of the resident. Residents had significantly more negative views of the effects of work hour limitations on patient care and safety than interns (Table 3).

Communication

All levels of residents reported difficulties with communication as a result of work hour limitations. The mean score for this scale was 1.98 (0.54), indicating frequent communication problems with nurses, other physicians, and patients. Interns and PGY-2 residents reported more difficulties with communication than PGY-3 residents (Table 3).

Satisfaction with Training

The overall mean score for this scale was 3.12 (0.74), suggesting that house staff had relatively neutral feelings about the effects of work hour limitations on their training. However, there were marked differences by program year. Interns perceived more positive effects of the limitations on this domain than did residents, with PGY-3 residents having the lowest mean score (Table 3). The questions in the scale relating to job satisfaction had much higher mean scores (3.83–3.92) than those asking about effects on education (2.18–3.12; Appendix), both important components of overall satisfaction.

Work–Rest Balance

We measured work–rest balance as an indicator of resident well-being by asking about fatigue during call months and having adequate time away from the hospital. The overall score for this scale was 2.95 (0.80). Interns perceived a significantly better work–rest balance than residents (Table 3).

Compliance with Work Hour Restrictions

Compliance with work hour limitations varied by rule and by training level (Table 4). Of the noncompliant house staff, 21 (41%) interns and 27 (47%) residents reported violations of the 30-hour and 10-hour limits of greater than 60 minutes. The most common reason for noncompliance was patient care responsibilities (PGY-1 85%, PGY-2 63%, PGY-3 56%). Interns were significantly more likely than residents to cite attending rounds as a major barrier to compliance (62% vs 5%, p < .001), whereas residents were much more likely to cite teaching responsibilities (60% vs 6%, p < .001). Interns were also significantly more likely to report educational conferences as a barrier to compliance (38% vs 4%, p < 0.001).

Table 4.

Percentage of Residents Reporting ≥1 Violations During a 4-week Block

Violation	PGY-1 (N = 52)	PGY-2 (N = 41)	PGY-3 (N = 39)
Worked >80 hours per week	59%	27%	30%
Had <4 days off per month	6%	17%	5%
Worked >30 consecutive hours*	94%	N/A	N/A
Had <10 hours off between shifts*	N/A	73%	67%

*PGY-1 interns were limited to 30 consecutive hours during one call shift, whereas PGY-2 and PGY-3 residents were to have 10 hours off between the end of the call night and the beginning of the postcall morning.

N/A = not applicable

DISCUSSION

There is mounting evidence that the current ACGME work hour limitations have led to some unintended consequences for residency training. Our findings suggest that whereas house staff perceive work hour limitations as having some positive effects on their well-being, they struggle to complete patient care responsibilities and teaching duties within the restricted hours.

Evaluating the impact of program changes is crucial to improving residency training. We developed scales to measure house staff perceptions of the impact of work hour limitations on several aspects of residency training. These metrics were developed in stepwise fashion, beginning with focus groups to identify issues important to house staff regarding work hour limitations, building survey items to characterize these issues, and finally using factor analysis to identify the underlying domains. This approach ensured that we assessed the areas of residency training that house staff felt were most affected by work hour limitations, increasing the validity of our findings.

House staff perceived negative effects of work hour limitations on their clinical experience. Senior residents, in particular, felt that the pressure to comply with work hour limits led to decreased thoroughness of workups and delays in patient care, which adversely affected their satisfaction with the level of care they provided. Furthermore, house staff reported direct patient care and teaching duties as the main reasons for noncompliance, suggesting that the workload for house staff may be too heavy for the current permitted work hours. A previous review of house staff work activities found that house staff spend up to 35% of their time on activities of marginal or no educational value.¹⁰ Transfer of these tasks to other providers is often proposed as a solution^11–13; however, shifting excess work from house staff is potentially very expensive, with estimates in the hundreds of millions to billions of dollars.^14–16 As there is little additional funding to offset these costs, it may be difficult for hospitals to afford these changes. In addition, programs must carefully implement workload reduction to ensure that the patient care and educational experiences required for residents to become competent physicians are not compromised.

House staff in our survey reported that they felt compliance with work hour limitations may have adversely affected patient care and safety. This is consistent with other studies, which have also shown the possibility of an increased risk of adverse events with work hour limits, primarily because of fragmentation in patient care.^17,18 Furthermore, inadequate supervision and problems with transfer of care between physicians may be as important as excessive work hours in causing mistakes.¹⁹ Our data, together with previous studies, suggest that any decrease in errors resulting from decreased fatigue may be at least partially offset by delays in patient care, discontinuity of care, and communication issues. Objective data are needed to determine whether the negative effects perceived by house staff have in fact affected patient outcomes.

Satisfaction with training is in part driven by satisfaction with the learning environment. Factors enhancing learning, such as contact with the attending physician, patient rounds, and seminars, are positively correlated with satisfaction with internship, whereas factors contributing to perceived mistreatment, including sleep deprivation, are negatively correlated.²⁰ Although house staff at the study institution were moderately satisfied with their training program in the setting of work hour limitations, the scores for the education questions for this scale were lower than those for questions about job satisfaction. This difference may indicate that any gains in job satisfaction made by decreasing sleep deprivation may be offset by disappointment caused by fewer learning opportunities.

Perceptions regarding the impact of work hour limitations varied by year of training. Senior residents were more likely to perceive problems with clinical experiences and patient care and safety, and to be less satisfied with indicators of resident well-being. These differences may reflect discontent with how the work hour limitations were implemented, frustration with a mid-residency change, or dissatisfaction with having to assume an increased share of the work previously done by interns. Conversely, senior residents have increased experience and responsibility, which may lead to a better understanding of the impact of work hour limitations on patient care and education, both of which may ultimately affect satisfaction with training. Follow-up studies are needed to determine whether the difference in perception between residents and interns will persist beyond the period of transition to the new system.

Compliance with work hour limitations in this study was poor. Although Internal Medicine programs had guidelines specifying an 80-hour work week and 1 day off per week before mandatory ACGME rules went into effect, the 30 consecutive hour and 10-hour limits were new. House staff were least compliant with these 2 rules, likely because patient care rarely followed such a rigid schedule. Whereas most violations were minor, over 40% of house staff had violations of greater than 60 minutes. Thus, attempting to comply with the limitations may be an additional stressor that could offset gains in resident well-being.

There was a high response rate to the survey, making it more likely that the opinions expressed in the survey are representative of the Internal Medicine house staff at the study institution. In addition, we were able to capture the sentiments of house staff who made the transition from 1 system to another, capturing valuable insight into the benefits and shortcomings of both systems. Although our findings are consistent with those seen in surveys from other disciplines,^21–23 this was a single center study of Internal Medicine house staff at an academic medical center during a period of transition, which may limit its generalizability. Although we are not aware of any other concurrent significant changes in the hospital and residency program environments during the study period, such changes would affect house staff experiences and confound our results. Additionally, we relied on self-report to assess behaviors, a limitation of survey methodology. Other studies have demonstrated that self-report overestimates compliance with desired behaviors,^24,25 suggesting that our findings may underestimate the impact of the limitations on those behaviors. We measured resident perceptions, not direct patient care or educational outcomes. These self-perceptions are subject to bias, particularly with senior residents, who were likely affected by prior experiences. Finally, difficulties with compliance, which may reflect local program structure, may have affected house staff attitudes toward the limitations.

Improvement in patient care and safety and in the education and well-being of house staff are the ultimate goals of changing the system of residency training. Our study indicates that the current ACGME work hour limitations have complex and perhaps unintended consequences on the residency training system, and demonstrates the need for ongoing evaluation to assess the impact of program changes on residency training.

Appendix

Table 5.

Mean (SD) Individual Question Scores

	Overall	PGY-1 (N = 52)	PGY-2 (N = 41)	PGY-3 (N = 39)
Clinical experience
Compliance with the work hour rules resulted in delays in reviewing my patient’s test results.*	2.39 (0.71)	2.52 (0.64)	2.41 (0.74)	2.15 (0.67)
Compliance with the work hour rules limited the thoroughness of my patient workups.*	2.44 (0.74)	2.71 (0.67)	2.36 (0.73)	2.12 (0.70)
Compliance with the work hour rules limited the time I had to teach other team members.*	2.26 (0.92)	2.80 (0.74)	2.13 (0.93)	1.68 (0.74)
Too much attention is directed at complying with the work hour rules.*	2.22 (0.88)	2.30 (0.72)	2.17 (0.85)	2.23 (0.98)
I am able to provide a level of care to my patients that satisfied me.§	3.33 (0.72)	3.46 (0.67)	3.26 (0.73)	3.22 (0.70)
How often did delays in patient care occur owing to multiple physician handoffs?†	1.60 (0.68)	1.60 (0.53)	1.41 (0.77)	1.74 (0.72)
Patient Care and Safety
How does restricting resident work hours affect patient safety?‡	2.81 (1.09)	3.29 (1.15)	2.55 (0.99)	2.36 (0.84)
How does restricting resident work hours affect patient care?‡	2.56 (1.02)	2.98 (1.09)	2.37 (0.92)	2.10 (0.72)
Doing what is best for my patients and complying with the work hour rules often conflict.*	2.56 (0.97)	2.89 (0.96)	2.36 (1.01)	2.26 (0.80)
Communication
How often did consult services have difficulty locating the primary team?†	1.80 (0.80)	1.74 (0.66)	1.65 (0.88)	2.05 (0.79)
How often did nurses have difficulty loading the appropriate member of the primary team?†	2.20 (0.67)	2.17 (0.59)	2.10 (0.77)	2.30 (0.66)
How often did work hour limitations impair communication between physicians and patients/families?†	1.94 (0.61)	1.96 (0.53)	1.75 (0.66)	2.10 (0.64)
Satisfaction with Training
How does restricting resident work hours affect your job satisfaction?‡	3.63 (1.18)	4.09 (0.89)	3.46 (1.29)	3.05 (1.19)
Compliance with the work hour rules has limited my ability to learn what I need to know to be an excellent physician.*	3.12 (0.93)	3.19 (0.95)	3.19 (0.98)	2.88 (0.79)
Compliance with the work hour rules limited my attending physician’s ability to teach my team.*	2.75 (1.03)	2.80 (1.06)	2.13 (0.90)	1.68 (1.01)
Restricting resident work hours is a good idea.§	3.92 (0.75)	4.18 (0.66)	3.79 (0.89)	3.69 (0.65)
How does restricting resident work hours affect resident education?‡	2.18 (1.13)	2.58 (1.13)	1.90 (1.07)	1.87 (1.06)
Work-Rest Balance
I have adequate time away from the hospital.§	2.83 (0.85)	3.23 (0.90)	2.53 (0.71)	2.60 (0.69)
I am well rested during my call months on the wards.§	3.07 (1.02)	3.64 (0.86)	2.86 (0.89)	2.53 (0.89)

*Scored “Strongly agree” = 1, “Agree” = 2, “Disagree” = 3, “Strongly disagree” = 4

^†Scored “Frequently” = 1, “Occasionally” = 2, “Rarely” = 3, “Never” = 4

^‡Scored “Negatively” = 1, “Somewhat negatively” = 2, “Not at all” = 3, “Somewhat positively” = 4, “Positively” = 5

^§Scored “Strongly disagree” = 1, “Disagree” = 2, “Agree” = 3, “Strongly Agree” = 4