Association of Employee Attributes and Exceptional Performance Rating at a National Center of the U.S. Centers for Disease Control and Prevention, 2011

Abstract

Context

Employee performance evaluation motivates and rewards exceptional individual performance that advances the achievement of organizational goals. CDC and its operating units evaluate employee performance annually and reward exceptional performance with a cash award or quality step increase in pay. A summary performance rating (SPR) of “exceptional” indicated personal achievements in 2011 that were beyond expectations described in the employee's performance plan.

Objective

To determine if personal attributes and job setting of civil service employees were associated with an exceptional SPR in NCHHSTP in 2011.

Design

Data from the CDC 2011 performance management database collected in 2012 were analyzed in 2013 to identify SPR, personal attributes, job-settings of full-time civil service employees. Multivariate logistic regression controlled for confounding and stratified analysis detected effect modifiers of the association between receiving an exceptional SPR in 2011 and gender, race/ethnicity, education, job location, job series, grade level, years in grade, years of federal service, supervisory role, and NCHHSTP division.

Results

Among the 1,037 employees, exceptional SPR was independently associated with: female gender (aOR: 1.7 [1.3,2.3]), advanced degrees (Doctorate: 1.7 [1.1,2.5]) Master's: 1.1, 2.0]), headquarters location (2.8 [1.9, 4.1]), higher pay grade (3.3 [2.4,4.5]) and years in grade (0-1yrs: 1.7 [1.3,2.4]; 2-4yrs: 1.5 [1.1,2.0]), division level (Division A: 5.0 [2.5,9.9]; Division B: 5.5 [3.5, 8.8]), and supervisory status (at a lower pay grade) (OR: 3.7 [1.1, 11.3]).

Conclusions

Exceptional SPR is independently associated with personal employee attributes and job-settings that are not modifiable by interventions designed to improve employee performance based on accomplishments.

Introduction

Employee performance evaluation motivates and rewards exceptional individual performance that advances the achievement of organizational goals. The Centers for Disease Control and Prevention (CDC) and its operating units evaluate employee performance annually and reward exceptional performance with a cash award or quality step increase in pay. During 2006 – 2011, the CDC Performance Management Appraisal Program (PMAP) was used to evaluate the work performance of CDC employees by focusing on results-oriented performance objectives related to the agency's goals.¹ PMAP covers all full time employees of the Civil Service and Title 42 Service Fellows. PMAP excludes CDC employees who are members of the Senior Executive Service (SES), the U.S. Public Health Service Commissioned Corps, Senior Biomedical Research Service, and Distinguished Consultants. Each employee covered by PMAP receives an SPR that is based on two supervisors’ joint assessment of the employee's individual accomplishments on 5-7 performance elements (activities) during the preceding 12 months (January – December). Individual accomplishments are noteworthy results of activities that advance the mission of the employee's work unit. Performance ratings fall into one of four categories as follows: Unacceptable, Minimally Successful, Fully Successful, and Exceptional. Employees who received an Exceptional rating were eligible for cash awards ranging from 2.5% to 5.0% of their annual salary, depending on the availability of funds. Employees who received a rating of Fully Successful were eligible for a performance cash award up to 2.0% of their annual salary. In 2009, CDC released a report on the distribution of SPR by employee attributes for calendar years 2006, 2007, and 2008. The study population excluded employees on Administratively Determined (AD) pay plans. During 2006-2008, approximately 7,600 employees received SPRs each year.¹ That study found that exceptional SPR was associated with supervisory role, higher pay grades, female gender, and white race.

In 2011, The National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention (NCHHSTP), one of the larger centers at CDC, was responsible for public health surveillance, prevention research, programs to prevent and control HIV and AIDS, other STDs, viral hepatitis, and TB. NCHHSTP had a budget of $1 billion, and a workforce of more than 1,800 full-time employees and contractors, including approximately 300 who are assigned to state and local health departments in the United States. NCHHSTP included an Office of the Director (OD) and four divisions (referred to as Divisions A-E). Since 2009, results from the Federal Employee Viewpoint Survey and focus groups commissioned by the NCHHSTP have raised concerns about the effectiveness and fairness of the CDC performance rating system². Fairness (equity) requires that the distribution of awards be unbiased by employee demographic attributes or job prestige attributes that are unrelated to accomplishment and high performance. One objective in the 2010-2015 NCHHSTP workforce development and capacity building strategic plan, is to “continuously recognize performance, contributions, and achievements of staff”.³ Achievement of this objective relies on an incentive award program that can be shown to be fair and effective with available data.

Performance ratings are expected to reflect the individual accomplishments of employees. In 2014, it is unknown if the findings from the 2006-2008 CDC-wide study were generalizable to each center, office, and institute that is part of the CDC or if the distribution of exceptional ratings by employee attributes has changed since 2008. Knowing whether personal attributes and job-setting of employees are associated with an exceptional SPR can help to inform efforts to improve the effectiveness and reliability of the PMAP in NCHHSTP. A fair and effective Incentive Awards Program is essential to develop the NCHHSTP workforce and to build its capacity to sustain a high level of performance in pursuit of the center's mission. The purpose of the study was to determine if personal employee attributes and job settings were associated with receiving an Exceptional PMAP rating within CDC/NCHHSTP in 2011.

The findings of this study will provide a baseline status of NCHHSTP's employee performance in the final year of the federal department-wide performance evaluation plan that was implemented during 2006 – 2011.⁴ Starting with the 2012 performance evaluation year, a new federal department-wide performance evaluation plan was implemented. The 2011 baseline analysis provides a basis of comparison for evaluating the impact of the new performance evaluation plan on determinants of the highest rating level (exceptional in 2011; outstanding in 2012). Continual monitoring and periodic reporting (every one or two years) can describe changes in the distribution of PMAP ratings within NCHHSTP in response to changes in policy, practice, and incentive award budgets over time. The results of this study can be used to assess the impact of ongoing efforts within NCHHSTP to ensure a more equitable distribution of exceptional performance ratings across categories of employees.

This report describes the creation of the data analysis file from centralized personnel records, modification of existing variables to enable statistical analyses, statistical analysis and interpretation of the findings in advancing NCHHSTP's employee performance goals and system of rewards for exceptional performance. Because the authors used deidentified data collected to support program administration, independent CDC officials determined that the study did not meet the federal definition of human research that requires institutional review board approval.

Methods

Data acquisition study design, and dataset creation

In December 2012, in response to a written request that included a project justification and appropriate authorization, the Workforce Engagement Branch in the Human Capital and Resources Management Office (HCRMO) extracted an analysis file from the CDC PMAP administrative database. The database included data derived from one or more U.S. Government Systems of Records covered by the Privacy Act (42 U.S.C. 552a). HCRMO selected NCHHSTP/CDC civil service employees with 2011 PMAP ratings from the sampling frame of all such CDC employees in the PMAP database in December 2012. NCHHSTP employees and their Divisions were identified by Office of Personnel Management administrative codes. The analysis file, in Microsoft Excel format, contained one record for each NCHHSTP civil service employee who had received a 2011 PMAP rating during January 2012. Each employee record included the following variables: administrative code, pay plan, grade, years-in-grade, PMAP summary rating, race/ethnicity, supervisory code, education, gender, job series, duty city and state.

Using a cross-sectional study design, the authors analyzed data on selected personal attributes, job attributes and performance ratings of 1,037 NCHHSTP/CDC civil service employees who had received a 2011 PMAP rating during January 2012.Similarly, data for comparing the attributes of NCHHSTP employees who received 2011 PMAP ratings with the 8,234 other CDC employees who worked in other CDC organizational units outside of NCHHSTP were obtained from the same source. .

Measures of employee attributes and PMAP ratings

Dependent variable (exceptional rating in 2011)

For the 2011 calendar year, the employee's direct supervisor determined a performance rating by comparing actual with expected accomplishments defined in the employee's performance plan at the start of the performance period. That rating is then reviewed by another person one level up in the leadership of the work unit to produce a consensus rating based on the joint assessment of the employee's individual accomplishments on 5-7 performance elements (activities) during the preceding 12 months (January – December 2011). Individual accomplishments are noteworthy results of activities that advance the center's mission. Performance ratings fall into one of four categories as follows: Unacceptable, Minimally Successful, Fully Successful, and Exceptional.

Independent variables (employee attributes)

Employee characteristics included in the analysis were gender, race/ethnicity, education level, geographic location, grade level, years in grade, job series, years of federal service and supervisory status. The variables were recoded from their native categories to new categories as follows: gender (female, male); race/ethnicity (white, Asian/PI, Hispanic, American Indian, black: no observations were excluded); educational Level (less than Masters level, Masters level, and Doctorate level); job location (headquarters, field staff); job series (scientists, other series); pay plan and grade level (GS/GP04-13 or AD00, GS/GP14-15: job grades reflect increasing levels of knowledge, skill, and responsibility associated with positions; grades 4-12 and AD00 are not managers; 13-15 are mid-level managers); years in grade: (0 – 1 years, 2 – 4 years, and 5+ years); supervisory status: (yes, no); years of federal service (1-10, 11-20, 21+ years); and Divisions: A, B, C, D, and E inside NCHHSTP.

Statistical Analysis

SAS version 9.3 was used to calculate the unadjusted proportion (with 95% confidence interval) of eligible employees who received an exceptional SPR in 2011 for each category of employee within strata, uniquely defined by select personal and job-related attributes as follows: Division A-E in NCHHSTP, Gender, Race/Ethnicity, Education Level, Geographic Location, GS Level, Years-in-Grade, Years in Government Service, Job Series, and Supervisory Status. The strength of association between receiving an exceptional SPR and employee attributes was assessed using stratified analyses and multivariate logistic regression. Bivariate analyses were conducted for exceptional SPR and each independent variable using Chi-square test to identify significant (p<0.05) differences between sub-groups defined by strata within each independent variable.

Next, logistic regression analyses were used to estimate both unadjusted odds ratios (OR) and adjusted odds ratios (AOR) for associations of exceptional SPR with some of the independent variables listed previously. A series of 3-factor logistic regression models was constructed including pay grade and job series as potential confounders and each employee attribute as the primary independent variable of interest. When supervisory status and divisions were examined for their associations with exceptional SPR, 4-factor logistic regression models were constructed with pay grade, job series, and job location as potential confounders in each model. To determine if significant covariates of SPR for all NCHHSTP employees were the same within each Division and among field employees only, comparisons of SPR prevalence by selected covariates and logistic regression models within those subgroups of employees were constructed. For each independent variable, the category with the lowest percentage of SPR served as the reference category. In each model, exceptional SPR served as the dependent variable. Odds ratios were considered statistically significant if p<0.05. To determine if the variables “years of government service” and “years in grade” were multicollinear in logistic regression models that included both variables, we examined the estimates of odds ratios and their standard errors for statistical stability when one variable was added after the other in the model: logit = Int + Yrs-in-Grade + Yrs-of-Gov-Service + GS Level. Finally, effect modification was assessed by stratified analysis between supervisory status and grade level to determine whether one variable modified the associations of the other with exceptional SPR.

The Wald Goodness-of-fit test was conducted to determine the statistical significance of each parameter estimate, interaction terms, and subsequent adjusted odds ratio estimates. Wald statistics having p-values less than 0.05 were considered to be statistically significant. A test for multicollinearity was necessary due to the overlap in the distributions of years of federal service and years in grade; multicollinearity was not present in models containing both variables.

Results

In 2011, 1,037 NCHHSTP employees received a PMAP evaluation. Missing values, whose numbers varied by variable, were excluded from the calculation of column percentages in the tables. Of these employees, 537 (52%) received an exceptional SPR; 64% were female; 50% were white and 35% black; 38% had masters and 38% had less than masters degrees; 83% were located at CDC headquarters in Atlanta; 50% were in other series and 50% in scientific job series; 73% were in GS/GP04-13 or AD00 job series; 41 % were in that grade more than 4 years and 33% were 2-4 years in grade; by division: 40% were in C, 25% in B, 16% in E, 12% in D and 6% in A.

Compared to employees who received 2011 PMAP ratings and worked in other CDC organizational units, NCHHSTP employees were significantly more likely to be non-white (49.9% vs 37.9%; p <0.001); to have doctorate or masters degrees (62.4% vs 52.2%; p <0.001); to work at headquarters (83.2% vs 76.7%; p<0.001); to be scientists (49.7% vs 31.9%; p <0.001); to be in lower grade levels (72.9% vs 71.2%; p <0.001); to be non-supervisory (89.4% vs 85.4%; p = 0.001) (Table 1A). Compared to employees who received exceptional 2011 PMAP ratings and worked in other CDC organizational units, NCHHSTP employees who received exceptional 2011 PMAP ratings were significantly more likely to be women (67.2% vs 62.4%; p = 0.03); to be non-white (44.2% vs 32.2%; p<0.001); to have doctorate or masters degrees (69.7% vs 53.7%; p <0.001); to work at headquarters (90.5% vs 73.8%; p<0.001); to be a non-supervisor (84.7% vs 80.6%; p=0.021) (Table 1B).

Table 1A.

Number and percentage of employees who received any 2011 PMAP rating in National Center for HIV/AIDS, viral Hepatitis, Sexually Transmitted Diseases and Tuberculosis Prevention and Other Centers for Disease Control and Prevention Organizational Units who had selected personal and job-related attributes, 2011

	NCHHSTP		Other CDC Organizational Units
Personal and job-related attributes	N	Column Percent	N	Column Percent	Chi-Square P-value
Gender
Women	648	63.7	5,080	61.7
Men	370	36.3	3,154	38.3	0.23
Race
White	464	50.1	4,795	62.1
Hispanic	50	5.4	235	3.0
Asian/PI	86	9.3	625	8.1
Black	327	35.3	2,069	26.8	<0.001
Educational Level
Doctorate	223	24.8	1511	18.7
Masters	338	37.6	2697	33.5
<Masters	339	37.7	3854	47.8	<0.001
Location
Headquarters	863	83.2	6,093	76.7
Field	174	16.8	1,851	23.3	<0.001
Job Series
Scientist	515	49.7	2,624	31.9
Other	522	50.3	5,610	68.1	<0.001
*GS/GP Level
GS/GP14-15	281	27.1	2,354	28.8
GS/GP04-13, AD00	756	72.9	5,810	71.2	<0.001
Supervisory Status
Supervisor	110	10.6	1,205	14.6
Non-supervisor	927	89.4	7,029	85.4	<0.001
Total	1,037	100	8,234	100

^*GS – General Schedule; GP – General Physician

Table 1B.

Number and percentage of employees who received exceptional 2011 PMAP rating in National Center for HIV/AIDS, viral Hepatitis, Sexually Transmitted Diseases and Tuberculosis Prevention and Other Centers for Disease Control and Prevention Organizational Units who had selected personal and job-related attributes, 2011

	NCHHSTP		Other CDC Organizational Units
Personal and job-related attributes	Exceptional (N)	Column Percent	Exceptional (N)	Column Percent	Chi-Square P-value
Gender
Women	356	67.2	2665	62.4
Men	174	32.8	1609	37.6	0.03
Race
White	274	55.8	2749	67.8
Hispanic	29	5.9	132	3.3
Asian/PI	47	9.6	307	7.6
Black	141	28.7	868	21.4	<0.001
Educational Level
Doctorate	143	30.3	826	19.6
Masters	186	39.4	1435	34.1
<Masters	143	30.3	1948	46.3	<0.001
Location
Headquarters	486	90.5	3018	73.8
Field	51	9.5	1074	26.3	<0.001
Job Series
Scientist	293	54.6	2304	53.9
Other	244	45.4	1970	46.1	0.77
*GS/GP Level
GS/GP14-15	203	37.8	1495	35.3
GS/GP04-13, AD00	334	62.2	2741	64.7	0.25
Supervisory Status
Supervisor	82	15.3	830	19.4
Non-supervisor	455	84.7	3444	80.6	0.02
Total	537	100.0	4274	100.0

^*GS – General Schedule; GP – General Physician

In the bivariate analysis of the distribution of exceptional SPR among NCHHSTP employees by select attributes, the percentage of employees who received an exceptional SPR in 2011 was lowest among men (47%), black employees (43%), employees with less than a master's degree, in positions in field locations away from Atlanta headquarters (29%), in other job series (47%), in GS/GP04-13 and AD00 grade levels (44%), in grade more than 4 years (46%), and in the division E (33%) (Table 2); and lowest among non-supervisors in GS/GP04-13 (44%) and GS/GP14-15 grade levels (71%), respectively (Table 3).

Table 2.

Association of personal and job-related attributes of National Center for HIV/AIDS, viral Hepatitis, Sexually Transmitted Diseases and Tuberculosis Prevention employees with exceptional 2011 summary Performance rating (unadjusted and adjusted Odds Ratios with 95% confidence intervals), 2011

Personal and job-related attributes	Exceptional N=537	Not Exceptional N=500	Unadjusted	Adjusted
	N (%)	N (%)	OR(CI)*	OR(CI)†
Gender
Women	356(54.9)	292(45.1)	1.4 (1.1, 1.8)	1.7 (1.3, 2.3)
Men	174(47.0)	196(53.0)	(ref)	(ref)
Race
White	274(59.1)	190(41.0)	1.9 (1.4, 2.5)	1.4 (1.0, 1.9)
Hispanic	29(58.0)	21(42.0)	1.8 (1.0, 3.3)	1.7 (0.9, 3.3)
Asian/PI	47(54.7)	39(45.3)	1.6 (1.0, 2.6)	1.1 (0.6, 2.0)
Black	141(43.1)	186(56.9)	(ref)	(ref)
Educational Level
Doctorate	143(64.1)	80(35.9)	2.3 (1.7, 3.2)	1.7 (1.1, 2.5)
Masters	186(55.0)	152(45.0)	1.6 (1.2, 2.1)	1.5 (1.1, 2.0)
<Masters	143(42.2)	196(57.8)	(ref)	(ref)
Location
Headquarters	486(56.3)	377(43.7)	3.1 (2.2, 4.4)	2.8 (1.9, 4.1)
Field	51(29.3)	123(70.7)	(ref)	(ref)
Job Series
Scientist	293(56.9)	222(43.1)	1.5 (1.2, 1.9)	1.1 (0.9, 1.5)
Other	244(46.7)	278(53.3)	(ref)	(ref)
**GS/GP Level
GS/GP14-15	203(72.2)	78(27.8)	3.4 (2.5, 4.6)	3.3 (2.4, 4.5)
GS/GP04-13, AD00	334(44.2)	422(56.8)	(ref)	(ref)
Years In Grade
0 - 1	145(54.7)	120(45.3)	1.4 (1.1, 2.0)	1.7 (1.3, 2.4)
2 - 4	197(57.3)	147(42.7)	1.6 (1.2, 2.1)	1.5 (1.1, 2.0)
5+	195(45.6)	233(54.4)	(ref)	(ref)
Years of Federal Service
1 - 10	230(52.2)	211(47.8)	1.3 (1.0, 1.7)	1.5 (1.1, 2.1)
11 - 20	143(59.6)	97(40.4)	1.7 (1.2, 2.4)	1.6 (1.1, 2.2)
21+	162(45.9)	191(54.1)	(ref)	(ref)
Divisions‡
A	48(76.2)	15(23.8)	6.4 (3.4, 12.1)	5.0 (2.5, 9.9)
B	126(76.4)	39(23.6)	6.5 (4.2, 10.0)	5.5 (3.5, 8.8)
C	218(51.5)	205(48.5)	2.2 (1.5, 2.9)	1.5 (1.0, 2.2)
D	57(46.7)	65(53.3)	1.8 (1.1, 2.7)	1.1 (0.7, 1.9)
E	88(33.3)	176(66.7)	(ref)	(ref)

^*Unadjusted Odds Ratio of an exceptional PMAP Rating by demographic groups

^**GS – General Schedule; GP – General Physician

^†Adjusted Odds Ratio of an exceptional PMAP Rating by demographic groups using 3-factor logistic regression models: Log-Odds(Exceptional) = Int + demographic variable + GS Level variable + job series variable

^‡Adjusted Odds Ratio of an exceptional PMAP Rating by demographic groups using 4-factor logistic regression models: Log-Odds(Exceptional) = Int + demographic variable + GS Level variable + job series variable + geographic location variable

Table 3.

Association of grade adjusted for supervisory status of National Center for HIV/AIDS, viral Hepatitis, Sexually Transmitted Diseases and Tuberculosis Prevention employees with exceptional 2011 summary performance rating (unadjusted and adjusted Odds Ratios with 95% cConfidence iIntervals), 2011

		Exceptional N=537	Not Exceptional N=500	Unadjusted
		N (%)	N (%)	OR(CI)
Grade	Supervisory
GS/GP 14-15	Supervisor	71(74.7)	24(25.3)	1.2(0.7,2.1)
	Non-Supervisor	132(71.0)	54(29.0)	(ref)
GS/GP GS 4-13, AD00	Supervisor	11(73.3)	4(26.7)	3.7(1.1,11.3)
	Non-Supervisor	323(43.6)	418(56.4)	(ref)

* Unadjusted Odds Ratio of an exceptional PMAP Rating by demographic groups

In the logistic regression models, statistically significant positive associations were found between exceptional SPR and each of the following employee attributes: female gender [AOR (95%CI) = 1.7 (1.3,2.3)]; doctorate degree [AOR (95%CI) = 1.7 (1.1,2.5)]; master's degree [AOR (95%CI) = 1.5 (1.1,2.0)]; headquarters location [AOR (95%CI) = 2.8 (1.9,4.1)]; GS/GP-14-15 grade level [AOR (95%CI) = 3.3 (2.4,4.5)]; 0-1 years in grade [AOR (95%CI) = 1.7 (1.3,2.4)]; 2-4 years in grade [AOR (95%CI) = 1.5 (1.1,2.0)]; 1-10 years of federal service [AOR (95%CI) = 1.5 (1.1-2.1)]; 11-20 years of federal service [AOR (95%CI) = 1.6 (1.1-2.2)]; working in Division A [AOR (95%CI) = 5.0 (2.5,9.9)]; or Division B [AOR (95%CI) = 5.5 (3.5,8.8)]; (Table 2). Despite the overlap in the distributions of years of federal service and years in grade estimates of the standard errors and adjusted odds ratios derived from models containing both variables were statistically stable indicating the absence of multicollinearity involving those variables. The positive association of exceptional SPR with supervisory role was modified by grade level as follows: only at the GS/GP04-13 and AD00 grade levels were supervisors (OR: 3.7 [1.1, 11.3]) more likely to receive an exceptional SPR than were non-supervisors. The results of sub-group analyses within Divisions and among field employees reinforced but did not change or further enlighten the interpretation of the significant positive associations between exceptional SPR and employee attributes described previously.

Discussion

This study of NCHHSTP employees who received PMAP ratings in 2011 shows that exceptional SPR is independently associated with the following personal employee attributes and job-settings: female gender; doctorate and master degrees; headquarters location; GS/GP-14-15 grade level; 0-1 and 2-4 years in grade; 1-10 and 11-20 years of federal service; and working in Divisions A or B. The significant independent association of selected employee attributes and job-settings with exceptional SPR is supported by adjusted odds ratios with a range of 1.5-5.5 times as high as the probability of exceptional SPR in the reference categories. There were no significant independent associations between an exceptional SPR rating and race or job series. A 2009 study of the distribution of PMAP ratings among CDC civil service employees who received PMAP ratings during 2006-2008 found that exceptional ratings were associated with the following personal and job-related employee attributes: female gender, supervisory status, higher pay grades, and white race.¹

Using the data available for this study, we are unable to explain why race was associated with exceptional ratings among all CDC employees who received PMAP ratings in 2006-2008 but not among NCHHSTP employees who received PMAP ratings in 2011. Two potential explanations are (1) better control of confounding with multivariable logistic regression in this study, and (2) changes in the behavior of PMAP raters and reviewers in NCHHSTP between 2008 and 2011. The finding regarding female gender in this study is consistent with the 2009 CDC-wide study. Although some personal and job-related attributes (e.g., gender, education, and grade level) might be mediators of the association between accomplishment and exceptional rating unrelated to rater (reviewer) bias, we are unable to disentangle those associations in a cross-sectional observational study design.

This study has several limitations. First, some employees who received PMAP ratings in 2011 while they worked in NCHHSTP are now part of other CDC NCIOs. Second, the personal and job-related employee attributes in the PMAP database (Performance Management Assessment System) are derived from other CDC personnel systems and the accuracy of those data elements were not assessed during this study. Third, the PMAP rules for assessment of performance ratings were not the same during 2006-2008 and in 2011. The current rules have been in place from July 2006 through December 2011. New rules were put in place in January 2012.⁵ Thus the comparison of results over time should be interpreted with caution. Fourth, the reliability of subgroup analyses, within Divisions and among field employees only, was limited by small sample sizes and simultaneous stratification on multiple variables. Fifth, the findings continue but do not resolve concerns about the effectiveness and fairness of the performance rating system that linger when demographic or job attributes are associated with exceptional ratings.

In summary, this study shows that Exceptional SPR is independently associated with personal employee attributes and job-settings. These attributes and job-settings are unlikely to be modifiable causes of exceptional performance and therefore are unsuitable targets of interventions designed to improve employee performance based on accomplishments. We are unaware of plausible cause-and-effect reasons why the personal and job-related employee attributes examined in this study should determine the results of objective employee performance assessment based on accomplishments. We speculate that variations in the performance environments between Divisions and between field and headquarters environments, yet to be discovered, might contain determinants of performance that are modifiable by changes in NCHHSTP's performance evaluation standards of practice. For reasons that existed before the findings of this study were available, NCHHSTP's workforce development and capacity building program had taken the following actions among others : (1) implementing the Coaching and Leadership Initiative for team leaders and other first level supervisors whose responsibilities include PMAP ratings; (2) expanding the Branch Chiefs Opportunities for Leadership Development training to include participation by higher level supervisors and management officials who are also involved in rating and reviewing performance; and (3) starting the NCHHSTP Learn@Lunch Career Development Series to provide all staff at all levels with learning opportunities for career growth, including a session entitled, “Preparing for your Performance Appraisal.” The findings of this study increase the salience of those ongoing training efforts to improve employee performance and its assessment in NCHHSTP.

Beyond efforts to replicate the findings of this study across the CDC, more research is needed to determine if these associations are the result of inaccuracy in the assessment of performance or assessment bias attributable to raters and reviewers of employee performance; whether some of the associations are causal and modifiable by policies and performance evaluation policies and standards of practice within the control of leadership in NCHHSTP and CDC. Except for ongoing training of employees, raters, and reviewers in order to improve the accuracy, fidelity of application, and validity of performance evaluation, we offer no interventions because their effectiveness is not supported by empirical evidence. Finally, we concede that the full implications of this study for the public health workforce beyond CDC and for public health practice exemplified by this operating unit of the CDC must await the results of efforts to replicate this cross-sectional study of associations, examine causal associations with appropriate studies, and the development of interventions that are effective in improving employee performance.