ABSTRACT
U.S. Military services use the Armed Services Vocational Aptitude Battery (ASVAB) and other indicators to select and classify about 150,000 enlisted personnel annually. The effectiveness of these assessments can be difficult to gauge due to differences in criteria across services and occupations. We developed a cross-service job performance model to be used in development of measures of first-term enlisted job performance. We synthesized job performance research, starting with the services’ Job Performance Measurement (JPM) projects (1980s−1990s) through recent projects, to form performance dimensions. The dimension structure was evaluated in a retranslation exercise with performance measurement experts. The resulting hierarchical taxonomy has three levels: (a) 33 dimensions, (b) 11 categories, and (c) four broad categories. The broad performance categories are: (a) Technical, (b) Organizational Citizenship and Peer Leadership, (c) Psychosocial Well-Being, and (d) Physical. A sample of military experts judged the importance of the 33 performance dimensions. Results showed that the four broad performance categories were highly relevant to each service and generalizable across services. While all dimensions were judged important, Psychosocial Well-Being dimensions were rated as the most important by each service. The dimension structure provides a framework for criterion development and future joint-service research.
KEYWORDS: Job performance, counterproductive work behavior, organizational citizenship, well-being, peer leadership
What is the public significance of this article?—This study identifies work behaviors that are required across entry-level military occupations. The behaviors are organized into broader categories. The behaviors and behavioral categories provide a foundation for cross-service assessment of entry-level job performance.
The Department of Defense (DoD) and the U.S. Coast Guard use scores on the Armed Services Vocational Aptitude Battery (ASVAB) and other measures to select about 150,000 new enlisted recruits every year and place them into military occupations. Research to validate these measures (predictors) involves correlating predictor scores with criteria of interest (e.g., training or job performance, attrition). Most validation research is done within each service using service-specific criteria. Joint-service research would allow more robust conclusions about the merits of predictors, but to date no standardized, core set of criterion variables exists. With that in mind, the services initiated a research effort to define common core criterion measures that can be implemented across services.
This paper describes the development of a job performance taxonomy to describe the domain of early career job performance, including performance in training and during the first term of enlistment, across services. The focus of this effort was on work behaviors that are common across military services and occupations. Therefore, occupation-specific tasks and behaviors were not part of the effort. A cross-service performance taxonomy could confer several benefits on the military services, such as providing (a) the basis for developing service-wide criterion measures, (b) mechanisms for identifying performance gaps or areas of need, and (c) a framework for conducting force-readiness research.
Seminal job performance measurement research
While job performance measurement has been studied for more than a century, landmark studies, Job Performance Measurement (JPM) programs, were conducted by U.S. military services in the 1980s and 1990s. JPM projects were congressionally mandated in response to misnorming of the ASVAB in the late 1970s (Sellman et al., 2017). Historically, the ASVAB had been validated as a predictor of training performance. The primary objective of the JPM projects was to evaluate the ASVAB as a predictor of on-the-job performance. Toward that end, the services created and tested a wide array of job performance measures, with each service proceeding independent of the others (Knapp & Campbell, 1993). All services created and administered hands-on tests of task performance for a sample of occupations and most used some form of written job knowledge test as well as supervisor and/or peer job performance ratings. In the end, JPM research indicated that the ASVAB predicts technical in-unit job performance, not only in the first term of enlistment, but also in second term leadership and supervisory positions (Oppler, McCloy, Campbell, 2001).
The Army’s JPM project, called Project A, was the largest effort, encompassing concurrent and longitudinal studies and samples that numbered in the tens of thousands. It was also unique in that the Army pursued the development of an empirical job performance model. Using data from the concurrent validation study (including nine military occupations with a median n of 603 per occupation), confirmatory factor analysis reduced scores from a host of criterion variables to five broad factors of first-term job performance: (a) Core Technical Proficiency (CTP), (b) General Soldiering Proficiency (GSP), (c) Effort and Leadership (ELS), (d) Personal Discipline (PD), and (e) Physical Fitness and Bearing (PFB). In turn, the five-factor model also fit data from a second sample having about the same composition as the first (Campbell et al., 2001). A four-factor model, combining CTP and GSP into a single “can do” factor fit the data almost as well as the five-factor model.
JPM projects showed that the choice of criterion in validation studies matters. One of the most important findings from Project A was that the ASVAB was the best predictor of CTP and GSP, but non-cognitive measures of temperament and interest were better than the ASVAB at predicting ELS, PD, and PFB performance (McHenry et al., 1990; Oppler, McCloy, Peterson et al., 2001). The research showed that non-cognitive measures should be included in the predictor set if the services choose to optimize ELS, PD, and PFB aspects of job performance.
The last few decades of performance measurement research
In the years after the JPM projects, the Army’s performance model provided the foundation for more generic models of job performance. Campbell et al. (1993) synthesized military and civilian literature to form an eight-factor model of performance that included supervisory and management performance dimensions, and Campbell (2012) later added leadership and management performance subfactors.
Other taxonomic research on job performance since the JPM projects can be characterized in terms of breadth across occupations (e.g., focusing on an industry or occupational group or across all jobs) whether the unit of analysis is one facet of job performance (e.g., adaptability, counterproductive work behavior) or many. Few efforts have tried to define performance constructs relevant across occupations. Many efforts have defined specific constructs relating to a facet of job performance including teamwork (O’Shea et al., 2009; Shuffler et al., 2012), adaptability (Pulakos et al., 2000), employee engagement (Macey & Schneider, 2008), organizational citizenship (Dorsey et al., 2017; Goffin et al., 2013; Organ, 1988), counterproductive work behavior (Dalal, 2005; Rotundo & Spector, 2017; Spector et al., 2010, 2006), ethical performance (Russell et al., 2017), self-directed learning (Garrison, 1997; Russell et al., 2006), and cross-cultural performance (Klafehn et al., 2019).
While there are probably hundreds of occupation-specific performance taxonomies (e.g., for managers, auditors, teachers), military-specific performance facets were most relevant to our current project. Those efforts have defined performance dimensions for combat performance (Wasko et al., 2012), situational awareness (Matthews et al., 2011), and basic and advanced training (Waugh & Russell, 2005). The Army and Air Force have also conducted projects to expand performance constructs in the years since the JPM projects (e.g., Lance et al., 1992; Sager et al., 2005).
Purpose of the present research
The objective of this research was to (a) build a first-term enlisted, cross-service job performance model and (b) evaluate its relevance within services as well as its generalizability across services.
Method
Job performance refers to work-related behavior – actions people take and things they say or do (Campbell et al., 1993). A job performance taxonomy places behaviors that covary into broader constructs that have meaning to the organization (Binning & Barrett, 1989). Unlike individual difference taxonomies, which are usually based on correlations among psychological constructs, performance taxonomies typically are developed using rational methods (i.e., the judgments of a group of people; Binning & Barrett, 1989; Flanagan, 1954). One systematic way to gather expert judgments to evaluate taxonomic inferences is “retranslation” (Smith & Kendall, 1963). Once a draft taxonomy is developed, a pool of experts attempt to categorize (or retranslate) behaviors into the draft constructs.
Our approach to develop a cross-service first-term job performance taxonomy was to (a) conduct a qualitative review of literature on job performance constructs, (b) create a draft taxonomy, and (c) conduct a retranslation exercise to evaluate and refine the draft taxonomy. The final step was to evaluate the relevance and generalizability of the performance taxonomy across the services and their many occupations by surveying experts knowledgeable about service-wide job requirements.
Job performance model
We developed the job performance model based on a qualitative review of the literature, followed by a retranslation exercise designed to evaluate the dimension structure of the taxonomy (Smith & Kendall, 1963).
Qualitative review
We conducted an exhaustive literature search to identify extant performance taxonomies and research on specific job performance dimensions or domains. In this search, we sought articles that provided behavioral definitions of dimensions whether they had one definition for one dimension, several dimensions of a facet of job performance (such as adaptability), or a full set of dimensions intended to describe the performance domain for all jobs. The search involved (a) searching Defense Technical Information Center (DTIC) and American Psychological Association (APA) databases for articles relating to job performance, job performance dimensions, and performance taxonomies, (b) querying research staff to learn of book chapters and other publications unlikely to show up in extant databases, and (c) gathering information from military experts. A panel of nine military selection and classification experts representing each service and the DoD assisted us throughout this effort. The panel also helped us gain access to performance rating dimensions used by the services as well as additional experts within each service when deeper service-specific knowledge was needed.
We identified job performance taxonomies that were relevant to first-term enlisted servicemember performance. Taxonomies clearly targeted toward managerial performance or only one occupation were not included. Several taxonomies that were reviewed contained many performance constructs as shown in Table 1. Others, shown in Table 2, focused on a particular domain of constructs, and still others, shown in Table 3, were military specific. The Campbell (2012) model is the most extensively researched and documented of the taxonomies and is relevant to entry-level positions. Therefore, it served as a scaffold for sorting other specific dimensions. We sorted the new dimensions sets into the Campbell dimensions, made adjustments to align with new content, and tailored behavioral definitions to capture dimension content. This process resulted in 33 draft dimension definitions. The draft dimensions were reviewed by several performance measurement experts.
Table 1.
Cross-job, multiple construct taxonomies.
| The Campbell Model (Campbell, 2012) |
| Job specific technical performance components. |
| Non-job-specific technical performance components |
| Written and oral communication task proficiency |
| Demonstrating effort |
| Maintaining personal discipline (Counterproductive work behavior) |
| Facilitating peer and team performance |
| Leadership performance factors: |
| Consideration, support, person-centered |
| Goal emphasis |
| Initiating structure, guiding, directing |
| Empowerment, facilitation |
| Training, coaching |
| Serving as a model |
| Management performance factors: |
| Goal setting, planning, organizing, and budgeting |
| Monitoring unit effectiveness |
| External representation |
| Staffing |
| Coordination |
| Decision making, problem solving, and strategic innovation |
| Administration |
| Commitment and compliance |
| The Great Eight Competencies (Bartram, 2005) |
| Leading and deciding |
| Supporting and cooperating |
| Interacting and presenting |
| Analyzing and interpreting |
| Creating and conceptualizing |
| Organizing and executing |
| Adapting and coping |
| Enterprising and performing |
| Leader Performance (Zaccaro et al., 2012) |
| Communicating information |
| Motivating |
| Setting direction |
| Planning and internal business control |
| Solving problems |
| Conflict management |
| External representation |
| Staffing |
Table 2.
Domain-specific taxonomies.
| Organizational Citizenship Behaviors (Dorsey et al., 2017) |
| Self-development |
| Helping |
| Cooperating |
| Courtesy |
| Motivating |
| Compliance |
| Loyalty |
| Representing |
| Cross-cultural Behavior (Klafehn et al., 2019) |
| Applies knowledge of another culture |
| Uses nonverbal communication |
| Works with an interpreter |
| Collects cultural Information |
| Strengthens cross-cultural relations |
| Manages conflicts and ethical challenges |
| Influences others through persuasion and negotiation |
| Maintains composure |
| Works with people from other cultures |
| Adaptability (Pulakos et al., 2000) |
| Learning work tasks, technologies, and procedures |
| Demonstrating cultural adaptability |
| Demonstrating interpersonal adaptability |
| Solving problems creatively |
| Handling work stress |
| Handling emergencies or crisis situations |
| Dealing with uncertain and unpredictable work situations |
| Demonstrating cultural adaptability |
| Demonstrating physically oriented adaptability |
| Ethical Performance Dimensions (Russell et al., 2017) |
| Workplace bullying |
| Defamation of others |
| Whistle-blowing |
| Rule abiding |
| Conflict of interest |
| Intellectual property |
| Confidentiality |
| Unfair treatment |
| Abuse of power |
| Engagement (Macey & Schneider, 2008) |
| Engagement as extra-role behavior |
| Engagement as organizational citizenship behavior |
| Engagement as adaptive behavior |
| Role expansion |
| Team Performance (Shuffler et al., 2012) |
| Communication |
| Backup behavior |
| Mutual trust |
| Shared mental models |
| Coordination |
| Motivation and confidence-building |
| Team leadership |
| Conflict management |
| Mission analysis, formulation, and planning |
| Goal specification |
| Strategy formulation |
| Mutual performance monitoring |
| Team orientation |
| Affect management |
| Adaptation |
| Active Learning (Russell et al., 2006) |
| Self-development |
| Information searching |
| Learning engagement |
| Learning self-management |
| Learning though information exchange |
| Counterproductive Work Behavior (Spector et al., 2010) |
| Absenteeism |
| Cyber loafing |
| Lateness |
| Production deviance |
| Sabotage |
| Service sabotage |
| Theft |
| Withdrawal |
| Workplace violence/aggression |
| Workplace bullying |
| Sexual harassment |
| Incivility |
| Substance abuse |
| Individual Team Performance (O’Shea et al., 2009) |
| Cooperation |
| Sociability |
| Dominance |
| Negativity |
Table 3.
Military taxonomies.
| First Term Performance (Sager et al., 2005) |
| Performs common tasks |
| Uses computers |
| Communicates orally |
| Communicates in writing |
| Self-directed learning |
| Relates to and supports peers |
| Exhibits cultural tolerance |
| Demonstrates teamwork |
| Exhibits self-management |
| Manages information |
| Solves problems and makes decisions |
| Follows Instructions and rules |
| Exhibits selfless service orientation |
| Demonstrates military presence |
| Adapts to changing situations |
| Demonstrates physical Fitness |
| Combat Performance (Wasko et al., 2012) |
| Field readiness |
| Awareness & Vigilance |
| Field/combat judgment |
| Emotional resilience |
| Physical Endurance |
| Physical courage |
| Military Training Performance (Waugh & Russell, 2005) |
| Delinquency |
| Support for peers |
| Peer leadership |
| Teamwork |
| Self-management |
| Respect for authority and orders |
| Self-control |
| Adaptability |
| Physical fitness and endurance |
| Other Specific Dimensions |
| Situation awareness (Matthews et al., 2011) |
Retranslation exercise
To evaluate the dimensions and hierarchical structures, we asked 17 researchers to categorize the 33 dimensions into (a) two categories, (b) four categories, and (c) 10 categories. All 17 researchers had substantial experience in performance measurement and/or military criterion development. They worked either for the DoD or for a DoD contractor and most had PhDs.
The two-, four-, and ten-category solutions were chosen to reflect (a) different levels of breadth, (b) widely used taxonomic structures, and (c) a structure that appeared to emerge from our sorting of dimensions. The Can-do/Will-do distinction (Campbell & Knapp, 2001) is frequently used in military research to differentiate aspects of performance that require technical knowledge, skill, and ability versus those that are a function of motivation and effort. Relatedly, Borman and Motowidlo (1993) differentiated task and contextual performance, where task performance involves performance on technical aspects of the job and contextual performance involves demonstrating effort and motivation in doing work. The four-category judgment was based primarily on Project A research, combining Project A’s two technical performance categories (GSP and CTP) into one technical performance category and updating it with more recent research on counterproductive work behavior. The 10-category solution was based on our own organization of the 33 performance dimensions.
For the two-category judgment, we defined Can-do/Task Performance and Will-do/Contextual Performance as follows, based on definitions from Borman and Motowidlo (1993) and Campbell and Knapp (2001).
Can-do/Task Performance: performance of activities that contribute to the organization’s technical core. Task activities usually vary between different jobs in the same organization. Task performance is usually predicted by knowledge, skills, and abilities. Task performance is role-prescribed (i.e., formally recognized as part of the job). Can-do performance typically is measured using maximal performance instruments.
Will-do/Contextual Performance: performance of activities that support the organizational, social, and psychological environment (e.g., organizational citizenship behaviors). Contextual activities are important across jobs. Motivational and personality characteristics are key determinants of contextual performance. Contextual activities may not be role-prescribed. Will-do performance is most often measured using typical performance measures.
Borman and Motowidlo (1993) discuss specific performance dimensions as being saturated with either task or contextual elements. Dimensions can vary somewhat in terms of how reliant performance is on task or contextual elements. So, for example, some dimensions might be saturated with both task and contextual elements. Managerial dimensions that involve both planning and dealing with people might be the best example of dually-saturated dimensions. Therefore, raters were asked to judge the Can-do/Task Performance vs. Will-do/Contextual Performance saturation of each of the 33 specific dimensions using the following scale:
-2 = Can-do/Task
-1 = Can-do/Task with some Will-do/Contextual saturation
0 = Equally Can-do/Task and Will-do/Contextual
+1 = Will-do/Contextual with some Can-do/Task saturation
+2 = Will-do/Contextual
For the four-category solution, we provided definitions of four categories derived from the Campbell (2012) model: (a) Technical Performance, (b) Counterproductive Work Behavior, (c) Citizenship and Peer Leadership, and (d) Physical Performance. Raters were asked to categorize each of the 33 dimensions into one of the four categories. Similarly, for the 10-category solution, we defined 10 performance categories that had emerged as we organized dimensions from the literature review. Raters were asked to sort each of the 33 specific performance dimensions into one of the 10 categories (see the note in Table 4 for the list of dimensions).
Table 4.
Summary of retranslation results.
| 2D (n = 17)1 |
4D (n = 17)2 |
10D (n = 16)3 |
|||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Broad Performance Category, Category, Dimension | M | SD | TP | CWB | CPL | PP | TP | COM | CI | CWB | SP | IS | IWR | OS | HS | PP | |
| A. Technical Performance | |||||||||||||||||
| A.1. Task Proficiency | |||||||||||||||||
| General Proficiency | −1.53 | 1.07 | 17 | 0 | 0 | 0 | 16 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Job-Specific Proficiency | −1.59 | 0.62 | 17 | 0 | 0 | 0 | 16 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| A.2. Decision Making, Problem Solving, and Innovation (judgment) | |||||||||||||||||
| Decision Making, Problem Solving, and Innovation | −1.18 | 0.88 | 17 | 0 | 0 | 0 | 9 | 0 | 0 | 0 | 0 | 2 | 5 | 0 | 0 | 0 | |
| A.3. Communication | |||||||||||||||||
| Written Communication | −1.29 | 1.05 | 17 | 0 | 0 | 0 | 1 | 15 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Oral Communication | −1.12 | 0.99 | 16 | 0 | 1 | 0 | 0 | 16 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Nonverbal Communication | −0.41 | 1 | 15 | 0 | 2 | 0 | 0 | 16 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| A.4. Safety Consciousness | |||||||||||||||||
| Safety Consciousness during Mission Operations | −0.47 | 1.23 | 13 | 0 | 4 | 0 | 2 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 12 | 0 | |
| Safety Consciousness in Everyday Work | 0.24 | 1.2 | 9 | 0 | 8 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 15 | 0 | |
| B.1. Planning and Structuring Work | |||||||||||||||||
| Providing Structure | −0.59 | 1.06 | 6 | 0 | 11 | 0 | 1 | 0 | 0 | 0 | 1 | 13 | 0 | 1 | 0 | 0 | |
| Teamwork | −0.06 | 1.25 | 8 | 1 | 8 | 0 | 3 | 0 | 0 | 0 | 5 | 8 | 0 | 0 | 0 | 0 | |
| Managing Responsibilities | 0.29 | 1.26 | 5 | 1 | 11 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 14 | 0 | 0 | 0 | |
| Learning Self-Management | 0.35 | 1.32 | 7 | 0 | 10 | 0 | 1 | 0 | 7 | 0 | 0 | 0 | 8 | 0 | 0 | 0 | |
| B.2. Conscientious Initiative | |||||||||||||||||
| Active Learning | 0.88 | 1.17 | 3 | 0 | 14 | 0 | 0 | 0 | 15 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | |
| Self-Development | 1.35 | 0.61 | 4 | 0 | 13 | 0 | 1 | 0 | 15 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Persistence | 1.18 | 1.19 | 3 | 0 | 14 | 0 | 0 | 0 | 15 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | |
| Initiative | 1.41 | 0.87 | 0 | 0 | 17 | 0 | 0 | 0 | 14 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | |
| B.3 Support for Peers | |||||||||||||||||
| Helping Peers | 0.94 | 0.83 | 2 | 0 | 15 | 0 | 0 | 0 | 0 | 0 | 15 | 1 | 0 | 0 | 0 | 0 | |
| Cooperating | 1.47 | 0.51 | 0 | 0 | 17 | 0 | 0 | 0 | 0 | 0 | 15 | 0 | 0 | 1 | 0 | 0 | |
| Courtesy & Respect | 1.53 | 0.87 | 0 | 0 | 17 | 0 | 0 | 1 | 1 | 0 | 13 | 0 | 1 | 0 | 0 | 0 | |
| Accepting Differences4 | 1.76 | 0.44 | 0 | 0 | 17 | 0 | 0 | 0 | 1 | 0 | 15 | 0 | 0 | 0 | 0 | 0 | |
| Motivating | 1.76 | 0.56 | 1 | 0 | 16 | 0 | 0 | 0 | 0 | 0 | 16 | 0 | 0 | 0 | 0 | 0 | |
| Serving as a Model | 1.76 | 0.56 | 1 | 0 | 16 | 0 | 0 | 0 | 2 | 0 | 10 | 1 | 1 | 2 | 0 | 0 | |
| B.4 Organizational Support | |||||||||||||||||
| Military Presence | 1.65 | 0.7 | 1 | 0 | 16 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 2 | 12 | 0 | 0 | |
| Selfless Service | 1.82 | 0.73 | 0 | 0 | 17 | 0 | 0 | 0 | 1 | 0 | 3 | 0 | 0 | 12 | 0 | 0 | |
| Support for the Organization5 | 0.76 | 1.3 | 2 | 0 | 15 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 15 | 0 | 0 | |
| Integrity/Moral Courage | 1.35 | 1.06 | 0 | 0 | 17 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 15 | 0 | 0 | 0 | |
| C. Psychosocial Well-Being | |||||||||||||||||
| C.1. Well-being | |||||||||||||||||
| Maintaining own Well-Being | 1.38 | 0.62 | 1 | 0 | 14 | 2 | 0 | 0 | 1 | 0 | 0 | 0 | 3 | 0 | 12 | 0 | |
| C.2. Counterproductive Work Behavior | |||||||||||||||||
| Loafing and Tardiness | 1.53 | 1.07 | 0 | 17 | 0 | 0 | 1 | 0 | 0 | 15 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Abusing Substances and Other Self-Destructive Behavior | 1.75 | 0.58 | 0 | 15 | 2 | 0 | 0 | 0 | 0 | 14 | 0 | 0 | 1 | 0 | 1 | 0 | |
| Bullying, Harassing, or Hurting Others | 1.82 | 0.53 | 0 | 17 | 0 | 0 | 0 | 0 | 0 | 14 | 1 | 0 | 0 | 0 | 0 | 0 | |
| Delinquency | 1.76 | 0.56 | 0 | 17 | 0 | 0 | 0 | 0 | 0 | 15 | 0 | 0 | 0 | 1 | 0 | 0 | |
| D. Physical Performance | |||||||||||||||||
| Physical Endurance | −0.25 | 1.53 | 0 | 0 | 0 | 17 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 16 | |
| Physical Fitness | −0.35 | 1.37 | 0 | 0 | 1 | 16 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 3 | 11 | |
Note. 1M = Mean, SD = Standard Deviation on the following scale: Can/do – will/do rating scale: −2 = Can-do/task, −1 = Can-do/task with some Will-do saturation, 0 = Equally Can-do/task and Will-do/contextual, +1 = Will-do/contextual with some Can-do/task saturation, +2 = Will-do/contextual.
2TP = Technical Performance, CWB = Counterproductive Work Performance, CPL = Citizenship and Peer Leadership Performance, and PP = Physical Performance.
3TP = Technical Performance, COM = Communication, CI = Conscientious Initiative, CWB = Counterproductive Work Performance, SP = Support for Peers, IS = Initiating Structure, IWR = Individual Work Responsibility, OS = Occupational Support, and PP = Physical Performance.
Consistency across raters on the judgments was high. For the Can-do/Task and Will-do/Contextual rating, the reliability estimates suggested strong consistency across raters (intraclass correlation coefficients [ICCs] of .69 [single-rater] and .95 [all raters]). The percent of raters agreeing on the categorization of the four-category structure ranged from 47% to 100%, with a mean of 88% (Avg. κ = .72). For 14 of the 33 dimensions, 100% of the raters agreed on the four-category grouping. Finally, for the 10-category structure, the percent of raters agreeing on the categorization of the specific 33 dimensions ranged from 50% to 100%, with a mean of 86% (Avg. κ = .74). For six of 33 dimensions, 100% of the raters agreed on the 10-category grouping.
Even though raters made the two-category ratings reliably, results suggested that the Can-do/Task vs. Will-do/Contextual structure did not work particularly well. As shown in Table 4, raters had difficulty making judgments for dimensions that were thought to be related to both Can-do/Task and Will-do/Contextual performance, as evidenced by higher standard deviations in Can-do/Will-do judgments for some of the dimensions (e.g., Teamwork, Managing Responsibilities, and Learning Self-Development).
After concluding that the two-category taxonomy (Can-do/Task and Will-do/Contextual) was insufficient, we turned to the four- and ten-category taxonomies. As reported earlier, both category structures were reasonably sound (Avg. κ = .72 and .74, respectfully). We decided to retain the two hierarchical tiers so that they could be used for different purposes needing different levels of specificity. But, we felt that the substance of the two tiers could be improved using the retranslation data. To improve the four-category taxonomy, we placed each of the 33 specific dimensions where they were classified by the raters. Our goal was to create the four categories such that they would be relatively homogeneous with regard to Can-do/Task and Will-do/Contextual saturation. For example, before the retranslation exercise, we had grouped the two Safety Consciousness dimensions and a dimension on well-being (Stress Adjustment) into a broader construct we had called Health and Safety. However, the safety and well-being dimensions were rated as having very different Can-do/Will-do saturations, with Can-do playing a much stronger role in Safety Consciousness than Stress Adjustment. Safety Consciousness was moved to the broader Technical Proficiency category because its Can-do/Task saturation aligned it with other Can-do/Task performance constructs in Technical Proficiency. We moved well-being into Psychosocial Well-Being category along with counterproductive work behaviors which were all rated as having a strong Will-do/Contextual saturation.
In the 10-category solution, two categories, Individual Work Responsibility and Health and Safety Conscientiousness, did not hold together well. We moved their constituent dimensions to other categories based on similar categorizations by raters. We also broke Decision Making, Problem Solving, and Innovation out into its own category (making an 11th category) and made minor wording changes to titles based on rater feedback.
Table 5 provides the entry-level skilled and technical job performance taxonomy. At the highest level, the taxonomy has four broad categories that are relatively homogeneous with regard to their Can-do/Will-do saturation. The second level has 11 categories, and the most specific level has 33 specific dimensions.
Table 5.
Performance taxonomy for entry-level skilled and technical occupations.
| Four Broad Performance Categories and 11 Categories | 33 Specific Dimensions | |
|---|---|---|
| A. Technical Performance | ||
| A.1. | Task Performance | General Proficiency |
| Job-Specific Proficiency | ||
| A.2. | Decision Making, Problem Solving, and Innovation | Decision Making, Problem Solving, and Innovation |
| A.3. | Communication | Written Communication |
| Oral Communication | ||
| Nonverbal Communication | ||
| A.4. | Safety Consciousness | Safety Consciousness during Mission Operations |
| Safety Consciousness in Everyday Work | ||
| B. Organizational Citizenship & Peer Leadership | ||
| B.1. | Planning and Structuring Work | Providing Structure |
| Teamwork | ||
| Self-Management | ||
| Learning Self-Management | ||
| B.2. | Conscientious Initiative | Active Learning |
| Self-Development | ||
| Persistence | ||
| Initiative | ||
| B.3. | Support for Peers | Helping Peers |
| Cooperating | ||
| Courtesy & Respect | ||
| Accepting Differences | ||
| Motivating | ||
| Serving as a Model | ||
| B.4. | Organizational Support | Military Presence |
| Selfless Service | ||
| Support for the Organization | ||
| Integrity/Moral Courage | ||
| C. Psychosocial Well-Being | ||
| C.1. | Adapting to Stressful Situations | Maintaining own Well-Being |
| C.2. | Counterproductive Work Behavior | Loafing and Tardiness |
| Abusing Substances and Other Self-Destructive Behavior | ||
| Bullying, Harassing, or Hurting Others | ||
| Delinquency | ||
| D. Physical Performance | Physical Endurance | |
| Physical Fitness | ||
Military expert survey
Having developed a taxonomic structure that has a great deal of support based on research literature and input from military research staff, we wanted to assess the generalizability of the constructs across the services and the criticality of the performance constructs. Toward that end, we solicited experts from each of the services to complete a survey to rate performance dimensions.
Sample
We sought experts who were not just knowledgeable about one occupation. We were looking for experts who had a good understanding of the non-technical performance requirements of many or all occupations in the service. We asked our panel of military selection and classification experts to identify at least three experts in each service who would meet the following criteria:
Have at least 5 years of experience in their organization.
Be broadly knowledgeable of their service’s occupations.
Be highly knowledgeable of their service’s mission.
Be able to respond to questions based, cumulatively, on their years of experience.
Twenty-six military experts were identified and responded to the survey. Sample sizes were: nine Army, three Navy, four Air Force, and 10 Marine Corps. The average tenure of the SMEs in their field across service was 21.81 (SD = 8.42), with a low average of 8.90 years (SD = 6.01) for the Marine Corps and a high average of 28.75 years (SD = 9.43) for the Air Force. The role of the survey participants fell into one of the following categories: (a) human capital research (e.g., personnel research psychologist, operations research analyst), (b) training (e.g., proponent specialist), (c) selection and accessions (e.g., selection and classification business analyst, chief of enlisted accessions policy), and (d) support programs (e.g., action officer combat and operational stress control).
Survey
The online survey asked military experts to make the following two ratings for each of the 33 job performance dimensions:
Importance across the service’s enlisted, first-term occupations on a 5-point rating scale ranging from “Not Important” to “Extremely Important”
Criticality to the service’s mission accomplishment on a 5-point rating scale ranging from “Not at all Critical” to “Extremely Critical”
Although similar, the two judgments (i.e., importance and criticality) offered somewhat different perspectives on the relevance of the performance dimension. Importance was intended to focus on day-to-day performance requirements. In making the criticality judgment, however, experts could consider the service’s ongoing initiatives and future plans.
Results
Within each service, we averaged the military experts’ importance and criticality ratings to create an overall relevance score. The overall relevance scores were highly reliable; ICCs were .95 (Army), .73 (Navy), .98 (Air Force), and .85 (Marine Corps), indicating sufficient expert agreement on the relevance of the dimensions within services.
We computed the grand mean of the services’ mean relevance scores to serve as an index of the extent to which a performance requirement generalized across services. Eight of 33 performance dimensions had grand means less than 3.0 (on a 5-point scale) in one or more of the services, suggesting these dimensions, though important within specific services, were not generalizable across services. While the full performance taxonomy includes all 33 dimensions, for the purpose of cross-service generalizability, we dropped these eight dimensions from further analysis: Nonverbal Communication, Written Communication, Learning/Training, Classroom Learning, Motivating, Serving as a Model, Military Presence, and Endurance.
As shown in Table 6, Psychosocial Well-Being (PWB) was rated most highly across the services (i.e., was the most generalizable). While both categories within PWB received high grand mean ratings across the services, Counterproductive Work Behavior was rated somewhat higher than Adapting to Stress (i.e., grand mean ratings of 4.48 and 3.77, respectively). Grand mean ratings for the other three broad performance categories (Physical Performance, Organizational Citizenship & Peer Leadership, and Technical Performance) varied little, ranging from m = 3.58 for Technical Performance to m = 3.65 for Physical Performance. Within Technical Performance, Safety Consciousness and Task Performance received higher grand mean ratings than the other categories, suggesting they might be slightly more generalizable across services. Within Organizational Citizenship and Peer Leadership (OCPL) Organizational Support received the highest grand mean rating (m = 3.85). In comparison, grand mean ratings for the other OCPL categories ranged from m = 3.54 for Support for Peers to m = 3.56 for Conscientious Initiative.
Table 6.
Mean relevance ratings for performance categories.
| |
Within Service Means |
Across Services |
||||
|---|---|---|---|---|---|---|
| Performance Category | Army | Navy | Air Force | Marine Corps | Grand Mean | SD |
| A. Technical Performance | 3.58 | 3.82 | 3.33 | 3.49 | 3.58 | .24 |
| A.1. Task Performance | 3.86 | 3.63 | 3.44 | 3.68 | 3.65 | .24 |
| A.2. Oral Communication | 3.11 | 3.83 | 3.25 | 3.50 | 3.42 | .38 |
| A.3. Decision Making, Problem Solving, and Innovation | 3.39 | 3.67 | 3.00 | 3.00 | 3.35 | .33 |
| A.4. Safety Consciousness | 3.94 | 4.17 | 3.63 | 3.78 | 3.88 | .27 |
| B. Organizational Citizenship & Peer Leadership | 3.41 | 3.96 | 3.45 | 3.68 | 3.62 | .15 |
| B.1. Planning and Structuring Work | 3.50 | 3.83 | 3.25 | 3.61 | 3.55 | .29 |
| B.2. Conscientious Initiative | 3.31 | 4.17 | 3.04 | 3.73 | 3.56 | .59 |
| B.3. Support for Peers | 3.42 | 3.67 | 3.5 | 3.56 | 3.54 | .13 |
| B.4. Organizational Support | 3.41 | 4.17 | 4.00 | 3.81 | 3.85 | .40 |
| C. Psychosocial Well-Being | 4.15 | 4.70 | 4.43 | 4.22 | 4.38 | .28 |
| C.1. Adapting to Stressful Situations | 3.72 | 3.83 | 3.75 | 3.75 | 3.77 | .06 |
| C.2. Counterproductive Work Behavior | 4.24 | 4.88 | 4.56 | 4.24 | 4.48 | .32 |
| D. Physical Performance | 3.83 | 3.67 | 3.00 | 4.11 | 3.65 | .44 |
| D.1 Fitness | 3.83 | 3.67 | 3.00 | 4.11 | 3.65 | .44 |
Note. For each performance dimension, mean relevance was computed by averaging importance and criticality ratings within service, then a unit-weighted average was computed across services. Sample sizes were 9 Army, 3 Navy, 4 Air Force, and 10 Marines. ICCs (C,k) were .95 (Army), .73 (Navy), .98 (Air Force), and .85 (Marines). The survey had 33 specific dimensions. Dimensions were retained only if they had mean ratings of at least 3.0 (on a 5-point scale) within each service. Twenty-five dimensions were retained and eight were dropped. The means presented here include only those 25 generalizable dimensions.
Full definitions of the performance constructs in the performance taxonomy appear in the Appendix.
Discussion
The four broad performance categories can be thought of as updated versions of the empirical factors identified in Project A (Campbell & Knapp, 2001). Technical performance is “Can-do/Task” performance encompassing Core Technical Proficiency and General Soldiering Proficiency. Organizational Citizenship and Peer Leadership overlaps substantially with Effort and Leadership from Project A, and Physical Performance maps to Project A’s Physical Fitness and Bearing.
We believe that one of the most important contributions of the new performance model is in the definition of PWB. Project A’s Personal Discipline reflected behaviors such as adhering to regulations, maintaining self-control, and demonstrating integrity. That aspect of the performance space was expanded in the current project to address recent research on counterproductive work behavior and adaptation to stress. Those behaviors are now encompassed in PWB, and the services are currently attempting to measure PWB through self-report measures like the Army Life Questionnaire (Nesbitt et al., 2020). While all four performance categories received high mean ratings within and across services. PWB was, overall, perceived to be the most important category across the services. This coincides with much concern in the last decade about resilience and stress reactions (e.g., see the 2011 special issue of American Psychologist on comprehensive Soldier fitness). PWB acknowledges the importance of behaviors stemming from psychological adjustment.
Another important component of the new performance model is the behavioral definition of specific dimensions within each category. This nested model allows the services to drill down to more specific performance constructs as needed for a particular purpose or concern. For example, if the concern were that recruits were deficient as peer leaders, the research team could focus criterion measures on specific dimensions of Organizational Citizenship and Peer Leadership.
One potential limitation of this research is the sample size for the survey. We are confident in these data for the general inference that we wanted to draw. That is, we wanted to identify job performance constructs that were broadly applicable across services. Therefore, we sought experts with broad knowledge within their service. Even so, we recommend that the services use the performance taxonomy in future job analytic work and continue to make judgments regarding its efficacy. Over time, the importance of different performance dimensions can change. For example, we suggest that one reason that Technical Performance was, on average, the least important category is because applicants are screened on the ASVAB and undergo rigorous technical training for their jobs. Therefore, the services are not likely to observe as many problems with Technical Performance in the field as they might with other dimensions for which applicants are not as directly selected or trained (e.g., Psychosocial Well-being). Selection and training initiatives or even other external factors such as reduced or enhanced physical demands could change the priority of performance categories over time.
The taxonomy has been used to categorize hundreds of criterion instruments and to guide the development of new cross-service criterion measures (i.e., performance rating scales and situational judgment tests).
APPENDIX.
Hierarchical Trainee and 1st Term Performance Taxonomy Definitions
| Broad Category | Category | Specific Dimension | Definition |
|---|---|---|---|
| A. Technical Performance | Performing job tasks proficiently; communicating clearly; making sound decisions; and being alert to safety and security concerns. | ||
| A.1. Task Performance | Being able to perform job-specific and Service-wide tasks proficiently. | ||
| Job-Specific Proficiency | Being able to perform job-specific tasks at the appropriate skill level (Campbell, 2012; Sager et al., 2005). | ||
| General Proficiency | Being able to perform Service-wide tasks at the appropriate skill level (e.g., navigation in the Army and Marine Corps) (Campbell, 2012; Sager et al., 2005). | ||
| A.2. Decision Making, Problem Solving, and Innovation | Making sound, timely decisions, even under pressure; analyzing situations and innovating solutions to problems; resolving conflicts; adapting plans and decisions as situations change (Campbell, 2012; Pulakos, 2000; Sager et al., 2005). | ||
| Decision Making, Problem Solving, and Innovation | Making sound, timely decisions, even under pressure; analyzing situations and innovating solutions to problems; resolving conflicts; adapting plans and decisions as situations change. | ||
| A.3. Communication | Conveying oral and written information clearly; using appropriate nonverbal communication. | ||
| Oral Communication | Conveying information in a clear, understandable, organized manner when speaking (Campbell, 2012; Sager et al., 2005). | ||
| Written Communicationa | Conveying information in a clear, understandable, organized manner when writing (Campbell, 2012; Sager et al., 2005). | ||
| Nonverbal Communicationa | Using “alternative, culturally appropriate methods to interpret and convey meaning when common language is not shared” (Klafehn et al., 2019, p. 4). | ||
| A.4. Safety and Security Consciousness | Following routine safety and security guidelines; and being alert to safety and security threats in non-routine situations. | ||
| Safety and Security Consciousness in Everyday Work | Following safety and security guidelines and instructions, noticing and alerting others to potential hazards in day-to-day work (Sager et al., 2005; Waugh & Russell, 2005). | ||
| Safety and Security Consciousness during Mission Operations | Being alert to enemy and environmental threats and taking actions that do not place self or others at unwarranted risk (Wasko et al., 2012). | ||
| B. Organizational Citizenship & Peer Leadership | Planning and structuring own work, and when in a leadership role, the work of others; taking initiative and persisting in work or training despite difficult conditions; supporting, helping, motivating, and respecting peers; and showing commitment to the organization, the team, and moral/ethical principles. | ||
| B.1. Planning and Structuring Work | Leading peer when given a leadership role; working with team members to plan work; planning and organizing own responsibilities and studying. | ||
| Providing Structure | Leading peers when given a leadership role, giving clear instructions, distributing tasks, and gaining others’ cooperation (Campbell, 2012; Waugh & Russell, 2005; Zaccaro, 2012). | ||
| Teamwork | Working with other team members to interpret the mission, set and prioritize team goals, and monitor team performance (Campbell, 2012; Shuffler et al., 2012). | ||
| Self-Management | Managing own responsibilities (e.g., work assignments, gear, equipment, personal finances, family, and personal well-being), and appearing on duty prepared for work. Setting personal work objectives (Sager et al., 2005; Wasko et al., 2012). | ||
| Learning/Training Self-Managementa | Planning, organizing, and using study time effectively (e.g., setting aside specific times to study; completing assignments on time) (Russell et al., 2006). | ||
| B.2. Conscientious Initiative | Taking initiative; persisting with extra effort despite obstacles; taking steps to enhance own knowledge and skill. | ||
| Classroom Learninga | Being actively engaged in own learning by searching for and obtaining information, taking notes in class, highlighting relevant material, practicing new skills, and participating/contributing during classes (Russell et al., 2006). | ||
| Self-Development | “Developing own knowledge and skills by taking courses on own time, volunteering for training and development opportunities offered within the organization; and trying to learn new knowledge and skills on the job from others or through new job assignments” (Dorsey et al., 2017, p. 450; see also Sager et al., 2005). | ||
| Persistence | “Persisting with extra effort despite difficult conditions and setbacks, accomplishing goals that are more difficult and challenging than normal completing work on time despite unusually short deadlines, and performing at a level of excellence that is significantly beyond normal expectations” (Dorsey et al., 2017, p. 450; see also Campbell, 2012; Sager et al., 2005). | ||
| Initiative | Taking the initiative to do all that is necessary to accomplish team or organizational objectives encountered, finding additional work to perform when own duties are completed, and volunteering for work assignments (Campbell, 2012; Dorsey et al., 2017; Sager et al., 2005). | ||
| B.3. Support for Peers | Helping and motivating peers; cooperating with others; being respectful and considerate; accepting individual differences; and modeling core values. | ||
| Helping Peers | “Helping others by offering suggestions about their work, showing them how to accomplish difficult tasks, teaching them useful knowledge or skills, directly performing some of their tasks, and providing emotional support for personal problems” (Dorsey et al., 2017, p. 450; see also Sager, 2005; Shuffler et al., 2012; Waugh & Russell, 2005). | ||
| Cooperating | Cooperating with others by accepting their suggestions, following their lead, being open-minded and adapting to others’ ways, and informing others of events or requirements that are likely to affect them (Dorsey et al., 2017; Pulakos et al., 2000). | ||
| Courtesy & Respect | “Showing consideration, courtesy, and tact in relations with others” (Dorsey et al., 2017, p. 450). | ||
| Accepting Differences | Showing “interest in and respect for people of other backgrounds or cultures by regularly engaging with them in a manner considerate of their norms” (Klafehn et al., 2019, p. 4; see also Sager et al., 2005). | ||
| Motivatinga | “Motivating others by applauding their achievements and successes, cheering them on in times of adversity, showing confidence in their ability to succeed, helping them overcome setbacks” (Dorsey et al., 2017, p. 450), and modeling leadership behavior (Campbell, 2012). | ||
| Serving as a Modela | Modeling core values by acting unselfishly, enduring hardships without complaint, treating others well, behaving ethically, and showing confidence and enthusiasm (Campbell, 2012). | ||
| B.4. Organizational Support | Complying with organizational rules; demonstrating selfless service; presenting a positive image of the Service; and demonstrating honesty and integrity. | ||
| Military Presencea | Presenting a positive and professional image of self and the military even when off duty, maintaining proper military appearance (Campbell, 2012; Sager et al., 2005). | ||
| Selfless Service | Committing to the greater good of the team or group putting organizational welfare ahead of individual goals (Sager et al., 2005). | ||
| Support for the Organization | “Complying with organizational rules and procedures, encouraging others to comply with organizational rules and procedures, and suggesting procedural, administrative, or organizational improvements” (Dorsey et al., 2017, p. 450; see also Campbell, 2012; Russell et al., 2017). | ||
| Integrity/Moral Courage | Demonstrating honesty and integrity in job-related matters, even when own self-interests might be jeopardized, taking steps to protect the security of military equipment/supplies, and voluntarily reporting thefts, misconduct, and any other violations of military order and discipline (Russell et al., 2017; Sager et al., 2005). | ||
| C. Psychosocial Well-Being | Maintaining emotional control in stressful situations; and not engaging in counterproductive work behaviors. | ||
| C.1. Adapting to Stressful Situations | Maintaining emotional control in stressful situations; noticing/monitoring own signs of stress from combat, work and home life and taking positive steps in managing stress reactions (Pulakos et al., 2000; Waugh & Russell, 2005; Wasko et al., 2012). | ||
| Adapting to Stressful Situations | Maintaining emotional control in stressful situations; noticing/monitoring own signs of stress from combat, work and home life and taking positive steps in managing stress reactions. | ||
| C.2. Counterproductive Work Behavior | Not engaging in delinquent behaviors or behaviors that affect the productivity of the organization (e.g., loafing, tardiness); not bullying, harassing, or hurting others; and not engaging in self-destructive behaviors. | ||
| Loafing and Tardiness | Arriving late for work or not showing up; spending work time on personal activities (e.g., surfing the web) (Spector et al., 2010). | ||
| Abusing Substances and Other Self-Destructive Behavior | Engaging in self-destructive behavior (e.g., alcohol or drug abuse) (Spector et al., 2010). | ||
| Bullying, Harassing, or Hurting Others | Engaging in deviant behavior directed at others (e.g., physical attacks, verbal abuse, harassment) (Campbell, 2012; Russell et al., 2017; Spector et al., 2010). | ||
| Delinquency | Engaging in deviant behaviors directed at the organization (e.g., theft, sabotage) (Campbell, 2012; Spector et al., 2010). | ||
| D. Physical Performance | Meeting fitness standards and sustaining physical performance over time. | ||
| D.1. Physical Endurance | Sustaining physical performance over long periods of time despite lack of sleep and difficult conditions. Adapting to environmental challenges (e.g., weather, terrain) (Pulakos et al., 2000; Wasko et al., 2012). | ||
| Physical Endurancea | Sustaining physical performance over long periods of time despite lack of sleep and difficult conditions. Adapting to environmental challenges (e.g., weather, terrain). | ||
| D.2. Physical Fitness | Meeting military standards for weight, physical fitness, and strength, maintaining own health. | ||
| Physical Fitness | Meeting military standards for weight, physical fitness, and strength, maintaining own health (Sager et al., 2005; Waugh & Russell, 2005). | ||
Note. Typically, several articles contributed conceptually to each dimension. We cited the most salient research for each specific dimension. The higher order category and broad category can be thought of as being supported by subsumed specific dimension citations. We chose to use an intact dimension definition by an author when we believed that the dimension definition reflected all important facets of a dimension.
aEight of 33 performance dimensions had average importance/criticality ratings less than 3.0 (on a 5-point scale) in one or more of the services, suggesting these dimensions were not generalizable across services. We dropped these eight dimensions from further cross-service analysis but retained them for within-service use. Some are more appropriate for training performance than job performance. Average importance/criticality was computed by first taking the average across ratings within service, then computing a unit-weighted average across services, and finally averaging importance and criticality metrics.
Correction Statement
This article has been corrected with minor changes. These changes do not impact the academic content of the article.
Funding Statement
This research was funded under Contract #: [FA8650-14-D6500-0007], Development of Criterion Measures for Evaluating Accession and Classification Testing, with Infoscitex Corporation.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
The data that support the findings of this study are available from the corresponding author, Teresa Russell, upon reasonable request.
References
- Bartram, D. (2005). The great eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90(6), 1185–1203. 10.1037/0021-9010.90.6.1185 [DOI] [PubMed] [Google Scholar]
- Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74(3), 478–494. 10.1037/0021-9010.74.3.478 [DOI] [Google Scholar]
- Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In Schmitt N. & Borman W. (Eds.), Personnel selection in organizations (pp. 71–98). Jossey-Bass. [Google Scholar]
- Campbell, J. P. (2012). Behavior, performance, and effectiveness in the 21st century. In Kozlowski S. (Ed.), The Oxford handbook of organizational psychology: Volume 1 (pp. 159–196). Oxford University Press. [Google Scholar]
- Campbell, J. P., Hanson, M. A., & Oppler, S. H. (2001). Modeling performance in a population of jobs. In Campbell J. P. & Knapp D. J. (Eds.), Exploring the limits in personnel selection and classification (pp. 307–334). Erlbaum. [Google Scholar]
- Campbell, J. P., & Knapp, D. J. (Eds.). (2001). Exploring the limits in personnel selection and classification. Erlbaum. [Google Scholar]
- Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of performance. In Schmitt N. & Borman W. C. (Eds.), Personnel selection in organizations (pp. 35–70). Jossey-Bass Publishers. [Google Scholar]
- Dalal, R. S. (2005). A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology, 90(6), 1241–1255. 10.1037/0021-9010.90.6.1241 [DOI] [PubMed] [Google Scholar]
- Dorsey, D. W., Cortina, J. M., Allen, M. T., Waters, S. D., Green, J. P., & Luchman, J. (2017). Adaptive and citizenship-related behaviors at work. In Farr J. L. & Tippins N. T. (Eds.), Handbook of employee selection (2nd ed., pp. 448–475). Routledge. [Google Scholar]
- Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. 10.1037/h0061470 [DOI] [PubMed] [Google Scholar]
- Garrison, D. R. (1997). Self-directed learning: Toward a comprehensive model. Adult Education Quarterly, 48(1), 18–33. 10.1177/074171369704800103 [DOI] [Google Scholar]
- Goffin, R. D., Woycheshin, Hoffman, B. J., & George, K. (2013). The dimensionality of contextual and citizenship performance in military recruits: Support for nine dimensions using self-, peer, and supervisor ratings. Military Psychology, 25(5), 478–488. 10.1037/mil0000012 [DOI] [Google Scholar]
- Klafehn, J., Ezzo, C., Anderson, L., Taylor, W., Ingerick, M., & Ford, L. (2019). Criterion development for evaluation of the cross-cultural competence assessment system (3CAS) [Unpublished memorandum]. U.S. Army Research Institute for the Behavioral and Social Sciences. [Google Scholar]
- Knapp, D. J., & Campbell, J. P. (1993). Building a joint-service classification research roadmap: Criterion-related issues (AL/HR-TP-1993-0028). Armstrong Laboratory. [Google Scholar]
- Lance, C. F., Teachout, M. S., & Donnelly, T. M. (1992). Specification of the criterion construct space: An application of hierarchical confirmatory factor analysis. Journal of Applied Psychology, 77(4), 437–452. 10.1037/0021-9010.77.4.437 [DOI] [Google Scholar]
- Macey, W. H., & Schneider, B. (2008). The meaning of employee engagement. Industrial and Organizational Psychology, 1(1), 3–30. 10.1111/j.1754-9434.2007.0002.x [DOI] [Google Scholar]
- Matthews, M. D., Eid, J., Johnsen, B. H., & Boe, O. C. (2011). A comparison of expert ratings and self-assessment s of situation awareness during a combat fatigue course. Military Psychology, 23(2), 125–136. 10.1080/08995605.2011.550222 [DOI] [Google Scholar]
- McHenry, J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology, 43(2), 335–354. 10.1111/j.1744-6570.1990.tb01562.x [DOI] [Google Scholar]
- Nesbitt, C. K., Salmon, E., & Kirkendall, C. D. (2020). Army Life Questionnaire. In Knapp D. J. & Kirkendall C. D. (Eds.), Tier One Performance Screen initial operational test and evaluation: Capstone report (Technical Report 1380). (pp. 23–35). U.S. Army Research Institute for the Behavioral and Social Sciences. [Google Scholar]
- O’Shea, P. G., Goodwin, G. F., Driskell, J. E., Salas, E., & Ardison, S. (2009). The many faces of commitment: Facet-level links to performance in military contexts. Military Psychology, 21(1), 5–23. 10.1080/08995600802565595 [DOI] [Google Scholar]
- Oppler, S. J., McCloy, R. A., & Campbell, J. P. (2001). The prediction of supervisory and leadership performance. In Campbell J. P. & Knapp D. J. (Eds.), Exploring the limits in personnel selection and classification (pp. 389–410). Erlbaum. [Google Scholar]
- Oppler, S. J., McCloy, R. A., Peterson, N. G., Russell, T. L., & Campbell, J. P. (2001). The prediction of multiple components of entry-level performance. In Campbell J. P. & Knapp D. J. (Eds.), Exploring the limits in personnel selection and classification (pp. 349–388). Erlbaum. [Google Scholar]
- Organ, D. W. (1988). Organizational citizenship behavior: The good soldier syndrome. Lexington Books D.C. Heath and Company. [Google Scholar]
- Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. (2000). Adaptability in the workplace: Development of a taxonomy of adaptive performance. Journal of Applied Psychology, 85(4), 612–624. 10.1037/0021-9010.85.4.612 [DOI] [PubMed] [Google Scholar]
- Rotundo, M., & Spector, P. E. (2017). New perspectives on counterproductive work behavior including withdrawal. In Farr J. L. & Tippins N. T. (Eds.), Handbook of employee selection (2nd ed., pp. 476–508). Routledge. [Google Scholar]
- Russell, T. L., Rosenthal, D., Paullin, C., & Putka, D. (2006). Addressing active learning concepts in the navy applicant management information system (NAMIS). Unpublished report. [Google Scholar]
- Russell, T. L., Sparks, T. E., Campbell, J. P., Handy, K., Ramsberger, P., & Grand, J. A. (2017). Situating ethical behavior in the nomological network of job performance. Journal of Business and Psychology, 32(3), 253–271. 10.1007/s10869-016-9454-9 [DOI] [Google Scholar]
- Sager, C. E., Russell, T. L., Campbell, R. C., & Ford, L. A. (2005). Future soldiers: Analysis of entry-level performance requirements and their predictors (Technical Report 1169). U.S. Army Research Institute for the Behavioral and Social Sciences. [Google Scholar]
- Sellman, W. S., Russell, T. L., & Strickland, W. J. (2017). Selection and classification in the U.S. military. In Farr J. L. & Tippins N. T. (Eds.), Handbook of employee selection (2nd ed., pp. 697–721). Routledge. [Google Scholar]
- Shuffler, M. L., Pavlas, D., & Salas, E. (2012). Teams in the Military: A review and emerging challenges. In Laurence J. H. & Matthews M. D. (Eds.), The Oxford handbook of military psychology (pp. 282–310). Oxford University Press. 10.1093/oxfordhb/9780195399325.013.0106 [DOI] [Google Scholar]
- Smith, E. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47(2), 149–155. 10.1037/h0047060 [DOI] [Google Scholar]
- Spector, P. E., Bauer, J. A., & Fox, S. (2010). Measurement artifacts in the assessment of counterproductive work behavior and organizational citizenship behavior: Do we know what we think we know? Journal of Applied Psychology, 95(4), 781–790. 10.1037/a0019477 [DOI] [PubMed] [Google Scholar]
- Spector, P. E., Fox, S., Penney, L. M., Brursema, K., Goh, A., & Kessler, S. (2006). The dimensionality of counterproductivity: Are all counterproductive behaviors created equal? Journal of Vocational Behavior, 68(3), 446–460. 10.1016/j.jvb.2005.10.005 [DOI] [Google Scholar]
- Wasko, L., Owens, K. S., Campbell, R., & Russell, T. (2012). Development of the combat/deployment performance rating scales. In Knapp D. J., Owens K. S., & Allen M. T. (Eds.), Validating future force performance measures (Army Class): In-unit performance longitudinal validation (Technical Report 1314) (pp. A-1–A-11). U.S. Army Research Institute for the Behavioral and Social Sciences. [Google Scholar]
- Waugh, G. W., & Russell, T. L. (2005). Predictor situational judgment test. In Knapp D. J., Sager C. E., & Tremble T. R. (Eds.), Development of experimental army enlisted selection and classification tests and job performance criteria (Technical Report 1168). (pp. 135-154). U.S. Army Research Institute for the Behavioral and Social Sciences. [Google Scholar]
- Zaccaro, S. J., Laport, K., & Jose, I. (2012). The attributes of successful leaders: A performance requirements approach. The Oxford Handbook of Leadership. Oxford Handbooks Online. http://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780195398793.001.0001/oxfordhb-9780195398793-e-1 [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author, Teresa Russell, upon reasonable request.