Job Analysis Overview

Web page under construction.

Job analysis is a fundamental part of Human Resource departments and the various functional areas. Analyzing a job involves the determination of what tasks make up a job, the tasks’ relative importance, and what knowledge, skills, and abilities (KSAs) are necessary to successfully carry out those tasks. Job analysis results may be utilized for a variety of personnel management purposes but are especially important for selection and promotional decisions.

Job analysis is undertaken prior to selection and promotional testing in order to determine what attributes qualify an individual for a specific job. In other words, the aspects identified through comprehensive job analysis serve as the criteria for tests and other assessments used in selection or promotion.

If an HR department wants to meet the legal requirements of personnel testing, the 1978 Uniform Guidelines on Employee Selection Procedures indicate that content-valid employment tests should have support that the content of the test is representative of the important aspects of the job position. Job analysis provides the documentation necessary to demonstrate content validity of personnel tests and supplies evidence for legal defensibility of selection methods.

The Uniform Guidelines on Employee Selection Procedures (Equal Employment Opportunity Commission, 1978) specify in Section 14 (c) 2:

"Job-analysis for content validity. There should be a job-analysis which includes an analysis of the important work behavior(s) required for successful performance and their relative importance and, if the behavior results in work product(s), an analysis of the work product(s). Any job-analysis should focus on the work behavior(s) and the tasks associated with them. If work behavior(s) are not observable, the job-analysis should identify and analyze those aspects of the behavior(s) that can be observed and the observed work products. The work behavior(s) selected for measurement should be critical work behavior(s) and/or important work behavior(s) constituting most of the job."

The process of job analysis can involve several methods of information-gathering, including observation, group meetings, interviews and surveys.

L.J. Craig & Associate's Inc. process

Our job analysis process typically begins with review of any pre-existing materials such as past job analyses and job descriptions. Next, interviews and job observations are conducted with a representative sample from the organization, with the purpose of obtaining task-level information about the job in question. The interviews and observations are complementary, as something that is accidentally omitted during an interview may be easily observed, and rarely performed but critical tasks that are not easily observed can be discussed in an interview. Job incumbents as well as supervisors – who may be more familiar with all aspects of the job – are interviewed. Interviews and observations provide I/O Solutions with a preliminary understanding of what skills and abilities are needed for successful performance in the specific job.

The Job Analysis Questionnaire

Information gathered through interviews and observation informs the creation of the Job Analysis Questionnaire (JAQ), a multisection survey instrument aimed at determining the knowledge, skills and abilities suitable for selection or promotional testing. Job incumbents’ ratings of task statements are obtained in the first section. The task statements are written to summarize major aspects and activities of the job and are organized by duty area (meaningful groupings of similar tasks). Some examples of duty areas include "Management and Leadership Activities" and "Training Activities." Using five-point Likert rating scales, respondents indicate which tasks are important and used frequently in their job position. They also indicate whether the tasks are applicable to their job. An example of a task statement is "Review in-box items in order to determine responsibilities for self and staff." The essentiality of each task is then calculated by using a proprietary formula that combines importance and frequency ratings for each task.

KSAs and Duty Areas

Ratings of importance and applicability for job knowledge, skill and ability areas are also collected in the JAQ. These ratings reflect the importance of the ability and knowledge areas to the job as a whole, across all duty areas, ensuring that the respective ability and knowledge areas are appropriate for testing. An example of a job knowledge area is "Agency Policies and Procedures." An example of an ability area is "Decision-Making Ability." Next, respondents rate the relative importance of duty areas.

Linkage

Finally, job duties are linked together with knowledge and ability areas by respondents in the final section of the JAQ. In conjunction with subject-matter expert input, this linkage section serves as a guideline for test formulation. Using a Likert scale, respondents rate to what degree a knowledge or ability is important to the performance of a duty area. For example, raters may be asked to judge to what degree the ability area of reading comprehension is important in the "Administrative Activities" duty area. The final knowledge linkage and ability linkage results provide a standardized score representing the importance of the knowledge or ability area across all duty areas, weighted based on the relative importance of the duty areas.

The results of the linkage analysis are used to determine an examination plan grounded in empirical data that is specifically linked to the job analysis. For more information, please see "Best Practices Related to Promotional Job Knowledge Exams."

References

Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor & Department of Justice (1978). "Uniform guidelines on employee selection procedures." Federal register, 43 (166), 38290-38315.

Morgeson, F. P. & Campion, M. A. (1997). "Social and cognitive sources of potential inaccuracy in job analysis." Journal of Applied Psychology, 82(5), 627-655.

Standards for Demonstrating Content Validity Evidence

What is Content Validity Evidence?

Evidence of content validity generally "consists of a demonstration of a strong linkage between the content of the selection procedure and important work behaviors, activities, worker requirements, or outcomes of the job" (Principles, 2003). In other words, a test is content valid to the degree that it "looks" like important aspects of the job. Of course, the process of demonstrating that a test looks like the job is more complicated than making a simple arm’s-length judgment. Inferences of job-relatedness are made based on rational judgments established by a set of best practices that seek to systematically link components of a job to components of a test.

When is it Appropriate?

A test can be supported by content validity evidence to the extent that the construct that is being measured is a representative sample of the content of the job or is a direct job behavior. Tests that assess job knowledge, supervisory skills and communication skills would be appropriate to validate with content validity evidence; however, tests that assess aptitude, personality, or more nebulous and multifaceted constructs like these should not be validated using content evidence.

Method of Establishing Content Evidence

In order to establish evidence of content validity, one needs to demonstrate "what important work behaviors, activities, and worker KSAOs are included in the (job) domain, describe how the content of the work domain is linked to the selection procedure, and explain why certain parts of the domain were or were not included in the selection procedure" (Principles, 2003). Further, it must be demonstrated that the selection procedure that measures a skill or ability should closely approximate an observable work behavior, or its product should closely approximate an observable work product (Uniform Guidelines, 1978). The method used to accomplish this goal involves a number of steps:

1. conduct a job-task analysis to identify essential job tasks, knowledge areas, skills and abilities;

2. link job tasks, knowledge areas or skills to the associated test construct or component that it is intended to assess;

3. use subject-matter experts internal to the department (where possible) to affirm the knowledge or skills that will be assessed in the test and the appropriateness and fidelity of the questions or scenarios that will be used (these can be accomplished in a number of ways, including the use of content-validity ratios [CVR] – systematic assessments of job-relatedness made by subject-matter experts);

4. document that the most essential knowledge areas and skills were assessed and explain why less essential knowledge and skills were excluded.

"The documented methods used in developing the selection procedure constitute the primary evidence for the inference that scores from the selection procedure can be generalized to the work behaviors and can be interpreted in terms of predicted work performance" (Principles, 2003). Therefore, the technical report that is used to document the methodology employed to develop the test is sufficient to serve as the evidence of content validity.

Rank-Ordering Candidates based on a Content-Valid Selection Procedure

In order to use rank-ordered selection, a test user must demonstrate that a higher score on the selection procedure is likely to result in better job performance. "Where a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance" (Uniform Guidelines, 1978). To the extent that the scoring system awards points based on the demonstration of knowledge or behaviors that distinguish between minimal and maximal performance, the selection procedure is likely to predict job performance.