External Review Addresses Validity of ACT's WorkKeys Program for Use in Selection
In 2010, ACT commissioned two external experts to evaluate the WorkKeys system relative to the EEOC's 1978 Uniform Guidelines on Employee Selection Procedures as well as more current professional standards. In a joint report issued in April 2010, two prominent industrial/organizational psychologists, Dr. Frank Schmidt of The University of Iowa and Dr. James C. Sharf of Sharf & Associates, Employment Risk Advisors, Inc., in Alexandria, VA, drew the following conclusions about the validity of the use of WorkKeys assessments as employee selection criteria:
- "The validity of the use of WorkKeys tests as selection criteria may properly be shown by using a content validation approach, which is the approach that is typically used by employers utilizing the WorkKeys job profiling process. The job profiling component involves subject matter experts who work with a trained facilitator to identify key tasks, skills, and skill levels necessary to perform the applicable position(s); the validation is thus employer-specific (i.e., local). This content validation approach is a professionally acceptable method of establishing validity as contemplated in the Uniform Guidelines."
- "The content validity of the WorkKeys system relative to the Uniform Guidelines can also be shown by using the [U.S. Department of Labor's] own database of job titles and corresponding skills, the Occupational Information Network, or O*NET. A detailed comparison of O*NET TASKS, ABILITIES, and WORK ACTIVITIES profiled for jobs at two representative employers who have used WorkKeys confirmed that the WorkKeys assessments assessed important skills and abilities for the jobs in question, as contemplated under the Uniform Guidelines."
- "An employer's use of the National Career Readiness Certificate (or any three or more of the WorkKeys foundational skills assessments) can also be shown to be valid under current professional standards without the need for a local validity study, based on meta-analytic validity generalization research and related research. The cumulative research findings of numerous professionals over many years, including the thirty years of research that has taken place since the Uniform Guidelines were released in 1978, show that [the] WorkKeys [system] meets the requirements for criterion-related validity, content validity, and construct validity, through application of validity generalization research findings and related research."
- "Cognitive ability tests such as the WorkKeys tests are fair to minority applicants and do not require individual fairness studies by every employer who uses the tests."
Several specific questions were also addressed by these nationally respected psychologists in a question-and-answer format, including the following:
Question: Does an employer's use of ACT's National Career Readiness Certificate or any three or more of the WorkKeys foundational skills assessments enable the employer to predict later performance of job applicants?
Answer: Yes. This report shows that the sum of measures of three or more specific aptitudes is a measure of [General Cognitive Ability] which has been shown to predict performance on all jobs.
Question: Under the EEOC's 1978 Uniform Guidelines on Employee Selection Procedures, are the WorkKeys assessments "job-related," content-valid selection procedures?
Answer: Yes. WorkKeys job profiling at the task and skill level meets a level of job analysis detail found in the generally accepted principles and practices of industrial psychology by identifying the critical skills and abilities used in performing work.
Question: Is an employer's use of the National Career Readiness Certificate or any three or more of the WorkKeys foundational skills assessments defensible under the generally accepted principles and practices of industrial psychology?
Answer: Yes.
Question: Does an employer's use of the National Career Readiness Certificate or any three or more of the WorkKeys foundational skills assessments require a local validation study?
Answer: No. The generally accepted principles of validity generalization support use of well studied cognitive ability measures such as WorkKeys without having to conduct a local validation study.
Question: Are WorkKeys cut scores legally defensible?
Answer: Yes. The WorkKeys job profiling system's reliance on subject matter experts to set such cut scores is supported by the generally accepted principles and practices of industrial psychology and is consistent with the Uniform Guidelines.
Question: Can the use of WorkKeys assessments — which are measures of specific cognitive aptitudes (also called cognitive skills) — be validated through "content" validity?
Answer: Yes.
Question: Taking account of the various considerations noted [in the study], does the WorkKeys system meet scientific and professional standards?
Answer: Yes. This report shows that an employer's use of WorkKeys tests and cut scores identified through a WorkKeys job profiling process, or an employer's use of three or more WorkKeys foundational skills assessments (such as those associated with the National Career Readiness Certificate) even without performing a job analysis, meets all relevant professional and scientific standards.
A concluding comment written by Dr. Sharf states:
"WorkKeys assessments are professionally defensible, content-valid measures of verbal, quantitative, and technical/problem-solving skills and abilities that are necessary prerequisites to successfully acquiring skills and performing tasks in the workplace. This conclusion is [also] supported by validity generalization principles, which are now well documented by research in the field and well accepted within contemporary industrial psychology. WorkKeys assessments measure specific cognitive skills/abilities/aptitudes which, when used in combinations of three or more assessments, are a measure of general cognitive ability. Thus, it is my opinion that use of the WorkKeys assessments and the National Career Readiness Certificate as described [in the study] is legally defensible relative to the Uniform Guidelines, and that the WorkKeys assessments are 'job related and consistent with business necessity,' as required under the Civil Rights Act of 1964."
The Authors of the Report
Dr. Frank Schmidt is a chaired professor on the faculty of The University of Iowa's College of Business. He is internationally known for his work in personnel testing, selection, and placement; the relation of selection and other personnel programs to workforce productivity and output; abilities, aptitudes, and personality traits; statistical issues in validation research; validity generalization in employment testing; and other related areas of interest to industrial psychology. Dr. Schmidt has received much recognition from professional societies and organizations, most recently a Career Achievement Award for Scientific Contributions from the Association of Test Publishers in 2007 and The James McKeen Cattell Award for Scientific Contributions to Applied Psychology from the Association for Psychological Science in 2008. An active researcher and writer over his career, Dr. Schmidt has approximately 200 professional publications, including half a dozen books and a meta-analysis software package.
Dr. James C. Sharf is president of Sharf & Associates, Employment Risk Advisors, Inc. He is a former chief psychologist for the U.S. Equal Employment Opportunity Commission (EEOC). In addition, Dr. Sharf has worked actively as a personnel research psychologist for the U.S. Office of Personnel Management and in various related capacities in private-sector companies. Dr. Sharf has taught and held positions at The George Washington University, The American University, and the University of Tennessee. Dr. Sharf has authored and presented almost 100 professional publications, professional society workshops, and guest lectures on fair employment law and public policy and has provided litigation support and expert testimony in over three dozen employment cases.
Access to the Study
Under the Uniform Guidelines, employers "are responsible for compliance with the guidelines" when they use procedures that have been developed by others in making employment decisions, but they may support their use of a particular procedure by referencing validity and fairness studies conducted by test publishers or distributors, in addition to any validity studies that the employer itself might have done and other relevant evidence. Although ACT cannot accept responsibility for any individual's or organization's compliance with applicable laws, if an employer's use of the WorkKeys system is challenged and becomes subject to the validity requirements set forth in the Uniform Guidelines, ACT will make this report and the results of any other relevant validity studies available to the employer at no cost to help demonstrate the employer's compliance with those requirements.
The information and resources provided by ACT regarding use of the WorkKeys system and the National Career Readiness Certificate (collectively, "the Tools") is not intended to be legal advice and employees should consult their own legal counsel regarding the federal, state, and local laws (collectively, "Laws") that may be applicable to them. ACT is not responsible for the use of the Tools in a manner that is in violation of such Laws, and use of the Tools does not ensure compliance with such Laws.
Contact us for more information about this report or call 800/WORKKEY (967-5539).