Are Your AI Selection Tools Validated? OFCCP Provides Guidance for Validation of AI-Based Algorithms

We have long counseled employers using or contemplating using artificial intelligence (“AI”) algorithms in their employee selection processes to validate the AI-based selection procedure using an appropriate validation strategy approved by the Uniform Guidelines on Employee Selection Procedures (“Uniform Guidelines”).  Our advice has been primarily based on minimizing legal risk and complying with best practices.  A recently updated Frequently Asked Questions (“FAQ”) from the Office of Federal Contract Compliance Programs (“OFCCP”) provides further support for validating AI-based selection procedures in compliance with the Uniform Guidelines.

On July 23, 2019, the OFCCP updated the FAQ section on its website to provide guidance on the validation of employee selection procedures.  Under the Uniform Guidelines, any selection procedure resulting in a “selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5) (or eighty percent) of the rate for the group with the highest rate will generally be regarded by Federal enforcement agencies as evidence of adverse impact,” which in turn requires the validation of the selection procedure.  These validation requirements are equally applicable to any AI-based selection procedure used to make any employment decision, including hiring, termination, promotion, and demotion.

As stated in the Uniform Guidelines, and emphasized in the FAQ, the OFCCP recognizes three methods of validation:

1. Content validation – a showing that the content of the selection procedure is representative of important aspects of performance on the job in question;
2. Criterion-related validation – production of empirical data demonstrating that the selection procedure is predictive or significantly correlated with important aspects of job performance; and
3. Construct validation – a showing that the procedure measures the degree to which candidates possess identifiable characteristics that have been determined to be important in successful performance on the job.

With the exception of criterion-related validating studies, which can be “transported” from other entities under certain circumstances, the Uniform Guidelines require local validation at the employer’s own facilities.

If a selection procedure adversely impacts a protected group, the employer must provide evidence of validity for the selection procedure(s) that caused the adverse impact. Thus, it is crucial that employers considering the implementation of AI-based algorithms in the selection process both conduct adverse impact studies and be prepared to produce one or more validation studies.

The new FAQ also provides important guidelines on the statistical methods utilized by OFCCP in evaluating potential adverse impact.  In accordance with the Uniform Guidelines, OFCCP will analyze the Impact Ratio – the disfavored group’s selection rate divided by the favored group’s selection rate.  Any Impact Ratio of less than 0.80 (referred to as the “Four – Fifths Rule”) constitutes an initial indication of adverse impact, but OFCCP will not pursue enforcement without evidence of statistical and practical significance.  For statistical significance, the OFCCP’s standard statistical tests are the Fisher’s Exact Test (for groups with fewer than 30 subjects) and the Two Independent-Sample Binomial Z-Test (for groups with 30 or more subjects).

With the publication of this new FAQ, employers – and particularly federal contractors – should be sure to evaluate their use of AI-based algorithms and properly validate all selection procedures under the Uniform Guidelines.  Moreover, although not addressed in the OFCCP’s new FAQ, employers should also ensure that their AI-based algorithms are compliant with all other state and federal laws and regulations.  Additional recommendations from Epstein Becker & Green’s Artificial Intelligence strategic industry group can be found here and here.