As we previously reported, on July 5, 2023, the New York City Department of Consumer and Worker Protection (DCWP) began enforcing Local Law 144 of 2021 (the “Law”) regulating the use of automated employment decision tools (AEDT).  In preparation for the July 5 enforcement date, last week, the DCWP published Frequently Asked Questions (FAQ) concerning the use of AEDTs on its fact page for the Law.  The FAQ contain an overview of the Law and general information and guidance regarding bias audit requirements, data requirements, independent auditors, responsibility for bias audits, notice requirements, and complaints.

As explained in the FAQ, the Law applies to employers and employment agencies that use AEDT:

1. for a job located in an office in New York City, at least part time;
2. a fully remote job associated with an office in New York City;
3. employment agencies located in New York City using an AEDT; or
4. agencies that are located outside of New York City, but that use an AEDT for a job located in an office in New York City (at least part time) or for a fully remote job associated with an office in New York City.

The FAQ also clarify that the Law applies not just to the use of AEDT in the final hiring and promotion decision, but also when AEDT are used to substantially assist “at any point in the hiring or promotion process.”  It does not, however, apply when an employer or employment agency uses an AEDT to assess someone, “who is not an employee being considered for promotion and who has not applied for a specific position for employment.”  Thus, according to the FAQ, the Law does not apply when employers or employment agencies employ AEDT merely to scan a resume bank or source potential candidates.

With respect to bias audits, the FAQ state that, while the employer or employment agency using an AEDT is ultimately responsible for ensuring completion prior to the first use, a vendor may commission an independent auditor to conduct a bias audit of its own tool.

The FAQ also confirm that the independent auditor has discretion when determining what “historical data” to use for the requisite bias audit.  For example, the auditor may limit the period of historical data or candidates’ location.  The auditor, however, must explain any such limitation in the bias audit.  Additionally, when the independent auditor uses historical data from multiple employers, the Law does not require the auditor to use data for the same type of position. 

Notably, when using historical data, employers and employment agencies may not impute or infer candidates’ demographic information.   If the auditor lacks sufficient historical data containing demographic information (for example, in cases in which the employer or employment agency does not collect demographic data from candidates), the auditor may use historical data of other employers or employment agencies, or test data.

The FAQ also confirm that independent auditors possess discretion in determining when to use “test data” (a term defined only as anything that is not “historical data”) to evaluate an AEDT.  Test data, per the Final Rule, may be used when there is insufficient historical data to conduct a “statistically significant bias audit,” and the FAQ clarifies that the DCWP “has not set a specific requirement for statistical significance.”  Furthermore, the FAQ state that there are no “set requirements for test data,” except that the bias audit summary must include the source and explanation of the test data, so as to allow for “flexibility and development of best practices.”

What Should Employers Do Now?

As we have stated in previous posts (which can be found here, here, here, here, and here), employers and employment agencies should start by determining whether they are using any AI-enabled tools that qualify as an AEDT.  Employers and employment agencies using an AEDT should immediately begin the process of conducting a bias audit and complying with the notice requirements. 

To learn more about Local Law 144 or about how to address and minimize the legal risks of artificial intelligence (AI), please register for and review one of our on-demand briefings:

• Auto-Hiring? What to Know When Using AI in Hiring
• Achieving Legal Compliance in AI: Minimizing Bias in Algorithms

Please contact Epstein Becker Green’s multidisciplinary team focused on providing artificial intelligence legal services for questions about Local Law 144 or other use cases.