Prove It or Lose It: New York City Cracks Down on AI Hiring Software

Artificial Intelligence (“AI”) programs have gained notoriety by injecting ease into otherwise burdensome and difficult daily tasks. However, as with most innovative advancements, AI has also drawn concern from skeptics regarding the everyday effects such programs may have.

With fast-growing scrutiny around AI’s use and evolvement, New York City has recently taken its own stab at regulating how these programs are implemented, specifically in the world of hiring candidates. Effective this month, New York City has rolled out a new rule that requires any automatic employment decision tool[1] (“AEDT”) to pass a third-party audit in order for businesses to continue using them (the “AI Law”). The AI Law requires that the AEDT undergo an extensive review to expose whether the program is free from any racist or sexist bias. Not only must companies publish their results, but they are prohibited from using any programs that haven’t been audited. In other words, businesses must prove it, or lose it.

Specifics of the AI Law

Under the AI Law, the use of any AEDT will be illegal unless: (1) the company has utilized a bias audit within a year of its use of the AEDT; (2) the results from the most recent audit are published; and (3) each candidate is provided with notice of the AEDT’s use and is given the opportunity to request an alternative selection process. The AI Law defines the required bias audit as “an impartial evaluation by an independent auditor” and is subject to certain minimum requirements, including a calculation of the selection rate for each race/ethnicity and a sex category and comparison of the same to determine an “impact ratio.” In order to further promote regulating the use of AEDTs, New York City has also provided guidance for individuals to file complaints in order to report an employer or employment agency that used an AEDT but failed to comply with the AI Law.

Employers can protect themselves from violating the AI Law’s provisions by first identifying whether they are utilizing any tools that would be subject to the AI Law and by ensuring that a proper audit is completed. A tool is subject to regulation if: (1) it is used relative to decisions regarding hiring and/or firing; (2) issues a score, classification, or recommendation that is used as: (a) the sole criterion; (b) a criterion that has considerable more influence than another; or (c) a criterion used to overrule other conclusive factors such as human decision-making; and (3) is derived from “machine learning, statistical modeling, data analytics, or artificial intelligence,” as more fully defined in the final regulation. Once a sufficient audit is complete, employers should consider developing a universal protocol for: (1) providing notice of their AEDT use; and (2) publishing the results in conformance with the AI Law in order to avoid any issues, such as complaints regarding the sufficiency and/or location of the posted content.

Conclusion

While certain restrictions may act as a deterrent to some, the sheer number of applicants being screened by companies will likely compel companies to continue seeking machine-guided ways to vet candidates for open positions. In doing so, it is important that employers take heed in their use of AEDT, not only in New York City, but around the country as attention remains on AI technology and the cry for further regulation gets louder.

——————————————————————–

This DarrowEverett Insight should not be construed as legal advice or a legal opinion on any specific facts or circumstances. This Insight is not intended to create, and receipt of it does not constitute, a lawyer-client relationship. The contents are intended for general informational purposes only, and you are urged to consult your attorney concerning any particular situation and any specific legal question you may have. We are working diligently to remain well informed and up to date on information and advisements as they become available. As such, please reach out to us if you need help addressing any of the issues discussed in this Insight, or any other issues or concerns you may have relating to your business. We are ready to help guide you through these challenging times.

Unless expressly provided, this Insight does not constitute written tax advice as described in 31 C.F.R. §10, et seq. and is not intended or written by us to be used and/or relied on as written tax advice for any purpose including, without limitation, the marketing of any transaction addressed herein. Any U.S. federal tax advice rendered by DarrowEverett LLP shall be conspicuously labeled as such, shall include a discussion of all relevant facts and circumstances, as well as of any representations, statements, findings, or agreements (including projections, financial forecasts, or appraisals) upon which we rely, applicable to transactions discussed therein in compliance with 31 C.F.R. §10.37, shall relate the applicable law and authorities to the facts, and shall set forth any applicable limits on the use of such advice.

[1] Defined as any process “derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making.”