Artificial intelligence applicant tracking systems are supposed to cut the work, but one major U.S. company discovered they also were cutting valid candidates.
In 2014, e-commerce giant Amazon began experimenting with its own software to streamline the process of sorting through piles of applicant resumes. The program had built-in artificial intelligence to look for the skills the company valued and then rank applicants with one to five stars.
“Everyone wanted this holy grail,” an unnamed source at Amazon told Reuters. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”
But recently, Amazon’s tech gurus discovered a problem: The system seemed to be docking applicants for being female.
Artificial intelligence engines make decisions based on existing data and a certain amount of “technology smarts” that can make logical predictions based on that data. Since most of Amazon’s programmers and technology employees over the past 10 years have been male, that’s the data the system used to create its sorting parameters.
In short, the algorithms in the technology based its decision-making on past results, the system “learned” to prefer male candidates.
For many HR departments, applicant tracking system (ATS) software seems like a savior. Such software can—and does—provide a valuable service in creating shortlists of candidates. But be careful how many limiting parameters are requested, or the system may end up excluding worthy candidates. Trusting too deeply on the results from ATS systems and other applicant sorting software and services can leave a company with incomplete search results at best and a discrimination lawsuit at worst.
LinkedIn also uses algorithmic ranking of job applicants, offering results to employers as a service. But John Jersin, vice president of LinkedIn Talent Solutions, doesn’t consider the service as a replacement for traditional recruiters. “I certainly would not trust any AI system today to make a hiring decision on its own,” he tells Reuters. “The technology is just not ready yet.”
Artificial intelligence algorithms can be valuable assistants to busy HR departments, but the temptation to rely on them too heavily is dangerous, civil rights groups say. “We are increasingly focusing on algorithmic fairness as an issue,” says Rachel Goodman, a staff attorney with the Racial Justice Program at the American Civil Liberties Union (ACLU).
