AI and particularly the use of algorithms is being implemented in a number of ways in the workplace, whether during the recruitment process, as part of a means of assessing performance, determining pay and promotions, and even as part of the selection process for redundancies (so-called ‘robo-firings’). However, it is important to remember that bias and, indeed, unlawful discrimination can occur for a number of reasons when using AI, including as a result of the objectives set for the algorithm, the data on which the algorithm is trained, the (often opaque) causal links identified by the algorithm, or the data used when running the algorithm.
In a now–infamous reported case, something clearly went wrong where one CV screening tool identified that being called Jared and playing lacrosse at high school were the two strongest predictors of high performance in the job. There has also been much publicity around the use of facial recognition technology. Uber drivers have claimed that a ‘racist’ algorithm left them locked out out of the ride-hailing app they were required to use, as it was biased against non-white faces.
Where algorithms such as these are used, disadvantaged employees might argue that an algorithm-based decision has directly or indirectly discriminated against them. Further, the obligation to make reasonable adjustments under disability laws poses further challenges. If such cases were to proceed to court, employers may need to prove that they did not discriminate or that the indirectly discriminatory impact of the algorithm is objectively justified. In many cases, the employer will not understand how an algorithm works (or even have access to the source code), so will face a significant challenge in doing this. Another key question for employers to consider is who will be on the hook if the algorithms they use in the workplace are found to be biased or discriminatory. Some are suggesting that employers should seek indemnities from the third party companies who developed the algorithm.
It remains to be seen how the courts would grapple with matters like this, but with the use of AI in the workplace rapidly becoming more widespread, it will no doubt only be a matter of time before these matters are tested.
