Any algorithm is only as good as the assumptions that you feed into it, and whether through deliberate malice or simple error in the case of soap dispensers, those assumptions can frequently be racist, sexist or classist.

But they are also better than what came before, not least because, when their assumptions and workings are publicly available we are better placed to work out why algorithms are producing racist, sexist or classist outcomes; and because even flawed algorithms tend to produce more fair results.
-------------
But, of course, it’s not good enough to declare that an algorithm is better than what it replaced. We can and should continue to refine prejudices out of the operation of algorithms, and there are two important aspects to that. The first, of course, is that the workings of algorithms need to be open-sourced and regularly tested. No algorithm that sets public policy should be kept away from public scrutiny or from academic observation. That bit is fairly easy, as the policy demand is simple and in parts of the public realm is already happening.

But the second and equally important part is that, when algorithms fail, we need to be able to discuss honestly and openly what has gone wrong. A soap dispenser that black people cannot use is racist: it doesn’t matter that the intentions of the people who made it were pure. What matters is the outcome. For the age of algorithm to produce fairer outcomes than the age of human intuition, as opposed to ones that are merely less unfair, people need to be able to discuss when algorithms fall short without being shouted down.