Chiang worries about the future of AI for its impact on labor and management of corporations. McKinsey is a notorious consulting firm that has worked alongside 90% of the top 100 US companies to do everything from "turbochargins" the opioid epidemic alongside Purdue Pharma, or Introducing more brutal policing tactics inside Riker's island in an attempt to reduce riots, and then fabricating results to indicate success just to name a few.
A.I exists as a black box to produce solutions that presents as a neutral objective decision maker, but can easily be warped to suit the needs of anybody who can commission one. Proponents of AI often imagine that simply putting check in place with the AI will allow it to be something that only benefits humanity, but fail to consider that there is nothing stopping a sufficiently motivated company from creating one that will.
I share a lot of the worries with Chiang, alongside my reading of AI machines aren't 'hallucinating'. But their makers are, I fear that the public's lack of knowledge (and the industry's lack of transparency) when it comes to machine learning will struggle to understand how non-objective a lot of the outputs from those tools can be.
Ted also discusses the proposed solutions from the A.I Industry like Universal Basic Income; but points out that without any semblance of a UBI framework in the US already, they are effectively passing the buck to the government to mitigate the impact of their work. He equates this to the leftist accelerationists, who believe the only way to truly exhaust neoliberal capitalism is to draw it to it's logical conclusion. I would argue that this is far more passive than accelerationists and lacks any of the actual desire for the world to be any different than it is. The only thing A.I companies want is to be richer, and I doubt many actually consider the better world they promise beyond its marketing potential.
Many of the defenders of AI technologies refer to people who are skeptical of its ability to positively impact humanity as neo-luddites. They misunderstand that the luddites themselves fought for labor rights rather than against technology, and their reputation as anti-progress is a result of a centuries old smear campaign they are both perpetuating and recreating.
Just as A.I. promises to offer managers a cheap replacement for human workers, so McKinsey and similar firms helped normalize the practice of mass layoffs as a way of increasing stock prices and executive compensation
If you think of A.I. as a broad set of technologies being marketed to companies to help them cut their costs, the question becomes: how do we keep those technologies from working as "capital's willing executioners"?