The downside of robot solutionism

Mapping the Abelard Snazz feedback loop

The last couple of weeks have been quiet thanks to other writing with other people, some of which is about to come out. In the meantime, this article reminded me of one of Alan Moore’s greatest and least-well-known creations. Abelard Snazz, “the man with the two-storey brain,” offers “to handle complex problems with even more complicated solutions.”

Someone at the Economist is following in Snazz’s footsteps. The problem: AI is “breaking the British state” by making it too easy for citizens to complain.

Citizens frustrated by the poor deal they get from the authorities are turning to artificial intelligence to file objections and appeals, and claim their dues. The resulting deluge of complaints, and demands, will overwhelm bureaucracies built for the age of the post and the telephone. Too little is being done to prevent the state from drowning.

The worry isn’t that the AI is bad, but that it threatens to become too good.

AI is superb at dealing with bureaucracy. It can digest small print, ferret out loopholes and draft appeal letters in seconds. Autonomous AI agents will accelerate the trend, filing tax complaints with minimal intervention. Unlike humans faced with officialdom, AI agents will never lose their temper—or the will to live. Officials complain about AI “slop” and hallucinations, but as the technology improves they will face the opposite problem: demands as well-crafted as a first-class lawyer’s.

This is structurally similar to the problems of the planet Twopp, where the state was overwhelmed by street crime, until it called in Abelard Snazz’s specialized consultancy services. Substitute in ‘thieving criminals’ for ‘aggrieved citizens’ and the analogy is nearly perfect.

It only took a few moments of cogitation for Snazz to come up with a solution.

Similarly, the Economist:

stop creating entitlements that are ripe for AI-fuelled claims. … prune the mass of procedural rights … The most important task is to act faster and more radically on remaking the state with AI. That means not just speeding up the old cycle of applications, decisions and appeals, but replacing it altogether. … Rather than just handling locals’ objections, a new AI-driven planning system could decide for itself whether a housing development meets a zoning code. Rather than processing benefits claims, it could devise personalised welfare interventions.

As the citizens of Twopp discover, automating the state turns out to have its own problems. Then, as now, robots really love proceduralism.

Abelard Snazz, of course has a solution to his previous solution.

The resulting system of robot police versus robot robbers expands and expands and expands, sucking in innocent bystanders and making ordinary life impossible.

So too for other efforts to counter the bureaucratic problems of AI with more AI. A couple of years ago, the New York Times described how this was already happening in US health insurance.

The list of treatments that needed pre-approval from insurers seemed ever broadening, and the denials seemed ever rising. … For a growing number of doctors, A.I. chatbots — which can draft letters to insurers in seconds — are opening up a new front in the battle to approve costly claims, accomplishing in minutes what years of advocacy and attempts at health care reform have not. … As doctors use A.I. to get faster at writing prior-authorization letters, Dr. Wachter said he had “tremendous confidence” that the insurance companies would use A.I. to get better at denying them. “You have automatic conflict,” said Dr. Wachter, who wrote a book about digital technology in medicine. “Their A.I. will deny our A.I., and we’ll go back and forth.”

The Economist’s leader writer waxes lyrical about the burgeoning of the AI state as rights and entitlements are pruned away, but has little to say about how citizens are likely to respond. Like other such proposals (the author seems to be riffing on the notion that the UK can e.g. solve its planning problems by making it harder for people to object), this tacitly assumes that citizens will not find other attack surfaces. As per Dan Davies, this is a heroic assumption.

Abelard Snazz, in fairness, proposes a more radical solution to his solution to his solution.

The consequences are not entirely satisfactory.

Beneath Alan Moore’s jokes, there lurks a very useful general point. Abelard Snazz believes that the solution is always more robots, even when robot solutionism is obviously the root cause of the problem. So too for the Economist, arguing that the solution to the problems caused by AI is more AI, with a healthy dollop of rights-pruning on the side. As a general matter, trying to solve a systemic problem by doubling down risks creating out-of-control feedback loops.

If you look around, you will see similar approaches to problem solving everywhere in AI. Frontier models are threatening to devastate cybersecurity? I see frontier models deployed for defense as well as offense. Ha, ha, ha! It’s brilliant. Ha, ha, ha.

While you can fill in the ‘you’re a genius, master’ yourself, as you like, such ideas may not always be wrong in the particular. Abelard Snazz; bureaucrats versus complainants; offensive versus defensive cybersecurity; these are all examples of technology driven arms races. Once an arms race has begun, strategic uncertainty means that it may be difficult to prevent escalation, even if it ends up making everyone worse off.

Equally - and this is where the Economist article misses the mark - you need to think about these dynamics as a system. How does the solution that you are proposing interact with the problem that it is intended to solve? Which other problems is it likely to give rise to? How will the solution be gamed? (and it will be gamed - AI bureaucracies will absolutely have their own politics). And so on. If you don’t think in this way, you’re liable to Abelard Snazz it up, providing ever more complicated solutions for already-complex problems, many of which you have created yourself thanks to your past problem solving efforts.

Next up, another ‘how science fiction explains the world we’re in’ special, likely on Monday.