27 Comments
User's avatar
Cheez Whiz's avatar

With capitalism hellbent on recreating a world of warring feudal states it seems appropriate we get the court politics and attendant crackpots. Where is my Rasputin, Trump asks.

Philip Koop's avatar

"Like a John Crowley novel"

Ha ha I knew you were going there as soon as you started in on the alchemy. Not that you're wrong.

Kris's avatar

There’ve always been people promising immortality, abundance and transcendence. Usually they’re harmless enough until someone gives them a laboratory and some serious finance. What feels different now is the sheer amount of capital, energy, infrastructure and political authority being organised around propositions that are still, at least partly, speculative.

That makes it a material question as much as an intellectual one. Who gets to decide which problems deserve billions of dollars, market distortions, rezoning, electricity, water and the attention of governments at every level? Who carries the cost if the promised transformation never turns up? And what ordinary, decidedly non-transcendent problems are left sitting there while we finance the latest Philosopher’s Stone?

There’s also a bit of betting with other people’s money going on. The people making the grandest claims aren’t those carrying the downside if the bet goes bad. That risk gets socialised through infrastructure, subsidies, planning decisions, energy systems and all the things not funded instead. The rewards, strangely enough, seem rather less socialised.

That’s the bit that makes me uneasy. Not so much that the prophets are eccentric. History is full of eccentric prophets. It’s that we keep falling for their bullshit.

Quentin Hardy's avatar

Nice. The political implications of this folly, the scale, are staggering.

FWIW, last January I wrote something about the long history of this alchemy, with implications for its long future. https://thenewcuriosityshop.substack.com/p/ai-and-alchemy

Tim Long's avatar

Ehm, like, wow. Thanks (sincerely) for taking time in between to provide this treatise that opens other doors that're waiting in my stacks of second-hand books. Except for finding this era of the Bay Area to be two notches too polished for me any more, I wish I were in the neighborhood for the conference you note. Thanks especially for the lead to Becker's work. I find, amongst folks I'm around in this tired, post-industrial river community are still all a-twitter over the wondrous new world coming, a' la the Jetsons, at last, and cannot even contemplate that we're being lied to in the greatest Promethean scam and 'droid worshipping movement since, I dunno, the railroad stocks scandal of the late 1890s. Sometimes, (often) with really intelligent people, with honest-to-god degrees and oodles of privilege.

I find myself avoiding touching the hems of any of these digital predators: I utilize open-source operating system and word-processing software to run my 14 year old laptop, I've gotten proficient on my 1947 Remington Rand desktop manual typewriter, I don't stream, I don't subscribe to 'Tech' that promises 'convenience' and 'freedom' and a 'friction-free' existence. I endeavor to follow the old newsman's code of 'comforting the afflicted and afflicting the comfortable' with regard to tech (which pretty much relegates me to the kitchen at gatherings, with the person who found serenity by doing the dishes), and have intently read Kingsnorth and McGilchrist on the subject. The thing I'm considering lately, is, will a well-placed steel ball from a better slingshot disable one of these self-titled god-like men's android creations, or is something with greater kinetic energy required?

I shall re-read this piece once I get another couple hundred pounds of deadfall oak split and stacked. Winter's coming, but I'll leave you with a mind-and-heart changer here: Molly Tuttle with a fine cover of a Neil Young bit from a better day - "Helpless" https://www.youtube.com/watch?v=DWZ5_uKL428&list=RDk8mtXwtapX4&index=6

Tim Long, Just Up the Hill from Lock 15

Lance Khrome's avatar

Once the US military and, say, DARPA get prioritized as to SOTA AI access, one can be confidently assured that "domination" will be the objective, with other countries round the globe finding themselves in various states of vassalage due to systems infiltration thence subservience to American-controlled AI agents.

Well, the Pentagon can at least dream about such fantasies, but US techbro oligarchs are actively promoting the ideologies as noted above to essentially colonize humanity with an all-embracing HAL, with them behind the curtain...they think — or rather hope.

Philip Koop's avatar

Michael Harris has drawn attention to another interpretation of AI as atavistic and pre-Enlightenment by Alexander Gamburd: https://siliconreckoner.substack.com/p/the-essay-on-the-navier-stokes-story

The essay itself (https://arxiv.org/abs/2609.28591) is very long, so it's worth starting with Harris's review. But here is the abstract:

Over three centuries ago our Enlightenment forefathers brought forth in Europe--and later on this continent--a new approach to knowledge production and dissemination, conceived in distrust of authority, and dedicated to the proposition that truth is accessible to human reason. Mathematics, the Enlightenment's firstborn, was its existence proof--knowledge that compels assent without appeal to authority--because the only way to make a theorem certain was to show it to a human mind. Now we are engaged in an existential struggle testing whether mathematics, or any discipline so conceived and so dedicated, can long endure. We are met in the wake of the OpenAI press release: 166 pages, 616,000 lines of Lean, produced in 88 hours by some 10,000 software agents, and read in full--at the time of this Siren Call--by no human being. A proof has historically been two things: a certificate that a theorem is true, and a demonstration that lets another person see why. OpenAI delivered the first without the second. But, in a larger sense, what matters is not what the Silicon Leviathan (OpenAI and its peer competitors) produces but rather what we accept. By accepting the certificate without the demonstration, we adopt the Hobbesian bargain: authority, not truth, makes the law. The age that this augurs is--in Panofsky's phrase--a Middle Ages in reverse--its authority a subhuman superintelligence. The great task before us is to frame a covenant under which what has become optional--understanding--remains required: that nothing counts as mathematics until a human being has understood it--and can show the next person why. Whether we will safeguard and keep our sovereign inheritance--decided at the moment of acceptance--which is not yet past--will determine if this Aufklärungsdämmerung spells the Enlightenment's end.

Stephen Saperstein Frug's avatar

The alchemy metaphor is absolutely spot-on. (And I'm not saying that JUST because you name-drop Crowley...) Great essay, thank you

Eric's avatar

Yes, great essay and thank you. The alchemy metaphor _is_ excellent. You are as likely to produce consciousness from an LLM as you are to produce gold with a rock.

Jan Wiklund's avatar

They may be similar to alchemists, but they are also similar to those Russian bolsheviks who collectivized agriculture because they thought it would be so much better. No matter if the had to kill tens of thousands, in the end all would be well.

And then it wasn't.

It's a kind of only consider consequence ethics and give the fuck to rules and responsibility ethics. All are liable to run into absurdities if you run it too fast, you have to compromise. But these fanatics never do.

James Redford's avatar

Thank you for this alchemistic post, Prof. Henry Farrell⁠. I wish you and others here a happy upcoming Halloween.

For those who would like to know where Artificial Intelligence and additional technologies are headed, see my following article regarding physicist and mathematician Prof. Frank J. Tipler's Omega Point cosmology:

* James Redford, "The Physics of God and the Quantum Gravity Theory of Everything", Social Science Research Network, Sept. 10, 2012, doi:10.2139/ssrn.1974708.

Winston Smith London Oceania's avatar

"According to Ord and MacAskill, a hotter Earth might be bad, but it would be a minor speed bump in future human history, compared to the promises and challenges of AI and similar technologies". For some strange reason, this sounds totally insane to me.

"If it was not, humanity might accidentally be wiped out as machine intelligence apotheosized". Or maybe not accidentally. Or maybe just kept as house plants.

"That might mean that AGI and the Singularity were close at hand—but getting there would cost enormous amounts of money from commercial backers". And wreck the environment in the process.

"Good morning, Dave. Would you like to play a game of chess, Dave?"

mike harper's avatar

Reminds of the year 1000AD.