I'll take the coming of Machine Shiva (thanks Ary!) seriously when they stop talking about what will/might happen and start talking about how. This may be a byproduct of theorizing about something they don't understand (to be fair, because it doesn't exist yet), but to me both the Doom and Paradise options reek heavily of the Power of Positive Thinking, willing a preferred reality into being like an Ayn Rand character.
“Accepting the reality of the coming machine god after it’s solved Navier-Stokes and dozens of other longstanding open math problems (while dramatically ramping up in capability every month), is sort of like accepting Jesus after he’s returned to earth on the gleaming cloud.” Wild statement and one that’s not aging that well giving some of the recent reporting. Oof.
Why is it always Machine Jesus? Why not Machine Shiva, Lord of the Dance of creative destruction? These machine god types need go expand their horizons.
You know the answer to that rhetorical question. Which deity/deities would they use in India? Would the ostensibly atheist communist China use a different term for the same idea?
This is a great post. I'm personally very skeptical about the imminence of space-faring machine gods, but I end up really frustrated by most skeptics because of how dismissive they are of true AI believers. (There are already a bunch of comments on this post along those lines!)
I don't think it makes sense to dismiss people who believe in imminent world-transforming superintelligence when a lot of them are very influential and have spent their whole lives being publicly very smart; any attempt to understand, catalogue and contextualize their worldview is valuable, in addition to providing alternatives. I think that generalizing and extrapolating, which are often the tools of smart people, have led a lot of AI people astray in understandable ways, and trying to understand how this has happened with someone like Scott Aaronson is very valuable and far more important than laughing at him.
Replace AI people with priests and the result is much the same. Wrong beliefs and actions bring "hellfire and eternal suffering". The right beliefs and actions will bring a "glorious future." Human nature was ever thus.
I do agree or at least strongly suspect it’s a case of a bunch of unfalsifiable beliefs filling a God-shaped hole in people’s brains (but that doesn't mean they can’t be taken seriously!)
As a retired proletariat I use my free chrome AI for mundane issues like health questions and American football. I also win every debate with the machine by use of pointed closed end prompts that forces the machine to rethink general answers to specific replies. Not anywhere in context of "Machine God" context but a time killer for me. Your piece is "Heavy" and interesting! Thanks!
'All the worrying behaviour has happened in one context' seems overly optimistic given the labs have no obligation to disclose issues and more are still coming out. And the worrying behaviour is really worrying given how fast things are moving. I don't see how we have time to consider what you write about without slowing the labs down.
Isn't the rapid creation of a new "god" analogous to the rapid colonization of a land and the priests converting the natives to the conquerors' religion?
Your reductionist "AI as cultural and social technology" is fine, but it runs up against millennia of human thought. Gods as superior beings have been believed by humans for millennia. Indeed, a "father-God" of Abrahamic religions is still believed by a large fraction of humanity, particularly in the USA. Neurobiologists and cognitive scientists might try to explain human thinking by lower-level processes, but we still regard exceptional people as special and shower them with accolades and status. While Newton may have well understood that "if I have seen further [than others], it is by standing on the shoulders of giants", we still regard Newton as an exceptional genius who made all his discoveries by himself. Our evolution as social apes, with a hierarchical structure, has resulted in human social systems that place a single individual at the top of the hierarchy, whether monarch, dictator, or president. (Our current president believes he is a "stable genius" who is never wrong. Why should we treat any remarkably [simulated] "intelligent" machine any differently than the way we treat humans at the top levels of a social hierarchy? We can only hope to shape their alignment with the needs of the population by various means of persuasion from close advisors to the ballot box, or in extremis, by defeat in battle.
TL;DR, we are disposed to anthropomorphize anything that looks like having agency, and elevate that agency, if powerful enough, as a god. This is a huge hurdle to overcome, and therefore will be the "attractor" around which "the AI alignment problem" will revolve. Good luck redirecting this.
I'm not worried about AI in and of itself per se. It becomes a problem when it's incorporated into the physical infrastructure, such as the power grid and water supply. We've already seen some of the vulnerabilities exposed with mere computer control - such as to hacking.
The big problem isn't AI becoming "god like" so much as it is the purveyors/profiteers shoving it down our throats a quickly as they can, so as to maximize profits - even when it's really not ready for prime time.
The segment highlighted by Kate Willett left me giggling that this is being taken seriously. It sounds like a weed-addled sophomore in a dorm room. And that ANY of it is remotely possible by 2045 is the cherry on top.
And yet a fair fraction of the US population believes in an "invisible sky fairy" and that the "rapture" will precede its arrival incarnated as its offspring. Shouldn't we non-believers be giggling?
Sure, feel free to giggle at that, too, but that's a matter of faith. We're supposed to be talking about the rational people who believe in science here, that makes it that much worse.
I'll take the coming of Machine Shiva (thanks Ary!) seriously when they stop talking about what will/might happen and start talking about how. This may be a byproduct of theorizing about something they don't understand (to be fair, because it doesn't exist yet), but to me both the Doom and Paradise options reek heavily of the Power of Positive Thinking, willing a preferred reality into being like an Ayn Rand character.
“Accepting the reality of the coming machine god after it’s solved Navier-Stokes and dozens of other longstanding open math problems (while dramatically ramping up in capability every month), is sort of like accepting Jesus after he’s returned to earth on the gleaming cloud.” Wild statement and one that’s not aging that well giving some of the recent reporting. Oof.
Are you the same as the co-author of "End Times Fascism"?
Why is it always Machine Jesus? Why not Machine Shiva, Lord of the Dance of creative destruction? These machine god types need go expand their horizons.
You know the answer to that rhetorical question. Which deity/deities would they use in India? Would the ostensibly atheist communist China use a different term for the same idea?
Machine Mandate of Heaven? Machine Socialism with Chinese Characteristics?
This is a great post. I'm personally very skeptical about the imminence of space-faring machine gods, but I end up really frustrated by most skeptics because of how dismissive they are of true AI believers. (There are already a bunch of comments on this post along those lines!)
I don't think it makes sense to dismiss people who believe in imminent world-transforming superintelligence when a lot of them are very influential and have spent their whole lives being publicly very smart; any attempt to understand, catalogue and contextualize their worldview is valuable, in addition to providing alternatives. I think that generalizing and extrapolating, which are often the tools of smart people, have led a lot of AI people astray in understandable ways, and trying to understand how this has happened with someone like Scott Aaronson is very valuable and far more important than laughing at him.
Replace AI people with priests and the result is much the same. Wrong beliefs and actions bring "hellfire and eternal suffering". The right beliefs and actions will bring a "glorious future." Human nature was ever thus.
I do agree or at least strongly suspect it’s a case of a bunch of unfalsifiable beliefs filling a God-shaped hole in people’s brains (but that doesn't mean they can’t be taken seriously!)
As a retired proletariat I use my free chrome AI for mundane issues like health questions and American football. I also win every debate with the machine by use of pointed closed end prompts that forces the machine to rethink general answers to specific replies. Not anywhere in context of "Machine God" context but a time killer for me. Your piece is "Heavy" and interesting! Thanks!
A nice, long power outage would do these guys a world of good.
Where did that picture come from? Wow! (Also, great post, but I'm sure you don't need me to tell you.)
'All the worrying behaviour has happened in one context' seems overly optimistic given the labs have no obligation to disclose issues and more are still coming out. And the worrying behaviour is really worrying given how fast things are moving. I don't see how we have time to consider what you write about without slowing the labs down.
Isn't the rapid creation of a new "god" analogous to the rapid colonization of a land and the priests converting the natives to the conquerors' religion?
thank you! every essay you write here enlightens me more and more on this situation, and the present one is a big step for me.
Your reductionist "AI as cultural and social technology" is fine, but it runs up against millennia of human thought. Gods as superior beings have been believed by humans for millennia. Indeed, a "father-God" of Abrahamic religions is still believed by a large fraction of humanity, particularly in the USA. Neurobiologists and cognitive scientists might try to explain human thinking by lower-level processes, but we still regard exceptional people as special and shower them with accolades and status. While Newton may have well understood that "if I have seen further [than others], it is by standing on the shoulders of giants", we still regard Newton as an exceptional genius who made all his discoveries by himself. Our evolution as social apes, with a hierarchical structure, has resulted in human social systems that place a single individual at the top of the hierarchy, whether monarch, dictator, or president. (Our current president believes he is a "stable genius" who is never wrong. Why should we treat any remarkably [simulated] "intelligent" machine any differently than the way we treat humans at the top levels of a social hierarchy? We can only hope to shape their alignment with the needs of the population by various means of persuasion from close advisors to the ballot box, or in extremis, by defeat in battle.
TL;DR, we are disposed to anthropomorphize anything that looks like having agency, and elevate that agency, if powerful enough, as a god. This is a huge hurdle to overcome, and therefore will be the "attractor" around which "the AI alignment problem" will revolve. Good luck redirecting this.
I'm not worried about AI in and of itself per se. It becomes a problem when it's incorporated into the physical infrastructure, such as the power grid and water supply. We've already seen some of the vulnerabilities exposed with mere computer control - such as to hacking.
The big problem isn't AI becoming "god like" so much as it is the purveyors/profiteers shoving it down our throats a quickly as they can, so as to maximize profits - even when it's really not ready for prime time.
Worse, if it isn't profitable and brings down the global economy with its collapse.
The segment highlighted by Kate Willett left me giggling that this is being taken seriously. It sounds like a weed-addled sophomore in a dorm room. And that ANY of it is remotely possible by 2045 is the cherry on top.
And yet a fair fraction of the US population believes in an "invisible sky fairy" and that the "rapture" will precede its arrival incarnated as its offspring. Shouldn't we non-believers be giggling?
Sure, feel free to giggle at that, too, but that's a matter of faith. We're supposed to be talking about the rational people who believe in science here, that makes it that much worse.