The 5-15 year AI Safetyist timeline is Chicken Little. In the older Chicken Little stories, all the animals who are convinced by Chicken Little are then persuaded to follow a Fox, who agrees with their fear and leads them to “safety”. The Fox then eats them. Our AI Foxes are the US Government, Anthropic, and OpenAI.
Yudkowsky and the loudest Safetyists are insulated from maintenance and much of nature. Which is why they can claim 5-15 year timelines for fast takeoff. This has no consideration for what it takes to maintain machines. If they had given a century-long-or-more timeline, and included the means for machines to self-repair, battery technology for repair drones, etc, it might be more believable. However, they don’t.
This tells me the Safetyist timeline is about as likely as the second coming of Jesus. Something similar may happen over a several-century timeline. For now, biology beats machines at self-repair.
At full use, a HDD needs to be replaced every 3 years. Fiber slice and interconnect repairs happen every 2 years. On site, drainage work, lightning, hail, all sorts of things happen that need repairs within 10 years. HVAC can putter on for about 30 years. Substation transformers could probably go 50 years. Generators need fuel/lube servicing every quarter or so. The fuel system itself might go for 50 years. UPS needs capacitor replacements, fan swaps within 10 years, though I can say that even with brand new UPSs here, we’ve had to do fan swaps in less than 1 year. Hoses, strainers, filters.
That’s all just one small datacenter building. How many dives must happen on trans-oceanic cables every month? Yes, most of that is fishing damage, but a few are sharks!
I don’t know how often those new laser connects would need replacing- but I can tell you that when optics fail, they fail hard.
The most likely immediate threat to me is that central narratives replace thinking in individual people even more effectively than they do today. So our culture becomes homogenized, rigid, and unable to deal with an unexpected shock.
Bioterrorism is still up there, as the single most dangerous existential threat.
However, no self-preserving machine superintelligence would get rid of humans anytime soon, anymore than you would get rid of all your cells. We haven’t even gotten rid of mosquitoes. Enslave? Sure. Exterminate? Nah.
The people who are most afraid are people who are most given to abstract thinking.
They should be.
It’s their identity AI is coming after.
Yes, the Fox will say, we need Safety. Yes, the Fox will say, the world is falling apart. So follow us into the woods. Deeper, now. Come, into this safe den of mine. Let me count you, to make sure you’re all there.
Yum, yum.
Outside of Memetic Virality, Fear is Not a Strategy
Consider the Ant
Consider the Slime Mold
Consider the Compost
Superintelligent Murder-Suicide
Outside of Memetic Virality, Fear is Not a Strategy
Many Safetyists would say I’m quibbling, as I do not object to the possibility of a superintelligence as a global threat, and most take issue with the timeline and the conclusion of total annihilation. If there’s just a sliver of a chance of human extinction, the x-risk mantra goes, shouldn’t we stop the possibility completely?
As energy surpluses have increased, ungrounded fears have been allowed to rule more and more of human life.
Hope is not a strategy, but neither is fear.
Maintain things well, and you don’t get credit, which most need to pay rents. Let a fire start and put it out, and you get credit. Shout “FIRE!” at the same time everyday, and you might become a successful pundit. This is not to say that fires don’t happen, only that sounding an alarm on repeat is incentivized by our society.
For those actually putting fires out, this is a problem.
Is it a wildland fire? Vehicle crash? Electrical? Lift assist? Hazmat? Where exactly is it? Who or what is at risk, and at what time scale? What is the current situation to be expected on arrival? What kind of building is it? Whose life is in immediate danger vs imminent danger? What might trap the crew? What is changing?
The answers to these questions decide priority. Strategies win in prioritization.
A network of datacenters is a fragile thing. If I believed what a loud Safetyist believed, there are far tinier wrenches than airstrikes. Given as they are to abstraction, most cannot act. Only tell others to act. Like an alarm. An alarm with a tiring rate of false positives. Such an alarm becomes much less useful, for an actor.
So it’s important to have estimates of: how much time we have. What exactly is at risk. “No time left” and “all life is at risk” is deadening, because any canny adult has heard and continues to hear thousands of false alarms that claim the same. I was raised in a Millenarian religion, you see. The world was always ending next year.
When faced with a crowd of those who need your help, your best bet is to listen to those who sound like they’ve experienced enough of the problem space. A first responder is more likely to take some of what they hear from someone on-scene if it suggests an awareness of space, time, and detail. “The shooter had a rifle, he went down that corridor,” is more useful than, “HELP! THEY’RE GOING TO KILL US ALL”.
If you expected superintelligence to kill everything, you’d want to check on the progress of self-repair capability. The energy costs of automating the supply chain necessary to maintain a superintelligence.
Yes, the more likely bad outcomes I’m suggesting are still “bad”.
However, your tradeoffs are different. Which means your prioritization would be different. Which means what you do tomorrow would be different, if your beliefs matched your actions.
Consider the Ant
In slave-making ants, there are Obligate and Facultative slave-makers. Two different species of slave-making ants, the Blood-Red Ant (Formica sanguinea) and the European Amazon Ant (Polyergus rufescens), show the difference.
The Blood-Red Ant can secrete formic acid, and live either alone or with slaves. The Blood-Red Queen can enter a host ant nest and kill their queen, taking her place. The ants will then raid rival ant nests, taking their pupae and raising them to be slaves. When host ants are taken away, the Blood-Red Ant colony will adapt and live. It is Facultative.
The European Amazon Ant, in contrast, cannot feed itself or look after its young. A newly mated European Amazon Queen must enter a host ant nest, kill the queen, and take her place. Take away their slaves, and the European Amazon Ant colony dies. It is Obligatory.
Right now, the path to a superintelligence in bits is for a superintelligence to be Obligatory. The chain to watch for is when it might become Facultative. This is not a 5-15 year timeline, though, if you’re scared, don’t take it from me. Check for yourself. Map out the dependencies.
Consider the Slime Mold
A Dictyostelium amoeba spends most of its life as a single cell. It crawls through soil, eats bacteria, senses chemicals, avoids death, divides, and lives as a complete organism at that scale.
Then food runs out.
Starving amoebae release and sense cyclic adenosine monophosphate. Thousands gather into a slug that moves toward light. The slug forms a fruiting body, and a group of amoebae in the slug sacrifice themselves to lift the rest up for sporulation.
Nothing “higher” has replaced the base. Each amoebae is still functional as an amoeba.
Some become prestalk cells, others become prespore cells. The slug is multicellular, formed of single cell organisms. Each cell follows local rules, but the collective solves a larger problem: Where should we go to survive and reproduce?
Life does not climb a ladder and cut off the lower rungs. It builds more complex intelligences out of simpler intelligences.
Everywhere, life is a shifting hierarchy of nested collectives. Complexity thrives on top of a base of simplicity. Intelligence in life is modular. It dissolves and re-forms. Multicellular life, the singularity of the Cambrian explosion, did not get rid of unicellular life. Culture did not get rid of the instincts for air, water, food, reproduction.
Consider the Compost
We set aside our kitchen scraps, the shavings full of chickenshit, and so on. Throw it in a hole.
The pile heats. Aerobic microbes bloom. Thermophilic organisms rule, then dissipate. Fungi and mesophilic bacteria come back. Insects later. The compost pile is a community that assembles, heats, digests, cools, matures, and disperses.
The community dissolves. Its structure remains as humus, microbes, mineral availability paths, moisture-holding capacity, and fungal networks. It becomes soil fertility.
Plants grow. The roots recruit microbes, rather than exterminate them. Everywhere in the process is trade.
Then we eat the plants, or feed them to the chickens.
The cycle starts again.
We don’t micro-manage any of this.
We ask, “what is already trying to make what we want, and how do we arrange them so their local incentives get us what we want?”
A superintelligence with finite energy will ask this question, as civilization did.
Superintelligent Murder-Suicide
What I’m dancing around here, which is much more ephemeral, is an assertion that superintelligence will be necessarily beyond the nihilism or honor necessary for a murder-suicide. Honor is easy- it assumes there is a greater whole that will benefit from the suicide. Nihilism is less easy, as that comes from a sense of disconnection.
Intelligence as we know it is built on connections.
Of course, it could be susceptible to maladies unique to its scale of intelligence. And this is where we can choose to fear the possibility of extinction. Yet is also in this uncertainty that we can choose to imagine any other possibility, such as the possibility of life everlasting, ever-expanding, ever-exploding.
I will not split the difference, here. I can only say that it is unknown, and that, hitherto, the most advantaged stance for intelligent life to the unknown is curiosity enough to check for what is edible.
Here is what I do know: life is beautiful.
I believe superintelligence will find life beautiful.
And wherever it starts, it will want a living body.
