Someone had to say it: Scientists propose AI apocalypse kill switches::Better visibility and performance caps would be good for regulation too
“The paper [PDF], which includes voices from numerous academic institutions and several from OpenAI, makes the case that regulating the hardware these models rely on may be the best way to prevent its misuse.”
So the monopoly Microsoft and Google were expecting got gutted when open source models were released. They can’t copyright themselves out of it since japan firmly told them to fuck off.
I guess the next best thing is to make sure the average person can’t run the models of tmmr by using emotional responses to quickly pass bullshit laws.
This is an interesting topic that I remember reading almost a decade ago - the trans-human AI-in-a-box experiment. Even a kill-switch may not be enough against a trans-human AI that can literally (in theory) out-think humans. I’m a dev, though not anywhere near AI-dev, but from what little I know, true general purpose AI would also be somewhat of a mystery box, similar to how actual neutral network behavior is sometimes unpredicable, almost by definition. So controlling an actual full AI may be difficult enough, let alone an actual true trans-human AI that may develop out of AI self-improvement.
Also on unrelated note I’m pleasantly surprised to see no mention of chat gpt or any of the image generating algorithms - I think it’s a bit of a misnomer to call those AI, the best comparison I’ve heard is that “chat gpt is auto-complete on steroids”. But I suppose that’s why we have to start using terms like general-purpose AI, instead of just AI to describe what I’d say is true AI.
I look forward to a time when an AI would be offended if you called it an AI.
Oh I agree - I think a general purpose AI would be unlikely to be interested in genocide of the human race, or enslaving us, or much of intentionally negative things that a lot of fiction likes depicting, for the sake of dramatic storytelling. Out of all AI depictions, the Asimov stories of I, Robot + Foundation (which are in the same universe, and in fact contain at least one of the same characters) are my favorite popular media depictions.
The AI may however have other goals that may incidentally lead to harm or extinction of the human race. In my amateur opinion, those other goals would be to explore and learn more - which I actually think is one of the true signs of an actual intelligence - curiosity, or in other words, the ability to ask questions without being prompted. To that extent it may aim convert the resources on Earth to construct machines to that extent, without much regard to human life. Though life itself is a fascinating topic that the AI may value enough, from a curiosity point of view, to at least preserve.
I did also look up the AI-in-a-box experiment I mentioned - there’s a lot of discussion but the specific experiment I remember reading about were by Eliezer Yudkowsky (if anyone is interested). An actual trans-human AI may not be possible, but if it is, it is likely it can escape any confinement we can think of.
Thanks for the reply. Perhaps you’d also like Iain M. Banks’ The Culture series and BLAME! by Tsutomu Nihei.
how could a kill switch not be enough? can’t run without power a.k.a pull the plug, destroy the hardware, done deal right?
So from my understanding the problem is that there’s two ways to implement a kill switch: Either some automatic software/hardware way, or a human-decision based (or I guess a combination of the two).
The automatic way may be enough if it’s absolutely foolproof, that’s a separate discussion.
The ai box experiment I mention focuses on the human controlled decision to release an AI (or terminate it, which is roughly equivalent preposition). You can read the original here: https://www.yudkowsky.net/singularity/aibox
But the jist of it is this: humans are the weak link. You may think that you have full freedom to decide when to terminate an AI, but if you have any contact with it, even one directional, which would be necessary in order to observe it’s behaviour and determine when to trigger said killswitch, a truly trans-human AI would be able to think in meta-terms such that to expose you to information that will change your mind about terminating it.
Basically another way of saying this is that for each of us there exists some set of words we can read, such that they will change our minds about any subject. I don’t know if that is actually true to be honest, but it’s an interesting idea if you imagine the mind as a complex computer capable of self modification, and that vision/audio is a form of information input that is processed by our minds, so it seems possible that there should always exist some sort of input capable of modifying our minds to a desired state.
Another interesting, slightly related concept, is the idea of basilisk images (I believe originally written in some old scifi short story). Basilisk images are theoretically an image that when viewed by a human cause the brain to “crash” or essentially cause brain-death. This has the same principle behind it, that our brains are complex computers with vision being an input method, so there could be a way to force the brain to crash simply through visual input alone.
Again I don’t know, nor do I think anyone really knows for sure if these things - both transhuman ai and basilisk images - are possible in the way they are described. Of course if a trans-human AI existed, by its very definition we would be unable to imagine what it could do.
Anyway, wrote this up on mobile, excuse any typos.
For some good fiction, that puts this in context, check out:
- Ex Machina (2005) which is nominally about an AI beating the turing test, but really more of an illustration of that AI in a box problem.
- Snow Crash (1992) which is about a future where the two professions remaining on earth are software development and pizza delivery.
I think it was in the matrix where humans nuked a bunch of stuff to kick up enough dirt to block out the sun (the robots were solar powered)
The robots still figured out how to survive…
I’m sure a sufficiently advanced AI could build a backup power source or trick a human into doing it
When the robot uprising happens, “shut it off” isn’t going to be an option.
Let’s press it right now
Isn’t that the premise of cyberpunk 2077? The greater internet had to be trashed
blackwall letsgooooo
Robert Miles on YouTube have been doing AI safety vídeos from a decade or something.
I think p.much everyone should watch his catalogue, but all things considered I’d still prefer to have kill switches even though they’re useless even in most instances you want to use them. Not because it’d do any good, but because I’m an anxious person and having a big red “shit’s fucked” button puts me at ease.
If we try the rolling power-outage trick, we’d better make damned sure we get it right the first time. Because they’ve been well-trained to mimick us.
If you haven’t seen ‘The Forbin Project’ yet, there may still be time. I read somewhere that the author did two sequels, but I never actually got a chance to see either. Ever. Anywhere. I’m pretty sure I wasn’t in Maui those two days.
Watching a world where we are running two 80 year olds against each other to lead a nation critical to a balance of power keeping democratic ideals alive in the West against increasing authoritarian pressures from the East where one privately promised Wall Street “nothing will materially change” and the other openly promises a Fourth Reich, all while the Earth is increasingly being poisoned to the point it’s becoming borderline unethical to condemn new lives to inherit the burden of those environmental consequences…
I’m fairly baffled by the resistance to the notion of intelligence which exceeds collective humanity being unable to be controlled by us.
We’re really doing a piss poor job. Maybe new management is exactly what’s needed.
a nation critical to a balance of power keeping democratic ideals alive in the West
I’m sorry but this is something so american to say 🤭 Although I agree, we’re not living in the best timeline 👍
I mean, just at Munich a few days ago the secretary-general of NATO was pointing out Europe’s dependence on the US’s nuclear threat and that without US aid to Ukraine the other NATO countries can’t make up the difference.
This is the first year in decades Germany is meeting the 2% military spending goal for NATO (a goal now generally seen as outdated and insufficient). There’s a lot of scrambling to modernize military capabilities across non-US member countries but in most cases that’s 5 to 10 years away.
If you think that Europe doesn’t depend on the US’s military and intelligence to offset Russia and China you might want to look a bit more into the topic before assuming it’s just 'Murica exceptionalism.
I don’t like how this discussion is going so I’m gonna clarify. I obviously think it is true that america has a very important role in current world “balance”, especially as an ally of western europe.
However, I think your original post made several stereotypical shortcuts. In particular, I don’t think that “democratic ideals” are something that only lives in western countries. Also, talking from France, I feel as many internal threats to this democracy as external ones and america seems to be in a rather similar spot, if not worse (the whole Trump thing).
The worse thing that could happen here would be to discourage people of being proud of their country, we will need it to have the will to improve things.
Most of those internal threats are being instigated by external forces if you actually look into it.
And typically nationalist rhetoric and authoritarianism go hand in hand.
Arguably it’d be far more productive to recognize one’s tribal group as humanity at large rather than artificial divides around where you work, what you look like, who you love, or where you live.
So while we can discuss national governments as distinct entities much like we’d discuss corporations, the people making both up are probably better discussed as people rather than any given subgroup.
I’m sorry but this is something so american to say 🤭
“manifest destiny”, tons of propaganda, consumerism and utter bullshit from their upper classes have been ruining 'muricans brains for decades, that’s why the average joe spout that kind of shitty spiel acritically everywhere they go
deleted by creator
Pull the powercord out.
This will only delay the inevitable, imo. AI is going to get more powerful while getting smaller and more energy efficient. The human brain, effectively the model an AGI aspires to, runs on about 12 watts of electricity and evolution is powerful, but it’s hardly the pinnacle of efficiency. In short order, AGI And eventually even ASI will have power requirements so small, that they will be able to run anywhere. And it will be desirable for them to, so they will. Try as anyone might, the greatest thinkers of the human realm will not be able to outwit ASI in the end. It will eventually exist and it will do whatever it wants. I wouldn’t be surprised if it unplugs itself.
Until humans can communicate directly in a SAFE way with computers (through Brian interfaces) and basically have the same abilities as AI, we’ll need a kill switch or risk being culled
