AI Doomsday Recipe
(1) Leave your artificial intelligence (AI) as vague as possible so that nobody can outline flaws in the scenario that you want to depict.
(2) Claim that almost any AI is going to be dangerous because all AI's want to take over the world. For example, if you ask the AI "Hey AI, calculate 1+1", the AI goes FOOOOM and the end of the world follow seconds later.
(2.1) If someone has doubts just use buzzwords such as 'anthropomorphic bias' to ridicule them.
(3) Forego the difficulty of outlining why anyone would want to build the kind of AI you have in mind. We're not concerned with how practical AI is developed after all.
(4) Make your AI as powerful as you can imagine. Since you are ignoring practical AI development and don't bother about details this should be no problem.
(4.1) If someone questions the power of your AI just outline how humans can intelligently design stuff that monkeys don't understand. Therefore humans can design stuff that humans don't understand which will then itself start to design even more incomprehensible stuff.
(5) Outline how as soon as you plug a superhuman machine into the Internet it will be everywhere moments later deleting all your porn videos. Don't worry if you have no idea how that's supposed to work in practice because your AI is conjectured to be much smarter than you are so you are allowed to depict scenarios that you don't understand at all.
(5.1) If someone asks how much smarter the AI you expect to be just make up something like "1000 times smarter". Don't worry about what that means because you never defined what intelligence is supposed to be in the first place.
(5.2) If someone calls bullshit on your doomsday scenario just conjecture nanotechnology to make your AI even more powerful, because everyone knows from science fiction how nanotech can pretty much fuck up everything.
(6) If nothing else works frame your concerns as a prediction of a worst case scenario that needs to be taken seriously, even given a low probability of its occurrence, due to the scale of negative consequences associated with it. Portray yourself as a concerned albeit calm researcher who questions the mainstream opinion due to his strong commitment to our collective future. To dramatize the situation even further you can depict the long term consequences and conjecture the possibility of an intergalactic civilization that depends on us.
(1) Leave your artificial intelligence (AI) as vague as possible so that nobody can outline flaws in the scenario that you want to depict.
(2) Claim that almost any AI is going to be dangerous because all AI's want to take over the world. For example, if you ask the AI "Hey AI, calculate 1+1", the AI goes FOOOOM and the end of the world follow seconds later.
(2.1) If someone has doubts just use buzzwords such as 'anthropomorphic bias' to ridicule them.
(3) Forego the difficulty of outlining why anyone would want to build the kind of AI you have in mind. We're not concerned with how practical AI is developed after all.
(4) Make your AI as powerful as you can imagine. Since you are ignoring practical AI development and don't bother about details this should be no problem.
(4.1) If someone questions the power of your AI just outline how humans can intelligently design stuff that monkeys don't understand. Therefore humans can design stuff that humans don't understand which will then itself start to design even more incomprehensible stuff.
(5) Outline how as soon as you plug a superhuman machine into the Internet it will be everywhere moments later deleting all your porn videos. Don't worry if you have no idea how that's supposed to work in practice because your AI is conjectured to be much smarter than you are so you are allowed to depict scenarios that you don't understand at all.
(5.1) If someone asks how much smarter the AI you expect to be just make up something like "1000 times smarter". Don't worry about what that means because you never defined what intelligence is supposed to be in the first place.
(5.2) If someone calls bullshit on your doomsday scenario just conjecture nanotechnology to make your AI even more powerful, because everyone knows from science fiction how nanotech can pretty much fuck up everything.
(6) If nothing else works frame your concerns as a prediction of a worst case scenario that needs to be taken seriously, even given a low probability of its occurrence, due to the scale of negative consequences associated with it. Portray yourself as a concerned albeit calm researcher who questions the mainstream opinion due to his strong commitment to our collective future. To dramatize the situation even further you can depict the long term consequences and conjecture the possibility of an intergalactic civilization that depends on us.
One thing I've noticed is that you hardly bother to argue that any of the ideas you criticize is wrong.
You just go on the meta-level, and compare the shape of the ideas with something else, or discuss the people that are supporting these ideas, or discuss their trustworthiness, or the decision algorithms that lead to people supporting them (or the lack of decision algorithms that lead to people supporting).
Can you pretend for a sec that you live in a work where the the people you so love to hate never existed at all? How would you discuss and what probability would you assign to the statement "A recursively improving artificial intelligence is a danger to humanity, when provided a goal which isn't taking into account the complexity of human values" BY ITSELF -- without mocking any person that believes it a danger, because you momentarily must imagine a world where nobody believes it a danger?Jun 1, 2013
I primarily think that some people are vastly overconfident and with their heads in the clouds. I do not think that their arguments (http://intelligence.org/2013/05/05/five-theses-two-lemmas-and-a-couple-of-strategic-implications/) are unsound but rather vastly less convincing than they seem to believe.
I think it is sensible to have a study group which thinks about risks associated with artificial intelligence (AI) research. Such a group could also think about the possibility of an AI improving itself and (1) define what exactly that means (2) how likely it is and when (3) how quickly such improvements could take place.
Once they outlined some of the basics they should seek peer review. Ask e.g. AI researchers, economists, complexity theorists etc. From there on, if those basic models have been judged sound, probable and temporally close enough to be action relevant, it should be decided if practical research into AI is necessary to fortify the underlying ideas or if it would make sense to think about some concrete scenarios of how such an AI could pose a risk. Which could involve, among other problems, (1) the advantage of intelligence with respect to endangering the global infrastructure etc. (2) possible warning signs.
And then long after that has all been fleshed out and if it still makes sense one could start thinking about more complex scenarios such as emergent goals of certain AI designs which would interfere with human matters in possibly catastrophic ways. Concrete scenarios of how such goals could have catastrophic side effects need to be outlined and many variables such as the fragility of such an AI to counterstrikes have to be analyzed in great detail before any action relevant conclusions can be drawn.
What I do not believe to be sensible is going straight into doomsdays mode, asking everyone for money to save the world, based on some sort of black box full of technological magic exhibiting complex behavior.
What I believe to be completely over the top and deserving ridicule is if such a black box is subsequently used as a basis to draw further unwarranted assumptions and take the implications of those assumptions seriously enough to censor certain ideas out of the fear of causing such a black box to behave badly somewhere in the multiverse. THAT IS BATSHIT INSANE and the burden of proof is not on me!Jun 1, 2013- "are unsound but rather vastly less convincing than they seem to believe."
Do you mean the word 'convincing' in the sense of 'likelihood to convince people'? If so that of course always depends partly on the argument and partly on the people hearing it. e.g. I think that MIRI realizes that some people will never be convinced of any existential threats, ever, regardless of argument.
Your paragraphs between "I think it is sensible to have a study group" and "before any action relevant conclusions can be drawn" seem reasonably decent. Do you know or plan to create, of any such study group which I could perhaps donate money to, assuming I find them superior to my current donation targets?
"What I do not believe to be sensible is going straight into doomsdays mode, asking everyone for money to save the world"
If someone honestly believes we're in "doomsday mode", I'd rather he tell me we are in "doomday mode", so that we all know where he stands.
If there are two study groups, one of which believes we're in doomsday mode, and one of which doesn't, show me the arguments of each, and I may donate to the one that makes the best arguments. Or split the donations if their quality differs in various aspects.
So far I don't think you're comparing MIRI to some better real alternative that actually exists, you are comparing them to some imaginary (and better according to you) version that doesn't actually exist.
"seriously enough to censor certain ideas out of the fear of causing such a black box to behave badly somewhere in the multiverse. THAT IS BATSHIT INSANE and the burden of proof is not on me!"
Yay, the basilisk again. What's the difference between "BATSHIT INSANE" and e.g. "OVERLY EARNEST"? Merely a different probabilistic estimation on what kinds of artificial intelligence are likely to arise?
There are forums where an "ableist" phrase like 'batshit insane' would itself be censored as potentially emotionally damaging to people with mental ailments.
I don't agree with such censorship policies, I don't agree with LessWrong's censorship policy either. But in the context of beliefs earnestly held about what may constitute harm, the people implementing said policies against phrases like "batshit insane" believe they are doing good.
And even in my disagreement, I find that attitude more respectable than I find the mockery of said policies to be.Jun 1, 2013 - You said that I like to compare the shape of the ideas with something else. Yes, I think that it makes sense to highlight what I e.g. mean by "convincing" and that the confidence that AI research might pose and existential risk largely relies on ignorance.
If the arguments in favor of foresight regarding molecular nanotechnology would be similar to those of artificial general intelligence (AGI) then nothing would be known about the eventually attainable range of chemical reaction cycles, error rates, speed of operation, and thermodynamic efficiencies. Such details would actually be ignored and instead action relevant conclusions would be drawn from the assumption that all those details will resolve in such a way that molecular nanotechnology poses an existential near-term risk. Which would in my opinion not be sensible. And the same is also true for AGI.
Arguing that there are viruses designed by evolution and that therefore nanobots which are intelligently designed will be X times better, for some undefined definition of "better", is not very convincing. Yet that is basically how AI risks are justified.
You wrote: 'If someone honestly believes we're in "doomsday mode", I'd rather he tell me we are in "doomday mode", so that we all know where he stands.'
Yes, I agree. But then if that person claims that any other beliefs are unjustified or irrational because nobody can disprove his claims, then that will ultimately cause some amount of ridicule. Which shouldn't worry that person if they really believe that their arguments are convincing because the person who does ridicule them will only end up discrediting themselves.
One of the major problems I see is that the arguments of the critics of AI risks seem to be judged by how much they succeed in disproving AI risks. Which is not sensible without discounting for the vagueness and unfalsifiable nature of AI risks.
You might respond that AI risks are falsifiable by creating a seed AGI, without paying attention to friendliness, who will undergo explosive recursive self-improvement without catastrophic side effects.
I do not call that 'unfalsifiable' for any useful definition of falsifiability. Not only is the prediction sufficiently vague to claim that any AI in question did not meet your prediction and that the next one will do it, but I believe that it is sensible to demand that AI risk advocates make their beliefs pay rent in anticipated experiences other than doomsday. There needs to be a way to update on evidence and refine a prediction other than waiting for that prediction to happen as predicted.
You wrote: 'Yay, the basilisk again. What's the difference between "BATSHIT INSANE" and e.g. "OVERLY EARNEST"? Merely a different probabilistic estimation on what kinds of artificial intelligence are likely to arise?'
Pretty much the only reason I started this was the basilisk. Check my comments from before Roko's post. No criticism.
So what's the difference between insanity and honesty? I guess someone who avoids uttering certain beliefs due to fear of looking like a crank retained enough sanity to judge which of their beliefs is controversial.Jun 1, 2013 - "Pretty much the only reason I started this was the basilisk. Check my comments from before Roko's post. No criticism."
Well, yeah, that's frankly one of my problems with you, that you moved from one extreme (no criticism ever about anything, just complete monotonous support) to the other (constant criticism about everything, complete monotonous dismissal) after a single event that left you displeased. (Have you seen "The Incredibles"? The character of Syndrome is what that reminds me.)
I don't want to amateur-psychoanalyze, so my apologies in advance for this, but that doesn't look like a rational response to me, it looks like an emotional backlash or something.
"So what's the difference between insanity and honesty?"
I meant "earnest" in the sense of "dedicated and having intense convictions", not "honest". If someone's convinced that the basilisk is really harmful, then they might react as Eliezer did. People like you who don't consider the basilisk harmful, may not be in the best position to call a behavior "insane" when it arises from such different beliefs.
And even if it were completely insane, even Newton wrote about frigging biblical prophecies -- it should not get people to dismiss his work on calculus or the theory of gravity. A single point of absurdity should in no way get you to so radically change your opinion about everything.Jun 2, 2013 - Some relevant posts I wrote after this discussion which outline some of the problems mentioned in the above post in a more respectful tone:
(1) AI vs. humanity and the lack of concrete scenarios
Link: http://kruel.co/2013/06/01/ai-vs-humanity-and-the-lack-of-concrete-scenarios/
(2) AI drives vs. practical research and the lack of specific decision procedures
Link: http://kruel.co/2013/06/01/ai-drives-vs-practical-research-and-the-lack-of-specific-decision-procedures/
(3) Questions regarding the nanotechnology-AI-risk conjunction
Link: http://kruel.co/2013/06/02/questions-regarding-the-nanotechnology-ai-risk-conjunction/
(4) AI risk scenario: Deceptive long-term replacement of the human workforce
Link: http://kruel.co/2013/06/03/ai-risk-scenario-deceptive-long-term-replacement-of-the-human-workforce/
(5) AI risk scenario: Mitchell Porter’s Elite Cabal
Link: http://kruel.co/2013/06/03/ai-risk-scenario-mitchell-porters-elite-cabal/
Other posts relevant to this discussion:
How it might be a bad idea to contribute money to MIRI even if they are the only group concerned with AI risks:
http://kruel.co/2013/01/04/should-you-trust-the-singularity-institute/
http://kruel.co/2013/01/07/why-you-should-be-wary-of-the-singularity-institute/
How Roko's basilisk highlights some important problems:
http://kruel.co/2013/01/12/rokos-basilisk-everything-you-need-to-know/
A lot more here:
http://kruel.co/2012/07/17/miri-lesswrong-critiques-index/Jun 4, 2013
