The purest case of an intelligence explosion would be an Artificial Intelligence rewriting its own source code. The key idea is that if you can improve intelligence even a little, the process accelerates. It's a tipping point. Like trying to balance a pen on one end - as soon as it tilts even a little, it quickly falls the rest of the way.Collection: Intelligence
We tend to see individual differences instead of human universals. Thus, when someone says the word 'intelligence,' we think of Einstein instead of humans.Collection: Intelligence
Intelligence is the source of technology. If we can use technology to improve intelligence, that closes the loop and potentially creates a positive feedback cycle.Collection: Intelligence
I am a full-time Research Fellow at the Machine Intelligence Research Institute, a small 501(c)(3) public charity supported primarily by individual donations.Collection: Intelligence
Since the rise of Homo sapiens, human beings have been the smartest minds around. But very shortly - on a historical scale, that is - we can expect technology to break the upper bound on intelligence that has held for the last few tens of thousands of years.Collection: Intelligence
Anything that could give rise to smarter-than-human intelligence - in the form of Artificial Intelligence, brain-computer interfaces, or neuroscience-based human intelligence enhancement - wins hands down beyond contest as doing the most to change the world. Nothing else is even in the same league.Collection: Change
Nothing you'll read as breaking news will ever hold a candle to the sheer beauty of settled science. Textbook science has carefully phrased explanations for new students, math derived step by step, plenty of experiments as illustration, and test problems.Collection: Science
There's a popular concept of 'intelligence' as book smarts, like calculus or chess, as opposed to, say, social skills. So people say that 'it takes more than intelligence to succeed in human society.' But social skills reside in the brain, not the kidneys.Collection: Intelligence
When you think of intelligence, don't think of a college professor; think of human beings as opposed to chimpanzees. If you don't have human intelligence, you're not even in the game.Collection: Intelligence
I keep trying to explain to people that the archetype of intelligence is not Dustin Hoffman in 'The Rain Man;' it is a human being, period. It is squishy things that explode in a vacuum, leaving footprints on their moon.Collection: Intelligence
By far, the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.Collection: Intelligence
The obvious choice isn't always the best choice, but sometimes, by golly, it is. I don't stop looking as soon I find an obvious answer, but if I go on looking, and the obvious-seeming answer still seems obvious, I don't feel guilty about keeping it.
You cannot 'rationalize' what is not rational to begin with - as if lying were called 'truthization.' There is no way to obtain more truth for a proposition by bribery, flattery, or the most passionate argument - you can make more people believe the proposition, but you cannot make it more true.
The human species was not born into a market economy. Bees won't sell you honey if you offer them an electronic funds transfer. The human species imagined money into existence, and it exists - for us, not mice or wasps - because we go on believing in it.
Textbook science is beautiful! Textbook science is comprehensible, unlike mere fascinating words that can never be truly beautiful. Elementary science textbooks describe simple theories, and simplicity is the core of scientific beauty. Fascinating words have no power, nor yet any meaning, without the math.
My successes already accomplished have mostly been taking existing science and getting people to apply it in their everyday lives.
I wouldn't be surprised if tomorrow was the Final Dawn, the last sunrise before the Earth and Sun are reshaped into computing elements.
A burning itch to know is higher than a solemn vow to pursue truth. To feel the burning itch of curiosity requires both that you be ignorant, and that you desire to relinquish your ignorance.
Do not flinch from experiences that might destroy your beliefs. The thought you cannot think controls you more than thoughts you speak aloud. Submit yourself to ordeals and test yourself in fire. Relinquish the emotion which rests upon a mistaken belief, and seek to feel fully that emotion which fits the facts.
Let the winds of evidence blow you about as though you are a leaf, with no direction of your own. Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated. Surrender to the truth as quickly as you can.
If our extinction proceeds slowly enough to allow a moment of horrified realization, the doers of the deed will likely be quite taken aback on realizing that they have actually destroyed the world. Therefore I suggest that if the Earth is destroyed, it will probably be by mistake.
The systematic experimental study of reproducible errors of human reasoning, and what these errors reveal about underlying mental processes, is known as the heuristics and biases program in cognitive psychology. This program has made discoveries highly relevant to assessors of global catastrophic risks.
A scientist worthy of a lab coat should be able to make original discoveries while wearing a clown suit, or give a lecture in a high squeaky voice from inhaling helium. It is written nowhere in the math of probability theory that one may have no fun.
The media thinks that only the cutting edge of science, the very latest controversies, are worth reporting on. How often do you see headlines like 'General Relativity still governing planetary orbits' or 'Phlogiston theory remains false'? By the time anything is solid science, it is no longer a breaking headline.
When something is universal enough in our everyday lives, we take it for granted to the point of forgetting it exists.
An anthropologist will not excitedly report of a newly discovered tribe: 'They eat food! They breathe air! They use tools! They tell each other stories!' We humans forget how alike we are, living in a world that only reminds us of our differences.
My parents were early adopters, and I've been online since a rather young age. You should regard anything from 2001 or earlier as having been written by a different person who also happens to be named 'Eliezer Yudkowsky.' I do not share his opinions.
In our skulls, we carry around 3 pounds of slimy, wet, greyish tissue, corrugated like crumpled toilet paper. You wouldn't think, to look at the unappetizing lump, that it was some of the most powerful stuff in the known universe.
The purpose of a moral philosophy is not to look delightfully strange and counterintuitive or to provide employment to bioethicists. The purpose is to guide our choices toward life, health, beauty, happiness, fun, laughter, challenge, and learning.
Transhumanists are not fond of death. We would stop it if we could. To this end, we support research that holds out hope of a future in which humanity has defeated death.
If I could create a world where people lived forever, or at the very least a few billion years, I would do so. I don't think humanity will always be stuck in the awkward stage we now occupy, when we are smart enough to create enormous problems for ourselves, but not quite smart enough to solve them.
I don't care where I live, so long as there's a roof to keep the rain off my books, and high-speed Internet access.
If you want to maximize your expected utility, you try to save the world and the future of intergalactic civilization instead of donating your money to the society for curing rare diseases and cute puppies.
Moore's Law of Mad Science: Every eighteen months, the minimum IQ necessary to destroy the world drops by one point.Collection: Science
You are personally responsible for becoming more ethical than the society you grew up in.Collection: Becoming
Many have stood their ground and faced the darkness when it comes for them. Fewer come for the darkness and force it to face them.Collection: Bravery
Litmus test: If you can't describe Ricardo's Law of Comparative Advantage and explain why people find it counterintuitive, you don't know enough about economics to direct any criticism or praise at "capitalism" because you don't know what other people are referring to when they use that word.Collection: Law
Remember, if you succeed in everything you try in life, you're living below your full potential and you should take up more difficult or daring things.Collection: Trying
By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.Collection: People
To confess your fallibility and then do nothing about it is not humble; it is boasting of your modesty.Collection: Humble
There is no justice in the laws of nature, no term for fairness in the equations of motion. The Universe is neither evil, nor good, it simply does not care. The stars don't care, or the Sun, or the sky. But they don't have to! WE care! There IS light in the world, and it is US!Collection: Stars
If you handed [character] a glass that was 90% full, he'd tell you that the 10% empty part proved that no one really cared about water.Collection: Character
If people got hit on the head by a baseball bat every week, pretty soon they would invent reasons why getting hit on the head with a baseball bat was a good thing.Collection: Baseball
Between hindsight bias, fake causality, positive bias, anchoring/priming, et cetera et cetera, and above all the dreaded confirmation bias, once an idea gets into your head, it's probably going to stay there.Collection: Ideas
I don't want to rule the universe. I just think it could be more sensibly organised.Collection: Thinking
Through rationality we shall become awesome, and invent and test systematic methods for making people awesome, and plot to optimize everything in sight, and the more fun we have the more people will want to join us.Collection: Fun
There were mysterious questions, but a mysterious answer was a contradiction in terms.Collection: Answers