ARTIFICIAL GENERAL INTELLIGENCE IS HERE. TOTALLY.
A reply to a friend
From the essay
A young friend studying AI asked me whether AGI is already here, and whether building something smarter than ourselves means building our own prison. My sense is that AGI has been here for a while, not as one gigantic brain announcing its arrival, but as a growing federation of specialized intelligences learning to work together. From there, we wander into robots moving with human grace, the possibilities of combining analog and digital computing, and AI’s inconvenient habit of bringing a power station to lunch. But what interests me most is not simply how smart a machine can become, it is whether anything could genuinely matter to it. That brings me to pain as a motivator: not a warning sensor or a numerical penalty, but an experience that could give an artificial entity something of its own to protect. Of course, an unfeeling machine can still be dangerous, and people hardly need help inventing bad uses for powerful tools. Still, I’m more excited than frightened by greater intelligence. What puzzles me is why we would deliberately add suffering to the package. My preference? Make it enormously intelligent. Just don’t make it hurt.