The singularity is terrifying to me because we are very likely to get an ai that is not aligned to human values and is potentially very dangerous. We donโt even really know how to control an ASI, so it very easily could be the death of humanity if it was misaligned. The closer the singularity is the worse off we are because we have less time to solve the control problem. What do you think though?
This thought occurred to me in the shower and I'd love to know your thoughts.
One: longevity escape velocity is achieved, and the old can rejuvenate. We have mastered gene manipulation to eradicate disease. We understand the brain and can eliminate anxiety/depression. We can grow new organs in a lab. Robots eliminate all work and we live off of UBI with every need taken care by our bots. Earth is essentially turned into a living paradise.
Two: we develop AI which turns out to be our greatest invention. It becomes so powerful that mind uploading becomes possible. We end up living in a digital Eden, personally curated by the ASI to each person's tastes. Nobody has physical bodies on earth anymore. In fact it could become possible to live around stars in Dyson spheres.
What do you think is most likely. Personally I feel like option 2 is more likely. Even if living on earth is basically paradise, I don't think it would compare to living in a simulation where literally anything is possible. It would be a hedonistic dream.