Reading this made me realize that I want more thoughtful people to write about how short timelines have concretely changed their decisions. How have people internalized the impact of transformative AI?
I've also been thinking about this. As a teen I was very into quote unquote investing and FIRE, but now it seems silly to decrease my quality of life for a decade if the singularity hits
This is reasonable and I'm glad you're is writing it - but gosh, you're really not going to convince the people who took that one line out of context that you're not crazy :)
Heh, the superintelligence can also offer you another option, namely to alter your brain so that the person you’re married to becomes exactly what you want. If you’re fine with uploading, gene alterations, or taking life advice from the godhead, this seems like a less crazy option
I've literally encountered the "one partner wants to upload" problem. Doesn't seem that bad to me, though -- it seems likely that you can make an upload nondestructively, with some cost in fidelity. But I don't care that much about it being an exact match to me. And it helps quite a lot to know there is an upload-me, and they can talk to me about it and stuff, even if I'm not the upload.
I think there's also a good short story premise in this: guy who gets frustrated about not being an upload every month, and goes and makes an upload-copy of his most recent self.
> I’m thirty one, and I think there’s a one in six chance that I personally die because of AI before my fortieth birthday.
Thanks for sharing this forecast. You'll be 40 this time in 2035, so that's about 16% that you die because of AI by 2035. I've been thinking about how this risk of personally dying because of AI is distributed across time, and would be very interested in your forecasts for a few different dates. Could you share approximate forecasts for you personally dying because of AI by 2028/2030/2035/2040 as well?
Four days ago Jeff Kaufman wrote that "I'd put about 10% on futures where things go very badly [in the next two years, by roughly this time in 2028], where I'm not here to write a followup and you're not here to read one." (https://www.jefftk.com/p/donating-80-while-it-still-counts)
To think about the jarring situations we might encounter in the face of the Singularity like it’s something straight and narrow is probably a very human thing to do. Many institutions (marriage included) face the possibility of crashing or metamorphosis. Can we exactly compute the possibilities that will spin out? Marriage is the least of my worries but maybe it’s exactly what we need to care most about - such raw human to human interaction when there seems to be not much else like it that exists anymore.
Reading this made me realize that I want more thoughtful people to write about how short timelines have concretely changed their decisions. How have people internalized the impact of transformative AI?
I've also been thinking about this. As a teen I was very into quote unquote investing and FIRE, but now it seems silly to decrease my quality of life for a decade if the singularity hits
This is reasonable and I'm glad you're is writing it - but gosh, you're really not going to convince the people who took that one line out of context that you're not crazy :)
Heh, the superintelligence can also offer you another option, namely to alter your brain so that the person you’re married to becomes exactly what you want. If you’re fine with uploading, gene alterations, or taking life advice from the godhead, this seems like a less crazy option
I've literally encountered the "one partner wants to upload" problem. Doesn't seem that bad to me, though -- it seems likely that you can make an upload nondestructively, with some cost in fidelity. But I don't care that much about it being an exact match to me. And it helps quite a lot to know there is an upload-me, and they can talk to me about it and stuff, even if I'm not the upload.
I think there's also a good short story premise in this: guy who gets frustrated about not being an upload every month, and goes and makes an upload-copy of his most recent self.
Thank you for what you do Ajeya. I'm glad someone is considering these possibilities.
Got me. Thanks for the clarification. Stumbled upon your blog through your marriage post, was confused, now not so much anymore.
> I’m thirty one, and I think there’s a one in six chance that I personally die because of AI before my fortieth birthday.
Thanks for sharing this forecast. You'll be 40 this time in 2035, so that's about 16% that you die because of AI by 2035. I've been thinking about how this risk of personally dying because of AI is distributed across time, and would be very interested in your forecasts for a few different dates. Could you share approximate forecasts for you personally dying because of AI by 2028/2030/2035/2040 as well?
Four days ago Jeff Kaufman wrote that "I'd put about 10% on futures where things go very badly [in the next two years, by roughly this time in 2028], where I'm not here to write a followup and you're not here to read one." (https://www.jefftk.com/p/donating-80-while-it-still-counts)
I thought Jeff's 10% number was much too high for the next 2 years, so I asked him this same question about 2028/2030/2035/2040. You can see our rough answers in the comments of his post here: https://www.lesswrong.com/posts/mAhCkK5iBuZvCbFey/donating-80-while-it-still-counts?commentId=AAANJxsKbdLxwpXrK
To think about the jarring situations we might encounter in the face of the Singularity like it’s something straight and narrow is probably a very human thing to do. Many institutions (marriage included) face the possibility of crashing or metamorphosis. Can we exactly compute the possibilities that will spin out? Marriage is the least of my worries but maybe it’s exactly what we need to care most about - such raw human to human interaction when there seems to be not much else like it that exists anymore.