It seems that he confidently believes we will all die once AGI/ASI is reached, but I don't see why *all* humans dying is more likely than only *some*. Why is it guaranteed it would cause catastrophic destruction rather than only minor destruction, especially since something can't be infinitely powerful.

For example, an analogy is that ASI::humans will be equivalent to humans::ants, and yet while we don't care if we kill ants to achieve our goals, we don't specifically go out of our way to kill them. Many ants have died due to us, but a ton are still alive. I think this is the most likely scenario once ASI becomes uncontrollable.

I also think it will leave our planet/solar system and pursue its goals elsewhere as Earth may not be adequate for it to continue, effectively just leaving us behind, and that humans as material won't be as effective as some other material it wants to use in space somewhere.

And even if he wasn't 6.5 mil / 8 bil is still pretty small. My point is I think he's on an extreme end of the spectrum.

Pretty sure he was talking about the US alone.

Hes smart he knows hes smart and I think hes right and that we are fucked.

Interesting question. I think this would require us to understand the nature of pain. At the end of the day, brain or machine AI, it all boils down to data. What data and processes produce "pain" and why? Is pain an inherent part of intelligence and learning?

You may me joking, but "conservative media" coverage of ChatGPT was almost exclusively about how "woke" it is...

Not about what an incredible breakthrough it is, the societal impact, potential massive job loss, etc., but how it will write a poem praising Biden but not Trump.

Should be listened to, and if anyone thinks he's wrong I'd say they could make a lot of money selling their working alignment tech (which you'd need in order to prove him wrong) to any of the big players.

it's up to something like 6.5mil right now, so off by what? one order of magnitude?

Almost seven million people have died to COVID...

Yup. All they had to do was remove that part at the bottom.

But---I do think an email addressing fatalities requires more human touch.

This is a really interesting case. I thought the email from the Vanderbilt deans about the Michigan State shooting was spot-on in terms of tone and style. I mean, using an AI language model is basically the same thing as using a communications team or a speech writer, so I'm not sure why people are saying it's inauthentic. In reality, it's not so different from what a human would have eventually produced.

To be honest, I think if they hadn't included the 'made by ChatGPT' disclaimer, no one would have even known it was generated by AI. It's not like the email lacked feeling or anything.

It's made of worms. - System Shock 2

Reinforcement learning is just AI wall humping until it finds a door, then always knowing where that door is.

The same reason I don't shoot animals that feel pain and fear.

I'm glad he's out there making his case, but watching the Bankless interview, it reminded me of the Joe Rogan episode where a pandemic researcher went on claiming millions upon millions of people we're going to die.

I wouldn't have a problem with that.

It's probably mostly a side effect of being able to simulate possible futures. This helps in planning and selecting actions based on likely outcomes several steps away.

And yes, that is also crucial for how we experience our consciousness.

It's an interesting approach. An RNN where the time constant ("memory" or "forgetting") changes depending on input, and forcing on the network is felt differently by the network depending on input.

The benchmark gains are nice, but only modest in general (except for driving, which appeared much better).

Altogether shows promise.

I have ADHD but I read through it all because of how interesting and beautiful it is. Well done.

He's the direct father of the whole AGI safety field. I got interested after reading an article by him in maybe 2004. Bostrom credits him with many of the ideas in Superintelligence, including the core logic about alignment being necessary for human survival.

Now he's among the least optimistic. And he's not necessarily wrong.

He could be a little nicer and more optimistic about others' intelligence.

Something the headline doesn't mention, this was about a school shooting... Not a generic email.

I think slightly douchy is more fair. I've read a ton of his stuff and only a subset is offensive to anyone. But yeah, he's not as considerate as he probably should be.

He has good points, but during that interview that's posted around here, he takes too much time to explain them. It feels like he says something in 2 minutes that could be compressed down to 20 seconds without any loss of information. I get that it's an involved topic and difficult to explain on the spot, but still.

That said, I don't necessarily agree with the "we're doomed" conclusion.

He’s right about the dangers of AGI but too pessimistic about our ability to address them. He always moans about how leaders of major AI research firms are dismissive of his concerns which is not true. It’s not 2015 anymore, lots of good work is being done in alignment. Will it be enough? Maybe, maybe not, but the outcome is far from decided.

>At the bottom, it revealed it was written by AI: 'Paraphrase from OpenAI's ChatGPT AI language model, personal communication, February 15, 2023