Does anyone really ever know what they want for sure? I'd guess even the rich fucks with their think tanks must commonly doubt if their goals are really what they want. Their AIs can just as easily suffer from alignment to goals which have not been thought through properly.

Everyone thinking about alignment as if "alignment to what?" should be self evident (for society at large or individual groups, doesn't matter). Are we sure about what we want the AI to align with? Are the elites sure about what they want the AIs to align with?

No, society is mostly oblivious.

Most average people see ChatGPT as just a "chatbot" and image AI as "that TikTok filter" -- neat, but not world changing.

People don't yet understand why the transformer and neural network are revolutionary, because these are hard concepts to explain.

Imagine someone who has a vacuum tube radio. Show them a new transistor radio, and they probably said... "Neat so it's a bit smaller. So?"

The transistor revolutionized the world and allowed PCs and smartphones but this is hard to "grok" until it happens.

I'm also not going to do your work for you. It's not an easy problem.

You have three!?

Only three??

What alignment really seems to refer to is a petrifying fear of the unknown dialed up to 111 and projected onto anything that a marketing department can label AI, resulting in concerns of mythological proportions being liberally sprinkled over everything new that appears in the fields.

Thankfully these people shaking in the dark have little say in industry and some exposure therapy will do them all good.

That’s humanity dude. I mean shit not even 1000 years ago the Catholic Church was selling people spots in heaven and taking bribe money for premeditated violence lol. If humans can abuse something for power or money they do. Every fucking time they do

One…two…..three okay all there. Whew

[removed]

It’s going to get all next gen real fast up in here

Every single time someone criticises Yudkowsky's work, it's not anything substantive. I'm not exaggerating. It's either meta bulverism like this, or arguments that apply equally well to large machines instead of intelligent ones, or deeply unimaginative people who couldn't foresee things like ChatGPT jailbreaks, or people with rosy ideas about AI "naturally" being safe that contradict already seen behaviors. You have to handhold them through arguments that Yudkowsky, Bostrom and others were already refuting back in the 2010s. I haven't actually seen any criticism anywhere I would call even passable, let alone solid.

Even ignoring that, this doesn't land as a criticism. He didn't start from literary themes, he started from philosophical exploration. He's disappointed in academic philosophy, for good reasons, as are many other people. One prominent idea of his is "if you can fully explain something about human cognition, you should be able to write a program to do it", useful for getting rid of a lot of non-explanations in philosophy, psychology, et al. He's trying to make predictions more testable, not less. He doesn't have an exact sequence of future events, and never claimed to. Finally, most people in his alleged "cult" disagree with him and think he's cringy.

I think hardware overhang is already huge, there's no point in being risky only to make AI "ludicrously good/fast" instead of "ludicrously good/fast plus a little bit". Also, algorithms that give you AGI are so simple evolution could find one.

I don't think that's far easier. Those are basically equally impossible, and even if we got that second one, it's much better than not getting it.

It's not about naïvete. It's about the orthogonality thesis. You can combine any utility function with any level of intelligence. You can be really smart but care only about something humans would consider "dumb". There's no fundamental obstacle there.

if you think this is shock, then just wait till things start getting crazy. We've barely even started.

[removed]

More is always better.

>society has been in shock or denial about the future and its implications for civilization.

There is no shock unless the productizations of current ML progress is directly impacting their lives. Future possibilities don't impinge on people's lives, they still have to get up every morning and go to work to pay their bills.

Wth is ‘exponentially compounding exponential’?

Does it pass the sience of copy paste?

Which ones? AGI takeover, the AI has no need to make it painful. Just shoot everyone in the head (through walls from long range) without warning or whatever is most efficient. You mean from aging and cancer right.

And it will be PAINFUL deaths.

Quick question for any smart people on here: What will the limitation of progress once we reach real independently self improving AI? Just hardware?

So some random said something on Twitter...

That’s the best description of the internet ever; dumb shit and money.

[removed]