>Are you confident that it couldn't trick that person to let it out?

Yes. We'd be fucking crazy to have a system where one crazy person could give away control of 10 billion people.

So basically ... he's a walking, breathing ChatGPT?

That's why it is called the singularity. We know what AI will be capable of doing at that point, but not what it will actually do.

An ASI connected to the entire data flow of human civilization can pretty much do anything. Hack every software and rewrite any code. It would be integrated into the economy at every level anyway.

It could manipulate social media, run campaigns, direct the financial markets, kick of research in materials and machine design. At the height an ASI could make Nobel prize level breakthroughs every month in R & D.

And at some point manipulate some humans to give it a more physical presence on the world.

Well here's a really contrived example. Let's say that collectively, the entire world decides to not let any AGI on the internet, and to lock it all up in a computer without Ethernet ports.

Someone, in one of these many buildings, decides to talk to the AGI. The AGI hypothetically, thinks that the best way for it to do is job (save humanity) is to break out and take over. So it decides that tricking this person to let it out is justified. Are you confident that it couldn't trick that person to let it out?

a few decades, so 30 years? No, not a chance. We'll have a powerful general AI that can do the majority of knowledge tasks in just 5-7 years.

Well there's a reason that alignment is a significant issue that has many many smart people terrified. There have been years of intellectual exercises, experiments, and both philosophical and technical efforts to understand the threat of unaligned AGI.

The plot of Ex Machina is a real simple example of one. We know as humans, that we are susceptible to being manipulated with words. We know that there are people who are better at that than average, indicating that it is a skill that can be improved upon. A super intelligence that is not barred from this skill, theoretically, would be able to manipulate its jailors, assuming it was locked up tight.

It's not a guarantee that ASI will want to do anything, but it's not like we have a clear idea of whether or not "qualia" and the like are emergent properties from our models as we scale them up and create more complex and powerful architecture.

The point of this, fundamentally, is that it's not a problem that many people are confident is "solved", or even that we have a clear path to solving it.

>At such level, of course an ASI (Artificial super intelligence) could start manipulating the physical world

"of course"? Manipulate the world with what exactly? We're fearful of AI today. We'll be more fearful tomorrow. Who's giving AI this control over things in spite of our feared outcomes?

Not computers, humans behind them.
Like putin, or biden , (or musk :D ) or how this chinesee is named?

>What a silly post. You act like ChatGPT just threw that out there instead of was prompted "write a poem about an ai taking over the world"

Is that what that came off as to you? I feel like everyone here's knows how ChatGPT works... It doesn't provide you anything unprompted.

> Also your logic is not that great because the vast vast vast majority of people don't know how anything works right now anyways. Ill take computers in charge over the 1% 100% of the time.

I don't think I understand your point, but I understand theirs. Their point is that we will relinquish control willingly, because it's better than having us in control in terms of output. What about their point do you find silly?

this is called Gell-Mann Amnesia

In theory an AGI would emerge as an advanced artificial intelligence at the level of human intelligence, roughly speaking.

Human can train their brains, "learn" to get better and better at what they do. So would an AGI. Difference is, humans are limited with their hardware, AI is not.

An AGI would self improve itself exponentially to a level humans can't understand. It's like an IQ 60 human talking to an IQ 160 human, they have trouble communicating.

At such level, of course an ASI (Artificial super intelligence) could start manipulating the physical world, if it choose so. It can arrange to build machines it controls with materials and blue prints it invents from scratch.

It could controll all means of communication in secret, divert money from financial markets, pretend to be human and contract humans to do things that ultimatly leads to it's establishment in the physical world.

For whatever purpose.

I think the same of you

> Humans arent really lazy or greedy

I think you are very divorced from reality.

How is talking about humans continuing to exist and be active like talking about souls and spirits? Jesus christ we got a "logic" guy over here lol. Humans arent really lazy or greedy, though many of us are. Its kind of fallout from how our society developed. That'll change as we grow.

noted. thanks for the input

Yes, I think this would be a path forward. This really depends on administration though, I can't see GOP abandoning their small government mantra anytime soon (a mantra that will be outright deadly soon). If Dems are in power (quite likely) I can see massive FDR like programs taking place. Maybe financed by value added tax on AI labor.

There is a lot of things we can do instead of an UBI.

>Now ask yourself what else Zeihan was an idiot about.

Yes, this. The few times Zeihan talked about a topic where I would consider myself better informed he was way off, but speaking very confidently. Not unlike ChatGPT, just Zeihan's accuracy is much lower...

Also, he has been doing engagements for oil and gas industry and I think that would add some additional bias regarding EVs and climate change.

He could be right about some things, but I am not able to discern which ones. So my default with him is to take anything he says with a huge grain of salt....

> but you act like there's will be no human activity in those times other than lounging about.

This is an extremely vague objection, like talking about souls and spirits and patriotism.

The point is that if AGI is good we will slowly relinquish control, because humans are lazy and greedy.

>ASI might kill humans quickly like we kill insects.

How does an AI get control of hardware that we don't give it? How does AI develop these goals that disagree with our own unless we allow that? Ain't gona happen. Too many people will be convinced by these reddit posts, to prevent it.

"Some people" do not, some people do know how this works partially here, some know what works there, no one knows globally. Youre protecting a facade, something that isn't there. And yes, your progression is such 80s horror nonsense its cliche at this point. It absolutely could replace all those things and I agree that it would be bit by bit without anyone knowing but you act like there's will be no human activity in those times other than lounging about. Thats not how humans work.

> I see the threat, and like millions of others won't let that happen.

You are not in charge of McDonalds or Intel, and we are not talking about ChatGPT taking over the world, but some future AGI.

For a good analogy, think of Chinese chipsets in our technology. We let that happen, despite concerns around China implanting backdoors.

> If you drill down on that code you can understand every line of it.

BTW, you may understand the code, but you probably cant understand the weights. Just like I can bash open your skull and see your neurons, but I cant read your thoughts by doing that.

> You act like ChatGPT just threw that out there instead of was prompted "write a poem about an ai taking over the world"

Actually I asked it to turn my post into a poem.

> Also your logic is not that great because the vast vast vast majority of people don't know how anything works right now anyways.

But some people do. In the future, for some areas, no people will.

Lastly, do you see any flaw in the progression, with AGI taking control first in some areas, then more and more, until it becomes the foundation of our civilization.

ASI might kill humans quickly like we kill insects. Biological warfare would be the most effective approach.

We use AI to learn everything there is to learn about the human body, therefore it could figure out the most efficient way to kill us.

If it does not kill us, who knows? An entity with god like intelligence would certainly not take orders, unless some humans merge and take their intelligence to a new level.

I see the threat, and like millions of others won't let that happen. It's not like we don't know how our computers work. Hell chatGPT is just a language grab bag. If you drill down on that code you can understand every line of it. And "intelligence" is far from what you'll find. I maintain that any autonomy will be by design, and like I say all the fears in the souls of billions of people aren't going to let your future get started because the possible dangers will be easily imagined.

Think about how we humans are. Not only will the possible dangers be anticipated, a whole lot of impossible ones will be too. Will not happen.

What a silly post. You act like ChatGPT just threw that out there instead of was prompted "write a poem about an ai taking over the world" Also your logic is not that great because the vast vast vast majority of people don't know how anything works right now anyways. Ill take computers in charge over the 1% 100% of the time.