Recent comments in /f/technology

nickyurick t1_j86gu7v wrote

Reply to comment by jamesj in Scientists Made a Mind-Bending Discovery About How AI Actually Works | "The concept is easier to understand if you imagine it as a Matryoshka-esque computer-inside-a-computer scenario." by Tao_Dragon

I'm sorry, thank you for the summery but is there an ELI5 of this consept or does it now... work if its that simplified? Is this like quantum shenanigans where i need 7 years of physics to get it?

1

BarryBosseran t1_j86f96n wrote

Reply to comment by C0rn3j in Opera is planning to incorporate ChatGPT by Parking_Attitude_519

Here are also American shareholders, according to your perception Opera is a Chinese-Norwegian - American browser LOL

Greenhouse Funds LLLP 2,100,397 1.82%

J. Goldman & Co. LP 1,265,220 1.10%

Toroso Investments LLC 1,261,481 1.10%

FIL Investment Management (Hong Kong) Ltd. 1,040,129 0.90%

Park West Asset Management LLC 1,026,900 0.89%

Roumell Asset Management LLC 895,087 0.78%

Genesis Investment Management LLP 720,796 0.63%

APG Asset Management NV 644,305 0.56%

Discovery Capital Management LLC 444,810 0.39%

0

Jorycle t1_j86dx1u wrote

Reply to comment by PM_ME_GAY_STUF in Scientists Made a Mind-Bending Discovery About How AI Actually Works | "The concept is easier to understand if you imagine it as a Matryoshka-esque computer-inside-a-computer scenario." by Tao_Dragon

Yeah I work in ML and I don't get what the novel discovery is here based on the article. This all just sounds like... what we already know. Like this line:

>"We show that it is possible for these models to learn from examples on the fly without any parameter update we apply to the model."

That's so routine it's not even interesting.

I'm guessing the actual study goes into what was found, I'll have to read it when I have time.

59

Vegetable_Tension985 t1_j86dprp wrote

Reply to comment by VoidAndOcean in Scientists Made a Mind-Bending Discovery About How AI Actually Works | "The concept is easier to understand if you imagine it as a Matryoshka-esque computer-inside-a-computer scenario." by Tao_Dragon

AI can't just be cracked open and debugged in such ways. Each model is simply replaced if training doesn't ever work the way it is wanted. But no, the higher level operations are not easily understood. Statisticians understand the math and techniques of their models but the economy is a different story.

1

spsteve t1_j869c9r wrote

Reply to comment by FatedMoody in Millions of passwords stolen from LastPass earlier than company disclosed: Report by BasedSweet

Next question; what happens if someone breaches last pass and destroys the vaults and nukes the backups (and given they've been so heavily breached, and I have 0 confidence in them corporately to store safe backups) then what.

My initial point was, there are lots of good reasons to argue against paper vs password manager, but loss isn't one of them. Anything can be lost, and with these companies getting breached at this level (including some having backups deleted) I don't think THAT is the argument to use.

Finally, I am genuinely curious; when have you used lastpass in an offline state? Like why??? LOL If your network is down, what are you signing into you don't have memorized?

1

neuronexmachina t1_j867ome wrote

Reply to Scientists Made a Mind-Bending Discovery About How AI Actually Works | "The concept is easier to understand if you imagine it as a Matryoshka-esque computer-inside-a-computer scenario." by Tao_Dragon

Link to MIT summary of study: Solving a machine-learning mystery: A new study shows how large language models like GPT-3 can learn a new task from just a few examples, without the need for any new training data.

Actual preprint and abstract: What learning algorithm is in-context learning? Investigations with linear models

>Neural sequence models, especially transformers, exhibit a remarkable capacity for in-context learning. They can construct new predictors from sequences of labeled examples (x,f(x)) presented in the input without further parameter updates. We investigate the hypothesis that transformer-based in-context learners implement standard learning algorithms implicitly, by encoding smaller models in their activations, and updating these implicit models as new examples appear in the context. Using linear regression as a prototypical problem, we offer three sources of evidence for this hypothesis. First, we prove by construction that transformers can implement learning algorithms for linear models based on gradient descent and closed-form ridge regression. Second, we show that trained in-context learners closely match the predictors computed by gradient descent, ridge regression, and exact least-squares regression, transitioning between different predictors as transformer depth and dataset noise vary, and converging to Bayesian estimators for large widths and depths. Third, we present preliminary evidence that in-context learners share algorithmic features with these predictors: learners' late layers non-linearly encode weight vectors and moment matrices. These results suggest that in-context learning is understandable in algorithmic terms, and that (at least in the linear case) learners may rediscover standard estimation algorithms. Code and reference implementations are released at this https URL.

20