You should look into torch dynamo and torchinductor, a good overview here: https://dev-discuss.pytorch.org/t/torchinductor-a-pytorch-native-compiler-with-define-by-run-ir-and-symbolic-shapes/747

They’re pretty useless

lol

I'm involved with the Fairlearn project, so once I figure out what's necessary from a company policy-side, my plan is to incorporate these methods into Fairlearn one day.

Sure! I'm always happy to chat about this.

Not much, what are they going to do when we have an AGI tell it to behave.

exception:

Windows not yet supported for torch.compile

[deleted]

I think you’ve entirely misunderstood what PyTorch is and how it functions.

PyTorch is a front-end to libtorch, which is the C++ backend. Libtorch itself is a wrapper to various highly optimised libraries as well as CUDA implementations of specific ops. Virtually nothing computationally expensive is done on the python layer.

It worked in preview. Does it just not optimize? I didn't see significant improvements (e.g. under 5%)

What are the most important concepts that I need to know for ML? Possible courses are below:
Algebra & Calculus II
Algebra & Calculus III
Bayesian Stats
Probability
Multivariate stats analysis
Stochastic processes
Time series
Statistical inference

To what extent should I know and be familiar with linear algebra?

[removed]

Or just plain accounting. Like what most recovered ancient written documents tend to be.

torch.compile does not work in windows :(

Yes, doable on a low budget if you have no fear of legal actions...

Follow https://crfm.stanford.edu/2023/03/13/alpaca.html

Yeah, but what if you run them through an SAT prep course first? It is impressive relative to other NLP but it could not be a better illustration of exactly those criticisms that it's responding to. Extremely formulaic.

It took more than 6 years from zero, because to reach GPT4 you had to develop transformers and all the GPTs before 4... The actual difference between ChatGPT and GPT4 is apparently in the data and some policies that regulate when it is allowed to answer (which are still incomplete). This is not remarkable.

I AGAIN fail to see how this relates to previous comments.

Would it be possible to train a quantzised model?

In your job as data scientists have you ever had to compare the quality of the probabilistic forecasts of 2 different models? if so, how do you proceed?

This work is exactly the kind of thing I'm interested in doing. Do you mind if I DM you for some career advice?

Former CEO of Github as well.

I just hope that whatever regulations Congress choose to implement actually end up being effective at promoting ethics while not crushing the field. After seeing Congress question Zuckerburg, I can’t say I have 100% faith in them. But I’m willing to be optimistic that they’ll be able to do a good job, especially since I believe that regulating AI has largely bipartisan support.

I think the most similar historic example is the human genome project where the government and private industry where both racing to be the first to fully decode the human genome but the US government was releasing its data and industry could use it to get even further ahead.

Its the classic prisoners dilemma. If both parties are secretive research is much slower and might never complete but with a small probability of completing the project first for a high private reward for the owner and low reward for society. If one party shares and the other does not, the withholding party gets a huge comparative boost for a high probability of a high private reward. If both parties share we have the best case with parties being able to split the work and share insights so less time is wasted for a very high probability of a high private and high public reward.

I think for AI we need mutual cooperation and to stop seeing ourselves as rivals. The rewards for AI cannot be privatized for the shared mutual good of humanity in general (“Humanity” regrettably does include Google and the spider piloting Zuckerberg’s body). Mutual beneficial agreement with enforceable punishment for contract breakers is what we need to defuse tensions, not an escalation of tensions.

What you don’t understand is it took 2 yrs to go from nothing to gpt-4. Do you understand how unbelievably fast that is? A tsunami of money is about to pour into this industry!

That’s really cool, it’d be awesome if something like that was built into TensorFlow or PyTorch.