like the comment- human reinforcement (good bot/bad bot)

I guess I just sign up

Speaking of this do you guys know of ways to inference and/or train models on graphics cards with insufficient vram? I have had some success with breaking up models into multiple models and then inferencing them as a boosted ensemble but thats obviously not possible with lots of architectures.

I'm just wondering if you can do that with an unfavorable architecture as long as its pretrained.

Yeah! I can sing and be creative in that sense. But no patience in learning guitar or piano. But when we can make stuff like that, better than jukebox ai, I can start on that hobby

It's not a user-facing product, it's a building block that would be useful to train music-oriented neural network, be they diffusers or other types of models.

It's probably going to take a little while before we see new models that leverage this library.

If you're looking for "stable diffusion but for music" right now, you could look at Riffusion (https://huggingface.co/riffusion/riffusion-model-v1)

>audioflux
>
> is a deep learning tool library for audio and music analysis, feature extraction.

> FFT hardware acceleration based on different platforms

???? I love me some FFTs but "hardware acceleration"?

Feedback on stratified k fold validation

I am doing some applied work with CNNs in the academic world.

I have a relatively small dataset.

I am doing 10 fold stratified cross validation(?) where I do an initial test-train split, and then the data in the train split is further cross validated to a 10 fold train-validate split.

I then run the ensemble of 10 train models against the test split, and I select the results from the best performing model against the test data as the predicted values for the test data.

Is this a reasonable strategy? Thank you!

https://github.com/oobabooga/text-generation-webui

I’ve had great results with this interface. It requires a little tweaking to get working with lower specs, but it utilizes a lot of optimization options including splitting the model between VRAM and CPU RAM. I’ve been running LLaMa 7b in 8bit and limiting to 8GB of VRAM.

In few-shot learning, are there gradient updates from the examples? If not, what difference does it make?

Yes. We have image AI, NLP text AI, video is on the way, probably later this year. I've been waiting for music AI. Jukebox was pretty meh. I know it can be way better.

Very cool! I have been looking for a better toolkit for audio analysis, this looks great!

[removed]

Question about ML research breakthroughs and narratives.

AlexNet was not the first and not the fastest and not the CNN that won the most prices - using Nvidia GPU CUDA cores for acceleration. Then why is it so often named as the 'it' paper in the popular MSM & AI YouTube Channels narrative around AI? Even Jensen Huang, CEO of Nvidia mentioned it in his keynote.

Is it because AlexNet can be traced back to 'Made in America' and sold to Google? And co-author is Chief Science Officer at OpenAI? And the others aren't.

Since you are training the LSTM by using labels, it is supervised or perhaps self-supervised depending on the specifics

/r/LocalLLaMA

> there's a number of efforts like llama.cpp/alpaca.cpp or openassistant but the problem is that fundamentally these things require a lot of compute, which you really cant step around.

It's honestly less than you'd expect. I have a Ryzen 5 1600 which I bought about 5 years ago for $200 (it's $79 now). I can run llama 7B on the CPU and it generates about 3 tokens/sec. That's close to what ChatGPT can do when it's fairly busy. Of course, llama 7B is no ChatGPT but still. This system has 32GB RAM (also pretty cheap) and I can run llama 30B as well, although it takes a second or so per token.

So you can't really chat in real time, but you can set it to generate something and come back later.

The 3 or 2 bit quantized versions of 65B or higher models would actually fit in memory. Of course, it would be even slower to run but honestly, it's amazing it's possible to run it at all on 5 year old hardware which wasn't cutting edge even back then.

It's pretty much all that's been posted here for the past week

There's a lot coming up. I'm looking into it right now, here's a tutorial I found:

https://medium.com/@martin-thissen/llama-alpaca-chatgpt-on-your-local-computer-tutorial-17adda704c23

​

Here's something unique, where a smaller LLM outperforms GPT-3.5 on specific tasks. It's multimodal and based on T5, which is much more runnable on consumer hardware.

https://arxiv.org/abs/2302.00923

Thanks for the tips! Will consider them!

I like to see all these advances optimizing machine learning more and more. In 10 years (being pessimistic) it will be very interesting, and I sincerely hope that neuromorphic processors leave the laboratory and become real, this would advance the area even further.

Ah! Iv only just started to dive into the machine learning rabbit hole so I was not sure of if was understanding the term correctly.

Im keen to check it out.

Awesome! Thanks for the explanation. "exact" vs "approximate"!

Looks neat. How does it compare to OpenSMILE? The license sure makes it an attractive alternative.

Hello guys, would you consider this supervised or unsupervised learning?

I am using Keras LSTM to generate new text, by tokenizing it, making n-grams from it and training the LSTM to predict the next word (token) by putting n-1 n-grams as a train sample, and as "labels" I am putting the last word (token) of the n-gram. Would you consider this supervised or unsupervised ML?

Technically, I do have a label for every n-gram, its own last word, but the dataset itself was not labeled beforehand. As I am new to ML I am a little bit confused and even ChatGPT sometimes says that its supervised, and sometimes unsupervised ML.

Thanks for any answers.

there's a number of efforts like llama.cpp/alpaca.cpp or openassistant but the problem is that fundamentally these things require a lot of compute, which you really cant step around.