OK, insofar as you care about adoption, I'd encourage you to clean up the README to make it much clearer as to what you're doing. Right now, you've got API call examples, but it isn't clear what is actually happening, why this wrapper is helpful/necessary, etc.

I can guess/infer all the above, but you want your README to make it really, really quick and easy for your readers to figure out what is going on.

do you have a link to a torrent/download for the 30B or 65B weights that works with Alpaca.cpp? reddit DMs are fine if don't want to post it publicly.

There are several. either collaboratively (look at together.computer hivemind petals) or on single no gpu machine with pipeline parallelism, but it requires reimplementing for every model, see e.g slowLLM on github for bloom176b

I'm using amp mixed precision which should be using fp16. It still requires training==false.

But the torch code also disables flash attention if autocast is enabled I'm not sure how to deal with that one.

would that mean for forcing MHA to use it, I should wrap the ctxmanager around the line where I forward through it?

with torch.backends.cuda.sdp_kernel(enable_flash=True, enable_mem_efficient=True):
            x = x + self.attn_head(x, x, x, need_weights=False)[0]

because that doesn't really seem to work :(

Thank you for the very detailed response!

Can you train with 24 gigs of vram ?

Can't wait to try this out 😁

Got it. Appreciate the insight.

I want to announce that we have released v1.1.0 which includes access for ChatGPT and GPT4 for Plus suscribers! :)

https://github.com/citiususc/Smarty-GPT/tree/v1.1.0

The main purpose of this project is joining in a single environment all the resources (models, prompts, APIs, etc.) related to LLMs. Moreover, we also think from an end-user perspective. It is heavily unlikely that a user would introduce a complex context in a query to a model or searcher. In this project, we try to bias the different model responses to answer in different ways/behaviors, but hidding this to end-users.

That's right :) Check the "purpose" in our readme

[deleted]

I don't think they can use llama cause of the limited open source rule fb put in llama. Wouldn't be as entirely open as pythia

Even for protein folding it has been overridden by deep models. It might be useful for critical tasks where error is not allowed and everything is deterministic, but I'm not expert of the field.

try fp16, that doesn't require training=False apparently.

Alright, thank you. Then I guess last question, if you happen to know; what is the current state of ILP in the ML/AI industry? Is it pretty much dead? Is it merely an interesting theory but hasn't found much application in the market? Does anyone see a bright future for it?

OK, I found out why. To use flash attention, I had to use fp16. It is a bit faster then using memory_efficient attention in my test.

[deleted]

Have you tried NFNets https://arxiv.org/abs/2102.06171

Currently there are no good methods to do this. There’s discussion of existing methods and many insights into the problem in this paper https://arxiv.org/abs/2206.14486

TL;DR: which images you should remove depends on the ratio between samples / parameters, no current method works anywhere near ideal, but you may see improvements if you choose the most expensive methods

Thanks for sharing! As far as I know LoRa approach increases efficiency, not so sure about quality Wiki. Maybe the paper can help you further.

Usually, the problem is the combinatorial nature of the possible number of rules that could apply. Here they seem to be able to find a subset of possible rules with a polynomial complexity, but as table 7 of the second paper contains tiny 'wrt ML/RL data) instances of problems, I would answer yes to your questions. ILP is something coming with strong guarantees, while ML comes with a statistical risk. Theses guarantees aren't free.

So the main problem with MIL or ILP is that it would not be able to handle the scale of the quantity of raw input data that the system would have to process?

[removed]