Recent comments in /f/MachineLearning
username001999 t1_jcsr60y wrote
Reply to comment by Quail-That in [R] ChatGLM-6B - an open source 6.2 billion parameter Eng/Chinese bilingual LLM trained on 1T tokens, supplemented by supervised fine-tuning, feedback bootstrap, and RLHF. Runs on consumer grade GPUs by MysteryInc152
Can you read Chinese? If so, you can read all about the Tiananmen protest on the Chinese internet or talk to actual Chinese citizens about it on WeChat.
asraniel t1_jcsr22m wrote
Reply to comment by raduqq in [P] The next generation of Stanford Alpaca by [deleted]
wo how does that work? soon a good chunk of the internet will be text written by gpt (including wikipedia). does that mean going forward you cant legally use the internet as a datasource to train a llm?
username001999 t1_jcsr1pi wrote
Reply to comment by extopico in [R] ChatGLM-6B - an open source 6.2 billion parameter Eng/Chinese bilingual LLM trained on 1T tokens, supplemented by supervised fine-tuning, feedback bootstrap, and RLHF. Runs on consumer grade GPUs by MysteryInc152
lol, whining about whataboutism is the last refuge of hypocrites.
yaosio t1_jcsqxwf wrote
Reply to comment by lxe in [P] The next generation of Stanford Alpaca by [deleted]
If doesn't matter what the license terms say if it can't be enforced.
lxe t1_jcsqmdi wrote
Reply to [P] The next generation of Stanford Alpaca by [deleted]
You should try fine tuning openchatkit — it’s Apache 2 licensed afaik. Or GPT-NEOX-20B if you have the hardware.
lxe t1_jcsqk7t wrote
Reply to comment by yaosio in [P] The next generation of Stanford Alpaca by [deleted]
Copyright and license terms are different things.
Spiritual-Reply5896 t1_jcsq4d9 wrote
Reply to comment by 127-0-0-1_1 in [D] PyTorch 2.0 Native Flash Attention 32k Context Window by super_deap
Exactly, I wanted to find out whether there is some research regarding these embeddings. I really think that by efficient pruning/organization of these "memories" its possible to generate quite advanced memory. Things like embedding consistency then becomes a big player - how much does length affect the embedding, what is the optimal information content vs string size...
Seromelhor t1_jcsov3a wrote
Reply to [P] The next generation of Stanford Alpaca by [deleted]
You can use NLLB from Facebook to translate the sentences from English to more than 200 other languages. That would be interesting.
FaceDeer t1_jcsot55 wrote
Reply to comment by raduqq in [P] The next generation of Stanford Alpaca by [deleted]
All these weird restrictions and regulations seem pretty squirrelly to me.
Maybe this could be "laundered" by doing two separate projects. Have one project gather the 2 million question/response interactions into a big archive, which is then released publicly. Then some other project comes along and uses it for training, without directly interacting with ChatGPT itself.
I'm sure this won't really stop a lawsuit, but the more complicated it can be made for OpenAI to pursue it the less likely they are to go ahead.
yaosio t1_jcsob5z wrote
Reply to comment by ThatInternetGuy in [P] The next generation of Stanford Alpaca by [deleted]
The output of AI can't be copyrighted so OpenAI has no say in what somebody does with the output.
xerca t1_jcsnz4j wrote
Reply to comment by Quail-That in [R] ChatGLM-6B - an open source 6.2 billion parameter Eng/Chinese bilingual LLM trained on 1T tokens, supplemented by supervised fine-tuning, feedback bootstrap, and RLHF. Runs on consumer grade GPUs by MysteryInc152
And derailing any topic that comes out of China into Tiananmen square is not acting bad faith? Especially given that the American company "Open"AI is heavily guarding and paywalling their models while this Chinese group is sharing theirs with the world for everyone to use.
Conflating anything that comes out of a country with 1.5 billion people with your incredibly shallow knowledge of history only serves to demonstrate your ignorance.
farmingvillein t1_jcsnx0f wrote
Reply to [R] ChatGLM-6B - an open source 6.2 billion parameter Eng/Chinese bilingual LLM trained on 1T tokens, supplemented by supervised fine-tuning, feedback bootstrap, and RLHF. Runs on consumer grade GPUs by MysteryInc152
"open source".
That license, lol:
> You will not use, copy, modify, merge, publish, distribute, reproduce, or create derivative works of the Software, in whole or in part, for any commercial, military, or illegal purposes.
> You will not use the Software for any act that may undermine China's national security and national unity, harm the public interest of society, or infringe upon the rights and interests of human beings.
> This license shall be governed and construed in accordance with the laws of People’s Republic of China. Any dispute arising from or in connection with this License shall be submitted to Haidian District People's Court in Beijing.
What a nightmare.
assimil8or t1_jcsnwuh wrote
Reply to [P] The next generation of Stanford Alpaca by [deleted]
Would UL2 be a good basis?
emotionalfool123 t1_jcsmthb wrote
extopico t1_jcsmc8t wrote
Reply to comment by username001999 in [R] ChatGLM-6B - an open source 6.2 billion parameter Eng/Chinese bilingual LLM trained on 1T tokens, supplemented by supervised fine-tuning, feedback bootstrap, and RLHF. Runs on consumer grade GPUs by MysteryInc152
Oh look a wumao deploys wahtaboutism!
kex t1_jcsm7kh wrote
Reply to comment by Anjz in [P] The next generation of Stanford Alpaca by [deleted]
I'd say enjoy it while it lasts, at the very least
Long19980 t1_jcsm4ni wrote
Reply to [P] The next generation of Stanford Alpaca by [deleted]
Can I see your python script? How did you balance your programming language data between the various languages?
Long19980 t1_jcsllwx wrote
Reply to comment by ThatInternetGuy in [P] The next generation of Stanford Alpaca by [deleted]
They can go cry about it.
raduqq t1_jcslcbz wrote
Reply to [P] The next generation of Stanford Alpaca by [deleted]
I thought their ToS doesn't allow you to train another model on the output of their ChatGPT or GPT-4 or other models.
Anjz t1_jcsktsf wrote
Reply to comment by throwaway957280 in [P] The next generation of Stanford Alpaca by [deleted]
It's probably untested in courts, there's so many loopholes and variables too, what's considered a competing AI model? Companies usually just spew a bunch of stuff in their terms of use, some of which have no legal basis.
RemindMeBot t1_jcsjtqk wrote
Reply to comment by retrogod_thefirst in [R] ChatGLM-6B - an open source 6.2 billion parameter Eng/Chinese bilingual LLM trained on 1T tokens, supplemented by supervised fine-tuning, feedback bootstrap, and RLHF. Runs on consumer grade GPUs by MysteryInc152
I will be messaging you in 2 days on 2023-03-21 05:56:26 UTC to remind you of this link
2 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) ^(delete this message to hide from others.)
| ^(Info) | ^(Custom) | ^(Your Reminders) | ^(Feedback) |
|---|
throwaway957280 t1_jcsjj07 wrote
Reply to comment by ThatInternetGuy in [P] The next generation of Stanford Alpaca by [deleted]
Is OpenAI actually legally allowed to do that? How is using their model for training different from training on copyrighted data which all these models do?
[deleted] OP t1_jcsjd9y wrote
Reply to [P] The next generation of Stanford Alpaca by [deleted]
For those who wish for LLaMA to become truly open source, please vote on this:
ninjasaid13 t1_jcssyvo wrote
Reply to [P] The next generation of Stanford Alpaca by [deleted]
can you finetune OpenAssistant's model? https://huggingface.co/OpenAssistant/oasst-sft-1-pythia-12b I hear their pythia language model isn't very good or maybe gpt-j which is more consumer grade friendly.