Recent comments in /f/MachineLearning

austacious t1_je0g6oi wrote

Reply to comment by truchisoft in [N] OpenAI may have benchmarked GPT-4’s coding ability on it’s own training data by Balance-

A healthy skepticism in AIML from those in the field is incredibly important and relatively hard to come by. Having the attitude that 'This is great and everything is wonderful' does not lead to meaningful progress addressing very real issues. It's very productive to point out shortcomings of otherwise highly effective models.

12

ArnoF7 t1_je0dzqg wrote

Reply to [N] OpenAI may have benchmarked GPT-4’s coding ability on it’s own training data by Balance-

Funnily, I actually found GPT-4 far worse than what I expected in terms of coding, especially after I looked at its impressive performance on other exams. I guess it’s still a progress in terms of LLM for coding, maybe just a little underwhelming compared to other standardized tests it aces? GPT-4’s performance on codeforces is borderline abhorrent.

And now you are telling me there is data leakage, so the actual performance would be even worse than what’s on paper???

5

EvilMegaDroid t1_je0d2a0 wrote

Reply to comment by Professional-Gap-243 in [D] FOMO on the rapid pace of LLMs by 00001746

There are many open source projects which in theory can do better than chatgpt.

The issue? Spend millions of dollars on the data to fed it.

Open source LLM are useless, the data is the important part.

Google microsoft etc can fed them their own data and they still spend millions of $,imagine how much it would cost for the normal joe to buy that data and the operating cost.

I doubt there will ever be an open source chat gpt that just works.

5

JigglyWiener t1_je0czvc wrote

Reply to [D] With ML tools progressing so fast, what are some ways you've taken advantage of them personally? by RedditLovingSun

I mostly automated the production of meat themed scripture. I used ChatGPT to write the scripts that run a madlib style series of prompts based on a lore I wrote about anthropomorphic flies that evolve to worship meat heaps through the openAI API and have ChatGPT3.5 generate the scripture. It then gets fed through Google's text to speech and I use it in my decaying meat livestream.

Also it spruced up my resume and let me build a mini application that summarizes online reviews for a side hustle I have helping a niche small business understand online marketing.

10

nxqv t1_je0cw14 wrote

Reply to comment by Impallion in [D] FOMO on the rapid pace of LLMs by 00001746

>People want to have a "large" impact - making company-wide differences, influence large swaths of people. I think the fear is that in the face of a ChatGPT, your little model or little application can only reach a handful of others.

Yes, it's this idea of wanting to make "as large of an impact as possible" that I was starting to chip away at. A lot of people - myself often included - feel dismayed when we think about our work only impacting a tiny corner of the world. It feels like you're "settling for less." But when you finish that thought, it sounds more like "settling for less than what I'm capable of" which has a lot to unpack.

And for the record, I think it's okay to want to make a big splash to satisfy your own ego. I wasn't trying to say that it's immoral. I just think it's important to understand that you're in that position and unpack how you got there. Mindfulness is the way to combat FOMO, as well as all sorts of other negative emotions.

>My solution is that we need to dig a little deeper. What does it mean to be human? What does it mean to live a good meaningful life? If your answer to that is that a good life worth living is one where you impact on the order of thousands or millions of humans, then yes we might be shifting away from that possibility. But humans are built for connection, and I think we will need to look inwards and realize that we don't need to influence thousands to experience that connection. You can make a little model or application that affects hundreds. You can write a song just for your friends and family. You can paint a piece of art that just hangs on your wall and gets a single compliment. To me that is already human connection, and is just as meaningful as making a large model that drives the next Google/Meta forward.

Yes yes yes.

2

sebzim4500 t1_je0c899 wrote

Reply to comment by TehDing in [P] two copies of gpt-3.5 (one playing as the oracle, and another as the guesser) performs poorly on the game of 20 Questions (68/1823). by evanthebouncy

> You can ask GPT to spell a word, or provide the words as individual "S P A C E D" characters and it will similarly do poorly- it has nothing to do with tokenization. GPT is capable of spelling, it can even identify that it is not playing well if you ask if something is a good guess- but continues to give poor answers.

Yeah, because 99.99% of the time when it sees words they are not written in the way. It's true that the model can just about figure out how to break a word up into characters, but it has to work hard at that and seemingly doesn't have many layers left for completing the actual task.

I would expect that a model trained with single character tokens would do far better at these word games (wordle, hangman, etc.) at the cost of being worse at almost everything else.

2