Recent comments in /f/MachineLearning
Oceanboi t1_jd6g49h wrote
Reply to [Project] Machine Learning for Audio: A library for audio analysis, feature extraction, etc by Leo_D517
How do these handmade features compare to features identified by CNNs? Only reason I ask is that I'm finishing up some thesis work on sound event detection using different spectral representations as inputs to CNNs (Cochleagram, Linear Gammachirp, Logarithmic Gammachirp, Approximate Gammatone filters, etc). Wondering how these features perform in comparison on similar tasks (UrbanSound8K) and where it fits in the larger scheme of things.
ZeryeZ t1_jd6fdt7 wrote
Which AI did you use and is there some way so I could try it myself?
Qwillbehr OP t1_jd6baxv wrote
Reply to comment by DB6135 in [D] Running an LLM on "low" compute power machines? by Qwillbehr
I played with it for a few minutes and noticed that the 16B alpaca model gave significantly better responses. From what I can tell though the issue seems to be in how dalai prompts alpaca.CPP (just tells it to complete the sentence with all possible outputs rather then just one of the possible answers). The 16B model fixed most of it for me
Suitable_Goose3637 t1_jd69ze2 wrote
Reply to comment by Fabulous-Possible758 in [P] Anyone interested in starting a Startup? by [deleted]
Not focused on that. I work in the film and television industry and I see a need for something in that market that AI can help with, but I got downvoted to hell and will look elsewhere. It wasn’t really a need to find people on Reddit, just exploring it as an option for trying to do things in a more cost effective way so we don’t burn through our capital so quickly. Instead I think we’re going to go with this firm in NYC that is giving us a quote on our project, just wanted to explore another option…clearly that wasn’t a good idea.
Fabulous-Possible758 t1_jd69ia0 wrote
Reply to comment by Suitable_Goose3637 in [P] Anyone interested in starting a Startup? by [deleted]
You might have better luck in the r/StableDiffusion, r/Midjourney, or r/AIGrinding if you’re looking for help with things related to image production. This sub tends to be a little more technical and broadly focused on ML in general.
Edit: And yeah, sorry you need to work on your pitch a little bit. Programmers get pitched about a thousand ideas a minute from people who think they have “the next big app.” We also have day jobs so putting a lot into someone else’s project has to come with some concrete ideas and promises.
Suitable_Goose3637 t1_jd68vr0 wrote
Reply to comment by Fabulous-Possible758 in [P] Anyone interested in starting a Startup? by [deleted]
We work on high end production in the film and tv space.
DB6135 t1_jd68vdi wrote
Reply to comment by not_particulary in [D] Running an LLM on "low" compute power machines? by Qwillbehr
What are the recommended parameters? I tried the 7B model with default settings but it kept generating repeated garbage output.
kilow4tt t1_jd68jnd wrote
Reply to [R] SPDF - Sparse Pre-training and Dense Fine-tuning for Large Language Models by CS-fan-101
Was there any effort to go from 75% sparsity during pre-training to a less sparse (e.g. 25%) sparsity during fine-tuning rather than strictly going from 75% sparsity to 0%?
sanxiyn t1_jd68827 wrote
You don't need leaked LLaMA weight. ChatGLM-6B weight is being distributed by the first party.
Suitable_Goose3637 t1_jd67s5i wrote
Reply to comment by Username912773 in [P] Anyone interested in starting a Startup? by [deleted]
Without doxing myself, I work in Television/Film/Advertising in Los Angeles.
disastorm t1_jd66swu wrote
Reply to comment by trnka in [D] Simple Questions Thread by AutoModerator
I see thanks, is that basically the equivallent of having "top_k" = 1?
Can you explain what these mean. From what I understand top_k means it considers the top K number of possible words at each step.
I can't exactly understand what top_p means, can they be use together?
aozorahime t1_jd66laf wrote
Reply to comment by 1azytux in Recent advances in multimodal models: What are your thoughts on chain of thoughts models? [D] by 1azytux
Ah, CoT. I think I have heard about this topic somewhere but forgot it already. yes it is similar to my next research proposal for Ph.D. regarding Visual QnA in mathematical reasoning. Thank you for the recommendation papers!
​
About the link that I mentioned, well it is just like a brief explanation of what people doing so far in terms of multimodal deep learning, the model, benchmark, dataset, etc. Since I get exposed with overwhelmed information about the current models, I think I need to look for this (just finished reading the NLP part).
​
Have you worked with multimodal before?
Fabulous-Possible758 t1_jd65m2u wrote
“Earlier this year, I formed an LLC with a long-time friend who shares the same expertise …”
Which is what, exactly?
Suitable_Goose3637 t1_jd652gm wrote
Reply to comment by PassionatePossum in [P] Anyone interested in starting a Startup? by [deleted]
My thoughts are that we would scale with funding. I’m doing this currently on the side while I do my normal job. The thought I had was if I were to start building the team now, it would be easier to go to investors. Right now it’s just my board of advisors and my co-founder. That’s not enough to move the needle in my opinion. I understand your concern and I thought this would be an issue. That’s why I’m also meeting with a company that already has a team in place to do this. So technically this post is a second option I thought would be interesting to explore. I can see it from your perspective that everyone that comes through here is all talk no action…but sometimes there are exemptions to that rule.
RainbowRedditForum t1_jd64o4e wrote
Reply to [D] Simple Questions Thread by AutoModerator
A CRNN is trained with logmel as input, calculated as follows:
the input audio is split in 30ms frames with 10ms hop size, and 40 logmel are calculated for each frame.
The CRNN performs a binary classification.
With this setup, are these two considerations true?
- two consecutive output labels generated by the CRNN are associated with two overlapped audio frames (each of size 30ms (0.03s) and hop size 10ms);
- for 10 minutes audio the CRNN should generate about 30000 output labels, each one associated with a 30ms frame with 10ms of overlap
CS-fan-101 OP t1_jd649ko wrote
Reply to comment by geneing in [R] SPDF - Sparse Pre-training and Dense Fine-tuning for Large Language Models by CS-fan-101
I wouldn't call it a workaround but rather an advantage.
Neural network models are made up of layers of neurons and connections between them. When there are missing connections, represented as zeros in the weight matrices, we refer to the model as sparse.
Sparsity comes in different forms. It is common for sparsity to occur naturally in the model structure itself if the pattern of connections is designed to only connect a subset of the neurons. Often, models are constructed this way intentionally with a predefined pattern and we refer to this as structured sparsity.
It turns out that even fully dense models, such as GPT, can be made sparse by inducing unstructured sparsity. In this form of sparsity, certain weights are set to zero, which effectively prunes the connections within the model. When the pruning is done without a fixed pattern, we refer to this as unstructured sparsity.
A key benefit of unstructured sparsity is the model retains the original baseline structure, without the need to create a new model architecture. Additionally, the sparse model can provide speedup in both training and inference.
The Cerebras CS-2 is designed to accelerate unstructured sparsity, whereas GPUs are not.
If you are interested in learning more, please check out our blog - https://www.cerebras.net/blog/harnessing-the-power-of-sparsity-for-large-gpt-ai-models
PassionatePossum t1_jd62upb wrote
I don't think you could have more red flags in a job posting if you tried: No technical know-how but a big unspecified vision. No employees. I guess that means no salary either. "Access to high-net worth individuals". So no funding, yet?
Sounds like you have nothing. No funding, no team, no idea whether what you are doing is technically feasible or how much it would cost or what resources you would need. And additionally you want people to sign up for "something" and assume all the financial risk themselves. Does that sound about right? It is basically the meme of the "ideas guy".
Nowado t1_jd62sxt wrote
Reply to comment by CheekProfessional146 in [Project] Machine Learning for Audio: A library for audio analysis, feature extraction, etc by Leo_D517
I'd recommend mixing wording a bit, maybe some older accounts?
Username912773 t1_jd60z75 wrote
Reply to comment by Suitable_Goose3637 in [P] Anyone interested in starting a Startup? by [deleted]
It seems you frequent r/promptart, r/midjourney and even r/aliens and r/collapse, all of this is to ask what field do you specialize in exactly? You say you work in advertising and marketing, but how does that translate to business experience? What expertise do you and your friend offer that differ from each other? Do you share the exact same expertise? You said you’ve worked with Hollywood productions before, how is this related to your startup? You use a lot of buzzwords and phrases such as “rise from the ashes” you also claim to have connections with “high net worth individuals” which is incredibly vague. This whole post just seems like you don’t necessarily have a plan.
geneing t1_jd6016n wrote
Reply to [R] SPDF - Sparse Pre-training and Dense Fine-tuning for Large Language Models by CS-fan-101
Is this a workaround for the weird Cerebras chip architecture? Would mainstream users who train on GPU benefit?
Suitable_Goose3637 t1_jd5zne1 wrote
Reply to comment by Username912773 in [P] Anyone interested in starting a Startup? by [deleted]
Absolutely, but that information is only available for people who are willing to sign an NDA.
Username912773 t1_jd5zjaz wrote
What exactly do you plan to do? This is all well and good, but do you actually have an idea for a unique product and a plan to market it?
cocochoco123 t1_jd5ysyi wrote
Hey I have vast experience in the UI/UX space. Do you have discord so we can discuss further?
keeplosingmypws t1_jd5xygm wrote
Reply to comment by KerfuffleV2 in [Project] Alpaca-30B: Facebook's 30b parameter LLaMa fine-tuned on the Alpaca dataset by imgonnarelph
I have the 16B parameter version of Alpaca.cpp (and a copy of the training data as well as the weights) installed locally on a machine with an Nvidia 3070 GPU. I know I can launch my terminal using the Discrete Graphics Card option, but I also believe this version was built for CPU use and I’m guessing that I’m not getting the most out of my graphics card
What’s the move here?
Traditional-Ad-8715 OP t1_jd6hd6i wrote
Reply to comment by ZeryeZ in [Project] AI Voice Narrated Audiobooks by Traditional-Ad-8715
Name of AI is called Elevenlabs, free to try