I can add it!

I hope this fails spectacularly

This should fail since the original work is not being redistributed. To wholly recreate a repo on which Codex was trained you’d have to literally start typing the original code, and even then the contextual suggestions would likely yield a different result from the original anyways. I could be mistaken but I remember reading about some litigation in this space concerning a model trained on copyrighted data. The court ruled in favor of the defendant because the resulting model couldn’t possibly reproduce the original work. It’s tricker here because technically you could recreate the original work, but you would have to know very well what the original work was to begin with to actually recreate it, and if that’s the case what’s the point of using copilot to begin with. I could be (and probably am) wrong.

Imagine trying to recreate PyTorch from scratch using Codex or copilot. IF, and that’s a big if, one did so the author of the recreation would still have to attribute it.

Not legal advice

I believe there could be other models which will fall in this category, image generation models specially.

This particular lawsuit is interesting as it says it violates github's own terms.

Is sentence rewrite available?

As others have said, you don't have enough examples to train a stylegan, but you can retrain Stable Diffusion on that few examples. That'll allow you to render anything you can imagine in your style too for example.

I am financing it. Free for everyone else.

How is it free? I thought they charged per token...

Are you going to pay for it?

Well I couldn't understand what your task was when you didn't say what it was until now.

Other than that, skimming through the paper it quite clearly says the following:

> Our present results do not indicate our procedure can generalize to motifs that are not present in the training set

Because what they're doing doesn't generalize, I think the starting assumptions (that there will be imprevements with a larger model) are wrong, and so the question is unnecessary... The issue is with the method or the data, they do not elaborate more than that.

I mean the dataset is extremely diverse. like millions clusters and every entry is noised when loaded on GPUs

[deleted]

That seems very bad. You want your train-dev-test to be different samples of the same distribution, so, not very different sets.

Furthermore, if you're using test for model validation, that means you will have no dataset to finally evaluate your model on. Reconsider your process.

Finally, again, I urge you to evaluate your dataset on an established evaluation metric for the task, not the loss you use to train the model. What is the exact task?

My best guess is coding mistake on your part. Good luck tivo

But DO will prevent overfitting, I don't have any overfitting it's not the relevant tool

The split has been done such as the train and test are highly different. the loss are almost equal on both datasets.

Well, clearly it’s not getting any better with what you’re trying. Maybe time to rethink

The model is equivariant no dataset augmentation, no DO as well. The model doesn't overfit as I said

Data augmentation, dropout?

It can be seen as an approximation of the variance between the noise and the noise predicted conditioned on some data.

If it's on the training set it is not even usable as a metric, and if it is not directly related to the performance it is not a good metric. You want to see how it acts on unseen data.

The loss here is for a denoiser, it can be seen as the variance between the noise and the noise predicted. So it's in this case a good metric

Loss doesn't matter, what are the validation metrics?

Hey everyone! We released this open-source tool for tracking and comparing embedding experiments, which we regularly do.

Feel free to give it a try! Getting started is easy: https://github.com/pentoai/vectory

[deleted]

The code is freely available and on Youtube you find videos on „How to use Stylegan“.

But:

• As already mentioned you‘ll need more data. 5-10k are already at the low end. You need more like 50-100k I think.
• You need some serious hardware to train something like this. For Stylegan they used 8 high end GPUs for a week. I guess for Stylegan2/3 even more. You might be happy with less than NVIDIA, but it‘s no quick thing to do.
• And even then there‘s no guarantee that it will turn out perfectly. It might need some fine-tuning, it‘s not a simple algorithm.