Yes, that's why i want to pass "all the text" into bert, because for example a word in a sentence has to have similar vector with the same word (with same meaning) in another sentence. How can i accomplish that, as the max tokens number of bert is 512?

If you're talking about the contextual embeddings that BERT is known for then those change depending on the sentence used, so you need to supply the full sentence.

How does it make assumptions about the depth of those 100 points it just makes up from the 3?

Github: https://github.com/shobrook/densify

[removed]

Do you mean I need size up in this part?

def ResNet50Regression():

`Res_input = layers.Input(shape=(178,))`

# 128

`width = 128`



`x = dens_block(Res_input,width)`

`x = identity_block(x,width)`

`x = identity_block(x,width)`



`x = dens_block(x,width)`

`x = identity_block(x,width)`

`x = identity_block(x,width)`



`x = dens_block(x,width)`

`x = identity_block(x,width)`

`x = identity_block(x,width)`



`x = layers.BatchNormalization()(x)`

`x = layers.Dense(1,activation='linear')(x)`

`model = models.Model(inputs=Res_input, outputs=x)`



`return model`

If all you want to see is the two curves close to each other, I guess you could size up the model, so that it overfits terribly. But is that really desirable?

If my assumption that you predict the whole graph autoregressively is correct, then I believe it works just fine. You should check the forecast horizon and think about what it is you want to achieve in the end

then is there any suggestion for me so that the model I have made can predict properly?

No this is not a modelling issue. It actually isnt a real issue at all. Predicting a very long trajectory is simply very hard. At each timestep, a slight error will occur which will exponentiate, even if the error per timestep is marginal. Imagine being asked to predict a certain stock price. Given some expertise and current information, you might be able to do it for tomorrow, but can you do it precisely for the next year?

>I guess this is timeseries forecasting. You should think about the lookahead. Probably, during training, the model only has to predict the next point, while during testing, it has to predict many values autoregressively

what should I do? should I change the model structure or use another model?

I guess this is timeseries forecasting. You should think about the lookahead. Probably, during training, the model only has to predict the next point, while during testing, it has to predict many values autoregressively

Why do you think it's a better idea to invest in colab/sagemaker instead of buying a gpu?

I think the monthly fees over the long run will exceed a gpu price.

Use colab, if you are treating the computer like a server colab should be a lot better

oh

I agree. I was joking myself when i built myself a pc for deep learning couple years ago. Selling it off within this month to get a macbook and will invest in colab/sagemaker.

dm if any1 is lookin to buy lol

Yea this 100%

Throw a pizza party with your money and use google colab instead

Thanks for the correction!

There is no 16GB 3070, only 8 GB. The 16 GB one was a rumor.

2060 if it's the 8 GB version, otherwise 1080. But obviously the best would be to not waste your money and just buy cloud compute... With these cards being 200-300$, that is roughly 200-300 hours of A100 training, which is much faster and enables you to train much larger models.

EDIT: I see you've gotten a 110$ offer for 1080, I'd say go with that. You'll be severely limited in what you can run but after you learn your lesson you can still sell it for 100$, or even more.

While I agree with the general statement that tensor core are better to have, memory on device is a bigger limiting factor in the long run.

Depending on the model, deep learning can require a significant amount of memory for training. I would suggest you save a little more and go for a 2070 (8GB) or if you can swing it the 3070 with 8GB.

(Edited with correction on the amount of memory available for the 3070)

I got a offer 1080 for $110 so thats why 1080 was in the choice

Neither are great choices, but I wouldn't buy a 10-series for DL anymore. You're giving up a lot by not having tensor cores.

2060

currently using 1650 d6 4gb