Recent comments in /f/deeplearning

MIKOLAJslippers t1_izy9s51 wrote

So you will need to implement that maths in your chosen language (easiest would be python and numpy as the syntax is almost the same as I shared). That’s the forward pass from inputs to outputs. You will also need to initialise the weight matrices w1 and w2 to something. Do you have any pretrained weights you can test it with? You may also need to add biases after the matmuls depending on the brief. Usually the case but not necessarily essential to make it train.

Presumably you will also need to then train your network so it’ll get a bit more tricky. You’ll need to implement a loss function based on error between the outputs and some target variable. Once you have the loss you can then use chain rule back through the network to get the delta w (weight gradients) for each weight (w1 and w2 and also any biases if you add those). You’ll then update your weights using some update rule which is usually just multiplying the weight gradients by the learning rate (usually denoted alpha).

Is any of this helpful? Which bit do you still not understand?

1

MIKOLAJslippers t1_izy3vs3 wrote

I don’t think anyone really wants to do your homework for you. Because is seems unlikely you weren’t told how to do this is some form

What’s your starting point here? Are you using python? Do you get numpy?

The maths you want is something like:

acts = act(in X w1)

To get hidden activation where act is your activation function, X is a matrix multiply and w1 has dims of len(in) by hidden size.

Then you do:

out = act(acts X w2)

Where w2 has dims len(acts) by output dims

3

dragnil_ t1_izxnlbo wrote

CS231n(2016) : https://youtube.com/playlist?list=PLkt2uSq6rBVctENoVBg1TpCC7OQi31AlC

Updated and Expanded version of above: Michigan Deep learning for computer vision(2019): https://youtube.com/playlist?list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r

You can find the respective course website using google search. From there you can find the assignments and suggested readings for both of the above courses.

Well, I am not an expert on the subject, but per my experience you have to learn things when you need them. If you think you have to fulfill all the prerequisites before diving into deep learning, you will be stacking different kinds of courses above the two mentioned above. There will be a long list of courses before you are actually starting deep learning. So, just start, go through the readings, assignments and when stuck ask questions, search on Google, see parts of some other courses that are actually relevant.

1

crimson1206 t1_izxfkya wrote

To correct the other comment all the examples they mentioned are from linear algebra, though calculus is important too.

To understand the deep learning you’ll need linear algebra and multidimensional calculus. For some parts of deep learning you’ll also need probability & statistics knowledge.

1

boosandy t1_izx7j90 wrote

Andrew Ng's course is the best out there beginners. He dilutes the complexities and slowly teaches all the math. You don't even need any book. That being said, his cpurse is based on Tensorflow. I would suggest learning pytorch on the side.

1