Trjtdtk

You're three for three

Can somebody explain Brexit in a few child-proof sentences?

Do these cracks on my tires look bad?

Is it possible to build a CPA Secure encryption scheme which remains secure even when the encryption of secret key is given?

What should I use for Mishna study?

What is the opposite of 'gravitas'?

General topology proving something for all of its points

QGIS Geometry Generator Line Type

When is separating the total wavefunction into a space part and a spin part possible?

Simulating a probability of 1 of 2^N with less than N random bits

Bob has never been a M before

Freedom of speech and where it applies

Partial sums of primes

Why does this part of the Space Shuttle launch pad seem to be floating in air?

Are taller landing gear bad for aircraft, particulary large airliners?

Adding empty element to declared container without declaring type of element

Can a Bard use an arcane focus?

How did Monica know how to operate Carol's "designer"?

Can I Retrieve Email Addresses from BCC?

Is there an Impartial Brexit Deal comparison site?

I'm in charge of equipment buying but no one's ever happy with what I choose. How to fix this?

Identify a stage play about a VR experience in which participants are encouraged to simulate performing horrific activities

The verb "to prioritize"

Teaching indefinite integrals

Gradient computation in neural networks

Adjusting weights in an convolutional neural networkShould weights on earlier layers change less than weights on later layers in a neural networkEliminate input in gradient by clever choosing of cost function in neural networksNeural networks - adjusting weightsHow to implement gradient descent for a tanh() activation function for a single layer perceptron?Vanishing Gradient in a shallow networkBackpropagation with multiple different activation functionshow to optimize the weights of a neural net when feeding it with multiple training samples?How does Gradient Descent and Backpropagation work together?Neural Networks - Back Propogation

0

$begingroup$

I am working on understanding gradient computation in neural networks. But there is an issue with my computation. Let the weights between input (X) and hidden layer between $W_ij$ and hidden layer and output be $W_jk$. $g1$ be the activation between hidden layer and input and linear activation between hidden layer and output. Then the gradient of error wrt to $W_ij$ comes out to be $W_jk*derivative(g1(w_ij* X))X$. Now lets suppose we have 4 inputs, 3 hidden nodes and 1 output. The gradient comes out to be of shape 1x4. But it should be 3x4 to update the weights as per gradient descent rule.

neural-network gradient-descent

share|improve this question

edited Mar 20 at 16:56

shaifali Gupta

asked Mar 20 at 14:34

shaifali Guptashaifali Gupta

769

$endgroup$

add a comment | 

0

$begingroup$

I am working on understanding gradient computation in neural networks. But there is an issue with my computation. Let the weights between input (X) and hidden layer between $W_ij$ and hidden layer and output be $W_jk$. $g1$ be the activation between hidden layer and input and linear activation between hidden layer and output. Then the gradient of error wrt to $W_ij$ comes out to be $W_jk*derivative(g1(w_ij* X))X$. Now lets suppose we have 4 inputs, 3 hidden nodes and 1 output. The gradient comes out to be of shape 1x4. But it should be 3x4 to update the weights as per gradient descent rule.

neural-network gradient-descent

share|improve this question

edited Mar 20 at 16:56

shaifali Gupta

asked Mar 20 at 14:34

shaifali Guptashaifali Gupta

769

$endgroup$

add a comment | 

$begingroup$

I am working on understanding gradient computation in neural networks. But there is an issue with my computation. Let the weights between input (X) and hidden layer between $W_ij$ and hidden layer and output be $W_jk$. $g1$ be the activation between hidden layer and input and linear activation between hidden layer and output. Then the gradient of error wrt to $W_ij$ comes out to be $W_jk*derivative(g1(w_ij* X))X$. Now lets suppose we have 4 inputs, 3 hidden nodes and 1 output. The gradient comes out to be of shape 1x4. But it should be 3x4 to update the weights as per gradient descent rule.

neural-network gradient-descent

share|improve this question

edited Mar 20 at 16:56

shaifali Gupta

asked Mar 20 at 14:34

shaifali Guptashaifali Gupta

769

$endgroup$

share|improve this question

edited Mar 20 at 16:56

shaifali Gupta

asked Mar 20 at 14:34

shaifali Guptashaifali Gupta

769

share|improve this question

edited Mar 20 at 16:56

shaifali Gupta

asked Mar 20 at 14:34

shaifali Guptashaifali Gupta

769

asked Mar 20 at 14:34

shaifali Guptashaifali Gupta

769

asked Mar 20 at 14:34

shaifali Guptashaifali Gupta

769