Trjtdtk

Is a tag line useful on a cover?

tikz: show 0 at the axis origin

I’m planning on buying a laser printer but concerned about the life cycle of toner in the machine

Finding angle with pure Geometry.

Dragon forelimb placement

Modeling an IP Address

Why not use SQL instead of GraphQL?

How do we improve the relationship with a client software team that performs poorly and is becoming less collaborative?

Theorems that impeded progress

Why do I get two different answers for this counting problem?

What does it mean to describe someone as a butt steak?

Can an x86 CPU running in real mode be considered to be basically an 8086 CPU?

Is it important to consider tone, melody, and musical form while writing a song?

Is a conference paper whose proceedings will be published in IEEE Xplore counted as a publication?

Watching something be written to a file live with tail

What are these boxed doors outside store fronts in New York?

Fencing style for blades that can attack from a distance

LaTeX closing $ signs makes cursor jump

How to find program name(s) of an installed package?

String Manipulation Interpreter

What do the dots in this tr command do: tr .............A-Z A-ZA-Z <<< "JVPQBOV" (with 13 dots)

How can I prevent hyper evolved versions of regular creatures from wiping out their cousins?

Is it possible to do 50 km distance without any previous training?

Arthur Somervell: 1000 Exercises - Meaning of this notation

My question is about dependency between hidden states for Back Propagation Through Time in RNN

0

$begingroup$

In one video lecture, professor Ali Ghodsi of University of Waterloo says that the first node of S(t)(hidden state of RNN at time t) has an effect only on the first node of S(t+1)(hidden state of RNN at time t+1), the second node of S(t) has an effect only on the second node of S(t+1), and so on. I do not see how that is correct cause the weight matrix W connecting the state at one time step to the state at the next time step is (p,p) dimensional which means each node of S(t) is connected to every node of S(t+1). What I have come to understand from various blogs on the internet about RNN is that nodes in the hidden state of some time step are connected to every node in the hidden state corresponding to the next time step which is contradictory to what professor Ghodsi is stating in the lecture.

What I want to clarify is whether the hidden nodes in RNN are usually connected in the same way as fully connected networks throughout the temporal dimension, or are they connected elementwise as the professor says.

I am providing a link to the video https://www.youtube.com/watch?v=AvyhbrQptHk&t=1704s where the claim is made a 43:02.

I had posted this question on StackOverflow, but that doesn't seem to be the correct platform for such topics.

I am new to this site. So, please be a little patient if you do not like the format of the question and let me know how I can improve the quality of my question.

Thanks in advance.

backpropagation recurrent-neural-net

share|improve this question

edited Mar 27 at 10:33

Sajid Ahmed

asked Mar 27 at 10:09

Sajid AhmedSajid Ahmed

314

$endgroup$

add a comment | 

0

$begingroup$

In one video lecture, professor Ali Ghodsi of University of Waterloo says that the first node of S(t)(hidden state of RNN at time t) has an effect only on the first node of S(t+1)(hidden state of RNN at time t+1), the second node of S(t) has an effect only on the second node of S(t+1), and so on. I do not see how that is correct cause the weight matrix W connecting the state at one time step to the state at the next time step is (p,p) dimensional which means each node of S(t) is connected to every node of S(t+1). What I have come to understand from various blogs on the internet about RNN is that nodes in the hidden state of some time step are connected to every node in the hidden state corresponding to the next time step which is contradictory to what professor Ghodsi is stating in the lecture.

What I want to clarify is whether the hidden nodes in RNN are usually connected in the same way as fully connected networks throughout the temporal dimension, or are they connected elementwise as the professor says.

I am providing a link to the video https://www.youtube.com/watch?v=AvyhbrQptHk&t=1704s where the claim is made a 43:02.

I had posted this question on StackOverflow, but that doesn't seem to be the correct platform for such topics.

I am new to this site. So, please be a little patient if you do not like the format of the question and let me know how I can improve the quality of my question.

Thanks in advance.

backpropagation recurrent-neural-net

share|improve this question

edited Mar 27 at 10:33

Sajid Ahmed

asked Mar 27 at 10:09

Sajid AhmedSajid Ahmed

314

$endgroup$

add a comment | 

$begingroup$

In one video lecture, professor Ali Ghodsi of University of Waterloo says that the first node of S(t)(hidden state of RNN at time t) has an effect only on the first node of S(t+1)(hidden state of RNN at time t+1), the second node of S(t) has an effect only on the second node of S(t+1), and so on. I do not see how that is correct cause the weight matrix W connecting the state at one time step to the state at the next time step is (p,p) dimensional which means each node of S(t) is connected to every node of S(t+1). What I have come to understand from various blogs on the internet about RNN is that nodes in the hidden state of some time step are connected to every node in the hidden state corresponding to the next time step which is contradictory to what professor Ghodsi is stating in the lecture.

What I want to clarify is whether the hidden nodes in RNN are usually connected in the same way as fully connected networks throughout the temporal dimension, or are they connected elementwise as the professor says.

I am providing a link to the video https://www.youtube.com/watch?v=AvyhbrQptHk&t=1704s where the claim is made a 43:02.

I had posted this question on StackOverflow, but that doesn't seem to be the correct platform for such topics.

I am new to this site. So, please be a little patient if you do not like the format of the question and let me know how I can improve the quality of my question.

Thanks in advance.

backpropagation recurrent-neural-net

share|improve this question

edited Mar 27 at 10:33

Sajid Ahmed

asked Mar 27 at 10:09

Sajid AhmedSajid Ahmed

314

$endgroup$

share|improve this question

edited Mar 27 at 10:33

Sajid Ahmed

asked Mar 27 at 10:09

Sajid AhmedSajid Ahmed

314

share|improve this question

edited Mar 27 at 10:33

Sajid Ahmed

asked Mar 27 at 10:09

Sajid AhmedSajid Ahmed

314

asked Mar 27 at 10:09

Sajid AhmedSajid Ahmed

314

asked Mar 27 at 10:09

Sajid AhmedSajid Ahmed

314