Trjtdtk

So I've drawn a neural network diagram below:

where $x_1, x_2,ldots,x_m$ are the input layer, $h_1, h_2$ are the hidden layer and $hat y_1, hat y_2,ldots hat y_k$ are the output layer. In the $W^h_im$ notation, it represents the weight, where $i$ is representing to which node it's pointing to in the hidden layer, which is either $h_1$ or $h_2$ and $m$ is representing from which input node.

For example, if $W^h_11$, this means it is the weight represented by the line connecting $x_1$ node to $h_1$ node. $W^o_ki$, where $k$ is the output node and $i$ is the hidden layer node. Therefore, $W^o_11$ is the arrow connecting $h_1$ node with $hat y_1$ node.

I'm currently working on the back propagation process of updating the gradient descent equation for $W^h_11$. Before this, I've only learnt to deal with a diagram with only one output node, but now it's multiple output nodes instead.

So I've attempted the beginning part of the calculus, where I'm not entirely sure if $hat y$ equation i wrote below is correct and also for the loss function?

For $W^h_11$:

$Input: (x,y)$

$haty = forward(x)$

$haty = sum ^K_k=1 h_1W^o_k1 + sum ^K_k=1 h_2W^o_k2$ *am i doing this correctly?

$h_1 = sigma(barh_1)$ where $barh_1 = sum_j=1^M x_jW^h_1j$

$Loss function: J_t(w) = frac12sum(haty-y)^2$ where $hat y$ is a vector of $hat y_1,hat y_2,..hat y_k$ *would the loss function be included with a summation?

Would like to know if I've done the beginning part right.

We have $$haty_colorbluek=h_1W_k1^o + h_2W_k2^o$$

If we let $haty = (haty_1, ldots, haty_K)^T$, $W_1^o=(W_11, ldots, W_K1)^T$, and $W_2^o=(W_12, ldots, W_K2)^T$

Then we have $$haty=h_1W_1^o+h_2W_2^o$$

I believe you are performing a regression, $$J(w) = frac12 |haty-y|^2=frac12sum_k=1^K(haty_k-y_k)^2$$

It is possible to weight individual term as well depending on applications.