Trjtdtk

How to manage monthly salary

Is there any use for defining additional entity types in a SOQL FROM clause?

Why did the Germans forbid the possession of pet pigeons in Rostov-on-Don in 1941?

Is it wise to hold on to stock that has plummeted and then stabilized?

What does 'script /dev/null' do?

Is it legal to have the "// (c) 2019 John Smith" header in all files when there are hundreds of contributors?

What to wear for invited talk in Canada

"listening to me about as much as you're listening to this pole here"

Why doesn't a const reference extend the life of a temporary object passed via a function?

How to make payment on the internet without leaving a money trail?

Manga about a female worker who got dragged into another world together with this high school girl and she was just told she's not needed anymore

When blogging recipes, how can I support both readers who want the narrative/journey and ones who want the printer-friendly recipe?

What is the command to reset a PC without deleting any files

Is domain driven design an anti-SQL pattern?

Patience, young "Padovan"

Shall I use personal or official e-mail account when registering to external websites for work purpose?

Why is my log file so massive? 22gb. I am running log backups

Email Account under attack (really) - anything I can do?

Find the number of surjections from A to B.

Where else does the Shulchan Aruch quote an authority by name?

Copycat chess is back

What are the advantages and disadvantages of running one shots compared to campaigns?

LWC and complex parameters

Landing in very high winds

Connection between piecewise linear basis functions and RELU activation function

1

$begingroup$

ReLU activation is defined as follows
$$sigma(x)=max(0, x).$$
Let's assume that I have deep network of 1 hidden layer, than output from my layer has form
$$ f(x)= sigma(Wx +b), $$
where matrix W represents the weights, b is bias and x stands for input data (output of previous layer).

What is connection between $f(x)$ and piecewise linear function, defined as
$$g(x_j)=sum_i=N^N alpha_i psi_i(x_j),$$
where $alpha_i$ stands for the coefficients and $psi_i$ for the basis function.

More precisely, I would like to identify basis functions and coefficients in the definition of $f(x)$.

deep-learning linear-algebra deep-network

share|improve this question

edited Mar 31 at 19:49

Community♦

1

asked Mar 29 at 8:37

user70435user70435

61

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  could you add more details in your question about the piecewise linear basis functions? The more details you add with what you thinking to compare, the more precise the answer you will get!
  $endgroup$
  – anu
  Mar 30 at 23:56
  

  
  
  
  

add a comment | 

1

$begingroup$

ReLU activation is defined as follows
$$sigma(x)=max(0, x).$$
Let's assume that I have deep network of 1 hidden layer, than output from my layer has form
$$ f(x)= sigma(Wx +b), $$
where matrix W represents the weights, b is bias and x stands for input data (output of previous layer).

What is connection between $f(x)$ and piecewise linear function, defined as
$$g(x_j)=sum_i=N^N alpha_i psi_i(x_j),$$
where $alpha_i$ stands for the coefficients and $psi_i$ for the basis function.

More precisely, I would like to identify basis functions and coefficients in the definition of $f(x)$.

deep-learning linear-algebra deep-network

share|improve this question

edited Mar 31 at 19:49

Community♦

1

asked Mar 29 at 8:37

user70435user70435

61

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  could you add more details in your question about the piecewise linear basis functions? The more details you add with what you thinking to compare, the more precise the answer you will get!
  $endgroup$
  – anu
  Mar 30 at 23:56
  

  
  
  
  

add a comment | 

$begingroup$

ReLU activation is defined as follows
$$sigma(x)=max(0, x).$$
Let's assume that I have deep network of 1 hidden layer, than output from my layer has form
$$ f(x)= sigma(Wx +b), $$
where matrix W represents the weights, b is bias and x stands for input data (output of previous layer).

What is connection between $f(x)$ and piecewise linear function, defined as
$$g(x_j)=sum_i=N^N alpha_i psi_i(x_j),$$
where $alpha_i$ stands for the coefficients and $psi_i$ for the basis function.

More precisely, I would like to identify basis functions and coefficients in the definition of $f(x)$.

deep-learning linear-algebra deep-network

share|improve this question

edited Mar 31 at 19:49

Community♦

1

asked Mar 29 at 8:37

user70435user70435

61

$endgroup$

share|improve this question

edited Mar 31 at 19:49

Community♦

1

asked Mar 29 at 8:37

user70435user70435

61

share|improve this question

edited Mar 31 at 19:49

Community♦

1

asked Mar 29 at 8:37

user70435user70435

61

edited Mar 31 at 19:49

Community♦

1

edited Mar 31 at 19:49

Community♦

1

asked Mar 29 at 8:37

user70435user70435

61

asked Mar 29 at 8:37

user70435user70435

61