Why do recurrent layers work better than simple feed-forward networks? Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsBatching in Recurrent Neural Networks (RNNs) when there is only a single instance per time step?Is it necessary to perform rolling-window on LSTMs?Neural net layer that preserves spatial informationWhat principle is behind semantic segmenation with CNNs?What are some methodologies for performing feature selection for simple feed-forward neural networks?1d time series to time series approximation using deep learningBest practices to modelize top layers over CNNUnderstanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTTWhat is exactly meant by neural network that can take different types of input?Understanding LSTM structure
How many times is it recorded in scripture - instances where Jesus ate a meal?
Generate an RGB colour grid
If windows 7 doesn't support WSL, then what does Linux subsystem option mean?
What font is "z" in "z-score"?
How do I make this wiring inside cabinet safer? (Pic)
Why did Roosevelt decide to implement a maximum wage through taxation rather than a simple ceiling?
Converter for RJ45 port to SFP+
NumericArray versus PackedArray in MMA12
Do I really need to have a message in a novel to appeal to readers?
Can anything be seen from the center of the Boötes void? How dark would it be?
If my PI received research grants from a company to be able to pay my postdoc salary, did I have a potential conflict interest too?
How to compare two different files line by line in unix?
How to convince students of the implication truth values?
Is it possible to add Lighting Web Component in the Visual force Page?
How often does castling occur in grandmaster games?
What is the meaning of the simile “quick as silk”?
How would a mousetrap for use in space work?
Catmull-Clark and Doo-Sabin Subdivision Codes
Are there mentions in Hinduism about instruments which allows one to know others thoughts and influence them? And is it sinful?
Amount of permutations on an NxNxN Rubik's Cube
Using audio cues to encourage good posture
2001: A Space Odyssey's use of the song "Daisy Bell" (Bicycle Built for Two); life imitates art or vice-versa?
AppleTVs create a chatty alternate WiFi network
How do I use the new nonlinear finite element in Mathematica 12 for this equation?
Why do recurrent layers work better than simple feed-forward networks?
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsBatching in Recurrent Neural Networks (RNNs) when there is only a single instance per time step?Is it necessary to perform rolling-window on LSTMs?Neural net layer that preserves spatial informationWhat principle is behind semantic segmenation with CNNs?What are some methodologies for performing feature selection for simple feed-forward neural networks?1d time series to time series approximation using deep learningBest practices to modelize top layers over CNNUnderstanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTTWhat is exactly meant by neural network that can take different types of input?Understanding LSTM structure
$begingroup$
On a time series problem that we try to solve using RNNs, the input usually has the shape $input features times timesteps times batchsize$ and we then feed this input into recurrent layers. An alternative would be to flatten the data so that the shape is $(input features times timesteps) times batchsize$ and use a fully connected layer for our time series task. This would clearly work and our dense network would be able to find dependencies between the data at different timesteps as well. So what is it that makes recurrent layers more powerful? I would be very thankful for an intuitive explanation.
machine-learning neural-network deep-learning lstm rnn
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
asked Mar 3 at 5:20
ZuberaZubera
61
$endgroup$
add a comment |
$begingroup$
On a time series problem that we try to solve using RNNs, the input usually has the shape $input features times timesteps times batchsize$ and we then feed this input into recurrent layers. An alternative would be to flatten the data so that the shape is $(input features times timesteps) times batchsize$ and use a fully connected layer for our time series task. This would clearly work and our dense network would be able to find dependencies between the data at different timesteps as well. So what is it that makes recurrent layers more powerful? I would be very thankful for an intuitive explanation.
machine-learning neural-network deep-learning lstm rnn
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
asked Mar 3 at 5:20
ZuberaZubera
61
$endgroup$
add a comment |
$begingroup$
On a time series problem that we try to solve using RNNs, the input usually has the shape $input features times timesteps times batchsize$ and we then feed this input into recurrent layers. An alternative would be to flatten the data so that the shape is $(input features times timesteps) times batchsize$ and use a fully connected layer for our time series task. This would clearly work and our dense network would be able to find dependencies between the data at different timesteps as well. So what is it that makes recurrent layers more powerful? I would be very thankful for an intuitive explanation.
machine-learning neural-network deep-learning lstm rnn
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
asked Mar 3 at 5:20
ZuberaZubera
61
$endgroup$
On a time series problem that we try to solve using RNNs, the input usually has the shape $input features times timesteps times batchsize$ and we then feed this input into recurrent layers. An alternative would be to flatten the data so that the shape is $(input features times timesteps) times batchsize$ and use a fully connected layer for our time series task. This would clearly work and our dense network would be able to find dependencies between the data at different timesteps as well. So what is it that makes recurrent layers more powerful? I would be very thankful for an intuitive explanation.
machine-learning neural-network deep-learning lstm rnn
machine-learning neural-network deep-learning lstm rnn
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
asked Mar 3 at 5:20
ZuberaZubera
61
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
asked Mar 3 at 5:20
ZuberaZubera
61
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
edited Mar 3 at 7:47
Vaalizaadeh
7,60562264
7,60562264
asked Mar 3 at 5:20
ZuberaZubera
61
asked Mar 3 at 5:20
ZuberaZubera
61
asked Mar 3 at 5:20
ZuberaZubera
61
61
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
The first reason is the number of parameters. The former case that you've mentioned, for each neuron there should be corresponding entries that would increase the number of training parameters. The other reason is that by employing simple feed-forward neurons you are somehow discarding the temporal information of your data which means you are discarding the sequence information in your data. This is somehow like the spatial data which is obtained by convolutional layers in CNNs.
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46559%2fwhy-do-recurrent-layers-work-better-than-simple-feed-forward-networks%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The first reason is the number of parameters. The former case that you've mentioned, for each neuron there should be corresponding entries that would increase the number of training parameters. The other reason is that by employing simple feed-forward neurons you are somehow discarding the temporal information of your data which means you are discarding the sequence information in your data. This is somehow like the spatial data which is obtained by convolutional layers in CNNs.
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
$endgroup$
add a comment |
$begingroup$
The first reason is the number of parameters. The former case that you've mentioned, for each neuron there should be corresponding entries that would increase the number of training parameters. The other reason is that by employing simple feed-forward neurons you are somehow discarding the temporal information of your data which means you are discarding the sequence information in your data. This is somehow like the spatial data which is obtained by convolutional layers in CNNs.
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
$endgroup$
add a comment |
$begingroup$
The first reason is the number of parameters. The former case that you've mentioned, for each neuron there should be corresponding entries that would increase the number of training parameters. The other reason is that by employing simple feed-forward neurons you are somehow discarding the temporal information of your data which means you are discarding the sequence information in your data. This is somehow like the spatial data which is obtained by convolutional layers in CNNs.
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
$endgroup$
The first reason is the number of parameters. The former case that you've mentioned, for each neuron there should be corresponding entries that would increase the number of training parameters. The other reason is that by employing simple feed-forward neurons you are somehow discarding the temporal information of your data which means you are discarding the sequence information in your data. This is somehow like the spatial data which is obtained by convolutional layers in CNNs.
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
answered Mar 3 at 7:45
VaalizaadehVaalizaadeh
7,60562264
7,60562264
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46559%2fwhy-do-recurrent-layers-work-better-than-simple-feed-forward-networks%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
