Why increasing the number of units or layers does not increase the accuracy and decrease the loss?Neural Network - Adjust number of hidden layers and neuronsWhy is my loss so high?Neural Network accuracy and loss guarantees?Regularization - Combine drop out with early stoppingLoss for CNN decreases and settles but training accuracy does not improveWhat to do if training loss decreases but validation loss does not decrease?train Neural Network with SGD and see that it overfits data.How can I increase the number of iterations per epoch in MATLAB?Is there any standard or normal range for the amount of LSTM loss function?Why do people use CrossEntropyLoss and not just a softmax probability as the loss?
Variable completely messes up echoed string
Would it be believable to defy demographics in a story?
Print a physical multiplication table
How to generate binary array whose elements with values 1 are randomly drawn
Deletion of copy-ctor & copy-assignment - public, private or protected?
How to get the n-th line after a grepped one?
Is there a term for accumulated dirt on the outside of your hands and feet?
What does Jesus mean regarding "Raca," and "you fool?" - is he contrasting them?
Generic TVP tradeoffs?
A Ri-diddley-iley Riddle
World War I as a war of liberals against authoritarians?
Are dual Irish/British citizens bound by the 90/180 day rule when travelling in the EU after Brexit?
Comment Box for Substitution Method of Integrals
How to define limit operations in general topological spaces? Are nets able to do this?
Do I need to be arrogant to get ahead?
How can an organ that provides biological immortality be unable to regenerate?
What (if any) is the reason to buy in small local stores?
Do US professors/group leaders only get a salary, but no group budget?
Optimising a list searching algorithm
What does "Four-F." mean?
Knife as defense against stray dogs
Practical application of matrices and determinants
Wrapping homogeneous Python objects
Is honey really a supersaturated solution? Does heating to un-crystalize redissolve it or melt it?
Why increasing the number of units or layers does not increase the accuracy and decrease the loss?
Neural Network - Adjust number of hidden layers and neuronsWhy is my loss so high?Neural Network accuracy and loss guarantees?Regularization - Combine drop out with early stoppingLoss for CNN decreases and settles but training accuracy does not improveWhat to do if training loss decreases but validation loss does not decrease?train Neural Network with SGD and see that it overfits data.How can I increase the number of iterations per epoch in MATLAB?Is there any standard or normal range for the amount of LSTM loss function?Why do people use CrossEntropyLoss and not just a softmax probability as the loss?
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
add a comment |
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
edited 2 days ago
Media
7,35062161
edited 2 days ago
Media
7,35062161
edited 2 days ago
Media
7,35062161
7,35062161
asked 2 days ago
user145959user145959
1268
asked 2 days ago
user145959user145959
1268
asked 2 days ago
user145959user145959
1268
1268
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47362%2fwhy-increasing-the-number-of-units-or-layers-does-not-increase-the-accuracy-and%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
edited 2 days ago
answered 2 days ago
MediaMedia
7,35062161
answered 2 days ago
MediaMedia
7,35062161
answered 2 days ago
MediaMedia
7,35062161
7,35062161
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47362%2fwhy-increasing-the-number-of-units-or-layers-does-not-increase-the-accuracy-and%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago