Why increasing the number of units or layers does not increase the accuracy and decrease the loss?Neural Network - Adjust number of hidden layers and neuronsWhy is my loss so high?Neural Network accuracy and loss guarantees?Regularization - Combine drop out with early stoppingLoss for CNN decreases and settles but training accuracy does not improveWhat to do if training loss decreases but validation loss does not decrease?train Neural Network with SGD and see that it overfits data.How can I increase the number of iterations per epoch in MATLAB?Is there any standard or normal range for the amount of LSTM loss function?Why do people use CrossEntropyLoss and not just a softmax probability as the loss?
Help rendering a complicated sum/product formula
Maths symbols and unicode-math input inside siunitx commands
Does .bashrc contain syntax errors?
Is honey really a supersaturated solution? Does heating to un-crystalize redissolve it or melt it?
Writing in a Christian voice
Using Past-Perfect interchangeably with the Past Continuous
Usage and meaning of "up" in "...worth at least a thousand pounds up in London"
What should I install to correct "ld: cannot find -lgbm and -linput" so that I can compile a Rust program?
Would it be believable to defy demographics in a story?
Geography in 3D perspective
Worshiping one God at a time?
World War I as a war of liberals against authoritarians?
How might a highly intelligent aquatic species (mermaids) communicate underwater?
Print a physical multiplication table
Violin - Can double stops be played when the strings are not next to each other?
Why didn't Héctor fade away after this character died in the movie Coco?
Do native speakers use "ultima" and "proxima" frequently in spoken English?
Suggestions on how to spend Shaabath (constructively) alone
Describing a chess game in a novel
Do I need to consider instance restrictions when showing a language is in P?
Why are there no stars visible in cislunar space?
Print last inputted byte
What is the significance behind "40 days" that often appears in the Bible?
What does "mu" mean as an interjection?
Why increasing the number of units or layers does not increase the accuracy and decrease the loss?
Neural Network - Adjust number of hidden layers and neuronsWhy is my loss so high?Neural Network accuracy and loss guarantees?Regularization - Combine drop out with early stoppingLoss for CNN decreases and settles but training accuracy does not improveWhat to do if training loss decreases but validation loss does not decrease?train Neural Network with SGD and see that it overfits data.How can I increase the number of iterations per epoch in MATLAB?Is there any standard or normal range for the amount of LSTM loss function?Why do people use CrossEntropyLoss and not just a softmax probability as the loss?
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
add a comment |
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
edited 2 days ago
Media
7,35062161
edited 2 days ago
Media
7,35062161
edited 2 days ago
Media
7,35062161
7,35062161
asked 2 days ago
user145959user145959
1268
asked 2 days ago
user145959user145959
1268
asked 2 days ago
user145959user145959
1268
1268
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47362%2fwhy-increasing-the-number-of-units-or-layers-does-not-increase-the-accuracy-and%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
edited 2 days ago
answered 2 days ago
MediaMedia
7,35062161
answered 2 days ago
MediaMedia
7,35062161
answered 2 days ago
MediaMedia
7,35062161
7,35062161
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47362%2fwhy-increasing-the-number-of-units-or-layers-does-not-increase-the-accuracy-and%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago