Why increasing the number of units or layers does not increase the accuracy and decrease the loss?Neural Network - Adjust number of hidden layers and neuronsWhy is my loss so high?Neural Network accuracy and loss guarantees?Regularization - Combine drop out with early stoppingLoss for CNN decreases and settles but training accuracy does not improveWhat to do if training loss decreases but validation loss does not decrease?train Neural Network with SGD and see that it overfits data.How can I increase the number of iterations per epoch in MATLAB?Is there any standard or normal range for the amount of LSTM loss function?Why do people use CrossEntropyLoss and not just a softmax probability as the loss?
Is it insecure to send a password in a `curl` command?
HP P840 HDD RAID 5 many strange drive failures
Do I need to consider instance restrictions when showing a language is in P?
Is honey really a supersaturated solution? Does heating to un-crystalize redissolve it or melt it?
Comment Box for Substitution Method of Integrals
Bash - pair each line of file
Does the attack bonus from a Masterwork weapon stack with the attack bonus from Masterwork ammunition?
Can a wizard cast a spell during their first turn of combat if they initiated combat by releasing a readied spell?
Knife as defense against stray dogs
In what cases must I use 了 and in what cases not?
Turning a hard to access nut?
Hausdorff dimension of the boundary of fibres of Lipschitz maps
Describing a chess game in a novel
What does Deadpool mean by "left the house in that shirt"?
두음법칙 - When did North and South diverge in pronunciation of initial ㄹ?
Suggestions on how to spend Shaabath (constructively) alone
Are dual Irish/British citizens bound by the 90/180 day rule when travelling in the EU after Brexit?
What can I do if I am asked to learn different programming languages very frequently?
Is it possible to stack the damage done by the Absorb Elements spell?
Recruiter wants very extensive technical details about all of my previous work
What is the significance behind "40 days" that often appears in the Bible?
gerund and noun applications
Why didn't Héctor fade away after this character died in the movie Coco?
Relation between independence and correlation of uniform random variables
Why increasing the number of units or layers does not increase the accuracy and decrease the loss?
Neural Network - Adjust number of hidden layers and neuronsWhy is my loss so high?Neural Network accuracy and loss guarantees?Regularization - Combine drop out with early stoppingLoss for CNN decreases and settles but training accuracy does not improveWhat to do if training loss decreases but validation loss does not decrease?train Neural Network with SGD and see that it overfits data.How can I increase the number of iterations per epoch in MATLAB?Is there any standard or normal range for the amount of LSTM loss function?Why do people use CrossEntropyLoss and not just a softmax probability as the loss?
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
add a comment |
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
$begingroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
$endgroup$
I have an LSTM neural network; when I increase the number of units, layers, epochs or add dropout, it seems it has no effect and still I have persistent errors and accuracies like the following:
loss: 3.5071 - acc: 0.0981 - val_loss: 6.7042 - val_acc: 0.0122
Why this happens and how can I fix it?
machine-learning neural-network deep-learning lstm loss-function
machine-learning neural-network deep-learning lstm loss-function
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
edited 2 days ago
Media
7,35062161
asked 2 days ago
user145959user145959
1268
edited 2 days ago
Media
7,35062161
edited 2 days ago
Media
7,35062161
edited 2 days ago
Media
7,35062161
7,35062161
asked 2 days ago
user145959user145959
1268
asked 2 days ago
user145959user145959
1268
asked 2 days ago
user145959user145959
1268
1268
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47362%2fwhy-increasing-the-number-of-units-or-layers-does-not-increase-the-accuracy-and%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
add a comment |
$begingroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
$endgroup$
The title of your question and the question itself are somehow different but I'll try to answer the question, the meaning of decreasing loss without changes in accuracy.
The reason is simply due to using probabilities. For instance, for classification task if you have an output $0.7$ for input and the last layer is a softmax, then you classify the input as that class which has $0.7$. Imagine you train more and that output changes to something like $0.95$. Consequently, the accuracy does not change because you already classify it as what it really is but the loss lessens.
To answer the question which is in the body of your post, there can be numerous reasons that I'll try to refer to them.
One of the possibilities is that your data of different classes have overlap in the current feature space. This may lead to high Bayes error. For instance, suppose you have two same inputs and the label of them are contradictory. In this situation, your performance cannot be improved. To check it whether you've got this problem or not, take a look at the histogram of your data.
Another problem can be the weakness of LSTM networks which cannot memorise numerous things. LSTM models are very good at things like considering the gender of a subject or the plural or singular form of subjects but in cases where they should consider many things simultaneously, they have difficulties.
Another reason can be the incorrect way of using dropout. At first, do not use it and let your network overfits the training data to find a good model. After fitting your data, try to use dropout.
You can also test Stacked LSTMs which are powerful models.
answered 2 days ago
MediaMedia
7,35062161
edited 2 days ago
answered 2 days ago
MediaMedia
7,35062161
answered 2 days ago
MediaMedia
7,35062161
answered 2 days ago
MediaMedia
7,35062161
7,35062161
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47362%2fwhy-increasing-the-number-of-units-or-layers-does-not-increase-the-accuracy-and%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Those accuracies are so low that I wonder whether something is seriously wrong. Can you provide some more context?
$endgroup$
– Ben Reiniger
2 days ago