Keras: extreme spike in loss during training The 2019 Stack Overflow Developer Survey Results Are In Unicorn Meta Zoo #1: Why another podcast? Announcing the arrival of Valued Associate #679: Cesar Manara 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsKeras difference beetween val_loss and loss during trainingThe validation loss < training loss and validation accuracy < training accuracySimple prediction with KerasHow to set input for proper fit with lstm?My Neural network in Tensorflow does a bad job in comparison to the same Neural network in KerasWhy is predicted rainfall by LSTM coming negative for some data points?Images Score Regression only regresses to the average of the target valuesWhy doesn't loss go down during Neural Net training?keras plotting loss and MSE
How to politely respond to generic emails requesting a PhD/job in my lab? Without wasting too much time
number sequence puzzle deep six
For what reasons would an animal species NOT cross a *horizontal* land bridge?
Is Cinnamon a desktop environment or a window manager? (Or both?)
Is every episode of "Where are my Pants?" identical?
How to read αἱμύλιος or when to aspirate
Identify 80s or 90s comics with ripped creatures (not dwarves)
Do working physicists consider Newtonian mechanics to be "falsified"?
Match Roman Numerals
how can a perfect fourth interval be considered either consonant or dissonant?
Is an up-to-date browser secure on an out-of-date OS?
Is there a way to generate uniformly distributed points on a sphere from a fixed amount of random real numbers per point?
What can I do if neighbor is blocking my solar panels intentionally?
Do ℕ, mathbbN, BbbN, symbbN effectively differ, and is there a "canonical" specification of the naturals?
different output for groups and groups USERNAME after adding a username to a group
Working through Single Responsibility Principle in Python when Calls are Expensive
How to fill page vertically?
What information about me do stores get via my credit card?
Merge multiple DataFrames Pandas
Make it rain characters
Did the UK government pay "millions and millions of dollars" to try to snag Julian Assange?
How to type a long/em dash `—`
What was the last x86 CPU that did not have the x87 floating-point unit built in?
Why are Marketing Cloud timestamps not stored in the same timezone as Sales Cloud?
Keras: extreme spike in loss during training
The 2019 Stack Overflow Developer Survey Results Are In
Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar Manara
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsKeras difference beetween val_loss and loss during trainingThe validation loss < training loss and validation accuracy < training accuracySimple prediction with KerasHow to set input for proper fit with lstm?My Neural network in Tensorflow does a bad job in comparison to the same Neural network in KerasWhy is predicted rainfall by LSTM coming negative for some data points?Images Score Regression only regresses to the average of the target valuesWhy doesn't loss go down during Neural Net training?keras plotting loss and MSE
$begingroup$
I am training an LSTM for time series forecasting and it has produced an extremly high loss value during one epoch:
Epoch 00043: saving model to /...
904/904 - 2s - loss: 0.7537 - mean_absolute_error: 0.5772 - val_loss: 1.4430
- val_mean_absolute_error: 0.7124
Epoch 00044: saving model to /...
904/904 - 2s - loss: 240372339275.7649 - mean_absolute_error: 56354.0078
- val_loss: 4.6229 - val_mean_absolute_error: 1.5681
Epoch 00045: saving model to /...
904/904 - 2s - loss: 1.3348 - mean_absolute_error: 0.7894 - val_loss: 2.2875
- val_mean_absolute_error: 1.1510
My model:
model = keras.Sequential()
model.add(keras.layers.LSTM(360, activation='relu', input_shape=(N_STEPS, n_features)))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.Dense(1, activation='relu'))
model.compile(optimizer='adam', loss='mse', metrics=['mae'])
What is the cause of this?
Theoretically, it shouldn't be able to have such a high loss unless it outputs very high values for that epoch. Which is strange since the model's output makes sense during other epochs.
python keras regression lstm accuracy
asked Feb 28 at 11:02
1b151b15
434
$endgroup$
add a comment |
$begingroup$
I am training an LSTM for time series forecasting and it has produced an extremly high loss value during one epoch:
Epoch 00043: saving model to /...
904/904 - 2s - loss: 0.7537 - mean_absolute_error: 0.5772 - val_loss: 1.4430
- val_mean_absolute_error: 0.7124
Epoch 00044: saving model to /...
904/904 - 2s - loss: 240372339275.7649 - mean_absolute_error: 56354.0078
- val_loss: 4.6229 - val_mean_absolute_error: 1.5681
Epoch 00045: saving model to /...
904/904 - 2s - loss: 1.3348 - mean_absolute_error: 0.7894 - val_loss: 2.2875
- val_mean_absolute_error: 1.1510
My model:
model = keras.Sequential()
model.add(keras.layers.LSTM(360, activation='relu', input_shape=(N_STEPS, n_features)))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.Dense(1, activation='relu'))
model.compile(optimizer='adam', loss='mse', metrics=['mae'])
What is the cause of this?
Theoretically, it shouldn't be able to have such a high loss unless it outputs very high values for that epoch. Which is strange since the model's output makes sense during other epochs.
python keras regression lstm accuracy
asked Feb 28 at 11:02
1b151b15
434
$endgroup$
add a comment |
$begingroup$
I am training an LSTM for time series forecasting and it has produced an extremly high loss value during one epoch:
Epoch 00043: saving model to /...
904/904 - 2s - loss: 0.7537 - mean_absolute_error: 0.5772 - val_loss: 1.4430
- val_mean_absolute_error: 0.7124
Epoch 00044: saving model to /...
904/904 - 2s - loss: 240372339275.7649 - mean_absolute_error: 56354.0078
- val_loss: 4.6229 - val_mean_absolute_error: 1.5681
Epoch 00045: saving model to /...
904/904 - 2s - loss: 1.3348 - mean_absolute_error: 0.7894 - val_loss: 2.2875
- val_mean_absolute_error: 1.1510
My model:
model = keras.Sequential()
model.add(keras.layers.LSTM(360, activation='relu', input_shape=(N_STEPS, n_features)))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.Dense(1, activation='relu'))
model.compile(optimizer='adam', loss='mse', metrics=['mae'])
What is the cause of this?
Theoretically, it shouldn't be able to have such a high loss unless it outputs very high values for that epoch. Which is strange since the model's output makes sense during other epochs.
python keras regression lstm accuracy
asked Feb 28 at 11:02
1b151b15
434
$endgroup$
I am training an LSTM for time series forecasting and it has produced an extremly high loss value during one epoch:
Epoch 00043: saving model to /...
904/904 - 2s - loss: 0.7537 - mean_absolute_error: 0.5772 - val_loss: 1.4430
- val_mean_absolute_error: 0.7124
Epoch 00044: saving model to /...
904/904 - 2s - loss: 240372339275.7649 - mean_absolute_error: 56354.0078
- val_loss: 4.6229 - val_mean_absolute_error: 1.5681
Epoch 00045: saving model to /...
904/904 - 2s - loss: 1.3348 - mean_absolute_error: 0.7894 - val_loss: 2.2875
- val_mean_absolute_error: 1.1510
My model:
model = keras.Sequential()
model.add(keras.layers.LSTM(360, activation='relu', input_shape=(N_STEPS, n_features)))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.Dense(1, activation='relu'))
model.compile(optimizer='adam', loss='mse', metrics=['mae'])
What is the cause of this?
Theoretically, it shouldn't be able to have such a high loss unless it outputs very high values for that epoch. Which is strange since the model's output makes sense during other epochs.
python keras regression lstm accuracy
python keras regression lstm accuracy
asked Feb 28 at 11:02
1b151b15
434
asked Feb 28 at 11:02
1b151b15
434
asked Feb 28 at 11:02
1b151b15
434
asked Feb 28 at 11:02
1b151b15
434
asked Feb 28 at 11:02
1b151b15
434
434
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Sorry I couldn't comment as it requires 50 Reputation. On Epoch 44 there is a huge spike in the loss. It is entirely possible that the model may have come across new data and it may have learned a few tricks up its sleeve. Try to plot loss of train & validation vs epoch to see if it underfits or overfits.
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46387%2fkeras-extreme-spike-in-loss-during-training%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Sorry I couldn't comment as it requires 50 Reputation. On Epoch 44 there is a huge spike in the loss. It is entirely possible that the model may have come across new data and it may have learned a few tricks up its sleeve. Try to plot loss of train & validation vs epoch to see if it underfits or overfits.
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
$endgroup$
add a comment |
$begingroup$
Sorry I couldn't comment as it requires 50 Reputation. On Epoch 44 there is a huge spike in the loss. It is entirely possible that the model may have come across new data and it may have learned a few tricks up its sleeve. Try to plot loss of train & validation vs epoch to see if it underfits or overfits.
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
$endgroup$
add a comment |
$begingroup$
Sorry I couldn't comment as it requires 50 Reputation. On Epoch 44 there is a huge spike in the loss. It is entirely possible that the model may have come across new data and it may have learned a few tricks up its sleeve. Try to plot loss of train & validation vs epoch to see if it underfits or overfits.
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
$endgroup$
Sorry I couldn't comment as it requires 50 Reputation. On Epoch 44 there is a huge spike in the loss. It is entirely possible that the model may have come across new data and it may have learned a few tricks up its sleeve. Try to plot loss of train & validation vs epoch to see if it underfits or overfits.
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
answered Feb 28 at 15:16
Justice_LordsJustice_Lords
1031
1031
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46387%2fkeras-extreme-spike-in-loss-during-training%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
