Binary Classification of Numeric Sequences with Keras and LSTMs Unicorn Meta Zoo #1: Why another podcast? Announcing the arrival of Valued Associate #679: Cesar Manara 2019 Moderator Election Q&A - Questionnaire 2019 Community Moderator Election ResultsLSTM neural network for music generationDot Product between two Keras intermediate variablesWhy does my model accuracy rise and then drop, with the loss sharing similar characteristics?Best model for Machine LearningKeras LSTM model for binary classification with sequencesHow to set input for proper fit with lstm?Neural network outputting same result for all inputsPython - Predicting data based on multidimensional array with KerasHow to reshape data for LSTM training in multivariate sequence predictionIN CIFAR 10 DATASET
Philosophical question on logisitic regression: why isn't the optimal threshold value trained?
How do I check if a string is entirely made of the same substring?
Israeli soda type drink
How much of a wave function must reside inside event horizon for it to be consumed by the black hole?
Can a stored procedure reference the database in which it is stored?
What's the difference between using dependency injection with a container and using a service locator?
How long after the last departure shall the airport stay open for an emergency return?
Why do games have consumables?
Retract an already submitted recommendation letter (written for an undergrad student)
Multiple fireplaces in an apartment building?
How to have a sharp product image?
What is it called when you ride around on your front wheel?
How to keep bees out of canned beverages?
Mistake in years of experience in resume?
What *exactly* is electrical current, voltage, and resistance?
I preordered a game on my Xbox while on the home screen of my friend's account. Which of us owns the game?
Can a level 2 Warlock take one level in rogue, then continue advancing as a warlock?
Raising a bilingual kid. When should we introduce the majority language?
Why do distances seem to matter in the Foundation world?
Sharepoint Designer Discontinuation - software to modify existing workflows
Bayes factor vs P value
Scheduling based problem
How to translate "red flag" into Spanish?
When do you need buffers/drivers on buses in a microprocessor design?
Binary Classification of Numeric Sequences with Keras and LSTMs
Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar Manara
2019 Moderator Election Q&A - Questionnaire
2019 Community Moderator Election ResultsLSTM neural network for music generationDot Product between two Keras intermediate variablesWhy does my model accuracy rise and then drop, with the loss sharing similar characteristics?Best model for Machine LearningKeras LSTM model for binary classification with sequencesHow to set input for proper fit with lstm?Neural network outputting same result for all inputsPython - Predicting data based on multidimensional array with KerasHow to reshape data for LSTM training in multivariate sequence predictionIN CIFAR 10 DATASET
$begingroup$
I'm attempting to use a sequence of numbers (of fixed length) in order to predict a binary output (either 1 or 0) using Keras and a recurrent neural network.
Each training example/sequence has 10 timesteps, each containing a vector of 5 numbers, and each training output consists of either a 1 or 0. The ratio of 1s to 0s is around 1:3. There are approximately 100,000 training examples.
I have tried implementing this using Keras, but the loss stops decreasing after the first epoch of training. I've also attempted modifying the hyper-parameters, but to no avail. Is there something I'm missing here?
The training inputs are as follows: (zero padded)
array([[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24829336, 0.96461449, 3.35142857, 0.74675 , 0.776075 ],
[1.248303 , 0.96427925, 0. , 1.317225 , 1.317225 ],
[1.24831488, 0.96409169, 2.74857143, 1.353775 , 1.377825 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24969672, 0.96336315, 0. , 1.319725 , 1.319725 ],
[1.24968077, 0.96331624, 0. , 1.33535 , 1.33535 ],
[1.24969598, 0.96330252, 5.01714286, 1.3508 , 1.3947 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[0. , 0. , 0. , 0. , 0. ],
[1.25715364, 0.95520672, 2.57714286, 1.04565 , 1.0682 ],
[1.25291274, 0.96879701, 7.76 , 1.311875 , 1.379775 ]],
...,
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24791079, 0.96561021, 4.44 , 0.7199 , 0.75875 ],
[1.25265263, 0.96117379, 2.09714286, 0.7636 , 0.78195 ],
[1.25868651, 0.96001674, 3.01142857, 1.35235 , 1.3787 ]]])
The training outputs are as follows:
array([[0.],
[0.],
[0.],
...,
[1.],
[0.],
[0.]])
This is the model I have attempted to train:
#Model
model = Sequential()
model.add(LSTM(100, input_shape= (10, 5)))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
print(model.summary())
model.fit(X_train, y_train, validation_data = (X_test, y_test), epochs = 100, batch_size = 1000)
classification keras lstm binary neural
asked Apr 6 at 17:15
George LeeGeorge Lee
11
$endgroup$
|
show 4 more comments
$begingroup$
I'm attempting to use a sequence of numbers (of fixed length) in order to predict a binary output (either 1 or 0) using Keras and a recurrent neural network.
Each training example/sequence has 10 timesteps, each containing a vector of 5 numbers, and each training output consists of either a 1 or 0. The ratio of 1s to 0s is around 1:3. There are approximately 100,000 training examples.
I have tried implementing this using Keras, but the loss stops decreasing after the first epoch of training. I've also attempted modifying the hyper-parameters, but to no avail. Is there something I'm missing here?
The training inputs are as follows: (zero padded)
array([[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24829336, 0.96461449, 3.35142857, 0.74675 , 0.776075 ],
[1.248303 , 0.96427925, 0. , 1.317225 , 1.317225 ],
[1.24831488, 0.96409169, 2.74857143, 1.353775 , 1.377825 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24969672, 0.96336315, 0. , 1.319725 , 1.319725 ],
[1.24968077, 0.96331624, 0. , 1.33535 , 1.33535 ],
[1.24969598, 0.96330252, 5.01714286, 1.3508 , 1.3947 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[0. , 0. , 0. , 0. , 0. ],
[1.25715364, 0.95520672, 2.57714286, 1.04565 , 1.0682 ],
[1.25291274, 0.96879701, 7.76 , 1.311875 , 1.379775 ]],
...,
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24791079, 0.96561021, 4.44 , 0.7199 , 0.75875 ],
[1.25265263, 0.96117379, 2.09714286, 0.7636 , 0.78195 ],
[1.25868651, 0.96001674, 3.01142857, 1.35235 , 1.3787 ]]])
The training outputs are as follows:
array([[0.],
[0.],
[0.],
...,
[1.],
[0.],
[0.]])
This is the model I have attempted to train:
#Model
model = Sequential()
model.add(LSTM(100, input_shape= (10, 5)))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
print(model.summary())
model.fit(X_train, y_train, validation_data = (X_test, y_test), epochs = 100, batch_size = 1000)
classification keras lstm binary neural
asked Apr 6 at 17:15
George LeeGeorge Lee
11
$endgroup$
$begingroup$
How many training instances do you have?
$endgroup$
– JahKnows
Apr 6 at 17:25
$begingroup$
I have around 100,000 instances
$endgroup$
– George Lee
Apr 6 at 17:52
1
$begingroup$
Welcome to SE.DataScience! Please provide these two: (1) ratio of 1s to all instances, and (2) value of loss for first, second, and third epochs. I may have an answer.
$endgroup$
– Esmailian
Apr 6 at 17:56
1
$begingroup$
Can you give us a snippet of the data please?
$endgroup$
– JahKnows
Apr 6 at 17:59
$begingroup$
(1) 1:4 (2) Loss actually flattens out after around 3-4 epochs, at around 0.5870, 0.5805, 0.5804
$endgroup$
– George Lee
Apr 6 at 18:26
|
show 4 more comments
$begingroup$
I'm attempting to use a sequence of numbers (of fixed length) in order to predict a binary output (either 1 or 0) using Keras and a recurrent neural network.
Each training example/sequence has 10 timesteps, each containing a vector of 5 numbers, and each training output consists of either a 1 or 0. The ratio of 1s to 0s is around 1:3. There are approximately 100,000 training examples.
I have tried implementing this using Keras, but the loss stops decreasing after the first epoch of training. I've also attempted modifying the hyper-parameters, but to no avail. Is there something I'm missing here?
The training inputs are as follows: (zero padded)
array([[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24829336, 0.96461449, 3.35142857, 0.74675 , 0.776075 ],
[1.248303 , 0.96427925, 0. , 1.317225 , 1.317225 ],
[1.24831488, 0.96409169, 2.74857143, 1.353775 , 1.377825 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24969672, 0.96336315, 0. , 1.319725 , 1.319725 ],
[1.24968077, 0.96331624, 0. , 1.33535 , 1.33535 ],
[1.24969598, 0.96330252, 5.01714286, 1.3508 , 1.3947 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[0. , 0. , 0. , 0. , 0. ],
[1.25715364, 0.95520672, 2.57714286, 1.04565 , 1.0682 ],
[1.25291274, 0.96879701, 7.76 , 1.311875 , 1.379775 ]],
...,
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24791079, 0.96561021, 4.44 , 0.7199 , 0.75875 ],
[1.25265263, 0.96117379, 2.09714286, 0.7636 , 0.78195 ],
[1.25868651, 0.96001674, 3.01142857, 1.35235 , 1.3787 ]]])
The training outputs are as follows:
array([[0.],
[0.],
[0.],
...,
[1.],
[0.],
[0.]])
This is the model I have attempted to train:
#Model
model = Sequential()
model.add(LSTM(100, input_shape= (10, 5)))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
print(model.summary())
model.fit(X_train, y_train, validation_data = (X_test, y_test), epochs = 100, batch_size = 1000)
classification keras lstm binary neural
asked Apr 6 at 17:15
George LeeGeorge Lee
11
$endgroup$
I'm attempting to use a sequence of numbers (of fixed length) in order to predict a binary output (either 1 or 0) using Keras and a recurrent neural network.
Each training example/sequence has 10 timesteps, each containing a vector of 5 numbers, and each training output consists of either a 1 or 0. The ratio of 1s to 0s is around 1:3. There are approximately 100,000 training examples.
I have tried implementing this using Keras, but the loss stops decreasing after the first epoch of training. I've also attempted modifying the hyper-parameters, but to no avail. Is there something I'm missing here?
The training inputs are as follows: (zero padded)
array([[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24829336, 0.96461449, 3.35142857, 0.74675 , 0.776075 ],
[1.248303 , 0.96427925, 0. , 1.317225 , 1.317225 ],
[1.24831488, 0.96409169, 2.74857143, 1.353775 , 1.377825 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24969672, 0.96336315, 0. , 1.319725 , 1.319725 ],
[1.24968077, 0.96331624, 0. , 1.33535 , 1.33535 ],
[1.24969598, 0.96330252, 5.01714286, 1.3508 , 1.3947 ]],
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[0. , 0. , 0. , 0. , 0. ],
[1.25715364, 0.95520672, 2.57714286, 1.04565 , 1.0682 ],
[1.25291274, 0.96879701, 7.76 , 1.311875 , 1.379775 ]],
...,
[[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
[0. , 0. , 0. , 0. , 0. ],
...,
[1.24791079, 0.96561021, 4.44 , 0.7199 , 0.75875 ],
[1.25265263, 0.96117379, 2.09714286, 0.7636 , 0.78195 ],
[1.25868651, 0.96001674, 3.01142857, 1.35235 , 1.3787 ]]])
The training outputs are as follows:
array([[0.],
[0.],
[0.],
...,
[1.],
[0.],
[0.]])
This is the model I have attempted to train:
#Model
model = Sequential()
model.add(LSTM(100, input_shape= (10, 5)))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
print(model.summary())
model.fit(X_train, y_train, validation_data = (X_test, y_test), epochs = 100, batch_size = 1000)
classification keras lstm binary neural
classification keras lstm binary neural
asked Apr 6 at 17:15
George LeeGeorge Lee
11
asked Apr 6 at 17:15
George LeeGeorge Lee
11
edited Apr 7 at 5:25
George Lee
asked Apr 6 at 17:15
George LeeGeorge Lee
11
asked Apr 6 at 17:15
George LeeGeorge Lee
11
asked Apr 6 at 17:15
George LeeGeorge Lee
11
11
$begingroup$
How many training instances do you have?
$endgroup$
– JahKnows
Apr 6 at 17:25
$begingroup$
I have around 100,000 instances
$endgroup$
– George Lee
Apr 6 at 17:52
1
$begingroup$
Welcome to SE.DataScience! Please provide these two: (1) ratio of 1s to all instances, and (2) value of loss for first, second, and third epochs. I may have an answer.
$endgroup$
– Esmailian
Apr 6 at 17:56
1
$begingroup$
Can you give us a snippet of the data please?
$endgroup$
– JahKnows
Apr 6 at 17:59
$begingroup$
(1) 1:4 (2) Loss actually flattens out after around 3-4 epochs, at around 0.5870, 0.5805, 0.5804
$endgroup$
– George Lee
Apr 6 at 18:26
|
show 4 more comments
$begingroup$
How many training instances do you have?
$endgroup$
– JahKnows
Apr 6 at 17:25
$begingroup$
I have around 100,000 instances
$endgroup$
– George Lee
Apr 6 at 17:52
1
$begingroup$
Welcome to SE.DataScience! Please provide these two: (1) ratio of 1s to all instances, and (2) value of loss for first, second, and third epochs. I may have an answer.
$endgroup$
– Esmailian
Apr 6 at 17:56
1
$begingroup$
Can you give us a snippet of the data please?
$endgroup$
– JahKnows
Apr 6 at 17:59
$begingroup$
(1) 1:4 (2) Loss actually flattens out after around 3-4 epochs, at around 0.5870, 0.5805, 0.5804
$endgroup$
– George Lee
Apr 6 at 18:26
$begingroup$
How many training instances do you have?
$endgroup$
– JahKnows
Apr 6 at 17:25
$begingroup$
How many training instances do you have?
$endgroup$
– JahKnows
Apr 6 at 17:25
$begingroup$
I have around 100,000 instances
$endgroup$
– George Lee
Apr 6 at 17:52
$begingroup$
I have around 100,000 instances
$endgroup$
– George Lee
Apr 6 at 17:52
1
1
$begingroup$
Welcome to SE.DataScience! Please provide these two: (1) ratio of 1s to all instances, and (2) value of loss for first, second, and third epochs. I may have an answer.
$endgroup$
– Esmailian
Apr 6 at 17:56
$begingroup$
Welcome to SE.DataScience! Please provide these two: (1) ratio of 1s to all instances, and (2) value of loss for first, second, and third epochs. I may have an answer.
$endgroup$
– Esmailian
Apr 6 at 17:56
1
1
$begingroup$
Can you give us a snippet of the data please?
$endgroup$
– JahKnows
Apr 6 at 17:59
$begingroup$
Can you give us a snippet of the data please?
$endgroup$
– JahKnows
Apr 6 at 17:59
$begingroup$
(1) 1:4 (2) Loss actually flattens out after around 3-4 epochs, at around 0.5870, 0.5805, 0.5804
$endgroup$
– George Lee
Apr 6 at 18:26
$begingroup$
(1) 1:4 (2) Loss actually flattens out after around 3-4 epochs, at around 0.5870, 0.5805, 0.5804
$endgroup$
– George Lee
Apr 6 at 18:26
|
show 4 more comments
0
active
oldest
votes
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48764%2fbinary-classification-of-numeric-sequences-with-keras-and-lstms%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f48764%2fbinary-classification-of-numeric-sequences-with-keras-and-lstms%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
How many training instances do you have?
$endgroup$
– JahKnows
Apr 6 at 17:25
$begingroup$
I have around 100,000 instances
$endgroup$
– George Lee
Apr 6 at 17:52
1
$begingroup$
Welcome to SE.DataScience! Please provide these two: (1) ratio of 1s to all instances, and (2) value of loss for first, second, and third epochs. I may have an answer.
$endgroup$
– Esmailian
Apr 6 at 17:56
1
$begingroup$
Can you give us a snippet of the data please?
$endgroup$
– JahKnows
Apr 6 at 17:59
$begingroup$
(1) 1:4 (2) Loss actually flattens out after around 3-4 epochs, at around 0.5870, 0.5805, 0.5804
$endgroup$
– George Lee
Apr 6 at 18:26