Unequal number of features in train and test data The Next CEO of Stack Overflow2019 Community Moderator ElectionFeature importance with high-cardinality categorical features for regression (numerical depdendent variable)Suitable aggregations (mean, median or something else) to make features?Preparing, Scaling and Selecting from a combination of numerical and categorical featuresadding logic combinations of boolean features in classificationPredicting with categorical dataAlways drop the first column after performing One Hot Encoding?Different number of features after using OneHotEncoderNumber of features of the model must match the input. Model n_features is `N` and input n_features is `X`.How to do feature engineering for email cleaning / text extraction?Target Encoding: missing value imputation before or after encoding
Apart from "berlinern", do any other German dialects have a corresponding verb?
Is it safe to use c_str() on a temporary string?
At which OSI layer a user-generated data resides?
Why do remote companies require working in the US?
Does it take more energy to get to Venus or to Mars?
Customer Requests (Sometimes) Drive Me Bonkers!
How to get regions to plot as graphics
Science fiction short story involving a paper written by a schizophrenic
Why is there a PLL in CPU?
Return the Closest Prime Number
If the heap is initialized for security, then why is the stack uninitialized?
How did people program for Consoles with multiple CPUs?
Why doesn't a table tennis ball float on a surface of steel balls? How do we calculate buoyancy here?
Can the Reverse Gravity spell affect the Meteor Swarm spell?
Tiptoe or tiphoof? Adjusting words to better fit fantasy races
Would this house-rule that treats advantage as a +1 to the roll instead (and disadvantage as -1) and allows them to stack be balanced?
Why do professional authors make "consistency" mistakes? And how to avoid them?
How to Reset Passwords on Multiple Websites Easily?
Why did we only see the N-1 starfighters in one film?
LWC - Unit Testing NavigationMixin.GenerateUrl
Inappropriate reference requests from Journal reviewers
How can I quit an app using Terminal?
What is the difference between Sanyaas and Vairagya?
What flight has the highest ratio of time difference to flight time?
Unequal number of features in train and test data
The Next CEO of Stack Overflow2019 Community Moderator ElectionFeature importance with high-cardinality categorical features for regression (numerical depdendent variable)Suitable aggregations (mean, median or something else) to make features?Preparing, Scaling and Selecting from a combination of numerical and categorical featuresadding logic combinations of boolean features in classificationPredicting with categorical dataAlways drop the first column after performing One Hot Encoding?Different number of features after using OneHotEncoderNumber of features of the model must match the input. Model n_features is `N` and input n_features is `X`.How to do feature engineering for email cleaning / text extraction?Target Encoding: missing value imputation before or after encoding
0
$begingroup$
After one hot encoding, I have different number of train and test features.
Does it matter if I add extra column with all zero values, so that the number of features become equal in both train and test dataset?
I am not considering to follow these things:
1) I am not combining the train and test dataset
2) Not doing PCA
feature-selection feature-engineering
asked Mar 22 at 14:34
pmdavpmdav
1
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
|
show 1 more comment
0
$begingroup$
After one hot encoding, I have different number of train and test features.
Does it matter if I add extra column with all zero values, so that the number of features become equal in both train and test dataset?
I am not considering to follow these things:
1) I am not combining the train and test dataset
2) Not doing PCA
feature-selection feature-engineering
asked Mar 22 at 14:34
pmdavpmdav
1
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
1
$begingroup$
Welcome to the site! How train and test have different formats? Does test set have a missing feature?
$endgroup$
– Esmailian
Mar 22 at 14:46
$begingroup$
Could you please elaborate on your problem. One hot encoding is usually carried out on the output rather than the features and so this problem becomes a little absurd. Please edit and explain the question again so that anyone will be able to help you.
$endgroup$
– thanatoz
Mar 23 at 9:51
$begingroup$
@Esmailian, Train set has less number of features than test set
$endgroup$
– pmdav
Mar 23 at 11:47
$begingroup$
@pmdav So model does not know about those extra features in test set, thus remove them and then run the model on test set.
$endgroup$
– Esmailian
Mar 23 at 11:50
$begingroup$
I have two separate train and test dataset. There are categorical features in both train and test dataset. After doing one hot encoding, I have unequal number of features in train and test dataset. This is due to unequal number of categorical values in both dataset. For example, test data become with 100 features while train with 105 features after one-hot encoding. Now, I want to make equal number of features in both dataset i.e. 105 features on both dataset. So, I am thinking to add extra 5 features/ columns with all values zeros. Does adding extra five columns have any impact?
$endgroup$
– pmdav
Mar 23 at 12:17
|
show 1 more comment
0
0
0
$begingroup$
After one hot encoding, I have different number of train and test features.
Does it matter if I add extra column with all zero values, so that the number of features become equal in both train and test dataset?
I am not considering to follow these things:
1) I am not combining the train and test dataset
2) Not doing PCA
feature-selection feature-engineering
asked Mar 22 at 14:34
pmdavpmdav
1
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
After one hot encoding, I have different number of train and test features.
Does it matter if I add extra column with all zero values, so that the number of features become equal in both train and test dataset?
I am not considering to follow these things:
1) I am not combining the train and test dataset
2) Not doing PCA
feature-selection feature-engineering
feature-selection feature-engineering
asked Mar 22 at 14:34
pmdavpmdav
1
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Mar 22 at 14:34
pmdavpmdav
1
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Mar 22 at 14:34
pmdavpmdav
1
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Mar 22 at 14:34
pmdavpmdav
1
asked Mar 22 at 14:34
pmdavpmdav
1
1
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
pmdav is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
$begingroup$
Welcome to the site! How train and test have different formats? Does test set have a missing feature?
$endgroup$
– Esmailian
Mar 22 at 14:46
$begingroup$
Could you please elaborate on your problem. One hot encoding is usually carried out on the output rather than the features and so this problem becomes a little absurd. Please edit and explain the question again so that anyone will be able to help you.
$endgroup$
– thanatoz
Mar 23 at 9:51
$begingroup$
@Esmailian, Train set has less number of features than test set
$endgroup$
– pmdav
Mar 23 at 11:47
$begingroup$
@pmdav So model does not know about those extra features in test set, thus remove them and then run the model on test set.
$endgroup$
– Esmailian
Mar 23 at 11:50
$begingroup$
I have two separate train and test dataset. There are categorical features in both train and test dataset. After doing one hot encoding, I have unequal number of features in train and test dataset. This is due to unequal number of categorical values in both dataset. For example, test data become with 100 features while train with 105 features after one-hot encoding. Now, I want to make equal number of features in both dataset i.e. 105 features on both dataset. So, I am thinking to add extra 5 features/ columns with all values zeros. Does adding extra five columns have any impact?
$endgroup$
– pmdav
Mar 23 at 12:17
|
show 1 more comment
1
$begingroup$
Welcome to the site! How train and test have different formats? Does test set have a missing feature?
$endgroup$
– Esmailian
Mar 22 at 14:46
$begingroup$
Could you please elaborate on your problem. One hot encoding is usually carried out on the output rather than the features and so this problem becomes a little absurd. Please edit and explain the question again so that anyone will be able to help you.
$endgroup$
– thanatoz
Mar 23 at 9:51
$begingroup$
@Esmailian, Train set has less number of features than test set
$endgroup$
– pmdav
Mar 23 at 11:47
$begingroup$
@pmdav So model does not know about those extra features in test set, thus remove them and then run the model on test set.
$endgroup$
– Esmailian
Mar 23 at 11:50
$begingroup$
I have two separate train and test dataset. There are categorical features in both train and test dataset. After doing one hot encoding, I have unequal number of features in train and test dataset. This is due to unequal number of categorical values in both dataset. For example, test data become with 100 features while train with 105 features after one-hot encoding. Now, I want to make equal number of features in both dataset i.e. 105 features on both dataset. So, I am thinking to add extra 5 features/ columns with all values zeros. Does adding extra five columns have any impact?
$endgroup$
– pmdav
Mar 23 at 12:17
1
1
$begingroup$
Welcome to the site! How train and test have different formats? Does test set have a missing feature?
$endgroup$
– Esmailian
Mar 22 at 14:46
$begingroup$
Welcome to the site! How train and test have different formats? Does test set have a missing feature?
$endgroup$
– Esmailian
Mar 22 at 14:46
$begingroup$
Could you please elaborate on your problem. One hot encoding is usually carried out on the output rather than the features and so this problem becomes a little absurd. Please edit and explain the question again so that anyone will be able to help you.
$endgroup$
– thanatoz
Mar 23 at 9:51
$begingroup$
Could you please elaborate on your problem. One hot encoding is usually carried out on the output rather than the features and so this problem becomes a little absurd. Please edit and explain the question again so that anyone will be able to help you.
$endgroup$
– thanatoz
Mar 23 at 9:51
$begingroup$
@Esmailian, Train set has less number of features than test set
$endgroup$
– pmdav
Mar 23 at 11:47
$begingroup$
@Esmailian, Train set has less number of features than test set
$endgroup$
– pmdav
Mar 23 at 11:47
$begingroup$
@pmdav So model does not know about those extra features in test set, thus remove them and then run the model on test set.
$endgroup$
– Esmailian
Mar 23 at 11:50
$begingroup$
@pmdav So model does not know about those extra features in test set, thus remove them and then run the model on test set.
$endgroup$
– Esmailian
Mar 23 at 11:50
$begingroup$
I have two separate train and test dataset. There are categorical features in both train and test dataset. After doing one hot encoding, I have unequal number of features in train and test dataset. This is due to unequal number of categorical values in both dataset. For example, test data become with 100 features while train with 105 features after one-hot encoding. Now, I want to make equal number of features in both dataset i.e. 105 features on both dataset. So, I am thinking to add extra 5 features/ columns with all values zeros. Does adding extra five columns have any impact?
$endgroup$
– pmdav
Mar 23 at 12:17
$begingroup$
I have two separate train and test dataset. There are categorical features in both train and test dataset. After doing one hot encoding, I have unequal number of features in train and test dataset. This is due to unequal number of categorical values in both dataset. For example, test data become with 100 features while train with 105 features after one-hot encoding. Now, I want to make equal number of features in both dataset i.e. 105 features on both dataset. So, I am thinking to add extra 5 features/ columns with all values zeros. Does adding extra five columns have any impact?
$endgroup$
– pmdav
Mar 23 at 12:17
|
show 1 more comment
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
pmdav is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47796%2funequal-number-of-features-in-train-and-test-data%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
pmdav is a new contributor. Be nice, and check out our Code of Conduct.
draft saved
draft discarded
pmdav is a new contributor. Be nice, and check out our Code of Conduct.
pmdav is a new contributor. Be nice, and check out our Code of Conduct.
pmdav is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47796%2funequal-number-of-features-in-train-and-test-data%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
Welcome to the site! How train and test have different formats? Does test set have a missing feature?
$endgroup$
– Esmailian
Mar 22 at 14:46
$begingroup$
Could you please elaborate on your problem. One hot encoding is usually carried out on the output rather than the features and so this problem becomes a little absurd. Please edit and explain the question again so that anyone will be able to help you.
$endgroup$
– thanatoz
Mar 23 at 9:51
$begingroup$
@Esmailian, Train set has less number of features than test set
$endgroup$
– pmdav
Mar 23 at 11:47
$begingroup$
@pmdav So model does not know about those extra features in test set, thus remove them and then run the model on test set.
$endgroup$
– Esmailian
Mar 23 at 11:50
$begingroup$
I have two separate train and test dataset. There are categorical features in both train and test dataset. After doing one hot encoding, I have unequal number of features in train and test dataset. This is due to unequal number of categorical values in both dataset. For example, test data become with 100 features while train with 105 features after one-hot encoding. Now, I want to make equal number of features in both dataset i.e. 105 features on both dataset. So, I am thinking to add extra 5 features/ columns with all values zeros. Does adding extra five columns have any impact?
$endgroup$
– pmdav
Mar 23 at 12:17