General question on EDA, correlations, classification, MLBinary classification model for sparse / biased dataModel that adapts to sample updatesHow does a convolutional ply differ from an ordinary convolutional network?How to improve precision under imbalanced classificationGeneral Machine Learning Workflow QuestionWhat are best practices for collaborative feature engineering?Does it make sense to “reorder” a categorical feature to make it monotonic?What data treatment/transformation should be applied if there are a lot of outliers and features lack normal distribution?How to model & predict user activity/presence time in a websiteGeneral question on the approach to optimise numbers
Is GOCE a satellite or aircraft?
Confusion about capacitors
What does "rf" mean in "rfkill"?
What is the difference between `a[bc]d` (brackets) and `ab,cd` (braces)?
Is thermodynamics only applicable to systems in equilibrium?
How to stop co-workers from teasing me because I know Russian?
If Earth is tilted, why is Polaris always above the same spot?
Examples of non trivial equivalence relations , I mean equivalence relations without the expression " same ... as" in their definition?
A non-technological, repeating, visible object in the sky, holding its position in the sky for hours
Binary Numbers Magic Trick
Do I have to worry about players making “bad” choices on level up?
What are the spoon bit of a spoon and fork bit of a fork called?
Do I have an "anti-research" personality?
Historically, were women trained for obligatory wars? Or did they serve some other military function?
Is it cheaper to drop cargo drop than to land it?
Pressure to defend the relevance of one's area of mathematics
Why do computer-science majors learn calculus?
Transfer over $10k
When did stoichiometry begin to be taught in U.S. high schools?
Will tsunami waves travel forever if there was no land?
Given what happens in Endgame, why doesn't Dormammu come back to attack the universe?
How to set printing options as reverse order as default on 18.04
Why does the Betti number give the measure of k-dimensional holes?
Asahi Dry Black beer can
General question on EDA, correlations, classification, ML
Binary classification model for sparse / biased dataModel that adapts to sample updatesHow does a convolutional ply differ from an ordinary convolutional network?How to improve precision under imbalanced classificationGeneral Machine Learning Workflow QuestionWhat are best practices for collaborative feature engineering?Does it make sense to “reorder” a categorical feature to make it monotonic?What data treatment/transformation should be applied if there are a lot of outliers and features lack normal distribution?How to model & predict user activity/presence time in a websiteGeneral question on the approach to optimise numbers
$begingroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
asked Mar 9 at 13:04
user69194user69194
11
$endgroup$
add a comment |
$begingroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
asked Mar 9 at 13:04
user69194user69194
11
$endgroup$
add a comment |
$begingroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
asked Mar 9 at 13:04
user69194user69194
11
$endgroup$
I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA and I am not sure which feature to include in my EDA, A or B. When I do correlations plots, nothing correlates with feature B, but some features do correlate with feature A. Which one should I take into account then, A or B correlations? Also, how can I make use of those correlations and scatterplots and pairplots anyway and are they important? If I am using random forest or NN, do I even need to bother with all of the pairplots and correlations to extract features from? I have around 150 features and not sure how to approach the problem of which features to use. I haven't found a source saying how to make a proper use of all of this in a real world scenarios. Any help is appreciated.
machine-learning feature-extraction data-science-model
machine-learning feature-extraction data-science-model
asked Mar 9 at 13:04
user69194user69194
11
asked Mar 9 at 13:04
user69194user69194
11
asked Mar 9 at 13:04
user69194user69194
11
asked Mar 9 at 13:04
user69194user69194
11
asked Mar 9 at 13:04
user69194user69194
11
11
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered Mar 9 at 17:26
SunilSunil
1045
$endgroup$
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
Mar 10 at 8:38
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
Mar 10 at 12:54
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46989%2fgeneral-question-on-eda-correlations-classification-ml%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered Mar 9 at 17:26
SunilSunil
1045
$endgroup$
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
Mar 10 at 8:38
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
Mar 10 at 12:54
add a comment |
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered Mar 9 at 17:26
SunilSunil
1045
$endgroup$
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
Mar 10 at 8:38
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
Mar 10 at 12:54
add a comment |
$begingroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered Mar 9 at 17:26
SunilSunil
1045
$endgroup$
If the relation between predictors is nearly 0, it's always better to drop that feature, the caveat here depends on the domain knowledge you have.
Did you check the correlation between B and the target variable and also A and target variable? if it's negative drop it, If it's significantly high .i.e greater 0.7, use that as your feature.
Yes, pair plots and scatter plots are really important, but it would be tedious to plot features with 150 variables.
answered Mar 9 at 17:26
SunilSunil
1045
answered Mar 9 at 17:26
SunilSunil
1045
answered Mar 9 at 17:26
SunilSunil
1045
answered Mar 9 at 17:26
SunilSunil
1045
1045
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
Mar 10 at 8:38
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
Mar 10 at 12:54
add a comment |
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
Mar 10 at 8:38
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
Mar 10 at 12:54
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
Mar 10 at 8:38
$begingroup$
I don't have any domain knowledge, I have some random data with features that I don't know much about. How do I use scatterplots and pairplots to determine which features to use? Or what should I be using them for?
$endgroup$
– user69194
Mar 10 at 8:38
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
Mar 10 at 12:54
$begingroup$
Do a correlation map or heat map to determine which features to use. negatively correlated features can be avoided, but with caution. Pair panels give a lot of information about your data points. You can also do a simple describe on your data and find about the skews.
$endgroup$
– Sunil
Mar 10 at 12:54
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46989%2fgeneral-question-on-eda-correlations-classification-ml%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
