Trjtdtk

What is IP squat space

How many prime numbers are there that can't be written as a sum of two composite numbers?

Could the Saturn V actually have launched astronauts around Venus?

Is having access to past exams cheating and, if yes, could it be proven just by a good grade?

Why doesn't the EU now just force the UK to choose between referendum and no-deal?

Good allowance savings plan?

Why are the outputs of printf and std::cout different

When do we add an hyphen (-) to a complex adjective word?

Does this property of comaximal ideals always holds?

Giving EXEC (@Variable) a Column name and Concatenation

Should we release the security issues we found in our product as CVE or we can just update those on weekly release notes?

Who is our nearest planetary neighbor, on average?

Did CP/M support custom hardware using device drivers?

Is it normal that my co-workers at a fitness company criticize my food choices?

Can anyone tell me why this program fails?

Force user to remove USB token

Why did it take so long to abandon sail after steamships were demonstrated?

How could a female member of a species produce eggs unto death?

Happy pi day, everyone!

Does splitting a potentially monolithic application into several smaller ones help prevent bugs?

Would it take an action or something similar to activate the blindsight property of a Dragon Mask?

Increase thickness of graph lines larger than ultra thick

What has been your most complicated TikZ drawing?

Are the common programs (for example: "ls", "cat") in Linux and BSD come from the same source code?

How to plan an analysis to prevent overfitting?

how to explain the behaviour: linear svm does better than non-linear RBFHow to represent target variable for chess AIMachine Learning: Writing PoemsHow to perform Logistic Regression with a large number of features?How to approach speech analysis?Model Selection with Oversampling/ Cross-Validation leads to similar test results in 2 approachesI have limited samples for one class, unlimited samples for the other class. Need to balance?ML algorithms for regression in the case of label noise with a known distribution?SciKit-Learn Decision Tree OverfittingHow can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?

2

$begingroup$

Coming from statistics, I'm freshly trying to learn machine learning. I've read a lot of tutorials about ML, but have no real training.

I'm working on a little project where my dataset have 6k lines and around 300 features.

As I've read in my tutorials, I split my dataset into a training sample (80%) and a testing sample (20%), and then train my algorithm on the training sample with cross-validation (5 folds).

As I re-ran my program twice (I've only tested KNN which I now know is quite not appropriate), I got really different results, with different sensitivity, specificity and precision.

I guess that if I re-run the program until metrics are good, my algorithm will be overfitted, and I also guess it would be because of the resample of test/training samples, but please correct me if I'm wrong.

If I'm going to try a lot of algorithms to see what I can get, should I fix my samples somewhere ? Is it even OK to do so ? (it would not always be in statistics)

In case it matters, I'm working with python's scikit-learn module.

*PS: my outcome is binary and my features are mostly binary, with few categorial and few numeric. I'm thinking about logistic, but which algorithm would be the best one ?

machine-learning project-planning

share|improve this question

asked 15 hours ago

Dan ChaltielDan Chaltiel

1113

New contributor

Dan Chaltiel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

2

$begingroup$

Coming from statistics, I'm freshly trying to learn machine learning. I've read a lot of tutorials about ML, but have no real training.

I'm working on a little project where my dataset have 6k lines and around 300 features.

As I've read in my tutorials, I split my dataset into a training sample (80%) and a testing sample (20%), and then train my algorithm on the training sample with cross-validation (5 folds).

As I re-ran my program twice (I've only tested KNN which I now know is quite not appropriate), I got really different results, with different sensitivity, specificity and precision.

I guess that if I re-run the program until metrics are good, my algorithm will be overfitted, and I also guess it would be because of the resample of test/training samples, but please correct me if I'm wrong.

If I'm going to try a lot of algorithms to see what I can get, should I fix my samples somewhere ? Is it even OK to do so ? (it would not always be in statistics)

In case it matters, I'm working with python's scikit-learn module.

*PS: my outcome is binary and my features are mostly binary, with few categorial and few numeric. I'm thinking about logistic, but which algorithm would be the best one ?

machine-learning project-planning

share|improve this question

asked 15 hours ago

Dan ChaltielDan Chaltiel

1113

New contributor

Dan Chaltiel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

Coming from statistics, I'm freshly trying to learn machine learning. I've read a lot of tutorials about ML, but have no real training.

I'm working on a little project where my dataset have 6k lines and around 300 features.

As I've read in my tutorials, I split my dataset into a training sample (80%) and a testing sample (20%), and then train my algorithm on the training sample with cross-validation (5 folds).

As I re-ran my program twice (I've only tested KNN which I now know is quite not appropriate), I got really different results, with different sensitivity, specificity and precision.

I guess that if I re-run the program until metrics are good, my algorithm will be overfitted, and I also guess it would be because of the resample of test/training samples, but please correct me if I'm wrong.

If I'm going to try a lot of algorithms to see what I can get, should I fix my samples somewhere ? Is it even OK to do so ? (it would not always be in statistics)

In case it matters, I'm working with python's scikit-learn module.

*PS: my outcome is binary and my features are mostly binary, with few categorial and few numeric. I'm thinking about logistic, but which algorithm would be the best one ?

machine-learning project-planning

share|improve this question

asked 15 hours ago

Dan ChaltielDan Chaltiel

1113

New contributor

Dan Chaltiel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|improve this question

asked 15 hours ago

Dan ChaltielDan Chaltiel

1113

New contributor

Dan Chaltiel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this question

asked 15 hours ago

Dan ChaltielDan Chaltiel

1113

New contributor

Dan Chaltiel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 15 hours ago

Dan ChaltielDan Chaltiel

1113

New contributor

Dan Chaltiel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 15 hours ago

Dan ChaltielDan Chaltiel

1113