Scikit learn train test split without mixing participants in trailsHow to split train/test in recommender systemsTraining Validation Testing set split for facial expression datasetConvolutional Neural Network not learning EEG dataFeatures standardization - Multilayer perceptronTraining score at parameter tuning lower than on hold out test set (RandomForestClassifier)Machine learning - 'train_test_split' function in scikit-learn: should I repeat it several times?Relationship between train and test errorHyper-parameter tuning when you don't have an access to the test dataOversampling before Cross-Validation, is it a problem?SciKit-Learn Decision Tree Overfitting
why `nmap 192.168.1.97` returns less services than `nmap 127.0.0.1`?
Is the U.S. Code copyrighted by the Government?
Why should universal income be universal?
Can the Supreme Court Overturn an Impeachment?
GraphicsGrid with a Label for each Column and Row
Writing bit difficult equation in latex
Redundant comparison & "if" before assignment
Are the IPv6 address space and IPv4 address space completely disjoint?
Strong empirical falsification of quantum mechanics based on vacuum energy density
Count the occurrence of each unique word in the file
What prevents the use of a multi-segment ILS for non-straight approaches?
The screen of my macbook suddenly broken down how can I do to recover
A question about fixed points and non-expansive map
Why does the Sun have different day lengths, but not the gas giants?
250 Floor Tower
Travelling outside the UK without a passport
What should you do when eye contact makes your subordinate uncomfortable?
Customize circled numbers
what is different between Do you interest vs interested in something?
What should you do if you miss a job interview (deliberately)?
When were female captains banned from Starfleet?
hline - width of entire table
Argument list too long when zipping large list of certain files in a folder
Why do we read the Megillah by night and by day?
Scikit learn train test split without mixing participants in trails
How to split train/test in recommender systemsTraining Validation Testing set split for facial expression datasetConvolutional Neural Network not learning EEG dataFeatures standardization - Multilayer perceptronTraining score at parameter tuning lower than on hold out test set (RandomForestClassifier)Machine learning - 'train_test_split' function in scikit-learn: should I repeat it several times?Relationship between train and test errorHyper-parameter tuning when you don't have an access to the test dataOversampling before Cross-Validation, is it a problem?SciKit-Learn Decision Tree Overfitting
$begingroup$
I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.
I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.
The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?
machine-learning
edited Mar 19 at 12:40
Simon Larsson
51910
asked Mar 18 at 17:01
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.
I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.
The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?
machine-learning
edited Mar 19 at 12:40
Simon Larsson
51910
asked Mar 18 at 17:01
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
$begingroup$
What do you mean by "splitting based on participant?"
$endgroup$
– Ethan
Mar 18 at 17:27
$begingroup$
@Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
$endgroup$
– Andrew
Mar 18 at 20:23
$begingroup$
I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
$endgroup$
– Simon Larsson
Mar 19 at 12:54
add a comment |
$begingroup$
I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.
I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.
The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?
machine-learning
edited Mar 19 at 12:40
Simon Larsson
51910
asked Mar 18 at 17:01
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.
I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.
The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?
machine-learning
machine-learning
edited Mar 19 at 12:40
Simon Larsson
51910
asked Mar 18 at 17:01
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited Mar 19 at 12:40
Simon Larsson
51910
asked Mar 18 at 17:01
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited Mar 19 at 12:40
Simon Larsson
51910
edited Mar 19 at 12:40
Simon Larsson
51910
edited Mar 19 at 12:40
Simon Larsson
51910
51910
asked Mar 18 at 17:01
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Mar 18 at 17:01
AndrewAndrew
112
asked Mar 18 at 17:01
AndrewAndrew
112
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$begingroup$
What do you mean by "splitting based on participant?"
$endgroup$
– Ethan
Mar 18 at 17:27
$begingroup$
@Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
$endgroup$
– Andrew
Mar 18 at 20:23
$begingroup$
I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
$endgroup$
– Simon Larsson
Mar 19 at 12:54
add a comment |
$begingroup$
What do you mean by "splitting based on participant?"
$endgroup$
– Ethan
Mar 18 at 17:27
$begingroup$
@Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
$endgroup$
– Andrew
Mar 18 at 20:23
$begingroup$
I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
$endgroup$
– Simon Larsson
Mar 19 at 12:54
$begingroup$
What do you mean by "splitting based on participant?"
$endgroup$
– Ethan
Mar 18 at 17:27
$begingroup$
What do you mean by "splitting based on participant?"
$endgroup$
– Ethan
Mar 18 at 17:27
$begingroup$
@Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
$endgroup$
– Andrew
Mar 18 at 20:23
$begingroup$
@Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
$endgroup$
– Andrew
Mar 18 at 20:23
$begingroup$
I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
$endgroup$
– Simon Larsson
Mar 19 at 12:54
$begingroup$
I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
$endgroup$
– Simon Larsson
Mar 19 at 12:54
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like
from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)
where groups is an array of group indices.
answered Mar 20 at 19:54
oW_oW_
3,261731
$endgroup$
add a comment |
$begingroup$
id_split = random.sample(range(50),25)
id_split.sort()
blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()
for i in range(50):
if i in train_split:
# print("Training: ", (i+1))
blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
else:
# print("Testing: ", (i+1))
blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])
I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.
answered Mar 20 at 19:48
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Andrew is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47544%2fscikit-learn-train-test-split-without-mixing-participants-in-trails%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like
from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)
where groups is an array of group indices.
answered Mar 20 at 19:54
oW_oW_
3,261731
$endgroup$
add a comment |
$begingroup$
You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like
from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)
where groups is an array of group indices.
answered Mar 20 at 19:54
oW_oW_
3,261731
$endgroup$
add a comment |
$begingroup$
You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like
from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)
where groups is an array of group indices.
answered Mar 20 at 19:54
oW_oW_
3,261731
$endgroup$
You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like
from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)
where groups is an array of group indices.
answered Mar 20 at 19:54
oW_oW_
3,261731
answered Mar 20 at 19:54
oW_oW_
3,261731
answered Mar 20 at 19:54
oW_oW_
3,261731
answered Mar 20 at 19:54
oW_oW_
3,261731
3,261731
add a comment |
add a comment |
$begingroup$
id_split = random.sample(range(50),25)
id_split.sort()
blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()
for i in range(50):
if i in train_split:
# print("Training: ", (i+1))
blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
else:
# print("Testing: ", (i+1))
blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])
I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.
answered Mar 20 at 19:48
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
id_split = random.sample(range(50),25)
id_split.sort()
blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()
for i in range(50):
if i in train_split:
# print("Training: ", (i+1))
blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
else:
# print("Testing: ", (i+1))
blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])
I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.
answered Mar 20 at 19:48
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
id_split = random.sample(range(50),25)
id_split.sort()
blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()
for i in range(50):
if i in train_split:
# print("Training: ", (i+1))
blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
else:
# print("Testing: ", (i+1))
blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])
I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.
answered Mar 20 at 19:48
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
id_split = random.sample(range(50),25)
id_split.sort()
blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()
for i in range(50):
if i in train_split:
# print("Training: ", (i+1))
blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
else:
# print("Testing: ", (i+1))
blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])
I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.
answered Mar 20 at 19:48
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited Mar 20 at 20:05
answered Mar 20 at 19:48
AndrewAndrew
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered Mar 20 at 19:48
AndrewAndrew
112
answered Mar 20 at 19:48
AndrewAndrew
112
112
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
Andrew is a new contributor. Be nice, and check out our Code of Conduct.
Andrew is a new contributor. Be nice, and check out our Code of Conduct.
Andrew is a new contributor. Be nice, and check out our Code of Conduct.
Andrew is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47544%2fscikit-learn-train-test-split-without-mixing-participants-in-trails%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
What do you mean by "splitting based on participant?"
$endgroup$
– Ethan
Mar 18 at 17:27
$begingroup$
@Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
$endgroup$
– Andrew
Mar 18 at 20:23
$begingroup$
I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
$endgroup$
– Simon Larsson
Mar 19 at 12:54