Trjtdtk

Has the laser at Magurele, Romania reached a tenth of the Sun's power?

Why is so much work done on numerical verification of the Riemann Hypothesis?

Is there any evidence that Cleopatra and Caesarion considered fleeing to India to escape the Romans?

How do you make your own symbol when Detexify fails?

Shouldn’t conservatives embrace universal basic income?

How to explain what's wrong with this application of the chain rule?

How to make money from a browser who sees 5 seconds into the future of any web page?

What is the English pronunciation of "pain au chocolat"?

What does "Scientists rise up against statistical significance" mean? (Comment in Nature)

Is there a nicer/politer/more positive alternative for "negates"?

Do we have to expect a queue for the shuttle from Watford Junction to Harry Potter Studio?

How would you translate "more" for use as an interface button?

"before" and "want" for the same systemd service?

Does "he squandered his car on drink" sound natural?

PTIJ: Why is Haman obsessed with Bose?

How much of a Devil Fruit must be consumed to gain the power?

Did the UK lift the requirement for registering SIM cards?

Quoting Keynes in a lecture

How do I fix the group tension caused by my character stealing and possibly killing without provocation?

Permission on Database

Has any country ever had 2 former presidents in jail simultaneously?

What is the highest possible scrabble score for placing a single tile

Doesn't the system of the Supreme Court oppose justice?

What does Apple's new App Store requirement mean

Scikit learn train test split without mixing participants in trails

How to split train/test in recommender systemsTraining Validation Testing set split for facial expression datasetConvolutional Neural Network not learning EEG dataFeatures standardization - Multilayer perceptronTraining score at parameter tuning lower than on hold out test set (RandomForestClassifier)Machine learning - 'train_test_split' function in scikit-learn: should I repeat it several times?Relationship between train and test errorHyper-parameter tuning when you don't have an access to the test dataOversampling before Cross-Validation, is it a problem?SciKit-Learn Decision Tree Overfitting

2

1

$begingroup$

I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.

I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.

The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?

machine-learning

share|improve this question

edited 2 days ago

Simon Larsson

51510

asked Mar 18 at 17:01

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What do you mean by "splitting based on participant?"
  $endgroup$
  – Ethan
  Mar 18 at 17:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
  $endgroup$
  – Andrew
  Mar 18 at 20:23
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
  $endgroup$
  – Simon Larsson
  2 days ago
  

  
  
  
  

add a comment | 

2

1

$begingroup$

I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.

I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.

The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?

machine-learning

share|improve this question

edited 2 days ago

Simon Larsson

51510

asked Mar 18 at 17:01

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What do you mean by "splitting based on participant?"
  $endgroup$
  – Ethan
  Mar 18 at 17:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @Ethan As in, I would like to split based on participant IDs and then classify exercises. For example, if I wanted to do a 50/50 train/test split, then I would have 25 participants data in the training, then test on the other 25 participants data.
  $endgroup$
  – Andrew
  Mar 18 at 20:23
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I tried editing to clear up some of the confusion about what you are asking. Maybe someone will be able to answer now, but I'm pretty sure it would be easier to implement this functionality yourself.
  $endgroup$
  – Simon Larsson
  2 days ago
  

  
  
  
  

add a comment | 

$begingroup$

I have dataset of trails. There are 90 participants that appear in 90 trails each. So 450 trails in total.

I'm looking to split my dataset so participants doesn't get mixed over training and test set. I want all trails of 25 participants in my training set and trails of the remaining 25 in the test set.

The reason is I want to test on totally unseen data to access overfitting. Is there a way of achieving this within scikit-learn?

machine-learning

share|improve this question

edited 2 days ago

Simon Larsson

51510

asked Mar 18 at 17:01

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|improve this question

edited 2 days ago

Simon Larsson

51510

asked Mar 18 at 17:01

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this question

edited 2 days ago

Simon Larsson

51510

asked Mar 18 at 17:01

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

edited 2 days ago

Simon Larsson

51510

edited 2 days ago

Simon Larsson

51510

asked Mar 18 at 17:01

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked Mar 18 at 17:01

AndrewAndrew

112

3 Answers
3

active

oldest

votes

2

$begingroup$

You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like

from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)

where groups is an array of group indices.

share|improve this answer

answered yesterday

oW_oW_

3,256730

$endgroup$

add a comment | 

1

$begingroup$

To split data into Training and Test sets in scikit-learn you can use the train_test_split function.

To implement this:

from sklearn.model_selection import train_test_split

sklearn.model_selection.train_test_split()

Then pass through the arguments that you would like to set as parameters. Some parameters of note you may like to tune are the sizes of the training and test sets as well as setting if the splits are shuffled or not.

You can read about this further on its scikit-learn documentation page here.

share|improve this answer

answered Mar 18 at 17:32

EthanEthan

576223

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yes, I am using that function. I don't think stratify is the correct parameter since all that is doing it looking to make sure proportions stay the same. I just want to split participant data as being used for training only or testing only.
  $endgroup$
  – Andrew
  Mar 18 at 22:09
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  train_test_split will by default, do a 75% training 25% testing split. Is this what you want to change? I am not sure I understand your question.
  $endgroup$
  – Ethan
  Mar 19 at 0:01
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  No, I want to split based on participants. Currently it randomly splits so all 50 participants are very likely going to be included in the training and testing portions.
  $endgroup$
  – Andrew
  2 days ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I want to do a 50% train split with the additional parameter that participants are mutually excluded to either the train or test split. So there are 25 participants worth of data on the train side and 25 participants worth of data on the test side. Basically, is there an easy way to randomize 50 participants into two groups? It seems like the answer is no and requires a custom script.
  $endgroup$
  – Andrew
  2 days ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What exactly do you mean by very likely to be included? If you have 50 samples, and pass them through train_test_split, all 50 samples will be included in either the training set or the test set. To your second comment, everything is mutually exclusive. No sample will be included in both the training and the test set.
  $endgroup$
  – Ethan
  2 days ago
  

  
  
  
  

 | 
show 4 more comments

0

$begingroup$

id_split = random.sample(range(50),25)
id_split.sort()

blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()

for i in range(50):
 if i in train_split:
# print("Training: ", (i+1))
 blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
 blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
 else:
# print("Testing: ", (i+1))
 blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
 blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])

I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.

share|improve this answer

edited yesterday

answered yesterday

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

Andrew is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47544%2fscikit-learn-train-test-split-without-mixing-participants-in-trails%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

2

$begingroup$

You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like

from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)

where groups is an array of group indices.

share|improve this answer

answered yesterday

oW_oW_

3,256730

$endgroup$

add a comment | 

2

$begingroup$

You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like

from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)

where groups is an array of group indices.

share|improve this answer

answered yesterday

oW_oW_

3,256730

$endgroup$

add a comment | 

$begingroup$

You can use one of scikit-learn's options for grouped data. In particular, GroupKFold should do the trick: something like

from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)

where groups is an array of group indices.

share|improve this answer

answered yesterday

oW_oW_

3,256730

$endgroup$

from sklearn.model_selection import GroupKFold
group_kfold = GroupKFold(n_splits=2)
group_kfold.get_n_splits(X, y, groups)

share|improve this answer

answered yesterday

oW_oW_

3,256730

answered yesterday

oW_oW_

3,256730

answered yesterday

oW_oW_

3,256730

1

$begingroup$

To split data into Training and Test sets in scikit-learn you can use the train_test_split function.

To implement this:

from sklearn.model_selection import train_test_split

sklearn.model_selection.train_test_split()

Then pass through the arguments that you would like to set as parameters. Some parameters of note you may like to tune are the sizes of the training and test sets as well as setting if the splits are shuffled or not.

You can read about this further on its scikit-learn documentation page here.

share|improve this answer

answered Mar 18 at 17:32

EthanEthan

576223

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yes, I am using that function. I don't think stratify is the correct parameter since all that is doing it looking to make sure proportions stay the same. I just want to split participant data as being used for training only or testing only.
  $endgroup$
  – Andrew
  Mar 18 at 22:09
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  train_test_split will by default, do a 75% training 25% testing split. Is this what you want to change? I am not sure I understand your question.
  $endgroup$
  – Ethan
  Mar 19 at 0:01
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  No, I want to split based on participants. Currently it randomly splits so all 50 participants are very likely going to be included in the training and testing portions.
  $endgroup$
  – Andrew
  2 days ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I want to do a 50% train split with the additional parameter that participants are mutually excluded to either the train or test split. So there are 25 participants worth of data on the train side and 25 participants worth of data on the test side. Basically, is there an easy way to randomize 50 participants into two groups? It seems like the answer is no and requires a custom script.
  $endgroup$
  – Andrew
  2 days ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What exactly do you mean by very likely to be included? If you have 50 samples, and pass them through train_test_split, all 50 samples will be included in either the training set or the test set. To your second comment, everything is mutually exclusive. No sample will be included in both the training and the test set.
  $endgroup$
  – Ethan
  2 days ago
  

  
  
  
  

 | 
show 4 more comments

1

$begingroup$

To split data into Training and Test sets in scikit-learn you can use the train_test_split function.

To implement this:

from sklearn.model_selection import train_test_split

sklearn.model_selection.train_test_split()

Then pass through the arguments that you would like to set as parameters. Some parameters of note you may like to tune are the sizes of the training and test sets as well as setting if the splits are shuffled or not.

You can read about this further on its scikit-learn documentation page here.

share|improve this answer

answered Mar 18 at 17:32

EthanEthan

576223

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yes, I am using that function. I don't think stratify is the correct parameter since all that is doing it looking to make sure proportions stay the same. I just want to split participant data as being used for training only or testing only.
  $endgroup$
  – Andrew
  Mar 18 at 22:09
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  train_test_split will by default, do a 75% training 25% testing split. Is this what you want to change? I am not sure I understand your question.
  $endgroup$
  – Ethan
  Mar 19 at 0:01
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  No, I want to split based on participants. Currently it randomly splits so all 50 participants are very likely going to be included in the training and testing portions.
  $endgroup$
  – Andrew
  2 days ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I want to do a 50% train split with the additional parameter that participants are mutually excluded to either the train or test split. So there are 25 participants worth of data on the train side and 25 participants worth of data on the test side. Basically, is there an easy way to randomize 50 participants into two groups? It seems like the answer is no and requires a custom script.
  $endgroup$
  – Andrew
  2 days ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What exactly do you mean by very likely to be included? If you have 50 samples, and pass them through train_test_split, all 50 samples will be included in either the training set or the test set. To your second comment, everything is mutually exclusive. No sample will be included in both the training and the test set.
  $endgroup$
  – Ethan
  2 days ago
  

  
  
  
  

 | 
show 4 more comments

$begingroup$

To split data into Training and Test sets in scikit-learn you can use the train_test_split function.

To implement this:

from sklearn.model_selection import train_test_split

sklearn.model_selection.train_test_split()

Then pass through the arguments that you would like to set as parameters. Some parameters of note you may like to tune are the sizes of the training and test sets as well as setting if the splits are shuffled or not.

You can read about this further on its scikit-learn documentation page here.

share|improve this answer

answered Mar 18 at 17:32

EthanEthan

576223

$endgroup$

from sklearn.model_selection import train_test_split

sklearn.model_selection.train_test_split()

share|improve this answer

answered Mar 18 at 17:32

EthanEthan

576223

answered Mar 18 at 17:32

EthanEthan

576223

answered Mar 18 at 17:32

EthanEthan

576223

0

$begingroup$

id_split = random.sample(range(50),25)
id_split.sort()

blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()

for i in range(50):
 if i in train_split:
# print("Training: ", (i+1))
 blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
 blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
 else:
# print("Testing: ", (i+1))
 blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
 blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])

I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.

share|improve this answer

edited yesterday

answered yesterday

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

0

$begingroup$

id_split = random.sample(range(50),25)
id_split.sort()

blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()

for i in range(50):
 if i in train_split:
# print("Training: ", (i+1))
 blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
 blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
 else:
# print("Testing: ", (i+1))
 blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
 blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])

I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.

share|improve this answer

edited yesterday

answered yesterday

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

id_split = random.sample(range(50),25)
id_split.sort()

blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()

for i in range(50):
 if i in train_split:
# print("Training: ", (i+1))
 blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
 blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
 else:
# print("Testing: ", (i+1))
 blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
 blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])

I just wrote a quick blurb to randomly select 25 participants out of the 50 participants. The data was already sorted to have the 90 trials from each participant consecutively (90 trials from ID1 followed by 90 trials from ID2, etc.). So I just appended to either train or test depending upon whether it was in the test or train split.

share|improve this answer

edited yesterday

answered yesterday

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

id_split = random.sample(range(50),25)
id_split.sort()

blinddata_train = pd.DataFrame()
blinddata_test = pd.DataFrame()
blindtarget_train = pd.DataFrame()
blindtarget_test = pd.DataFrame()

for i in range(50):
 if i in train_split:
# print("Training: ", (i+1))
 blinddata_train=blinddata_train.append(data[(90*i):(90*i+89)])
 blindtarget_train=blindtarget_train.append(target[(90*i):(90*i+89)])
 else:
# print("Testing: ", (i+1))
 blinddata_test=blinddata_test.append(data[(90*i):(90*i+89)])
 blinddtarget_test=blindtarget_test.append(target[(90*i):(90*i+89)])

share|improve this answer

edited yesterday

answered yesterday

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

answered yesterday

AndrewAndrew

112

New contributor

Andrew is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

answered yesterday

AndrewAndrew

112