Choosing sample from large dataset?Python + Neural Nets + Large dimension Large DatasetHow to sample a statistically uniform datasetStrategy for dealing with giant sample sizeHow to Choose a Sample for Multiply ClassifiersHow to randomly sample crops from plain image with points only if crop contains n points inside?One hot encoding large datasetHow to implement feature selection for categorical variables (especially with many categories)?sample n unique items from datasetSub-sampling so that sample statistics match population statisticsSequence extraction in a dataset
Shouldn’t conservatives embrace universal basic income?
How do I tell my boss that I'm quitting soon, especially given that a colleague just left this week
Will number of steps recorded on FitBit/any fitness tracker add up distance in PokemonGo?
Why does Carol not get rid of the Kree symbol on her suit when she changes its colours?
How to convince somebody that he is fit for something else, but not this job?
Review your own paper in Mathematics
Is it necessary to use pronouns with the verb "essere"?
Is there any evidence that Cleopatra and Caesarion considered fleeing to India to escape the Romans?
Why the "ls" command is showing the permissions of files in a FAT32 partition?
How does electrical safety system work on ISS?
Do we have to expect a queue for the shuttle from Watford Junction to Harry Potter Studio?
How to make money from a browser who sees 5 seconds into the future of any web page?
Has the laser at Magurele, Romania reached a tenth of the Sun's power?
Did the UK lift the requirement for registering SIM cards?
Taxes on Dividends in a Roth IRA
US tourist/student visa
How to explain what's wrong with this application of the chain rule?
Can you use Vicious Mockery to win an argument or gain favours?
A variation to the phrase "hanging over my shoulders"
How much of a Devil Fruit must be consumed to gain the power?
PTIJ: Why is Haman obsessed with Bose?
Which Article Helped Get Rid of Technobabble in RPGs?
"before" and "want" for the same systemd service?
What does "Scientists rise up against statistical significance" mean? (Comment in Nature)
Choosing sample from large dataset?
Python + Neural Nets + Large dimension Large DatasetHow to sample a statistically uniform datasetStrategy for dealing with giant sample sizeHow to Choose a Sample for Multiply ClassifiersHow to randomly sample crops from plain image with points only if crop contains n points inside?One hot encoding large datasetHow to implement feature selection for categorical variables (especially with many categories)?sample n unique items from datasetSub-sampling so that sample statistics match population statisticsSequence extraction in a dataset
$begingroup$
How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?
python dataset sampling
asked 8 hours ago
Dishant KothiaDishant Kothia
1
$endgroup$
add a comment |
$begingroup$
How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?
python dataset sampling
asked 8 hours ago
Dishant KothiaDishant Kothia
1
$endgroup$
2
$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago
add a comment |
$begingroup$
How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?
python dataset sampling
asked 8 hours ago
Dishant KothiaDishant Kothia
1
$endgroup$
How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?
python dataset sampling
python dataset sampling
asked 8 hours ago
Dishant KothiaDishant Kothia
1
asked 8 hours ago
Dishant KothiaDishant Kothia
1
asked 8 hours ago
Dishant KothiaDishant Kothia
1
asked 8 hours ago
Dishant KothiaDishant Kothia
1
asked 8 hours ago
Dishant KothiaDishant Kothia
1
1
2
$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago
add a comment |
2
$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago
2
2
$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago
$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.
You can do:
df_unique = df.drop_duplicates()
df_sample = df.sample(n)
df_final = pd.concat([df_unique, df_sample], axis=0)
In the above code, n is the amount of sample you want.
In this way you can assure that every unique row is in your dataset and you have more samples on it.
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47743%2fchoosing-sample-from-large-dataset%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.
You can do:
df_unique = df.drop_duplicates()
df_sample = df.sample(n)
df_final = pd.concat([df_unique, df_sample], axis=0)
In the above code, n is the amount of sample you want.
In this way you can assure that every unique row is in your dataset and you have more samples on it.
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
$endgroup$
add a comment |
$begingroup$
Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.
You can do:
df_unique = df.drop_duplicates()
df_sample = df.sample(n)
df_final = pd.concat([df_unique, df_sample], axis=0)
In the above code, n is the amount of sample you want.
In this way you can assure that every unique row is in your dataset and you have more samples on it.
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
$endgroup$
add a comment |
$begingroup$
Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.
You can do:
df_unique = df.drop_duplicates()
df_sample = df.sample(n)
df_final = pd.concat([df_unique, df_sample], axis=0)
In the above code, n is the amount of sample you want.
In this way you can assure that every unique row is in your dataset and you have more samples on it.
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
$endgroup$
Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.
You can do:
df_unique = df.drop_duplicates()
df_sample = df.sample(n)
df_final = pd.concat([df_unique, df_sample], axis=0)
In the above code, n is the amount of sample you want.
In this way you can assure that every unique row is in your dataset and you have more samples on it.
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
answered 4 hours ago
Victor OliveiraVictor Oliveira
3407
3407
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47743%2fchoosing-sample-from-large-dataset%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago