Choosing sample from large dataset?Python + Neural Nets + Large dimension Large DatasetHow to sample a statistically uniform datasetStrategy for dealing with giant sample sizeHow to Choose a Sample for Multiply ClassifiersHow to randomly sample crops from plain image with points only if crop contains n points inside?One hot encoding large datasetHow to implement feature selection for categorical variables (especially with many categories)?sample n unique items from datasetSub-sampling so that sample statistics match population statisticsSequence extraction in a dataset

Shouldn’t conservatives embrace universal basic income?

How do I tell my boss that I'm quitting soon, especially given that a colleague just left this week

Will number of steps recorded on FitBit/any fitness tracker add up distance in PokemonGo?

Why does Carol not get rid of the Kree symbol on her suit when she changes its colours?

How to convince somebody that he is fit for something else, but not this job?

Review your own paper in Mathematics

Is it necessary to use pronouns with the verb "essere"?

Is there any evidence that Cleopatra and Caesarion considered fleeing to India to escape the Romans?

Why the "ls" command is showing the permissions of files in a FAT32 partition?

How does electrical safety system work on ISS?

Do we have to expect a queue for the shuttle from Watford Junction to Harry Potter Studio?

How to make money from a browser who sees 5 seconds into the future of any web page?

Has the laser at Magurele, Romania reached a tenth of the Sun's power?

Did the UK lift the requirement for registering SIM cards?

Taxes on Dividends in a Roth IRA

US tourist/student visa

How to explain what's wrong with this application of the chain rule?

Can you use Vicious Mockery to win an argument or gain favours?

A variation to the phrase "hanging over my shoulders"

How much of a Devil Fruit must be consumed to gain the power?

PTIJ: Why is Haman obsessed with Bose?

Which Article Helped Get Rid of Technobabble in RPGs?

"before" and "want" for the same systemd service?

What does "Scientists rise up against statistical significance" mean? (Comment in Nature)



Choosing sample from large dataset?


Python + Neural Nets + Large dimension Large DatasetHow to sample a statistically uniform datasetStrategy for dealing with giant sample sizeHow to Choose a Sample for Multiply ClassifiersHow to randomly sample crops from plain image with points only if crop contains n points inside?One hot encoding large datasetHow to implement feature selection for categorical variables (especially with many categories)?sample n unique items from datasetSub-sampling so that sample statistics match population statisticsSequence extraction in a dataset













0












$begingroup$


How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling







share|improve this question









$endgroup$







  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago















0












$begingroup$


How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling







share|improve this question









$endgroup$







  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago













0












0








0





$begingroup$


How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling







share|improve this question









$endgroup$




How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling




python dataset sampling






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 8 hours ago









Dishant KothiaDishant Kothia

1




1







  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago












  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago







2




2




$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago




$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago










1 Answer
1






active

oldest

votes


















0












$begingroup$

Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



You can do:



df_unique = df.drop_duplicates()
df_sample = df.sample(n)

df_final = pd.concat([df_unique, df_sample], axis=0)


In the above code, n is the amount of sample you want.
In this way you can assure that every unique row is in your dataset and you have more samples on it.






share|improve this answer









$endgroup$












    Your Answer





    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("mathjaxEditing", function ()
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    );
    );
    , "mathjax-editing");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "557"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47743%2fchoosing-sample-from-large-dataset%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0












    $begingroup$

    Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



    You can do:



    df_unique = df.drop_duplicates()
    df_sample = df.sample(n)

    df_final = pd.concat([df_unique, df_sample], axis=0)


    In the above code, n is the amount of sample you want.
    In this way you can assure that every unique row is in your dataset and you have more samples on it.






    share|improve this answer









    $endgroup$

















      0












      $begingroup$

      Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



      You can do:



      df_unique = df.drop_duplicates()
      df_sample = df.sample(n)

      df_final = pd.concat([df_unique, df_sample], axis=0)


      In the above code, n is the amount of sample you want.
      In this way you can assure that every unique row is in your dataset and you have more samples on it.






      share|improve this answer









      $endgroup$















        0












        0








        0





        $begingroup$

        Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



        You can do:



        df_unique = df.drop_duplicates()
        df_sample = df.sample(n)

        df_final = pd.concat([df_unique, df_sample], axis=0)


        In the above code, n is the amount of sample you want.
        In this way you can assure that every unique row is in your dataset and you have more samples on it.






        share|improve this answer









        $endgroup$



        Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



        You can do:



        df_unique = df.drop_duplicates()
        df_sample = df.sample(n)

        df_final = pd.concat([df_unique, df_sample], axis=0)


        In the above code, n is the amount of sample you want.
        In this way you can assure that every unique row is in your dataset and you have more samples on it.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered 4 hours ago









        Victor OliveiraVictor Oliveira

        3407




        3407



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Data Science Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47743%2fchoosing-sample-from-large-dataset%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Marja Vauras Lähteet | Aiheesta muualla | NavigointivalikkoMarja Vauras Turun yliopiston tutkimusportaalissaInfobox OKSuomalaisen Tiedeakatemian varsinaiset jäsenetKasvatustieteiden tiedekunnan dekaanit ja muu johtoMarja VaurasKoulutusvienti on kestävyys- ja ketteryyslaji (2.5.2017)laajentamallaWorldCat Identities0000 0001 0855 9405n86069603utb201588738523620927

            Image
            Vuonna 1953 syntyneetSuomalaiset kasvatustieteilijätSuomalaiset professoritTurun yliopiston professorit 9. maaliskuuta1953TurkuTurun yliopistonSuomalaisen tiedeakatemian Marja Vauras Henkilötiedot Syntynyt 9. maaliskuuta 1953 (ikä 66) Turku Kansallisuus Suomi Ammatti kasvatustieteen professori Koulutus ja ura Instituutti Turun yliopisto Aiheesta muualla Marja Vauras Turun yliopiston tutkimusportaalissa Infobox OK Marja Merja Sinikka Vauras (s. 9. maaliskuuta 1953 Turku) on suomalainen filosofian tohtori, ja Turun yliopiston kasvatustieteen professori. [1] [2] Hän on toiminut myös Turun yliopiston Kasvatustieteellisen tiedekunnan dekaanina. [3] Suomalaisen tiedeakatemian jäsen Vauras on ollut vuodesta 2013. [2] Lähteet | ↑ Suomen professorit 1640–2007 , Professoriliitto, 2008 (ISBN  978-952-99281-1-8 ja 978-952-99281-2-5 , viitattu 7 huhtikuuta 2019 ) ↑ a b Suomalaisen Tiedeakatemian varsinaiset jäsenet pä...
            Read more

            Which is better: GPT or RelGAN for text generation?2019 Community Moderator ElectionWhat is the difference between TextGAN and LM for text generation?GANs (generative adversarial networks) possible for text as well?Generator loss not decreasing- text to image synthesisChoosing a right algorithm for template-based text generationHow should I format input and output for text generation with LSTMsGumbel Softmax vs Vanilla Softmax for GAN trainingWhich neural network to choose for classification from text/speech?NLP text autoencoder that generates text in poetic meterWhat is the interpretation of the expectation notation in the GAN formulation?What is the difference between TextGAN and LM for text generation?How to prepare the data for text generation task

            Image
            Neighboring nodes in the network Is it canonical bit space? Why do I get two different answers for this counting problem? Python: return float 1.0 as int 1 but float 1.5 as float 1.5 Drawing a Grid/Tikz image Why is it a bad idea to hire a hitman to eliminate most corrupt politicians? Why is this clock signal connected to a capacitor to gnd? Blender 2.8 I can't see vertices, edges or faces in edit mode How to say in German "enjoying home comforts" Is there an expression that means doing something right before you will need it rather than doing it in case you might need it? Can a virus destroy the BIOS of a modern computer? Saudi Arabia Transit Visa Why "Having chlorophyll without photosynthesis is actually very dangerous" and "like living with a bomb"? What is the word for reserving something for yourself before others do? Why do bosons tend to occupy the same state? Is the Joker left-handed? What method can I use to des...
            Read more

            Is flight data recorder erased after every flight?When are black boxes used?What protects the location beacon (pinger) of a flight data recorder?Is there anywhere I can pick up raw flight data recorder information?Who legally owns the Flight Data Recorder?Constructing flight recorder dataWhy are FDRs and CVRs still two separate physical devices?What are the data elements shown on the GE235 flight data recorder (FDR) plot?Are CVR and FDR reset after every flight?What is the format of data stored by a Flight Data Recorder?How much data is stored in the flight data recorder per hour in a typical flight of an A380?Is a smart flight data recorder possible?

            Image
            Why are prions in animal diets not destroyed by the digestive system? How to increase the size of the cursor in Lubuntu 19.04? As matter approaches a black hole, does it speed up? Why is B♯ higher than C♭ in 31-ET? Nominativ or Akkusativ What does this colon mean? It is not labeling, it is not ternary operator Didn't attend field-specific conferences during my PhD; how much of a disadvantage is it? Applying a GPO to local users except local administrators on Workgroup computers Why has the UK chosen to use Huawei infrastructure when Five Eyes allies haven't? Using column size much larger than necessary Can my company stop me from working overtime? Will 700 more planes a day fly because of the Heathrow expansion? How to use dependency injection and avoid temporal coupling? Point of the Dothraki's attack in GoT S8E3? How I can I roll a number of non-digital dice to get a random number between 1 and 150? Validation rule Scheduled Apex I need a d...
            Read more