Choosing sample from large dataset?Python + Neural Nets + Large dimension Large DatasetHow to sample a statistically uniform datasetStrategy for dealing with giant sample sizeHow to Choose a Sample for Multiply ClassifiersHow to randomly sample crops from plain image with points only if crop contains n points inside?One hot encoding large datasetHow to implement feature selection for categorical variables (especially with many categories)?sample n unique items from datasetSub-sampling so that sample statistics match population statisticsSequence extraction in a dataset

Shouldn’t conservatives embrace universal basic income?

How do I tell my boss that I'm quitting soon, especially given that a colleague just left this week

Will number of steps recorded on FitBit/any fitness tracker add up distance in PokemonGo?

Why does Carol not get rid of the Kree symbol on her suit when she changes its colours?

How to convince somebody that he is fit for something else, but not this job?

Review your own paper in Mathematics

Is it necessary to use pronouns with the verb "essere"?

Is there any evidence that Cleopatra and Caesarion considered fleeing to India to escape the Romans?

Why the "ls" command is showing the permissions of files in a FAT32 partition?

How does electrical safety system work on ISS?

Do we have to expect a queue for the shuttle from Watford Junction to Harry Potter Studio?

How to make money from a browser who sees 5 seconds into the future of any web page?

Has the laser at Magurele, Romania reached a tenth of the Sun's power?

Did the UK lift the requirement for registering SIM cards?

Taxes on Dividends in a Roth IRA

US tourist/student visa

How to explain what's wrong with this application of the chain rule?

Can you use Vicious Mockery to win an argument or gain favours?

A variation to the phrase "hanging over my shoulders"

How much of a Devil Fruit must be consumed to gain the power?

PTIJ: Why is Haman obsessed with Bose?

Which Article Helped Get Rid of Technobabble in RPGs?

"before" and "want" for the same systemd service?

What does "Scientists rise up against statistical significance" mean? (Comment in Nature)



Choosing sample from large dataset?


Python + Neural Nets + Large dimension Large DatasetHow to sample a statistically uniform datasetStrategy for dealing with giant sample sizeHow to Choose a Sample for Multiply ClassifiersHow to randomly sample crops from plain image with points only if crop contains n points inside?One hot encoding large datasetHow to implement feature selection for categorical variables (especially with many categories)?sample n unique items from datasetSub-sampling so that sample statistics match population statisticsSequence extraction in a dataset













0












$begingroup$


How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling







share|improve this question









$endgroup$







  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago















0












$begingroup$


How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling







share|improve this question









$endgroup$







  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago













0












0








0





$begingroup$


How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling







share|improve this question









$endgroup$




How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?






python dataset sampling




python dataset sampling






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 8 hours ago









Dishant KothiaDishant Kothia

1




1







  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago












  • 2




    $begingroup$
    It is hard to understand what you are asking. Could you rephrase the question?
    $endgroup$
    – Simon Larsson
    7 hours ago







2




2




$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago




$begingroup$
It is hard to understand what you are asking. Could you rephrase the question?
$endgroup$
– Simon Larsson
7 hours ago










1 Answer
1






active

oldest

votes


















0












$begingroup$

Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



You can do:



df_unique = df.drop_duplicates()
df_sample = df.sample(n)

df_final = pd.concat([df_unique, df_sample], axis=0)


In the above code, n is the amount of sample you want.
In this way you can assure that every unique row is in your dataset and you have more samples on it.






share|improve this answer









$endgroup$












    Your Answer





    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("mathjaxEditing", function ()
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    );
    );
    , "mathjax-editing");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "557"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47743%2fchoosing-sample-from-large-dataset%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0












    $begingroup$

    Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



    You can do:



    df_unique = df.drop_duplicates()
    df_sample = df.sample(n)

    df_final = pd.concat([df_unique, df_sample], axis=0)


    In the above code, n is the amount of sample you want.
    In this way you can assure that every unique row is in your dataset and you have more samples on it.






    share|improve this answer









    $endgroup$

















      0












      $begingroup$

      Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



      You can do:



      df_unique = df.drop_duplicates()
      df_sample = df.sample(n)

      df_final = pd.concat([df_unique, df_sample], axis=0)


      In the above code, n is the amount of sample you want.
      In this way you can assure that every unique row is in your dataset and you have more samples on it.






      share|improve this answer









      $endgroup$















        0












        0








        0





        $begingroup$

        Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



        You can do:



        df_unique = df.drop_duplicates()
        df_sample = df.sample(n)

        df_final = pd.concat([df_unique, df_sample], axis=0)


        In the above code, n is the amount of sample you want.
        In this way you can assure that every unique row is in your dataset and you have more samples on it.






        share|improve this answer









        $endgroup$



        Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.



        You can do:



        df_unique = df.drop_duplicates()
        df_sample = df.sample(n)

        df_final = pd.concat([df_unique, df_sample], axis=0)


        In the above code, n is the amount of sample you want.
        In this way you can assure that every unique row is in your dataset and you have more samples on it.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered 4 hours ago









        Victor OliveiraVictor Oliveira

        3407




        3407



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Data Science Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47743%2fchoosing-sample-from-large-dataset%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Marja Vauras Lähteet | Aiheesta muualla | NavigointivalikkoMarja Vauras Turun yliopiston tutkimusportaalissaInfobox OKSuomalaisen Tiedeakatemian varsinaiset jäsenetKasvatustieteiden tiedekunnan dekaanit ja muu johtoMarja VaurasKoulutusvienti on kestävyys- ja ketteryyslaji (2.5.2017)laajentamallaWorldCat Identities0000 0001 0855 9405n86069603utb201588738523620927

            Image
            Vuonna 1953 syntyneetSuomalaiset kasvatustieteilijätSuomalaiset professoritTurun yliopiston professorit 9. maaliskuuta1953TurkuTurun yliopistonSuomalaisen tiedeakatemian Marja Vauras Henkilötiedot Syntynyt 9. maaliskuuta 1953 (ikä 66) Turku Kansallisuus Suomi Ammatti kasvatustieteen professori Koulutus ja ura Instituutti Turun yliopisto Aiheesta muualla Marja Vauras Turun yliopiston tutkimusportaalissa Infobox OK Marja Merja Sinikka Vauras (s. 9. maaliskuuta 1953 Turku) on suomalainen filosofian tohtori, ja Turun yliopiston kasvatustieteen professori. [1] [2] Hän on toiminut myös Turun yliopiston Kasvatustieteellisen tiedekunnan dekaanina. [3] Suomalaisen tiedeakatemian jäsen Vauras on ollut vuodesta 2013. [2] Lähteet | ↑ Suomen professorit 1640–2007 , Professoriliitto, 2008 (ISBN  978-952-99281-1-8 ja 978-952-99281-2-5 , viitattu 7 huhtikuuta 2019 ) ↑ a b Suomalaisen Tiedeakatemian varsinaiset jäsenet pä...
            Read more

            Which is better: GPT or RelGAN for text generation?2019 Community Moderator ElectionWhat is the difference between TextGAN and LM for text generation?GANs (generative adversarial networks) possible for text as well?Generator loss not decreasing- text to image synthesisChoosing a right algorithm for template-based text generationHow should I format input and output for text generation with LSTMsGumbel Softmax vs Vanilla Softmax for GAN trainingWhich neural network to choose for classification from text/speech?NLP text autoencoder that generates text in poetic meterWhat is the interpretation of the expectation notation in the GAN formulation?What is the difference between TextGAN and LM for text generation?How to prepare the data for text generation task

            Image
            Neighboring nodes in the network Is it canonical bit space? Why do I get two different answers for this counting problem? Python: return float 1.0 as int 1 but float 1.5 as float 1.5 Drawing a Grid/Tikz image Why is it a bad idea to hire a hitman to eliminate most corrupt politicians? Why is this clock signal connected to a capacitor to gnd? Blender 2.8 I can't see vertices, edges or faces in edit mode How to say in German "enjoying home comforts" Is there an expression that means doing something right before you will need it rather than doing it in case you might need it? Can a virus destroy the BIOS of a modern computer? Saudi Arabia Transit Visa Why "Having chlorophyll without photosynthesis is actually very dangerous" and "like living with a bomb"? What is the word for reserving something for yourself before others do? Why do bosons tend to occupy the same state? Is the Joker left-handed? What method can I use to des...
            Read more

            Is this part of the description of the Archfey warlock's Misty Escape feature redundant?When is entropic ward considered “used”?How does the reaction timing work for Wrath of the Storm? Can it potentially prevent the damage from the triggering attack?Does the Dark Arts Archlich warlock patrons's Arcane Invisibility activate every time you cast a level 1+ spell?When attacking while invisible, when exactly does invisibility break?Can I cast Hellish Rebuke on my turn?Do I have to “pre-cast” a reaction spell in order for it to be triggered?What happens if a Player Misty Escapes into an Invisible CreatureCan a reaction interrupt multiattack?Does the Fiend-patron warlock's Hurl Through Hell feature dispel effects that require the target to be on the same plane as the caster?What are you allowed to do while using the Warlock's Eldritch Master feature?

            Image
            How to Reset Passwords on Multiple Websites Easily? How can a function with a hole (removable discontinuity) equal a function with no hole? What is the opposite of 'gravitas'? Return the Closest Prime Number Term for the "extreme-extension" version of a straw man fallacy? How to check is there any negative term in a large list? How does it work when somebody invests in my business? How do I rename a Linux host without needing to reboot for the rename to take effect? Avoiding estate tax by giving multiple gifts Purchasing a ticket for someone else in another country? Energy of the particles in the particle accelerator How can I get through very long and very dry, but also very useful technical documents when learning a new tool? How did Doctor Strange see the winning outcome in Avengers: Infinity War? How to create a 32-bit integer from eight (8) 4-bit integers? What can we do to stop prior company from asking us questions? How to be diplo...
            Read more