Trjtdtk

Greatest common substring

Identify a stage play about a VR experience in which participants are encouraged to simulate performing horrific activities

Difference between -| and |- in TikZ

Can a controlled ghast be a leader of a pack of ghouls?

What is the term when two people sing in harmony, but they aren't singing the same notes?

What is the opposite of 'gravitas'?

Why does this part of the Space Shuttle launch pad seem to be floating in air?

Do Legal Documents Require Signing In Standard Pen Colors?

How can "mimic phobia" be cured or prevented?

What does the "3am" section means in manpages?

Is camera lens focus an exact point or a range?

Did US corporations pay demonstrators in the German demonstrations against article 13?

Drawing ramified coverings with tikz

Proof of Lemma: Every integer can be written as a product of primes

Lightning Web Components - Not available in app builder

NIntegrate: How can I solve this integral numerically? NIntegrate fails while Integrate works

I2C signal and power over long range (10meter cable)

Female=gender counterpart?

A social experiment. What is the worst that can happen?

Giant Toughroad SLR 2 for 200 miles in two days, will it make it?

Find fails if filename contains brackets

Simulating a probability of 1 of 2^N with less than N random bits

What does this horizontal bar at the first measure mean?

Can a theist be saved?

How to visualize change in a distribution with a few outliers that account for a very large percent of the total?

Classification affected by a lot of outliers in features? How do you deal with outliers?How to visualise very large graphs with 250M nodes and 500M+ edges?How to decide for the contamination value (proportion of the outliers) in my dataset?How would you visualize data that comes in the millions of records?How to best visualize data when outliers lead to lack of contrasting colors for the rest of the plot?How to make multiple regression perform better for outliers? (without reducing effect of them)What are the differences between Glueviz and Orange that come installed with Anaconda and how do they compare on performance and workflows?

1

$begingroup$

I'm working on an edtech product where some of our traffic lands on webpages about textbooks.

Textbooks belong to subjects like Algebra, Calculus and Spanish. In each of our subjects, we have "whales" - individual books that account for a large percent (~20%) of total subject traffic.

Year over year, these whales grow or shrink (sometimes books are replaced in a school, or schools drop textbooks altogether). This change in whale traffic contributes to a big change in overall subject traffic.

I'm trying to figure out how to visualize this change, given a dataset that looks something like the table attached below (it shows only 2 months traffic per book, but I have access to all the months).

I've tried overlapping histograms (and boxplots), where each histogram is a month. But this visualization doesn't indicate how huge my whales (outliers) are, and how much influence they have.

Any help on chart types or how to otherwise tell this story would be much appreciated.

visualization outlier distribution

share|improve this question

asked Mar 19 at 19:31

samthebrandsamthebrand

1063

New contributor

samthebrand is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

1

$begingroup$

I'm working on an edtech product where some of our traffic lands on webpages about textbooks.

Textbooks belong to subjects like Algebra, Calculus and Spanish. In each of our subjects, we have "whales" - individual books that account for a large percent (~20%) of total subject traffic.

Year over year, these whales grow or shrink (sometimes books are replaced in a school, or schools drop textbooks altogether). This change in whale traffic contributes to a big change in overall subject traffic.

I'm trying to figure out how to visualize this change, given a dataset that looks something like the table attached below (it shows only 2 months traffic per book, but I have access to all the months).

I've tried overlapping histograms (and boxplots), where each histogram is a month. But this visualization doesn't indicate how huge my whales (outliers) are, and how much influence they have.

Any help on chart types or how to otherwise tell this story would be much appreciated.

visualization outlier distribution

share|improve this question

asked Mar 19 at 19:31

samthebrandsamthebrand

1063

New contributor

samthebrand is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I'm working on an edtech product where some of our traffic lands on webpages about textbooks.

Textbooks belong to subjects like Algebra, Calculus and Spanish. In each of our subjects, we have "whales" - individual books that account for a large percent (~20%) of total subject traffic.

Year over year, these whales grow or shrink (sometimes books are replaced in a school, or schools drop textbooks altogether). This change in whale traffic contributes to a big change in overall subject traffic.

I'm trying to figure out how to visualize this change, given a dataset that looks something like the table attached below (it shows only 2 months traffic per book, but I have access to all the months).

I've tried overlapping histograms (and boxplots), where each histogram is a month. But this visualization doesn't indicate how huge my whales (outliers) are, and how much influence they have.

Any help on chart types or how to otherwise tell this story would be much appreciated.

visualization outlier distribution

share|improve this question

asked Mar 19 at 19:31

samthebrandsamthebrand

1063

New contributor

samthebrand is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share|improve this question

asked Mar 19 at 19:31

samthebrandsamthebrand

1063

New contributor

samthebrand is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share|improve this question

asked Mar 19 at 19:31

samthebrandsamthebrand

1063

New contributor

samthebrand is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked Mar 19 at 19:31

samthebrandsamthebrand

1063

New contributor

samthebrand is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked Mar 19 at 19:31

samthebrandsamthebrand

1063

1 Answer
1

active

oldest

votes

2

$begingroup$

To visualize change in the size of multiple entities that are contributing to a total through time, e.g. total (t) = book_1 (t) + book_2 (t) + ..., we can use Stacked Area Plot. This plot can be used for normalized and un-normalized (absolute) values.

Preprocessing

1. For large number of entities, to avoid cognitive load, we can keep only
   those entities that become significant (whale) at some point in the plot, and group all the others under an "ordinary" entity. This way, cognitive load is minimized and only those entities that matter at some point are distinguished. For example, distinguishing books that have more than 10% of traffic at some point in the plotted time span.




2. If total fluctuation is very high, logarithm of values can be plugged into the plot.

share|improve this answer

edited Mar 20 at 21:37

answered Mar 19 at 19:57

EsmailianEsmailian

1,756115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  This is helpful. But these charts get very noisy with hundreds of observations per period. (We have hundreds of books per subject.) And it takes a lot of cognitive load, at a glance, to tell what's growing and what's shrinking. I do appreciate the suggestion!
  $endgroup$
  – samthebrand
  Mar 19 at 20:35
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @samthebrand I updated the answer for this problem
  $endgroup$
  – Esmailian
  Mar 19 at 21:03
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks @esmailian. This is a good answer, but I don't think it is THE answer, given that I'm interested in visualizing YOY change. For a stacked area chart I would have to look at latest period and then compare its relative size to the period twelve months prior. This feels hard. There's got to be a better way.
  $endgroup$
  – samthebrand
  Mar 19 at 21:31
  
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

samthebrand is a new contributor. Be nice, and check out our Code of Conduct.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47636%2fhow-to-visualize-change-in-a-distribution-with-a-few-outliers-that-account-for-a%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

2

$begingroup$

To visualize change in the size of multiple entities that are contributing to a total through time, e.g. total (t) = book_1 (t) + book_2 (t) + ..., we can use Stacked Area Plot. This plot can be used for normalized and un-normalized (absolute) values.

Preprocessing

1. For large number of entities, to avoid cognitive load, we can keep only
   those entities that become significant (whale) at some point in the plot, and group all the others under an "ordinary" entity. This way, cognitive load is minimized and only those entities that matter at some point are distinguished. For example, distinguishing books that have more than 10% of traffic at some point in the plotted time span.




2. If total fluctuation is very high, logarithm of values can be plugged into the plot.

share|improve this answer

edited Mar 20 at 21:37

answered Mar 19 at 19:57

EsmailianEsmailian

1,756115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  This is helpful. But these charts get very noisy with hundreds of observations per period. (We have hundreds of books per subject.) And it takes a lot of cognitive load, at a glance, to tell what's growing and what's shrinking. I do appreciate the suggestion!
  $endgroup$
  – samthebrand
  Mar 19 at 20:35
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @samthebrand I updated the answer for this problem
  $endgroup$
  – Esmailian
  Mar 19 at 21:03
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks @esmailian. This is a good answer, but I don't think it is THE answer, given that I'm interested in visualizing YOY change. For a stacked area chart I would have to look at latest period and then compare its relative size to the period twelve months prior. This feels hard. There's got to be a better way.
  $endgroup$
  – samthebrand
  Mar 19 at 21:31
  
  

  
  
  
  

add a comment | 

2

$begingroup$

To visualize change in the size of multiple entities that are contributing to a total through time, e.g. total (t) = book_1 (t) + book_2 (t) + ..., we can use Stacked Area Plot. This plot can be used for normalized and un-normalized (absolute) values.

Preprocessing

1. For large number of entities, to avoid cognitive load, we can keep only
   those entities that become significant (whale) at some point in the plot, and group all the others under an "ordinary" entity. This way, cognitive load is minimized and only those entities that matter at some point are distinguished. For example, distinguishing books that have more than 10% of traffic at some point in the plotted time span.




2. If total fluctuation is very high, logarithm of values can be plugged into the plot.

share|improve this answer

edited Mar 20 at 21:37

answered Mar 19 at 19:57

EsmailianEsmailian

1,756115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  This is helpful. But these charts get very noisy with hundreds of observations per period. (We have hundreds of books per subject.) And it takes a lot of cognitive load, at a glance, to tell what's growing and what's shrinking. I do appreciate the suggestion!
  $endgroup$
  – samthebrand
  Mar 19 at 20:35
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @samthebrand I updated the answer for this problem
  $endgroup$
  – Esmailian
  Mar 19 at 21:03
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks @esmailian. This is a good answer, but I don't think it is THE answer, given that I'm interested in visualizing YOY change. For a stacked area chart I would have to look at latest period and then compare its relative size to the period twelve months prior. This feels hard. There's got to be a better way.
  $endgroup$
  – samthebrand
  Mar 19 at 21:31
  
  

  
  
  
  

add a comment | 

$begingroup$

To visualize change in the size of multiple entities that are contributing to a total through time, e.g. total (t) = book_1 (t) + book_2 (t) + ..., we can use Stacked Area Plot. This plot can be used for normalized and un-normalized (absolute) values.

Preprocessing

1. For large number of entities, to avoid cognitive load, we can keep only
   those entities that become significant (whale) at some point in the plot, and group all the others under an "ordinary" entity. This way, cognitive load is minimized and only those entities that matter at some point are distinguished. For example, distinguishing books that have more than 10% of traffic at some point in the plotted time span.




2. If total fluctuation is very high, logarithm of values can be plugged into the plot.

share|improve this answer

edited Mar 20 at 21:37

answered Mar 19 at 19:57

EsmailianEsmailian

1,756115

$endgroup$

share|improve this answer

edited Mar 20 at 21:37

answered Mar 19 at 19:57

EsmailianEsmailian

1,756115

answered Mar 19 at 19:57

EsmailianEsmailian

1,756115

answered Mar 19 at 19:57

EsmailianEsmailian

1,756115