Combiation of stemming and stop word removal consequences on standard errorsRegular Expressions in WordIncrease F1 score on a text corpusCan I create a word cloud of crowdfunding donors using word cloud?Can we apply community detection algorithms for word vector space?Word embedding vectors for keyphrase extractionBuild a relevancy scoring model of articles using NLPPlots with shaded standard deviationword/sentence alignment for English documentWord embeddings for Information Retrieval - Document search?Product classification in hierarchical categories based on multiple parameters and non-standard descriptions
Is a party consisting of only a bard, a cleric, and a warlock functional long-term?
In a future war, an old lady is trying to raise a boy but one of the weapons has made everyone deaf
Science-fiction short story where space navy wanted hospital ships and settlers had guns mounted everywhere
Why would a flight no longer considered airworthy be redirected like this?
Set readonly fields in a constructor local function c#
Does Mathematica reuse previous computations?
What does さっさ mean?
Is it normal that my co-workers at a fitness company criticize my food choices?
Why doesn't the EU now just force the UK to choose between referendum and no-deal?
Identifying the interval from A♭ to D♯
How to deal with taxi scam when on vacation?
Define, (actually define) the "stability" and "energy" of a compound
What's the difference between /ɪ/ and /i(ː)/?
How to read the value of this capacitor?
Co-worker team leader wants to inject his friend's awful software into our development. What should I say to our common boss?
Software described as 香ばしい
Professor being mistaken for a grad student
What approach do we need to follow for projects without a test environment?
Brexit - No Deal Rejection
At what level can a dragon innately cast its spells?
PTIJ: Who should I vote for? (21st Knesset Edition)
Most cost effective thermostat setting: consistent temperature vs. lowest temperature possible
Gravity magic - How does it work?
What do Xenomorphs eat in the Alien series?
Combiation of stemming and stop word removal consequences on standard errors
Regular Expressions in WordIncrease F1 score on a text corpusCan I create a word cloud of crowdfunding donors using word cloud?Can we apply community detection algorithms for word vector space?Word embedding vectors for keyphrase extractionBuild a relevancy scoring model of articles using NLPPlots with shaded standard deviationword/sentence alignment for English documentWord embeddings for Information Retrieval - Document search?Product classification in hierarchical categories based on multiple parameters and non-standard descriptions
$begingroup$
I've read an article of Greene, Ceron, Schumacher and Fazekas which called The Nuts and Bolts of Automated Text Analysis: Comparing Different Document Pre-Processing Techniques in FourCountries.
In this article, the authors state that using stemming and stop word removing separately decreases the standard errors of scaling estimations (compared to no-preprocessed texts). But when using the combination of them, it increases it.
So can you help me why using these techniques separately it increases the precision while using the combination it the S.E. go higher.
Thanks in advance, J
text-mining
New contributor
$endgroup$
add a comment |
$begingroup$
I've read an article of Greene, Ceron, Schumacher and Fazekas which called The Nuts and Bolts of Automated Text Analysis: Comparing Different Document Pre-Processing Techniques in FourCountries.
In this article, the authors state that using stemming and stop word removing separately decreases the standard errors of scaling estimations (compared to no-preprocessed texts). But when using the combination of them, it increases it.
So can you help me why using these techniques separately it increases the precision while using the combination it the S.E. go higher.
Thanks in advance, J
text-mining
New contributor
$endgroup$
add a comment |
$begingroup$
I've read an article of Greene, Ceron, Schumacher and Fazekas which called The Nuts and Bolts of Automated Text Analysis: Comparing Different Document Pre-Processing Techniques in FourCountries.
In this article, the authors state that using stemming and stop word removing separately decreases the standard errors of scaling estimations (compared to no-preprocessed texts). But when using the combination of them, it increases it.
So can you help me why using these techniques separately it increases the precision while using the combination it the S.E. go higher.
Thanks in advance, J
text-mining
New contributor
$endgroup$
I've read an article of Greene, Ceron, Schumacher and Fazekas which called The Nuts and Bolts of Automated Text Analysis: Comparing Different Document Pre-Processing Techniques in FourCountries.
In this article, the authors state that using stemming and stop word removing separately decreases the standard errors of scaling estimations (compared to no-preprocessed texts). But when using the combination of them, it increases it.
So can you help me why using these techniques separately it increases the precision while using the combination it the S.E. go higher.
Thanks in advance, J
text-mining
text-mining
New contributor
New contributor
New contributor
asked 7 hours ago
JudgeJudge
1
1
New contributor
New contributor
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "557"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Judge is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47333%2fcombiation-of-stemming-and-stop-word-removal-consequences-on-standard-errors%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Judge is a new contributor. Be nice, and check out our Code of Conduct.
Judge is a new contributor. Be nice, and check out our Code of Conduct.
Judge is a new contributor. Be nice, and check out our Code of Conduct.
Judge is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f47333%2fcombiation-of-stemming-and-stop-word-removal-consequences-on-standard-errors%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown