Trjtdtk

Is `x >> pure y` equivalent to `liftM (const y) x`

How can I kill an app using Terminal?

Opposite of a diet

What is paid subscription needed for in Mortal Kombat 11?

Is there a problem with hiding "forgot password" until it's needed?

How to Reset Passwords on Multiple Websites Easily?

Is a stroke of luck acceptable after a series of unfavorable events?

Inappropriate reference requests from Journal reviewers

Applicability of Single Responsibility Principle

Is this apparent Class Action settlement a spam message?

How do I extract a value from a time formatted value in excel?

Is the destination of a commercial flight important for the pilot?

I'm in charge of equipment buying but no one's ever happy with what I choose. How to fix this?

Return the Closest Prime Number

How does the UK government determine the size of a mandate?

Unreliable Magic - Is it worth it?

Do sorcerers' Subtle Spells require a skill check to be unseen?

Failed to fetch jessie backports repository

How to be diplomatic in refusing to write code that breaches the privacy of our users

Pole-zeros of a real-valued causal FIR system

How do we know the LHC results are robust?

Different result between scanning in Epson's "color negative film" mode and scanning in positive -> invert curve in post?

Is oxalic acid dihydrate considered a primary acid standard in analytical chemistry?

Anatomically Correct Strange Women In Ponds Distributing Swords

NLP - How to detect the presence of a phrase and it's derivatives

0

0

$begingroup$

I have a dataset with a free form text field as one of the variables. Essentially I want to determine if a record has the phrase "The cat is not present". However, this phrase could be written as "cat is not present", "cat- not present", "There is no cat", "cat: not present", "no cat here to report", "report: no cat", and many other derivatives. I also want to exclude situations like "I was outside playing with my friend Bob. It was sunny. It was warm. Cat was not present. Overall, it was a good day" because this has "useful" context.

The end goal is to calculate the number of records that has "no cat is present" (minus instances where this phrase or derivatives has context) vs "cat is present"

machine-learning python nlp

share|improve this question

edited Mar 22 at 20:25

DataNoob7

asked Mar 20 at 22:22

DataNoob7DataNoob7

193

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  This is surely an NLP problem. Your last sentence however made it slightly difficult, and make it understand context. One easy solution that comes to my mind is: 1. do a binary classification with a lot of such labeled data. 2. Another solution can be do a multi class classification. There are codes avaibable on internet. Both of these solutions are simple. 3. Third solution can be to some use some attention or semantic similarity kind of solution.
  $endgroup$
  – Sandeep B
  Mar 21 at 13:02
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for the reply. I am still a beginner in applying these types of solutions, so it might not be so simple! That being said, I would rather not copy and paste, and tweak. I would like to learn it and write it myself. Please excuse my ignorance, but is there more formal names for these type of NLP techniques? I would like to use the method that provides the most confidence (which is likely your third option). I don't think I will need to create a test / training set because we are using the whole population for a specific period of time.
  $endgroup$
  – DataNoob7
  Mar 21 at 20:13
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I have looked up solutions 1 and 2, but I don't think it answers the question due to the variety of which these instances can appear. Unless I am misunderstanding.
  $endgroup$
  – DataNoob7
  2 days ago
  

  
  
  
  

add a comment | 

0

0

$begingroup$

I have a dataset with a free form text field as one of the variables. Essentially I want to determine if a record has the phrase "The cat is not present". However, this phrase could be written as "cat is not present", "cat- not present", "There is no cat", "cat: not present", "no cat here to report", "report: no cat", and many other derivatives. I also want to exclude situations like "I was outside playing with my friend Bob. It was sunny. It was warm. Cat was not present. Overall, it was a good day" because this has "useful" context.

The end goal is to calculate the number of records that has "no cat is present" (minus instances where this phrase or derivatives has context) vs "cat is present"

machine-learning python nlp

share|improve this question

edited Mar 22 at 20:25

DataNoob7

asked Mar 20 at 22:22

DataNoob7DataNoob7

193

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  This is surely an NLP problem. Your last sentence however made it slightly difficult, and make it understand context. One easy solution that comes to my mind is: 1. do a binary classification with a lot of such labeled data. 2. Another solution can be do a multi class classification. There are codes avaibable on internet. Both of these solutions are simple. 3. Third solution can be to some use some attention or semantic similarity kind of solution.
  $endgroup$
  – Sandeep B
  Mar 21 at 13:02
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for the reply. I am still a beginner in applying these types of solutions, so it might not be so simple! That being said, I would rather not copy and paste, and tweak. I would like to learn it and write it myself. Please excuse my ignorance, but is there more formal names for these type of NLP techniques? I would like to use the method that provides the most confidence (which is likely your third option). I don't think I will need to create a test / training set because we are using the whole population for a specific period of time.
  $endgroup$
  – DataNoob7
  Mar 21 at 20:13
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I have looked up solutions 1 and 2, but I don't think it answers the question due to the variety of which these instances can appear. Unless I am misunderstanding.
  $endgroup$
  – DataNoob7
  2 days ago
  

  
  
  
  

add a comment | 

$begingroup$

I have a dataset with a free form text field as one of the variables. Essentially I want to determine if a record has the phrase "The cat is not present". However, this phrase could be written as "cat is not present", "cat- not present", "There is no cat", "cat: not present", "no cat here to report", "report: no cat", and many other derivatives. I also want to exclude situations like "I was outside playing with my friend Bob. It was sunny. It was warm. Cat was not present. Overall, it was a good day" because this has "useful" context.

The end goal is to calculate the number of records that has "no cat is present" (minus instances where this phrase or derivatives has context) vs "cat is present"

machine-learning python nlp

share|improve this question

edited Mar 22 at 20:25

DataNoob7

asked Mar 20 at 22:22

DataNoob7DataNoob7

193

$endgroup$

share|improve this question

edited Mar 22 at 20:25

DataNoob7

asked Mar 20 at 22:22

DataNoob7DataNoob7

193

share|improve this question

edited Mar 22 at 20:25

DataNoob7

asked Mar 20 at 22:22

DataNoob7DataNoob7

193

asked Mar 20 at 22:22

DataNoob7DataNoob7

193

asked Mar 20 at 22:22

DataNoob7DataNoob7

193