Trjtdtk

What do I do when my TA workload is more than expected?

Deal with toxic manager when you can't quit

What to do when moving next to a bird sanctuary with a loosely-domesticated cat?

Can a flute soloist sit?

Nested ellipses in tikzpicture: Chomsky hierarchy

Why did Peik Lin say, "I'm not an animal"?

How to substitute curly brackets with round brackets in a grid of list

What other Star Trek series did the main TNG cast show up in?

Student Loan from years ago pops up and is taking my salary

How to design a circuit to convert 100 mV and 50 Hz Sine wave to square wave?

How to read αἱμύλιος or when to aspirate

Why don't hard Brexiteers insist on a hard border to prevent illegal immigration after Brexit?

Was credit for the black hole image misappropriated?

Is this wall load bearing? Blueprints and photos attached

What was the last x86 CPU that did not have the x87 floating-point unit built in?

Why are PDP-7-style microprogrammed instructions out of vogue?

Are spiders unable to hurt humans, especially very small spiders?

Can we generate random numbers using irrational numbers like π and e?

Can withdrawing asylum be illegal?

Is it ethical to upload a automatically generated paper to a non peer-reviewed site as part of a larger research?

Why can't devices on different VLANs, but on the same subnet, communicate?

Is it ok to offer lower paid work as a trial period before negotiating for a full-time job?

Can I visit the Trinity College (Cambridge) library and see some of their rare books

Am I ethically obligated to go into work on an off day if the reason is sudden?

Doc2vec most similar document to a query string

1

1

$begingroup$

I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title and abstract. Now, I want to do a search on all of the professors and find which professor is the most similar to a query string, such as "deep learning" or "computer networking". I have tried to use the infer_vector() to create a doc2vec representation of the query string using the already generated model and calculate the cosine similarity between the vectors. But I got terrible results. For example, when I search for "computer networking", it will give me the result of professor from History.
Is there any recommendation of how to find most similar document to a query string?

word2vec similarity natural-language-process information-retrieval similar-documents

share|improve this question

asked Mar 30 at 19:42

qiqiqiqi

61

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Welcome to this site! An easy-to-check alternative would be to use Euclidean distance instead of cosine similarity.
  $endgroup$
  – Esmailian
  Mar 30 at 19:57
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Hm, I tried the euclidean distance, but it gave me similar results to the cosine similarity methods I tried. Is it possible that my query string is too short to give good results?
  $endgroup$
  – qiqi
  Mar 30 at 23:15
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  You can go for an easier-to-pass evaluation to see how far off is the model. To this end, see if a good result shows up in top 3, 5, 10 closest matches. Also, keywords in a paper are way more important than the abstract according to your queries, place a special emphasis on them.
  $endgroup$
  – Esmailian
  Mar 31 at 7:58
  

  
  
  
  

add a comment | 

1

1

$begingroup$

I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title and abstract. Now, I want to do a search on all of the professors and find which professor is the most similar to a query string, such as "deep learning" or "computer networking". I have tried to use the infer_vector() to create a doc2vec representation of the query string using the already generated model and calculate the cosine similarity between the vectors. But I got terrible results. For example, when I search for "computer networking", it will give me the result of professor from History.
Is there any recommendation of how to find most similar document to a query string?

word2vec similarity natural-language-process information-retrieval similar-documents

share|improve this question

asked Mar 30 at 19:42

qiqiqiqi

61

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Welcome to this site! An easy-to-check alternative would be to use Euclidean distance instead of cosine similarity.
  $endgroup$
  – Esmailian
  Mar 30 at 19:57
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Hm, I tried the euclidean distance, but it gave me similar results to the cosine similarity methods I tried. Is it possible that my query string is too short to give good results?
  $endgroup$
  – qiqi
  Mar 30 at 23:15
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  You can go for an easier-to-pass evaluation to see how far off is the model. To this end, see if a good result shows up in top 3, 5, 10 closest matches. Also, keywords in a paper are way more important than the abstract according to your queries, place a special emphasis on them.
  $endgroup$
  – Esmailian
  Mar 31 at 7:58
  

  
  
  
  

add a comment | 

$begingroup$

I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title and abstract. Now, I want to do a search on all of the professors and find which professor is the most similar to a query string, such as "deep learning" or "computer networking". I have tried to use the infer_vector() to create a doc2vec representation of the query string using the already generated model and calculate the cosine similarity between the vectors. But I got terrible results. For example, when I search for "computer networking", it will give me the result of professor from History.
Is there any recommendation of how to find most similar document to a query string?

word2vec similarity natural-language-process information-retrieval similar-documents

share|improve this question

asked Mar 30 at 19:42

qiqiqiqi

61

$endgroup$

share|improve this question

asked Mar 30 at 19:42

qiqiqiqi

61

share|improve this question

asked Mar 30 at 19:42

qiqiqiqi

61

asked Mar 30 at 19:42

qiqiqiqi

61

asked Mar 30 at 19:42

qiqiqiqi

61