Trjtdtk

Why limits give us the exact value of the slope of the tangent line?

Antipodal Land Area Calculation

Why is it faster to reheat something than it is to cook it?

Belief In God or Knowledge Of God. Which is better?

Maximum summed subsequences with non-adjacent items

If Windows 7 doesn't support WSL, then what is "Subsystem for UNIX-based Applications"?

Converted a Scalar function to a TVF function for parallel execution-Still running in Serial mode

How to pronounce 伝統色

Should there be a hyphen in the construction "IT affin"?

QGIS virtual layer functionality does not seem to support memory layers

What does Turing mean by this statement?

Why does it sometimes sound good to play a grace note as a lead in to a note in a melody?

Can a new player join a group only when a new campaign starts?

Why do early math courses focus on the cross sections of a cone and not on other 3D objects?

How to compare two different files line by line in unix?

Random body shuffle every night—can we still function?

Did Mueller's report provide an evidentiary basis for the claim of Russian govt election interference via social media?

Significance of Cersei's obsession with elephants?

Is it possible to force a specific program to remain in memory after closing it?

What happened to Thoros of Myr's flaming sword?

Is it possible for SQL statements to execute concurrently within a single session in SQL Server?

Would it be easier to apply for a UK visa if there is a host family to sponsor for you in going there?

Do wooden building fires get hotter than 600°C?

Why weren't discrete x86 CPUs ever used in game hardware?

How do I recommend items to out of training users based on its recent views?

1

$begingroup$

I used Spark's ALS implementation of matrix factorization (Collaborative Filtering for Implicit Feedback) to train user and item embeddings.

Since we have a lot of users in system, I had to sample some users to train model to avoid overfitting.

Now how do I construct user embeddings for out of training users. I tried constructing user embeddings by averaging item embeddings for user's items. But when I compared performance of average vector vs original user embeddings, it is not that great.

So how would I generate user embeddings using item matrix and rating matrix?

machine-learning recommender-system apache-spark embeddings matrix-factorisation

share|improve this question

asked Apr 2 at 20:00

Sn.Sn.

61

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Why do you sample users?. You wont overfit the model using more data...
  $endgroup$
  – lsmor
  Apr 8 at 11:22
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  We have more than 15 million users, so if I add all them spark ALS training, it crashes with out of memory error. So sampling is the best work around.
  $endgroup$
  – Sn.
  Apr 9 at 0:04
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What you are facing is the cold start problem. Essentialy there is no way to recommend products for new users besides using some default recommendations. There is something call fold-in but I am not sure if it works good. You can split your data in chunks, train the model for the first chunk, save it, and then retrain the model for the second chunk and so on. This way you don't need the whole data set in memory but training will take longer
  $endgroup$
  – lsmor
  Apr 9 at 8:48
  

  
  
  
  

add a comment | 

1

$begingroup$

I used Spark's ALS implementation of matrix factorization (Collaborative Filtering for Implicit Feedback) to train user and item embeddings.

Since we have a lot of users in system, I had to sample some users to train model to avoid overfitting.

Now how do I construct user embeddings for out of training users. I tried constructing user embeddings by averaging item embeddings for user's items. But when I compared performance of average vector vs original user embeddings, it is not that great.

So how would I generate user embeddings using item matrix and rating matrix?

machine-learning recommender-system apache-spark embeddings matrix-factorisation

share|improve this question

asked Apr 2 at 20:00

Sn.Sn.

61

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Why do you sample users?. You wont overfit the model using more data...
  $endgroup$
  – lsmor
  Apr 8 at 11:22
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  We have more than 15 million users, so if I add all them spark ALS training, it crashes with out of memory error. So sampling is the best work around.
  $endgroup$
  – Sn.
  Apr 9 at 0:04
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What you are facing is the cold start problem. Essentialy there is no way to recommend products for new users besides using some default recommendations. There is something call fold-in but I am not sure if it works good. You can split your data in chunks, train the model for the first chunk, save it, and then retrain the model for the second chunk and so on. This way you don't need the whole data set in memory but training will take longer
  $endgroup$
  – lsmor
  Apr 9 at 8:48
  

  
  
  
  

add a comment | 

$begingroup$

I used Spark's ALS implementation of matrix factorization (Collaborative Filtering for Implicit Feedback) to train user and item embeddings.

Since we have a lot of users in system, I had to sample some users to train model to avoid overfitting.

Now how do I construct user embeddings for out of training users. I tried constructing user embeddings by averaging item embeddings for user's items. But when I compared performance of average vector vs original user embeddings, it is not that great.

So how would I generate user embeddings using item matrix and rating matrix?

machine-learning recommender-system apache-spark embeddings matrix-factorisation

share|improve this question

asked Apr 2 at 20:00

Sn.Sn.

61

$endgroup$

share|improve this question

asked Apr 2 at 20:00

Sn.Sn.

61

share|improve this question

asked Apr 2 at 20:00

Sn.Sn.

61

asked Apr 2 at 20:00

Sn.Sn.

61

asked Apr 2 at 20:00

Sn.Sn.

61