Evaluate recommendations after training a model

Question

First of all, I would like to create a recommender system. With the help of a neural.network, this is supposed to make a prediction of which articles user X is most likely to buy.

I have already trained a model with the right datasets and the help of the neuMF model (you can also look at the different layers in the picture).

[Source https://arxiv.org/abs/1708.05031]

My dataset contains the following:

The column event contains whether the user has looked at an item (view), placed it in the shopping cart (addtocart) or bought it (transaction).

I have already found example implementations of how they determine the recommendations. The following was written about it:

Now that I've trained my model, I'm ready to recommend songs for a given playlist, However. one issue that I encountered (see below) is that I need the embedding of that new playlist (as stored in my model) in order to find the closest relevant playlists in that embedding space using kmeans, I am not sure how to get around this issue- as is. it seems that I have to retrain my whole model each time I get an input playlist in order to get that playlist embedding, Therefore, I just test my model on a randomly chosen playlist (which happens to be rock and oldies. mostly!) from the training set.

To recommend songs, I first cluster the learned embeddings for all of the training playlists, and then select "neighbor" playlists for my given test playlist as all of the other playlists in that same cluster. I then take all of the tracks from these playlists and feed the test playlist embedding and these "neighboring" tracks into my model for prediction. This ranks the "neighboring" tracks by how likely they are (under my model) to occur next in the given test playlist.

[Source https://github.com/caravanuden/spotify_recsys]

I've just trained my model and now I'd like to make a recommendation as to which items User X is most likely to buy. Do I have to carry out another implementation of an algorithm that determines, for example, the nearest neighbors ( knn ) or is it sufficient to train the model and then derive the data from it?

How do I proceed after I have trained the model with the data, how do I get the recommendations from it? What is state of the art in this area in order to receive the recommendations from the trained model?

Thanks in advance. Looking forward to suggestions, ideas and answers.

Answer 1

It depends on your use case for the model. This is twofold, firstly because of the performance (speed) required for your specific use case, and secondly in regards to the main weakness (in my opinion) with the neuMF model: if a user interacts with some more items, the predictions will not change, since they were not part of the training. Because of this, if it is used in an real-time-online setting, the recommendations will essentially be based on previous behavior, and will not take into account the current session, unless the model is retrained.

The neuMF model is particularly good at batch predictions for interval recommendations. If you, for example, would like to recommend items to users in a weekly email, then you would for each user, predict the output probability for each item, and then select top n (eg. 10) probabilities and recommend those. (You would have to retrain the model next week, in order to get other predictions based on the users' latest item interactions.) So if there are 10000 unique items, for each user, make 10000 individual predictions, and recommend n -items based on those. The main drawback is of course that these 10000 predictions takes a while to perform. Because of this, it might not be suitable for real-time online predictions. On the other hand, if you are clever with parallelization of the predictions, this limitation could be surpassed as well, although, might be unnecessary. This because, as explained previously, the predictions will not change depending on current user interactions.

Using knn to cluster users in the embedding-space, and then take these users' items, and feed them into the model seems unnecessary, and in my option, defeats the purpose of the whole model-architecture. This because the whole point of the neuMF model is to generalize a given user's interaction with items among all the other users' interaction, and base the recommendations on that, so that you can, given a user and an item, get the probability for that specific item.