Page Summary

outlined_flag

• This page focuses on the training data and process for a softmax deep neural network recommendation system.

• Negative sampling is crucial to avoid "folding," where embeddings from different categories are incorrectly grouped together.

• Negative sampling involves training the model on both positive (relevant) and negative (irrelevant) examples.

• Compared to Matrix Factorization, softmax DNNs are more flexible but computationally expensive and susceptible to folding.

• While Matrix Factorization is better for large-scale applications, DNNs excel at capturing personalized preferences for recommendation tasks.

The previous page explained how to incorporate a softmax layer into a deepneural network for a recommendation system. This page takes a closer look at thetraining data for this system.

Training data

The softmax training data consists of the query features \(x\) anda vector of items the user interacted with (represented as aprobability distribution \(p\)). These are marked in blue inthe following figure. The variables of the model are the weightsin the different layers. These are marked as orange in the followingfigure. The model is typically trained using any variant ofstochastic gradient descent.

Negative sampling

Since the loss function compares two probability vectors\(p, \hat p(x) \in \mathbb R^n\) (the ground truth andthe output of the model, respectively), computing thegradient of the loss (for a single query \(x\)) can beprohibitively expensive if the corpus size \(n\) is too big.

You could set up a system to compute gradients only on the positive items(items that are active in the ground truth vector). However, if the systemonly trains on positive pairs, the model may suffer from folding, asexplained below.

Folding

In the following figure, assume that each color represents a different category of queries and items. Each query (represented as a square) only mostly interacts with the items (represented as a circle) of the same color. For example, consider each category to be a different language in YouTube. A typical user will mostly interact with videos of one given language.

The model may learn how to place the query/item embeddings of a given color relative to each other (correctly capturing similarity within that color), but embeddings from different colors may end up in the same region of the embedding space, by chance. This phenomenon, known asfolding, can lead to spurious recommendations: at query time, the model may incorrectly predict a high score for an item from a different group.

Negative examples are items labeled "irrelevant" to a given query. Showing the model negative examples during training teaches the model that embeddings of different groups should be pushed away from each other.

Instead of using all items to compute the gradient (which can be tooexpensive) or using only positive items (which makes the model prone tofolding), you can use negative sampling. More precisely, you compute anapproximate gradient, using the following items:

• All positive items (the ones that appear in the target label)
• A sample of negative items (\(j\) in \({1, …, n}\))

There are different strategies for sampling negatives:

• You can sample uniformly.
• You can give higher probability to items j with higherscore \(\psi(x) . V_j\). Intuitively, these are examplesthat contribute the most to the gradient); these examples are oftencalled hard negatives.

• For a more comprehensive account of thetechnology, architecture, and models used in YouTube, seeDeep Neural Networksfor YouTube Recommendations.
• SeeXin et al., Folding:Why Good Models Sometimes Make Spurious Recommendations for more detailson folding.
• To learn more about negative sampling, seeBengio and Senecal,Adaptive Importance Sampling to Accelerate Training of a NeuralProbabilistic Language Model.

On matrix factorization versus softmax

DNN models solve many limitations of Matrix Factorization, but are typicallymore expensive to train and query. The table below summarizes some of theimportant differences between the two models.

In summary:

• Matrix factorization is usually the better choice for large corpora.It is easier to scale, cheaper to query, and less prone to folding.
• DNN models can better capture personalized preferences, but areharder to train and more expensive to query. DNN models are preferableto matrix factorization for scoring because DNN models can usemore features to better capture relevance. Also, it is usuallyacceptable for DNN models to fold, since you mostly care aboutranking a pre-filtered set of candidates assumed to be relevant.