The name softmax is a play on words. A "hard" max assigns probability 1 to the item with the largest score \(y_i\). By contrast, the softmax assigns a non-zero probability to all items, giving a higher probability to items that have higher scores. When the scores are scaled, the softmax \(h(\alpha y)\) converges to a "hard" max in the limit \(\alpha \to \infty\).

Loss Function

Finally, define a loss function that compares the following:

• \(\hat p\), the output of the softmax layer (a probability distribution)
• \(p\), the ground truth, representing the items the user hasinteracted with (for example, YouTube videos the user clicked or watched).This can be represented as a normalized multi-hot distribution (aprobability vector).

For example, you can use the cross-entropy loss since you are comparingtwo probability distributions.

Softmax Embeddings

The probability of item \(j\) is given by\(\hat p_j = \frac{\exp(\langle \psi(x), V_j\rangle)}{Z}\),where \(Z\) is a normalization constant that does not depend on \(j\).

In other words, \(\log(\hat p_j) = \langle \psi(x), V_j\rangle - log(Z)\),so the log probability of an item \(j\) is (up to an additive constant)the dot product of two \(d\)-dimensional vectors, which can be interpretedas query and item embeddings:

• \(\psi(x) \in \mathbb R^d\) is the output of the last hidden layer. We call it the embedding of the query \(x\).
• \(V_j \in \mathbb R^d\) is the vector of weights connecting the last hidden layer to output j. We call it the embedding of item \(j\).

DNN and Matrix Factorization

In both the softmax model and the matrix factorization model,the system learns one embedding vector\(V_j\) per item \(j\). What we called theitem embedding matrix \(V \in \mathbb R^{n \times d}\) in matrixfactorization is now the matrix of weights of the softmax layer.

The query embeddings, however, are different. Instead of learningone embedding \(U_i\) per query \(i\), the system learns a mappingfrom the query feature \(x\) to an embedding \(\psi(x) \in \mathbb R^d\).Therefore, you can think of this DNN model as a generalization of matrixfactorization, in which you replace the query side by a nonlinearfunction \(\psi(\cdot)\).

Can You Use Item Features?

Can you apply the same idea to the item side? That is, instead of learningone embedding per item, can the model learn a nonlinear function that mapsitem features to an embedding? Yes. To do so, use a two-towerneural network, which consists of two neural networks:

• One neural network maps query features\(x_{\text{query}}\) to query embedding\(\psi(x_{\text{query}}) \in \mathbb R^d\)
• One neural network maps item features\(x_{\text{item}}\) to item embedding\(\phi(x_{\text{item}}) \in \mathbb R^d\)

The output of the model can be defined as the dot product of\(\langle \psi(x_{\text{query}}), \phi(x_{\text{item}}) \rangle\).Note that this is not a softmax model anymore. The new model predictsone value per pair \((x_{\text{query}}, x_{\text{item}})\)instead of a probability vector for each query \(x_{\text{query}}\).