1. Training a separate model that detects whether a video is click-bait.
2. Running this model on the candidate list.
3. Removing the videos that the model classifies as click-bait.

Another re-ranking approach is to manually transform the score returnedby the ranker.

• video age (perhaps to promote fresher content)
• video length

This section briefly discusses freshness, diversity, and fairness.These factors are among many that can help improve your recommendationsystem. Some of these factors often require modifying different stagesof the process. Each section offers solutions that you might applyindividually or collectively.

Freshness

Most recommendation systems aim to incorporate the latest usage information,such as current user history and the newest items. Keeping the model freshhelps the model make good recommendations.

Solutions

• Re-run training as often as possible to learn on the latest training data.We recommend warm-starting the training so that the model does not haveto re-learn from scratch. Warm-starting can significantly reduce trainingtime. For example, in matrix factorization, warm-start the embeddings foritems that were present in the previous instance of the model.
• Create an "average" user to represent new users in matrix factorizationmodels. You don't need the same embedding for each user—youcan create clusters of users based on user features.
• Use a DNN such as a softmax model or two-tower model. Since the model takesfeature vectors as input, it can be run on a query or item that was notseen during training.
• Add document age as a feature. For example, YouTube can add a video's ageor the time of its last viewing as a feature.

Diversity

If the system always recommend items that are "closest" to the queryembedding, the candidates tend to be very similar to each other. Thislack of diversity can cause a bad or boring user experience. For example,if YouTube just recommends videos very similar to the video the user iscurrently watching, such as nothing but owl videos(as shown in the illustration), the user will likely lose interest quickly.

Solutions

• Train multiple candidate generators using different sources.
• Train multiple rankers using different objective functions.
• Re-rank items based on genre or other metadata to ensure diversity.

Fairness

Your model should treat all users fairly. Therefore, make sureyour model isn't learning unconscious biases from the training data.

Solutions

• Include diverse perspectives in design and development.
• Train ML models on comprehensive data sets. Add auxiliary data whenyour data is too sparse (for example, when certain categories areunder-represented).
• Track metrics (for example, accuracy and absolute error) on eachdemographic to watch for biases.
• Make separate models for underserved groups.