Skip to main content

Diversity Factor

Shaped allows you to fine-tune the diversity of your recommendations, ensuring users see a wider range of relevant items rather than just the top predicted results. This can lead to a more engaging and surprising user experience.

This guide explains how to control the diversity of recommendations using the diversity_factor parameter at inference time.

Understanding Diversity​

In recommendation systems, relevance usually refers to how well an item matches a user's preferences based on their past behavior and item attributes. However, simply showing the most relevant items can lead to a phenomenon known as "filter bubbles," where users are only exposed to a narrow range of similar content.

Diversity, on the other hand, aims to introduce variety and novelty into recommendations. By incorporating diversity into your ranking strategy, you can:

  • Reduce redundancy: Avoid recommending very similar items.
  • Increase serendipity: Surface unexpected but potentially interesting items.
  • Improve user satisfaction: Provide a more engaging and less repetitive experience.

Configuring diversity_factor​

Shaped utilizes the diversity_factor parameter to balance relevance and diversity in your recommendations. This parameter accepts a value between 0 and 1, where:

  • 0: Prioritizes relevance, ignoring diversity. This setting essentially ranks items solely based on their predicted relevance scores.
  • 1: Prioritizes diversity, potentially sacrificing some relevance. This setting emphasizes variety, even if it means recommending items with slightly lower predicted relevance.

Values between 0 and 1 allow you to fine-tune the trade-off between relevance and diversity according to your specific use case and desired user experience.

You can override the model's default diversity_factor at inference time using the Rank API. This provides flexibility to adjust diversity dynamically based on the context:

shaped rank --model_name my_recommendation_model --user_id "XA123F2" --diversity_factor 0.7

This example overrides the model's default diversity_factor of 0.5 and sets it to 0.7 for this specific ranking request, emphasizing diversity more strongly.

How Shaped Handles Diversity: Maximal Marginal Relevance (MMR)​

Shaped employs the Maximal Marginal Relevance (MMR) algorithm to incorporate diversity into the ranking process. MMR works by iteratively selecting items for the recommendation list, considering both:

  • Relevance: How well an item matches the user's preferences.
  • Diversity: How different the item is from those already included in the list.

This approach ensures that the final ranked list contains a mix of relevant and diverse items, striking a balance controlled by the diversity_factor parameter.