The Limitations of Single-Objective Optimization

Traditionally, recommender systems have focused primarily on predicting item relevance. While accuracy remains crucial, optimizing solely for this metric can lead to several drawbacks:

• Lack of Diversity:  Focusing solely on relevance often results in homogenous recommendations, trapping users in "filter bubbles" and limiting their exposure to new and potentially interesting items.
• Ignoring Business Goals:  Maximizing clicks or predicted ratings might not always align with a company's broader objectives, such as increasing conversions, promoting specific items, or maximizing long-term user engagement.
• Overlooking User Experience:  A laser focus on relevance can neglect other crucial aspects of user experience, such as serendipity, novelty, and explainability, which contribute to satisfaction and long-term usage.

The Multi-Objective Optimization Landscape

Multi-objective optimization aims to address these limitations by considering multiple criteria simultaneously. This involves defining a set of objectives and finding solutions that achieve a desirable balance between them. The landscape of multi-objective optimization in recommendations and search has been categorized by Jannach D and Abdollahpouri H (2023) as follows:

• Quality Objectives: These relate directly to the perceived quality of recommendations for users.  Common quality objectives include:
  
  • Relevance (Accuracy):  Predicting how likely a user is to interact with an item.
  • Diversity:  Ensuring a variety of items are recommended, covering different categories or attributes.
  • Novelty:  Surfacing items that are new or unexpected to the user.
  • Serendipity:  Recommending surprisingly relevant items that the user might not have discovered otherwise.

• Multi-Stakeholder Objectives:  Recommendation systems often operate within a complex ecosystem involving multiple stakeholders with potentially conflicting interests. These include:
  
  • Users: Seeking relevant, diverse, and novel recommendations.
  • Platform Providers:  Aiming to maximize revenue, engagement, and customer retention.
  • Item Providers:  Wanting their items to be exposed to relevant users.

• Time Horizon Objectives:  Optimizing for short-term goals (e.g., clicks) might conflict with long-term objectives (e.g., customer lifetime value).  Strategies need to balance immediate gratification with sustainable user engagement.

• User Experience (UX) Objectives:  The UI/UX of a recommendation system significantly impacts user satisfaction. 

• Engineering Objectives:  These relate to the practical implementation and maintenance of recommendation systems.

Value Modeling: A Practical Approach to Multi-Objective Optimization

Value modeling provides a structured, industry-standard approach to defining and optimizing for multiple objectives. It assigns a numerical score to each item, reflecting its contribution to various objectives, and then combines these scores into a final ranking. This "value model" (or "scoring function") allows for a nuanced balance between potentially competing goals.

‍

How Value Models Work:

Platforms like Facebook, Twitter, YouTube, Instagram, Amazon, LinkedIn, and TikTok employ value models based on predicted user engagement. For example, a social media platform might predict the probability of a user liking, commenting, sharing, or simply dwelling on a post. These predictions, generated by machine learning models trained on historical interaction data, form the core of the value model.

‍

A simplified example:

w_like * Pr(like) + 
w_comment * Pr(comment) + 
w_share * Pr(share) + 
w_dwell_time * Pr(dwell_time)

Here, Pr(like) represents the predicted probability of a user liking a post, and w_like is the corresponding weight, reflecting the importance of likes to the platform's goals. These weights can be:

• Skewed towards specific engagement types: YouTube prioritizes watch time, TikTok prioritizes retention.
• Negative for undesirable actions: Clicking "See Fewer Posts Like This" on Instagram.
• Personalized: Facebook News Feed tailors weights to individual users.
• Algorithmically determined: Google and LinkedIn optimize weights for overarching metrics like retention.
• Dynamically adjusted: Instagram's Explore view adjusts weights based on UI changes and shifting priorities.

‍

Beyond Engagement:

Value models rarely focus solely on engagement.  They often incorporate additional factors, such as "integrity signals," which penalize low-quality content or boost items aligned with platform policies. For instance:

w_like * Pr(like) + 
w_comment * Pr(comment) + 
w_share * Pr(share) + 
w_dwell_time * Pr(dwell_time) - 
w_low_quality * Pr(low_quality) + 
w_authoritative * Pr(authoritative)

This example adds weights for the probability of content being low-quality or authoritative. Real-world value models are typically more complex, incorporating numerous engagement types, integrity signals, and other factors. However, the underlying principle of combining weighted predictions remains consistent across platforms.

Exploring New Content: The Role of Multi-Armed Bandits

Value modelling helps define the ideal balance of objectives given our current understanding of item values and user preferences. However, user preferences and item performance can change over time, and we often face uncertainty about the true value of different items. This introduces the exploration-exploitation dilemma: should we recommend items we believe are good (exploitation) or explore new items to learn more about their potential (exploration)?

Multi-armed bandits (MAB) provide a powerful framework for tackling this dilemma in dynamic recommendation environments. Imagine each recommendation candidate as a "slot machine arm" (a "bandit"). We don't know the payoff of each arm beforehand. The goal of a MAB algorithm is to learn the value of each arm over time by strategically exploring different options and exploiting the arms that seem to be performing well.

Typically industry recommendation systems will use value modelling at the ranking stage to determine the initial order of candidate items being shown. MAB is then used in the re-ranking stage to shuffle the list of candidates in a way that also optimizes for new, exploratory items.

Conclusion

Multi-objective optimization and value modeling are essential for building next-generation recommendation and search systems. By moving beyond a narrow focus on relevance and embracing a more holistic approach, businesses can create product experiences that benefit both users and their bottom line.

In practice, implementing value models requires robust infrastructure, careful objective design, and continuous adaptation to changing user behavior. Companies looking to adopt this approach often face challenges in defining objectives, balancing trade-offs, and integrating these models into production systems. As industry best practices evolve, new tools and frameworks are emerging to streamline this process, making sophisticated multi-objective optimization more accessible to teams of all sizes.

References, inspiration and Further Reading:

• Multi-objective Recommender Systems: A Survey
• How Platform Recommenders Work
• Recommendation Systems
• Multi-Objective Optimization
• Scaling the Instagram Explore recommendations system
• Instagram Explore Recommender System
• Facebook News Feed Ranking
• Twitter Personalized Notifications
• Youtube multi-objective ranking