What we as marketers can learn about originality and diversity from Diversity-Rewarded CFG Distillation via Google DeepMind.

Welcome back to 🐹 Hamsterdam Research, where we’re looking at a subtopic from a Google DeepMind paper, Diversity-Rewarded CFG Distillation.

Our focus will be on what originality and diversity mean in the context of the paper, and why we should care as marketers.

The paper was submitted to ArXiv on October 8th, 2024, and includes several authors, including Geoffrey Cideron, Andrea Agostinelli, Johan Ferret, Sertan Girgin, Romuald Elie, Olivier Bachem, Sarah Perrin, and Alexandre Ramé.

In short, this paper presents a new approach to improving the quality and diversity of AI-generated content, particularly for creative applications like music generation. It introduces “diversity-rewarded Classifier-Free Guidance (CFG) distillation,” a technique that helps AI models produce higher-quality and more diverse outputs without increasing computational cost at deployment.

Here’s the paper’s full abstract with my bolding:

“Generative models are transforming creative domains such as music generation, with inference-time strategies like Classifier-Free Guidance (CFG) playing a crucial role. However, CFG doubles inference cost while limiting originality and diversity across generated contents. In this paper, we introduce diversity-rewarded CFG distillation, a novel finetuning procedure that distills the strengths of CFG while addressing its limitations. Our approach optimises two training objectives: (1) a distillation objective, encouraging the model alone (without CFG) to imitate the CFG-augmented predictions, and (2) an RL objective with a diversity reward, promoting the generation of diverse outputs for a given prompt. By finetuning, we learn model weights with the ability to generate high-quality and diverse outputs, without any inference overhead. This also unlocks the potential of weight-based model merging strategies: by interpolating between the weights of two models (the first focusing on quality, the second on diversity), we can control the quality-diversity trade-off at deployment time, and even further boost performance. We conduct extensive experiments on the MusicLM (Agostinelli et al., 2023) textto-music generative model, where our approach surpasses CFG in terms of quality-diversity Pareto optimality. According to human evaluators, our finetuned-then-merged model generates samples with higher quality-diversity than the base model augmented with CFG. Explore our generations at googleresearch.github.io/seanet/musiclm/diverse_music.”

– Abstract (my bolding)

I’m a proponent of marketers learning everything we can about generative AI. As the authors mention, “Art and entertainment — domains historically driven by
human creativity — are undergoing a profound transformation thanks to AI generative models.”

We also rely on it for creative content brainstorming and production, including increasingly multimodal content like video, images, and music.

As marketers, the strategies explored in the DeepMind paper allow us to maintain both quality and originality in AI-generated content, making it more engaging for audiences.

Originality and diversity are important in generative models, especially for creative fields, because these models are expected to not only meet user expectations but also provide innovative and varied outputs.

What originality and diversity speak to is a wider range of creative possibilities, rather than generating overly similar content.

Current approaches like Classifier-Free Guidance (CFG), a “particularly popular method for image and audio generation,” like in DALL-E, can improve quality and alignment of outputs with user prompts, but it can also hinder “the exploration of novel and diverse ideas — a cornerstone of creativity” — by causing the model to produce similar outputs across different generations.

As marketers, it is important for us to understand that “optimising quality usually reduces diversity,” as we might want to produce diverse multimodal content for different marketing campaigns, but we should be aware of existing model limitations in that regard.

On the other hand, increasing diversity, “e.g., when increasing temperature at inference,” can reduce quality.

The authors introduce a reinforcement learning-based method with a diversity award to address this and encourage broader exploration of novel ideas, which is key to increased creativity.

They measure diversity “by comparing pairs of generations by first embedding them and then computing their negative (cosine) similarity in this embedding space.”

As a result, the model can generate more varied outputs while still maintaining quality.

So why should marketers care about originality and diversity from generative models?

These enhance the creative potential of a model, making its generated content less predictable and potentially more engaging. As marketers, this means we have the ability to produce a broader range of content to better capture audience interests, align with campaign goals, and keep viewers engaged with fresh ideas.

Thanks for checking out this rendition of Hamsterdam Research. 🐹

Until next time, enjoy the vibes:

Thanks for reading. Happy marketing! 🤗