Meta has introduced GEM (Generative Ads Recommendation Model), the most advanced foundational ad recommendation model the company has developed to date.
This system represents a significant shift in the way Meta selects and delivers adverts to users across platforms such as Facebook and Instagram, introducing an approach inspired by large language models (LLMs) applied to recommendation systems.
The objective of GEM is clear: to improve advertising performance and increase return on investment (ROI) for advertisers through a deeper understanding of user behaviour.
Improvements in Campaign Performance
Since its launch, Meta has observed notable improvements in campaign performance thanks to this model.
Specifically:
- Conversions on Instagram have increased by 5%
- Conversions in the Facebook Feed have grown by 3%
In addition, GEM’s architecture is four times more efficient, driving performance improvements compared with the previous ranking models used by the platform.
This increase in efficiency enables more effective optimisation of ad delivery at scale.
Innovations in the GEM Architecture
One of the main challenges for recommendation systems is processing the enormous volume of interactions that occur between users and adverts.
GEM introduces several technical innovations to address this challenge.
Advanced Feature Modelling
The model divides data into two main groups:
- Sequential features, such as a user’s interaction history
- Non-sequential features, such as age, location, or ad format
Each group is processed using attention mechanisms specifically designed for that type of information.
Processing Long Interaction Histories
Understanding user intent requires analysing long sequences of events, such as clicks, views, or interactions with content.
To address this challenge, GEM uses a parallel pyramid structure that can process thousands of historical events with reduced storage costs.
This allows the system to better capture signals indicating purchase intent or genuine interest in a product.
InterFormer: Learning Between Signals
Another key innovation is InterFormer, a system that enables learning from the interaction between multiple features without losing information from the user’s complete history.
This helps identify complex patterns that influence the probability of conversion.
Cross-Platform Learning Across the Meta Ecosystem
GEM also introduces an important concept: multi-domain learning.
The model learns from interactions occurring across different platforms within the Meta ecosystem:
- Business Messaging
For example, if a user frequently interacts with video content on Instagram, that information can help improve ad predictions in the Facebook Feed.
This approach enables the creation of far more comprehensive and accurate recommendation models.
Knowledge Transfer to Other Models
GEM also acts as a central model that trains and enhances other smaller recommendation systems.
To achieve this, it uses a knowledge transfer framework based on:
- Knowledge distillation
- Representation learning
- Parameter sharing
This system is twice as effective as traditional methods, enabling hundreds of specialised models to benefit from the learnings generated by GEM.
Large-Scale Training
Training a model of this magnitude requires highly advanced technological infrastructure.
Meta has redesigned its training system for GEM using:
- Thousands of GPUs
- Advanced compilation with PyTorch 2.0
- Custom GPU kernels
Thanks to these optimisations, Meta has achieved a 23-fold increase in effective training FLOPs, allowing the model to scale to unprecedented levels.
The Future of GEM: Multimodal Models
The next step in GEM’s evolution will be to transform it into a multimodal model.
This means it will be capable of analysing and integrating information from:
- Text
- Images
- Audio
- Video
With this approach, Meta aims to build a system capable of understanding both organic content and advertisements within a single artificial intelligence model.
Conclusion
The introduction of GEM marks an important step forward in the evolution of advertising recommendation systems.
As models become larger and more capable of processing vast volumes of behavioural data, advertising platforms are moving towards more intelligent systems that better understand user intent and interests.
For advertisers, this represents a clear opportunity: more relevant campaigns, more efficient optimisation, and stronger performance outcomes.
