Meta's AI agent Muse achieves 100x cost efficiency advantage for recommender-feed workloads compared to generative chat on equivalent infrastructure.

Devansh Tandon, who leads recommendation research at Meta, argues that the next massive consumer AI application will not be a chatbot but the recommendation feed itself. Speaking on the AI Engineer podcast, Tandon laid out a structural case for why content feeds like Instagram, Facebook, TikTok, and YouTube hold a decisive economic advantage over AI chat apps: they decode pointers to content rather than generating every output token, making them up to 100 times cheaper to run per hour of consumer engagement.

The argument rests on what Tandon calls a "tokens in, engagement out" flywheel: train a model, run inference on it, convert better recommendations into engagement (daily active users and time spent), then monetize through ads or subscriptions, and finally use that revenue to fund the next training run. Every step up the scaling curve is one revolution around the flywheel. Both Instagram and ChatGPT have hundreds of millions of users and are powered by large models. But the economics of serving those users could not be more different.

The first pillar of Tandon's case is that recommendation systems obey the same scaling laws as large language models. Before the LLM wave, recommender models were the largest production machine learning systems in big tech, and they remain among the largest models served at billion-plus daily-active-user scale. Meta's HSTU papers from 2024 and 2026 both show that scaling model size, compute, and data produces clear offline gains in recommendation quality. More concretely, Meta's recent earnings reports credit Instagram Reels with 30% year-on-year watch time growth. Tandon attributes part of that to simplifying the ranking architecture to enable efficient model scaling and doubling the length of user interaction sequences used for training Instagram while increasing the richness of each interaction. "These scaling curves aren't just academic research," he noted. "They're driving real product impact at scale for some of the biggest consumer businesses in the world."

The second pillar is where the economics become decisive: the structural cost gap between what feeds and chat apps output. A content feed's output is not content itself, but a semantic ID—a compact token that acts as an address pointing to a video or post that creators have already uploaded. The supply of that content is effectively free, subsidized by a healthy creator economy. An AI chat app, by contrast, must manufacture every single output token itself at inference time. A single chat turn emits a few thousand tokens, each one generated by expensive model computation. Content feeds operate with roughly 1–10 billion active parameters and output semantic ID tokens (pointers to existing content); a few thousand tokens per interaction. AI chat apps operate with 10–100 billion active parameters and manufacture every token of content at inference. The result: generating an hour of consumer engagement through a feed costs up to 100 times less than the same hour of engagement through a chat interface.

The enabling technology is semantic IDs. Many teams are simply swapping traditional hash IDs for semantic IDs inside existing recommendation models and seeing meaningful gains. A hash ID is an arbitrary number that changes whenever the underlying data shifts, meaning a model can only memorize it. A semantic ID is a learned representation that captures what the content actually is, giving the model something it can genuinely learn from. More importantly for the economics, semantic IDs compress aggressively: a three-minute Instagram Reel would consume roughly 10,000 tokens if fed raw into a model's context window—enough to blow through most context budgets in a single item. Semantic IDs compress that same video to about 10 tokens. They use hierarchical structure: similar Reels share their first three semantic tokens—the first representing sports, the next two representing tennis, and the final token individuating the specific video. This structure allows a model to reason over long user-interaction sequences (months or years of scrolling) without running out of context, and it is what makes the entire system economical: the model is not regenerating content, it is just looking up an address in a space that the creator economy has already populated.

Tandon reduces the build process to three conceptual steps: tokenize content to create a language for the domain, adapt the LLM so it is bilingual in English and that domain language, then prompt the model with user information so it can decode recommendations directly from the content corpus. He renders this as a five-layer architecture. At the bottom sit semantic IDs. Above that is a base LLM foundation model, either open-weights or an internal first-party build. Then comes pre-training to bridge English and recommender tokens—teaching the model to translate between the two languages. Above that is post-training to steer the model toward recommendation tasks, such as predicting engagement.