AMD acquires Taalas, a startup that hard-codes AI model weights directly into inference silicon, enabling 16K+ tokens/sec performance for Llama 3.1-8B.

AMD is acquiring Taalas, a Canadian AI startup founded in Toronto in 2023 that specializes in building inference chips with an unconventional design. The company emerged from stealth in February with an approach that embeds a model's architecture and trained parameters directly into silicon, enabling exceptionally fast inference while locking each chip to a single model.

Taalas's technology delivers significant performance gains. A demonstration chip achieved over 16,000 tokens per second per user when running Llama 3.1-8B, representing performance many times faster than competing hardware solutions. Google is reportedly pursuing a similar approach for chips optimized around Gemini.

AMD plans to integrate the technology into its accelerator roadmap and offer it as a system-level solution alongside its Instinct GPU lineup. Vamsi Boppana, SVP of AMD's AI division, stated that the acquisition strengthens the company's AI portfolio. Ljubisa Bajic, co-founder of Taalas, noted that AMD provides the scale and reach necessary for the startup to advance its mission. The acquisition remains subject to standard regulatory approvals.