Tuesday, November 17, 2026
AI Infrastructure · News & Analysis
Home › Chips & Hardware › Report
Chips & Hardware · Report

Smallest.ai partnered with Tenstorrent to deliver on-premises voice agent hardware, enabling inference at the edge using Tenstorrent's custom silicon.

Edge inference hardware path broadens deployment options for voice agents outside cloud infrastructure, diversifying AI chip demand.
Trade pressSlicast · October 1, 2026 at 17:09 UTC · US · Source: Unite.AI
importance 55

Tenstorrent and Smallest.ai announced a partnership on October 1, 2026, to deliver production-grade, on-premises voice AI infrastructure. The joint stack runs Smallest.ai's Lightning V2 real-time text-to-speech model natively on Tenstorrent Galaxy Blackhole servers, designed to reduce deployment costs while meeting the performance demands of real-time voice applications. The companies positioned the offering for organizations seeking to operate real-time voice agents entirely on-premises with full data sovereignty, targeting high-volume, data-sensitive workloads across financial services, telecom, healthcare, and enterprise environments. Lightning V2 is available on Tenstorrent hardware immediately with usage-based pricing and no upfront commitments.

"There shouldn't need to be a choice between performance and cost," said Amr Elashmawi, Vice President of Strategy & Business Development at Tenstorrent. "Tenstorrent has built efficient and scalable compute that reduces the cost of existing workflows and makes entirely new workloads practical."

The Galaxy Blackhole platform pairs 32 Blackhole ASICs delivering 23 PFLOPS of Block FP8 compute with 6.2 GB of on-chip SRAM operating at 2.9 PB/s and 1 TB of GDDR6 memory at 16 TB/s. Each ASIC connects via ten 400 GbE links for a 32 TB/s accelerator fabric, and systems scale out through up to 56 800 GbE QSFP-DD ports. The 6U air-cooled chassis pairs an AMD EPYC 9004 host processor with up to 576 GB of DDR5 memory, runs Ubuntu 22.04, draws 8 to 10 kW on average, and carries a list price of $160,000.

The partnership is grounded in peer-reviewed research. A paper authored by Smallest.ai's Ranjith M S, Chief Technology Officer Akshat Mandloi, and Chief Executive Officer Sudarshan Kamath—"Rewriting TTS Inference Economics: Lightning V2 on Tenstorrent Achieves 4x Lower Cost Than NVIDIA L40S"—documents the engineering achievement. First submitted March 24, 2026, and revised April 7, 2026, the paper details how reduced-precision execution unlocks substantial gains. According to the authors, text-to-speech models are significantly more numerically fragile than large language models because they generate continuous waveforms through iterative refinement, causing small numerical errors to compound across denoising steps and surface as audible artifacts.

The research shows that more than 95% of Lightning V2's layers execute at LoFi computational fidelity and more than 80% of the model deploys in BlockFloat8 without measurable audio degradation. The authors report a 4x compute reduction in the diffusion acoustic model, an 8x reduction in the neural vocoder, approximately 2x reduction in model size, and 1.8x reduction in memory transfer volume. On output quality, a DNSMOS perceptual score registered 3.872 for an NVIDIA L40S baseline versus 3.801 on Tenstorrent's P150—a 0.071 difference the authors characterize as falling within the range of minor perceptual variation—with a normalized word error rate of 0.009 between the two systems.

In single-device testing, the L40S sustained a concurrency of 3 at 300-millisecond latency against a listed cost of $9,000. The P150 and P100 each ran a concurrency of 1 at 250-millisecond latency at costs of $1,400 and $1,000 respectively, yielding reported cost improvements of 2.6x and 3.6x. At fleet scale, modeling a workload of 550 simultaneous five-second TTS requests at full utilization, the authors calculated that sustaining it would require 11 L40S GPUs at roughly $100,000 in accelerator cost, versus 27 P100 accelerators at roughly $27,000 or 27 P150 accelerators at roughly $37,000—a 3-4x reduction in upfront cost. The paper also documents a heavily optimized layer of approximately 6 billion multiply-accumulate operations executing in about 60 microseconds on the L40S versus about 31 microseconds on the P150, with projections that wider kernel-level optimization could yield 8-12x cost-normalized improvement over the L40S baseline.

The research highlights that numerical fragility impairs conventional debugging approaches. The authors documented a case where outputs from an L40S GPU and AMD EPYC 7352 CPU showed a Pearson Correlation Coefficient of only 0.72 despite identical inputs, yet produced perceptually indistinguishable audio. In another instance, a layer whose coefficient rounded to 1.0 caused audible breakage that required more than a month to isolate. The authors conclude that conventional tensor-level similarity metrics are unreliable indicators of perceptual quality in TTS systems, necessitating end-to-end perceptual validation for reduced-precision deployments.

The authors acknowledge that certain layers remain too numerically sensitive for reduced-fidelity or block floating-point execution without perceptual degradation, limiting full-model low-precision coverage, and that compiler and program configurations require further optimization. In a September 28, 2026 technical post, Smallest.ai characterized Lightning V2 as the world's first TTS model to deliver high-quality speech synthesis under joint BlockFloat8 quantization and reduced-precision arithmetic. The company noted that its newer Lightning V3 already surpasses V2 in quality and architectural efficiency, with plans to deploy Lightning V3 on Tenstorrent hardware using the same co-design principles and targeting similar or greater cost breakthroughs.

Read the original
Smallest.ai partnered with Tenstorrent to… · Slicast