At Hot Chips 2026, Oxmiq Labs presented an analysis comparing High Bandwidth Fabric against HBM for AI compute workloads.

Oxmiq Labs is presenting HBF in AI Compute at Hot Chips 2026, making the case for high-bandwidth flash as a dedicated capacity tier for AI inference. The presentation outlines HBF hardware specifications across three grades, with maximum user bandwidth scaling from 0.384 TB/s to 3.072 TB/s and UCIe data rates climbing from 8 GT/s to 32 GT/s. Capacity tops out at 512 GiB on a 16-high stack. Oxmiq’s core argument is that HBF delivers eight to sixteen times the capacity of HBM at an equivalent cost.

Within the broader memory technology landscape, Oxmiq maps technologies using alpha and beta, with beta tracking cost and alpha tracking bandwidth. In this framework, HBF occupies a distinct capacity niche rather than serving as a lower-cost alternative to HBM. This distinction becomes critical when examining model architectures. Large Mixture-of-Experts (MoE) models operate differently from dense models regarding batch size and tokens per second. MoE architectures introduce a sub-phase where smaller batches hold the full model in memory, making lower bandwidth acceptable.

Oxmiq reframes inference economics around the cost per token, arguing that the true metric depends on how memory is utilized for holding and feeding data via bandwidth. The company evaluates three deployment configurations within the same cost envelope: an HBM-only baseline, an all-HBF setup, and a 2x HBF to 6x HBM mix. These options trade peak bandwidth for expanded capacity. While the HBM baseline achieves 22 TB/s of peak bandwidth with 288 GB of capacity, the all-HBF configuration drops to 12.8 TB/s while expanding capacity to 4 TB.

Simulations using a 72-GPU rack demonstrate these trade-offs concretely. Running a decode-centric Kimi-K2 1T model at FP4 with a 1M-token input and 1K-token output, Oxmiq found that HBF provides approximately 14 times the capacity for roughly 0.6 times the bandwidth compared to an equivalent HBM rack. Performance varies significantly with context length. With a short 256-by-256 context, HBF wins on cost at low batch sizes but leaves 85 percent “dead capacity.” Extending to long-context workloads (1M tokens in, 1K out) shifts the advantage; capacity remains beneficial only while bandwidth demand stays low, beyond which HBM regains its edge.

In mixed deployments, HBM can function as a hot-expert cache. Because expert popularity tends to flatten across diverse queries, caching yields returns primarily at low batch sizes or when similar queries are grouped together. A preliminary simulation plots rack dollars per token against single-server capacity, demonstrating that cheaper dollars per gigabyte does not equal cheaper dollars per token. This mathematical exercise clarifies the boundaries of HBF’s viability and optimal deployment zones. Mapping the same trade across batch sizes reveals that HBM maintains its advantage at the rack scale, whereas HBF aligns best with deployments prioritizing capacity per node. A secondary analysis zeroing in on the bandwidth-versus-capacity split within a single server reinforces that matching memory to deployment scale matters as much as the sticker price per gigabyte.

Implementing HBF introduces specific software requirements. Optimal bandwidth requires 64 KB access chunks, and the memory exhibits approximately 24 hours of power-on data retention at 85 degrees Celsius, necessitating host-managed lifecycle handling. Since HBF is read-optimized and write-constrained, data placement becomes a software-driven challenge. Oxmiq reviews the inference engine ecosystem, spanning production frameworks like vLLM and SGLang to vendor-optimized stacks such as TensorRT-LLM and AWS Neuron. The company identifies vLLM as the production default and primary focus for HBF integration.

Analyzing a model like Kimi-K3, Oxmiq notes that roughly 93 percent of bytes consist of MoE expert weights totaling 1.45 TB. Consequently, HBF is best suited for the MoE expert pool and KV-cache offload, while HBM retains attention weights and frequently accessed hot data. Inside vLLM, Oxmiq proposes a plugin that replaces host CPU pinned memory with HBF for KV caches and MoE expert pools. A GPU equipped with four HBM and four HBF stacks would achieve 2.2 TB of total memory with approximately 17.4 TB/s of peak bandwidth, substantially exceeding current offload capabilities.

Oxmiq identifies sparse-attention models, such as DeepSeek Sparse Attention and Compressed Sparse Attention, as strong candidates for HBF due to their alignment with low-bandwidth, high-capacity workloads, though adoption remains conditional on model design. The company also highlights expert parallelism as a key use case. Cheap HBF capacity reduces the need for extensive EP shards and minimizes all-to-all communication. Instead of sharding experts across eight GPUs and incurring inter-layer traffic, two nodes could store experts locally in HBF.

Oxmiq concludes that HBF is not a universal replacement but a specialized capacity instrument tailored to a narrow segment of inference workloads. It excels specifically where bandwidth demand is low, such as MoE models running small batches with long-context sparse KV caches. This measured perspective contrasts with more optimistic narratives from many HBF promoters, offering a more balanced framing. Ultimately, matching memory architecture to workload characteristics and deployment scale matters more than raw cost-per-gigabyte metrics. Before HBF becomes widely practical, robust allocators and placement policies will be essential. Modern AI-native coding tools may accelerate development of these missing software components, making implementation less burdensome than it otherwise would be.