Qualcomm is advancing a chip architecture that places compute units directly adjacent to memory to reduce latency and improve bandwidth efficiency.

Qualcomm has detailed how its High Bandwidth Compute (HBC) architecture addresses a costly, often overlooked bottleneck in AI: data movement. By stacking DRAM directly onto a logic die and executing data-intensive operations where the data resides, HBC eliminates the need for wider interfaces between memory and processors. The Dragonfly AI250 will ship with HBC Gen 1, while the upcoming Dragonfly AI300 will feature Gen 2. For independent software vendors (ISVs) optimizing inference stacks and silicon teams evaluating packaging strategies, this architectural shift fundamentally alters how the industry measures AI performance.

Generative AI workloads expose a fundamental constraint that raw compute speed cannot overcome. Generating each token requires fetching vast quantities of model parameters and accumulated context from memory, making the actual arithmetic negligible compared to the cost of data delivery. While compute capability has surged over the past decade, memory bandwidth has lagged significantly. Transformer model sizes have grown approximately 240× every two years, whereas AI hardware memory capacity has increased by only 2× during the same period. This divergence leaves an increasing portion of every accelerator idle, stalled while waiting for operands to arrive.

Figure 1. Compute capability has climbed steeply across generations. Memory bandwidth has grown far more slowly, and this widening gap increasingly dictates real-world AI performance. (Source: Qualcomm)

The industry’s conventional solution attempts to push more bandwidth through the same foundational layout: keeping memory and compute as discrete components and widening the interconnect between them. This approach quickly encounters physical and economic ceilings. Wider interfaces require additional wires and pins, escalating cost and complexity while delivering diminishing returns. Every bit transferred between separate memory and compute dies consumes energy and latency; at current scales, this data movement dominates the power budget. Furthermore, advanced memory-integration techniques, including 2.5D interposers such as CoWoS, entail prohibitive costs and severe supply constraints, restricting top-tier performance to a narrow segment of deployments.

Qualcomm’s Solution: Moving Logic Adjacent to Memory

Qualcomm’s HBC architecture positions compute logic directly adjacent to memory, eliminating the chip-boundary crossings that data must otherwise traverse. The design vertically stacks an LPDDR DRAM package atop a logic or compute die, linking the layers via dense through-silicon vias (TSVs) rather than relying on a traditional long-edge HBM interface. Operating on a standard 2D organic substrate allows Qualcomm to bypass supply-constrained 2.5D interposer packaging altogether. Qualcomm categorizes this architecture as near-memory computing—distinct from compute-in-memory paradigms—and claims it delivers up to 6× higher bandwidth per watt than conventional HBM, alongside substantially greater capacity per watt than on-chip SRAM can provide.

Figure 2. Two competing philosophies: Once data movement dominates the energy budget, computing adjacent to the data rather than transporting it fundamentally reshapes system economics. (Source: Qualcomm)

The architecture does not supplant the primary processor. A Qualcomm Dragonfly AI accelerator retains responsibility for complex, flexible orchestration tasks, while HBC assumes the memory-bound operations that are prohibitively expensive to feed. Each component executes the workload best suited to its design.

Qualcomm has integrated HBC Gen 1 into the Dragonfly AI250 rack-scale platform, which will deliver 133 TB/s of effective bandwidth per card—an 18× increase in effective memory bandwidth over the Dragonfly AI200’s LPDDR5X configuration. HBC Gen 2 will debut with the forthcoming Dragonfly AI300 platform, targeting a 54× increase relative to the AI200 baseline. Speaking at a recent analyst meeting, Qualcomm Executive Vice President and General Manager of Technology Planning, Edge Solutions & Data Center Durga Malladi reiterated the roadmap unveiled at the company’s Investor Day. She confirmed that first-generation HBC will reach commercialization in fiscal 2027, with the immediate focus shifting from research and test chips to volume production.

Qualcomm also positioned HBC as a modular infrastructure component, decoupled from any single proprietary platform. Customers are expected to deploy Qualcomm CPUs independently, HBC independently, or in combination, without requiring adoption of Qualcomm’s other accelerator offerings. This modular framing aligns with Qualcomm’s broader inference strategy: anticipating AI deployment across cloud, edge, and hybrid environments, the company plans to support models from the open ecosystem rather than developing proprietary foundation models. Qualcomm emphasized that efficiency—measured by energy consumption and token-generation throughput—is becoming an increasingly decisive competitive metric.

The recently acquired Modular software stack further strengthens Qualcomm’s value proposition. Designed to run on third-party hardware platforms, Qualcomm markets the stack as open and widely accessible, citing substantial performance improvements over incumbent alternatives. The company noted overwhelmingly positive feedback from hyperscalers during initial pilot engagements.

During the Q&A session, analysts concentrated on implementation details. Questions covered packaging technology, TSV alignment, thermal constraints, memory vendor participation, optical interconnects, manufacturing yields, software-ecosystem implications, JEDEC standards compliance, and future disaggregated memory architectures. Qualcomm withheld several technical specifics, citing active collaborations with memory partners and undisclosed product roadmaps.

Qualcomm contends that AI infrastructure purchasers require a revised evaluation framework. Traditional headline compute metrics are losing relevance as buyers prioritize delivered performance for memory-bound workloads, alongside performance-per-watt and performance-per-dollar ratios. This shifts the focus away from peak theoretical throughput. Energy consumption emerges as both the primary operational expense and the hard physical constraint at hyperscale, granting architectures that minimize data movement a compounding structural advantage. Within this paradigm, memory transitions from a passive storage medium to an active computational participant, with ramifications spanning chip architecture to data-center economics.

Qualcomm positions HBC as a direct response to a structural industry challenge: the scarce resource in computing has shifted from raw arithmetic to the timely, cost-effective delivery of data. The company’s roadmap—spanning Gen 1 on the AI250 to Gen 2 on the AI300—provides ISVs and silicon engineering teams with a definitive design target. Simultaneously, it offers CIOs and IT procurement leaders a new framework for evaluating AI infrastructure investments, moving beyond peak FLOPS marketing claims.

Figure 3. Capacity, bandwidth, and compute scale synergistically with HBC. Conventional architectures force designers to choose among them. (Source: Qualcomm)

For silicon engineering teams, the near-memory paradigm restores viable packaging options previously restricted by CoWoS supply bottlenecks. For ISVs, it elevates the importance of software capable of scheduling workloads based on memory locality rather than sheer core counts. Meanwhile, CIOs negotiating multi-year AI infrastructure agreements gain a secondary evaluation metric beyond peak FLOPS: total energy consumed per token served—a figure that scales linearly with operational expenditures as AI fleets expand.

HBC provides Qualcomm with a credible foothold in the inference infrastructure market without forcing direct competition with NVIDIA and AMD on raw compute metrics. The near-memory strategy tackles a genuine physical limitation, while standard-substrate packaging circumvents the CoWoS supply constraints currently pressuring rival vendors. With commercialization still two fiscal years away, execution risk persists, and the pace at which Gen 2 achieves scale will heavily depend on continued cooperation from memory vendors.

An Industry Inflection Point? HBC may mark a pivotal shift in AI infrastructure design. For the past decade, the industry has scaled accelerators by continuously adding compute density and widening memory interfaces. Qualcomm’s data indicates this trajectory is approaching its physical and economic limits as workloads intensify. Should near-memory architectures proliferate across data centers, PCs, automotive systems, extended reality (XR), and industrial applications, the competitive landscape will pivot from peak FLOPS to bandwidth-per-watt efficiency. This transition will fundamentally reshape how ISVs optimize software and how CIOs evaluate AI infrastructure acquisitions across multi-year procurement cycles.

If you liked what you read here, don’t be stingy, share it with your friends.