AI Infrastructure Summit highlights memory and design advances as drivers of next-generation system architecture.

At the 2025 AI Infra Summit in Santa Clara, California, three companies showcased advances in memory, chip, and system design. John Overton from Kove presented their Linux-based memory software that runs on any hardware and allows sharing memory between servers to increase memory utilization, CPU and GPU utilization. He noted that while GPUs and CPUs have been scaling, conventional memory systems have not, leading to overprovisioning in servers and processing bottlenecks. Kove's SDM software can be installed in a system in 15 minutes and allows unlimited memory accessed from virtualized elastic memory pools across servers, supporting up to 64PiB of DRAM per process. The software can hide latency, making memory look local to a CPU even when it isn't, and can hide latency between memory that is over 150 meters away, working across Infiniband and RoCE fabrics.

Kove demonstrated that for AI inference, their solution can run 3-5X faster, with results shown working with partners such as Red Hat and SuperMicro for key-value cache at scale, using benchmarks with Redis and Valkey. The company claimed their software can also improve vector search and enable larger concurrent processing. Pliops showcased its XDP LightningAI at the Summit, a GenAI native memory stack to power inference and retrieval workloads for hyperscale and enterprise applications. The product consists of an ASIC called the Extreme Data Processor (XDP), a software stack, distributed nodes, and uses a GPU-initiated Key-Value I/O interface. According to Pliops, deploying XDP LightningAI in data centers offers significant cost savings: instead of needing four times as many GPU servers to support 4X more users for LLM inferencing, existing GPU infrastructure can be maintained while adding XDP LightningAI servers, resulting in 67% optimization in rack space, 66% reduction in power consumption, 58% annual OpEx savings, and 69% decrease in initial investment costs. Pliops is collaborating with Tensormesh, inference optimization software, with the combined deployment of LightningAI and Tensormesh's shared KV cache architecture delivering fast time-to-first token and GPU savings across multi-GPU clusters.

Charles Alpert from Cadence, an electronic design software company that enables device and system simulation, discussed various challenges in AI infrastructure including energy consumption—driving GW data centers—thermal management, and the time to operationalize that infrastructure. He explained how improvements can come through adding AI for design alongside designing for AI to create continuous improvements in data centers and their devices in a virtuous cycle. Cadence has tools for data center design as well as traditional semiconductor design, including tools that work with 3D stacks of die and Multiphysics digital twin simulation. Over half the chips built today use AI technology, Alpert noted, with this expected to accelerate to 90% in the next few years using agentic AI. Agentic AI should lead to levels of autonomous design similar to those used for autonomous driving, with Cadence's new EDA tools offering multi-physics capability to enable designing 3D devices made from stacking semiconductor die—often called heterogeneous integration—which requires massive system level integration and is resource intensive.

Cadence's Palladium Emulator, which Jensen Huang from Nvidia called an essential tool for designing NVIDIA's most advanced chips, including the Blackwell series, has been expanded beyond chip design toward creating digital twins of data centers including all functional components. The company's Millennium M2000 system enables much faster chip design as well as system design, serving as a tool for Cadence's Digital Twin Ecosystem for data center design. Together, these three companies demonstrated how addressing different aspects of AI infrastructure—memory sharing for inference acceleration, memory stack optimization, and digital twin design tools—can drive improvements across the industry.