Lumai, an optical computing company spun from Oxford research, argues that AI infrastructure must specialize hardware fo

As frontier AI companies project a roughly 1,000x increase in effective compute demand over the next five years, delivering this capacity using conventional digital accelerators would require an estimated 100 trillion dollars in infrastructure investment and around 1,000 gigawatts of additional electrical capacity. Prefill, which processes the input context before generation begins, and decode, which generates output tokens, have fundamentally different computational characteristics. Prefill is dominated by highly parallel matrix multiplication and is compute-bound, while decode is driven more heavily by memory bandwidth and data movement.

As leading AI infrastructure providers move toward disaggregating prefill and decode into separate infrastructure pools, Lumai sees the next logical step as specializing the hardware powering each stage. According to Phil Burr, Head of Product at Lumai, "If the workloads are fundamentally different, it makes sense to stop asking the same hardware to do both jobs. Disaggregating prefill and decode is an important step. But the bigger opportunity is to match the compute architecture to the workload."

AI applications are changing the economics of inference through longer context windows, retrieval-augmented generation, multimodal inputs, and agentic workflows, all of which increase the information that must be processed before token generation begins. A single user request can now trigger multiple model calls, each carrying accumulated context from everything that came before it. Today, a growing share of inference compute runs on hardware built to do two jobs adequately rather than one well. As context length grows or an agentic workflow adds another interaction or call, more of the available power is consumed by prefill, widening the gap between the inference capacity a watt could deliver and what it actually delivers.

At scale, this creates three infrastructure challenges. First, power becomes the binding constraint, as data centers effectively convert a fixed megawatt budget into inference capacity, and the efficiency of the prefill tier determines the quantity of useful compute that power can deliver before a model generates its first token. Second, GPU capacity ends up stranded, as GPUs absorb more prefill computation and increasingly process context instead of generating tokens for which they are better suited. Third, inference economics becomes a constraint, as the cost of processing context can make long-context and agentic applications increasingly difficult to price competitively. Burr notes that "the real issue is what happens at scale. Power is fixed, GPU capacity is finite, and every interaction adds more context to process. We have gotten to the point where the economics of inference comes down to the efficiency of the hardware running it."

Lumai Iris Nova uses light rather than electricity to perform the matrix multiplications that define the prefill workload, completing each vector-matrix multiplication in a single optical cycle. The significance is that optical computing creates the possibility of treating prefill as a purpose-built infrastructure workload rather than running it on general-purpose silicon. In Lumai testing, Iris Nova ran billion-parameter large language models in real time and demonstrated approximately 10 times more compute per watt than GPU-based equivalents on prefill workloads, with up to 90 percent lower energy consumption than conventional GPU architecture. Moving prefill onto purpose-built hardware frees GPU capacity for token generation, enabling existing infrastructure to be more productive without wholesale replacement.

Lumai, the optical compute company spun out of world-leading optics research at the University of Oxford in 2021, is building next-generation AI infrastructure for the Inference Era. The company is the recipient of the Falling Walls Award for Science Breakthrough of the Year 2025, was part of Intel Ignite's first London cohort, and won Best Overall Technology at the OCP Future Technologies Symposium. Burr concludes that "our view is not that every accelerator needs to be replaced—it is that we should stop asking every accelerator to solve every problem. The future AI data center will be increasingly heterogeneous, with different technologies optimized for different stages of inference."