Zhipu deploys its Recursive System Intelligence (RSI) framework to optimize compute efficiency and win the 30-billion-unit compute cost war.

On September 17, Tang Jie unusually published a lengthy post addressing one of the most prominent industry topics: RSI. Zhipu has also been quietly advancing its RSI strategy, embedding it at the infrastructure layer to build an inference system powered by GLM-5.3. This very infrastructure enabled GLM-5.3-Flash—previously known globally under the anonymous moniker Ox-Alpha—to rapidly become one of the most widely deployed models online within a week of its release, processing over 62 trillion tokens in just six days. More importantly, this entire operation ran exclusively on 100,000 domestic chips. Historically, reliance on NVIDIA stemmed from the weak underlying software ecosystem of domestic chips, fraught with countless operator compatibility and communication challenges. At a scale of 100,000 chips, complexity escalates exponentially; hundreds of cards may experience hardware failures, network disconnections, or packet loss daily. Yet, as Tang Jie revealed, Zhipu managed to transition from the first successful full-scale run to routing all real-world production traffic in merely two weeks—a process that typically requires seasoned infrastructure engineering teams several weeks or even months to complete.

Facing imperfect hardware ecosystems, Zhipu applied software patches to hardware and pushed optimization to the extreme. The system not only successfully ran large models but also boosted end-to-end throughput by 3.2 times. This reflects Zhipu’s core strategic judgment: under strict external compute restrictions, whether China can achieve AGI breakthroughs depends not merely on the number of chips owned, but on how efficiently software can mobilize and utilize compute. By leveraging RSI, Zhipu aims to establish a bottom-level decoupling capability, enabling AI to autonomously adapt across different hardware ecosystems and overcome compatibility barriers. In essence, Zhipu seeks to transform the industry-wide challenge of domestic compute being difficult to use into technical initiative where AI automatically mines hardware limits. Combining domestic chip clusters, an AI-deeply-involved infrastructure system, and a globally recognized model, Tang Jie remarked, “We are still far from the ultimate goal of RSI, but that minimal loop of ‘the model optimizes the system, and the system feeds back to the model’ has truly begun to turn.”

Deploying a large model across 100,000 domestic chips is far from trivial. Tang Jie noted at least three major hurdles, each capable of overwhelming even experienced engineering teams. First, domestic chips inherently lack the memory bandwidth and data transfer speeds of NVIDIA, making direct replication of mature NVIDIA workflows impossible. Second, the software stack remains highly immature, with many foundational components missing. There are no ready-made operational manuals, many critical operators lack implementations, and extensive low-level documentation often requires speculative deduction and validation. Third, the model’s own load limits must be considered. GLM-5.3-Flash features a novel architecture designed to simultaneously process text and images while supporting a 1-million-token ultra-long context window. This causes KV cache capacity pressure to skyrocket geometrically.

Zhipu’s solution centers on a GLM-5.3-driven Infra Agent that integrates key technologies to compensate for hardware shortcomings through software engineering. For instance, it trades communication for memory by employing intra-node tensor parallelism and layer splitting, dividing the model by layers while allowing multiple chips within a single server to jointly handle core attention and output computations. It trades computation for memory via the ReplaySSM strategy: when memory is insufficient, data is temporarily discarded and recalculated on-demand by the chips, exchanging minimal computation time for valuable memory space. It trades precision for capacity using W8A8 quantization, compressing both model weights and activations to 8 bits to cut storage and bandwidth usage by over half. Additionally, the most memory-intensive KV cache undergoes dynamic compression mixing INT8, FP8, and BF16 precisions based on data importance, maximizing per-card VRAM utilization. Building on this, Zhipu further decouples the encoding, prefill, and decoding stages to gain scheduling flexibility. Coupled with rapid Infra Agent iterations, domestic chip utilization and single-token costs now approach NVIDIA’s levels.

However, the most critical challenge arises when the Infra Agent stalls—not because it cannot write code, but because it lacks understanding of why conditions deteriorated. Real-world inference systems are not static codebases but dynamically intertwined ecosystems spanning bottom-layer chips, network communication, memory allocation, and upper-layer services. Any minor deviation triggers cascading effects, a phenomenon known in systems engineering as sparse feedback. The system merely reports outcomes, such as throughput dropping by 20%, leaving engineers unaware whether the root cause lies in low-level code, memory scheduling, or data transmission, which specific change caused the crash, or which direction to proceed next. If AI were to troubleshoot manually, each attempt would require hours of full-cycle testing, making exploration prohibitively slow due to high trial-and-error costs. Traditional system optimization is already saturated with logs, tests, and performance analysis tools, yet they remain fragmented across engineering phases. Experienced infrastructure engineers rely on engineering intuition to pinpoint issues. Zhipu’s objective is to quantify this intuition into a feedback mechanism, enabling AI to trace and verify problems accurately, cost-effectively, and in real time. Zhipu believes future generational gaps among tech giants will be determined by the depth of model participation in constructing its own systems. Whoever closes the loop on model-training infrastructure first will capture compounding intellectual growth, achieving non-linear leaps despite physical compute constraints. To address this, Zhipu developed a layered verification framework called Dense Feedback.

Zhipu decomposes Dense Feedback into three components: correctness feedback, system behavior feedback, and performance feedback. These correspond to three fundamental infrastructure questions: Is the calculation correct?, Where is the bottleneck?, and Which solution is optimal? Understanding this system is best illustrated through three real-world engineering cases demonstrating how human engineering intuition is encoded into AI capabilities.

Case One: Long Context Accuracy Drift (Correctness Feedback). When handling long contexts across multiple chips, the system partitions text segments for parallel processing before concatenating results. This frequently causes accuracy drift, increasing output deviations. A senior infrastructure engineer would trace the execution chain step-by-step, capturing intermediate results and plotting error curves to identify which concatenation step corrupted the data. Zhipu codified this logic into a correctness feedback system. Acting like a microscope, the AI instantly identified that the KDA kernel’s default low-level precision caused rounding errors to snowball. Upon pinpointing the issue, the AI automatically matched solutions from an open-source repository, upgrading the computation to a higher-precision composite algorithm to resolve the drift.

Case Two: KV Transmission Concurrency Bottleneck (System Behavior Feedback). During large model inference, the prefill stage processes user prompts and generates required caches. Ideally, the system should compute on the GPU while simultaneously transferring cache data across nodes. However, combined execution time proved 20% slower than computation alone, indicating data movement was queuing rather than overlapping with processing. Traditionally, resolving this required painstakingly reviewing massive full-chain timing diagrams, cross-referencing low-level C++ source code such as DeepEP, and speculating whether an unrelased global interpreter lock (GIL) paralyzed the Python thread responsible for data transfer. Zhipu transformed this workflow into a system behavior feedback mechanism, effectively equipping the AI with a full-chain X-ray scanner. It automatically pulls execution timelines to verify computation-communication overlap, flagging abnormal time slices in red. Upon detecting anomalies, the AI traces the call chain across Python and C++ boundaries, inspecting lock states at the lowest interface level. Once located, it auto-generates fixes and validates them against both timing and performance metrics, ensuring actual overlap and end-to-end throughput gains. Problems that previously took days to resolve are now fixed in seconds.

Case Three: Decode Kernel Redundant Computation (Performance Feedback). Text generation relies on a core computational module—the decode kernel. If inefficient, it bottlenecks overall speed. Senior engineers typically inspect low-level assembly or operator code manually. To maximize multi-chip parallelism, systems often employ tiling strategies, splitting massive tasks into four concurrent blocks. This inadvertently forces shared preliminary steps to be recalculated four times, a classic case of parallelism for parallelism’s sake. Zhipu distilled common kernel optimization patterns into a reusable optimization template library, integrated with performance feedback to give the AI an automated search engine. Performance feedback first identifies the bottleneck; the AI then queries the template library. This library was curated by analyzing expert-written kernels from projects like SGLang, Flash Linear Attention, and DeepGEMM. Upon matching a scenario, the AI directly applies the optimized pattern to refactor code, runs a micro-test to validate efficacy, retains successful implementations, and feeds them back into the library, creating a compounding improvement cycle. From problem identification to resolution, AI troubleshooting now operates at the operator precision level, transcends language boundaries, and accelerates compounding engineering gains.

The future of infrastructure RSI envisions AI engineers possessing senior system diagnostics capabilities, executing faster and more granularly at the engineering layer, while human engineers oversee risk management and business decisions. The infrastructure system evolves into a high-speed, automated pipeline.

This shift from selling tools to selling efficiency underscores how RSI was compelled by market demand. Historically, Zhipu earned recognition as the domestic Claude due to its robust coding capabilities, capturing the majority of China’s coding market. However, this year marked a decisive strategic pivot toward foundational infrastructure. Why this major shift? Insights from Zhipu’s September 16 investor conference call reveal the rationale.

Zhipu executives disclosed that following the February launch of GLM-5, market demand surged explosively, driving API call volumes up tenfold and completely depleting compute reserves within the launch week. Severely constrained by compute scarcity, Zhipu was forced to urgently suspend its primary revenue-generating product, the Coding Plan. Unable to meet surging demand, Zhipu adopted an All-in-Infra strategy and swiftly acquired Zhongke Jiahe, boosting overall compute utilization efficiency by two to three times. Even so, the gap left by the Coding Plan remained unfillable. Compute emerged as Zhipu’s core growth constraint: given sufficient compute, the company could immediately convert product availability into direct revenue. Market validation arrived quickly. Following a $4 billion funding round in July, Zhipu fully relaunched the Coding Plan after a six-month hiatus, with sales jumping over 15 times, reaffirming the revenue scales directly with available compute principle.

Zhipu’s leadership meticulously modeled compute economics. A saturated upfront investment of 30 billion RMB in compute—allocated 40% to model training and 60% to inference—would yield substantial returns. If the 60% inference allocation operated at full capacity, leveraging GLM-5.3’s theoretical 80% gross margin, annual revenue could surpass 40 billion RMB. This would recover all compute expenditures within a single year while simultaneously covering training and R&D costs. Conversely, a conservative 10 billion RMB investment would inevitably stall R&D, lose orders, and result in missed opportunities on both fronts. For large model companies, scaling compute is therefore a mandatory, non-negotiable trajectory.

To execute this, Zhipu deployed a three-tier strategy. First, it constructed a 1GW ultra-large compute data center to solve compute availability, while exclusively utilizing domestically produced chips to control costs and mitigate long-term supply chain risks. Second, it implemented a cloud revenue-sharing model, transforming open-source models into commoditized cloud offerings. Zhipu signed revenue-sharing agreements with multiple overseas cloud hyperscalers and leading domestic cloud providers. Starting in October, GLM-series open-source models will be listed as hosted APIs across these platforms, effectively outsourcing compute pressure, operational costs, and global distribution while pursuing light-asset monetization. Third, in the enterprise co-work segment, GLM-5.3 achieved early validation in cybersecurity scenarios. One hundred security firms have already integrated GLM models, spanning mainstream vendors, internet giants, research institutions, and financial sectors. With Gartner projecting global information security terminal spending to reach $248.9 billion in 2026 and $372.6 billion by 2030, Zhipu faces substantial expansion potential. Ultimately, Zhipu’s heavy reinvestment in infrastructure is the inevitable evolution following the Coding demand explosion.