Samsung integrates processing-in-memory technology onto AMD MI100 GPU, advancing AI accelerator architecture.

Samsung has unveiled a supercomputer containing 96 AMD Instinct MI100 GPUs, each equipped with processing-in-memory (PIM) chips, which the company claims can significantly improve the performance and energy efficiency of training large AI models. The supercomputer was disclosed Tuesday at an industry event in South Korea. Processing-in-memory technology reduces the amount of data that needs to move between the CPU and DRAM by shifting some computation inside the memory itself.

According to Choi Chang-kyu, head of the AI Research Center at Samsung Electronics Advanced Institute of Technology, the cluster was able to train Google's Text-to-Test Transfer Transformer (T5) language model 2.5 times faster while using 2.7 times less power compared to the same cluster configuration without the PIM chips. "It is the only one of its kind in the world," Choi said. Samsung has projected that its PIM technology could reduce annual energy use of a cluster by 2,100 Gigawatt hours and cut 960,000 tons of carbon emissions—equivalent to the amount of carbon 16 billion urban trees would absorb over a decade.

The performance gains stem from Samsung's use of high-bandwidth memory (HBM) on each PIM chip, a technology increasingly used in the industry for high-performance computing and AI workloads. What distinguishes Samsung's HBM-PIM chips from other companies' HBM implementations is that each memory bank on the PIM chip includes a processing unit inside, reducing bottlenecks associated with moving data between the CPU and memory.

Samsung is working to drive adoption of its PIM technology through software development using SYCL, a royalty-free, cross-architecture programming abstraction layer. The company also plans to bring the technology to market through what it calls the AXDIMM, short for accelerated DIMM. Wider adoption would become evident if PIM begins appearing in new supercomputers deployed by research labs, academic institutions, and other organizations in the coming years.