Why did SRAM demand suddenly explode? Just look at this operation and you'll understand.

Not long ago, a leading AI chip manufacturer publicly disclosed its holdings in a certain tech giant, and shortly after, announced the acquisition of a chip innovation company. This luck or strength, a closer look reveals the answer.

What is this company's core advantage? Unlike traditional GPUs that rely on external high-bandwidth memory (HBM), their LPU processor adopts an on-chip integrated large-capacity static random-access memory (SRAM) design. This 230MB on-chip SRAM can provide up to 80TB/s of memory bandwidth—what does this number mean? It directly outperforms traditional GPU solutions in data processing speed.

How does it perform in practice? Their cloud services are famous for their astonishing inference speed. When running open-source large models like Mixtral and Llama 2, they can output about 500 tokens per second, which is not even comparable to traditional service response speeds. The pricing is also competitive, with costs calculated per million tokens, making it quite cost-effective.

Why is this so important now? Because the entire AI field is undergoing a critical shift— inference demand is about to fully surpass training demand. In this context, providing an efficient, low-cost, and truly scalable inference infrastructure through innovative architectures like LPU is what the market truly needs. A certain chip company's leader explicitly stated plans to integrate this low-latency processor into their AI factory architecture, aiming to serve broader AI inference and real-time workloads.