DeepX Chip Claims 100B AI Inference Under 5 W

DeepX has taped out its second-generation AI accelerator, the DX-M2, and claims it can perform inference for models with up to 100 billion parameters while drawing less than 5 W. According to EE Times, the claim matters because it points toward running sophisticated generative AI on edge hardware that has limited power and thermal headroom, such as robots and drones. The company is a South Korean edge AI chip startup, and the announcement focused on making local AI more practical for battery-powered devices.

DeepX says the DX-M2 can run large models under 5 W

DeepX said it completed the tape-out of its second-generation AI accelerator, the DX-M2, using custom processor-in-memory technology from Samsung. The company said its first-generation DX-M1 was already in mass production and targeted vision applications. The DX-M2 was designed for edge markets including robotics, drones and other devices, and DeepX claimed it could run models with up to 100 billion parameters at less than 5 W.

The chip was designed to handle models with 20 billion to 100 billion parameters, or about 3 billion to 20 billion active parameters, at speeds of 10 to 30 tokens per second depending on the model. DeepX said this was within the same power budget as the M1, which was 5 W. The company also said its neural processing unit could reach 80 TOPS, and that memory bandwidth tripled from 50 to 150 GB/s compared with the first generation. DeepX said its NPU doubled power efficiency through design improvements, and moving to a 2-nm process doubled it again. The M2 could handle large mixture-of-experts language models and supported speculative decoding and TurboQuant, according to DeepX.

The claim is significant for power-constrained design, but it remains a first-party statement. The source did not detail how the performance would be validated or what thermal and software challenges might remain.

What the 6.7x performance claim compares

The performance figure came from DeepX simulations. The company said offloading general matrix-vector multiplication operations from the DX-M2 into LPDDR5x-PIM resulted in a 6.7x gain in tokens per second for Llama3.1-8B compared with the DX-M2 alone. DeepX said the DX-M2 with LPDDR5x-PIM was close to optimal. Its next-generation DX-M3 was expected to support official LPDDR6 PIM technology, which DeepX said could lead to a full theoretical performance advancement of 5x to 8x higher performance.

DeepX founder and CEO Lokwon Kim told EE Times that memory bandwidth was the most important change between the M1 and M2. Kim said the 6.7x performance gain required 2.7x the power consumption overall, but DeepX believed it could get a 2x performance uplift within the same power envelope as non-PIM operation. DeepX said substantial manual compiler effort was required to support this mode, and that a prototype compiler was running based on simulation. The company was making hardware modules to test the setup in a physical environment.

The figure was specific to Llama3.1-8B in DeepX's simulation, so it should not be read as a blanket guarantee for all language models.

Why memory bandwidth and processor-in-memory matter

DeepX explained that most AI inference operations are constrained by moving weights between memory and compute, especially general matrix-vector multiplication, during which random memory accesses are needed and can slow computation. PIM schemes were described as a way to compute memory-bound operations such as GEMV inside the memory chip. DeepX said PIM was set to become a JEDEC standard for LPDDR6, while it had worked with Samsung Memory on integrating the DX-M2 with Samsung LPDDR5x-PIM. DeepX said it spent 1.5 years collaborating with Samsung Memory and received LPDDR5x PIM samples for in-house testing.

The discussion suggested a shift in where compute happens, though the source noted the added complexity. The tradeoff was not simply speed; it involved power, software effort and hardware maturity.

Edge AI goals and practical questions

DeepX said it was targeting key sectors, including working with Taiwanese OEMs to find the right balance for AI PCs such as laptops and workstations. The DX-M2 was designed to bring generative AI to robots, drones and other edge devices, enabling inference of models up to 100 billion parameters at less than 5 W. Kim said in comments reported by EE Times that with 5 W power consumption the chip could go into drones or other battery-powered devices, and that the goal was to enable deployment of sophisticated AI in all devices.

For developers, the practical question is whether such efficiency claims translate into deployable systems. DeepX acknowledged that the extra compute did not come for free, and that compiler support and physical testing remained areas of work. The available source material did not provide independent benchmarks or third-party validation.

Where creators and developers can explore related ideas in Grafy

For teams exploring how language models behave, how prompts and outputs evolve, or how to compare multiple AI systems, Grafy's Chat app at /chat offers multi-model chat, while Grafy's Graph app at /graph maps prompts, outputs and model history. The DeepX story is about custom silicon and inference efficiency, so the connection to Grafy is indirect. Creators and developers interested in local or edge-oriented AI workflows could use Grafy to prototype prompts, compare model outputs and organize experiments before working with specialized hardware. The Grafy News Desk notes that such workflows can help teams test orchestration ideas even before they have access to custom silicon.

To explore this yourself, open Grafy Chat: chat with many AI models side by side.

Frequently asked questions

Does DeepX teach smaller models how to run larger ones?

No. The source material describes an AI accelerator and inference claims, not a teaching method for smaller models. DeepX said the DX-M2 could run models up to 100 billion parameters, but that was a capability claim about the chip. The discussion focused on memory bandwidth, processor-in-memory and performance. It did not describe the chip as teaching a smaller model.

Does the 6.7x token-rate gain apply to every large language model?

No. According to DeepX, the 6.7x simulation result was tied to Llama3.1-8B in one comparison. The company said the DX-M2 with LPDDR5x-PIM produced that gain versus the DX-M2 alone, but the source did not establish that it would be universal across models or workloads.

Does the DX-M2 include LPDDR5x-PIM at launch?

DeepX said it taped out the DX-M2 with access to Samsung custom PIM and received LPDDR5x-PIM samples for testing. It claimed simulations showed offloading GEMV into that memory produced a 6.7x token-rate gain for Llama3.1-8B. The source did not clearly state when such a version would be commercially launched. DeepX was working on compiler support and physical testing.

Source: EE Times

---

Source: Hacker News — front page — DeepX claims inference up to 100B parameters at less than 5 W