Samsung has demonstrated LPDDR5X-PIM, a version of low-power DRAM that places processing units inside the memory itself. Presented at Hot Chips 2026, the technology is designed to reduce the volume of data shuttled between memory and a separate AI processor during inference—one of the central performance and energy constraints facing generative AI on phones, PCs and other power-limited devices.
In a Llama 3.1 8B inference test, Samsung reported token throughput roughly 3.01 times higher than conventional LPDDR5X while retaining the same package footprint, according to independent reporting on the presentation.[1] That is a consequential demonstration, but not yet a declaration of broad product readiness: the result is from one company presentation, one model and a specific test configuration. The key question is whether the approach can deliver comparable gains across real workloads, software stacks and high-volume memory products.
By the numbers
- 3.01x: Reported AI-inference speed improvement versus conventional LPDDR5X in Samsung’s Llama test.
- Llama 3.1 8B: The language model used in the reported demonstration.
- 8x: Bandwidth figure highlighted in reporting on Samsung’s Hot Chips presentation.[1]
- Same package footprint: The reported physical-package comparison with standard LPDDR5X.
Why memory movement has become an AI hardware problem
Modern AI accelerators can perform enormous numbers of arithmetic operations, but inference is often constrained by a less glamorous task: feeding those operations with model weights and intermediate data quickly enough. Large language models repeatedly read vast parameter sets from memory. If the compute engine must wait for data to travel across a memory interface, performance falls and energy is spent moving bits rather than producing useful output.
This is commonly called the memory wall. It becomes particularly acute for autoregressive language-model inference, where each newly generated token depends on earlier computation and the model must repeatedly access weights. Faster conventional memory helps, as do larger caches, wider interfaces and more powerful neural-processing units. But each approach has cost, power, packaging or capacity trade-offs.
Processing-in-memory, or PIM, takes a different route. Rather than moving all relevant data to a processor, it moves selected operations closer to the data already stored in DRAM. Samsung’s LPDDR5X-PIM concept integrates logic units into the memory architecture so that some AI work can be performed in or near the memory arrays. The objective is not to replace a CPU, GPU or NPU with DRAM. It is to offload the repetitive, data-intensive portions of a workload that suffer most from conventional memory traffic.
That distinction matters. The strongest argument for PIM is efficiency: fewer transfers can mean lower latency, less interface congestion and lower energy per inference. In devices where the battery, thermal envelope and board area are tightly controlled, those gains can be more valuable than simply adding another block of peak compute throughput.
What Samsung demonstrated
Samsung’s presentation centered on LPDDR5X, the low-power DRAM family used widely in mobile devices and increasingly in AI-capable PCs. The company added processing capability to the memory rather than proposing a wholly separate accelerator package. Reporting on the Hot Chips session said the design delivered 3.01 times the inference performance of standard LPDDR5X on Llama 3.1 8B and did so without increasing package footprint.[1]
The footprint claim is commercially important. Mobile and compact-client hardware is constrained not only by watts but also by millimeters. A design that requires extra memory packages, a larger logic die or a more complicated board may erase much of its system-level value. Retaining the established LPDDR5X package envelope would, if it translates into production hardware, give Samsung a more practical integration story than a discrete PIM component aimed only at specialized servers.
The reported bandwidth figure also frames the intended benefit. AI inference is sensitive to effective bandwidth—the useful rate at which the system can access model data—not merely the signaling speed at the edge of a DRAM package. Local processing can raise effective bandwidth for supported operations because data need not traverse the external interface for every step. That is different from claiming that every application will see an eightfold improvement in system memory bandwidth.
Samsung is the central player in this announcement, both as a leading memory supplier and as a company with a broad device and semiconductor business that could provide future deployment paths. Hot Chips, the semiconductor-architecture conference where the work was presented, is an important venue for early disclosure of technical designs, but a conference presentation remains earlier evidence than a qualified, shipping memory part supported by a broad software ecosystem.
The opportunity: smarter DRAM in familiar device categories
LPDDR5X-PIM could be relevant wherever local AI models are restricted by memory traffic: smartphones, tablets, laptops, edge gateways, robots and embedded systems. These devices increasingly run speech recognition, translation, image generation, computer vision and language assistants locally for privacy, response time or cloud-cost reasons. Yet many cannot accommodate the power consumption, cooling and memory capacity of data-center-class AI systems.
For device makers, in-memory processing creates a possible third path between two familiar choices: run a model on a conventional local accelerator or send the request to the cloud. A local system that generates tokens faster while using less energy could improve responsiveness and allow more tasks to remain on-device. It could also reduce the need to overprovision the main NPU simply to overcome a memory bottleneck.
For the memory industry, the idea raises the value of DRAM beyond capacity, speed grade and power rating. Samsung and its rivals have already pursued higher-bandwidth memory for AI servers, where stacked HBM is becoming strategically important. LPDDR5X-PIM points toward a related but distinct market: adding intelligence to ubiquitous low-power memory for the edge. If the architecture matures, competition could shift toward memory products differentiated by supported operations, compiler integration, firmware and validated AI-model performance.
Promises, constraints and the evidence still needed
A 3.01x gain on a named model is encouraging, but it does not establish a universal multiplier. Inference performance depends on model architecture, parameter precision, batch size, sequence length, quantization method, memory capacity, host processor and the fraction of the workload that can actually run on the PIM units. A benefit in Llama 3.1 8B may not map directly to multimodal models, vision transformers, recommendation systems or smaller models that are already compute-bound.
Programming is another major constraint. PIM hardware needs compilers, runtimes and model-serving software that can identify suitable operations, partition work between the host accelerator and memory, and schedule transfers without creating new overhead. Developers will expect frameworks to make this largely invisible. If every model requires hand-tuned kernels or a bespoke deployment path, adoption will be limited even if benchmark results are strong.
Memory manufacturing and qualification also demand caution. DRAM is produced at enormous scale with exacting requirements for yield, reliability, thermals and cost. Introducing logic into a low-power memory design can add complexity, and any power consumed near the arrays must be managed without compromising the characteristics that make LPDDR attractive. Customers will want details on capacity options, data rates, total system power, error handling, software support and cost before treating the demonstration as a product roadmap.
The most valuable next disclosures would therefore be broader benchmarks, energy-per-token measurements, supported precision formats, the share of an inference pipeline executed in memory, and evidence from a production-oriented device platform. Comparisons against modern NPUs and aggressively optimized standard LPDDR systems would be especially useful. Those data would show whether Samsung’s result reflects a durable architecture advantage or a narrow benchmark optimization.
A wider shift from peak compute to data locality
The industry is not abandoning faster AI processors. It is recognizing that compute alone does not solve workloads whose cost is dominated by fetching and moving data. HBM, larger caches, chiplet interconnects, near-memory accelerators and PIM designs all address variants of the same problem: keeping computation close to the data it needs.
Samsung’s LPDDR5X-PIM demonstration is notable because it applies that principle to low-power DRAM rather than only to premium data-center hardware. If production versions can preserve familiar LPDDR integration while offering measurable performance-per-watt gains, the technology could make local AI more capable without demanding dramatically larger batteries or more elaborate cooling. That would be a meaningful change in the hardware economics of on-device inference.
For now, the announcement should be viewed as a strong technical signal rather than a shipping-product verdict. Samsung has shown a plausible way to attack an increasingly important bottleneck. Commercial impact will depend on whether the design, its software and its manufacturing economics can make that advantage repeatable beyond a single conference benchmark.
Editor’s Take
I see the most practical value here in the choice of LPDDR5X. AI hardware headlines often focus on giant accelerator clusters, but a memory-side capability that fits the physical constraints of a phone or thin laptop could matter to far more products. Reducing traffic is usually a cleaner systems-engineering win than continually raising clocks or adding peak TOPS that a memory-starved model cannot fully use.
The 3.01x result deserves attention, not extrapolation. I would watch for joules per token, model coverage, compiler support and a customer device—not another isolated bandwidth figure. If Samsung can make PIM accessible through mainstream AI software and retain the cost and reliability expectations of LPDDR, smarter memory could become one of the more useful enablers of local AI. Until then, this is a promising architecture demonstration, not proof that conventional LPDDR has been displaced.
