Near-memory Dequantization Architecture In Custom HBM for LLM inference (SK hynix)


Researchers from SK hynix published a technical paper titled “StreamDQ: Near-Memory Weight DeQuantization in Custom HBM for Scalable AI Inference Acceleration.” The paper proposes StreamDQ for “a lightweight architectural enhancement that enables on-the-fly dequantization in the memory subsystem for high-throughput, large-batch LLM inference,” and reports “up to 7.08× speedup and ... » read more

Microarchitecture Tailored to 3D-Stacked Near-Memory Processing LLM Decoding (U. of Edinburgh, Peking U., Cambridge et al.)


A new technical paper, "Rethinking Compute Substrates for 3D-Stacked Near-Memory LLM Decoding: Microarchitecture-Scheduling Co-Design," was published by researchers at University of Edinburgh, Peking University, University of Cambridge, University of Chinese Academy of Sciences, and the Hong Kong University of Science and Technology. Abstract "Large language model (LLM) decoding is a majo... » read more

A New Breed Of Engineer


The industry loves to move in straight lines. Each generation of silicon is more-or-less a linear extrapolation of what came before. There are many reasons for this – products continue to evolve within the industry, adding new or higher performance interfaces, risk levels are lower when the minimum amount is changed for any chip spin, existing software is more likely to run with only minor mo... » read more