Topic: quantization

Latest on “quantization”

The AI news that matters for people who make things — what happened, and how to put it to work in Grafy.

CodeQuant2 Introduces Adaptive Clustering for Low‑Precision Reasoning Model Quantization

Researchers unveil CodeQuant2, a clustering‑based LUT quantization framework that mitigates early‑step activation spikes in large reasoning models.

Try in Grafy Codequantizationlarge reasoning modelslow precisioncodequant2

Arxiv AI Recent12 h ago4 min read

FOSCAR boosts LLM decode attention 3.4× using 2‑bit KV caches

Researchers unveiled FOSCAR, a kernel library that folds affine transforms into low‑bit KV caches.

Try in Grafy Codellmkv-cache

MachineLearningMastery6 days ago8 min read

A Decision Tree for Choosing LLM Inference Optimizations

MachineLearningMastery published a decision tree for choosing LLM inference optimizations by bottleneck. Read on for the six questions and trade-offs.

Try in Grafy Chatllm inferenceinference optimization