IndexCache, a new sparse attention optimizer, delivers 1.82x faster inference on long-context AI models

Researchers at Tsinghua University and Z.ai built IndexCache to eliminate redundant computation in sparse attention models ...
WinBuzzer

Google’s TurboQuant Algorithm Slashes LLM Memory Use by 6x

Google has published TurboQuant, a KV cache compression algorithm that cuts LLM memory usage by 6x with zero accuracy loss, ...