LLM Efficiency

NVIDIA says TensorRT Edge-LLM ran Qwen3.6-27B 6.4x faster on Jetson AGX Thor, highlighting cache and quantization gains for edge agents.

Google Research has publicly released TurboQuant, a training-free AI memory compression algorithm suite that delivers a 6x reduction in KV cache memory usage and an 8x speedup in attention computation, potentially cutting enterprise AI inference costs by more than 50%.