Cloudflare released Unweight, a lossless compression system that reduces LLM size by 15-22% without sacrificing output accuracy. The tool, which saves roughly 3 GB of VRAM on Nvidia H100 GPUs for Llama-3.1-8B, has been open-sourced on GitHub with plans to extend compression.
This project introduces the Python library "torch-nvenc-compress," which leverages the GPU's NVENC/NVDEC hardware to compress LLM activations and KV cache, aiming to overcome PCIe bandwidth bottlenecks in multi-GPU setups. It measures a parallel-path overlap at 67% of theoretical max, improving communication between consumer GPUs.
![torch-nvenc-compress: GPU NVENC silicon as a PCIe bandwidth multiplier — PCA + pure-ctypes Video Codec SDK wrapper. Parallel-path overlap measured at 67% of theoretical max on a real GEMM + encode workload. [P]](/cdn-cgi/image/width=3840,quality=75,format=webp/https://external-preview.redd.it/vqLrMLU0urgSqpiud1c7Ilq7WSsJhRPX63HDDrDRN6M.png?width=640&crop=smart&auto=webp&s=0d43a15121928a0c4b5e3a9730e67ff06df77324)
This article explores the relationship between entropy, compression, and intelligence, serving as the first part of a series. It aims to redefine the understanding of these fundamental concepts.
This research introduces generic triple-latent sequence models, which use a running token state and compressed pair-memory to capture higher-order token interactions. These models show improvement over a Transformer baseline on language-model benchmarks, though a retrieval extension enhances recall but is slower.
This article reframes the Fujimoto Infinite Dot Theory (FIDT) from a universal codec to a domain-specific generator for D-FUMT₈ theories. This new positioning, developed with Claude Opus 4.7's collaboration, achieves byte-exact reconstruction and high compression.
This work introduces a B-spline-based decoupling framework for compressing Transformer models. It generalizes existing tensor-based methods, addressing their limitations in numerical instability or limited expressiveness by exploiting the properties of B-splines.
Este estudo explora a compressão de texto gerado por LLMs em regimes com e sem perdas, apresentando métodos que melhoram a eficiência em 2x, como adaptadores LoRA e reescritas concisas. Introduz também a compressão interativa por Perguntas e Respostas (QA), um protocolo que transfere um bit por resposta para recuperar uma parte significativa da capacidade de modelos maiores.
Google recently launched TurboQuant, a novel algorithmic suite and library for applying advanced quantization and compression to large language models (LLMs) and vector search engines. This tool is an indispensable element of RAG systems.
This content discusses the Opus codec, highlighting its capabilities in providing high-quality music coding with low delay. It focuses on the technical aspects that enable efficient and performant audio compression.