transformer compression

AutoCompress: Critical Layer Isolation for Efficient Transformer Compression

AutoCompress is a transformer compression method based on the empirical finding that Layer 0 carries disproportionately high task-critical information. Its Critical Layer Isolation (CLI) architecture achieves 2.47x compression on GPT-2 Medium with 59.5% parameter reduction, significantly outperforming a uniform bottleneck baseline.