Trials and tribulations fine-tuning & deploying Gemma-4 [P]

An ML team documented the technical challenges faced while fine-tuning and deploying Gemma-4. Key issues included PEFT's incompatibility with Gemma 4's custom layers, SFTTrainer silently breaking KV-sharing attention, and DeepSpeed ZeRO-3 saving half-empty LoRA adapters.