CLIP

2 items

ARTICLEDEV.to AI·28d ago

Fine-tuning CLIP on a Niche Domain: How I Got +26pp Accuracy on Architectural Styles and What You Can Apply to Your Own Domain

This article details the process of fine-tuning OpenCLIP ViT-B/32 for architectural styles, achieving a +26 percentage point increase in accuracy. The author focuses on the critical decisions made before and after the training loop that were responsible for this significant result, rather than the training loop optimization itself.

CLIP Vision-Language Models Machine Learning computer vision

RESEARCHarXiv CS.LG·4/21/2026

Matched-Learning-Rate Analysis of Attention Drift and Transfer Retention in Fine-Tuned CLIP

This paper investigates how adaptation methods (Full FT vs. LoRA) and optimization scale jointly shape attention drift and transfer retention in fine-tuned CLIP models. A controlled matched-learning-rate comparison reveals that the learning rate strongly modulates structural change, with Full FT showing marked contraction at higher rates while LoRA remains entropy-positive.

CLIP Optimization attention fine-tuning