Revealing Interpretable Failure Modes of VLMs

Vision-Language Models (VLMs) can exhibit catastrophic failures in real-world situations despite their broad reasoning capabilities. REVELIO is introduced as a framework to systematically uncover interpretable failure modes in VLMs by combining diversity-aware beam search and Gaussian-process Thompson Sampling to map the failure landscape.