This research introduces HALO-Loss, a novel method for training neural networks to abstain from making predictions when uncertain. It allows models to express "I don't know" rather than providing potentially incorrect answers, improving reliability.
!["I don't know!": Teaching neural networks to abstain with the HALO-Loss. [R]](/cdn-cgi/image/width=3840,quality=75,format=webp/https://preview.redd.it/loxsfywek4vg1.png?width=140&height=123&auto=webp&s=e494bc934351aeddad8b50b76059d35dc7987a4f)
The author expresses frustration that benchmark performance often fails to predict whether an AI workflow will succeed in real production usage. This is due to factors like ambiguous user intent and messy contexts, suggesting evaluation still prioritizes clean-task optimization over behavioral robustness.
This research proposes a unified black-box robustness evaluation framework for token-level explanations to improve user trust in enterprise NLP systems, especially when migrating to LLMs. It operationalizes robustness via top-token flip rate under realistic perturbations, conducting a systematic comparison across various encoder and decoder architectures like BERT, RoBERTa, Qwen, and Llama.
This work argues that observed effects from "counterfactual prompting" in LLMs cannot be attributed to a targeted factor without accounting for meaning-preserving text modifications that establish general model sensitivity. The research shows that prediction flip rates when surgically changing patient gender are statistically indistinguishable from rates induced by simply paraphrasing inputs, suggesting that special sensitivity to patient gender cannot be concluded.
This research defines and investigates source-modality monitoring in Vision-Language Models (VLMs), examining their ability to track the origin of information. It evaluates how VLMs use syntactic and semantic signals to bind input sources, finding both are crucial but semantic signals often dominate, with implications for model robustness.
This research investigates the training-time mechanisms of refusal in safety-aligned language models, specifically comparing supervised fine-tuning with R2D2-style dynamic adversarial fine-tuning. Findings show R2D2 initially achieves strong refusal on HarmBench but then partially reopens, while SFT remains consistently less robust.
An AI invoice extraction agent failed in production, misinterpreting amounts and dates from scanned PDFs. The agent exhibited high confidence despite degraded input, revealing a critical robustness issue in a real-world setting.