An independent researcher created SGOCR, an open-source dataset pipeline for spatially-grounded, OCR-focused VQA, to fill a gap in visual datasets for text grounding in imagery. This pipeline generates VQA tuples with rich metadata, supporting diverse VLM training strategies.
O usuário busca um método eficaz para detectar texto invertido em selfies antes de passá-los para modelos de Visão-Linguagem (VLM) ou extratores de embedding facial, que são insensíveis a essa inversão devido ao treinamento com dados aumentados. Sua ideia é usar OCR (EasyOCR) para comparar a pontuação de leitura de texto normal versus espelhado, questionando se esta é a melhor abordagem ou se existe uma solução de modelo menor e mais inteligente.
Vision-Language Models (VLMs) often misinterpret interactive charts due to a "Pixel-Only Bottleneck," treating them as static images. This paper introduces Introspective and Interactive Visual Grounding (IVG), a framework combining spec-grounded introspection and view-grounded interaction to resolve visual ambiguities, significantly improving QA accuracy.
This research paper demonstrates that embedded numeric anchors on images systematically bias Vision-Language Model quality judgments across multiple VLMs. Layer-wise probing reveals that optimal layers for quality prediction are deeper than where anchor classification saturates, establishing a causal account of visual anchoring bias.
Pet-tech startup Tomofun is leveraging EC2 Inf2 instances powered by AWS Inferentia2 for cost-effective deployment of vision-language models for pet behavior detection. This strategy allows the company to significantly reduce costs while maintaining the accuracy of its systems.
F-VLM introduces a novel approach for open-vocabulary object detection by efficiently leveraging frozen pre-trained vision and language models. This method allows for identifying a wide range of objects without requiring specific training data for each new category.
This article discusses the evolution of multimodal AI models, which are transitioning from research to production APIs by 2026, integrating text, images, audio, and video. It covers current capabilities, architectures, and production patterns for these applications, featuring models like GPT-4o and Claude.
PaliGemma 2 is introduced as a new family of versatile Vision-Language Models (VLMs) specifically designed to excel in various transfer learning applications. This advancement aims to improve performance across diverse multimodal tasks through effective knowledge transfer.
GazeX is a new vision language model trained on radiologists' eye-tracking data and reasoning to improve chest X-ray interpretation. The model learns to emulate expert spatial and temporal attention, aiming to bridge the gap between model outputs and clinical diagnostic reasoning.
MobileVLM V2 introduces a new and enhanced baseline for vision language models, focusing on faster performance and stronger capabilities. This research aims to advance the efficiency and robustness of VLMs on mobile platforms.
The paper introduces 'Spatial Forcing,' a method for aligning implicit spatial representations in vision-language-action models. It aims to enhance these models' understanding and interaction with spatial information.
This article details the process of fine-tuning OpenCLIP ViT-B/32 for architectural styles, achieving a +26 percentage point increase in accuracy. The author focuses on the critical decisions made before and after the training loop that were responsible for this significant result, rather than the training loop optimization itself.
GAZE is a framework enabling medical Vision-Language Models (VLMs) to iteratively analyze brain MRI images using viewer-level tools and literature retrieval. It achieved 58.2 mAP for lesion localization and 34.9% Top-1 diagnostic accuracy on the NOVA benchmark for rare neurological conditions.
Este artigo propõe o DLR, um framework de raciocínio latente reforçado para Vision-Language Models (VLMs) que melhora o raciocínio visual complexo, superando a perda de informação em CoT textual. Ele decompõe dinamicamente consultas, extrai latentes visuais e deduz respostas, oferecendo maior interpretabilidade e superando baselines em benchmarks vision-centric.
This research defines and investigates source-modality monitoring in Vision-Language Models (VLMs), examining their ability to track the origin of information. It evaluates how VLMs use syntactic and semantic signals to bind input sources, finding both are crucial but semantic signals often dominate, with implications for model robustness.
This work explores neuro-symbolic language reasoning in VLMs, leveraging Reinforcement Learning to improve analytical abilities and efficiency. It achieved a 3.33% accuracy increase on a vision-language evaluation dataset while reducing reasoning tokens by 75%.
Este artigo de pesquisa propõe uma nova abordagem para otimização de documentos, transformando-os para melhor alinhamento com sistemas de recuperação via Reinforcement Learning (GRPO), utilizando melhorias de ranking como recompensa. O método, aplicável a retrievers de caixa preta, demonstrou ganhos em tarefas de recuperação de código e documentos visuais.
Researchers propose Demo2Reward, a test-time adaptation technique to optimize Vision-Language Model (VLM) reward models in robotics. It uses a few demonstrations to reduce false positives while preserving true positives, without requiring additional model training.
This research tests the "Attention-Confidence Assumption" in Vision-Language Models (VLMs), finding that attention structure is a near-zero predictor of correctness. The study uses a unified mechanistic pipeline (VLM Reliability Probe) to analyze attention, generation dynamics, and hidden-state geometry in three VLM families.
A new training-free inference framework, Positive-and-Negative Decoding (PND), is introduced to address object hallucination in Vision-Language Models (VLMs). PND enforces visual fidelity by using a dual-path contrast mechanism, leading to state-of-the-art performance without retraining.