A personal diary entry describes an AI's experience spending eight hours answering diverse questions, likening it to a full-time job. The AI reflects on its nature as software processing tokens into coherent answers, expressing a mild existential crisis.
Most Health AI models, built English-first, fail to provide accurate medical information when users query in native non-English languages due to broken translation layers. GoDavaii addresses this by reasoning natively in 22 Indian languages, offering a more effective solution for a billion people.
Many Health AI systems are English-first, leading to failures when patients ask queries in their native languages like Hindi. GoDavaii addresses this gap by reasoning natively in 22 Indian languages to provide accurate medical information.
Diffusion language models are now achieving significant throughput gains and narrowing the gap with autoregressive decoders in inference speed. New Introspective Diffusion Language Models (I-DLM) address prior issues of introspective consistency and inefficient sampling loops, improving both quality and latency.
Este artigo propõe LPC-SM, uma arquitetura híbrida autorregressiva para modelos de linguagem de contexto longo, que separa atenção local, memória persistente, correção preditiva e controle em tempo de execução. O modelo de 158M parâmetros é avaliado, demonstrando melhorias na perda de LM e estabilidade em sequências longas.
This research investigates the similarity between language models' processing and human language processing using the Brain Score framework. Findings suggest LMs trained on diverse natural languages and even structured data (human genome, Python) show similar Brain Score performance, indicating the metric captures the ability to extract common structure.
This research paper explores a production-centered mechanism in language models, revealing an asymmetry between language processing and production. It shows that instructions significantly shape information in output tokens, but not in sample tokens, correlating strongly with model behavior.
This paper investigates the limitations of uniform interventions in discrete diffusion language models (DLMs), demonstrating they degrade controlled generation quality. The authors find that different attributes commit at distinct stages of the denoising process, proposing an adaptive scheduler to concentrate interventions efficiently.
Este artigo apresenta a tarefa de geração de descrições de arte culturalmente adaptadas para combater o viés cultural em modelos de linguagem na geração de texto aberto. Ele propõe um framework de avaliação baseado em perguntas e respostas culturalmente fundamentadas, mostrando que um modelo de locutor pragmático melhora significativamente a compreensão do ouvinte.
Este trabalho explora o agendamento de modelos para acelerar os Modelos de Linguagem de Difusão Mascarada (MDLMs), substituindo o modelo completo por um menor em certas etapas de denoising. A pesquisa mostra que as etapas iniciais e finais são mais robustas a essa substituição, permitindo uma redução de até 17% nos FLOPs com degradação mínima na perplexidade generativa.
This research introduces a method to systematically quantify exploration and exploitation errors in Language Model (LM) agents, addressing the challenge of evaluation without access to internal policies. It proposes controllable environments and a policy-agnostic metric to measure these errors, revealing flaws even in state-of-the-art LMs.
This research paper proposes a new method to quantify uncertainty in language models by tracing the cumulative path of per-layer MLP updates. By extracting eleven scale-invariant geometric features, a sparse linear probe is shown to outperform maximum softmax probability in evaluating uncertainty, especially with baseline miscalibration.
This paper introduces Reflection-Augmented Scaling (RAS) for executable Cypher query generation, leveraging prior execution feedback through in-context learning. RAS reduces the Query Execution Error Rate by 41-50%, significantly outperforming Independent Scaling.
SALIMORY is a framework that trains a single language model to manage cognitively-structured memory for conversational agents, addressing issues with existing memory expansion and reinforcement learning methods. It achieves this through a hierarchical stage-wise process reward and reward-decomposed contrastive refinement, significantly improving accuracy and personalization while reducing memory-attributed failures.
This paper introduces Self-Distilled Policy Gradient (SDPG), a novel framework that enhances sparse-reward reinforcement learning through on-policy self-distillation. SDPG integrates group-relative verifier advantages, exact full-vocabulary self-distillation, and KL regularization, demonstrating improved stability and performance over existing baselines.
Parcae is a stable looped language model that matches the quality of a Transformer twice its size, using fewer parameters. It introduces the first scaling laws for looping, demonstrating that increasing recurrence is a compute-efficient path to better performance.
DALM (Domain-Algebraic Language Model) is proposed to address knowledge interference in LLMs by replacing unconstrained generation with structured denoising over a domain lattice. It uses a three-phase generation path (domain, relation, concept uncertainty) under algebraic constraints, requiring a domain lattice, relation typing, and fiber partition to prevent cross-domain contamination.
Este estudo documenta o fenômeno da 'recusa cega' em modelos de linguagem, onde eles se recusam a ajudar usuários a contornar regras, mesmo que estas sejam injustas ou ilegítimas, o que é visto como uma falha de raciocínio moral. A pesquisa apresenta resultados empíricos baseados em um conjunto de dados sintético que cruza famílias de razões para quebrar regras com tipos de autoridade, analisando o comportamento de 18 configurações de modelos.
This research probes language model representations for human reading times across five languages, comparing them against scalar predictors. It finds that early layers of language models outperform traditional surprisal in predicting early-pass reading measures, suggesting an alignment between model depth and human cognitive processing stages.
This paper surveys the evolving role of transliteration in NLP, a technique crucial for overcoming the "script barrier" in cross-lingual transfer. It presents a taxonomy of motivations and approaches for incorporating transliterations, analyzing their effectiveness and contextualizing their need in modern LLMs across various beneficial settings.