entropy

5 items

RESEARCHarXiv CS.LG·4/6/2026

From Broad Exploration to Stable Synthesis: Entropy-Guided Optimization for Autoregressive Image Generation

O artigo analisa a interação entre Chain-of-Thought (CoT) e Reinforcement Learning (RL) na geração de imagens a partir de texto (T2I) usando uma análise sistemática baseada em entropia. Ele revela que menor entropia dos tokens de imagem e do CoT textual se correlaciona com melhor qualidade de imagem, propondo a estratégia Entropy-Guided Group Relative Policy Optimization (EG-GRPO) para otimização com base na incerteza.

Optimization deep learning reinforcement learning Text-to-Image Generation

RESEARCHarXiv CS.CL·4/9/2026

The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?

Este artigo investiga a correlação entre a dinâmica interna de entropia e o raciocínio correto em Large Language Models (LLMs), um enigma ainda sem solução. Propõe a Hipótese de Informatividade Gradual (SIA), que afirma que os modelos raciocinam corretamente ao acumular informações relevantes sobre a resposta por meio de prefixos informativos, um processo reforçado por métodos de treinamento padrão.

information theory LLMs machine learning Reasoning

RESEARCHarXiv CS.LG·15d ago

When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

This research proposes that LLM reasoning is a dynamic decoding state, not a static property, observable through early-stage entropy dynamics during generation. Tasks benefiting from Chain-of-Thought exhibit consistent entropy reduction, interpreted as a phase-transition to a structured reasoning regime.

AI models LLMs Chain-of-Thought Reasoning

RESEARCHDEV.to AI·17d ago

The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models

This research explores the entropy mechanism within reinforcement learning, specifically its application to enhance reasoning capabilities in language models. It investigates how entropy can be leveraged to improve the learning process and decision-making for more robust language model reasoning.

language models reinforcement learning learning Reasoning

DOCTowards Data Science·2/3/2025

Quantifying Uncertainty — A Data Scientist’s Intro To Information Theory — Part 2/5: Entropy

This content provides an intuitive understanding of Entropy and its applications in Machine Learning and Data Analysis. It also includes Python code examples to facilitate learning.

information theory learning machine learning data analysis