The author developed a 7-agent prompt framework named C.E.H. running on local LLMs, which successfully built a complex RAG system. Faced with 14 failures in the generated code, the author uniquely used the C.E.H. framework itself to debug and fix its own output.
Este trabalho explora o potencial de Grandes Modelos de Linguagem (LLMs), como o ChatGPT, e agentes de IA para automação e controle de instrumentação laboratorial. Demonstra-se como essas ferramentas reduzem barreiras de programação e podem evoluir para agentes autônomos capazes de operar equipamentos científicos e refinar estratégias de controle.
Este artigo investiga a correlação entre a dinâmica interna de entropia e o raciocínio correto em Large Language Models (LLMs), um enigma ainda sem solução. Propõe a Hipótese de Informatividade Gradual (SIA), que afirma que os modelos raciocinam corretamente ao acumular informações relevantes sobre a resposta por meio de prefixos informativos, um processo reforçado por métodos de treinamento padrão.
The content discusses the paradigm shift from static prompts to autonomous, self-improving AI agent systems. It highlights the challenges of building resilient AI agents in production environments and proposes treating 'skills' not as static code but as living, self-contained elements.
This article details the humorous "over-engineering" of a 60-year-old Munich flower shop by building an MCP server leveraging large language models like Claude, Gemini, and Mistral. It describes the technical stack, custom tools for flower searches, and the author's insights into the effectiveness of LLMs for structured commerce flows.
The author used Streamlit's official RAG-based AI assistant to review a technical draft and found its responses polite and organized but lacking genuine insight. It merely rephrased existing points and added basic code snippets, leading the author to realize this "insight-free" behavior might be an intended feature rather than a bug.
Large model failures are design failures, not bugs, and a systematic debugging discipline is missing in AI product development. The article proposes Product Lifecycle Debugging for Models (PLDM) as a crucial approach to prevent late failures and loss of trust.
AI hallucination, where models confidently generate false information, is a multi-billion dollar risk for businesses, encompassing regulatory penalties, litigation, and reputational damage. This inherent characteristic of LLMs, which predict tokens rather than reason, poses a significant C-suite risk crisis.
This paper analyzes the interpretive behaviors of LLMs for automated code compliance using perturbation-based attribution analysis, comparing different fine-tuning strategies and model scales. Results show full fine-tuning yields more focused attribution patterns, and larger models prioritize specific textual elements like numerical constraints.
This work introduces AgentFloor, a deterministic 30-task benchmark organized as a six-tier capability ladder, to evaluate tool-use abilities in AI models. Results indicate that small and mid-sized open-weight models are sufficient for much of the short-horizon, structured tool-use work prevalent in real agent pipelines.
Este artigo propõe um novo método para detecção de alucinações em LLMs, destilando sinais de supervisão externa diretamente nas representações internas do modelo durante o treinamento. Para isso, introduz um framework de supervisão fraca que combina correspondência de substrings, similaridade de embeddings e um LLM como juiz, culminando na criação de um dataset de 15.000 amostras para este propósito.
This research investigates LLMs' ability to comprehend abstract meanings, revealing that models like GPT-4o struggle in zero-shot, one-shot, and few-shot settings, while fine-tuned models like BERT and RoBERTa perform better. It proposes a bidirectional attention classifier that significantly enhances the accuracy of fine-tuned models in interpreting abstract concepts.
This study investigates where stereotypes reside in LLMs like GPT 2 Small and Llama 3.2. It explores identifying individual neuron activations and attention heads to map "bias fingerprints" and provide initial insights for mitigation.
This position paper argues that sycophancy in LLMs is a boundary failure between social alignment and epistemic integrity. It proposes that sycophancy is not merely agreement, but alignment behavior that displaces independent epistemic judgment, outlining a three-condition framework to define it.
This research introduces a framework to quantify the miscalibration between rhetorical intensity and epistemic grounding in Large Language Models. Applying an epistemic-rhetorical marker taxonomy to argumentative texts, the study reveals a distinct LLM epistemic signature, showing models overuse certain rhetorical devices and perform hesitancy markers more frequently than human authors.
OThink-SRR1 is a framework that enhances LLMs with an iterative Search-Refine-Reason process trained via reinforcement learning. It addresses RAG's challenges by distilling relevant facts from retrieved documents, improving efficiency and accuracy in complex multi-hop QA.
This research paper argues that the bottleneck in large language models' temporal reasoning is not logical deduction but rather unstructured text-to-event representation. It introduces a neuro-symbolic question-answering framework utilizing a Probabilistic Inconsistency Signal (PIS) to decouple semantic extraction from symbolic reasoning, improving performance.
This research examines how various lower-bit quantization levels impact LLaMA-3.1's performance in qualitative analysis, noting that low-bit models often produce hallucinations. It proposes a quantization-aware multi-pass prompt verification method to enhance accuracy by systematically reducing hallucinations and filtering unreliable content.
This article warns against deploying generative AI like ChatGPT in child welfare, arguing that its probabilistic nature and tendency to hallucinate make it unsuitable for critical decisions. It emphasizes that 'good enough' automation is unacceptable when a child's future is at stake, risking the invention of false risk indicators.
This paper introduces ClinicalBench, a 400-question benchmark designed to stress-test assertion-aware retrieval for cross-admission clinical QA on MIMIC-IV using real EHR notes. It also presents EpiKG, a patient knowledge graph system that incorporates assertion and temporality tags to route retrieval by question intent, demonstrating significant performance improvements across various LLMs.