O Occursus Benchmark é uma plataforma de benchmarking de código aberto que testa se múltiplas LLMs colaborando podem superar um único modelo. A ferramenta avalia 22 estratégias de orquestração em quatro provedores de LLMs, usando julgamento cego duplo para pontuar a qualidade das saídas.
StepPRM-RTL is a novel framework that enhances LLM-based RTL code generation by combining stepwise trajectory modeling, process-reward modeling (PRM), and retrieval-augmented fine-tuning (RAFT). It uses dense feedback from a PRM to guide reinforcement-style updates and Monte Carlo Tree Search (MCTS) to enrich the training dataset.
Este artigo critica a técnica de 'chunking' em sistemas RAG, destacando seus problemas de perda de contexto e erros em documentos estruturados, como registros clínicos. Propõe a indexação ciente da estrutura e a sumarização como métodos mais eficazes para lidar com dados complexos.
This article delves into cost-effective alternatives to GPT-4o, revealing how other AI models can offer significant savings for developers. It provides direct cost comparisons, highlighting options like DeepSeek V4 Flash and Qwen3-32B.
This article provides a tutorial on integrating locally hosted open-source large language models such as Mistral, Gemma, and Llama 3 for language tasks like text classification. It demonstrates how to achieve this for free using Ollama and the Scikit-LLM Python library.
ReaComp compiles LLM reasoning into symbolic program synthesizers to overcome the inefficiency and unreliability of LLMs on hard program synthesis tasks. These standalone solvers achieve higher accuracy and efficiency, outperforming LLMs and significantly reducing token usage in neuro-symbolic hybrid settings.
Research reveals that single-position intervention fails to transfer tasks in Llama-3.2-3B despite high probing accuracy, suggesting distributed task encoding. However, multi-position intervention achieves up to 96% transfer, pinpointing for the first time the causal locus of in-context learning task identity.
CHAL (Council of Hierarchical Agentic Language) is a new multi-agent dialectic framework proposed to optimize beliefs in defeasible domains. It addresses current limitations of multi-agent debate for LLM reasoning, where defeasible argumentation is treated as an engine for belief optimization.
This article explores the inherent limitations of AI models, particularly Large Language Models (LLMs), stressing the importance of understanding these boundaries for robust product development. It details hallucination as a key limitation, explaining that LLMs generate plausible, not necessarily true, text without an internal fact-checker.
This paper introduces a two-dimensional early exit strategy for LLM classification tasks, coordinating layer-wise and sentence-wise exiting. The method achieves multiplicative computational savings and speed-ups of 1.4-2.3x over optimal layer-wise early exit for simpler tasks, applicable across various state-of-the-art LLMs.
This research explores the robustness implications of Sparse Autoencoders (SAEs) against jailbreak attacks on Large Language Models (LLMs). Integrating pretrained SAEs at inference time significantly reduces jailbreak success rates by up to 5x and decreases cross-model attack transferability across various LLM families.
This practical playbook guides developers in building AI-core SaaS products, detailing essential changes and new considerations. It covers architectural patterns, LLM integration, agent development, cost control, testing, safety, and multi-tenancy, offering actionable advice for rapid deployment.
Today's top stories feature the merger of speculative checkpointing in llama.cpp to accelerate local LLM inference and a new Ollama multimodal tool for local audio/video analysis. Additionally, a detailed comparison between MLX and GGUF is provided for optimizing Gemma 4 deployment on consumer hardware.
As enterprises adopt large language models, robust AI governance and security are essential to prevent data leaks, regulatory penalties, and reputational damage. A defense-in-depth approach is crucial to mitigate threats like prompt injection and data contamination, ensuring compliance with regulations such as GDPR and the EU AI Act.
This research investigates LLM prompt sensitivity by comparing instruction-based and example-based prompting styles. It finds that despite performance variation, LLMs share common underlying mechanisms, specifically "lexical task heads" which are attention heads that literally describe the task and trigger answer production.
Este conteúdo apresenta CGD-PD, uma camada leve para modelos de linguagem grandes (LLMs) que melhora a resposta a perguntas lógicas de três vias (Verdadeiro/Falso/Desconhecido). Ele aborda falhas recorrentes como inconsistência de negação e previsões 'Desconhecido' epistêmicas, utilizando decisões consistentes e desambiguação baseada em prova para maior precisão.
Nsanku is a systematic benchmark evaluating the zero-shot machine translation performance of 19 LLMs across 43 Ghanaian languages. It utilizes Bible sentences and metrics like BLEU and chrF, with gemini-2.5-flash achieving the highest overall average score.
The paper introduces HealthCraft, a public reinforcement-learning environment designed to evaluate the safety of frontier language models in emergency medicine. It focuses on trajectory-level safety, tool misuse, and clinical pressure, built on a FHIR R4 world state and offering 195 tasks for comprehensive assessment.
This paper proposes SENSE (Semantic Embedding Navigation with Soft-gated Evaluation) to enhance Retrieval-based Speculative Decoding (RSD) for LLMs. SENSE addresses RSD's rigid lexical dependencies by using robust semantic alignment and a soft-gated evaluation module to validate semantic equivalence.
This paper presents a zero-shot multi-label topic classification framework, systematically investigating how per-article knowledge graph augmentation affects its performance. The authors test eight methods across fifteen LLMs and eight multi-label datasets, finding that keyword-enhanced classification is the best performing method in the base framework.