The article compares Gemma 4 31B and Qwen 3.5 27B, identifying them as the best models for local use on 24GB GPUs. The author praises Qwen 3.5 27B for its superior reasoning and long-context analysis capabilities without hallucinations, marking a significant evolution.
Este artigo propõe LPC-SM, uma arquitetura híbrida autorregressiva para modelos de linguagem de contexto longo, que separa atenção local, memória persistente, correção preditiva e controle em tempo de execução. O modelo de 158M parâmetros é avaliado, demonstrando melhorias na perda de LM e estabilidade em sequências longas.
The article discusses whether Retrieval-Augmented Generation (RAG) is becoming obsolete due to the rise of large context windows in new LLMs. It argues that RAG remains relevant, primarily due to its cost-effectiveness, lower latency, and efficiency in handling frequently updated proprietary data.
LoSA introduces Locality Aware Sparse Attention to address memory-bound attention and the KV Inflation problem in block-wise diffusion language models, especially for long contexts. It optimizes performance by reusing cached attention for stable tokens and applying sparse attention only to active tokens, significantly reducing KV index loading.
The article describes an experiment where an LLM's (Qwen3.5-35B-A3B with 1M tokens) KV cache is used as a "document store" by prefilling and persisting it to answer queries, eliminating embeddings and vector databases. The AI engineering agent, NEO, autonomously implemented this Cache-Augmented Generation system in just 30 minutes.
This content details a "Divide & Conquer" framework that enables smaller language models to outperform larger ones like GPT-4o on long context tasks. It addresses the performance degradation of LLMs with growing context windows by breaking documents into parallel chunks.