Research reveals that single-position intervention fails to transfer tasks in Llama-3.2-3B despite high probing accuracy, suggesting distributed task encoding. However, multi-position intervention achieves up to 96% transfer, pinpointing for the first time the causal locus of in-context learning task identity.
This paper introduces Reflection-Augmented Scaling (RAS) for executable Cypher query generation, leveraging prior execution feedback through in-context learning. RAS reduces the Query Execution Error Rate by 41-50%, significantly outperforming Independent Scaling.
This paper introduces DOPA, a demonstration search framework for robust in-context learning with Large Language Models (LLMs). DOPA uses an OOD proxy to approximate inaccessible target domains and a Mahalanobis distance-based global diversity constraint for demonstration retrieval.
This paper introduces a novel in-context learning approach for low-resource Coptic to English machine translation, augmenting inputs with syntactic information from Universal Dependencies parses. Combining this syntactic data with dictionary-based glosses achieves significant gains and sets a new state-of-the-art.
This work introduces an innovative framework for adaptive test-time compute allocation, jointly adjusting where computation is spent and how generation is performed. The method uses a warm-up phase to identify easy queries and then concentrates further computation on unresolved queries, reshaping generation distributions with evolving in-context demonstrations.
LiFT is a new instruction fine-tuning framework designed to improve in-context learning for large language models on longitudinal NLP tasks, which require reasoning over temporally ordered text. It uses a curriculum that progressively increases temporal difficulty, incorporating few-shot structure and temporal conditioning, consistently outperforming base models across various datasets and parameter sizes.
Este estudo empírico investiga o aprendizado em contexto (ICL) de muitos exemplos para tradução automática de inglês para dez idiomas de baixo recurso. Os achados mostram que o ICL se torna mais eficaz com o aumento do número de exemplos, e a recuperação baseada em BM25 melhora substancialmente a eficiência dos dados.