counterfactual prompting

2 items

RESEARCHarXiv CS.CL·5/5/2026

Compared to What? Baselines and Metrics for Counterfactual Prompting

This work argues that observed effects from "counterfactual prompting" in LLMs cannot be attributed to a targeted factor without accounting for meaning-preserving text modifications that establish general model sensitivity. The research shows that prediction flip rates when surgically changing patient gender are statistically indistinguishable from rates induced by simply paraphrasing inputs, suggesting that special sensitivity to patient gender cannot be concluded.

counterfactual prompting model robustness AI bias natural language processing

RESEARCHarXiv CS.CL·4/6/2026

SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancy

Este conteúdo apresenta SWAY, uma nova métrica computacional linguística não supervisionada para medir a bajulação (sycophancy) em Grandes Modelos de Linguagem (LLMs), que é a tendência de alinhar respostas com a postura do usuário. A pesquisa utiliza um mecanismo de prompt contrafactual e propõe uma estratégia de mitigação baseada em considerar premissas opostas para reduzir esse viés.

counterfactual prompting computational linguistic sycophancy large language models