How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

This paper measures the consistency and specificity of language model circuits using edge attribution patching across multiple tasks and models. It finds high within-task circuit reuse that is necessary for performance, but also significant overlap across tasks, indicating circuits are not task-specific.