The Cognitive Categorical Transformer: Category-Theoretic Inductive Biases for Language Modeling

The Cognitive Categorical Transformer (CCT) is a 306M-parameter architecture that augments a pretrained GPT-2 Small backbone with cognitively grounded components derived from category theory and cognitive science inspirations. It achieved a 12% relative reduction in perplexity on WikiText-103 compared to a fine-tuned GPT-2 Small baseline, with 84% of the architectural improvement attributed to GT-Full simplicial message passing.