On the Persistent Effects of Lexicality in Large Language Mod

This work investigates the persistent effect of lexical overlap, rather than semantic content, on representations extracted from large language models (LLMs) and its implications. The authors find that lexical influence extends across model depths, architectures, and training regimes, even in models trained for semantic similarity.