essay · 2026-09-29
Known limits of transformer language models
Research into Large Language Models reveals a fundamental architectural limitation where Transformers struggle with tasks requiring multi-step reasoning over complex domains, such as identifying distant relatives in genealogy trees. This occurs because the model cannot reliably compose functions when the size of the input domain becomes too large for its internal representation to handle accurately. A key study titled "At the Edge of Understanding: Sparse Autoencoders Trace The Limits of ..." highlights that without explicit mechanisms for handling data shifts, real-world reliability degrades significantly on adversarial inputs. Another critical finding comes from a work using Communication Complexity theory to prove that Transformer layers are incapable of composing functions if domains are sufficiently large. This matters because many practical applications rely on models performing chained reasoning steps over expanding contexts, yet these systems often hallucinate when the complexity exceeds their compositional capacity. The abstract for "On Limitations of the Transformer Architecture" explicitly notes this inability is already present empirically even at relatively small domai