essay · 2026-09-09
Known limits of transformer language models
Research into transformer limitations reveals that models often fail not due to lack of data, but because intrinsic computational and statistical bounds make certain failure modes unavoidable. Specifically, studies indicate that autoregressive token dependence can confuse logical causality with mere sequence order, significantly increasing the risk of errors in reasoning tasks [47]. Furthermore, evidence suggests that likelihood-optimised representations actively hinder a model's ability to generalize symmetrically across different contexts [45]. In visual-language systems, hallucinated objects tend to appear disproportionately late during generation, indicating a structural weakness rather than random noise [45]. These findings imply that confidence-aware mechanisms are necessary because models generate statistically probable text without built-in truth verification capabilities. The core issue is that finite description length and sample complexity enforce an irreducible generalization error regardless of model scale [Under review as submission to TMLR On the Fundamental Limits of LLMs at Scale].
Sources
- Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers
- Fundamental Limits of LLMs - emergentmind.com
- The Limits of Today's AI Models
- Understanding Hallucinations in Large Visual and Language Models | ACM Computing Surveys
- Under review as submission to TMLR On the Fundamental Limits of LLMs at Scale