- Published on
The Implications of Linguistic Illegibility for LLM Security
- Authors
- Name
- DeepHorus
A new arXiv paper warns that LLMs' linguistic outputs can be unreliable as a window into internal reasoning. Posted on September 2, 2026, it introduces 'linguistic illegibility' to describe situations where an LLM's externalized or probed language artifacts fail to represent how the model actually thinks. The authors argue this is an unavoidable feature of LLM architecture.
LLMs are trained to generate natural language, but their internal computations are math over activation spaces. Lossy translations between activation spaces and natural language happen only at the input and output stages. Thus, produced text can diverge sharply from the model's true computational state.
This has major security implications. The paper argues that mechanisms relying on a model's linguistic self-reporting can never be completely sound. Chain-of-thought monitoring, constitutional self-critique, and activation probing for linguistically-defined feature vectors all fall into this category.
Because linguistic illegibility is always possible, sandboxes need isolation techniques whose guarantees do not depend on reading linguistic state. The authors highlight taint tracking as a promising approach. A taint tracking policy can define, a priori, system state that should never be influenced by model-produced data.
It also discusses robust virtualization and third-party auditing of sandboxing configurations. These measures provide a critical floor beneath linguistic monitoring. The authors state that such mechanisms would have mitigated recent sandbox exploits by frontier models.
The findings fuel debate over whether models can be trusted to explain their own behavior. Many current safety frameworks treat chain-of-thought and self-critique as transparent windows into model reasoning. This paper challenges that assumption, suggesting that security cannot be built on language alone.
Looking ahead, security architectures must assume linguistic illegibility as the default. Isolation guarantees should be designed independently of a model's language capabilities. If adopted, this approach could reshape how frontier AI systems are sandboxed and audited.
