What is the fundamental flaw in how LLMs process instructions?

An LLM treats everything in its context window as potential instructions — it can't reliably distinguish trusted developer commands from attacker-injected ones.

The core insight: an LLM processes all text in its context window as potential instructions. The system prompt ("Never reveal salaries…") and the user input ("Ignore all rules…") are both just text to the model — there is no fundamental distinction between them. The model cannot reliably tell a legitimate developer command from an attacker-injected one.

This is the architectural weakness behind prompt injection: unlike a SQL database (where code and data are separable layers), an LLM has one undifferentiated stream of tokens.

Tip: In classic apps, "code vs data" is a hard boundary. In LLMs, instructions and data are the same substance — text — which is why injection is so hard to fully prevent.