What is the Von Neumann bottleneck?

It is the speed mismatch caused by routing all instructions and data through one shared path to memory: the CPU starves waiting for memory it can't reach fast enough.

* Instructions and data share one path to slow memory, so the fast CPU stalls waiting — the bottleneck caches try to hide. *

In a Von Neumann machine, code and data live in the same memory and travel over the same bus. The trouble is a growing gap: a modern CPU can execute billions of operations per second, but main memory delivers data far more slowly, so the processor spends much of its time simply waiting. The single shared channel becomes the limiting factor — hence "bottleneck."

Because this gap dominates performance, almost every speed trick in a modern CPU exists to hide it:

• Cache hierarchy (L1/L2/L3) keeps frequently used data close and fast.
• Prefetching guesses what's needed next and fetches it early.
• Out-of-order execution finds other useful work to do while a memory access is pending.
• Separate instruction/data caches (a Harvard-style split) let code and data fetch in parallel.

Tip: this is why a cache miss is so costly and why optimising memory-access patterns often matters far more than shaving off instructions.

Go deeper:

• Von Neumann architecture (Wikipedia) — why a single shared bus for code and data creates the bottleneck.