Quiz Entry - updated: 2026.07.14
Why is "Random Access Memory" a misleading abstraction?
Because access time is not constant — it varies ~50× depending on whether the data sits in a register, a cache level, or main RAM.
* The memory hierarchy trades speed for capacity, so access latency spans from zero-cycle registers to ~200-cycle RAM and far slower SSD. *
The abstraction says: "Access any memory location in constant time" Reality: Access time depends on whether data is cached!
Memory hierarchy (typical modern system):
| Level | Size | Latency | Bandwidth |
|---|---|---|---|
| Registers | ~1 KB | 0 cycles | — |
| L1 cache | 32-64 KB | ~4 cycles | ~1 TB/s |
| L2 cache | 256-512 KB | ~12 cycles | ~500 GB/s |
| L3 cache | 8-64 MB | ~40 cycles | ~200 GB/s |
| Main RAM | 8-64 GB | ~200 cycles | ~50 GB/s |
| SSD | 256GB-2TB | ~100,000 cycles | ~5 GB/s |
Why caches exist:
- CPU runs at ~4 GHz, RAM runs at ~3.2 GHz but with ~60ns latency
- Without caches, CPU would wait 200+ cycles for EVERY memory access
- Caches exploit locality - recently used data is likely to be used again
The "Memory Mountain" shows throughput vs. access pattern:
- Peak: ~7000 MB/s (small data, stride 1, in L1)
- Valley: ~1000 MB/s (large data, large stride, in main memory)
Why this matters for programmers:
- Loop order can make code 86x faster (cache-friendly access)
- Data structure choice affects performance enormously
- "Premature optimization" often means ignoring cache effects!
Go deeper:
A Gallery of Memory Mountains (CS:APP) — Bryant & O'Hallaron's throughput-vs-size-and-stride plots showing RAM access is anything but uniform.
Memory hierarchy (Wikipedia) — the register→cache→RAM→disk latency ladder behind the non-uniform-latency reality.- Random-access memory (Wikipedia) — where the idealised equal-access-time abstraction comes from, and its limits.