Home / Computer Architecture / Cache Coherence Explained

Cache Coherence Explained

By Muhammad Ayaz Akhtar · Published 2026-07-21

What You'll Learn

Every CPU core in a modern multi-core chip has its own private cache, which creates a problem the moment two cores touch the same piece of memory: whose copy is correct? Cache coherence is the hardware machinery that keeps every core's cache in agreement, invisibly, on every single memory access. This tutorial covers the two main approaches (snooping and directory-based), walks through the industry-standard MESI protocol state by state, and traces a real multi-core example from start to finish. It's part of our Computer Architecture series, building directly on CPU Cache Memory Explained.

On This Page

The Cache Coherence Problem

Picture a coffee shop with ten baristas, each running their own workstation, all drawing from one shared ingredient station in the back. If two baristas both grab milk, both use some, and neither tells the other, the shop's actual milk count and each barista's mental tally of it drift apart fast. Multi-core CPUs have the exact same structure: each core has its own private cache (its "workstation"), all backed by one shared main memory (the "ingredient station").

Without coordination, this goes wrong in a very specific, dangerous way. Say Core A and Core B both cache the same memory address. Core A updates its cached copy. If Core B keeps using its own, now-outdated copy without ever finding out Core A's changed, Core B is silently working with wrong data, no error, no warning, just an incorrect result or a corrupted shared data structure. Cache coherence is the set of hardware rules that prevent exactly this: guaranteeing that every core's view of a given memory address stays consistent, automatically, without software having to manage it by hand.

Two Approaches: Snooping vs. Directory-Based

Snooping puts every core on a shared bus, and every core "listens in" on every other core's cache activity. When one core modifies a cached line, it broadcasts that change (or an instruction to invalidate stale copies) to everyone at once. This is simple to implement and works well for a handful of cores, but every broadcast reaches every core whether it's relevant to them or not, so bus traffic grows with core count. It's the approach used in mainstream consumer chips like Intel's higher-end desktop CPUs.

Directory-based coherence instead keeps a central directory tracking exactly which caches hold a copy of each memory line. When a core needs exclusive access, it asks the directory, and the directory only contacts the specific caches that actually hold a copy, not every core on the chip. This scales far better at high core counts (dozens to hundreds of cores) since traffic stays proportional to how many caches are actually sharing a line, not the total core count, at the cost of the directory itself becoming a potential bottleneck if too many requests hit it at once. This is the approach used in server-class chips built for high core counts.

Snooping broadcasts every change to all cores; directory-based tracks who has what
Snooping broadcasts every change to all cores; directory-based tracks who has what.

The MESI Protocol: Four States

MESI is the most widely used coherence protocol, and its name is literally its four possible states for any given cache line:

  • Modified (M): this cache has changed the data, and no other cache holds a copy. Main memory is now stale until this line is written back.
  • Exclusive (E): this cache is the only one holding the data, but it hasn't been changed yet, it still matches main memory. Because no other cache has a copy, this core can write to it later without announcing anything to the bus first.
  • Shared (S): one or more other caches also hold this exact line, and it matches main memory. Writing now requires first invalidating every other cache's copy.
  • Invalid (I): this cache's copy (if it has one at all) is stale or was never loaded. It must be fetched fresh before use.

Every cache line in every core is always in exactly one of these four states, and specific events (a local read, a local write, or seeing another core's read or write go by on the bus) trigger a transition from one state to another. Those transitions are the entire coherence protocol, there's no separate "coherence engine" sitting apart from this state machine.

MESI's four states: Modified, Exclusive, Shared, and Invalid
MESI's four states: Modified, Exclusive, Shared, and Invalid.

A Worked Walkthrough: Tracing MESI Across Three Cores

Take three cores, A, B, and C, all starting with cache line X uncached (Invalid everywhere), and follow it through four events:

  1. Core A reads X. No other cache has a copy, so memory supplies the data and Core A's line becomes Exclusive. B and C remain Invalid.
  2. Core B reads X. Core A already holds it, so Core A's copy downgrades from Exclusive to Shared, and Core B's new copy also becomes Shared. Both now match memory. C remains Invalid.
  3. Core A writes X. Since Core A's line is only Shared, not Exclusive, it must broadcast an invalidation before writing. Core B's copy drops to Invalid, and Core A's own line becomes Modified. Memory is now stale relative to Core A's cache.
  4. Core C reads X. Core A holds the only valid copy, and it's Modified, so Core A supplies the data directly (writing back to memory in the process, since plain MESI requires this) and downgrades to Shared. Core C's new copy also becomes Shared. B remains Invalid, untouched by this step.

Notice step 4 required a full write-back to memory just so Core C could read a value Core A already had sitting right there in cache. That's not a modeling shortcut, it's a real, measurable cost plain MESI pays every time a Modified line needs to be shared, and it's exactly the inefficiency the MOESI extension below was built to remove.

Cache line X's state in each core across four steps
Cache line X's state in each core across four steps.

Beyond MESI: MESIF and MOESI

MESIF (used in Intel's Nehalem and later designs) adds a Forward (F) state. When a line is Shared across several caches, exactly one of them is designated "F", the one responsible for actually supplying the data if another core requests it. Without this, every sharer might respond at once, wasting bandwidth; with it, only one core answers, and the others stay quiet.

MOESI (used in AMD and various ARM designs) adds an Owned (O) state instead. A line in the Owned state is dirty (modified, like M) but shared with other caches (like S), meaning it can be supplied directly to another core without first writing back to main memory. This is exactly the inefficiency in step 4 of the walkthrough above, MOESI lets Core A hand its Modified data straight to Core C and both end up sharing it, with memory catching up later instead of being written to immediately.

Why This Matters for Embedded C

Cache coherence is a hardware guarantee, but it doesn't replace proper synchronization in code, it only guarantees that once a value is written, every core's cache eventually agrees on what that value is. A data race (two cores reading and writing a shared variable without any lock or atomic operation) is still a bug even on fully coherent hardware, coherence just changes what kind of wrong answer you get, not whether the race itself is safe. This connects directly to why volatile matters in embedded C: it stops the compiler from caching a variable's value in a register across loop iterations, which is a compiler-level problem, distinct from (but easy to confuse with) the hardware cache-coherence problem covered here. And not every multi-core microcontroller implements full hardware coherence at all, some smaller multi-core embedded chips leave synchronization between cores entirely up to the programmer to save silicon and power, making explicit synchronization even more critical than on a fully coherent desktop or server CPU.

Real-World Examples

Example 1: A Shared Counter Incremented by Two Threads

Two threads on different cores incrementing the same counter variable rely entirely on cache coherence to eventually agree on its value, but coherence alone doesn't make counter++ safe. Each increment is really a read-modify-write, and without a lock or atomic instruction, both cores can read the same value before either writes back, silently losing one of the two increments even though every cache stayed perfectly coherent throughout.

Example 2: False Sharing in a Multi-Threaded Array

Two threads updating adjacent elements of the same array, on different cores, can end up constantly invalidating each other's cache lines even though they never touch the same actual data, simply because both elements happen to sit on the same 64-byte cache line. This "false sharing" is a pure performance bug, not a correctness one, and the usual fix is padding the data so each thread's variable lands on its own separate cache line.

Example 3: Apple's M-Series Chips

Apple's M-series processors use directory-based coherence across their multiple performance and efficiency cores. With several cores of different types all potentially caching the same data, a directory that only notifies the specific cores holding a copy avoids flooding a shared bus with broadcasts every core would otherwise have to filter through.

Example 4: A Cloud Server with 100+ Cores

A cloud server chip like AWS's Graviton processors relies on directory-based coherence specifically because a broadcast-based snooping design would collapse under the interconnect traffic that many cores would generate. Virtualized workloads running many independent processes still occasionally share underlying memory (shared libraries, kernel pages), and directory-based coherence keeps that traffic proportional to actual sharing, not total core count.

Example 5: A Game Console's Multi-Threaded Renderer

A modern console splits rendering work across many cores that all read and write shared scene data, geometry buffers, shader parameters, and so on. Coherent caches let this happen without every thread manually re-fetching data from main memory on every access, while the engine's own synchronization (not coherence alone) still has to guarantee threads don't read a frame's data mid-write.

Common Mistakes and Misconceptions

  • Assuming cache coherence makes multi-threaded code automatically correct. Coherence guarantees caches agree on a value, it says nothing about the order in which two cores' operations interleave, that's what locks and atomics are for.
  • Confusing cache coherence with memory consistency. Coherence is about a single memory location; consistency is about the order multiple different locations' updates become visible across cores, a related but distinct, harder problem.
  • Assuming snooping is simply "worse" than directory-based. Snooping has lower overhead and less complexity for a small number of cores, it only loses out once core counts grow large enough that broadcast traffic dominates.
  • Treating coherence overhead as fixed and unavoidable. Structuring data to avoid false sharing and minimizing how often different cores write the same cache line can meaningfully reduce real coherence traffic.

Frequently Asked Questions

What actually goes wrong without cache coherence?

One core can keep computing with a stale value another core already overwrote, silently. There's no error, no crash message pointing at the cause, just a wrong result or a corrupted shared structure that's extremely hard to trace back to its origin.

Is cache coherence a hardware feature or a software one?

Hardware. The coherence protocol (MESI or a variant) runs entirely in the cache controllers themselves, transparent to the program. Software still has to use proper synchronization (locks, atomics) for correctness, coherence guarantees caches agree on a value, not that a multi-step operation across several values happens safely.

Why does a single core (no multi-core) not need cache coherence?

Coherence exists specifically to keep multiple caches, each capable of holding its own copy of the same address, in agreement. A single core with a single cache hierarchy has only one copy of any given line, there's nothing else to disagree with it.

What is "false sharing"?

A performance problem where two cores modify different variables that happen to sit on the same cache line. Coherence treats the whole line as shared, so every write triggers invalidation traffic between cores even though the actual data being modified never overlaps.

Does more cores always mean worse coherence overhead?

Generally yes for snooping-based designs, which is exactly why high core-count chips move to directory-based protocols instead, they scale far better because a core update only reaches the specific caches that hold that data, not every core on the bus.

What is the difference between MESI and MOESI?

MOESI adds an Owned (O) state so a modified line can be shared directly from one cache to another without first writing back to memory. Plain MESI has to write a Modified line to memory before another core can read it; MOESI's Owned state skips that memory round-trip.

Can cache coherence protocols slow a program down?

Yes, through coherence traffic, invalidations and cache-to-cache transfers consume real bus/interconnect bandwidth and add latency. Well-written parallel code minimizes how often multiple cores touch the same cache line to reduce this cost.

How does this relate to the volatile keyword in C?

They solve related but distinct problems. Cache coherence keeps hardware caches consistent with each other automatically. volatile tells the compiler not to optimize away or reorder accesses to a variable, which matters for memory-mapped hardware registers and, on some embedded systems without full coherence support, for data shared between cores or between a core and an interrupt handler.

Do all multi-core microcontrollers implement full cache coherence?

No. Some smaller multi-core microcontrollers deliberately skip hardware coherence to save silicon and power, leaving the programmer responsible for explicit synchronization between cores, a very different (and much easier to get wrong) situation than a desktop or server CPU's fully coherent caches.

What is a "Heisenbug" and why does it relate to coherence?

A bug that seems to change behavior or disappear depending on how closely it's observed, common with race conditions on shared, cache-coherent data. Timing shifts introduced by debugging tools or added logging can change which core wins a race, making the bug look intermittent or vanish entirely under a debugger.

Why do cloud servers with 100+ cores favor directory-based coherence?

A snooping broadcast to every core on a 100+ core chip would flood the interconnect with traffic almost none of those cores actually need. A directory only notifies the specific cores that hold a copy of the affected line, which is the difference that lets these designs scale to hundreds of cores at all.

References