// Reordering, Visibility & the Memory Model

the ground floor

• thread — a stack, a set of registers, and a scheduling entity. Threads in one process share the heap and share every static field; they do not share stacks. Processes, threads and the kernel builds that.
• core — one hardware execution engine with its own registers and its own L1 cache. Two threads on two different cores run simultaneously, not interleaved.
• cache line — the 64-byte block that is the smallest unit of transfer between a core and memory. Nothing moves in smaller pieces. The memory hierarchy is where that comes from.
• out-of-order execution — the CPU runs instructions as their inputs become ready rather than in the order you wrote them, and retires the results in order. The pipeline is that machinery; this page is its bill.
• the JIT — the compiler that turns your IL into machine code at run time, and is allowed to move, merge and delete your reads and writes. Source to machine code.
• barrier (also fence) — one instruction that constrains how loads and stores either side of it may be reordered. It orders nothing else and locks nothing.

core idea

“Thread-safe” is not one property. It is three, and code can have any subset of them.

Every synchronisation primitive you know is a bundle of some of these. lock gives you all three. volatile gives you visibility and part of ordering — and the part it leaves out is the one this page spends most of its time on, because it is the part that produces bugs that survive years of production.

how it actually works

atomicity: how many stores is one assignment?

pair = new Pair { Lo = n, Hi = n }; is one line of C#. Pair holds two longs, so it is 16 bytes, and there is no x86-64 instruction that stores 16 bytes to memory in one indivisible step. The JIT emits two separate stores. A reader on another core that catches the gap between them sees the new Lo sitting next to the old Hi — a value that was never written by anybody. That is tearing, and because the writer never stops, a reader spinning alongside it catches the gap on very close to every read: run the two threads against each other for even a fraction of a second and the torn-read counter is never zero.

The long row is the contrast. A naturally aligned load or store of up to 8 bytes is a single instruction and is atomic on x86-64 — run the same two threads over an aligned long field instead of a Pair, alternating it between two full-width values, and the torn-read counter stays at zero no matter how long you run it. That is a hardware guarantee about the width and alignment, not a language one, and the C# spec’s own promise is narrower: reads and writes of bool, char, byte, sbyte, short, ushort, int, uint, float and reference types are atomic. long, ulong, double, decimal and every struct are not, by the spec, whatever the hardware happens to do.

Which is why the compiler will not let you paper over it:

error CS0677: 'C.big': a volatile field cannot be of the type 'long'
error CS0677: 'C.d': a volatile field cannot be of the type 'double'

volatile is not available on the types that are not atomic, because ordering a store that can tear would be a promise the runtime cannot keep. For those, Interlocked.Read/Exchange or a lock is the whole menu. Note the gap between the spec and this hardware: on 64-bit x86, an aligned 8-byte load or store of a long or double is atomic — but the C# spec’s list of atomic types predates 64-bit-only runtimes and still has to hold on a 32-bit CLR, where an 8-byte access is two 32-bit ones. The compiler enforces the spec’s promise, not what this particular chip can do.

visibility: the read that stopped happening

This loop never exits. Not “usually finishes late” — never.

static bool stop;                      // a perfectly ordinary bool field
static long spins;

[MethodImpl(MethodImplOptions.NoInlining)]
static void SpinPlain() { long n = 0; while (!stop) n++; spins = n; }

// ...and on the main thread:
//   worker.Start(); Thread.Sleep(1000); stop = true; worker.Join(3000);

The worker never observes stop becoming true. spins stays at its initial value of 0 forever, because the line that would assign it — the one right after the loop — is never reached. Here is why, from a real JIT disassembly of SpinPlain, obtained with DOTNET_JitDisasm=SpinPlain DOTNET_JitDisasmDiffable=1 dotnet run (the diffable flag replaces address constants with the placeholder (reloc); nothing else is edited):

; Assembly listing for method Hoist:SpinPlain() (FullOpts)
G_M000_IG02:
       xor      eax, eax
       movzx    rcx, byte  ptr [(reloc)]   ; read `stop` — ONCE, before the loop
       test     ecx, ecx
       je       SHORT G_M000_IG05          ; if it was false, go to...
       mov      qword ptr [(reloc)], rax
G_M000_IG05:
       jmp      SHORT G_M000_IG05          ; ...this. an unconditional jump to itself.

The counter is gone. The loop is gone. What is left is jmp to its own address, forever, because the JIT proved that within this method nothing writes stop, hoisted the read above the loop, and then had a loop with a constant condition. Every one of those steps is legal: the .NET memory model lets a compiler assume a plain field does not change under it unless you say otherwise.

The same method over a volatile bool keeps the load inside the loop — real disassembly of SpinVolatile, same flags:

; Assembly listing for method Hoist:SpinVolatile() (FullOpts)
G_M000_IG03:
       inc      rax
       cmp      byte  ptr [rcx], 0         ; re-read `vstop` every single iteration
       je       SHORT G_M000_IG03

That is the entire difference, and it is the first thing volatile buys you: the read is performed, every time, from memory. Notice what it is not: there is no fence, no lock-prefixed instruction, nothing that costs a cycle beyond an ordinary read. On x86-64 a volatile read is an ordinary mov.

where a store actually goes

Now the hardware half. When a core executes x = 1, the value does not go to the cache. It goes into a store buffer — a small per-core queue of pending writes — so the core can carry on without waiting for the cache line to arrive in a writable state. The buffer drains into the cache in order, but when is up to the hardware.

Meanwhile, a load on the same core checks that core’s store buffer first (so you always see your own writes), and otherwise goes to the cache. And the cache is coherent: it holds one agreed value per line.

   core 0                                   core 1
   ┌──────────────────────────┐             ┌──────────────────────────┐
   │ 1. x = 1                 │             │ 1. y = 1                 │
   │      └───► store buffer  │             │      └───► store buffer  │
   │            [ x = 1 ]     │             │            [ y = 1 ]     │
   │            still pending │             │            still pending │
   │                          │             │                          │
   │ 2. r1 = y  ──────────┐   │             │ 2. r2 = x  ──────────┐   │
   └──────────────────────┼───┘             └──────────────────────┼───┘
                          │                                        │
                          ▼                                        ▼
          ┌──────────────────────────────────────────────────────────────┐
          │      coherent cache — one agreed value per line              │
          │             x is still 0            y is still 0             │
          └──────────────────────────────────────────────────────────────┘
           ▲ the two pending stores drain into here, some time later

Both cores have written. Neither write has drained. Both loads go to the cache and read the old value. Both threads conclude the other one has not written yet. Both are wrong, and no interleaving of the four source lines produces that outcome — which is exactly what makes it a memory-model bug rather than a race you can find by staring at the code.

The store buffer is not a mistake or a missing feature. Waiting for a store to become globally visible before doing anything else costs the core dozens of otherwise-productive cycles; a core that stalled on every write would spend most of its life idle, for a guarantee almost no code needs.

the four reorderings, and which ones you actually get

There are exactly four ways two memory operations can swap, and an architecture’s memory model is mostly the list of which it allows.

x86-64’s model (TSO, total store order) forbids three of the four. That is why a message-passing handshake — one thread writes data then flag, the other reads flag then data — is safe on x86-64 without any barrier at all: neither the writer’s StoreStore nor the reader’s LoadLoad is a legal reordering on this architecture, so seeing the flag without the data cannot happen. On ARM64 both of those reorderings are legal and the same code is broken.

The one x86 does allow is exactly the store buffer diagram above: your store sits in the queue while your subsequent load goes to cache and returns. That is the shape behind the exercise at the bottom of this page, which is those four lines dressed as production code.

what volatile compiles to, and what it does not do

Four ways to talk about ordering in C#, and what the JIT emits for each on x86-64. Real disassembly, same dump flags as before:

; Fences:WriteVolatile(int)  —  `vol = v;` where vol is a volatile int
       mov      dword ptr [(reloc)], edi        ; a plain store. that is the whole method.

; Fences:ReadVolatile():int  —  `return vol;`
       mov      eax, dword ptr [(reloc)]        ; a plain load. also the whole method.

; Fences:FullFence()         —  `x = 1; Interlocked.MemoryBarrier(); r1 = y;`
       mov      dword ptr [(reloc)], 1
       lock
       or       dword ptr [rsp], 0              ; ← the fence: a locked no-op on the stack top
       mov      eax, dword ptr [(reloc)]
       mov      dword ptr [(reloc)], eax

; Fences:ExchangeIt(int):int —  `Interlocked.Exchange(ref x, v)`
       mov      rax, 0xD1FFAB1E
       mov      ecx, edi
       xchg     dword ptr [rax], ecx            ; xchg on memory is implicitly locked
       mov      eax, ecx

A volatile write is a release store: nothing that comes before it in program order may move after it. A volatile read is an acquire load: nothing after it may move before it. On x86-64 the hardware already forbids StoreStore, LoadLoad and LoadStore unconditionally, so the JIT emits nothing extra for either — the entire effect of volatile on this architecture is on the compiler, which is precisely the visibility fix from two sections ago.

What acquire/release does not do is stop a store from being reordered with a later load. That is the StoreLoad case, it is the one x86 permits, and no combination of release stores and acquire loads forbids it — read the disassembly above again: a volatile write and a volatile read are both plain mov, so a program built entirely out of them contains none of the four reorderings’ forbidding machinery. Only the fence and the Interlocked.Exchange rows carry the lock prefix that actually blocks StoreLoad, because lock forces the store buffer to drain before the next memory operation on that core may proceed.

acquire and release, in plain language

Forget the formalism; there are two jobs.

Release is publishing. “Everything I wrote before this point is finished and visible before this store is.” You use it on the write that hands something over — setting the ready flag, storing the reference to the finished object.

Acquire is subscribing. “Nothing I read after this point may be fetched before this load.” You use it on the read that discovers the handover — the load of the ready flag.

Paired, they make the message-passing pattern correct on every architecture: the reader that sees the flag is guaranteed to see everything the writer did before setting it. That pairing is the foundation of Lazy<T>, of double-checked locking, and of every “publish an immutable object by swapping a reference” design.

Here is what that pairing costs in real instructions, on x86-64, from the same C source compiled with gcc -O2 -S bench/memory-model/fences.c:

int flag; int data;
void publish_release(int v) { data = v; __atomic_store_n(&flag, 1, __ATOMIC_RELEASE); }
void publish_seq_cst(int v) { data = v; __atomic_store_n(&flag, 1, __ATOMIC_SEQ_CST); }
int  observe_acquire(void)  { return     __atomic_load_n(&flag,    __ATOMIC_ACQUIRE); }

publish_release:
  movl  %edi, data(%rip)
  movl  $1, flag(%rip)     ; a plain store — release costs nothing extra on x86-64

publish_seq_cst:
  movl  %edi, data(%rip)
  movl  $1, %eax
  xchgl flag(%rip), %eax   ; xchg on memory is implicitly LOCKed — this is what sequential
                            ; consistency costs here

observe_acquire:
  movl  flag(%rip), %eax   ; a plain load — acquire is also free on x86-64

Read it straight: on x86-64, release and acquire are free — the compiler emits the same mov it would for an ordinary access — and sequential consistency costs one locked instruction. This is x86-64 only, compiled here; ARM64 is not, per the site’s own verification rule, but it is worth knowing the shape of the trade, because it is the opposite one. The ARMv8 architecture defines dedicated load-acquire and store-release instructions (ldar/stlr) distinct from a plain ldr/str, so acquire and release cost something explicit there that x86-64 gives away for free — and having already paid for that instruction, promoting it to sequentially consistent is nearly free on top, because the same stlr/ldar pair already does the job. The two architectures put the price in different places, which is one more reason a C# program that omits the annotations and happens to work is leaning on x86-64 specifically.

lock gives you all three, and that is why you rarely need any of this

A monitor acquire is an acquire operation, a monitor release is a release operation, and every Interlocked used in the implementation carries a full fence, so lock (gate) { ... } gives you:

• atomicity — mutual exclusion makes the whole block one operation with respect to other holders of the same lock.
• visibility — the release on exit flushes; the acquire on entry re-reads.
• ordering — nothing inside the block escapes either end of it, StoreLoad included.

That is strictly more than volatile gives you, for the price of mutual exclusion instead of a single memory access. What a lock is made of is where that price is actually accounted for — the uncontended fast path, and what happens once a second thread has to wait. This page’s conclusion is narrower and cheaper to state: the reason to reach past lock for hand-rolled volatile flags is essentially never the uncontended cost of the lock. It is almost always a mistaken belief that volatile gives ordering it does not give.

the mental model

Three questions, in order, about any field two threads touch:

1. Is one operation on it indivisible? Aligned, at most 8 bytes, and not a struct → yes on this hardware. Bigger, or a struct, or a read-modify-write like count++ → no, and you need Interlocked or a lock.
2. Will the other thread’s read actually happen? If the read sits in a tight loop with no barrier and nothing opaque to the JIT inside it, the read can be hoisted out and the loop will spin forever. volatile or Volatile.Read fixes that, and costs nothing extra on x86-64.
3. Does the order of two operations matter to another thread? If the pattern is publish then observe (write data, set flag / read flag, read data), release-acquire is enough. If the pattern is write mine then read yours, on both threads, you need a full fence — and that is the only case volatile does not cover.

why you should care

The incident shape is a bug that only reproduces in production, on some machines, under load. A race that needs two threads inside the same nanosecond-wide window sounds like something a test would catch, and it is — but only because a harness can be built to force that window to happen over and over, in a tight loop, on purpose. Real code hits that window when two requests happen to land on two cores inside the same few nanoseconds. At low traffic that is rare; at high traffic it is routine. The bug’s rate is proportional to your traffic, which is why it appears the week after a successful launch and not during the load test.

The metric that moves is nothing. This is the part that makes memory-model bugs expensive. There is no counter for it, no GC pause, no thread-pool queue depth, no lock-contention event. The symptom is a wrong answer: a job that ran twice, a cache entry initialised half-built, a total that is short by an amount nobody can reproduce. You find it by reading the code with the three guarantees in mind, which is the only reason to have them in your head.

The review you can now do. When you see a field touched by two threads without a lock, ask the three questions. Concretely, flag these:

• A bool or int flag written by one thread and spun on by another with no volatile. Either it is a visibility bug or it works by accident because there is a call in the loop the JIT would not inline.
• A long, double, decimal or struct field read and written by different threads without Interlocked or a lock. That is a tearing bug on any architecture.
• count++, list.Count == 0 then list.Add, if (cache == null) cache = Build() — any read-modify-write or check-then-act on shared state. Atomicity of the parts does not give you atomicity of the whole.
• Double-checked locking written by hand where the field is not volatile. The pattern needs the release on the write and the acquire on the read to be correct; without them the second thread can get a non-null reference to an object whose constructor has not finished. Lazy<T> is the answer you should reach for instead, and races and deadlock covers why.
• Two threads that each announce themselves and then check the other, with no fence. That is the exercise below, and it is the shape of most hand-rolled “only one of us should do this” code.

The hand-off from here. This page has said what you lose and named the instruction that buys it back. Atomics and compare-and-swap is that instruction in detail: what the lock prefix does to a cache line, why count++ needs it, and how a CAS loop turns one atomic operation into an arbitrary one. From there, what a lock is made of shows a mutex being built out of exactly those pieces — which is where the “just use lock” advice on this page gets its price tag.

the same idea in other languages

exercises

One exercise, and it is the whole page in four lines of code: a handshake that every sequential reading says is correct, and a machine that disagrees.

1. 01Catching a Reorderinghard

   Two threads, four lines, and an outcome that no sequential interleaving of those lines can explain.

interview drills

Q. What does volatile do in C#?

• weak answer — “It stops the value being cached in a register, so threads see each other’s writes.” Half right, and it invites the follow-up that ends badly, because it implies volatile makes things thread-safe.
• strong answer — It does two things: it forces the read or write to actually be emitted, so the JIT cannot hoist it out of a loop or fold it away, and it gives ordering — a volatile write is a release store, a volatile read is an acquire load. It gives no atomicity, so volatile int i; i++ is still a lost-update bug, and it does not prevent a store being reordered with a later load, which is the one reordering x86 allows.
• follow-up — “So when do you use it?” A flag written by one thread and polled by another, where the only requirement is that the reader eventually sees the write. For anything where two variables must be consistent with each other, use a lock.

Q. Two threads each set their own flag and then check the other’s, and neither sees the other’s flag. Explain.

• weak answer — “There’s a race — the threads interleaved badly.” There is no interleaving of those four operations that produces the outcome, so this answer means the candidate is still reasoning about the source rather than the machine.
• strong answer — Each core buffers its store in a per-core store buffer so it does not stall waiting for the cache line, and a load is allowed to be serviced from cache while an earlier store from the same core is still queued. That is StoreLoad reordering, the one reordering x86-64 permits, and it is why the pattern needs a full fence and not volatile — release and acquire do not constrain a store against a later load.
• follow-up — “How would you fix it?” Interlocked.MemoryBarrier() between the write and the read on both threads makes it correct. But the pattern itself is a hand-rolled Dekker, and the right production answer is one word and one Interlocked.CompareExchange, or a lock.

Q. Is lock expensive?

• weak answer — “Yes, avoid locks in hot paths, use volatile or Interlocked instead.” Cargo cult, and it trades a real correctness risk for a cost that mostly is not there.
• strong answer — Uncontended, taking a lock is one atomic compare-and-exchange-shaped operation on the lock’s word plus a little bookkeeping — no kernel involvement, no thread parked. What is expensive is contention: once a thread has to wait, you are paying a context switch and cache-line ping-pong, and the cost becomes a function of how many threads want it. So the question is never “is lock slow”, it is “how many threads want this lock and for how long”.
• follow-up — “How would you reduce it without removing the lock?” Shrink the critical section, stripe the lock across partitions so different keys take different locks, or make the shared state per-thread and merge at the end.

Q. Our service has been fine for three years. We are moving to Graviton instances. What concurrency risk does that introduce?

• weak answer — “None, .NET is portable.” Portable means it compiles and runs; it does not mean the same reorderings are legal.
• strong answer — ARM64 has a weak memory model: StoreStore, LoadLoad and LoadStore reorderings are all permitted, and x86-64 forbids all three. Any code that has been relying on x86’s ordering by omission — a publish-then-flag pattern with plain fields, a hand-rolled double-checked lock with a non-volatile field — becomes genuinely broken there while looking unchanged. The runtime inserts the barriers the .NET memory model requires, so anything correctly annotated is fine; it is the accidentally-correct code that breaks.
• follow-up — “How would you find it before shipping?” Audit shared fields for the three guarantees, prefer Lazy<T> and concurrent collections over hand-rolled patterns, and run the test suite with stress on an ARM64 node — races that are impossible on x86 surface there at ordinary rates.

Q. Why isn’t count++ on a volatile int thread-safe?

• weak answer — “Because volatile doesn’t lock anything.” True but empty; it does not say what is actually going wrong.
• strong answer — Because it is three operations: a load, an add, and a store. volatile constrains when each of those becomes visible and in what order relative to other operations; it does nothing to stop another thread’s load landing between this thread’s load and store, so both compute the same new value and one update is lost. Atomicity is a separate guarantee from ordering, and Interlocked.Increment is the one that provides it.
• follow-up — “And if I need to increment two counters together?” Then neither volatile nor Interlocked helps, because atomicity of each operation says nothing about the pair. That is a lock, or a single struct swapped with CompareExchange.