the ground floor
- machine code — the bytes an x86-64 core decodes and executes.
add eax, ecxis three bytes. What the CPU does with them is the page underneath this one. - register — one of the sixteen named 64-bit slots inside the core. Everything the CPU computes on has to be in a register first; memory is where values wait.
- IL — Intermediate Language (also written CIL or MSIL): the instruction set the C#
compiler actually emits. It is a stack machine — no registers, no addresses, no CPU in
mind — and it is what lives inside a
.dll. - assembly — in .NET, a
.dllor.exe: IL plus metadata, the tables describing every type, method and field. Note the collision: this page also says “assembly” for assembly language. Where it matters, this page writes “the assembly” for the file and “assembly code” or “machine code” for the instructions. - JIT — the just-in-time compiler inside the runtime. It turns a method’s IL into machine code the first time that method is called, and sometimes again later. .NET’s is called RyuJIT.
- tier — one JIT compilation of a method at a particular quality setting. A hot method gets compiled more than once, at more than one tier, in the same process.
core idea
There are two compilers between your C# and the CPU, and the one that decides your performance
runs inside your process, while it is serving traffic. Roslyn (csc) translates C# to IL and
performs almost no optimisation worth the name — it does not allocate registers, does not
inline, does not remove a bounds check. RyuJIT does all of that, per method, at runtime, and it
does it twice: once fast and badly to get the method running, then again slowly and well
once the method has proved it matters.
| the C# compiler (Roslyn) | the JIT (RyuJIT) | |
|---|---|---|
| runs | at build time, on your machine | at run time, in your process |
| turns | C# into IL and metadata | IL into machine code for this CPU |
| decides | overload resolution, lambda and async rewriting, constant literals |
registers, inlining, bounds-check removal, unrolling, branch layout |
| knows | your source | which CPU features exist, which types actually showed up, how often each method ran |
what -c Release changes |
very little in the IL | everything — a Debug build turns tiering off permanently, see below |
That split is why “compiled languages are fast and JIT languages are slow” is a category error. The JIT is not a handicap the runtime pays; it is a compiler with information a build-time compiler cannot have.
how it actually works
five lines of C#, 24 bytes of IL, 29 bytes of machine code
One method, followed all the way down. Every listing below is real output from this box, from
the files in bench/il-jit-codegen/.
static int SumLoop(int[] a)
{
int s = 0;
for (int i = 0; i < a.Length; i++) s += a[i];
return s;
}Roslyn turns that into IL. This listing is decoded from the actual method-body bytes in the
built assembly — bench/il-jit-codegen/il-dump.cs reads them with
MethodBody.GetILAsByteArray() and decodes them against the runtime’s own opcode table, so
nothing here is transcribed by hand:
.method SumLoop // 24 bytes of IL, maxstack 3, 2 locals
.locals [0] int32
.locals [1] int32
IL_0000: ldc.i4.0 ; push 0
IL_0001: stloc.0 ; s = pop()
IL_0002: ldc.i4.0
IL_0003: stloc.1 ; i = pop()
IL_0004: br.s IL_0010 ; jump to the test — a for loop checks first
IL_0006: ldloc.0 ; push s
IL_0007: ldarg.0 ; push a
IL_0008: ldloc.1 ; push i
IL_0009: ldelem.i4 ; pop array, pop index, push a[i]
IL_000a: add ; pop two, push the sum
IL_000b: stloc.0 ; s = pop()
IL_000c: ldloc.1
IL_000d: ldc.i4.1
IL_000e: add
IL_000f: stloc.1 ; i = i + 1
IL_0010: ldloc.1 ; push i
IL_0011: ldarg.0
IL_0012: ldlen ; push a.Length
IL_0013: conv.i4
IL_0014: blt.s IL_0006 ; if i < len, jump to the body
IL_0016: ldloc.0
IL_0017: ret
Four things about IL are worth carrying around, and they are all visible above.
It is a stack machine. There are no registers. Every instruction pops its operands off an
evaluation stack and pushes its result back. maxstack 3 is the compiler telling the runtime
how deep that stack ever gets. Real CPUs do not work this way at all, which is exactly the
point: IL describes what to compute without committing to where the values live, and the
JIT makes that decision later for a CPU Roslyn never saw.
Locals are numbered slots, not names. s is local 0 and i is local 1. The names survive
only in the debug symbols.
ldelem.i4 is one instruction, and it is defined to bounds-check. The IL specification says
ldelem throws IndexOutOfRangeException when the index is out of range. Roslyn cannot elide
that check because IL has no way to express an unchecked array read. Everything you have heard
about bounds-check elimination in .NET therefore happens below this listing — in the JIT.
Roslyn optimised nothing. It did not hoist a.Length out of the loop, it did not keep s
in anything resembling a register, and in Release configuration this IL is essentially
identical. The IL is a faithful transcription.
the JIT compiles a method the first time it is called
Nothing in that assembly is machine code. The runtime installs a stub in every method’s slot,
and the first call to SumLoop lands in the stub, which invokes the JIT, which compiles the
method, patches the slot to point at the fresh code, and jumps into it. The second call goes
straight there.
build time run time
────────── ────────
Program.cs ──csc──▶ Program.dll first call ──▶ stub ──▶ JIT ──▶ tier-0 code
├─ IL bytes │
└─ metadata next ~30 calls ─────────────────▶ │ counted
▼
hot ──▶ JIT again ──▶ tier-1 code (optimised,
using the counts collected
by the tier-0 code)
That first call is not free, and the cost is not hidden anywhere clever — it is the JIT
compiling the method, on the same thread that made the call, before that call can proceed.
Every frame between your code and SumLoop is waiting on the compiler, not on the loop. A
method called exactly once pays this cost and gets nothing back for it. A method called a
million times pays it exactly once and amortises it over all the rest — which is the whole bet
tiering makes.
tier 0: get it running
Here is what the JIT emitted for SumLoop the first time it was called. Real output, unedited,
from DOTNET_JitDisasm=SumLoop dotnet run bench/il-jit-codegen/tiered-codegen.cs -c Release:
; Assembly listing for method P:SumLoop(int[]):int (Instrumented Tier0)
; Instrumented Tier0 code
; rbp based frame
; compiling with minopt
G_M000_IG01: ;; offset=0x0000
push rbp
sub rsp, 64
lea rbp, [rsp+0x40]
xor eax, eax
mov dword ptr [rbp-0x34], eax
mov dword ptr [rbp-0x38], eax
mov gword ptr [rbp-0x30], rdi
G_M000_IG02: ;; offset=0x0016
mov dword ptr [rbp-0x40], 0x3E8
xor eax, eax
mov dword ptr [rbp-0x34], eax
xor eax, eax
mov dword ptr [rbp-0x38], eax
jmp SHORT G_M000_IG04
G_M000_IG03: ;; offset=0x0029
mov rdi, 0x7EBE0DFEC1F8
call CORINFO_HELP_COUNTPROFILE32
mov rax, gword ptr [rbp-0x30]
mov ecx, dword ptr [rbp-0x38]
cmp ecx, dword ptr [rax+0x08]
jae SHORT G_M000_IG08
mov edx, ecx
lea rax, bword ptr [rax+4*rdx+0x10]
mov eax, dword ptr [rax]
add eax, dword ptr [rbp-0x34]
mov dword ptr [rbp-0x34], eax
mov eax, dword ptr [rbp-0x38]
inc eax
mov dword ptr [rbp-0x38], eax
G_M000_IG04: ;; offset=0x005B
mov eax, dword ptr [rbp-0x40]
dec eax
mov dword ptr [rbp-0x40], eax
cmp dword ptr [rbp-0x40], 0
jg SHORT G_M000_IG06
G_M000_IG05: ;; offset=0x0069
lea rdi, [rbp-0x40]
mov esi, 16
call CORINFO_HELP_PATCHPOINT
G_M000_IG06: ;; offset=0x0077
mov rax, gword ptr [rbp-0x30]
mov eax, dword ptr [rax+0x08]
cmp eax, dword ptr [rbp-0x38]
jg SHORT G_M000_IG03
mov rdi, 0x7EBE0DFEC1FC
call CORINFO_HELP_COUNTPROFILE32
mov eax, dword ptr [rbp-0x34]
G_M000_IG07: ;; offset=0x0095
add rsp, 64
pop rbp
ret
G_M000_IG08: ;; offset=0x009B
call CORINFO_HELP_RNGCHKFAIL
int3
; Total bytes of code 161
161 bytes of machine code for 24 bytes of IL, and four separate things in it are not your program:
| what | why it is there |
|---|---|
s at [rbp-0x34], i at [rbp-0x38] |
tier 0 does not allocate registers; every local gets a frame slot and every use is a load and a store |
two call CORINFO_HELP_COUNTPROFILE32 |
instrumentation. Tier-0 code counts how often each block runs, so the next compilation can use real numbers. This is Dynamic PGO |
mov [rbp-0x40], 0x3E8 and call CORINFO_HELP_PATCHPOINT |
a patchpoint: a countdown from 1000 iterations, checked every trip, so a long-running loop can be rescued mid-flight — see OSR below |
cmp ecx, [rax+0x08] / jae / call CORINFO_HELP_RNGCHKFAIL |
the bounds check. [rax+0x08] is the array’s length field, eight bytes into the object |
Tier-0 code is deliberately unoptimised. Its job is to start executing as soon as possible, because most methods in a real program run a handful of times and compiling them well would be wasted work.
tier 1: get it fast
The same process, the same method, once it has been called enough times:
; Assembly listing for method P:SumLoop(int[]):int (Tier1)
; Tier1 code
; optimized code
; optimized using Dynamic PGO
; with Dynamic PGO: fgCalledCount is 726400
G_M000_IG01: ;; offset=0x0000
push rbp
mov rbp, rsp
G_M000_IG02: ;; offset=0x0004
xor eax, eax ; s = 0
mov ecx, dword ptr [rdi+0x08] ; ecx = a.Length
test ecx, ecx ; empty array?
jle SHORT G_M000_IG05
G_M000_IG03: ;; offset=0x000D
add rdi, 16 ; rdi = &a[0]
G_M000_IG04: ;; offset=0x0011
add eax, dword ptr [rdi] ; s += *p ← the whole loop
add rdi, 4 ; p++
dec ecx ; trip count--
jne SHORT G_M000_IG04
G_M000_IG05: ;; offset=0x001B
pop rbp
ret
; Total bytes of code 29
161 bytes became 29. s is eax and never touches memory. The index i does not exist any
more — the JIT rewrote a[i] into a moving pointer and turned the loop test into a countdown to
zero, because dec/jne sets the flags for free. The instrumentation is gone. The bounds
check is gone, which is the subject of
the bounds-check exercise.
fgCalledCount is 726400 is the JIT telling you what the tier-0 instrumentation counted: this
method had run 726,400 times when it was recompiled. That number is why the second compilation
can be aggressive — it is not guessing that the method is hot.
an honest note about that listing
In this run the runtime printed the tier-0 listing twice, byte for byte identical, before the tier-1 one. It is shown once above. The order of compilations — tier 0 first, tier 1 much later — is what the output shows and what matters here.
what “warmed up” means
“Warmed up” is not a vibe — it is a specific, discrete event: the pointer in the method’s slot gets rewritten from the tier-0 code above to the tier-1 code above, and every call after that one runs different bytes. Two conditions gate it, and the runtime checks both:
- A call count. The method has to be called some number of times — on the order of a few dozen — before the runtime bothers recompiling it. Most methods in a real program run once or twice and would never earn back the cost of a second, more thorough compile.
- A grace period after the process starts. The runtime does not start counting calls at all until the process has been running for a little while. Startup calls hundreds of methods in a burst; counting from the first instant would trigger a flood of recompilations competing with the work of starting up in the first place.
Both are real knobs (DOTNET_TC_CallCountThreshold, DOTNET_TC_CallCountingDelayMs) and both
exist for the same reason: promoting too eagerly wastes compiler time on methods that were
never going to be hot, and promoting too late leaves a method slow for longer than it needs to
be. Two details matter more than the exact numbers. The transition is a step, not a ramp —
one call runs the tier-0 code above, the very next call runs the tier-1 code above, because
promotion is a pointer write, not a gradual improvement. And it is per method, not
per process — a service can be fully warmed up on its hot path and still running tier-0 code on
a controller it hits once a minute.
OSR: rescuing a loop that is already running
The patchpoint in the tier-0 listing exists for the method that is called once and runs for a minute — a batch job’s outer loop. Counting calls will never promote it, because there is only one call.
So tier-0 code counts loop iterations too. When the countdown at [rbp-0x40] hits zero, the
patchpoint helper compiles an optimised version of the method starting at that loop, copies
the live locals from the tier-0 frame into the new frame, and jumps into it — mid-loop, on the
same thread, without returning. That is on-stack replacement (OSR). bench/il-jit-codegen/osr-demo.cs
is one call to a method whose loop runs fifty million trips — call-counting can never promote a
method that is only ever called once, so this is OSR or nothing. Real output, unedited, from
DOTNET_JitDisasm=LongLoop dotnet run bench/il-jit-codegen/osr-demo.cs -c Release:
; Assembly listing for method P:LongLoop(int[]):long (Tier1-OSR)
; Tier1-OSR code
; OSR variant for entry point 0x16
; optimized code
G_M000_IG01: ;; offset=0x0000
mov rax, qword ptr [rbp]
push rax
mov rbp, rsp
mov rax, gword ptr [rbp+0x30]
mov rdx, qword ptr [rbp+0x28]
mov edi, dword ptr [rbp+0x24]
mov ecx, dword ptr [rbp+0x20]
G_M000_IG02: ;; offset=0x0016
mov esi, dword ptr [rax+0x08]
jmp SHORT G_M000_IG04
G_M000_IG03: ;; offset=0x001B
cmp ecx, esi
jae SHORT G_M000_IG09
mov r8d, ecx
movsxd r8, dword ptr [rax+4*r8+0x10]
add rdx, r8
inc ecx
G_M000_IG04: ;; offset=0x002C
cmp esi, ecx
jg SHORT G_M000_IG03
; … the outer 50,000,000-trip counter, then …
G_M000_IG08: ;; offset=0x0045
add rsp, 96
pop rbp
ret
; Total bytes of code 81
The label says everything: (Tier1-OSR), OSR variant for entry point 0x16 — this is not a
fresh call into LongLoop, it is a resumption, and its opening instructions
(mov rax, [rbp+0x30] …) are reloading the live locals — the array reference, the running sum,
both loop counters — out of the tier-0 frame that was already in flight when the switch
happened. The practical consequence: a Main that does all its work in one long loop does
get optimised, but only after the first 1000 iterations of the loop the patchpoint sits in, and
the transition costs a compile happening mid-flight rather than between calls.
inlining, and what the JIT does afterwards
Inlining is replacing a call with the callee’s body. Its direct saving — the call and the return no longer happen at all — is real but small on its own; see the call-cost exercise for what a call actually costs. The reason inlining is the JIT’s most important decision is what becomes possible after it, when the callee’s code is sitting in the caller’s expression graph where the optimiser can see it.
bench/il-jit-codegen/inlining.cs gives eight callees exactly one call site each, prints each
one’s IL size out of its own metadata, and the disassembly of each caller says whether the call
survived. Four of the eight carry an exception-handling region, because the folklore says all
four are uninlinable and only one of them is:
| callee | IL bytes | shape | inlined? | caller’s machine code |
|---|---|---|---|---|
Small |
6 | x * 2 + 1 |
yes | 7 bytes |
WithTry |
15 | same arithmetic, wrapped in try/catch |
no | 14 bytes |
TryFinallyWork |
27 | same arithmetic in a try, with a finally that increments a static array element |
yes | 29 bytes |
UsingReal |
26 | same arithmetic under a using, on a class whose Dispose increments a static array element |
yes | 29 bytes |
LockMethod |
52 | same arithmetic inside a lock on a static object |
yes | 166 bytes |
Guarded |
21 | argument check that throws, then the arithmetic | yes | 74 bytes |
Big60 |
829 | 60 straight-line arithmetic statements | yes | 984 bytes |
Big90 |
1249 | 90 of them | no | 13 bytes |
The folklore is “small methods get inlined”. The evidence says a 15-byte method was refused while an 829-byte one was accepted. Four rules are doing that work:
- A
catchhandler in the callee stops it, at any size.WithTryis six bytes smaller thanGuarded, computes the same expression, and is the only one of the eight refused for its shape rather than its size. This is the part of the folklore that is still true. try/finally,usingandlockdo not stop it — not on this runtime. All three inlined. Thelockis the convincing case:CallLockMethodcame out at 166 bytes containingMonitor:TryEnter_FastPath,Monitor:Enter_Slowpathand a real exception-path funclet that callsMonitor:Exit_Slowpath, so the callee’s EH region was carried into the caller rather than being folded away before the inliner looked. (InTryFinallyWorkandUsingRealit was folded away, because a protected region containing nothing butx * 2 + 1cannot throw — which is why thelockresult is the one that settles the question.) The switch that governs this is real and observable: re-running the same file underDOTNET_JitInlineMethodsWithEH=0refuses all four EH callees, and every one of those callers collapses to the 14 bytesCallWithTryalready had. Whatever was true of earlier runtimes, check it on the one you ship rather than carrying the rule forward.- A
throwis not an exception handler.Guardedvalidates its argument and throws; it inlines fine, and the JIT moved the throwing path to a cold block at the end of the emitted code so the hot path stays contiguous. Argument validation does not cost you inlining. - Size is a budget, not a cliff at 32 or 100 bytes, and it is much larger than people quote. Somewhere between 829 and 1249 bytes of IL, for this shape at this call site, the JIT decided against it.
And here is the payoff. CallSmall is Small(x) + Small(x + 1), i.e. (2x+1) + (2x+3):
; Assembly listing for method Inl:CallSmall(int):int (Tier1)
; 0 inlinees with PGO data; 2 single block inlinees; 0 inlinees without PGO data
G_M000_IG02:
add edi, edi ; edi = 2x
lea eax, [rdi+rdi+0x04] ; eax = 2x + 2x + 4 = 4x + 4
ret
; Total bytes of code 7
Two calls, two multiplications, two additions and one more addition in the source. Two instructions in the result, and neither of them is a multiply. That is the real argument for inlining: it is the enabling transformation for constant folding, common-subexpression elimination and everything else, because an optimiser can only reason about code it can see.
the four reasons your micro-benchmark is lying to you
Every mechanism above is also a reason a naive “time version A, time version B” comparison can report the wrong answer, without either implementation being buggy.
-
Tiering. Compare two implementations before either one has been called enough times to promote, and you are comparing two pieces of deliberately unoptimised tier-0 code — not the code either one runs in production. The tier-0 and tier-1 listings above are not the same program run at different speeds; they are two different compiled artifacts, and only one of them is what a warmed-up service actually executes.
-
Inlining context. A callee measured behind an isolated
NoInliningwrapper — which is how you have to measure a call in isolation — compiles differently than the same callee sitting at its real call site, where it might inline and vanish into theCallSmallshape above, or hit a size budget the isolated version never faced. -
Dead-code elimination. If nothing observable consumes a result, and the JIT can see straight through the computation that produced it, the whole thing can be deleted.
bench/il-jit-codegen/dead-code-elimination.cscalls a method that computes a value and never uses it; once the callee is small enough to inline, here is the entire compiled body:; Assembly listing for method P:CallUnusedResult(int) (Tier1) ; optimized using Synthesized PGO ; 0 inlinees with PGO data; 1 single block inlinees; 0 inlinees without PGO data G_M000_IG01: G_M000_IG02: ret ; Total bytes of code 1One byte:
ret. Nothing was computed, because nothing needed to be. A comparison that calls a method purely for its return value and never reads that value can end up timing this — an empty function — on one or both sides. -
Debug versus Release. Covered in full below: a Debug build never enters tiering at all — it compiles straight to MinOpts, on purpose, and stays there for the life of the process.
None of these four is exotic. All four are things RyuJIT does correctly, on every request your service serves — they only become a trap when someone times two code paths without accounting for which compiled artifact was actually running.
Debug is not “Release with symbols”
A Debug build sets DebuggableAttribute(DisableOptimizations) on the assembly, and the JIT
honours it by compiling every method with MinOpts — forever. It does not merely start out
unoptimised and warm up; there is no tier-1 state for it to warm up into, ever. Not even the
tiering machinery runs — the runtime skips straight past it, because it already knows where
this method is going to end up. Compiling bench/il-jit-codegen/tiered-codegen.cs without
-c Release, with no other flags at all:
$ DOTNET_JitDisasm=SumLoop dotnet run bench/il-jit-codegen/tiered-codegen.cs
; Assembly listing for method P:SumLoop(int[]):int (MinOpts)
; Total bytes of code 133
One compilation, straight to MinOpts, with no tier-0 or tier-1 label anywhere in the output. 133 bytes against 29. Debug code is not tier-0 code either — it is what tier 0 would look like if the runtime had also promised never to recompile it. Every implementation compared in a Debug build is being compared on the strength of the debugger’s convenience, not on anything that will ship.
the mental model
Four sentences, and they cover most of what this page is for.
- Roslyn transcribes; the JIT optimises. If you are wondering whether the C# compiler is clever enough to fix something, the answer is almost always no — and it does not matter, because the compiler that is clever runs later.
- Every hot method is compiled at least twice. The first version exists to start running; the second one exists to run well and is built from counts the first one collected.
- “Warmed up” is a real state with a real boundary — a call count plus a start-up grace period, both shown above — and it is a step, a pointer rewrite, not a gradual improvement.
- You can just look.
DOTNET_JitDisasm=MethodName dotnet run -c Releaseprints the exact instructions. Questions like “did it inline”, “did the bounds check go away”, “is this loop unrolled” have answers in seconds, and the answer is often not the one you would have argued for.
why you should care
The first requests after a deploy run different code than the requests an hour later. A
rolling deploy puts a process into a load balancer’s rotation with every method at tier 0 — the
deliberately unoptimised code shown above, no register allocation, extra instrumentation calls
in every hot loop — while the JIT is also compiling in the background, competing for the same
cores as your request handlers. That is the shape behind the classic “p99 spikes for the first
stretch after every deploy, then settles” graph, and it is why warming a process with real
traffic before it joins the rotation is worth the trouble. It is also why a health check that
only pings /healthz proves nothing: the code path it warms is not the code path you serve.
Any comparison that ignores the four reasons above is not comparing two speeds of your code. It is comparing whatever artifacts the JIT happened to produce for reasons that have nothing to do with which implementation is actually better — which tier each side reached, whether the callee inlined at its real call site, whether the result was even consumed, whether it was a Debug build. Fix all four and what is left is a fair comparison; ignore any one of them and a number can point the wrong way with total confidence.
Constant factors are decided here. Big-O tells you which algorithm
wins as n grows; the JIT decides everything about the constant in front. Bounds checks removed
or kept, a call inlined or not, a value in a register or in a frame slot — none of that is
visible in the source, and two implementations of the same complexity class can compile to very
different code for reasons only DOTNET_JitDisasm will show you.
The code review you can now do. Prefer the loop shape the JIT recognises — the idiomatic
for (int i = 0; i < arr.Length; i++), not a hand-rolled variant. Do not scatter
[MethodImpl(MethodImplOptions.AggressiveInlining)] — the JIT’s default answer is usually
right, forced inlining grows code and evicts instruction cache, and the table above shows the
inliner is far more generous than its reputation. Do not wrap a hot little helper in
try/catch unless it needs one; that alone makes it uninlinable at any size.
if you need an actual number
This page never times anything, on purpose — the four reasons above are exactly why a raw
Stopwatch loop is easy to get wrong. If you need a real comparison, use BenchmarkDotNet: it
waits for tiering to settle before it starts the clock, keeps every result alive so it cannot
be deleted, isolates each benchmark in its own process, and reports allocations alongside
time. It can also dump the disassembly of the method it just ran, which is the same technique
this page uses by hand.
the same idea in other languages
| language | what the machinery is called | the trap |
|---|---|---|
| Java | javac emits bytecode; HotSpot interprets it, then compiles with C1, then C2 — the same two-tier shape, plus an interpreter below it |
HotSpot can deoptimise: code compiled on the assumption that only one implementation of an interface exists is thrown away when a second class is loaded, and execution falls back to the interpreter. A comparison that runs long enough to trigger this can change behaviour partway through for that reason alone. |
| C / C++ | one compiler, at build time; -O0 through -O3 pick how hard it tries |
There is no tiering, so a C program’s first iteration runs the same code as its last — but the compiler will still delete a computation whose result you never use, exactly like reason 3 above. That is why Google Benchmark ships benchmark::DoNotOptimize. Same trap, no runtime to blame. |
| Go | compiled ahead of time to machine code; no JIT, no tiers | Inlining and escape-analysis decisions are frozen at build time and you inspect them with go build -gcflags=-m rather than by dumping runtime code. Carrying “it gets faster once it’s been running a while” into Go is meaningless; carrying “it never needs a settling-in period” into .NET produces the tiering trap above. |
| Python (CPython) | source is compiled to bytecode (the .pyc) and executed by an interpreter loop |
There is no machine code for your function to inspect — dis.dis(f) shows bytecode, and the only machine code is the interpreter’s own. The bytecode never changes shape based on how many times a function ran, so there is no tier-0/tier-1 split to trip over. |
| JavaScript (V8) | several tiers, promoted on call and loop counts, with inline caches keyed on object shape | A function is compiled against the shapes it has seen. Pass an object with a different set of properties and V8 deoptimises and recompiles it — the JS analogue of reason 1 above, except the trigger is a shape change instead of a call count. |
exercises
The bounds-check exercise below is the JIT observed rather than argued about: two loops that look almost identical, and the compiled code that says which one the JIT could actually prove safe.
Two index loops over the same array — one the JIT can prove safe, one it cannot.
interview drills
Q. Our p99 spikes for about a minute after every deploy and then settles. Where would you look?
- weak answer — “cold caches” and stop there. It is a real cause and it is rarely the whole story; the follow-up will be “which cache, and why exactly a minute?”.
- strong answer — Name the layers and say how you would separate them. A fresh process runs every method at tier 0 until it has been called enough times and the start-up grace period has expired, so hot paths are running deliberately unoptimised code, and the JIT is burning CPU compiling while the process serves traffic. On top of that sit cold in-memory caches, unopened connection pools and an empty file-system cache. The way to tell them apart is to warm the process with real traffic before it joins the load balancer and see which part of the spike disappears first.
- follow-up — “How would you get rid of the JIT part?” ReadyToRun precompiles to machine code at publish time so the first call has something to run, though it is still less optimised than tier 1 and gets recompiled once hot; Native AOT removes the JIT entirely at the cost of dynamic-code features and of the JIT’s runtime knowledge of the actual CPU and the actual call pattern.
Q. A colleague swaps a hand-rolled loop for library code, calls it an improvement, and has a number from a quick before/after comparison to prove it. What do you check before you accept the number?
- weak answer — “Can you re-run it?” Re-running a broken comparison produces the same misleading number very reliably.
- strong answer — Four questions, all about whether the comparison was even looking at real code: was each side actually running tier-1 code, or did one side get more calls before the clock started than the other; is the callee inlined the same way at its real call site as it was in the isolated comparison; is the result of each path actually consumed by something the optimiser cannot see through, or could the whole computation have collapsed the way the dead-code example above did; and was it a Debug build, which never leaves tier 0 no matter how long you wait. Any one of the four can make an honest comparison meaningless.
- follow-up — “How do you check whether the JIT actually kept the work?” Read the disassembly
with
DOTNET_JitDisasmand look at what is actually there. A method whose entire body collapsed to a singleretdid not run the computation at all — the comparison measured nothing, on that side, at every iteration.
Q. C arrays do not bounds-check. Does that make C# arrays inherently worse for a tight numeric loop?
- weak answer — “Yes, that’s the price of memory safety.” True in principle, and it skips past the mechanism entirely, which is what the interviewer is actually asking about.
- strong answer — Usually there is no check to pay for. The JIT proves the index is in range
for the idiomatic
for (int i = 0; i < a.Length; i++)loop and emits none — the tier-1 code above is four instructions, and not one of them is a comparison against the length. When it cannot prove the index safe, it often clones the loop instead of checking every element: one guard before the loop, then a fast body identical to the proven case. Even in the case where a check does survive per element, it is one predictable, never-taken branch sitting right next to the read, and a predictor that gets a never-taken branch right on every iteration is not costing you a stall. - follow-up — “When does it actually matter?” When the check blocks a bigger optimisation the
JIT would otherwise make on the loop, or in a tight kernel with an indirection in the
innermost loop, where a
Span<T>slice or a rawrefgets the proof back — and the rawrefgets memory corruption back too if the invariant you were relying on turns out to be wrong.
Q. When would you add [MethodImpl(MethodImplOptions.AggressiveInlining)]?
- weak answer — “On hot small methods, to help the JIT.” The JIT already inlines those; the attribute mostly changes behaviour where the JIT said no, which is where it had a reason.
- strong answer — Rarely, and only after checking what actually happened without it. The
inliner is far more generous than folklore suggests — it accepted an 829-byte-IL callee in
the table above — so a refusal usually means something specific: a
catchhandler in the callee (refused here at 15 bytes of IL, whiletry/finally,usingandlockcallees all inlined), a virtual or interface call it could not devirtualise, or a size budget it decided against. Forcing it grows the caller and pressures the instruction cache, so it is worth it mainly for a small wrapper on a genuinely hot path, verified withDOTNET_JitDisasmrather than assumed. - follow-up — “What stops the JIT inlining a method that is only twenty bytes?” A
catchhandler: the 15-bytetry/catchcallee above was refused while an 829-byte callee with no exception handler was accepted.try/finally,usingandlockcallees all inlined on .NET 10 here,lockwith its exception-path funclet intact — whatever folklore says about those three, check the runtime you actually ship on.
Q. Your profiler blames a method that does not appear in the stack traces, or attributes 40% of the time to a line that cannot cost that. What is going on?
- weak answer — “The profiler is wrong.” It is usually reporting exactly what it sampled.
- strong answer — Optimised code does not map one-to-one onto source. Inlining moves a callee’s cost into its callers and can make the callee vanish from stacks entirely; the reverse also happens, where one caller’s frame accumulates the cost of five inlined helpers. Loop rewriting fuses several source statements into single instructions, so the line table becomes approximate. The fix is to read the emitted code for the method in question rather than argue with the line numbers.
- follow-up — “How do you get honest stacks then?” Sample at the instruction level and treat
attribution within a method as approximate, or temporarily mark the suspect method
NoInliningto give it a frame of its own — accepting that you have then changed the thing you are profiling.
cheat sheet — il jit
recognize it
- p99 spikes for about a minute after every deploy and then settles on its own — a fresh process runs every hot method at tier 0 until it has been called a few dozen times *and* the start-up grace period expires
- a tight timing loop prints an implausibly small per-operation cost, or zero — the JIT proved the result was never used and deleted the work you meant to measure
- the profiler blames a method that never appears in the stack traces, or charges an outsized share to a line that cannot cost it — inlining moved the cost into the caller
- two implementations rank differently in Debug and Release, or before and after warmup — the build and the tier change which one wins, not just by how much
- a JIT listing whose header says
Instrumented Tier0orMinOptswhen you thought you were reading the optimised code
key tricks
DOTNET_JitDisasm=MethodName dotnet run -c Releaseprints the exact instructions — did it inline, did the bounds check go, is it vectorised, answered in ten seconds- write the loop shape the JIT already proves safe:
for (int i = 0; i < a.Length; i++), or pass aSpan<T>instead of an array plus a separate count - break the dependency chain before reaching for anything exotic — a single accumulator makes every addition wait on the one before it; independent accumulators give the out-of-order scheduler more than one ready to run at a time, with no unsafe code
- measure with BenchmarkDotNet: warm up in-process, consume every result so nothing gets deleted, report a median of several runs, and check allocations alongside the clock
DOTNET_TieredCompilation=0,DOTNET_TC_CallCountingDelayMsandDOTNET_TC_CallCountThresholdmove the tier-0/tier-1 boundary while you investigate
common bugs
- "compiled languages are fast, JIT languages are slow" — the JIT is a compiler that knows which CPU it is on, which types actually showed up and how often each block ran
- sprinkling
[MethodImpl(MethodImplOptions.AggressiveInlining)]— the inliner accepted an 829-byte-IL callee here; a refusal usually means atry/catch, andusingandlockcompile into one - "bounds checks cost extra" — the idiomatic loop has none left to pay for, and even a check that genuinely survives is two instructions (a
cmpand a never-takenjae) sitting next to work the loop was already doing - caching
a.Lengthin a local, or rewritingforeachasfor— the JIT already knew the length could not change, so no check goes away either way, andforeachover an array compiles to the identical loop as the equivalentfor - reading
Stopwatch.ElapsedMilliseconds(along) on a sub-millisecond workload and concluding two implementations are the same speed