Recompiling Functions
These docs are a bit more scrappy.
Today is 5th October 2026, time is running out, I got 6 weeks!
Time for the fun part.
And that's turning each discovered function into one C++ function.
Where this happens
Recompiler::recompile (src/xbox-recomp2/src/recompiler.rs).
What goes in
For each function, the recompiler has:
- The function: its start address and size.
- Its body (usually): the instructions discovery found, and how they connect.
- The config: midasm hooks, switch tables, banned and invalid ranges.
- The executable: the bytes to decode, symbols and imports.
A running example
We'll follow one small function from Shadow the Hedgehog (NTSC-U) the whole way through.
It's at 0x11410:
0x11410: mov eax, [0x005356F0] ; read a global
0x11415: test eax, eax
0x11417: jnz 0x11423 ; already set? skip to the end
0x11419: call 0x113B0 ; otherwise, compute it...
0x1141E: mov [0x005356F0], eax ; ...and store it
0x11423: ret
In C, that's roughly:
int cached_value;
int get_cached_value(void) {
if (cached_value == 0)
cached_value = compute_value(); // sub_113B0
return cached_value;
}
And here's the C++ XboxRecomp2 writes for it:
X86_PROTECTED_FUNC(sub_11410) {
uint32_t val_uint32_0 = 0;
// mov eax, [0x005356F0]
ctx.eax.u32 = X86_LOAD_U32(0x5356f0);
// test eax, eax
ctx.eflags.update_test_flags_U32(ctx.eax.u32 & ctx.eax.u32);
// jnz 0x00011423
if (X86_CC_NZ(ctx)) goto loc_11423;
// call 0x000113B0
ctx.SubEsp(1); //Adjust stack pointer for call instruction
sub_113B0(ctx, base);
// mov [0x005356F0], eax
X86_STORE_U32(0x5356f0, ctx.eax.u32);
loc_11423:
// ret
ctx.esp.u32 += 0x4; //Adjust esp for return instruction
return;
}
With Optimizations Disabled*
Every generated function receives the same two parameters:
-
ctx: the guest CPU state. e.g.- Registers (
ctx.eax,ctx.esp, ...), - Flags (
ctx.eflags) - x87 FPU.
- Registers (
-
base: a pointer to the start of guest memory. e.g.X86_LOAD_U32(0x5356f0)reads 4 bytes atbase + 0x5356f0.
So the guest's registers live in ctx, and its RAM lives at base.
The C++ just does to them what the original instructions did.
The rest of this page builds that output one piece at a time.
At a glance
Each function goes through three stages:
flowchart LR
BUILD["Build the function model"]
PASSES["Run optimisation passes"]
WRITE["Write the model as C++"]
BUILD --> PASSES --> WRITE -
Build (
codegen/function/build/mod.rs) groups the function's instructions into a function model (codegen/function/model/mod.rs).The model holds blocks of instructions, with their labels and exits.
-
Passes (
codegen/optimizations/) optimise code before it's emitted. -
Write (
codegen/function/write/mod.rs) turns the model into C++.
Why build a 'model' first?
C++ needs some things before the code that uses them:
- Hooks must be declared above the function that calls them.
- Scratch variables must be declared at the top of the function.
And an optimisation often needs to see the whole function before it can change one instruction.
Step by Step
Build the function model
Where this happens
codegen/function/build/mod.rs: build, called by build_function in codegen/function/mod.rs.
1. Find labels and hooks
Where this happens
codegen/function/build/labels/mod.rs:
collect_labels_from_bodyfor bodiescollect_labelsfor user defined ranges.
Jumps inside a function become C++ gotos.
A goto needs a label to land on:
if (X86_CC_NZ(ctx)) goto loc_11423;
...
loc_11423:
Labels are named loc_<address>, in uppercase hex.
This follows IDA Pro convention most people are used to.
Which addresses get a label?
Where this happens
codegen/function/build/labels/mod.rs:
collect_labels_from_bodyfor function bodies.collect_labelsandcollect_jump_targetfor user defined ranges.
Basically every time a jmp instruction pops up in original ASM
An instruction gets a label when it's a target of:
- a
jmp, or a conditional jump likejnz,jeorloope. - a case in a
[[switch]], i.e. ajmpthrough a jump table. - a midasm hook jump (
jump_address,jump_address_on_true,jump_address_on_false). - an instruction that runs right after another, but isn't right after it.
- For example, if an (in)sane compiler decided to put data between two instructions.
- The generated code would need a
gototo reach it; see Write each instruction.
The function entry, of course also gets one.
In our example, there's only a goto for 0x11423 .
It's where jnz jumps to.
Hooks
Where this happens
codegen/function/build/labels/mod.rs: collect_labels_from_body and collect_labels fill LabelPass::hook_addresses.
This pass also makes keeps track of addresses that have a midasm hook.
We'll need them later.
2. Pick the instructions
Decide which bytes are code, and in what order to write them.
Where this happens
codegen/function/build/mod.rs:
SelectedWalkfor bodies.InstructionWalkfor user defined ranges.
Both feed build.
There are two ways to do this.
From the function body (the usual way)
Where this happens
codegen/function/build/mod.rs: SelectedWalk::next_step.
Discovery saves every instruction it reaches in a function.
And all of the branches/control flow between them.
Recall FunctionBody, which holds:
Instructions (address, length):
0x11410 (5) 0x11415 (2) 0x11417 (2)
0x11419 (5) 0x1141E (5) 0x11423 (1)
Edges:
0x11410 -> 0x11415 NextInstruction
0x11415 -> 0x11417 NextInstruction
0x11417 -> 0x11423 BranchTaken ; jnz jumps
0x11417 -> 0x11419 BranchNotTaken ; jnz falls through
0x11419 -> 0x1141E NextInstruction ; call returns here
0x1141E -> 0x11423 NextInstruction
0x11410: mov eax, [0x005356F0] ; read a global
0x11415: test eax, eax
0x11417: jnz 0x11423 ; already set? skip to the end
0x11419: call 0x113B0 ; otherwise, compute it...
0x1141E: mov [0x005356F0], eax ; ...and store it
0x11423: ret
Instructions are always written out in address order; i.e. same order you read it in disassembly.
There's some awkward cases to be aware of
Thankfully we handle them relatively cleanly.
But if you're a nerd and want to know; the fact we replay the decoded instructions gives us a few things for 'free'.
- Data inside a function. Bytes between instructions are never decoded.
- So a jump table sitting in middle of code is unaffected.
- Though I don't think any sane compiler would emit this; it is valid.
- Entry isn't the lowest address. Some code sits before the function's entry.
- See Jump to the entry.
From a user defined range (fixes.functions)
Where this happens
codegen/function/build/mod.rs: InstructionWalk::next_step.
You know, for those moments where discovery gets a function wrong.
Functions listed in [fixes.functions] use a simpler mechanism.
Every byte from the start address to start + size is decoded.
0x11410 decode -> 5 bytes, next at 0x11415
0x11415 decode -> 2 bytes, next at 0x11417
...
0x11423 decode -> 1 byte, reached start + size, stop
Easy peasy! Of course, banned_addresses and invalid_instructions are skipped.
Use this when discovery got a function wrong.
You decide the range, so whatever's in it gets recompiled.
The catch: any data inside the range gets decoded as code too.
Decode
zydis decodes the bytes:
bytes: 75 0A
decoded: jnz 0x11423 (2 bytes)
Run optimisation passes
Where this happens
codegen/optimizations/mod.rs, called by Recompiler::generate in recompiler.rs.
See Optimizations.
Write the model as C++
Where this happens
codegen/function/mod.rs: write_function.
- Calls
writeincodegen/function/write/mod.rs. writecallswrite_body.
3. Name the function
Where this happens
codegen/function/mod.rs: write_function.
Pick a name.
The name comes from the symbol at the function's address:
- The entry point is
_xstart. - Functions without a symbol get
sub_<address>, e.g.sub_11410.- Follows IDA Pro convention, which folks are already familiar with.
- Easier to match up against a disassembly.
4. Declare hooks
Where this happens
codegen/function/write/midasm_hook.rs: write_hook_declaration.
Hooks are your own C++ functions, called from the generated code.
From ShadowRecomp:
extern bool EasierUnlockLastStoryMidAsmHook(X86Context& __restrict ctx, uint8_t* base);
In order to call your own C++ replacement for the original, we use extern. The clang compiler will later link it up.
Calling convention is standard. See midasm hooks for how to write the C++ side.
5. Write the opening lines
Where this happens
codegen/function/write/mod.rs: write.
The opening line is written:
X86_PROTECTED_FUNC(sub_11410) {
So each function exists under two names:
__imp__sub_11410: the recompiled code itself.sub_11410: a public entry pointing at it.- Other generated code calls this one.
This is how you replace a whole function.
Basically, if your host projects defines its own sub_11410, every call goes to your function.
Should there be a need to call it, the original will be in __imp__sub_11410.
Your whole function hooks/overrides must be specified in config!
Add the overrides [fixes.function_hooks]! Some optimisations break ABI, so failure to do can lead to crashes.
(Though more likely- build will just fail, as I added some safeguards...)
6. Declare scratch variables
Where this happens
codegen/function/write/locals.rs: declare.
codegen/function/write/mod.rs: write_local.
write_body calls both after optimisation passes.
Some instructions need a temporary value in their C++.
For example, an indirect call loads the target first:
// call [eax]
val_uint32_0 = X86_LOAD_U32(ctx.eax.u32);
ctx.SubEsp(1); //Adjust stack pointer for call instruction
X86_CALL_INDIRECT_FUNC(val_uint32_0);
So the write step checks the instructions after optimisation passes, and lists what's needed at the top of the function:
| Variable | Declared for |
|---|---|
uint32_t val_uint32_0 | call, rep movsw, cmpsd |
uint32_t val_uint32_1 | cmpsd (needs two) |
uint64_t val_uint64_0, val_uint64_1 | punpckldq, punpckhdq |
x87_float val_double_0 | x87 math and compares (fadd, fsin, fcomip, ...), sqrtss |
x87_float val_double_1, val_double_2 | fsincos (takes all three) |
Each variable is declared once, at the top of the function.
In our example, the call asks for val_uint32_0:
uint32_t val_uint32_0 = 0;
7. Jump to the entry, if needed
Where this happens
codegen/function/write/mod.rs: write_entry, called from write_body. entry_of in codegen/function/build/mod.rs chooses the entry action.
TODO: Investigate and dedupe the duplicated code
Have not investigated scope of this, but a lot of 'forced inlining' like here could lead to unnecessary instruction cache thrashing.
Sometimes the first instruction in address order isn't the function entry
There's a few cases like tail calls (jmp), where a function jumps to another location to complete its work.
In that case, we include that other code inside the function body. And make the function body jump to the real entry using goto.
clang is able to optimise this out, thankfully.
; ... called from function 0x1AE10 (0x1AE25)
0x1ACD0: mov eax, [esp+0x04]
0x1ACD4: shr eax, 0x10
0x1ACD7: cmp ax, [ecx+0x02]
0x1ACDB: jnz 0x1ACED
0x1ACDD: mov dx, [esp+0x04]
0x1ACE2: cmp dx, [ecx]
0x1ACE5: jnz 0x1ACED
0x1ACE7: mov eax, [ecx+0x04]
0x1ACEA: ret 0x04
0x1ACED: xor eax, eax
0x1ACEF: ret 0x04
; ... other functions live here, not ours ...
0x1AE10: mov eax, [esp+0x04] ; the entry
0x1AE14: shr eax, 0x10
0x1AE17: cmp ax, 0xFFFF
0x1AE1B: jz 0x1AE2A
0x1AE1D: movsx eax, ax
0x1AE20: mov ecx, [ecx]
0x1AE22: lea ecx, [ecx+eax*8]
0x1AE25: jmp 0x1ACD0 ; pulls the block above into this function
0x1AE2A: xor eax, eax
0x1AE2C: ret 0x04
Labels stay in address order, so the entry ends up in the middle. The goto at the top makes execution start there anyway:
X86_PROTECTED_FUNC(sub_1AE10) {
goto loc_1AE10; // execution really starts here
loc_1ACD0:
// mov eax, [esp+0x04]
...
// ret 0x04
return;
loc_1AE10:
// mov eax, [esp+0x04]
...
// jmp 0x0001ACD0
goto loc_1ACD0; // the tail jmp, now a plain goto
loc_1AE2A:
// xor eax, eax
...
}
8. Write each instruction
Where this happens
codegen/function/write/mod.rs: write_block, write_guest and write_instruction. One block of the function model at a time.
The main event 🧑🍳🦔. Every instruction becomes 1-more lines of C++.
For each instruction, in address order, up to five things are written:
- A label, if Find labels and hooks marked this address.
- A comment with the original instruction, e.g.
// jnz 0x00011423. - A hook call, if there's a hook at this address.
- The instruction's C++.
- A block exit, if needed: a
gotoor an abort.
Here's our jnz and the instruction it jumps to, split into those parts:
// jnz 0x00011423 // <-- 2. comment
if (X86_CC_NZ(ctx)) goto loc_11423; // <-- 4. C++
...
loc_11423: // <-- 1. label
// ret // <-- 2. comment
ctx.esp.u32 += 0x4; //Adjust esp for return instruction
return; // <-- 4. C++
Write the comment
Where this happens
codegen/function/write/mod.rs: write_instruction_prefix calls DisasmFormatter::format in decode/format.rs.
The decoded instruction is printed in Intel syntax as a // comment.
Call the hook
Where this happens
codegen/function/write/midasm_hook.rs:
write_hook_call_for_bodywrite_hook_call
If there's a midasm hook here, its call goes before the instruction's own C++. From ShadowRecomp:
// cmp edi, 0x0A
if (EasierUnlockLastStoryMidAsmHook(ctx.edi)) {
goto loc_23D64F; // jump_address_on_true
}
else {
}
ctx.eflags.update_compare_flags_U32(ctx.edi.u32, 0xAu);
Each register is passed as ctx.<register>, so the hook can read and change it.
| Hook keys set | Call becomes |
|---|---|
Only name (and registers) | Hook(...); |
return | Hook(...); return; |
jump_address | Hook(...); goto loc_...; |
return_on_true / return_on_false | if (Hook(...)) { return; } else { ... } |
jump_address_on_true / _on_false | if (Hook(...)) { goto ...; } else { ... } |
Emit the instruction
Where this happens
codegen/function/write/emit/instructions/:
- one file per mnemonic, such as
add.rs lookupin itsmod.rspicks the file
Each emitter knows how to turn one instruction into C++. Some examples:
| Guest instruction | C++ |
|---|---|
add eax, ecx | X86_ADD32(ctx.eax.u32, ctx.ecx.u32, ctx); |
add eax, [ecx] | X86_ADD32(ctx.eax.u32, X86_LOAD_U32(ctx.ecx.u32), ctx); |
add [ecx], eax | X86_ADD32_MEM(ctx.ecx.u32, ctx.eax.u32, ctx, base); |
add al, cl | X86_ADD8(ctx.eax.u8_lo, ctx.ecx.u8_lo, ctx); |
jnz 0x1040 | if (X86_CC_NZ(ctx)) goto loc_1040; |
A few patterns to notice:
-
Registers are fields of
ctx:eaxisctx.eax.u32axisctx.eax.u16alisctx.eax.u8_lo
-
Memory goes through macros:
X86_LOAD_U32(addr)reads atbase + addrX86_STORE_U32(addr, value)writes atbase + addr
- Helpers update the flags:
testbecomesctx.eflags.update_test_flags_U32(...)- a later
jnzreads them throughX86_CC_NZ(ctx)
- The macros live in
x86_context.h, which is generated alongside the code.
Looks something like this:
ctx.out.line("if (X86_CC_Z(ctx)) {").indent();
ctx.out.line("return;").dedent();
ctx.out.line("}");
Calls and jumps
Where this happens
codegen/function/write/emit/instructions/control_transfer/call.rsandjmp.rs:emitcodegen/function/write/emit/instructions/control_transfer/helpers.rs:print_conditional_branchandprint_function_call_addr
Control flow needs a bit more care. It depends on where the target is.
Inside the function (the target is in the body): a goto.
// jnz 0x00011423
if (X86_CC_NZ(ctx)) goto loc_11423;
Another function: call it by name.
// call 0x000113B0
ctx.SubEsp(1); //Adjust stack pointer for call instruction
sub_113B0(ctx, base);
A jmp to another function is a tail call. It becomes a call, then a return:
// jmp 0x00012070
sub_12070(ctx, base);
return;
Through a register or memory (call eax, call [eax]):
The address isn't known until runtime, so we look it up.
// call [eax]
val_uint32_0 = X86_LOAD_U32(ctx.eax.u32);
ctx.SubEsp(1); //Adjust stack pointer for call instruction
X86_CALL_INDIRECT_FUNC(val_uint32_0);
X86_CALL_INDIRECT_FUNC looks up the host function for a guest address.
// `x86_func_mapping.cpp`
X86FuncMapping X86FuncMappings[] = {
{ 0x11000, sub_11000 },
{ 0x11010, sub_11010 },
...
Kernel imports (call [0x00429AE4], see Kernel thunks): called directly e.g. __imp__RtlEnterCriticalSection(ctx, base);.
Switch tables: a jmp through a switch table becomes a C++ switch. Cases inside the function use goto. Cases in another function call it, then return.
// jmp [eax*4+0x180B4]
switch (ctx.eax.u32) {
case 0:
goto loc_1801E;
case 1:
goto loc_18036;
case 2:
goto loc_180A5;
case 3:
goto loc_18059;
case 4:
goto loc_1806F;
default:
// With `debug = true` in `[main]`, the default case reports and aborts.
// With `debug = false`, it is just `X86_UNREACHABLE();`.
fprintf(stderr, "Unhandled switch index at 0x18017: %u (5 entries); check [[switch]] configuration.\n", static_cast<unsigned>(ctx.eax.u32));
fflush(stderr);
std::abort();
}
loc_1801E:
// push edi
X86_PUSH(ctx, base, ctx.edi.u32);
...
Platform-specific memory
Where this happens
GuestPlatform::routeinplatform.rsis the hook every platform implements.Xbox::routeinsrc/xbox-recomp2-xbox/src/platform/mod.rsis the Xbox version.Xbox::routecallsrouteinfs.rsandds.rs, in the same directory.
Some addresses mean something special on an Xbox.
Two examples:
- The
fssegment, which holds the kernel's per-CPU data. - The GPU's registers.
Exit the block
Where this happens
block_exitincodegen/function/build/mod.rschooses the exit during Build.write_blockincodegen/function/write/mod.rswrites the exit.write_blockcallswrite_barrierin the same file for aborts.
C++ runs top to bottom. The guest might not.
Instructions are written in address order.
If execution would continue somewhere other than the next written line, something must be added.
With a function body, after each instruction:
| Situation | Written after it |
|---|---|
| Continues to the next written instruction | nothing |
| Continues somewhere else | goto loc_<target>; |
Ends the block with jmp or ret | nothing (it leaves by itself) |
Ends the block with int3 | abort |
| Ends the block with an unsupported instruction | abort |
| Ends the block with anything else | abort |
In the case of Continues somewhere else; it looks something like this:
// mov eax, 1
ctx.eax.u32 = 0x1u;
goto loc_11020; // the next written line isn't next in execution
loc_11010:
...
Say that for some insane reasons the function had data in middle of it, or we just couldn't decode. That's how we handle it.
Why abort instead of carrying on?
Makes debugging easier. If we abort, we know we f*cked up somewhere.
9. Close the function
The closing brace is written:
return;
}
The finished text is appended to the current x86_recomp.N.cpp unit. See Write the outputs.
When things go wrong
Each problem is printed to the console with its address. The generated code also gets a printf or an abort, which fires when the game reaches that spot.
| You see | Cause | Fix |
|---|---|---|
UNKNOWN FUNCTION REACHED | A call to an unknown function | Add it to [fixes.functions]. Garbage address: check boundaries. |
Missing instruction | No emitter for it | Replace the function, or add an emitter |
Function doesn't end with JMP or RET | A [fixes.functions] range is too short | Check its size in a disassembler |
Selected block ends without control transfer | Wrong boundaries or missed code | Set the range with [fixes.functions] |
Selected instruction ... | Bad bytes: decode, mapping, length | Report it, or use [fixes.functions] |
INT3 reached | Breakpoint or padding | Execution went somewhere it shouldn't |
Unhandled switch index | No case for the index | See Switch Tables |
midasm hook jumps outside function | Hook target is outside the function | See Find labels and hooks |