Rerius: Bug Fixes

Found by compiling with -Wall -Wextra and hardening with AddressSanitizer + UndefinedBehaviorSanitizer, then exercising -X (every analysis pass) against several real binaries (rerius itself, /bin/ls, /bin/cat, /bin/bash, python3.12).

1. Heap buffer overflow in DSA analysis (critical): include/core/dax.h, src/analysis/dsa.c#

DAX_DSA_FUNC_SIZE was 65536, documented as covering the full opaque struct dsa_func_t. The real struct (defs[512] + uses[1024] + chains[256] + phis[64] + bookkeeping) is 172584 bytes: every calloc(1, DAX_DSA_FUNC_SIZE) call site (dsa.c, interactive.c, main.c) under-allocated by ~107KB. Any DSA run that populated more than the first ~65KB of defs[]/uses[]/chains[] wrote past the end of the heap allocation.

Fix: raised DAX_DSA_FUNC_SIZE to 262144 (headroom for future struct growth) and added a compile-time size check right after struct dsa_func_t's definition in dsa.c, so if the struct ever outgrows the constant again the build fails loudly instead of silently overflowing.

2. --dsa / --poly / --aire / --vm-trace rejected as "unknown flag": src/analysis/correct.c#

main.c handles these four long flags and --help documents them, but the separate KNOWN_FLAGS table used by the argument-correction/validation pass in correct.c was never updated when they were added, so every invocation using them printed a spurious error unknown flag message (the flag still worked, but the tool's own validator was wrong).

Fix: added all four flags to KNOWN_FLAGS.

3. NULL-pointer write / crash in parse3(): src/analysis/symexec.c#

dax_symexec_func() calls parse3(ops2, a1t, NULL, NULL) when it only wants the first operand, but parse3() unconditionally wrote to all three output pointers (a1[0]=a2[0]=a3[0]='\0'). Passing NULL reliably segfaulted (confirmed with ASan: SEGV ... WRITE ... parse3 src/analysis/symexec.c:217), which crashed -X / -P on real binaries such as /bin/ls.

Fix: parse3() now routes any NULL operand slot through a local scratch buffer instead of dereferencing NULL.

4. Signed-integer-overflow UB in raw instruction word decoding (pervasive): analysis.c, decomp.c, disasm.c, entropy.c, symexec.c#

Reading a little-endian 32-bit instruction word from a uint8_t *code buffer was written as e.g. code[off+3]<<24. code[off+3] is uint8_t, which promotes to signed int before the shift; for any byte ≥ 0x80 (common in real ARM64 encodings) <<24 overflows a 32-bit signed int, which is undefined behavior in C. UBSan caught this directly: runtime error: left shift of 250 by 24 places cannot be represented in type 'int'.

This exact pattern appeared in 132 places across 5 files.

Fix: every byte is now cast to uint32_t before shifting ((uint32_t)code[off+3]<<24), applied consistently via a scripted pass so all call sites match. macho.c's bswap32() was already safe (operates on a uint32_t value) and was left untouched.

5. Signed integer overflow in switch-table case counting: src/analysis/analysis.c#

n_cases = strtol(comma + 1, NULL, 0) + 1;: a large parsed immediate (e.g. LONG_MAX) overflows on the + 1. The result was clamped to [2,512] immediately after, so impact was limited, but the overflow itself is still UB.

Fix: clamp the parsed value to 511 before adding 1, so the addition can never overflow.

6. Missing bounds check on legacy instruction prefixes: src/arch/x86_decode.c#

insn->prefixes is a fixed uint8_t[4] (see include/arch/x86.h), but the prefix-parsing loop wrote insn->prefixes[insn->nprefixes++] = pf without checking nprefixes against the array size. A crafted/degenerate byte stream with more than 4 stacked legacy prefix bytes before MAX_BUF would write past the end of the array (UBSan: index 4 out of bounds for type 'uint8_t [4]').

Fix: guard the write with if (insn->nprefixes < (uint8_t)sizeof(insn->prefixes)). Excess prefix bytes are still consumed (parsing position still advances) so decoding of the rest of the instruction is unaffected.

---

Verification#

• Clean build with -Wall -Wextra: warning count dropped from 614 to 76 (remaining warnings are cosmetic: -Wmisleading-indentation, -Wformat-truncation: not correctness bugs).
• -Warray-bounds warnings (7, all from bug #1): eliminated entirely.
• ASan+UBSan build run against rerius itself, /bin/ls, /bin/cat with -X (every analysis pass enabled): clean exit, no crashes, no sanitizer reports.