Files
disasm/TODO.md
T
tmerritt 2ddaf102a0 Audit comment standardization + new tool (2026-10-19)
Standardized all 18 top-level projects to canonical audit comment format:
  ; CPU:$XXXX PRG:XXXX AA BB

Changes per project:
- CIRCUS: fixed 4,427 comments (CPU:XXXX -> CPU:$XXXX)
- Excitebike (JU): fixed 48 double-semicolon comments (; state $XX  ; CPU:...)
- 9 projects received canonical comments via new audit_comment_tool.py:
  Balloon Fight, Baseball, Bomberman, Burger Time, Clu Clu Land,
  Defender 2, Donkey Kong 3, Donkey Kong Jr., Tennis

New tool: _tools/audit_comment_tool.py
- Uses nesdis.py tracer as ground truth for instruction bytes/addresses
- Parses source while tracking PRG offset, emits canonical comments
- Handles .byte/.word/.dword directives, instruction comments (short format)
- Carves PRG from rom.nes when prg.bin missing

Documentation updates:
- TODO.md: Updated current state, item 3 (tool built), repo conventions, tool list
- AGENTS.md: Added audit_comment_tool.py to _tools/, updated audit convention

Verification: verify_pass1.py reports all 18 projects byte-identical with 'canoni' format.
2026-10-19 09:07:14 -04:00

23 KiB
Raw Blame History

TODO / Handoff

Written so a fresh session can pick this up cold. Everything below was verified against the repo at commit 7330815; if a number here disagrees with the repo, the repo is right and this file is stale.

Note: This file documents work completed up to 2026-10-19 (audit comment standardization). The actual repo state is always the source of truth.

The contract

Byte-identical rebuild is the hard requirement. In every project's copilot/:

make        # must print "Byte-identical ROM verified."  (or "BYTE-IDENTICAL" in _wip/)

A build that assembles but does not print that message is a failure. Requires ca65/ld65 on PATH. Never edit bytes to make a build pass — fix the source.

Two tiers exist, and the difference is real:

  • Top level (17 projects) — make compares the whole ROM against copilot/rom.nes.
  • _wip/ (19 projects) — make only runs cmp reconstructed_prg.bin prg.bin. Header and CHR are not proven for those. Promotion means adding copilot/rom.nes, switching to a whole-ROM compare, and changing ../../../_tools/ to ../../_tools/ in the Makefile. A promoted project also needs header.s, chr.s and a HEADER/PRG/CHR linker.cfg, which a Pass-0 skeleton does not carry — Lunar Pool (U) (2026-10-18) is the worked example.

Current state

  • 18 top-level projects, all byte-identical (Lunar Pool promoted 2026-10-18).
  • 19 _wip/ projects, all PRG-identical. Pyramid (U) is the only one with Pass 1.
  • 9 commits ahead of origin/master, nothing pushed.
  • _tools/verify_pass1.py rebuilds and re-traces everything: all 36 faithful, none truncated. The earlier-reported re-trace crash on MUSCLE (U) / Donkey Kong Jr. (JU) was a false alarm — nesdis.operand_value() already handles the a: prefix via token.removeprefix("a:").lstrip("$"). The make half was always working.
python3 _tools/verify_pass1.py            # all projects
python3 _tools/verify_pass1.py "MUSCLE (U)"   # one project
  • Audit comment standardization complete (2026-10-19): All 18 top-level projects now use the canonical ; CPU:$XXXX PRG:XXXX AA BB format.
    • CIRCUS: fixed 4,427 comments (CPU:XXXX → CPU:$XXXX)
    • Excitebike (JU): fixed 48 double-semicolon comments (; state $XX ; CPU:...)
    • 9 projects received canonical comments via _tools/audit_comment_tool.py: Balloon Fight, Baseball, Bomberman, Burger Time, Clu Clu Land, Defender 2, Donkey Kong 3, Donkey Kong Jr., Tennis

Open work, highest value first

1. Joypad Test Cartridge (U) — Pass 3 complete

The only top-level project whose Pass 3 was undone; now complete.

  • Pass 1 redone (6.6x: 282 → 1,773 instructions, 580 → 3,836 bytes / 3.5% → 23.4%).
  • Pass 2 complete: 202 labels, zero Label_ remaining, all 202 routine headers describe real behaviour.
  • Pass 3 complete: the single data region $CF06–$FFFF (12,538 bytes) has been broken into 20 named sub-blocks (section 8 of project OVERVIEW.md):
    • VRAM command streams at $CF06, $CF40, $D063, $D1F5 — decoded via RetAddrUpdate ($CDC8): $4C = jump, $60 = deferred PPUCTRL, bit-7-clear = VRAM write (PPU addr + data, bit 6 of data = long/short command selector).
    • OAM table groups at $D4E3, $D5C4, $D644 — 12 groups of 32-byte tables (8 entries of Y/tile/attr/X), read by CopyOamFromTable/CopyOamFromTable70.
    • Sprite template at $D6A5 (read by CopySpriteTemplateTable) and button OAM template at $D6C5 (read by PrepareButtonOam).
    • $FF ROM padding ($D6E5–$FFF9) and vectors ($FFFA–$FFFF).
  • Build verified byte-identical.

Remaining open questions (not Pass 3 blocking):

  • Which player/button each MarkerVar01–MarkerVar12 belongs to — not established, named by position deliberately.
  • HandleButtonPress1–8 write OAM tiles in order $0202, $0206, $020E, $020A, ... — sprites 1, 2, 4, 3, 5, 6, 7, 8. The out-of-order third/fourth entries are unexplained.

2. Mario Bros (JU) — Pass 2 substantial, reach-limited

_wip/Mario Bros (JU)/. Pass 1 complete: 72.2% code (5,735 instructions), byte accounting exact, BYTE-IDENTICAL.

Pass 2 substantial — 252 subroutine entry points named:

  • 90 semantic names: audio/sound routines in $F8xx-$F9xx (clearly identifiable from APU register writes), gameplay helpers like CheckZpB3Return, CheckRangeReturn012, ToggleZpFC_Bit3
  • 165 address-based names (SubCxxxx) for the remainder
  • All 252 jsr/jmp targets resolved from raw $XXXX to labels
  • Zero raw $XXXX call operands remain

667 Label_Cxxxx internal branch targets remain — local control-flow targets within routines, not subroutine entry points. These do not need semantic names for the call graph to be readable.

The real blocker is reach, not naming. 4,555 bytes (27.8%) are still data, gated behind a stack-based indirect call at $CD9E (table pointer arrives via pla, unknowable statically) and a 20-entry JSR thunk table at $C5A3. Resolving the pushed table addresses is the key to the remaining 28%.

  • Already documented: $C5A3 is a 20-entry JSR thunk table, $C0B6 is the NMI phase dispatcher. Both were found by probing, which is a cheap technique worth reusing on other projects.

2b. Lunar Pool (U) — Pass 2 and Pass 3 complete, promoted to the top level

Lunar Pool (U)/ (promoted out of _wip/ 2026-10-18 by git mv, so the relative copilot/rom.nes symlink is live and tool paths are ../../_tools/). Pass 1 complete: 67.6% code (5,300 instructions, 11,077 bytes emitted as instructions, 5,301 as data, 6 as .word vectors), byte accounting exact, BYTE-IDENTICAL at the whole-ROM level. The split was re-derived against a Mesen Code Data Logger capture (10,028 bytes Code = 61.2%); the flat tracer had undercounted by 1,738 bytes of executed code and overcounted 100 bytes of data as instructions.

Promotion detail worth reusing: a Pass-0 skeleton has no header.s, chr.s or HEADER/PRG/CHR linker.cfg, so promotion is not just a symlink and a cmp edit — copy the shape from Burger Time (U)/copilot/ (header.s/chr.s are one-line .incbins; the linker needs HEADER: start = $0000, size = $0010 before PRG). Result here: reconstructed.nes 24,592 bytes, SHA-256 6da37fd3…c8a1, equal to the original Lunar Pool (U).nes.

  • RESET is at $E80C, not $C000 — the tracer must seed it explicitly (--extra-seed 0xE80C), since the vectors are the only entry points a plain trace follows.
  • Correction to an earlier claim in this file and in the two OVERVIEW.mds: there is no jmp ($7961) RAM-pointer dispatch at $F788/$F7F6/$F828. Those were flat-tracer phantoms. $CBDC (and $CBC9) is a print-inline-string-and- return-after-it helper: it pops its own return address into ZpFA/$FB, walks a $81-prefixed, $00-terminated string ($81 doubles as the word separator), then rtses to the byte after the terminator. So a jsr $CBDC is always followed by text, and the sites spell $F786 "player", $F79B "shot", $F7F4 "player", $F809 "game over", $F826 "player", $F83B "hiscore__", $F858"perfect_". Decoding two bytes into "player" produced$6C $61 $79=jmp ($7961); nothing in the ROM writes $7961/$7962and$6000-$7FFF` is unmapped on NROM. Same trap to watch for in other titles.
  • Also corrected: the old Pass 2 claim. No SubC<addr> label exists in this prg.s and git log -S SubC finds no commit that added one; the file used to be all address-derived Label_<addr> with only Reset/NmiHandler/IrqHandler semantic. Do not copy the old "218 renamed / 374 internal" line into other docs. Pass 2 then finished for real: 705 labels, zero Label_<addr> left. Naming rule that was applied: jsr/jmp targets and logical block heads get PascalCase semantic names, internal branch targets get <Block>_<Role> (_Loop, _Next, _Done, _Store, _Return, …), and jsr/jmp operands stay raw numeric (this file's convention, as in Joypad Test Cartridge (U)) while branch operands are symbolic.
  • Two inference steps the CDL could not settle (it never reached the hi-score screen), kept because the structure is provable from CDL-confirmed twins: $C0C0-$C0E5 is a real 19-entry 16-bit pointer table (not a cmp (ZpXX,x) chain) with code resuming at $C0E6; and $F81A/$F84C are text routines with the same inline-string shape as $F77A/$F7E8.
  • Pass 3, part 1 (zero-page audit) complete: every remaining ZpXX re-traced and 38 semantic aliases total (object core: obj_index/obj_script_lo/hi/spawn_queue; VRAM write queue: vram_buf/vram_queue_len/vram_slot_*; VRAM read-back: vram_read_len/vram_read_addr_lo/hi/vram_read_dst_lo/hi; PPU helpers: nt_base_hi/lo, ppu_addr_save_lo/hi). 66 stay neutral with recorded reasons. Ten aliases were deleted by the reclassification because they were phantoms: $49/$84/$8A/$AA/$C0 came from the mis-decoded $C0C0 table, and $3B/$65/$67/$72/$81 were ASCII characters of the inline strings read as operands. Full evidence + residue in locals.inc.
  • Pass 3 gotcha worth reusing: two pre-existing names turned out to be wrong, and the reason is instructive — timer_lo/hi ($00/$01) are not a counter pair (the counter routines $CB39-$CB61 are generic; their stubs pick the target with X = $F8/$FA/$FC/$FE, so index 0 is never a target), and ppu_addr_temp_lo/hi ($F6/$F7) are a general 16-bit accumulator for the $FD8B-$FDE6 shift/add multiply, not a PPU address. Not renamed again (churn), recorded in locals.inc instead.
  • Pass 3, part 2 (tables) complete: 33 data structures named. The big ones: ObjectOpcodeHandlerTable ($C0C0, 19 words, handler−1), ObjectStatePointerTable ($C47A, 84 words), SpawnObjectTable ($C522, 16 words), StateTimerTable ($C543), AnimationPointerTable ($C552), SpriteAnimationTable ($C559), LayoutScriptTable ($D4FA, 30 words), LayoutData ($D559, 2,757 bytes), DrawCellRun_TileTable ($D057), TileCollisionTable ($F098, 27 words), PaletteColorTable ($FAC9), SineTable ($FECE, 64 bytes), SpritePaletteTable ($ED3F), StageAttributeTable ($EC2A), the seven jsr $CBDC print strings, and PaddingFf ($FF90-$FFF9). Five .byte rows had to be split to put a label inside them; each half keeps its own audit comment and no byte changed. The $FFFA-$FFFF vectors are .word NmiHandler, Reset, IrqHandler. Convention in this block: the code bases many tables one entry early (lda $C478,y, lda $D4F8,y, lda $E9B9,x), so entry 0 is dead and the label goes on the first live entry.
  • Residue the CDL could not settle — recorded, not guessed. Everything below is still .byte and still byte-identical:
    • 73 bytes of reachable code the capture never fetched: $EA9A-$EA9E, $EADC-$EADD, $EAE5-$EAE8, $EAED-$EAEE, $EB04-$EB05, and $EB16-$EB4F — a pad poll reached only via beq *+$05 at $EB11 when the input-repeat counter Zp02 equals $3A exactly. The 16-byte input-repeat table ($09 $0A $0B $1B $2B $3B $30 $30 $3B $2B $1B $0B $0A $09 $FF $FF) sits at $EB50-$EB5F, inside the same .byte run.
    • 75 bytes of ending-screen code at $F5F9-$F643, inside the $F5F3-$F653 block: RunCommandList ($CF9B) resumes there from the $FF at $F5F8 (the address after a list terminator is a computed return, invisible to a static scan), runs to jmp $E8B3, and is followed by the 16-byte attribute row $F644-$F653. Needs a completed game, so natural play never reached it.
    • ~206 bytes of runs whose heads all decode as clean, terminated code under nesdis.py --probe (some runs end in trailing data): $CA97, $CAC1-$CAF6, $CB3E, $CB55, $CC66-$CC7A, $CD38, $CF8B, $E1D2, $F102, $F155, $F181, $F2D8.
    • Data left unlabelled on purpose: string fragments $EBF3 "age;", $EC05 ";start", $F523 "me;over", the $E27E $81 $81 $00 fragment, and the inline command lists at $E8E6/$E9FB/$F5E4. Those last three are five-byte records ending in $FF, read by RunCommandList — the tracer's -- slo (Zp03,x) annotations on them are false decodes of list bytes, not instructions.
    • $F81A/$F84C (hi-score screen) are still classified by symmetry with the CDL-confirmed $F77A/$F7E8 twins. Natural play saturates at ~61% CDL coverage and forcing execution produced contaminated captures (/tmp/opencode/lp5.cdl, lp6.cdl — do not use). Beating HI. 0005000 or sinking a "perfect" shot would settle them.
  • _tools/cdl_reclass.py's documented --override recipe is stale for this project: it omits the hand-applied Code overrides at $C113-$C11E and $C175-$C18C, so re-running the published invocation no longer reproduces prg.s.
  • Three undocumented opcodes this project forced into nesdis.py: $72 (RRA abs), $42 (NOP #imm), $FA (NOP abs,X). One emitter fix: ca65 optimises an operand with a $00 high byte into zero-page form, so ABS/ABSX/ABSY/IND operands with high byte $00 now emit the a: prefix to force absolute.

3. Audit comments for Excitebike — TOOL BUILT, 9 PROJECTS COMPLETE

Excitebike (JU) is a hand-written source with semantic labels, and 1,013 of its 5,376 instruction lines carry no byte-identity audit comment, so those lines cannot be mapped back to ROM bytes without re-assembling. (An earlier revision of this item named Pac-Man as well. That was wrong on both counts: Pac-Man (USA) (Namco) is fully commented, 4,695/4,695, and the two _wip/Pac-Man folders are 12-line .incbin Pass-0 skeletons with no disassembly at all. Corrected 2026-10-18.)

  • ca65 -l gives offset + bytes, and the offset indexes prg.bin directly. Byte values must come from prg.bin, not the listing — ca65 prints rr rr for relocatable words.
  • Blocker: ca65's listing has no source line numbers. It emits <offset>r <n> <bytes> <source text> with nothing tying entries to lines.
  • Solution: _tools/audit_comment_tool.py uses nesdis.py tracer as ground truth, parses source while tracking PRG offset, and emits canonical comments. Successfully applied to 9 previously-uncommented projects (2026-10-19).
  • Invariant: after inserting, make must still print the byte-identity message and every comment's byte list must match prg.bin at the stated offset. A wrong offset that still assembles is the failure mode to watch.

4. Projects with Label_ residue

Project Label_ definitions refs in code Note
Wild Gunman (U) 718 539 Pass 1 redone; Pass 2/3 pending
Donkey Kong Jr. (JU) 688 840 Pass 1 verified; Pass 2/3 pending
Balloon Fight (JU) 497 711
NES PowerPad Test Card (U) 46 43 otherwise Pass 3 complete

The other 12 top-level projects have zero Label_ and are Pass 3 complete. Note these counts are address-derived labels only — some are legitimately internal branch targets and may never deserve a semantic name.

5. MUSCLE (U) — coverage plateau

Pass 1/2/3 complete and promoted to top level as #16. The stage-payload trace plateaus at ~41% of $B0FB–$BEBF and will not go higher by cycling stages:

  • $B0FB–$BEBF is a shared pool, not per-stage data: per-actor bytecode + compressed-VRAM RLE + music note streams. $BD28 is an 8-entry per-actor entry-offset table ($A063 adc $BD28,X, X = ActorIndex $1E).
  • Forcing all 8 actor indices added zero new bytes.
  • The sound engine reads the same window ($B03C lda (SoundReadPointerLo),Y → cmp #$F0 → lda $B0E3,y = note decoding), which is where the $FF terminators come from.
  • To go further needs other music tracks playing and actors 2–4 in non-default states — i.e. scripted match play, not button taps. Out of scope for a coverage tracer.
  • Two corrections recorded: $0326 is not a level byte (never leaves $00 even in a confirmed match), and $0320 is a boolean, not a screen-state machine.

Gotchas that will bite you

These cost real time this session. All are recorded so they don't cost it twice.

  • Undocumented opcodes silently truncate a trace. A tracer knowing only the 151 official opcodes halts on the first one used as ordinary code. Joypad died at $C09E on DCP $3C20 ($CF) and lost 3,256 bytes of code. nesdis.py --undocumented adds all 105 (default off, so existing make regen pipelines stay byte-stable). Mario Bros: 14.6% → 14.9% from this alone, but combined with seeding it mattered more.
  • ca65 V2.19 implements none of the undocumented opcodes. They cannot be written as mnemonics. nesdis.py emits them as explicit .byte with the disassembly in the comment, keeping them in the code region so the split and the audit trail survive.
  • RESET and IRQ are usually the same address. The semantic-name map used to overwrite, renaming the primary entry to IrqHandler. Now setdefault, in RESET/NMI/IRQ priority order.
  • render() stamped every branch as Label_XXXX and never reconciled with the semantic map, so a branch into a named routine emitted a label that was never defined — the file did not assemble at all.
  • Control-flow targets must win over fall-through. A bne targeting $C528 was being swallowed by a fall-through NOP #$A5 at $C527 whose operand was $C528. No assembler branches mid-instruction. Fixed in nesdis.py.
  • Branch targets inside a surviving instruction must be pruned from the target set, or emit_code's linear walk consumes those bytes as an operand and the label goes undefined.
  • NMI must fire once per frame, not per $2002 read. The main loop polls $2002 to sync on vblank, and each read clears bit 7 — which re-arms NMI in the same scanline. Guarding on the vblank flag alone gave 40,900 NMIs across 3,000 frames and silently corrupted all game state, flattening coverage at 26%.
  • Input goes on joy1 = $4016 with a real latch/shift register. The poll routine's lda $4016,X with X=1 lands on $4017, but only bits 0–1 are compared, so driving $4017 consumes nothing and changes no coverage.
  • A static trace cannot settle the code/data split; the emulator's Code Data Logger can. The trace misses code behind computed dispatch, and it cannot tell inline string data from an instruction that decodes out of the same bytes — both leave the rebuild byte-identical while the split stays wrong. Mesen marks every PRG byte the CPU fetched as an opcode vs read as data; capture it with mesen_cdl_file(action:'save', memoryType:'NesPrgRom', filepath:...) (cumulative — it does not reset) and feed it to _tools/cdl_reclass.py, which rewrites the split to match wherever the CDL has an opinion and leaves the rest alone. Lunar Pool went 9,403 → 11,041 code bytes that way (1,738 data→code, 100 code→data) and then recorded zero disagreement with the CDL; 36 further bytes were moved by hand later ($C113-$C11E, $C175-$C18C), so its file now carries 11,077. Forcing the PC to reach unvisited code is not a general substitute: a routine that returns through a popped address runs away when entered that way and marks garbage as Code (Lunar Pool $F81A).
  • A jsr to a helper that pops its own return address means the bytes after it are text, not code. Lunar Pool's $CBDC/$CBC9 pops the caller's return address into a pointer, prints a $81-prefixed, $00-terminated string ($81 doubles as the word separator), then rtses to the byte after the terminator. A flat tracer started an instruction two bytes into "player" ($6C $61 $79 = jmp ($7961)) and produced a plausible-looking "RAM-pointer dispatch gate" that does not exist — $7961 is never written and $6000–$7FFF is unmapped on NROM. Suspect this shape whenever a jmp ($xxxx) operand spells ASCII.
  • Odd vector targets are not bugs. Burger Time's $FF7F NMI and CIRCUS's $4CB7 IRQ point into unused space. That is common in this set — verify before assuming a handler is broken.
  • Vectors are not in the iNES header. They are the three little-endian words at the end of the 16 KB PRG ($FFFA–$FFFF in the $C000 mirror). Bytes 0–9 of rom.nes carry only sizes, mirroring and mapper. The snippet to regenerate the whole column is in the root OVERVIEW.md.
  • py65 is not a build dependency and must be unpacked on demand: python3 -m pip download --no-deps -d <dir> py65 && unzip -q <dir>/py65-*.whl -d <dir>/py65lib. /tmp is volatile — a reboot wiped it once. Anything reusable belongs in _tools/.
  • ROM filenames contain spaces and parentheses. copilot/rom.nes is a relative symlink to a space-free name. Keep it working; don't rename it.
  • .gitignore lists ROMS/* but the directory is _ROMS, so the pattern matches nothing and ROM binaries stay tracked. Check before committing large blobs.

Repo conventions that must survive

  • Every instruction carries a trailing ; CPU:$XXXX PRG:XXXX AA BB comment with PRG offset and exact emitted bytes. This is the byte-identity audit trail. Preserve it even while renaming labels or restructuring files. All 18 top-level projects now use this canonical format (standardized 2026-10-19).
  • Unreached or unresolved bytes stay as .byte/.word/.incbin with ROM provenance. Never delete or reformat them away.
  • Two separate symbol files: symbols.inc = NES hardware constants only; locals.inc = zero page and forward references. Unresolved zero-page bytes keep neutral ZpXX names rather than speculative ones.
  • Routine headers follow the template in prg.s: global + local description, parameters, returns, clobbers.
  • Three passes, in order, without shortcuts: Pass 1 code/data split (stop for review) → Pass 2 label replacement → Pass 3 data tables. Do not skip straight to guessy rewrites.
  • make regen (where it exists) must keep reproducing prg.s byte-for-byte on the standard library alone. MUSCLE has this pipeline; re-check it after any nesdis.py change.

Where the detail lives

  • Root OVERVIEW.md — inventory of all 36 projects, per-project detail, the Pass 1 methodology note, and the audit-comment TODO with its full analysis.
  • <Project>/OVERVIEW.md — per-project architecture and pass status. The most detailed ones: MUSCLE (U) (Pass 3b/3c/3d), Joypad Test Cartridge (U), _wip/Mario Bros (JU).
  • _tools/nesdis.py — Pass 1 tracer (--undocumented, --auto-seed, --extra-seed, --table, --data, --probe).
  • _tools/verify_pass1.py — faithfulness + completeness check across all projects.
  • _tools/audit_comment_tool.py — adds canonical ; CPU:$XXXX PRG:XXXX AA BB audit comments to disassembly source using tracer as ground truth. Usage: python3 _tools/audit_comment_tool.py "Project Name".
  • _tools/cdl_reclass.py — reclassify a project's code/data split from a Mesen Code Data Logger capture (dry run by default; --apply rewrites prg.s, then make must still print BYTE-IDENTICAL). Its docstring covers capture and the safety rules.
  • _tools/bytecode_dis.py, pass2.py, trace_stage_payload.py — per-pass helpers.