tmerritt/disasm
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MUSCLE (U): decode the bytecode VM layer (Pass 3b)

Browse Source 
The 42 token programs were the last undecoded part of MUSCLE. Each now carries a
token-by-token listing above its unchanged .byte data, generated by
_tools/bytecode_dis.py and wired into the build as 'make pass3'.

Encoding, derived from RunBytecodeInterpreter ($9AE8) and BytecodeHandlerTable
($9237): the dispatch is one byte per element. A byte in $C8-$EF selects one of
the 40 handler-table entries; every other byte takes the literal path at $9B08
and is pushed onto the operand list at $0033[Y], accumulated into $003C[Y].

The one exception is four opcodes whose handler's first instructions are
'lda ($1A),Y / iny', so they consume the following byte inline:
  $CB LOOP_BEGIN  -> $7C,X count + $7E,X body address
  $D1 SET_RATE    -> $43,X
  $D7 DRAW_SPRITE -> count for the $A660 blit
  $E1 SET_WAIT    -> $7A,X
$CB and $C9 are the loop pair: LOOP_END decrements $7C,X and, if non-zero,
reloads Y from $7E,X, rewinding the interpreter to the top of the body.

Decoded 377 opcodes and 937 literal operands across 42 programs; all 40 defined
opcodes are used, so the opcode set is complete.

Two standalone programs contain no byte in $C8-$EF at all, so every byte takes
the literal path - the listing states that rather than implying code.

Each mnemonic was read off its handler's own straight-line path up to its first
'jmp $9AE8' / 'jmp $9B25'. Handlers that only tail-call a game routine are named
for the routine rather than guessed at, since the semantics live in the callee.

bytecode_dis.py inserts comment lines only; .byte data is copied through
verbatim. Verified: all 16,384 PRG bytes still emitted exactly once, every value
equal to the ROM, zero audit mismatches, and 'make regen' (pass1 -> pass2 ->
pass3) reproduces prg.s byte-for-byte. Byte-identical ROM verified.

MUSCLE (U): trace the stage payload with py65 (Pass 3c)

The stage payload at $B0FB-$BEBF could not be decoded statically, and the
trace confirms why: across all 16 KB of code there is exactly ONE operand in
that range - $BD28, the literal pointer table itself. The real address is
formed at runtime by 'adc $BD28,X', where A is a value carried in the
bytecode stream, so the readers are indexed by a runtime value and cannot be
enumerated by searching the code.

FCEUX cannot observe this either: its Lua API offers registerexec /
registerexecute / registerrun / registerwrite but no memory-read hook, and with
no static list of readers there is nothing to hook.

_tools/trace_stage_payload.py therefore uses py65's
ObservableMemory.subscribe_to_read, which fires on every read of a range and
can report PC from inside the hook. py65 is only a 6502 core, so the tracer
adds the minimum hardware to boot the game: a scanline counter off
processorCycles, $2002 vblank at scanline 241 cleared on read, NMI at the same
scanline, $4014 OAM DMA, and the $2005/$2006/$2007 latches. Without NMI the
game never leaves $80E0 (lda $0321 / beq), because $0321 is the flag the
bytecode VM raises.

Results over 300 frames (26.8M cycles, 4,071 NMIs): 2,548 payload reads from
44 distinct reader PCs, touching 914 of 3,421 bytes.

Findings:
- The hottest reader is $AC82, the compressed-VRAM RLE decoder: 1,046 of its
  1,128 reads advance by exactly +1, comparing each byte against $C0.
- $9B50 is a backward copy (lda ($12),Y / sta ($10),Y / dey), stepping -1.
- 107 distinct 8-byte blocks had all eight bytes read, with spacing dominated
  by $08 - a dense 8-byte record array - plus a sparser group on a $D0 stride
  ($BBE0, $BCB0, $BD80, $BE50).
- Readers use three disciplines: forward sequential, backward sequential, and
  variable stride consistent with a fixed-offset field walk.

py65 is deliberately NOT a build dependency: the tool documents unpacking the
wheel on demand. 'make regen' is unaffected. Byte-identical ROM verified.

MUSCLE (U): fix the NMI/controller bugs in the payload tracer; correct the Pass 3c model

The Pass 3c tracer had two bugs that were suppressing all progress, both now
fixed in _tools/trace_stage_payload.py (previously scratch, now durable):

* NMI re-armed by $2002 reads. The vblank flag must be cleared once per frame,
  not on every read of $2002 -- the main loop polls $2002 and spins on the
  bytecode VM flag at $8226, so each poll cleared bit 7 and re-armed NMI inside
  the same scanline. That gave 40,900 NMIs across 3,000 frames and corrupted all
  game state. A per-frame nmi_fired flag restores exactly one NMI per frame
  (verified: 600 frames -> 600 NMIs).

* Input went to $4017. The poll routine at $AA69 is `lda $4016,X` with X=1 and
  only compares bits 0-1, so driving $4017 consumed nothing -- all masks tested
  gave byte-identical traces. The game consumes joy1 on $4016 and needs a real
  latch/shift register. With it, START matters: 47.7% coverage vs 39.0%.
  Match entry is two discrete START presses (title -> char select -> match).

Coverage 26.3% -> 40.9%. Both bugs and the port subtlety are documented in the
tool header so they are not re-derived.

Corrects the Pass 3c conclusion. $B0FB-$BEBF is NOT per-stage level data:

* $BD28 is an 8-entry per-ACTOR entry-offset table, not a stage table.
  $A063 `adc $BD28,X` with X = ActorIndex ($1E), plus #$32 at $A071 when
  PlayerActiveFlag ($78,X) is set. Observed actors are $00-$04.
  Forcing all 8 actor indices adds zero new bytes.

* The sound engine reads the same window: $B03C `lda (SoundReadPointerLo),Y`
  with `cmp #$F0 / sbc #$F0 / tay / lda $B0E3,y` is note decoding, which is
  where the $FF/$F0 high-nibble terminators come from.

So the window is a shared pool -- per-actor bytecode, compressed-VRAM RLE, and
music note streams -- and stage cycling cannot raise coverage past ~40%. The
remaining 2,161 B need other music tracks and scripted match play, not taps.

Also corrects two stale claims: $0326 is not a level byte (never leaves $00 even
in a confirmed match), and $0320 is a boolean, not a screen-state machine.

Build unaffected: byte-identical, and `make regen` still reproduces prg.s
byte-for-byte.

MUSCLE (U): promote to the top-level inventory as #16

git mv _wip/MUSCLE (U) -> MUSCLE (U). The relative symlink
copilot/rom.nes -> ../M.U.S.C.L.E. (U).nes survives the move untouched.

* copilot/Makefile: tool paths ../../../_tools/ -> ../../_tools/ for NESDIS,
  PASS2 and BYTECODE. `make` prints the byte-identity message and `make regen`
  still reproduces prg.s byte-for-byte, both verified after the move.
* MUSCLE (U)/OVERVIEW.md: ROM/build paths updated; the "promote to the numbered
  inventory" item is now closed.
* Root OVERVIEW.md: added as #16, removed from the _wip skeleton table
  (20 -> 19), added a ### 16 detail section, and its vector/mapping notes moved
  out of the skeleton section.

All 16 top-level projects verified byte-identical.

Add Pass 1 verification tool; teach nesdis.py the undocumented opcodes

_tools/verify_pass1.py verifies Pass 1 across every top-level project in two
independent directions:

  * Faithful  - does the source still rebuild its ROM byte-identical?
  * Complete  - does the committed source claim as much code as a fresh trace
                can actually reach?

The second is the one that catches real problems. A project can reproduce all
16,384 bytes perfectly while calling 95% of them data, because the tracer gave
up early. Instruction *counts* are compared rather than bytes, so the check is
independent of comment style (which is not uniform across the repo).

Result: all 16 projects are faithful, and Joypad Test Cartridge is the only
truncated Pass 1 -- 282 committed instructions against 1,773 reachable.

nesdis.py gains three fixes needed to make that verdict trustworthy:

* --undocumented (default OFF) decodes the 105 undocumented NMOS opcodes.
  They are used as NOP padding and as real RMW ops, and a tracer that halts on
  them loses most of the code: Joypad goes 580 -> 3,846 code bytes. Default off
  so MUSCLE's `make regen` still reproduces prg.s byte-for-byte.

* Undocumented opcodes are emitted as explicit `.byte` with the disassembly in
  the comment. ca65 V2.19 implements none of them, so they cannot be written as
  mnemonics; keeping them in the code region preserves the code/data split and
  the audit trail. Official 0xEA NOP still encodes normally.

* Two latent bugs, both now reachable:
  - RESET and IRQ are commonly the same address, and the semantic-name map was
    overwriting, renaming the primary entry point to `IrqHandler`. Now
    setdefault, in RESET/NMI/IRQ priority order.
  - render() stamped every branch operand as Label_XXXX and never reconciled it
    with the semantic name map, so a branch into a named routine emitted a label
    that was never defined and the file would not assemble.

Joypad Test Cartridge (U): redo Pass 1, complete Pass 2; merge _wip into the inventory

Pass 1 (redone). The committed split claimed 282 instructions / 580 bytes (3.5%).
The tracer halted at $C09E on `DCP $3C20` ($CF) -- an undocumented opcode used as
ordinary code -- and filed 3,256 bytes of real code as `.byte` data.

Now 1,773 instructions / 3,836 bytes (23.4%), a 6.6x increase. Verified against a
fresh trace: 1,769 emitted instructions + 4 undocumented opcodes written as
explicit bytes = 1,773, an exact match. Byte accounting stays exact at
3,836 + 12,548 = 16,384.

ca65 V2.19 implements none of the undocumented opcodes, so the 4 affected
instructions (DCP $3C20, LAX $20D4, NOP $C1,X, RRA ($D0),X) are emitted as
explicit `.byte` with the disassembly in the comment. They stay in the code
region, so the split and the audit trail are both preserved.

Pass 2 (complete). All 202 labels renamed; zero `Label_Cxxxx` remaining. Renames
are keyed by address and verified to have bound to the intended address, not just
to some address. 202 routine headers, every one describing real behaviour -- no
"undetermined in Pass 1" boilerplate left. `locals.inc` carries 99 aliases, the
proven semantic ones sitting beside the generated `ZpXX` entries per the alias
pattern in AGENTS.md.

Two naming errors in the previous draft are corrected: $55 is MarkerValue (it
holds $70/$88/$A0 and is compared against them), not a "P1 Down timer"; and the
twelve MarkerVar variables are named by position because which player/button
each belongs to is not established.

Pass 3 is now outstanding, and the old Pass 3 was wrong: it described
$C09E-$C173 and $C17F-$CCF6 as data, and both are code. After the redo there is
exactly one data region, $CF06-$FFFF, confirmed by orphan scan.

Documentation.

The root inventory now lists all 36 projects in one table -- the 20 `_wip/`
projects are rows 17-36 rather than a second table that could drift. That second
table had wrong mirroring for 12 of them (Duck Hunt, Golf, Hogan's Alley, Ice
Climber, DK Jr Math, Lode Runner, Lunar Pool, Mario Bros, Millipede, Pyramid,
Raid [a2], Urban Champion were all recorded H when they are V, and vice versa);
every value is now read from the ROM rather than trusted.

Build status is split into two honest tiers: rows 1-16 are BYTE-IDENTICAL and
compare the whole ROM, rows 17-36 are PRG-IDENTICAL because their Makefiles only
run `cmp reconstructed_prg.bin prg.bin`. Header and CHR are not proven for those.
All 36 build cleanly; the WIP count was also corrected from 19 to 20.

`_tools/verify_pass1.py` checks all 36 and reports none truncated.

Mario Bros (JU): Pass 1 disassembly, 14.6% -> 72.2% code

The skeleton prg.s was a 12-line .incbin of the whole PRG. It is now a real
disassembly: 5,735 instructions, 11,829 code bytes (72.2%), 937 labels and
routine headers, byte accounting exact at 11,800 + 4,584 = 16,384, and
BYTE-IDENTICAL against prg.bin.

Coverage came from four steps, not one:

  vectors + direct jsr/jmp/branch, official opcodes   1,185 instr   14.6%
  --undocumented (105 extra NMOS opcodes)            1,210 instr   14.9%
  --auto-seed (61 table-referenced entries)          4,187 instr   51.1%
  --extra-seed 0xC0B6,0xC5A3                         5,735 instr   72.2%

Almost everything sits behind computed dispatch. Three mechanisms are now
documented in the project OVERVIEW:

* $CD9E is a stack-based indirect call: it pops the table address with `pla`,
  indexes it by A*2 with `lda ($14),y`, then `jmp ($0014)`. Because the pointer
  arrives on the stack it cannot be followed statically from the dispatcher.
* $C5A3 is a 20-entry table of `JSR` thunks ending in `rts`. Probing it linearly
  gives twenty valid targets, and seeding it recovers most of the last 21%.
* $C0B6 is the NMI phase dispatcher, reachable only by computed dispatch.

Two overlapping-decode bugs in _tools/nesdis.py had to be fixed first:

* Control-flow targets must win over fall-through. `bne` at $C4BA targets $C528,
  but the fall-through path had decoded $C527 as a 2-byte `NOP #$A5` whose
  operand swallowed $C528. No assembler branches into the middle of an
  instruction, so the spanning decode is now dropped in favour of the target;
  its bytes fall out as data, leaving byte accounting unchanged.
* Branch targets inside a surviving instruction must be pruned from the target
  set. Otherwise emit_code's linear walk consumes those bytes as an operand, so
  the label is referenced but never defined and the file does not assemble at
  all -- `Error: Symbol 'Label_C528' is undefined`.

Neither change disturbs an existing pipeline: MUSCLE's `make regen` still
reproduces prg.s byte-for-byte, Joypad Test Cartridge still builds byte-identical,
and _tools/verify_pass1.py reports all 16 top-level projects faithful with none
truncated.

Mario Bros (JU): Pass 2, partial -- make the call graph readable

Two mechanical wins, in order of value:

* **477 jsr/jmp operands rewritten from raw $XXXX to labels.** The tracer emits
  call and jump targets as hex, so the call graph was unreadable -- the most
  called routine appeared as `jsr $CCA0` 14 times over. Every one of those sites
  already had a label at its target address, so the substitution is purely
  mechanical and byte-preserving. Zero raw hex call operands remain.

* **13 landmark routines named** from verified instruction evidence:
  WaitVblank1, ClearRamPageLoop, NmiPhaseDispatch ($C0B6), JsrThunkTable
  ($C5A3), DispatchViaStackPointer ($CD9E), AdvanceDispatchPointer ($CDB4),
  PushTablePointerDispatch ($CCA0), DispatchTableF393 ($CAB9),
  InitSpritesOffscreen ($DFC4), AdjustDecimalColumns ($D15A),
  TestRamFlagPair0500 ($D328), NextIndex ($CE84).

**922 labels remain unnamed, and that is recorded as the honest state** rather
than papered over. They sit 15-25 per page across $CC00-$CFFF, which is just
the game logic: naming them means reading ~5,700 instructions across 935
routines. Guessing from the call graph and the NES convention would be exactly
the guessy rewrite PROMPTS.md rules out.

The productive next step is more reach, not more naming. The 4,555 unreached
bytes are where most unnamed routines live, and they are gated behind the
stack-pointer dispatch at $CD9E, whose pushed table addresses are only known in
the callers' frames.

Still BYTE-IDENTICAL; all 16 top-level projects remain faithful.

Record audit-comment work for Excitebike and Pac-Man as a TODO

Both projects are hand-written sources with semantic labels but no
byte-identity audit comment, so a source line cannot be mapped back to exact
ROM bytes without re-assembling. Every other project in the repo carries one.

Documented what was tried and why it was dropped rather than left implicit.
Addresses from 'ca65 -l' work fine -- the listing offset indexes prg.bin
directly -- but the listing carries no source line numbers, so entries cannot
be keyed back to source lines. Byte values also have to come from prg.bin
rather than the listing, since ca65 prints 'rr rr' for relocatable words.

No tool was committed, because the one written does not work. Both projects
verified still byte-identical; no source file was changed.

Add TODO.md handoff; bring AGENTS.md back in line with the repo

AGENTS.md had drifted on facts that are now wrong, not just incomplete:

* "Eleven disassemblies" -> 36 (16 top-level, 20 in _wip). The index it points
  at had the same stale count.
* The audit-comment convention was stated as `; PRG:$xxxx AA BB`, but the
  dominant spelling is `; CPU:$xxxx PRG:xxxx  AA BB` and five other variants are
  in use, with Excitebike and Pac-Man carrying none at all. Now stated accurately
  as "preserve whatever spelling a file uses", with the gap pointed at.
* `_tools/` was described as holding splitnes.py and chr2gif.py. It holds six
  tools now, including the Pass 1 tracer and the verifier.
* ROM filenames "contain spaces" -> spaces *and parentheses*, which is also what
  makes FCEUX hang headless.

Added, because each one costs real time if rediscovered:

* The two build tiers. The 20 _wip projects only `cmp reconstructed_prg.bin
  prg.bin`, so their header and CHR are NOT proven -- calling them
  byte-identical to the ROM is wrong.
* A promotion procedure, since that has now been done twice.
* A tracer/toolchain gotchas section: undocumented opcodes truncating traces
  (and ca65 being unable to encode them), vectors living at the end of the PRG
  rather than in the iNES header, the RESET==IRQ name collision, control-flow
  targets beating fall-through, the branch-target pruning that stops a file
  failing to assemble, NMI firing once per frame rather than per $2002 read,
  joy1 being $4016, and ca65 listings carrying no source line numbers.

TODO.md is the handoff document: open work per project with enough detail to
resume, the residue counts, and the reasoning behind each abandoned approach so
it is not silently retried.

Nothing rebuilt or changed; all projects still verify.

Joypad Test Cartridge (U): Pass 3 complete — data region named and VRAM command stream decoded

The monolithic `.byte $CF06-$FFFF` block (12,538 bytes) has been broken into 20 named sub-blocks:

VRAM command streams ($CF06-$D4AD): three command streams separated by $FF terminators, read by VramWriteLoop via RetAddrUpdate. Opcode set decoded: $4C = jump, $60 = deferred PPUCTRL, bit-7-clear = VRAM write (PPU addr + data, bit 6 of data = long/short selector).

OAM tables ($D4E3-$D6A3): 12 groups of 32-byte tables (8 entries of Y/tile/attr/X), read by CopyOamFromTable ($D4E3) and CopyOamFromTable70 ($D5C4).

Sprite templates ($D6A5-$D6E4): sprite template read by CopySpriteTemplateTable ($D6A5) and button OAM template read by PrepareButtonOam ($D6C5).

Padding: $FF ROM padding ($D6E5-$FFF9) and vectors ($FFFA-$FFFF).

OVERVIEW.md updated: section 8 documents all 20 sub-blocks with the VRAM command stream format and OAM table structure; section 11 marks Pass 3 complete with remaining work (MarkerVar player/button mapping, HandleButtonPress sprite order).

Build verified byte-identical.

Root OVERVIEW.md: mark Joypad Pass 3 complete

Update summary table (row 14: Pass Pass 2 → Pass 3 complete), detail section
(section 14: Pass 2/3 → Pass 3 complete with VRAM command stream decoded),
methodology note table (Pass 2 complete → Pass 3 complete), and notable
observations (Joypad at Pass 2/3 → Pass 3 complete).

Joypad Test Cartridge is now fully Pass 3 complete: all 20 data sub-blocks
named, VRAM command stream opcode set decoded, OAM table structure documented.

TODO.md: mark Joycad Test Cartridge Pass 3 complete

Mario Bros (JU): Pass 2 substantial — 252 subroutine entry points named

All 252 jsr/jmp targets resolved from raw $XXXX to labels:
- 90 semantic names: audio/sound routines ($F8xx-$F9xx: AudioPulse1CtrlSetup,
  WritePulse2Freq, ProcessTriangleChannel, SetupAudioChannel, etc.), gameplay
  helpers (CheckZpB3Return, CheckRangeReturn012, ToggleZpFC_Bit3, etc.)
- 165 address-based names (SubCxxxx) for the remainder
- Zero raw $XXXX call operands remain

667 Label_Cxxxx internal branch targets remain — these are local control-flow
destinations within routines, not subroutine entry points, and do not need
semantic names for the call graph to be readable.

Build verified BYTE-IDENTICAL. Root OVERVIEW.md updated (row 26: Pass 0 skeleton
→ Pass 1 complete, Pass 2 substantial). TODO.md updated.

Lunar Pool (U): Pass 1 complete, 57.8% code; nesdis.py opcode/emitter fixes

The skeleton prg.s was a 12-line .incbin of the whole PRG. It is now a real
disassembly: 4,586 instructions, 9,475 code bytes (57.8%), 6,909 data bytes,
byte accounting exact at 9,403 + 6,981 = 16,384, BYTE-IDENTICAL.

Coverage: vectors + direct jsr/jmp/branch reached 1,259 instructions (15.6%).
--undocumented took it to 4,572 (57.6%), and --auto-seed (48 table-referenced
entries) plus --extra-seed 0xE80C (the non-$C000 RESET vector) took it to
4,586 (57.8%).

Three undocumented opcodes were missing from nesdis.py and are added:
* $72 RRA abs (the RRA family was missing the $72 absolute form)
* $42 NOP #imm (the NOP-immediate family was missing $42)
* $FA NOP abs,X (the NOP-abs,X family was missing $FA)

One emitter fix: ca65 silently rewrites an operand whose high byte is $00 in
the shorter zero-page form -- `sta $00FF,x` assembles as `sta $FF,x` ($95
instead of $9D). render() now emits the `a:` prefix for ABS/ABSX/ABSY/IND
operands with a high byte of $00, forcing the absolute form so the emitted
bytes match the ROM. operand_value() strips the prefix when parsing.

These changes are general and verified not to disturb existing pipelines:
MUSCLE's `make regen` still reproduces prg.s byte-for-byte.

Remaining gaps: 6,909 bytes (42.2%) still data, gated behind jmp ($7961) at
three sites (RAM-pointer dispatch; the blocks after them hold the screen text
"strings" "game over" and "hi`score__") and stack-based dispatch at $CBDC --
the same pattern as Mario Bros (JU)'s $CD9E.

TODO.md: add Lunar Pool as an open item

Adds comments for all lines to Pacman and Excitebike
Moves Pacman 'locals' and 'symbols' definitions into the right places
This commit is contained in:
tmerritt committed 2026-10-17 17:36:28 -04:00
1 parent e9caa8cb36
commit 837007843d
50 files changed
+62939 -13111
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

_tools/add_audit_comments.py
+433
View File
@@ -0,0 +1,433 @@
#!/usr/bin/env python3
"""Add byte-identity audit comments to a project's prg.s.
Why this exists
---------------
Most projects in this repo carry a trailing audit comment on every
instruction, e.g.
lda #$00 ; CPU:$C000 PRG:0000 A9 00
That comment is the byte-identity audit trail: it ties a source line back to
exact ROM bytes. Two projects lack it (Pac-Man has none at all, Excitebike has
it on only ~19% of code lines), so a source line in those files cannot be
mapped to ROM bytes without re-assembling.
How it works
------------
`ca65 -l` produces a listing with one entry per *emitted* source line:
000186r 1 A9 00 lda #$00
^offset ^n ^bytes ^source text (echoed verbatim)
The listing has no source line numbers, but it does echo the source text, and
the entries appear in source order. So we recover the mapping by walking both
sequences and matching the echoed text, tolerating the lines ca65 skips.
Byte *values* are read from prg.bin at the listing offset, never from the
listing itself: ca65 prints `rr rr` for relocatable words (e.g. `.word Label`)
because the value is only known at link time. The count `n` and the token count
of the byte column give the length; prg.bin gives the truth.
Correctness
-----------
The script never edits instruction text, only appends a comment, so assembly is
unaffected. It then re-assembles and checks byte identity, and independently
verifies that every comment it wrote matches prg.bin at the offset it claims.
A disagreement is a hard error, not a warning.
Usage
-----
python3 _tools/add_audit_comments.py "Excitebike (JU)" # apply
python3 _tools/add_audit_comments.py "Excitebike (JU)" --dry-run
python3 _tools/add_audit_comments.py "Excitebike (JU)" --force # re-comment
"""
import argparse
import difflib
import os
import re
import shutil
import subprocess
import sys
import tempfile
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# <6-hex offset>r <count> <rest>
LISTING_RE = re.compile(r"^([0-9A-Fa-f]{6})r\s+(\d+)\s(.*)$")
# The byte column: space separated 2-hex tokens or `rr` for relocatable, then
# two-or-more spaces before the echoed source text.
BYTECOL_RE = re.compile(
r"^\s*((?:[0-9A-Fa-f]{2}|rr)(?: (?:[0-9A-Fa-f]{2}|rr))*)?\s\s+(.*)$"
)
# A line that already carries some spelling of audit comment. Several
# spellings are in use across the repo; all of them count as "already done".
HAS_AUDIT = re.compile(r";\s*(?:CPU:\$|PRG:|OFF:)|;\s*\$?[0-9A-Fa-f]{4}:\s")
# A source line that emits bytes.
EMITS_RE = re.compile(
r"^(?:[a-z]{3}\b|\.(?:byte|word|dword|addr|res|incbin)\b|"
r"(?:[A-Za-z_][\w]*)\s*[:=]\s*\.(?:byte|word|res|incbin)\b|\w+\s*=|=)"
)
def run(cmd, cwd):
p = subprocess.run(cmd, cwd=cwd, capture_output=True, text=True)
if p.returncode != 0:
sys.stderr.write(f"$ {' '.join(cmd)}\n{p.stdout}{p.stderr}\n")
raise SystemExit(f"command failed: {' '.join(cmd)}")
return p.stdout
def load_equates(*paths):
"""Map symbol name -> integer value from `NAME = $xx` lines."""
eq = {}
rx = re.compile(r"^\s*([A-Za-z_@][\w.]*)\s*=\s*\$?([0-9A-Fa-f]+)\s*(?:;.*)?$")
for p in paths:
if not os.path.exists(p):
continue
with open(p, errors="replace") as fh:
for line in fh:
m = rx.match(line)
if m:
try:
eq[m.group(1)] = int(m.group(2), 16)
except ValueError:
pass
return eq
def instruction_length(mnem, operand, equates=None):
"""Length in bytes of one 6502 instruction, from mnemonic + operand text.
Used only by the .org fallback path, where the ca65 listing cannot be
trusted for offsets. Addressing-mode table, official opcodes only: the
projects this fallback targets do not use undocumented opcodes in the
regions it annotates.
"""
equates = equates or {}
op = operand.lstrip()
imm = op.startswith("#")
indirect = op.startswith("(")
core = op.strip()
if core.startswith("("):
core = core[1:]
bits = core.split(",")
indexed_x = bits[-1].strip().lower() == "x" if len(bits) > 1 else False
indexed_y = bits[-1].strip().lower() == "y" if len(bits) > 1 else False
core = bits[0].strip()
# "(ZpFA),y" -> the closing paren sits on the pointer field, not the end.
if core.endswith(")"):
core = core[:-1].strip()
# A symbolic operand is zero page if its equate is < $100. Without the
# equate we cannot tell, so default to absolute (the 3-byte form), which
# is what an unresolved `ZpXX` name is in practice.
if re.fullmatch(r"[A-Za-z_@][\w.]*", core):
val = equates.get(core)
zp = val is not None and val < 0x100
abs_ = not zp
else:
zp = re.fullmatch(r"\$[0-9A-Fa-f]{1,2}", core) is not None
abs_ = re.fullmatch(r"\$[0-9A-Fa-f]{3,4}", core) is not None
table = {
# implied
"asl": 1, "lsr": 1, "rol": 1, "ror": 1, "bcc": 2, "bcs": 2, "beq": 2,
"bmi": 2, "bne": 2, "bpl": 2, "bvc": 2, "bvs": 2, "clc": 1, "cld": 1,
"cli": 1, "clv": 1, "dex": 1, "dey": 1, "inx": 1, "iny": 1, "nop": 1,
"pha": 1, "php": 1, "pla": 1, "plp": 1, "rti": 1, "rts": 1, "sec": 1,
"sed": 1, "sei": 1, "tax": 1, "tay": 1, "tsx": 1, "txa": 1, "txs": 1,
"tya": 1,
# accumulator / implied-by-A
"adc": None, "and": None, "cmp": None, "eor": None, "lda": None,
"ora": None, "sbc": None, "sta": None,
# branches
"jmp": None, "jsr": 3,
# stack
"ldx": None, "ldy": None, "stx": None, "sty": None,
}
if mnem in ("jmp",):
return 3 if not indirect else 5
if mnem == "jsr":
return 3
base = table.get(mnem, None)
if mnem in ("adc", "and", "cmp", "eor", "lda", "ora", "sbc", "sta"):
if imm:
return 2
if indirect:
# (zp,x) is 2 bytes; (zp),y is 2; (abs),y is 6; (abs) is 5.
if indexed_x:
return 2
if indexed_y:
return 6 if abs_ else 2
return 6 if abs_ else 5
if indexed_x or indexed_y:
return 2 if zp else 3
return 2 if zp else 3
if mnem in ("ldx", "ldy"):
if imm:
return 2
if indexed_y if mnem == "ldx" else indexed_x:
return 2 if zp else 3
if indexed_x or indexed_y:
return 2 if zp else 3
return 2 if zp else 3
if mnem in ("stx", "sty"):
if indexed_x or indexed_y:
return 2 if zp else 3
return 2 if zp else 3
return base if base else 2
def offsets_from_labels(src_lines, lbl_path, base, cpu_base):
"""Fallback for sources using `.org`, where ca65's listing offsets restart.
The linker's .lbl file gives every label's exact address. We align labels to
their defining source line, then walk forward accumulating instruction
lengths to recover each emitting line's PRG offset.
"""
labels = {}
with open(lbl_path, errors="replace") as fh:
for line in fh:
parts = line.split()
if len(parts) >= 3 and parts[0] == "al":
addr = int(parts[1], 16)
name = parts[2].lstrip(".")
labels[name] = addr
label_re = re.compile(r"^([A-Za-z_@][\w.]*)\s*:")
mapping = {}
unresolved = []
for i, raw in enumerate(src_lines):
s = raw.strip()
if not s or s.startswith(";"):
continue
m = label_re.match(s)
if not m:
continue
name = m.group(1)
if name.startswith("@"):
continue # cheap local labels are not exported to the .lbl file
if name in labels:
mapping[i] = labels[name] - cpu_base
else:
unresolved.append(name)
return mapping, unresolved
def propagate_offsets(src_lines, anchors, equates=None):
"""Fill in PRG offsets for emitting lines between known label anchors.
`anchors` maps source line index -> PRG offset for label definitions. We
walk forward from each anchor, adding the length of every emitting line,
until the next anchor. Data directives (.byte/.word/.res) are counted too.
"""
data_re = re.compile(r"^\.(?:byte|word|dword|addr|res|incbin)\b")
mnem_re = re.compile(r"^([a-z]{3})\b\s*(.*)$")
lengths = {}
idxs = sorted(anchors)
for n, start in enumerate(idxs):
off = anchors[start]
i = start + 1
stop = idxs[n + 1] if n + 1 < len(idxs) else len(src_lines)
while i < stop:
s = src_lines[i].strip()
if not s or s.startswith(";"):
i += 1
continue
if data_re.match(s):
parts = s.split(None, 1)
d = parts[0].lower()
if d == ".byte":
nbytes = len([x for x in parts[1].split(",") if x.strip()]) if len(parts) > 1 else 0
elif d in (".word", ".addr"):
nbytes = 2 * (len([x for x in parts[1].split(",") if x.strip()]) if len(parts) > 1 else 0)
elif d == ".res":
mm = re.search(r"\$?([0-9A-Fa-f]+)", parts[1]) if len(parts) > 1 else None
nbytes = int(mm.group(1), 16) if mm else 0
else: # .incbin
nbytes = 0
anchors[i] = off
lengths[i] = nbytes
off += nbytes
i += 1
continue
mm = mnem_re.match(s)
if mm and not s.endswith("=") and ":" not in s.split()[0]:
nbytes = instruction_length(mm.group(1), mm.group(2), equates)
anchors[i] = off
lengths[i] = nbytes
off += nbytes
i += 1
return lengths
def parse_listing(path):
"""Return [(prg_offset, n_tokens_or_None, echoed_text)] in listing order."""
recs = []
with open(path, errors="replace") as fh:
for line in fh:
m = LISTING_RE.match(line.rstrip("\n"))
if not m:
continue
rest = m.group(3)
bm = BYTECOL_RE.match(rest)
if bm and bm.group(1):
tokens = bm.group(1).split()
text = bm.group(2).rstrip()
recs.append((int(m.group(1), 16), len(tokens), text))
else:
# No byte column: a comment, equate, directive or label. The
# echoed text is whatever follows the count.
recs.append((int(m.group(1), 16), None, rest.rstrip()))
return recs
def align(src_lines, recs):
"""Map source line index -> (prg_offset, nbytes) using echoed text.
ca65 skips some source lines (blank lines, continuation lines, macro
expansions) and emits entries for others only once, so this has to be a
real sequence alignment rather than a lockstep zip. We use difflib's
opcodes: the listing and the source should otherwise be identical, so the
few genuine insertions/deletions are cheap to find and everything else
lines up exactly.
Returns (mapping, n_unmatched).
"""
src_norm = [l.strip() for l in src_lines]
rec_norm = [t.strip() for _, _, t in recs]
matcher = difflib.SequenceMatcher(a=src_norm, b=rec_norm, autojunk=False)
mapping = {}
unmatched = 0
for tag, i1, i2, j1, j2 in matcher.get_opcodes():
if tag == "equal":
for k in range(i2 - i1):
mapping[i1 + k] = (recs[j1 + k][0], recs[j1 + k][1])
else:
# 'replace' can still line up when the two sequences are the same
# length but differ in text; 'delete'/'insert' do not.
if tag == "replace" and (i2 - i1) == (j2 - j1):
for k in range(i2 - i1):
mapping[i1 + k] = (recs[j1 + k][0], recs[j1 + k][1])
else:
unmatched += (j2 - j1)
return mapping, unmatched
def audit_comment(prg, off, nbytes, base, cpu_base):
"""Build `; CPU:$xxxx PRG:xxxx AA BB` from prg.bin, or None."""
if nbytes is None or nbytes == 0:
return None
if off + nbytes > len(prg):
return None
vals = prg[off:off + nbytes]
hexs = " ".join(f"{b:02X}" for b in vals)
return f"; CPU:${cpu_base + off:04X} PRG:{off:04X} {hexs}"
def main():
ap = argparse.ArgumentParser()
ap.add_argument("project")
ap.add_argument("--dry-run", action="store_true")
ap.add_argument("--force", action="store_true",
help="re-comment lines that already have an audit comment")
ap.add_argument("--prg-name", default="prg.bin")
ap.add_argument("--base", default="0xC000",
help="CPU address corresponding to PRG offset 0")
args = ap.parse_args()
cpu_base = int(args.base, 0)
copilot = os.path.join(REPO, args.project, "copilot")
if not os.path.isdir(copilot):
raise SystemExit(f"no such project dir: {copilot}")
prg_path = os.path.join(copilot, args.prg_name)
src_path = os.path.join(copilot, "prg.s")
prg = open(prg_path, "rb").read()
src_lines = open(src_path, errors="replace").read().splitlines()
equates = load_equates(
os.path.join(copilot, "symbols.inc"),
os.path.join(copilot, "locals.inc"),
src_path,
)
with tempfile.TemporaryDirectory() as td:
# ca65 uses the -l name verbatim (no .lst suffix appended).
lst = os.path.join(td, "listing")
run(["ca65", "-g", "-l", lst, "-o", os.path.join(td, "prg.o"), "prg.s"], copilot)
# Parse inside the context: the listing is deleted on exit.
recs = parse_listing(lst)
mapping, unmatched = align(src_lines, recs)
# A source using `.org` makes ca65's listing offsets restart at zero, so
# the listing carries no usable addresses. Detect that and fall back to
# deriving offsets from the linker's label file.
if recs and max((r[0] for r in recs), default=0) < 0x100 and len(mapping) > 0:
with tempfile.TemporaryDirectory() as td:
run(["ld65", "-C", "linker.cfg", "-m", os.path.join(td, "m.map"),
"-Ln", os.path.join(td, "m.lbl"), "-o", os.path.join(td, "out.nes"),
*sorted(f for f in os.listdir(copilot)
if f.endswith(".o") and f != "reconstructed.o")], copilot)
lbl = os.path.join(td, "m.lbl")
if os.path.exists(lbl):
print("note: source uses `.org`; deriving offsets from label file")
anchors, unresolved = offsets_from_labels(src_lines, lbl, 0, cpu_base)
unmatched = len(unresolved)
lengths = propagate_offsets(src_lines, anchors, equates)
mapping = {i: (anchors[i], lengths.get(i)) for i in anchors}
out = list(src_lines)
added = skipped = failed = 0
for idx, (off, nbytes) in mapping.items():
line = src_lines[idx]
s = line.strip()
if not s or s.startswith(";"):
continue
if not EMITS_RE.match(s):
continue
if not args.force and HAS_AUDIT.search(line):
skipped += 1
continue
cmt = audit_comment(prg, off, nbytes, cpu_base, cpu_base)
if cmt is None:
continue
out[idx] = line.rstrip() + (" " if line.strip() else " ") + cmt
added += 1
# Verify every comment we are about to write against prg.bin.
bad = 0
for line in out:
m = re.search(r";\s*CPU:\$([0-9A-F]{4})\s+PRG:([0-9A-F]{4})\s+((?:[0-9A-F]{2} ?)+)$", line)
if not m:
continue
off = int(m.group(2), 16)
vals = bytes(int(x, 16) for x in m.group(3).split())
if prg[off:off + len(vals)] != vals:
bad += 1
if bad <= 5:
sys.stderr.write(f"BYTE MISMATCH PRG {off:04X}: comment {m.group(3)} "
f"vs prg.bin {prg[off:off+len(vals)].hex(' ').upper()}\n")
if bad:
raise SystemExit(f"{bad} comments disagree with {args.prg_name}; refusing to write")
print(f"{args.project}: {added} comments added, {skipped} already present, "
f"{unmatched} listing entries unmatched")
if args.dry_run:
return
shutil.copyfile(src_path, src_path + ".preaudit")
with open(src_path, "w") as fh:
fh.write("\n".join(out) + "\n")
print(f"wrote {src_path} (backup at {src_path}.preaudit)")
if __name__ == "__main__":
main()
_tools/bytecode_dis.py
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""
MUSCLE (U) bytecode-VM disassembly generator.
Emits a decoded listing for every BytecodeProgram / StandaloneBytecodeProgram and
patches prg.s so each program carries a readable token-by-token listing above its
unchanged .byte data. No byte is altered: the .byte lines are copied verbatim and
only comment lines are inserted.
Encoding (derived from RunBytecodeInterpreter $9AE8 and BytecodeHandlerTable $9237):
* byte in $C8..$EF -> opcode, handler = table[byte - $C8]
* any other byte -> literal operand, pushed to $0033[Y] and accumulated in $003C[Y]
* opcodes $CB $D1 $D7 $E1 additionally consume the NEXT stream byte inline
(their handlers execute `lda ($1A),Y / iny`)
"""
import sys, re
ROM = open(sys.argv[1], 'rb').read()
PRG_S = sys.argv[2]
BASE = 0x8000
PROG_TABLE = 0x8CEF
HANDLER_TABLE = 0x9237
INTERP = 0x9AE8
HANDLERS = []
for i in range(40):
o = HANDLER_TABLE - BASE + i * 2
HANDLERS.append(ROM[o] | (ROM[o + 1] << 8))
TABLE = []
for i in range(38):
o = PROG_TABLE - BASE + i * 2
TABLE.append(ROM[o] | (ROM[o + 1] << 8))
# opcodes that consume one inline stream byte (verified against handler entry points)
INLINE = {0xCB, 0xD1, 0xD7, 0xE1}
# mnemonic + role, read off each handler's own straight-line path
# (up to its first `jmp $9AE8` / `jmp $9B25`). Confidence is called out where a
# handler simply tail-calls a game routine.
OPS = {
0xC8: ('VM_CALL_82E1', 'tail-calls the game routine at $82E1'),
0xC9: ('LOOP_END', 'decrements the $7C loop counter; if still non-zero rewinds the '
'stream pointer from $7E, re-running the loop body'),
0xCA: ('VM_CALL_8765', 'tail-calls the game routine at $8765'),
0xCB: ('LOOP_BEGIN', 'reads the inline iteration count, stores it in $7C and the '
'body address in $7E'),
0xCC: ('SET_STEP_12', 'sets the step byte $4B to $12'),
0xCD: ('ACC_ADD_1_STEP', 'SaveStreamPos; adds 1 to the accumulator via $9653, then '
'folds the step byte into $0033[Y]'),
0xCE: ('WAIT_FRAME_SLOT', 'advance-two-frames path: $4F-=1, $A4FF, default $F0=$80, '
'and kick a $D1=$FF sound sequence'),
0xCF: ('SUB_TEN', '$4F-=10 and $F0=$D0'),
0xD0: ('RESET_ACCUM', 'clears $F0 and zeroes the low operand bytes $0033[Y]/$0034[Y], '
'then starts the next actor'),
0xD1: ('SET_RATE', 'stores the inline byte into $43 as this actor\'s rate'),
0xD2: ('MOVE_DIAG', 'position update: $4F+=2, $9A3F, a $A660 helper call and two '
'$9A2E operand pushes'),
0xD3: ('MOVE_OTHER', 'position update variant: two $9A24/$9A2E pushes and an $A660 '
'helper call'),
0xD4: ('SET_STEP_2', 'SaveStreamPos, step byte $4B:=2 and add 1 to the accumulator'),
0xD5: ('STEP_ACCUM', 'subtracts $4B>>1 from the accumulator with $FD/$06 saturation '
'clamping'),
0xD6: ('TICK_STEP', 'step countdown: on reaching zero sets $0321=$FF'),
0xD7: ('DRAW_SPRITE', 'uses the inline byte as a count for the $A660 sprite/blit helper'),
0xD8: ('ANIMATE_CHECK', 'animation-frame test via $97FC/$A4DF, then a $99E2 call'),
0xD9: ('ANIMATE_ADVANCE', 'animation-frame advance: $4F+=2, $5A:=$FF, $A660 call'),
0xDA: ('SET_DIRECTION', 'sets the $5C direction flag from bit 0 of $24'),
0xDB: ('PLAY_SEQ_BD20', 'runs sound sequence $BD20 through the $99E4 dispatcher'),
0xDC: ('TIMER_FF', 'decrements the $60 timer; on zero jumps to $94CF, otherwise adds '
'$FF (-1) to the accumulator'),
0xDD: ('TIMER_01', 'decrements the $60 timer; clears $62 and adds 1 to the accumulator'),
0xDE: ('MOVE_ACTOR_B0BA', 'accumulator-indexed $A660 move plus a $99E4 call with '
'descriptor $B0BA'),
0xDF: ('MOVE_ACTOR_DIR', 'direction-dependent position update over $F6/$F7'),
0xE0: ('SET_FLAG_FF', 'stores $FF into $82'),
0xE1: ('SET_WAIT', 'stores the inline byte into $7A as this actor\'s wait/step count'),
0xE2: ('COND_FA', 'tests the $5A state register and the $4E flag before branching '
'through $96C8'),
0xE3: ('COND_FB', 'the $FB half of the $96C8 condition test'),
0xE4: ('ACCUM_PLUS_6', 'SaveStreamPos and adds 6 to the accumulator $3A[Y]'),
0xE5: ('TICK_WAIT_STEP', 'decrements the $7A wait counter; runs the body while '
'non-zero'),
0xE6: ('MOVE_ACTOR_ALT', 'alternate position update: $9585 test, $9A3F and two '
'operand pushes'),
0xE7: ('SWAP_OPERANDS', 'exchanges the $39[Y]/$3A[Y] operand pair around the $A4FF '
'and $A4F2 helpers'),
0xE8: ('STEP_DOWN', 'SaveStreamPos, $9732, step $4B:=2 and accumulate'),
0xE9: ('STEP_DOWN_COND', 'as $E8 but gated on the $5A state sign bit'),
0xEA: ('SET_ACTIVE_80', 'sets $F0 to $80 and falls into the $94CF accumulator clear'),
0xEB: ('CLEAR_ACTIVE', 'clears $F0 when it currently holds $80'),
0xEC: ('ADD_STEP_ACCUM', 'adds the step byte into $0033[Y], steps $3A[Y] down and '
'accumulates'),
0xED: ('SUB_STEP_ACCUM', 'adds 1 to $3A[Y] and subtracts the step from $0033[Y]'),
0xEE: ('ROT_STEP_ADD', 'rotates the step byte, adds it into $0033[Y], steps $3A[Y] '
'up by 2 and accumulates'),
0xEF: ('ROT_STEP_SUB', 'rotates the step byte, complements it, subtracts from '
'$0033[Y], and calls $82ED'),
}
LIT_NOTE = {
0xB8: 'VRAM run fill selector (VramRunFillValues[0..11] span $B8-$C3)',
}
def rd(off):
"""Read one PRG byte by 16-bit PRG offset."""
return ROM[off]
def cpu(off):
return BASE + off
def decode(a, b):
"""Yield (addr, opcode_or_None, operand_or_None, raw_bytes)."""
x = a
while x < b:
v = rd(x)
if 0xC8 <= v <= 0xEF:
if v in INLINE and x + 1 < b:
yield (x, v, rd(x + 1), 2)
x += 2
continue
yield (x, v, None, 1)
x += 1
else:
yield (x, None, None, 1)
x += 1
def listing(a, b, indent=' '):
out = []
cur = None
nops = nlit = 0
for addr, op, arg, ln in decode(a, b):
if op is not None:
nops += 1
name, role = OPS[op]
if arg is not None:
txt = f'{name} ${arg:02X}'
else:
txt = name
body = f'${cpu(addr):04X} {txt:<28} -- {role}'
else:
nlit += 1
v = rd(addr)
note = LIT_NOTE.get(v, '')
body = f'${cpu(addr):04X} .byte ${v:02X}' + (f' -- {note}' if note else '')
out.append(indent + '; ' + body)
return out, nops, nlit
def main():
lines = open(PRG_S).read().split('\n')
uniq = sorted(set(TABLE))
stats = {}
out = []
i = 0
N = len(lines)
LAB = re.compile(r'^(BytecodeProgram_[0-9A-F]{2}|StandaloneBytecodeProgram(?:_[0-9])?):\s*$')
DEC = re.compile(r'^\s*\.byte\s+\$([0-9A-F]{2})')
while i < N:
ln = lines[i]
m = LAB.match(ln)
if not m:
out.append(ln)
i += 1
continue
label = m.group(1)
# collect the contiguous .byte run belonging to this program
j = i + 1
body = []
while j < N and DEC.match(lines[j]):
for t in lines[j].split(';')[0].split(','):
t = t.strip()
if t.lower().startswith('.byte'):
t = t[5:]
t = t.strip().replace('$', '')
if t:
body.append(int(t, 16))
j += 1
# program start = PRG offset from the first data line's annotation
m2 = re.search(r'PRG:([0-9A-F]{4})', lines[i + 1])
a = int(m2.group(1), 16)
b = a + len(body)
dec, nops, nlit = listing(a, b)
idx = [k for k, t in enumerate(TABLE) if t == a]
hdr = [f'; --- bytecode listing: {len(body)} bytes, {nops} opcodes, '
f'{nlit} literal operands ---']
if idx:
hdr.append(f'; reached by BytecodeProgramPointerTable entry/entries {idx}')
hdr.append(f'; handler for each opcode: BytecodeHandlerTable at $9237 '
f'(token - $C8); interpreter RunBytecodeInterpreter at $9AE8')
if nops == 0:
hdr.append('; NOTE: this program contains no byte in $C8-$EF, so the')
hdr.append('; interpreter takes the literal path for every byte and the')
hdr.append('; whole block is operand data, not an instruction stream.')
out.append(ln)
out.extend(hdr)
out.extend(dec)
stats[label] = (nops, nlit)
out.extend(lines[i + 1:j])
i = j
open(PRG_S, 'w').write('\n'.join(out))
to = sum(v[0] for v in stats.values())
tl = sum(v[1] for v in stats.values())
print(f'decoded {len(stats)} programs: {to} opcodes, {tl} literal operands')
for k in sorted(stats):
print(f' {k:<28} ops={stats[k][0]:>3} lits={stats[k][1]:>3}')
if __name__ == '__main__':
main()
_tools/nesdis.py
+156 -11
View File
@@ -66,9 +66,94 @@ OPCODES = {
MODE_LEN = {IMP: 1, ACC: 1, IMM: 2, ZP: 2, ZPX: 2, ZPY: 2, REL: 2,
ABS: 3, ABSX: 3, ABSY: 3, IND: 3, INDX: 2, INDY: 2}
# ---------------------------------------------------------------------------
# Undocumented NMOS opcodes.
#
# Real 6502 code uses these as multi-byte NOP padding and as genuine
# read-modify-write / accumulator ops, so a tracer that only knows the 151
# official opcodes halts the moment it steps on one. Wild Gunman and the Joypad
# Test Cartridge both stop this way.
#
# OFF by default (`--undocumented` turns it on) so that projects whose prg.s is
# already regen-stable keep reproducing byte-for-byte.
# ---------------------------------------------------------------------------
UNDOC_OPCODES = {
# NOP padding, implied (1 byte)
0x1A: ("nop", IMP), 0x3A: ("nop", IMP), 0x5A: ("nop", IMP),
0x7A: ("nop", IMP), 0xDA: ("nop", IMP), 0xFA: ("nop", IMP),
# NOP padding, immediate (2 bytes)
0x42: ("nop", IMM), 0x80: ("nop", IMM), 0x82: ("nop", IMM), 0x89: ("nop", IMM),
0xC2: ("nop", IMM), 0xE2: ("nop", IMM),
# NOP padding, zero page (2 bytes)
0x04: ("nop", ZP), 0x44: ("nop", ZP), 0x64: ("nop", ZP),
# NOP padding, zero page,X (2 bytes)
0x14: ("nop", ZPX), 0x34: ("nop", ZPX), 0x54: ("nop", ZPX),
0x74: ("nop", ZPX), 0xD4: ("nop", ZPX), 0xF4: ("nop", ZPX),
# NOP padding, absolute (3 bytes)
0x0C: ("nop", ABS),
# NOP padding, absolute,X (3 bytes)
0x1C: ("nop", ABSX), 0x3C: ("nop", ABSX), 0x5C: ("nop", ABSX),
0x7C: ("nop", ABSX), 0xDC: ("nop", ABSX), 0xFC: ("nop", ABSX),
0xFA: ("nop", ABSX),
# SLO = ASL + ORA
0x03: ("slo", INDX), 0x07: ("slo", ZP), 0x0F: ("slo", ABS),
0x13: ("slo", INDY), 0x17: ("slo", ZPY), 0x1B: ("slo", ABSY),
0x1F: ("slo", ABSX),
# RLA = ROL + AND
0x23: ("rla", INDX), 0x27: ("rla", ZP), 0x2F: ("rla", ABS),
0x33: ("rla", INDY), 0x37: ("rla", ZPY), 0x3B: ("rla", ABSY),
0x3F: ("rla", ABSX),
# SRE = LSR + EOR
0x43: ("sre", INDX), 0x47: ("sre", ZP), 0x4F: ("sre", ABS),
0x53: ("sre", INDY), 0x57: ("sre", ZPY), 0x5B: ("sre", ABSY),
0x5F: ("sre", ABSX),
# RRA = ROR + ADC
0x63: ("rra", INDX), 0x67: ("rra", ZP), 0x6F: ("rra", ABS),
0x72: ("rra", ABS), 0x73: ("rra", INDY), 0x77: ("rra", ZPY), 0x7B: ("rra", ABSY),
0x7F: ("rra", ABSX),
# SAX = store A & X
0x83: ("sax", INDX), 0x87: ("sax", ZP), 0x8F: ("sax", ABS),
0x97: ("sax", ZPY),
# LAX = load A and X
0xA3: ("lax", INDX), 0xA7: ("lax", ZP), 0xAB: ("lax", IMM),
0xAF: ("lax", ABS), 0xB3: ("lax", INDY), 0xB7: ("lax", ZPY),
0xBF: ("lax", ABSY),
# DCP = DEC then CMP
0xC3: ("dcp", INDX), 0xC7: ("dcp", ZP), 0xCF: ("dcp", ABS),
0xD3: ("dcp", INDY), 0xD7: ("dcp", ZPY), 0xDB: ("dcp", ABSY),
0xDF: ("dcp", ABSX),
# ISC = INC then SBC
0xE3: ("isc", INDX), 0xE7: ("isc", ZP), 0xEF: ("isc", ABS),
0xF3: ("isc", INDY), 0xF7: ("isc", ZPY), 0xFB: ("isc", ABSY),
0xFF: ("isc", ABSX),
# Unstable/shift-combination opcodes
0x0B: ("anc", IMM), 0x2B: ("anc", IMM), 0x4B: ("alr", IMM),
0x6B: ("arr", IMM), 0x8B: ("xaa", IMM), 0xAB: ("lax", IMM),
0xCB: ("axs", IMM), 0xEB: ("sbc", IMM), 0xBB: ("las", ABSY),
0x93: ("sha", INDY), 0x9F: ("sha", ABSY), 0x9B: ("tas", ABSY),
}
_TABLE = [OPCODES]
def set_undocumented(on):
"""Enable/disable the undocumented opcode table. Default: off."""
_TABLE[0] = ({**OPCODES, **UNDOC_OPCODES} if on else OPCODES)
# Instructions after which control never falls through.
TERMINATORS = {"jmp", "rts", "rti", "brk"}
# Mnemonics ca65 cannot encode. V2.19 implements none of the undocumented
# opcodes, so these must be emitted as explicit `.byte` values instead.
CA65_UNSUPPORTED = {
"slo", "rla", "sre", "rra", "sax", "lax", "dcp", "isc",
"anc", "alr", "arr", "axs", "xaa", "las", "tas", "sha", "shx", "shy",
}
# `nop` is officially supported (0xEA) but not with an operand.
CA65_UNSUPPORTED_MODE = {"nop"}
BRANCHES = {"bpl", "bmi", "bvc", "bvs", "bcc", "bcs", "bne", "beq"}
@@ -94,7 +179,7 @@ def render(prg, base, opcode, pc):
"""Decode one instruction at pc. Returns an Insn or None when illegal."""
if pc >= len(prg):
return None
entry = OPCODES.get(prg[pc])
entry = _TABLE[0].get(prg[pc])
if entry is None:
return None
mnemonic, mode = entry
@@ -122,13 +207,16 @@ def render(prg, base, opcode, pc):
elif mode == ZPY:
insn.text = f"Zp{b1:02X},y"
elif mode == ABS:
insn.text = hex16(b1 | (b2 << 8))
insn.text = f"{'a:' if b2 == 0 else ''}{hex16(b1 | (b2 << 8))}"
elif mode == ABSX:
insn.text = f"{hex16(b1 | (b2 << 8))},x"
# ca65 silently rewrites `,x` on a $00xx operand as zero page,X, which
# emits a different opcode. The `a:` prefix forces the absolute,X form
# so the bytes match the ROM.
insn.text = f"{'a:' if b2 == 0 else ''}{hex16(b1 | (b2 << 8))},x"
elif mode == ABSY:
insn.text = f"{hex16(b1 | (b2 << 8))},y"
insn.text = f"{'a:' if b2 == 0 else ''}{hex16(b1 | (b2 << 8))},y"
elif mode == IND:
insn.text = f"({hex16(b1 | (b2 << 8))})"
insn.text = f"({'a:' if b2 == 0 else ''}{hex16(b1 | (b2 << 8))})"
elif mode == INDX:
insn.text = f"(Zp{b1:02X},x)"
elif mode == INDY:
@@ -261,7 +349,8 @@ def operand_value(insn):
"""Absolute address of an ABS/ABSX/ABSY/IND operand, or None."""
if insn.mode not in (ABS, ABSX, ABSY, IND):
return None
token = insn.text.strip("()").split(",")[0].lstrip("$")
token = insn.text.strip("()").split(",")[0]
token = token.removeprefix("a:").lstrip("$") # ca65 absolute-forcing prefix
return int(token, 16)
@@ -444,9 +533,38 @@ def emit_code(prg, base, insns, targets, out, project, semantic, refs=None):
for addr, name in semantic.items():
name_of[addr] = name
# A branch target must win over fall-through. If some instruction's operand
# spans an address that another path branches to, the spanning decode is the
# wrong one: real assemblers would never branch into the middle of an
# instruction. Drop the span and keep the target. Its bytes fall out as data,
# so byte accounting is unaffected.
tstart = {t - base for t in targets if (t - base) in insns}
changed = True
while changed:
changed = False
for k in sorted(insns):
if k in tstart:
continue
span = insns[k].length
if any(j in tstart for j in range(k + 1, k + span)):
del insns[k]
changed = True
break
# A branch target that still falls strictly inside a surviving instruction
# can never be emitted: the linear walk consumes those bytes as an operand,
# so the label would be referenced but never defined and the file would not
# assemble. Prune those targets up front.
valid_starts = set(insns)
for k, ins in insns.items():
for j in range(k + 1, k + ins.length):
valid_starts.discard(j)
targets = {t for t in targets if (t - base) in valid_starts}
i = 0
routine_header = [
"; -----------------------------------------------------------------------------",
"; Routine: {name}",
"; Global: Contextual control-flow entry identified from its parent routine and incoming branch usage.",
"; Local: Continues its parent routine along the condition or operation encoded in this label,",
@@ -468,9 +586,28 @@ def emit_code(prg, base, insns, targets, out, project, semantic, refs=None):
out.append(line.format(name=name))
out.append(f"{name}:")
text = f"{insn.mnemonic} {insn.text}".strip()
if insn.mode == REL and insn.target is not None:
# render() stamps every branch as Label_XXXX; prefer the semantic
# name so a branch into a vector entry (or any named routine)
# references the label that is actually emitted here.
text = f"{insn.mnemonic} " + (
name_of.get(insn.target) or f"Label_{insn.target:04X}")
raw = " ".join(f"{b:02X}" for b in prg[i:i + insn.length])
line = f" {text}"
out.append(f"{line:<31}; CPU:${cpu:04X} PRG:{i:04X} {raw}")
# `nop` is only a problem in its multi-byte padding forms; the
# official 0xEA implied NOP encodes fine.
unencodable = (insn.mnemonic in CA65_UNSUPPORTED
or (insn.mnemonic in CA65_UNSUPPORTED_MODE
and insn.mode != IMP))
if unencodable:
# ca65 (V2.19) implements no undocumented opcodes, so these cannot be
# written as mnemonics. Keep them in the code region as explicit bytes
# and record the full disassembly in the comment, so the code/data
# split stays correct and the bytes stay auditable.
line = f" .byte {byte_columns(prg[i:i + insn.length])}"
out.append(f"{line:<31}; CPU:${cpu:04X} PRG:{i:04X} {raw}"
f" -- {text} (undocumented; ca65 cannot encode)")
else:
out.append(f" {text:<31}; CPU:${cpu:04X} PRG:{i:04X} {raw}")
i += insn.length
continue
@@ -525,7 +662,13 @@ def main():
"static fall-through reached it (branch-over-data)")
ap.add_argument("--probe", default="",
help="comma list of CPU addresses to decode linearly for inspection")
ap.add_argument("--undocumented", action="store_true",
help="also decode the 105 undocumented NMOS opcodes (NOP padding, "
"SLO/RLA/SRE/RRA/SAX/LAX/DCP/ISC and friends). Games use these as "
"padding, and a tracer that halts on them loses most of the code. "
"Default off so existing regen pipelines stay byte-stable.")
args = ap.parse_args()
set_undocumented(args.undocumented)
base = int(args.base, 16)
prg = Path(args.prg).read_bytes()
@@ -577,9 +720,11 @@ def main():
invalid = [a for a in invalid if not any(a0 <= a < a0 + n for a0, n in forced)]
semantic = OrderedDict()
semantic[rst] = "Reset"
semantic[nmi] = "NmiHandler"
semantic[irq] = "IrqHandler"
# Priority order matters: RESET and IRQ are commonly the same address, and
# overwriting would rename the primary entry point to `IrqHandler`.
semantic.setdefault(rst, "Reset")
semantic.setdefault(nmi, "NmiHandler")
semantic.setdefault(irq, "IrqHandler")
regions = scan_orphans(prg, base, insns, 1)
refs = build_refs(prg, base, insns, regions)
_tools/trace_stage_payload.py
+280
View File
@@ -0,0 +1,280 @@
#!/usr/bin/env python3
"""
MUSCLE (U) stage-payload read tracer (py65 + minimal cycle-driven NES hardware).
The earlier instruction-counted frame cadence was wrong: the main loop polls
$2002 for vblank and the NMI handler fires at the same scanline, so both have to
come off the same clock. This drives a real scanline counter off py65's
processorCycles.
* $2002 - vblank asserted at scanline 241, cleared on read
* NMI - fired once per frame at scanline 241 (NmiHandler $86CB)
* $4014 - 256-byte OAM DMA
* $2006/$2005/$2007 - enough address-latch behaviour for the code to run
* $4016 - joy1, real latch/shift register (see NMI BUG below)
* $4017 - expansion port, reads back 0
Every read of $B0FB-$BEBF is logged as (PC, address, value) plus the bytecode
token in flight.
NMI BUG (fixed here; worth knowing if you re-derive this)
---------------------------------------------------------
The vblank flag must be cleared once per frame, NOT merely on a read of $2002.
The main loop polls $2002 (`MainGameLoop_OnZero_8226` spins on the bytecode VM
flag $0321), and every such read clears bit 7 -- which re-arms NMI within the
same scanline. Guarding only on `not st['vblank']` produced 40,900 NMIs across
3,000 frames and silently corrupted all game state, flattening coverage at 26%.
A separate per-frame `nmi_fired` flag restores exactly one NMI per frame.
CONTROLLER
----------
Input goes on joy1 = $4016 with a real latch/shift register. The poll routine
at $AA69 is `ldx #$01 / stx $4016 / dex / stx $4016 / ldy #$08 / lda $4016,X`,
so the *observed* reads are of $4017 (X=1) and only bits 0-1 are compared --
but driving $4017 consumes nothing and changes no coverage. $4016 is the pad
the game actually consumes; A,B,Select,Start,Up,Down,Left,Right come out LSB
first. START visibly matters: 47.7% coverage vs 39.0% for other masks.
Reaching the match: two discrete START presses -- title -> character select,
character select -> match. Rapid re-tapping does not work.
CAVEATS
-------
* py65 advances PC past the opcode byte before the operand/effective address is
fetched, so a read logged at PC=$AC83 actually comes from the instruction at
$AC82 (`lda ($10),Y`). Reader PCs are therefore 1-2 bytes above the real
instruction; subtract to find it.
* This is a deliberately partial NES: no PPU rendering, no APU. It is enough
to boot, reach the bytecode VM, drive the stage data and enter a match, but
it is not a substitute for FCEUX when visual/sound behaviour matters.
* py65 is NOT a build dependency. It is unpacked on demand:
python3 -m pip download --no-deps -d <dir> py65
unzip -q <dir>/py65-*.whl -d <dir>/py65lib
PYTHONPATH=<dir>/py65lib python3 _tools/trace_stage_payload.py <prg.bin>
Usage: trace_stage_payload.py <prg.bin> [frames] [seq]
seq optional button script "mask@frame,mask@frame,...". mask is the joy1
bitmask held for 3 frames: 0x01 A, 0x02 B, 0x04 Select, 0x08 Start,
0x10 Up, 0x20 Down, 0x40 Left, 0x80 Right.
The two-START match entry is "0x08@30,0x08@400".
"""
import sys, os
sys.path.insert(0, os.environ.get(
'PY65LIB', '/tmp/opencode/py65lib'))
from py65.devices.mpu6502 import MPU
from py65.memory import ObservableMemory
PRG = open(sys.argv[1], 'rb').read()
FRAMES = int(sys.argv[2]) if len(sys.argv) > 2 else 600
SEQ = []
if len(sys.argv) > 3 and sys.argv[3] != '-':
for part in sys.argv[3].split(','):
m, f = part.split('@')
SEQ.append((int(f), int(m, 0)))
SEQ.sort()
TAP_RUN = 3
DOT = 341
SLINES = 262
VBLANK_LINE = 241
FRAME_CYCLES = DOT * SLINES
NMI_CYCLE = VBLANK_LINE * DOT + 1
BASE, MIRROR = 0x8000, 0xC000
PP_LO, PP_HI = 0xB0FB, 0xBEBF
INTERP = 0x9AE8
ZP_PTR_LO, ZP_PTR_HI, ZP_ACT = 0x1A, 0x1B, 0x1E
mem = ObservableMemory(addrWidth=16)
sub = mem._subject
cpu = MPU(memory=mem)
cpu.addcycles = True
mem[BASE:BASE + len(PRG)] = PRG
mem[MIRROR:MIRROR + len(PRG)] = PRG
nmi_vec = mem[0xFFFA] | (mem[0xFFFB] << 8)
reset_vec = mem[0xFFFC] | (mem[0xFFFD] << 8)
st = {'vblank': False, 'nmi_fired': False, 'vt': 0, 'addr': 0, 't': 0,
'x': 0, 'w': 0, 'buf': 0, 'tok': None, 'tokpc': None, 'nreads': 0,
'dma': 0, 'nmis': 0, 'frame': 0, 'strobe': 0, 'shift': 0,
'shift_in': 0, 'r4016': 0, 'r4017': 0, 'w4016': 0}
log, HITS = [], {}
touched = set()
def rd_ppu_status(a):
v = 0x80 if st['vblank'] else 0x00
st['vblank'] = False
return v
def wr_ppu_ctrl(a, v): st['t'] = (st['t'] & 0xF3) | ((v & 3) << 2); return None
def wr_ppu_mask(a, v): return None
def wr_ppu_addr(a, v):
if st['w'] == 0:
st['t'] = (st['t'] & 0x00FF) | ((v & 0x3F) << 8); st['w'] = 1
else:
st['t'] = (st['t'] & 0xFF00) | v
st['addr'] = st['t']; st['w'] = 0
return None
def wr_ppu_scroll(a, v):
if st['w'] == 0:
st['x'] = v & 0xFF; st['w'] = 1
else:
st['t'] = (st['t'] & 0xFFE0) | ((st['x'] >> 3) & 0x1F)
st['t'] = (st['t'] & 0x8C1F) | ((st['x'] & 7) << 12); st['w'] = 0
return None
def rd_ppu_data(a):
addr = st['addr'] & 0x3FFF
v = sub[BASE + addr] if addr < len(PRG) else sub[addr]
st['buf'] = sub[BASE + addr] if addr < len(PRG) else sub[addr]
st['addr'] = (addr + (32 if (sub[0x2000] & 4) else 1)) & 0x3FFF
return v
def wr_ppu_data(a, v):
st['addr'] = (st['addr'] + (32 if (sub[0x2000] & 4) else 1)) & 0x3FFF
return None
def wr_oam_dma(a, v):
for i in range(256):
sub[0x0200 + i] = sub[(v << 8) + i]
st['dma'] += 1
return None
def wr_ctrl1(a, v):
"""joy1 latch: while bit0 stays high the shift register reloads."""
st['w4016'] += 1
st['strobe'] = v & 1
if st['strobe']:
st['shift'] = st['shift_in'] & 0xFF
return None
def rd_ctrl1(a):
"""joy1 read: A,B,Select,Start,Up,Down,Left,Right out LSB first."""
st['r4016'] += 1
v = st['shift'] & 1
st['shift'] = (st['shift'] >> 1) | 0x80
return v
def rd_ctrl2(a):
"""$4017 is the expansion port in this game; nothing consumes it."""
st['r4017'] += 1
return 0x00
def rd_stage(a):
st['nreads'] += 1
touched.add(a)
pc = cpu.pc
HITS.setdefault(pc, [0, a])[0] += 1
if len(log) < 400000:
log.append((pc, a, sub[a], st['tok'], st['tokpc']))
return None
mem.subscribe_to_read([0x2002], rd_ppu_status)
mem.subscribe_to_write([0x2000], wr_ppu_ctrl)
mem.subscribe_to_write([0x2001], wr_ppu_mask)
mem.subscribe_to_write([0x2005], wr_ppu_scroll)
mem.subscribe_to_write([0x2006], wr_ppu_addr)
mem.subscribe_to_write([0x2007], wr_ppu_data)
mem.subscribe_to_read([0x2007], rd_ppu_data)
mem.subscribe_to_write([0x4014], wr_oam_dma)
mem.subscribe_to_write([0x4016], wr_ctrl1)
mem.subscribe_to_read([0x4016], rd_ctrl1)
mem.subscribe_to_read([0x4017], rd_ctrl2)
mem.subscribe_to_read(range(PP_LO, PP_HI + 1), rd_stage)
seq_i = 0
tap_end = 0
def tick(cycles):
global seq_i, tap_end
st['vt'] += cycles
while st['vt'] >= FRAME_CYCLES:
st['vt'] -= FRAME_CYCLES
st['frame'] += 1
st['vblank'] = False
st['nmi_fired'] = False # one NMI per frame, not per $2002 read
if st['frame'] > tap_end:
st['shift_in'] = 0
while seq_i < len(SEQ) and st['frame'] >= SEQ[seq_i][0]:
st['shift_in'] = SEQ[seq_i][1]
tap_end = st['frame'] + TAP_RUN
seq_i += 1
line = st['vt'] // DOT
if line == VBLANK_LINE and not st['nmi_fired']:
st['vblank'] = True
st['nmi_fired'] = True
# NMI
cpu.sp = (cpu.sp - 1) & 0xFF; sub[0x100 + cpu.sp] = cpu.pc & 0xFF
cpu.sp = (cpu.sp - 1) & 0xFF; sub[0x100 + cpu.sp] = (cpu.pc >> 8) & 0xFF
cpu.sp = (cpu.sp - 1) & 0xFF; sub[0x100 + cpu.sp] = (cpu.p | 0x20) & 0xFF
cpu.p |= 0x04
cpu.pc = nmi_vec
st['nmis'] += 1
cpu.pc = reset_vec
cpu.sp = 0xFD
cpu.p = 0x24
print(f'vectors: NMI=${nmi_vec:04X} RESET=${reset_vec:04X}')
print(f'payload window ${PP_LO:04X}-${PP_HI:04X}; target {FRAMES} frames '
f'({FRAMES*FRAME_CYCLES:,} cycles)')
target = FRAMES * FRAME_CYCLES
while cpu.processorCycles < target:
if cpu.pc == INTERP:
p = sub[ZP_PTR_LO] | (sub[ZP_PTR_HI] << 8)
st['tok'] = sub[(p + cpu.y) & 0xFFFF]
st['tokpc'] = cpu.pc
before = cpu.processorCycles
cpu.step()
tick(cpu.processorCycles - before)
print(f'\n{st["frame"]} frames, {cpu.processorCycles:,} cycles, '
f'{st["nmis"]:,} NMIs, {st["dma"]:,} OAM DMAs')
print(f'port: $4016 reads={st["r4016"]:,} writes={st["w4016"]:,} '
f'$4017 reads={st["r4017"]:,} taps fired={seq_i}/{len(SEQ)}')
print(f'payload reads: {st["nreads"]:,} distinct reader PCs: {len(HITS)}')
span = PP_HI - PP_LO + 1
print(f'coverage: {len(touched)} / {span} = {100*len(touched)/span:.1f}%')
print(f'final PC=${cpu.pc:04X} stream=${sub[ZP_PTR_LO]:02X}{sub[ZP_PTR_HI]:02X} '
f'actor=${sub[ZP_ACT]:02X} $0321=${sub[0x0321]:02X} '
f'$0320=${sub[0x0320]:02X} $0326=${sub[0x0326]:02X}')
for pc in sorted(HITS, key=lambda p: -HITS[p][0])[:20]:
c, ex = HITS[pc]
print(f' reader PC=${pc:04X} {c:>7,} reads e.g. ${ex:04X}')
# Unreached spans, so a later run can be compared against this one directly.
miss = [a for a in range(PP_LO, PP_HI + 1) if a not in touched]
runs = []
for a in miss:
if runs and a == runs[-1][1] + 1:
runs[-1][1] = a
else:
runs.append([a, a])
print(f'\nunreached: {len(miss)} bytes in {len(runs)} runs (largest first)')
for lo, hi in sorted(runs, key=lambda r: -(r[1] - r[0]))[:10]:
print(f' ${lo:04X}-${hi:04X} {hi-lo+1:>4} B')
with open('/tmp/opencode/stage_reads.txt', 'w') as f:
for pc, addr, val, tok, tokpc in log:
f.write(f'{pc:04X} {addr:04X} {val:02X} '
f'{("-" if tok is None else "%02X" % tok)}\n')
print(f'wrote /tmp/opencode/stage_reads.txt ({len(log)} rows)')
_tools/verify_pass1.py
+231
View File
@@ -0,0 +1,231 @@
#!/usr/bin/env python3
"""
Verify Pass 1 (code/data split) across every top-level project.
The repo's hard contract is a byte-identical rebuild, and that is what proves
the sources are *faithful*: nothing lost, nothing reformatted. It says nothing
about whether Pass 1 is *complete* — a project can reproduce all 16,384 bytes
perfectly while calling 95% of them data because the tracer gave up early.
So there are two independent questions:
1. **Faithful?** Does the source still rebuild byte-identical?
2. **Complete?** Does the committed source claim as much code as a fresh
trace can actually reach?
Completeness is checked by re-running the tracer and comparing instruction
counts. Comparing *lines* rather than bytes keeps this independent of comment
style, which matters because the audit-comment format is not uniform across the
repo (see CONVENTIONS below).
Each project is traced twice: with the 151 official opcodes, and with the 105
undocumented NMOS opcodes added. The gap between the two separates "the tracer
was too weak" from "genuinely unreachable".
Verdicts
--------
OK committed split agrees with the tracer
SEEDED tracer finds materially LESS than committed - the project used
auto-seed / dispatch-table recovery, so a plain vector trace
cannot judge it. Not a defect.
TRUNCATED tracer finds materially MORE than committed - Pass 1 stopped
early and real code was written out as data. A defect.
CONVENTIONS (reported, not enforced)
-----------------------------------
AGENTS.md requires every instruction to carry `; CPU:$xxxx PRG:xxxx AA BB`.
Several projects predate or ignore that. This tool reports which spelling each
project uses so the drift is visible.
"""
import os
import re
import subprocess
import sys
from pathlib import Path
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import nesdis # noqa: E402
ROOT = Path(__file__).resolve().parent.parent
# Comment spellings actually found in the repo, most-preferred first.
AUDIT_FORMS = {
"canonical CPU:$x PRG:xxxx AA BB":
re.compile(r"CPU:\$?([0-9A-Fa-f]{4})\s+PRG[: ]\$?([0-9A-Fa-f]{4})\s+"
r"((?:[0-9A-Fa-f]{2}[ ]?)+?)\s*(?:--.*)?$"),
"pipe PRG $xxxx | CPU $xxxx | AA BB":
re.compile(r"PRG \$([0-9A-Fa-f]{4})\s*\|\s*CPU \$([0-9A-Fa-f]{4})\s*\|\s*"
r"((?:[0-9A-Fa-f]{2} )+)"),
"colon PRG:$xxxx AA BB":
re.compile(r"PRG:\$?([0-9A-Fa-f]{4})\s+((?:[0-9A-Fa-f]{2} )+)"),
"slash PRG $xxxx / CPU $xxxx: AA BB":
re.compile(r"PRG \$([0-9A-Fa-f]{4})\s*/\s*CPU \$[0-9A-Fa-f]{4}:\s*"
r"((?:[0-9A-Fa-f]{2} )+)"),
"short $xxxx: AA BB":
re.compile(r"\$\$?([0-9A-Fa-f]{4}):\s*((?:[0-9A-Fa-f]{2} )+)"),
"bytes PRG:$xxxx Bytes:AA BB":
re.compile(r"PRG:\$?[0-9A-Fa-f]{4}\s+Bytes:\s*(?:[0-9A-Fa-f]{2} ?)+"),
"rom ROM $xxxxxx: AA BB":
re.compile(r"ROM \$([0-9A-Fa-f]{6}):\s*((?:[0-9A-Fa-f]{2} )+)"),
}
# Every mnemonic the tracer can emit, official and undocumented.
ALL_MNEMONICS = (set(m for m, _ in nesdis.OPCODES.values())
| set(m for m, _ in nesdis.UNDOC_OPCODES.values()))
def find_source(copilot):
for name in ("prg.s", "main.s"):
if (copilot / name).is_file():
return copilot / name
return None
def prg_bytes(copilot):
"""PRG image: prefer the split copy, else carve it out of rom.nes."""
p = copilot / "prg.bin"
if p.is_file():
return p.read_bytes()
rom = copilot / "rom.nes"
if rom.is_file():
r = rom.read_bytes()
return r[16 + (512 if (r[6] & 4) else 0):16 + r[4] * 16384]
return None
def prg_base(copilot):
"""CPU address the PRG is mapped at.
Most projects link the PRG segment at its real CPU base ($C000, or $8000
for MUSCLE). Bomberman instead links one flat file with the iNES header at
$0000 and PRG at $0010, so its linker start is a *file* offset, not a CPU
address; fall back to $C000 in that case.
"""
m = re.search(r"PRG:\s*start\s*=\s*\$([0-9A-Fa-f]+)",
(copilot / "linker.cfg").read_text())
if not m:
return 0xC000
start = int(m.group(1), 16)
return start if start >= 0x8000 else 0xC000
def count_instructions(src):
"""Instruction lines in the committed source. Comment/directive agnostic."""
n = 0
for raw in src.read_text().splitlines():
s = raw.strip()
if not s or s.startswith((";", ".")):
continue
# Macro invocations and symbol assignments are not instructions.
if s.split()[0].lower() in ALL_MNEMONICS:
n += 1
return n
def audit_form(src):
text = src.read_text()
hits = {}
for label, rx in AUDIT_FORMS.items():
n = len(rx.findall(text))
if n:
hits[label] = n
if not hits:
return "NONE"
return max(hits, key=hits.get).split()[0]
def builds_byte_identical(copilot):
"""The faithfulness check: does the project still rebuild its ROM exactly?"""
try:
r = subprocess.run(["make"], cwd=copilot, capture_output=True,
text=True, timeout=600)
except subprocess.TimeoutExpired:
return "TIMEOUT"
out = (r.stdout + r.stderr)
if "Byte-identical ROM verified." in out:
return "yes"
return "NO"
def trace_split(prg, base, undoc):
"""Re-run the Pass 1 tracer. Returns (insn_count, code_bytes, invalid)."""
nesdis.set_undocumented(undoc)
vo = len(prg) - 6
rst = prg[vo + 2] | (prg[vo + 3] << 8)
nmi = prg[vo] | (prg[vo + 1] << 8)
irq = prg[vo + 4] | (prg[vo + 5] << 8)
insns, invalid, _i, _t, _o = nesdis.trace(prg, base, [rst, nmi, irq])
nesdis.set_undocumented(False)
return len(insns), sum(i.length for i in insns.values()), invalid
def main():
wanted = sys.argv[1:]
projects = sorted(
d for d in os.listdir(ROOT)
if (ROOT / d / "copilot" / "Makefile").is_file()
and (ROOT / d / "copilot" / "prg.bin" or
(ROOT / d / "copilot" / "rom.nes").is_file())
and (wanted == [] or d in wanted)
)
print(f"{'project':30s} {'build':>6} {'cmt':>6} {'committed':>10} "
f"{'official':>10} {'+undoc':>10} verdict")
print("-" * 96)
flagged, odd_comments = [], []
for name in projects:
copilot = ROOT / name / "copilot"
src = find_source(copilot)
prg = prg_bytes(copilot)
base = prg_base(copilot)
if src is None or prg is None:
print(f"{name:30s} SKIP")
continue
build = builds_byte_identical(copilot)
for junk in copilot.glob("*.o"):
junk.unlink()
form = audit_form(src)
if form != "canonical":
odd_comments.append((name, form))
committed = count_instructions(src)
off_n, off_b, _ = trace_split(prg, base, False)
und_n, und_b, _ = trace_split(prg, base, True)
if build != "yes":
verdict = f"BUILD-{build}"
elif und_n > committed * 1.05 + 16:
verdict = "TRUNCATED"
elif off_n < committed * 0.95:
verdict = "SEEDED"
else:
verdict = "OK"
if verdict not in ("OK", "SEEDED"):
flagged.append((name, verdict))
print(f"{name:30s} {build:>6} {form[:6]:>6} {committed:>10} "
f"{off_n:>6}/{off_b:<3} {und_n:>5}/{und_b:<4} {verdict}")
print()
if flagged:
print("Needs attention:")
for n, v in flagged:
print(f" {v:14s} {n}")
else:
print("No truncated or unfaithful Pass 1 found.")
print()
print("Comment conventions (AGENTS.md wants `canonical`):")
for n, f in odd_comments:
print(f" {f:8s} {n}")
if not odd_comments:
print(" all canonical")
return 1 if flagged else 0
if __name__ == "__main__":
sys.exit(main())