49e117bfccae264308a12a3e71822d1e1ad3b200
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  tmerrittandClaude Opus 4.8
|
49e117bfcc |
Fix Pass 3 symbol application and add promotion criteria to PROMPTS.md
- Apply symbols.inc hardware register names to Defender 2 (U) (51 operands)
- Apply symbols.inc hardware register names to Donkey Kong 3 (JUE) (36 operands)
- Add APU register symbols (-) to Defender 2 (U) and Donkey Kong 3 (JUE) symbols.inc
- Apply symbols to Donkey Kong Jr. Math (U) (80 operands: 11 single + 69 hardware)
- Apply symbols to Excitebike (JU) (786 operands: 785 single + 1 hardware)
- Add promotion criteria to PROMPTS.md: symbol/local substitution,
ROM-specific semantic naming, and verification checklist
Add branch/jump target naming requirement to PROMPTS.md
All branches and jumps must target symbolic labels with descriptive names
derived from program flow role (e.g., _Loop, _Next, _Done, _Store, _Return),
never raw address literals. Adds criterion 8 to verification checklist.
Rename all OVERVIEW.md to README.md; add root README content to top-level
- Rename all OVERVIEW.md files to README.md throughout the repo (47 files)
- Prepend README.md content to top-level OVERVIEW.md before rename
- Fix apply_symbols.py NAME_RE regex to handle spaced assignments
- Apply symbols.inc to Defender 2 (U) (51 hardware operands)
- Apply symbols.inc to Donkey Kong 3 (JUE) (36 hardware operands)
- Apply symbols.inc to Donkey Kong Jr. Math (U) (80 operands: 11 single + 69 hardware)
- Apply symbols.inc to Excitebike (JU) (786 operands: 785 single + 1 hardware)
- Add missing Zp18/FrameCounter aliases to Excitebike prg.s
- Build all 27 top-level projects: byte-identical
Promote Donkey Kong Jr. Math back to _wip tier
Moves code from `copilot` folder and verifies each builds after
Add pre-commit hook for byte-identical verification
Add .git/hooks/pre-commit that runs 'make' then 'make clean' in each
project directory before committing. This ensures every project builds
byte-identical before any commit is accepted.
Also includes all file moves from the Donkey Kong Jr. Math (U) demotion
and other file restructuring from the session.
moves code out of copilot folder, its all AI generated.
Moves NROM games into NROM folder
removes copilot folder from Lunar Pool
Adds makefile to build all projects recursively
Updates docs to reflect changes in layout
Repromotes Donkey Kong Jr Math
Repromotes Excitebike
Promotes Popeye (JU) (PRG 1)
Updates Makefile to provide summary of build failure/success
Promote Raid on Bungeling Bay (U): Pass 2/3 partial, whole-ROM byte-identical
Move _wip/NROM/Raid on Bungeling Bay (U) to NROM/ and switch the build
from a PRG-only compare to a whole-ROM one:
- header.s/chr.s added (one-line .incbin of header.bin / chr.bin)
- linker.cfg gains HEADER ($0000, $0010) and CHR ($0000, $2000)
areas ahead of PRG ($C000, $4000)
- rom.nes symlink to the space-containing original ROM name
- Makefile builds header.o/prg.o/chr.o into reconstructed.nes and
compares it against rom.nes ('Byte-identical ROM verified.')
- one leftover neutral indirect operand jmp (Zp01) at $E2EC becomes
jmp (IndDispatch01); apply_symbols.py reports 0 substitutions pending
README.md / TODO.md / AGENTS.md updated: 29 promoted, 7 in _wip/.
Pac-Man (U): Pass 2 complete (label renaming) + Pass 3 partial (ZP aliases, table names)
Pass 2 — fresh map derived from the committed split, not from any drafted
map (the drafted maps in other projects were audited and found not
applicable):
- all 194 address-derived Label_XXXX definitions renamed to semantic
routine-entry names plus <Routine>_<Role>_<addr> internal branch
targets; zero Label_ remains
- all 49 jsr/jmp call sites resolved from raw $XXXX to labels; zero
raw $XXXX call operands remain
- routinemap.tsv / derived_names.tsv record the derivation
Pass 3 (partial):
- 15 proven semantic zero-page aliases added to locals.inc
(GameState, NmiFlag, ScrollX/Y, PpuCtrlShadow, OamDmaPage,
CocktailModeFlag, ActivePlayerFlag, GameplaySuspended, FrameCounter,
StateTimer, Player1/2Input, ActiveInput, CopyrightVerified)
- 112 operands substituted into prg.s (11 single + 101 hardware)
- key data tables named: ProgrammerSig ($C0EB), GameStateDispatchTable
($C253), TitleScreenData ($C2FD), EntityPtrTable ($D912),
AudioCommandTable ($EFDD)
Also corrects the README's claim that the RESET vector points at padding:
$FF00 holds real boot code (sei/cld, PPU off, double vblank wait, then
jmp ColdStart at $C042).
.gitignore: ignore reconstructed_prg.bin (build artifact of the _wip
Makefiles; '**/reconstructed.*' does not match the underscore form).
Promote Pac-Man (U): Pass 2/3 complete, whole-ROM byte-identical
Move _wip/NROM/Pac-Man (U) to NROM/ and switch the build from a PRG-only
compare to a whole-ROM one:
- header.s/chr.s added (one-line .incbin of header.bin / chr.bin)
- linker.cfg gains HEADER ($0000, $0010) and CHR ($0000, $2000)
areas ahead of PRG ($C000, $4000)
- rom.nes symlink to 'Pac-Man (U) [!].nes' (the original ROM name
contains a space and brackets)
- Makefile builds header.o/prg.o/chr.o into reconstructed.nes and
compares it against rom.nes
- reconstructed.nes is 24,592 bytes, MD5 a69d310f8cae9397fbd39f204672b5c7
- equal to the original ROM; the header/prg/chr splits were checked
against the original before promotion and are byte-exact
Pass 3 residue closed. The last 7 multi-alias zero-page operands are
resolved per site through a new sites.tsv, each proven from context
rather than guessed:
- jmp ($0010) at $C250/$D3CA/$D90F/$D9B6 -> (DispatchPtrLo); each
is immediately preceded by sta DispatchPtrLo/sta DispatchPtrHi
- jmp ($00FE) at $EFDA -> (AudioHandlerPtrLo); preceded by
sta AudioHandlerPtrLo/sta AudioHandlerPtrHi
- sta a:$0000,y at $C048/$C089 -> a:Zp00,y; the literal zero-page
base in the two ZP clear loops, so the neutral name is used rather
than the semantic WorkPtrLo. The 'a:' prefix survives the
substitution, so the bytes stay 99 00 00 (sta abs,Y)
apply_symbols.py now reports 119 operands substituted (101 hardware,
11 single-name, 7 site) and 0 residue.
New tool _tools/align_audit_comments.py: re-aligns the audit comment to
column 35 after a substitution lengthens an operand (nesdis.py pads each
instruction to 31 characters). Whitespace-only, so the byte identity
cannot change. 405 lines re-aligned in this project; all 2,387 audit
comments still match prg.bin at the stated PRG offset, 0 mismatches.
Docs: root README.md / TODO.md / AGENTS.md updated - 30 promoted, 6 in
_wip/; a per-project detail entry for Pac-Man (U) added; the stale
'Donkey Kong 3 (JUE) has no README.md' claim corrected (it has a
318-line one).
Promote Raid on Bungeling Bay (U) [a2]: Pass 3 complete, whole-ROM byte-identical
Move _wip/NROM/Raid on Bungeling Bay (U) [a2] to NROM/ and switch the build
from a PRG-only compare to a whole-ROM one:
- header.s/chr.s added (one-line .incbin of header.bin / chr.bin)
- linker.cfg gains HEADER ($0000, $0010) and CHR ($0000, $2000)
areas ahead of PRG ($C000, $4000)
- rom.nes symlink to 'Raid on Bungeling Bay (U) [a2].nes' (the original
ROM name contains spaces and brackets)
- reconstructed.nes is 24,592 bytes, MD5 659d340a1e9d420eee28df13319b0094
- equal to the original ROM; the header/prg/chr splits were checked
against the original before promotion and are byte-exact
Pass 3 - 15 semantic zero-page aliases, each applied only at the sites
where its role is proven:
- PpuCtrlShadow ($C2), 15 sites: the reset entry primes it right after
'sta PPUCTRL', and every other use is an ora/and/eor followed by
'sta PPUCTRL'
- NmiPathSelector ($C4), 5 sites: written 1 at $C3E9 and 0 at $C962
- GameState ($D6), 6 sites: 'cmp #$02' and used as an index at $C4C7
- FrameSubCounter ($55), 6 sites: set to $0A at $C520, decremented per
NMI at $C07B
- PrngPeriodCounter ($EA) / PrngSeedB ($EB) / PrngState ($EC), 13 sites:
the PRNG state
- PtrLo ($00) / PtrHi ($01), 8 sites: the 16-bit pair $1C13 and $1610
compared at $E785/$E791
- IndDispatch01 ($01) / IndDispatch02 ($02), 3 sites: the computed
dispatch at $E2EC
- IndDispatch1B ($1B) / IndDispatch1C ($1C), 6 sites: the dispatch at
$E7A3
- IndDispatch17 ($17) / IndDispatch19 ($19): the dispatches at $E79D
and $E7A0
Addresses that carry two names keep the neutral ZpXX name where the other
role applies - $01 is PtrHi at the 16-bit compare and IndDispatch01 at the
dispatch; $17 is IndDispatch17 at the dispatch and Zp17 for the absolute
RAM $0017 store in the copy loop at $E895/$E8D0.
New tool _tools/apply_zp_aliases.py does the per-site substitution:
apply_symbols.py only rewrites *numeric* operands, so it cannot rename an
operand that is already written as a neutral ZpXX name. The tool keeps the
ca65 'a:' prefix (dropping it would turn 'sta a:Zp17,y' into a zero-page
store and change the bytes) and rewrites only the operand token, so the
audit comment survives byte for byte.
Naming correction in the promoted sibling. Finishing this dump showed that
$EA was misnamed 'VBlankWaitCounter': the routine at $C412 is the game's
pseudo-random generator (the classic eor/asl/php/lsr/plp/rol scrambler,
called from thirteen gameplay sites, twice in a row at $CDF4/$CDF9 with the
second result masked #$07), so $EA is the scrambler's phase counter.
Renamed PrngPeriodCounter in both dumps and named $EB/$EC PrngSeedB /
PrngState. The sibling's two 'sta a:IndDispatch17,y' operands were also
wrong - they are absolute stores to RAM $0017, not the dispatch pointer -
and are now 'a:Zp17,y', matching the file's 'a:' = absolute convention.
Audit-comment alignment re-done with _tools/align_audit_comments.py (1,040
lines here, 1,080 in the sibling). Both files still match prg.bin at every
stated PRG offset: 5,467 and 5,483 audit comments, 0 mismatches.
Docs: root README.md / TODO.md / AGENTS.md updated - 31 promoted, 5 in
_wip/; per-project detail entries added for both Raid dumps.
Promote Pac-Man (Tengen) (U): Pass 2/3 from scratch, whole-ROM byte-identical
Move _wip/NROM/Pac-Man (Tengen) (U) to NROM/ and switch the build from a
PRG-only compare to a whole-ROM one:
- header.s/chr.s added (one-line .incbin of header.bin / chr.bin)
- linker.cfg gains HEADER ($0000, $0010) and CHR ($0000, $2000)
areas ahead of PRG ($C000, $4000)
- rom.nes symlink to 'Pac-Man (Tengen) (U) [!].nes'
- reconstructed.nes is 24,592 bytes, MD5 d34e7a74d8d2c231b6cc601210e28ad0
- equal to the original ROM; the header/prg/chr splits were checked
against the original before promotion and are byte-exact
Disassembled from scratch. No name, map or file was carried over from the
sibling 'Pac-Man (U)' dump; the map was derived from this dump's own
committed split and the two were only compared afterwards.
Pass 2 (new tool _tools/apply_pass2_map.py):
- all 194 Label_XXXX definitions renamed to semantic routine entries plus
<Routine>_<Role> branch targets; zero Label_ remains
- all 48 raw $XXXX jsr/jmp operands resolved to labels; zero raw $XXXX
call or branch operands remain
- the map is in routinemap.tsv
Pass 3:
- 30 semantic zero-page aliases, each applied only at the sites where its
role is proven (per-site choice in sites.tsv; the addresses that carry
two roles keep the neutral ZpXX name elsewhere). _tools/
derive_zp_aliases.py records how the set was derived, and locals.inc
records the reason for every remaining neutral slot
- 28 data tables named (new tool _tools/name_data_region.py), including
CharacterDataTable ($C66A) and CharacterNameStrings ($C67E), where the
ROM itself names Blinky, Pinky, Inky and Clyde - which is why the four
indexed entities are called ghosts and not 'actors'
- apply_symbols.py reports 0 substitutions pending and 0 residue
- audit-comment alignment re-done with _tools/align_audit_comments.py;
all 2,479 audit comments still match prg.bin at the stated PRG offset
Pass 1 correction - six command handlers recovered (new tool
_tools/recode_region.py). ScriptCommandTable ($EFDD) is read by the traced
instruction at $EFCC, and six of its entries point at $F00A, $F022, $F03A,
$F052, $F062 and $F072 - all of which the tracer had left as data, and two
of which it had mis-split across the 'jmp $EF61' at $F01F and $F04F.
Those 120 bytes are code here, decoded straight from the ROM; the six
handlers tile $F00A-$F081 exactly. The same gap exists in the promoted
'Pac-Man (U)' dump, which has not been corrected - recorded in both READMEs.
Residue (documented, not guessed): the unreached main game logic at
$C45A-$C55F is still .byte data. It is code - the tracer's own decoder
walks it cleanly and the dispatch table at $C511 points into it - but
nothing reaches it, so it cannot be entered without a guess. The same gap
exists in the promoted 'Pac-Man (U)' dump. Re-deriving it is a Pass 1 job;
a Mesen Code Data Logger capture is the tool the repo uses for it.
Docs: root README.md / TODO.md / AGENTS.md updated - 32 promoted, 4 in
_wip/; a per-project detail entry added for the Tengen dump.
Promote Pyramid (U): Pass 1 corrected, Pass 2/3 complete, whole-ROM byte-identical
Move _wip/NROM/Pyramid (U) to NROM/ and switch the build from a PRG-only
compare to a whole-ROM one:
- header.s/chr.s added (one-line .incbin of header.bin / chr.bin)
- linker.cfg gains HEADER ($0000, $0010) and CHR ($0000, $2000)
areas ahead of PRG ($C000, $4000)
- rom.nes symlink to 'Pyramid (U) (Sachen).nes'
- reconstructed.nes is 24,592 bytes, MD5 aa285ff17f25dd05da5996b7466c59e2
- equal to the original ROM; the header/prg/chr splits were checked
against the original before promotion and are byte-exact
Pass 1 correction - the committed split was a strict subset of the correct
one. This game dispatches through 'jmp' trampoline tables that live
*inside* the data regions, and the tracer cannot follow a jmp there, so
828 bytes of code were still marked as data. Four such tables were found
($C01E 3 entries, $C02A 6, $C042 2, $C054 5) plus ten more entries at
$C000-$C01B; all sixteen targets decode as valid instructions. Seeding
them, the $C800 boot mainline and six further code starts recovers the
whole lot, plus a five-byte stub at $DF89 (lda $33 / jmp $DECB) that sat
in a hole between two routines. The re-trace loses NO committed code byte
and takes the split from 34.5% to 39.6% (3,130 instructions / 6,493 code
bytes). 'make regen' reproduces it, so the split stays verifiable.
Pass 2 - a fresh map derived from this dump's own committed split:
- 428 Label_XXXX definitions renamed; zero Label_ residue
- 320 raw $XXXX jsr/jmp operands resolved; zero raw $XXXX branch
operands remain
- 431 routine entries and the 32 data blocks named from the code
- the internal branch-target names are derived mechanically from the
branch that reaches them by the new _tools/derive_branch_names.py
(beq -> OnZero, bne -> OnNonzero, bcc/bcs -> the carry cases, a
backwards branch -> Loop)
- recorded in routinemap.tsv
Pass 3:
- 24 semantic zero-page aliases, each applied only where its role is
proven; the 57 slots that stay neutral are each listed with the reason
in locals.inc (the neutral set lives in zp_aliases.inc so that
'make regen' can rebuild locals.inc from prg.bin)
- 32 data tables named - TaskVectorTable1/2, LevelScriptData,
MusicPeriodTable, SoundEnvelopeTable, the actor tables - recorded in
data_region_names.tsv
- apply_symbols.py reports 0 substitutions pending and 0 residue; no
hardware register is left as a raw literal
- the audit comments are now canonical on every line, data included
(the old file carried only 'PRG:$xxxx' on its 1,358 data lines);
all 3,762 audit comments match prg.bin at their stated PRG offset
Kept as-is, not 'cleaned up':
- sta $4100 stays a raw operand. The iNES header claims mapper 0 but
the boot code writes past the NES register space; the ROM does not
explain it, so it is recorded rather than guessed
- the 'A'/'V' check on RAM $0120/$0121 and its seed/clear routines
are named for what they do, not for what they are for
- the ninth entry of the $C3A6 task vector table ($D820) is a junk
slot and stays data
Docs: root README.md / TODO.md / AGENTS.md updated - 33 promoted, 3 in
_wip/; a per-project detail entry added.
Hogan's Alley (JU): Pass 1 correction, Pass 2, Pass 3, promoted to NROM/
The light-gun game whose engine dispatches through *inline* tables:
`IndirectCall` at $C34D pops the return address of the `jsr` that called
it and reads a 16-bit handler pointer from the word that follows, so all
six handler tables ($C124 19 entries, $C219 4, $C89C 14, $CDD1 1, $D234 2,
$D3D6 14) live inside the code stream and the static tracer cannot follow
them.
Pass 1 correction. The committed split was a strict subset of the correct
one -- 41.8% of the PRG was classified as code. A fresh Mesen Code Data
Logger capture was merged with the four captures already in the folder
(hogans_merged.cdl, the OR of five per-byte flag arrays, 6,544 code bytes)
and _tools/cdl_reclass.py rewrote the split to match what the CPU actually
fetched: 1,216 bytes moved from data to code, five the other way, 59 labels
emitted. The split is now 49.1% code (3,950 instructions, 8,041 bytes).
The four branch targets the reclassifier could not prove were turned into
labels by the new _tools/resolve_raw_branches.py.
Pass 2. 206 routine entries hand-named from this dump's own code and 413
internal branch targets derived mechanically by _tools/derive_branch_names.py:
630 Label_XXXX definitions gone, 484 raw $XXXX call operands resolved, 15
call targets the tracer had never labelled given a label. Recorded in
routinemap.tsv. No SubC<addr> residue was needed -- every entry the engine
reaches is named after what it does.
Pass 3. 60 semantic zero-page aliases, each applied only where its role is
proven (zp_aliases.inc, zp_alias_sites.tsv); the slots that stay neutral are
each listed with the reason. All 34 data blocks named
(data_region_names.tsv). apply_symbols.py reports 1 residue: `sta a:$00B4,y`
at $C203, the same address as Zp40/ActorPalette but forced to absolute
addressing, so it must stay numeric.
Promotion. header.s, chr.s and a HEADER/PRG/CHR linker.cfg added, rom.nes
symlinked to the original ROM; the whole-ROM cmp passes on the first
promoted build. There is deliberately no `make regen` target: the Pass 1
split here is the static trace corrected by a CDL capture, and the base the
correction was applied to is not what the current _tools/nesdis.py
produces, so prg.s is the maintained source of truth.
Tools changed:
nesdis.py --split/--note/--provenance options, so a data
region can carry its own name and explanation;
a forced-data address is no longer reported as a
recovered routine entry
apply_pass2_map.py can create the label for a call target the tracer
left unlabelled; no longer double-labels it
apply_zp_aliases.py --at-file, an address-based per-site list
derive_branch_names.py keeps the map's own header instead of writing a
fixed one
resolve_raw_branches.py new: turns `bXX *+$NN` operands into labels
build_hogans_map.py new: the hand-written Pass 2 map builder
The existing regen pipelines (MUSCLE (U), Urban Champion (JU)) still
reproduce prg.s byte for byte.
Apply the symbols.inc register names at Pass 1; fix a dropped byte in cdl_reclass
nesdis.py now names the hardware registers as it renders the operands, so a
project's Pass 1 output already reads `lda PPUSTATUS` instead of `lda $2002`
and the Pass 3 apply_symbols.py step has nothing left to do for them.
* nesdis.py --symbols FILE reads a symbols.inc-style file and substitutes
the register names. Only unambiguous entries in the $2000-$4017 window
are used: $4017 is both JOY2 and APUFRAMECOUNTER, so it stays numeric
and apply_symbols.py reports it as multi-alias residue, exactly as
before. The default is the symbols.inc next to --out, so a new project
gets the names without asking; --no-symbols opts out.
* The renderer records the operand's numeric value on the Insn, because
operand_value() used to re-parse the operand text and could no longer
read a name.
* The controller bit masks in symbols.inc (PAD_A and friends) are
constants, not addresses, and are still never substituted.
MUSCLE (U) and Urban Champion (JU) keep --no-symbols: their committed
prg.s was produced with the registers named by the Pass 3 apply_symbols.py
step, and the columns in those files reflect the in-place substitution.
Adding the missing apply_symbols.py step to both regen pipelines makes
them reproduce their committed prg.s byte for byte again - neither did
before this change (MUSCLE was 440 lines out, Urban Champion 772).
cdl_reclass.py had a bug that cost a byte: a region whose last source row
straddled the region's end was replaced by a regeneration of the region
only, so the tail of that row was dropped. Mario Bros (JU) hit it at
$C595, where the rebuild came out one byte short and every label after it
shifted. A region now covers the whole rows it replaces.
pass2.py reflow takes an optional column argument; the default is
unchanged (the ';' lands on the same column as before).
Mario Bros (JU): Pass 1 correction, Pass 2, Pass 3, promoted to NROM/
A two-player platformer whose handler tables are *inline*: the pointers sit
in the middle of the code stream, right after the `jsr` that reads them, so
the static tracer could never follow them.
Pass 1 correction. The committed split was a strict subset of the correct
one -- only 72.0% of the PRG was classified as code. A Mesen Code Data
Logger capture of this ROM (mario_mesen.cdl, 10,915 code bytes) showed
1,010 bytes the tracer had left as data were code; two branch targets the
capture never reached ($E4CB and $E639) were forced with --override for 57
more, so 1,067 bytes moved from data to code and 22 moved the other way.
The split is now 78.4% code (6,210 instructions, 12,847 bytes). The three
branch targets the reclassifier could not prove were turned into labels by
_resolve_raw_branches.py, which now refuses to put a label on a `.byte` row.
Pass 2. 206 routine entries and 519 internal branch targets: 931
Label_XXXX definitions gone, 51 raw $XXXX call operands resolved and 12 call
targets the tracer had never labelled given a label. Zero SubC residue --
the 185 entries a previous session had left as address-derived residue were
all named from this dump's own code. Recorded in routinemap.tsv.
Pass 3. 25 semantic zero-page aliases, each applied only where its role is
proven (zp_aliases.inc); the slots that stay neutral are each listed with the
reason. 35 data tables named -- GameStateHandlerTable, EnemyBehaviourTable,
EnemyScriptTable, EnemyTypePointerTable, SoundRoutineTable,
NoteFrequencyTable and the rest -- and recorded in
data_region_names.tsv. apply_symbols.py reports 0 residue.
Promotion. header.s, chr.s and a HEADER/PRG/CHR linker.cfg added, rom.nes
symlinked to the original ROM; the whole-ROM cmp passes on the first
promoted build. There is deliberately no `make regen` target: the Pass 1
split here is the static trace corrected by a CDL capture, and the base the
correction was applied to is not what the current _tools/nesdis.py
produces, so prg.s is the maintained source of truth.
Tools:
apply_pass2_map.py renames a label of any spelling -- SubC<addr> is
the honest residue form -- and its references
follow it; one combined pattern, so a 16,000-line
file does not need a substitution per label
resolve_raw_branches.py refuses to label a data row
build_mario_map.py new: the hand-written Pass 2 map builder
Remove the old _wip/NROM/Mario Bros (JU) folder (promoted to NROM/)
Mario Bros (JU): add the promoted-tier header, CHR, rom.nes symlink and zero-page aliases
Moves _ROMS/<ROM> into _wip/<MAPPER>/<ROM>
Sync the docs and the skeleton tool with the 857-project _wip tier
The _wip/ tier now holds 857 projects across 30 mapper folders, all
building byte-identical for the whole ROM. README.md, TODO.md and
AGENTS.md still described the old 39-project tier, so bring them in line
with WIP.md, which carries the tier's index.
The tier is no longer 39 projects of NROM-256 with a 16/32 KB PRG:
only two projects have a 16 KB PRG, 151 have 32 KB, 523 have 128 KB and
one has 1.5 MB, so most of it is a banked mapper whose image does not fit
the 6502's address space. Those projects link their MEMORY areas at .nes
file offsets rather than CPU addresses, which is the only option and the
honest model; the 39 older projects keep the CPU-address layout because
their PRG really is one contiguous window.
make_skeleton.py gains the two layouts (--layout flat|cpu), the board
name for the mapper, and a fix for find_rom() mistaking the build
output reconstructed.nes for the source ROM.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
tmerritt
|
837007843d |
MUSCLE (U): decode the bytecode VM layer (Pass 3b)
The 42 token programs were the last undecoded part of MUSCLE. Each now carries a token-by-token listing above its unchanged .byte data, generated by _tools/bytecode_dis.py and wired into the build as 'make pass3'. Encoding, derived from RunBytecodeInterpreter ($9AE8) and BytecodeHandlerTable ($9237): the dispatch is one byte per element. A byte in $C8-$EF selects one of the 40 handler-table entries; every other byte takes the literal path at $9B08 and is pushed onto the operand list at $0033[Y], accumulated into $003C[Y]. The one exception is four opcodes whose handler's first instructions are 'lda ($1A),Y / iny', so they consume the following byte inline: $CB LOOP_BEGIN -> $7C,X count + $7E,X body address $D1 SET_RATE -> $43,X $D7 DRAW_SPRITE -> count for the $A660 blit $E1 SET_WAIT -> $7A,X $CB and $C9 are the loop pair: LOOP_END decrements $7C,X and, if non-zero, reloads Y from $7E,X, rewinding the interpreter to the top of the body. Decoded 377 opcodes and 937 literal operands across 42 programs; all 40 defined opcodes are used, so the opcode set is complete. Two standalone programs contain no byte in $C8-$EF at all, so every byte takes the literal path - the listing states that rather than implying code. Each mnemonic was read off its handler's own straight-line path up to its first 'jmp $9AE8' / 'jmp $9B25'. Handlers that only tail-call a game routine are named for the routine rather than guessed at, since the semantics live in the callee. bytecode_dis.py inserts comment lines only; .byte data is copied through verbatim. Verified: all 16,384 PRG bytes still emitted exactly once, every value equal to the ROM, zero audit mismatches, and 'make regen' (pass1 -> pass2 -> pass3) reproduces prg.s byte-for-byte. Byte-identical ROM verified. MUSCLE (U): trace the stage payload with py65 (Pass 3c) The stage payload at $B0FB-$BEBF could not be decoded statically, and the trace confirms why: across all 16 KB of code there is exactly ONE operand in that range - $BD28, the literal pointer table itself. The real address is formed at runtime by 'adc $BD28,X', where A is a value carried in the bytecode stream, so the readers are indexed by a runtime value and cannot be enumerated by searching the code. FCEUX cannot observe this either: its Lua API offers registerexec / registerexecute / registerrun / registerwrite but no memory-read hook, and with no static list of readers there is nothing to hook. _tools/trace_stage_payload.py therefore uses py65's ObservableMemory.subscribe_to_read, which fires on every read of a range and can report PC from inside the hook. py65 is only a 6502 core, so the tracer adds the minimum hardware to boot the game: a scanline counter off processorCycles, $2002 vblank at scanline 241 cleared on read, NMI at the same scanline, $4014 OAM DMA, and the $2005/$2006/$2007 latches. Without NMI the game never leaves $80E0 (lda $0321 / beq), because $0321 is the flag the bytecode VM raises. Results over 300 frames (26.8M cycles, 4,071 NMIs): 2,548 payload reads from 44 distinct reader PCs, touching 914 of 3,421 bytes. Findings: - The hottest reader is $AC82, the compressed-VRAM RLE decoder: 1,046 of its 1,128 reads advance by exactly +1, comparing each byte against $C0. - $9B50 is a backward copy (lda ($12),Y / sta ($10),Y / dey), stepping -1. - 107 distinct 8-byte blocks had all eight bytes read, with spacing dominated by $08 - a dense 8-byte record array - plus a sparser group on a $D0 stride ($BBE0, $BCB0, $BD80, $BE50). - Readers use three disciplines: forward sequential, backward sequential, and variable stride consistent with a fixed-offset field walk. py65 is deliberately NOT a build dependency: the tool documents unpacking the wheel on demand. 'make regen' is unaffected. Byte-identical ROM verified. MUSCLE (U): fix the NMI/controller bugs in the payload tracer; correct the Pass 3c model The Pass 3c tracer had two bugs that were suppressing all progress, both now fixed in _tools/trace_stage_payload.py (previously scratch, now durable): * NMI re-armed by $2002 reads. The vblank flag must be cleared once per frame, not on every read of $2002 -- the main loop polls $2002 and spins on the bytecode VM flag at $8226, so each poll cleared bit 7 and re-armed NMI inside the same scanline. That gave 40,900 NMIs across 3,000 frames and corrupted all game state. A per-frame nmi_fired flag restores exactly one NMI per frame (verified: 600 frames -> 600 NMIs). * Input went to $4017. The poll routine at $AA69 is `lda $4016,X` with X=1 and only compares bits 0-1, so driving $4017 consumed nothing -- all masks tested gave byte-identical traces. The game consumes joy1 on $4016 and needs a real latch/shift register. With it, START matters: 47.7% coverage vs 39.0%. Match entry is two discrete START presses (title -> char select -> match). Coverage 26.3% -> 40.9%. Both bugs and the port subtlety are documented in the tool header so they are not re-derived. Corrects the Pass 3c conclusion. $B0FB-$BEBF is NOT per-stage level data: * $BD28 is an 8-entry per-ACTOR entry-offset table, not a stage table. $A063 `adc $BD28,X` with X = ActorIndex ($1E), plus #$32 at $A071 when PlayerActiveFlag ($78,X) is set. Observed actors are $00-$04. Forcing all 8 actor indices adds zero new bytes. * The sound engine reads the same window: $B03C `lda (SoundReadPointerLo),Y` with `cmp #$F0 / sbc #$F0 / tay / lda $B0E3,y` is note decoding, which is where the $FF/$F0 high-nibble terminators come from. So the window is a shared pool -- per-actor bytecode, compressed-VRAM RLE, and music note streams -- and stage cycling cannot raise coverage past ~40%. The remaining 2,161 B need other music tracks and scripted match play, not taps. Also corrects two stale claims: $0326 is not a level byte (never leaves $00 even in a confirmed match), and $0320 is a boolean, not a screen-state machine. Build unaffected: byte-identical, and `make regen` still reproduces prg.s byte-for-byte. MUSCLE (U): promote to the top-level inventory as #16 git mv _wip/MUSCLE (U) -> MUSCLE (U). The relative symlink copilot/rom.nes -> ../M.U.S.C.L.E. (U).nes survives the move untouched. * copilot/Makefile: tool paths ../../../_tools/ -> ../../_tools/ for NESDIS, PASS2 and BYTECODE. `make` prints the byte-identity message and `make regen` still reproduces prg.s byte-for-byte, both verified after the move. * MUSCLE (U)/OVERVIEW.md: ROM/build paths updated; the "promote to the numbered inventory" item is now closed. * Root OVERVIEW.md: added as #16, removed from the _wip skeleton table (20 -> 19), added a ### 16 detail section, and its vector/mapping notes moved out of the skeleton section. All 16 top-level projects verified byte-identical. Add Pass 1 verification tool; teach nesdis.py the undocumented opcodes _tools/verify_pass1.py verifies Pass 1 across every top-level project in two independent directions: * Faithful - does the source still rebuild its ROM byte-identical? * Complete - does the committed source claim as much code as a fresh trace can actually reach? The second is the one that catches real problems. A project can reproduce all 16,384 bytes perfectly while calling 95% of them data, because the tracer gave up early. Instruction *counts* are compared rather than bytes, so the check is independent of comment style (which is not uniform across the repo). Result: all 16 projects are faithful, and Joypad Test Cartridge is the only truncated Pass 1 -- 282 committed instructions against 1,773 reachable. nesdis.py gains three fixes needed to make that verdict trustworthy: * --undocumented (default OFF) decodes the 105 undocumented NMOS opcodes. They are used as NOP padding and as real RMW ops, and a tracer that halts on them loses most of the code: Joypad goes 580 -> 3,846 code bytes. Default off so MUSCLE's `make regen` still reproduces prg.s byte-for-byte. * Undocumented opcodes are emitted as explicit `.byte` with the disassembly in the comment. ca65 V2.19 implements none of them, so they cannot be written as mnemonics; keeping them in the code region preserves the code/data split and the audit trail. Official 0xEA NOP still encodes normally. * Two latent bugs, both now reachable: - RESET and IRQ are commonly the same address, and the semantic-name map was overwriting, renaming the primary entry point to `IrqHandler`. Now setdefault, in RESET/NMI/IRQ priority order. - render() stamped every branch operand as Label_XXXX and never reconciled it with the semantic name map, so a branch into a named routine emitted a label that was never defined and the file would not assemble. Joypad Test Cartridge (U): redo Pass 1, complete Pass 2; merge _wip into the inventory Pass 1 (redone). The committed split claimed 282 instructions / 580 bytes (3.5%). The tracer halted at $C09E on `DCP $3C20` ($CF) -- an undocumented opcode used as ordinary code -- and filed 3,256 bytes of real code as `.byte` data. Now 1,773 instructions / 3,836 bytes (23.4%), a 6.6x increase. Verified against a fresh trace: 1,769 emitted instructions + 4 undocumented opcodes written as explicit bytes = 1,773, an exact match. Byte accounting stays exact at 3,836 + 12,548 = 16,384. ca65 V2.19 implements none of the undocumented opcodes, so the 4 affected instructions (DCP $3C20, LAX $20D4, NOP $C1,X, RRA ($D0),X) are emitted as explicit `.byte` with the disassembly in the comment. They stay in the code region, so the split and the audit trail are both preserved. Pass 2 (complete). All 202 labels renamed; zero `Label_Cxxxx` remaining. Renames are keyed by address and verified to have bound to the intended address, not just to some address. 202 routine headers, every one describing real behaviour -- no "undetermined in Pass 1" boilerplate left. `locals.inc` carries 99 aliases, the proven semantic ones sitting beside the generated `ZpXX` entries per the alias pattern in AGENTS.md. Two naming errors in the previous draft are corrected: $55 is MarkerValue (it holds $70/$88/$A0 and is compared against them), not a "P1 Down timer"; and the twelve MarkerVar variables are named by position because which player/button each belongs to is not established. Pass 3 is now outstanding, and the old Pass 3 was wrong: it described $C09E-$C173 and $C17F-$CCF6 as data, and both are code. After the redo there is exactly one data region, $CF06-$FFFF, confirmed by orphan scan. Documentation. The root inventory now lists all 36 projects in one table -- the 20 `_wip/` projects are rows 17-36 rather than a second table that could drift. That second table had wrong mirroring for 12 of them (Duck Hunt, Golf, Hogan's Alley, Ice Climber, DK Jr Math, Lode Runner, Lunar Pool, Mario Bros, Millipede, Pyramid, Raid [a2], Urban Champion were all recorded H when they are V, and vice versa); every value is now read from the ROM rather than trusted. Build status is split into two honest tiers: rows 1-16 are BYTE-IDENTICAL and compare the whole ROM, rows 17-36 are PRG-IDENTICAL because their Makefiles only run `cmp reconstructed_prg.bin prg.bin`. Header and CHR are not proven for those. All 36 build cleanly; the WIP count was also corrected from 19 to 20. `_tools/verify_pass1.py` checks all 36 and reports none truncated. Mario Bros (JU): Pass 1 disassembly, 14.6% -> 72.2% code The skeleton prg.s was a 12-line .incbin of the whole PRG. It is now a real disassembly: 5,735 instructions, 11,829 code bytes (72.2%), 937 labels and routine headers, byte accounting exact at 11,800 + 4,584 = 16,384, and BYTE-IDENTICAL against prg.bin. Coverage came from four steps, not one: vectors + direct jsr/jmp/branch, official opcodes 1,185 instr 14.6% --undocumented (105 extra NMOS opcodes) 1,210 instr 14.9% --auto-seed (61 table-referenced entries) 4,187 instr 51.1% --extra-seed 0xC0B6,0xC5A3 5,735 instr 72.2% Almost everything sits behind computed dispatch. Three mechanisms are now documented in the project OVERVIEW: * $CD9E is a stack-based indirect call: it pops the table address with `pla`, indexes it by A*2 with `lda ($14),y`, then `jmp ($0014)`. Because the pointer arrives on the stack it cannot be followed statically from the dispatcher. * $C5A3 is a 20-entry table of `JSR` thunks ending in `rts`. Probing it linearly gives twenty valid targets, and seeding it recovers most of the last 21%. * $C0B6 is the NMI phase dispatcher, reachable only by computed dispatch. Two overlapping-decode bugs in _tools/nesdis.py had to be fixed first: * Control-flow targets must win over fall-through. `bne` at $C4BA targets $C528, but the fall-through path had decoded $C527 as a 2-byte `NOP #$A5` whose operand swallowed $C528. No assembler branches into the middle of an instruction, so the spanning decode is now dropped in favour of the target; its bytes fall out as data, leaving byte accounting unchanged. * Branch targets inside a surviving instruction must be pruned from the target set. Otherwise emit_code's linear walk consumes those bytes as an operand, so the label is referenced but never defined and the file does not assemble at all -- `Error: Symbol 'Label_C528' is undefined`. Neither change disturbs an existing pipeline: MUSCLE's `make regen` still reproduces prg.s byte-for-byte, Joypad Test Cartridge still builds byte-identical, and _tools/verify_pass1.py reports all 16 top-level projects faithful with none truncated. Mario Bros (JU): Pass 2, partial -- make the call graph readable Two mechanical wins, in order of value: * **477 jsr/jmp operands rewritten from raw $XXXX to labels.** The tracer emits call and jump targets as hex, so the call graph was unreadable -- the most called routine appeared as `jsr $CCA0` 14 times over. Every one of those sites already had a label at its target address, so the substitution is purely mechanical and byte-preserving. Zero raw hex call operands remain. * **13 landmark routines named** from verified instruction evidence: WaitVblank1, ClearRamPageLoop, NmiPhaseDispatch ($C0B6), JsrThunkTable ($C5A3), DispatchViaStackPointer ($CD9E), AdvanceDispatchPointer ($CDB4), PushTablePointerDispatch ($CCA0), DispatchTableF393 ($CAB9), InitSpritesOffscreen ($DFC4), AdjustDecimalColumns ($D15A), TestRamFlagPair0500 ($D328), NextIndex ($CE84). **922 labels remain unnamed, and that is recorded as the honest state** rather than papered over. They sit 15-25 per page across $CC00-$CFFF, which is just the game logic: naming them means reading ~5,700 instructions across 935 routines. Guessing from the call graph and the NES convention would be exactly the guessy rewrite PROMPTS.md rules out. The productive next step is more reach, not more naming. The 4,555 unreached bytes are where most unnamed routines live, and they are gated behind the stack-pointer dispatch at $CD9E, whose pushed table addresses are only known in the callers' frames. Still BYTE-IDENTICAL; all 16 top-level projects remain faithful. Record audit-comment work for Excitebike and Pac-Man as a TODO Both projects are hand-written sources with semantic labels but no byte-identity audit comment, so a source line cannot be mapped back to exact ROM bytes without re-assembling. Every other project in the repo carries one. Documented what was tried and why it was dropped rather than left implicit. Addresses from 'ca65 -l' work fine -- the listing offset indexes prg.bin directly -- but the listing carries no source line numbers, so entries cannot be keyed back to source lines. Byte values also have to come from prg.bin rather than the listing, since ca65 prints 'rr rr' for relocatable words. No tool was committed, because the one written does not work. Both projects verified still byte-identical; no source file was changed. Add TODO.md handoff; bring AGENTS.md back in line with the repo AGENTS.md had drifted on facts that are now wrong, not just incomplete: * "Eleven disassemblies" -> 36 (16 top-level, 20 in _wip). The index it points at had the same stale count. * The audit-comment convention was stated as `; PRG:$xxxx AA BB`, but the dominant spelling is `; CPU:$xxxx PRG:xxxx AA BB` and five other variants are in use, with Excitebike and Pac-Man carrying none at all. Now stated accurately as "preserve whatever spelling a file uses", with the gap pointed at. * `_tools/` was described as holding splitnes.py and chr2gif.py. It holds six tools now, including the Pass 1 tracer and the verifier. * ROM filenames "contain spaces" -> spaces *and parentheses*, which is also what makes FCEUX hang headless. Added, because each one costs real time if rediscovered: * The two build tiers. The 20 _wip projects only `cmp reconstructed_prg.bin prg.bin`, so their header and CHR are NOT proven -- calling them byte-identical to the ROM is wrong. * A promotion procedure, since that has now been done twice. * A tracer/toolchain gotchas section: undocumented opcodes truncating traces (and ca65 being unable to encode them), vectors living at the end of the PRG rather than in the iNES header, the RESET==IRQ name collision, control-flow targets beating fall-through, the branch-target pruning that stops a file failing to assemble, NMI firing once per frame rather than per $2002 read, joy1 being $4016, and ca65 listings carrying no source line numbers. TODO.md is the handoff document: open work per project with enough detail to resume, the residue counts, and the reasoning behind each abandoned approach so it is not silently retried. Nothing rebuilt or changed; all projects still verify. Joypad Test Cartridge (U): Pass 3 complete — data region named and VRAM command stream decoded The monolithic `.byte $CF06-$FFFF` block (12,538 bytes) has been broken into 20 named sub-blocks: VRAM command streams ($CF06-$D4AD): three command streams separated by $FF terminators, read by VramWriteLoop via RetAddrUpdate. Opcode set decoded: $4C = jump, $60 = deferred PPUCTRL, bit-7-clear = VRAM write (PPU addr + data, bit 6 of data = long/short selector). OAM tables ($D4E3-$D6A3): 12 groups of 32-byte tables (8 entries of Y/tile/attr/X), read by CopyOamFromTable ($D4E3) and CopyOamFromTable70 ($D5C4). Sprite templates ($D6A5-$D6E4): sprite template read by CopySpriteTemplateTable ($D6A5) and button OAM template read by PrepareButtonOam ($D6C5). Padding: $FF ROM padding ($D6E5-$FFF9) and vectors ($FFFA-$FFFF). OVERVIEW.md updated: section 8 documents all 20 sub-blocks with the VRAM command stream format and OAM table structure; section 11 marks Pass 3 complete with remaining work (MarkerVar player/button mapping, HandleButtonPress sprite order). Build verified byte-identical. Root OVERVIEW.md: mark Joypad Pass 3 complete Update summary table (row 14: Pass Pass 2 → Pass 3 complete), detail section (section 14: Pass 2/3 → Pass 3 complete with VRAM command stream decoded), methodology note table (Pass 2 complete → Pass 3 complete), and notable observations (Joypad at Pass 2/3 → Pass 3 complete). Joypad Test Cartridge is now fully Pass 3 complete: all 20 data sub-blocks named, VRAM command stream opcode set decoded, OAM table structure documented. TODO.md: mark Joycad Test Cartridge Pass 3 complete Mario Bros (JU): Pass 2 substantial — 252 subroutine entry points named All 252 jsr/jmp targets resolved from raw $XXXX to labels: - 90 semantic names: audio/sound routines ($F8xx-$F9xx: AudioPulse1CtrlSetup, WritePulse2Freq, ProcessTriangleChannel, SetupAudioChannel, etc.), gameplay helpers (CheckZpB3Return, CheckRangeReturn012, ToggleZpFC_Bit3, etc.) - 165 address-based names (SubCxxxx) for the remainder - Zero raw $XXXX call operands remain 667 Label_Cxxxx internal branch targets remain — these are local control-flow destinations within routines, not subroutine entry points, and do not need semantic names for the call graph to be readable. Build verified BYTE-IDENTICAL. Root OVERVIEW.md updated (row 26: Pass 0 skeleton → Pass 1 complete, Pass 2 substantial). TODO.md updated. Lunar Pool (U): Pass 1 complete, 57.8% code; nesdis.py opcode/emitter fixes The skeleton prg.s was a 12-line .incbin of the whole PRG. It is now a real disassembly: 4,586 instructions, 9,475 code bytes (57.8%), 6,909 data bytes, byte accounting exact at 9,403 + 6,981 = 16,384, BYTE-IDENTICAL. Coverage: vectors + direct jsr/jmp/branch reached 1,259 instructions (15.6%). --undocumented took it to 4,572 (57.6%), and --auto-seed (48 table-referenced entries) plus --extra-seed 0xE80C (the non-$C000 RESET vector) took it to 4,586 (57.8%). Three undocumented opcodes were missing from nesdis.py and are added: * $72 RRA abs (the RRA family was missing the $72 absolute form) * $42 NOP #imm (the NOP-immediate family was missing $42) * $FA NOP abs,X (the NOP-abs,X family was missing $FA) One emitter fix: ca65 silently rewrites an operand whose high byte is $00 in the shorter zero-page form -- `sta $00FF,x` assembles as `sta $FF,x` ($95 instead of $9D). render() now emits the `a:` prefix for ABS/ABSX/ABSY/IND operands with a high byte of $00, forcing the absolute form so the emitted bytes match the ROM. operand_value() strips the prefix when parsing. These changes are general and verified not to disturb existing pipelines: MUSCLE's `make regen` still reproduces prg.s byte-for-byte. Remaining gaps: 6,909 bytes (42.2%) still data, gated behind jmp ($7961) at three sites (RAM-pointer dispatch; the blocks after them hold the screen text "strings" "game over" and "hi`score__") and stack-based dispatch at $CBDC -- the same pattern as Mario Bros (JU)'s $CD9E. TODO.md: add Lunar Pool as an open item Adds comments for all lines to Pacman and Excitebike Moves Pacman 'locals' and 'symbols' definitions into the right places |
||
|
tmerritt
|
9aef410424 |
Promotes Donkey Kong Jr
Adds remaining NROM as _wip Update gitignore to not keep object files Adds AGENTS.md as this is an AI helped project |
||
|
tmerritt
|
25423d3828 |
Roms Completed:
Baseball Clu Clu Land Excitebike Pac-Man (Namco) Tennis Balloon Fight Bomberman (U) Burger Time (U) Defender 2 (U) |