Promote Donkey Kong (JU): Pass 3 complete, whole-ROM byte-identical

Pass 3 named the 10 data regions in prg.s, then the project was promoted out of _wip/ so `make` now proves header + PRG + CHR against the original ROM instead of only the PRG bank. Data regions named: LevelPtrTable $C000-$C03B (60 B) level header data ObjectPtrTable $C03C-$C043 (8 B) 4 pointer pairs LevelDataPtrTable $C044-$C79D (1882 B) level data pointers SpriteInitPtrTable labelled at $C4B0 see residue note UnreachedCode_D650 $D650-$D696 (71 B) unreached code UnreachedCode_E144 $E144-$E165 (34 B) unreached code MusicDataTable $F55B-$FA47 (1261 B) music/sound data APUDataTable $FAE6-$FB74 (143 B) APU pulse/noise tables SfxDataTable $FE59-$FFEF (407 B) sound-effect data VectorTable $FFF6-$FFFF (10 B) padding + NMI/RESET/IRQ Promotion (per the AGENTS.md checklist): - git mv "_wip/Donkey Kong (JU)" "Donkey Kong (JU)" - add copilot/rom.nes -> ../Donkey Kong (JU).nes symlink - add header.s / chr.s (one-line .incbin of header.bin / chr.bin) - linker.cfg gains HEADER ($0000/16 B) and CHR ($0000/8 KB) - Makefile switched from `cmp reconstructed_prg.bin prg.bin` to `cmp reconstructed.nes rom.nes` - no _tools path change (this Makefile never referenced _tools/) and no `make regen` pipeline exists for it Doc corrections: - project OVERVIEW.md said Mirroring Vertical; header byte 6 = $00 means Horizontal (the root index was right) — fixed - root OVERVIEW.md: row 18 now BYTE-IDENTICAL, tier paragraph and summary table refreshed (they had drifted around the Lunar Pool and Urban Champion promotions), new per-project detail entry - counts corrected to 19 top-level / 17 _wip in OVERVIEW.md, TODO.md and AGENTS.md Residue recorded rather than guessed: - Pass 2 is partial: 919 Label_ definitions remain (214 semantic labels applied); noted in TODO.md so nobody reads the promotion as full semantic naming - SpriteInitPtrTable's label sits at $C4B0: the routine at $C807 reads $C4A7,x/$C4A8,x, but that table starts mid-row and a label there would split the canonical 16-byte audit-comment line Verification: make clean && make -> "Byte-identical ROM verified." verify_pass1.py -> 19/19 top-level build, Donkey Kong (JU) OK (6014 instr / 12505 B), all comments canonical Doc-drift sweep: build tiers, mirroring, and stale status tails Reconciled the docs against verified reality after the Lunar Pool / Urban Champion / Donkey Kong (JU) promotions and the 2026-10-19 audit-comment standardization. No source behaviour changed except the two build fixes below. Build tier (whole-ROM vs PRG-only) - AGENTS.md: the 17 `_wip/` projects are real disassemblies at the PRG-build tier, not "skeletons"; header.s/chr.s/rom.nes are promotion additions; verify_pass1.py covers the top-level projects. - Root OVERVIEW.md: `_wip` prose, audit-comment narrative and summary table refreshed (37.5% figure, SEEDED vs OK wording, re-trace false alarm). - Promoted projects whose build blocks still claimed `make` emits `reconstructed_prg.bin` compared to `prg.bin` (Balloon Fight, Baseball, CIRCUS, Clu Clu Land, Excitebike, NES PowerPad, Pac-Man (USA), Tennis): now describe the whole-ROM `reconstructed.nes` vs `rom.nes`. - Lunar Pool / Urban Champion / CIRCUS also had stale `cd "_wip/..."` build paths and "Promotion" items; retargeted to their top-level folders. Mirroring (header byte 6 bit 0: 0 = Horizontal, 1 = Vertical) - Corrected DK Jr. prose "horizontal mirroring (flag 6 = $01)" -> vertical, and both `_wip/Pac-Man` variant notes "Both use vertical mirroring" -> horizontal (both are byte-6 = $00). - Re-audited every project OVERVIEW: 0 rows and 0 prose claims now disagree. Stale status tails - Retitled/restatused the `_wip` OVERVIEWs to their real `prg.s` (Golf, Ice Climber (U), Lode Runner, Millipede, Pac-Man Tengen, Pac-Man (U), Popeye 0/1, Raid, Raid a2, Hogan's Alley, Mario Bros); Golf's skeleton-era contents rows and Next Steps corrected. - Dropped the skeleton-era "## Next Steps (per `PROMPTS.md`)" heading and the "(from the skeleton)" wording across all project OVERVIEWs. - Rewrote Lunar Pool / Urban Champion Next Steps (both already Pass 1-3 complete and promoted; their tails still described pending Pass 2 and promotion); fixed root row 13 (Wild Gunman) and row 20 (Duck Hunt). Also lands two code fixes carried with this sweep: - `_wip/Hogan's Alley (JU)`: $E9F1-$E9FD and $EA13-$EA14 reclassified as data with canonical audit comments (the seeded build had decoded a data table base as code, breaking the build since 2ae09f7). `make` -> BYTE-IDENTICAL again; reconstructed_prg.bin/.lbl refreshed. - `_wip/Pyramid (U)`: `Label_D72E` -> `NMI:` (byte-identical). Verification: make (Hogan's Alley, Pyramid) -> BYTE-IDENTICAL, BYTE-IDENTICAL _tools/verify_pass1.py -> 19/19 build, all canonical, "No truncated or unfaithful Pass 1" mirroring audit (rows + prose) -> 0 mismatches Donkey Kong (JU): complete Pass 2 — all 919 Label_ definitions named The last non-zero Label_ residue in the repo is now gone. Every one of the 919 address-derived definitions in copilot/prg.s is replaced: * 319 routine entry points (jsr/jmp targets): - 102 named from observed behaviour (LoadSpriteInitPointerToZp00 $C807, CopyVramDataToBuffer $CBDF, CheckSpriteListCollision $D8AD, UpdateRandomSeed $F4ED, ReadControllers $F50E, UpdateSoundEngine $FA48, ...) - 217 left as SubC<addr> — honest residue where the body is too context-dependent to name without guessing (Golf: 108, Mario Bros: 223) - the 77 already-named entries untouched * 600 internal branch targets named <Routine>_Block / <Routine>_Block_n in address order within their owning routine (DK Jr./Balloon Fight convention). Ownership is control-flow containment, so a loop that physically precedes its own entry lands correctly (FlushPpuBuffer_Block $F1EC under $F228). * Two proposal collisions disambiguated by indexing zero-page slot: UpdateObjectListZp5D/DispatchObjectStateZp5E ($DA16/$DA4C) vs UpdateObjectListZpAE/DispatchObjectStateZpAF ($E1E9/$E250). jsr/jmp operands stay numeric (Wild Gunman style); only branch operands are symbolic. Applied with _tools/pass2.py apply (919 defs + 2,622 refs rewritten, comments re-aligned); the ROM still rebuilds byte-identically and verify_pass1.py reports 19/19 canonical with no truncated split. Docs synced: project OVERVIEW.md gains a Pass 2 Details section, root OVERVIEW.md rows + conventions, TODO.md section 4 closed. dk_full_map.tsv now records the complete applied map (its zp rows remain unapplied proposals). _wip status correction: five false "Pass 2 complete" claims + map audit The `_wip` tier's "Pass 2 complete" headers counted their draft maps' rows, not the names actually present in `prg.s`, and described the count of *neutral* `ZpXX` slots in `locals.inc` as "zero-page semantic aliases". Verified against the sources and corrected in the root inventory rows, the project OVERVIEWs and TODO: project named/total Label_ left drafted map Pinball (JU) 180 / 681 501 210 rows, 179 applied Ice Climber (JE) 76 / 973 897 255 rows, 76 applied Donkey Kong Jr. Math 64 / 1035 971 243 rows, 62 applied Duck Hunt (JUE) 31 / 472 441 272 rows, 31 applied Ice Climber (U) 2 / 973 971 none - Pass 2 not started No `_wip` project is Pass 2 complete. Ice Climber (U) was the worst case: the 2026-10-19 sweep itself restated it "Pass 2 complete" while describing its 971 `Label_` definitions in the same paragraph - a copy-paste from (JE); (U) has no map file at all. Drafted maps audited row by row against the committed split: they are not applicable. 100/243 (DK Jr. Math), 96/255 (Ice Climber (JE)) and 17/210 (Pinball) rows point at addresses that are not instruction boundaries - they were built against an older code/data split - while most of the rest are unlabelled boundaries that no operand references, or (Duck Hunt) competing candidate names for one label (`Label_C2BF` alone carries 205). Rows that are both insertable and corroborated by control flow: 0 in every map. Recorded as stale and kept for provenance; Pass 2 here needs a fresh map derived from the committed split. Also corrected while verifying: - Mario Bros entry counts (docs said 252 / 90 + 165 / 477 sites / 156 neutral slots; the file holds 262 distinct `jsr`/`jmp` targets = 39 semantic + 223 `SubC<addr>`, 478 sites, 151 slots). - root `_wip` prose line/label ranges (4,482-17,180 lines, 192-971 `Label_`, 11,126 total) and the Ice Climber code figure (74.6% / 6,112 instr, not 74.7% / 6,113). - TODO §4 re-scoped to the promoted tier, with the `_wip` residue table added. Verification: `make` across all 17 `_wip` projects -> BYTE-IDENTICAL (17/17); `_tools/verify_pass1.py` -> 19/19 build, all canonical, "No truncated or unfaithful Pass 1 found". No source bytes changed. Balloon Fight (JU): apply locals.inc/symbols.inc names to prg.s (Pass 3) Substitutes symbol names for 922 numeric operands: 125 hardware registers (symbols.inc) and 797 of 813 zero-page/RAM operands (locals.inc). The 15 multi-alias addresses were resolved per routine; 16 operands stay numeric because $13 is used there as a plain high-byte scratch beside Temp12 and none of its three aliases fits. - _tools/apply_symbols.py: the substitution tool (dry run by default; --sites supplies per-routine choices for multi-alias addresses, --report lists the residue). Only the operand token is rewritten, so the `; CPU:$.... PRG:.... XX XX` audit comments survive byte for byte. - prg.s: operand tokens only. make prints "Byte-identical ROM verified." - OVERVIEW.md (project): instruction quotes now match prg.s, new alias-reuse table, and section 10 rewritten from the stale "497 Label_" prose to the real naming state and the 16-site residue. - OVERVIEW.md (root), TODO.md, AGENTS.md: status, counts, tool list. verify_pass1.py: 19/19 OK, all audit comments canonical. Bomberman (U): apply locals.inc/symbols.inc names to prg.s (Pass 3) Substitutes 362 zero-page operands in copilot/prg.s with the names already declared in copilot/locals.inc: 354 from single-name addresses and 8 from the two multi-alias addresses, resolved per site: $E0/$E1 IndirectPtrLo/Hi in the RAM-clear loop (InitializeSystemAndRunMainLoop -> Main_WriteSoundDispatchIndex_C317, which writes $00 across $0100-$07FF via (ptr),y), SoundDelay0 (the base of the per-channel delay table, indexed by X) in the sound engine; No symbols.inc substitution was needed: copilot/prg.s contains no numeric hardware operands ($2000-$4017) at all. All 362 operands were already written with two hex digits, so no ca65 `a:` prefix was required. make -> Byte-identical ROM verified. Operand token only; mnemonics, spacing and the trailing `; CPU:$.... PRG:.... XX XX` audit comments are untouched. Remaining numeric operands (758) address bytes that no .inc file names: 505 zero-page (133 addresses), 171 RAM $0100-$07FF (46 addresses) and 82 table reads in $8000+ (54 addresses). Bomberman (U): substitute 24 in-file table labels at ROM operand sites Uses the new `--labels` option of `apply_symbols.py` with the ld65 `-Ln` output (`reconstructed.lbl`). Only labels that prg.s actually defines and that live in the PRG (+) are considered, so CHR/header segment labels cannot leak in. 24 numeric operands (`lda ,x`, `adc ,y`, etc.) now read `lda SystemPaletteData,x`, `adc GameplayObjectClassTable,y`, etc. All 24 point at exact label addresses; the remaining 58 ROM-space numeric operands point at interior bytes or unlabelled tables and stay numeric. make -> Byte-identical ROM verified. Bomberman (U): update docs for Pass 3 symbol application - Project OVERVIEW.md: corrected source file from main.s to prg.s/header.s/chr.s; output name from reconstructed_bomberman.nes to reconstructed.nes; rewritten §10 Zero-Page/RAM Symbols table to match locals.inc (with aliases and unnamed residue documented); fixed §4 step 1/4 to use IndirectPtrLo/Hi and SoundDelay0 names; fixed §11 InesHeader location (now header.s); added §2 Pass 3 summary table; added §13 notes on legacy main.s and Main_WriteSoundDispatchIndex_C317 misnomer. - Root OVERVIEW.md §3: updated source line count (7,553), file list, and added Pass 3 summary. Both builds verified byte-identical. Pass 3 symbol application to all 19 top-level projects (2026-10-19) Ran _tools/apply_symbols.py --write on all 19 promoted projects to substitute symbolic names from symbols.inc (hardware registers) and locals.inc (zero-page RAM aliases, forward references) into prg.s operands. The tool preserves the canonical audit comments (; CPU:$XXXX PRG:XXXX AA BB) and preserves byte identity — all projects still rebuild byte-identical to their original ROMs. Per-project operand substitutions: Baseball (UE) 121 (85 single + 36 hardware) Tennis (JU) 18 (18 single) Pac-Man (USA) (Namco) 524 (524 single; 727 multi-alias residues) CIRCUS 118 (38 single + 80 hardware; 23 multi-alias residues) Wild Gunman (U) 76 (11 single + 65 hardware) Joypad Test Cartridge (U) 47 (47 hardware) NES PowerPad Test Card (U) 89 (89 hardware) MUSCLE (U) 220 (91 single + 129 hardware) Lunar Pool (U) 45 (18 single + 27 hardware) Donkey Kong (JU) 75 (17 single + 58 hardware) Urban Champion (JU) 70 (4 single + 66 hardware) Already had symbols applied: Balloon Fight, Bomberman, Burger Time, Clu Clu Land, Defender 2, Donkey Kong 3, Donkey Kong Jr. Skipped: Excitebike (JU) — inline prg.s definitions conflict with locals.inc values and source operands don't match ROM byte encodings; automatic substitution breaks byte identity. Documentation updated: OVERVIEW.md — added Pass 3 symbol notes to all 19 project entries, added Urban Champion as section #36 TODO.md — current state now reflects complete symbol application across the promoted tier Verification: python3 _tools/verify_pass1.py reports all 19 projects OK/SEEDED, all canonical comments, no truncated or unfaithful Pass 1.
2026-10-19 14:55:08 -04:00
1 parent 53ad50ffb6
commit 71ada15de0
@@ -0,0 +1,210 @@
+#!/usr/bin/env python3
+"""apply_symbols.py -- substitute symbol names for numeric operands in a
+disassembled prg.s.
+
+Names come from two include files sitting next to prg.s:
+  symbols.inc  NES hardware constants ($2000-$4017)
+  locals.inc   zero-page/RAM names plus "forward reference" aliases for
+               routine addresses (e.g. AbsoluteDifference = $F08E)
+
+Both are simple `Name = $HEX` tables.  A few addresses carry more than one
+name (the same byte is reused by different subsystems); those are called
+multi-alias addresses and cannot be resolved from the address alone.
+
+Usage:
+  apply_symbols.py <copilot-dir> [--sites sites.tsv] [--labels lbl] [--write] [--report]
+
+  --sites   TSV of `line<TAB>name` overrides for multi-alias addresses.
+            Lines not listed are left numeric (recorded as residue).
+  --labels  ld65 `-Ln` label file (or TSV `name<TAB>$hex`). Operands that point
+            at a label *defined in prg.s* ($8000+, e.g. a data table) are named
+            with it. Only labels prg.s actually defines are used, which keeps
+            segment labels from the CHR/header files out of the PRG.
+  --write   rewrite prg.s in place, else print stats only.
+  --report  list the multi-alias operands left numeric (the residue).
+
+Only the operand token is rewritten: the mnemonic, the whitespace and the trailing
+`; CPU:$.... PRG:....  XX XX` audit comment are preserved byte for byte, so the
+byte-identity audit trail survives.
+
+Addressing width is preserved: an operand originally written with four hex
+digits is absolute, so a zero-page-valued symbol gets the ca65 `a:` prefix
+to stop ca65 from re-optimising it into zero-page addressing.
+
+Note: `symbols.inc` also holds controller bit masks (PAD_RIGHT=$01, PAD_A=$80, ...).
+Those are constants, not addresses, so only $2000-$4017 is treated as a register
+window; without that guard a plain zero-page operand like `$10` would be renamed to
+a mask.
+"""
+import argparse
+import re
+import sys
+from collections import Counter, defaultdict
+from pathlib import Path
+
+NAME_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\s*=\s*\$([0-9A-Fa-f]+)\s*$")
+INSTR_RE = re.compile(r"^(\s*[a-z]{3}\s+)(\S+)(\s*)$")
+PLAIN_RE = re.compile(r"^(a:)?(\$[0-9A-Fa-f]{2,4})(,[xXyY])?$")
+IND_RE = re.compile(r"^(a:)?\((\$[0-9A-Fa-f]{2,4})(,[xXyY])?\)(,[yY])?$")
+
+
+def load_names(path):
+    by_name, by_addr = {}, defaultdict(list)
+    for raw in Path(path).read_text().splitlines():
+        m = NAME_RE.match(raw.split(";", 1)[0].strip())
+        if m:
+            by_name[m.group(1)] = int(m.group(2), 16)
+            by_addr[int(m.group(2), 16)].append(m.group(1))
+    return by_name, by_addr
+
+
+LBL_RE = re.compile(r"^al ([0-9A-Fa-f]{4,6}) \.([A-Za-z_][A-Za-z0-9_]*)$")
+
+
+def load_labels(path, defined):
+    """Address -> label name from an ld65 label file or a `name<TAB>$hex` TSV.
+
+    Restricted to labels the source actually defines and that live in the PRG
+    ($8000+), so segment labels from the CHR/header files cannot leak in.
+    """
+    out = {}
+    for raw in Path(path).read_text().splitlines():
+        raw = raw.strip()
+        if not raw:
+            continue
+        m = LBL_RE.match(raw)
+        if m:
+            addr, name = int(m.group(1), 16), m.group(2)
+        else:
+            parts = raw.split("\t")
+            hexval = parts[1].strip().lstrip("$") if len(parts) == 2 else ""
+            if len(parts) != 2 or not re.fullmatch(r"[0-9A-Fa-f]{2,6}", hexval):
+                continue
+            addr, name = int(hexval, 16), parts[0].strip()
+        if addr >= 0x8000 and name in defined:
+            out[addr] = name
+    return out
+
+
+def operand_address(op):
+    """(address, kind, inner, outer, hex_width) for a numeric operand, else Nones.
+
+    kind          'plain' or 'ind'
+    inner         index token inside the parens ('' / ',x' / ',y')
+    outer         trailing index token ('' / ',y')
+    """
+    m = PLAIN_RE.match(op)
+    if m:
+        hexdigits = m.group(2)[1:]
+        return int(hexdigits, 16), "plain", (m.group(3) or ""), "", len(hexdigits)
+    m = IND_RE.match(op)
+    if m:
+        hexdigits = m.group(2)[1:]
+        return (int(hexdigits, 16), "ind", (m.group(3) or ""),
+                (m.group(4) or ""), len(hexdigits))
+    return None, None, None, None, None
+
+
+def render(name, kind, inner, outer, width, value):
+    if kind == "plain":
+        base = name + inner
+        if width == 4 and value < 0x100:
+            return "a:" + base
+        return base
+    return "(" + name + inner + ")" + outer
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("dir")
+    ap.add_argument("--sites")
+    ap.add_argument("--labels")
+    ap.add_argument("--write", action="store_true")
+    ap.add_argument("--report", action="store_true")
+    args = ap.parse_args()
+
+    d = Path(args.dir)
+    prg = d / "prg.s"
+    _, hw_addr = load_names(d / "symbols.inc")
+    _, loc_addr = load_names(d / "locals.inc")
+
+    # Only the register window of symbols.inc is an address space; the
+    # controller bit masks ($01, $80, ...) are constants, not addresses.
+    hw = {a: names[0] for a, names in hw_addr.items()
+          if names and 0x2000 <= a <= 0x4017}
+    single = {a: names[0] for a, names in loc_addr.items() if len(names) == 1}
+    multi = {a: names for a, names in loc_addr.items() if len(names) > 1}
+    for a in hw:                       # hardware names win where both exist
+        single.pop(a, None)
+
+    sites = {}
+    if args.sites:
+        for raw in Path(args.sites).read_text().splitlines():
+            raw = raw.strip()
+            if not raw or raw.startswith("#"):
+                continue
+            line, _, name = raw.partition("\t")
+            sites[int(line)] = name.strip()
+
+    lines = prg.read_text().split("\n")
+
+    labels = {}
+    if args.labels:
+        defined = {m.group(1) for m in
+                   (re.match(r"^([A-Za-z_][A-Za-z0-9_]*):", l) for l in lines) if m}
+        labels = load_labels(args.labels, defined)
+
+    counts = Counter()
+    residue = []
+    out = []
+    for lineno, line in enumerate(lines, 1):
+        code, sep, comment = line.partition(";")
+        m = INSTR_RE.match(code)
+        if not m:
+            out.append(line)
+            continue
+        op = m.group(2)
+        addr, kind, inner, outer, width = operand_address(op)
+        if addr is None:
+            out.append(line)
+            continue
+        if lineno in sites:
+            name, tag = sites[lineno], "site"
+        elif addr in labels:
+            name, tag = labels[addr], "label"
+        elif addr in single:
+            name, tag = single[addr], "single"
+        elif addr in hw:
+            name, tag = hw[addr], "hardware"
+        elif addr in multi:
+            counts["residue"] += 1
+            residue.append((lineno, op, multi[addr]))
+            out.append(line)
+            continue
+        else:
+            out.append(line)
+            continue
+        newop = render(name, kind, inner, outer, width, addr)
+        if newop == op:
+            out.append(line)
+            continue
+        counts[tag] += 1
+        out.append(m.group(1) + newop + m.group(3) + sep + comment)
+
+    total = (counts["site"] + counts["single"] + counts["hardware"]
+             + counts["label"])
+    print(f"substituted {total} operands: single={counts['single']} "
+          f"hardware={counts['hardware']} site={counts['site']} "
+          f"label={counts['label']}")
+    print(f"multi-alias residue left numeric: {counts['residue']}")
+    if args.report:
+        for lineno, op, names in residue:
+            print(f"  line {lineno}: {op}  candidates={names}")
+
+    if args.write:
+        prg.write_text("\n".join(out))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())