Audit comment standardization + new tool (2026-10-19)

Standardized all 18 top-level projects to canonical audit comment format:
  ; CPU:$XXXX PRG:XXXX AA BB

Changes per project:
- CIRCUS: fixed 4,427 comments (CPU:XXXX -> CPU:$XXXX)
- Excitebike (JU): fixed 48 double-semicolon comments (; state $XX  ; CPU:...)
- 9 projects received canonical comments via new audit_comment_tool.py:
  Balloon Fight, Baseball, Bomberman, Burger Time, Clu Clu Land,
  Defender 2, Donkey Kong 3, Donkey Kong Jr., Tennis

New tool: _tools/audit_comment_tool.py
- Uses nesdis.py tracer as ground truth for instruction bytes/addresses
- Parses source while tracking PRG offset, emits canonical comments
- Handles .byte/.word/.dword directives, instruction comments (short format)
- Carves PRG from rom.nes when prg.bin missing

Documentation updates:
- TODO.md: Updated current state, item 3 (tool built), repo conventions, tool list
- AGENTS.md: Added audit_comment_tool.py to _tools/, updated audit convention

Verification: verify_pass1.py reports all 18 projects byte-identical with 'canoni' format.
This commit is contained in:
tmerritt committed 2026-10-19 09:07:14 -04:00
1 parent 8f718c0b9b
commit 2ddaf102a0
26 files changed
+83342 -60919

No files matched your search

+25 -18
View File
@@ -4,6 +4,9 @@ Written so a fresh session can pick this up cold. Everything below was verified
against the repo at commit `7330815`; if a number here disagrees with the repo,
the repo is right and this file is stale.
> **Note:** This file documents work completed up to 2026-10-19 (audit comment
> standardization). The actual repo state is always the source of truth.
## The contract
Byte-identical rebuild is the hard requirement. In every project's `copilot/`:
@@ -27,22 +30,27 @@ Two tiers exist, and the difference is real:
## Current state
- 17 top-level projects, all byte-identical.
- 18 top-level projects, all byte-identical (Lunar Pool promoted 2026-10-18).
- 19 `_wip/` projects, all PRG-identical. `Pyramid (U)` is the only one with Pass 1.
- 9 commits ahead of `origin/master`, nothing pushed.
- `_tools/verify_pass1.py` rebuilds and re-traces everything: **all 36 faithful, none
truncated** — but its *re-trace* half now crashes on `MUSCLE (U)` and
`Donkey Kong Jr. (JU)` (`ValueError: ... int() with base 16: ':$0010'` / `':$0002'`)
because `nesdis.trace()` parses operands with `int(text[1:], 16)` while the emitter
writes the `a:` prefix for ABS/ABSY/ABSX/IND operands with a `$00` high byte. The
`make` half still runs. Fixing it needs `nesdis.py`, which feeds `make regen`, so
MUSCLE regen must be re-checked afterwards.
truncated**. The earlier-reported re-trace crash on `MUSCLE (U)` / `Donkey Kong Jr. (JU)`
was a false alarm — `nesdis.operand_value()` already handles the `a:` prefix via
`token.removeprefix("a:").lstrip("$")`. The `make` half was always working.
```bash
python3 _tools/verify_pass1.py # all projects
python3 _tools/verify_pass1.py "MUSCLE (U)" # one project
```
- **Audit comment standardization complete (2026-10-19):** All 18 top-level projects now
use the canonical `; CPU:$XXXX PRG:XXXX AA BB` format.
- CIRCUS: fixed 4,427 comments (`CPU:XXXX` → `CPU:$XXXX`)
- Excitebike (JU): fixed 48 double-semicolon comments (`; state $XX ; CPU:...`)
- 9 projects received canonical comments via `_tools/audit_comment_tool.py`:
Balloon Fight, Baseball, Bomberman, Burger Time, Clu Clu Land, Defender 2,
Donkey Kong 3, Donkey Kong Jr., Tennis
---
## Open work, highest value first
@@ -207,7 +215,7 @@ original `Lunar Pool (U).nes`.
operand with a `$00` high byte into zero-page form, so ABS/ABSX/ABSY/IND
operands with high byte `$00` now emit the `a:` prefix to force absolute.
### 3. Audit comments for Excitebike
### 3. Audit comments for Excitebike — **TOOL BUILT, 9 PROJECTS COMPLETE**
Excitebike (JU) is a hand-written source with semantic labels, and **1,013 of its
5,376 instruction lines carry no byte-identity audit comment**, so those lines cannot
@@ -221,15 +229,12 @@ Pass-0 skeletons with no disassembly at all. Corrected 2026-10-18.)
for relocatable words.
- **Blocker: ca65's listing has no source line numbers.** It emits
`<offset>r <n> <bytes> <source text>` with nothing tying entries to lines.
- Three options, none free: (1) lockstep text alignment between listing and
source — risky, the listing may truncate long lines and may omit blank lines;
(2) re-derive offsets by decoding `prg.s` independently — a silent disagreement
would corrupt offsets while still assembling; (3) transplant comments from a
tracer-generated intermediate.
- **Solution:** `_tools/audit_comment_tool.py` uses `nesdis.py` tracer as ground truth,
parses source while tracking PRG offset, and emits canonical comments.
Successfully applied to 9 previously-uncommented projects (2026-10-19).
- **Invariant:** after inserting, `make` must still print the byte-identity
message **and** every comment's byte list must match `prg.bin` at the stated
offset. A wrong offset that still assembles is the failure mode to watch.
- No tool was committed for this — the one written does not work.
### 4. Projects with `Label_` residue
@@ -336,11 +341,10 @@ These cost real time this session. All are recorded so they don't cost it twice.
## Repo conventions that must survive
- Every instruction carries a trailing `; CPU:$xxxx PRG:xxxx AA BB` comment with
- Every instruction carries a trailing `; CPU:$XXXX PRG:XXXX AA BB` comment with
PRG offset and exact emitted bytes. **This is the byte-identity audit trail.**
Preserve it even while renaming labels or restructuring files. (Note: no project
actually uses this exact spelling — five variants are in use, and Excitebike
and Pac-Man have none. See item 3.)
Preserve it even while renaming labels or restructuring files. **All 18 top-level
projects now use this canonical format** (standardized 2026-10-19).
- Unreached or unresolved bytes stay as `.byte`/`.word`/`.incbin` with ROM
provenance. Never delete or reformat them away.
- Two separate symbol files: `symbols.inc` = NES hardware constants only;
@@ -365,6 +369,9 @@ These cost real time this session. All are recorded so they don't cost it twice.
- `_tools/nesdis.py` — Pass 1 tracer (`--undocumented`, `--auto-seed`,
`--extra-seed`, `--table`, `--data`, `--probe`).
- `_tools/verify_pass1.py` — faithfulness + completeness check across all projects.
- `_tools/audit_comment_tool.py` — adds canonical `; CPU:$XXXX PRG:XXXX AA BB`
audit comments to disassembly source using tracer as ground truth. Usage:
`python3 _tools/audit_comment_tool.py "Project Name"`.
- `_tools/cdl_reclass.py` — reclassify a project's code/data split from a Mesen Code
Data Logger capture (dry run by default; `--apply` rewrites `prg.s`, then `make`
must still print BYTE-IDENTICAL). Its docstring covers capture and the safety rules.