tmerritt/disasm
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Xevious (U): Pass 1 disassembly

Browse Source 
The first _wip project to get a real disassembly.  4,165 instructions /
8,444 code bytes (25.8%), 24,324 data bytes, 0 unclassified -- every byte
of the 32 KB PRG is now classified as code or data.

The split needed three stages, all recorded in prg.s's header:

  * nesdis.py --undocumented --auto-seed at base $8000 gave 3,272
    instructions / 6,555 bytes.  The undocumented opcodes matter here: the
    trace stops dead at the first one it cannot decode.
  * Four hand-seeded entries ($C08B, $DC1E, $E0D5, $E134).  Each is a
    branch target from decoded code that sat inside an untraced data run,
    so the tracer never reached it.
  * A merged four-session Mesen Code Data Logger capture (kept as
    xevious_mesen.cdl) promoted 1,902 bytes from data to code and
    demoted 22 the trace had called code.

Two tool bugs fell out and are fixed here:

  * nesdis.py emitted the undocumented single-byte NOPs ($1A/$3A/$5A/
    $7A/$DA/$FA) as `nop`, which ca65 assembles as the *official* $EA --
    the rebuild stopped being byte-identical at $DF82.  The check now
    keys on the opcode byte, not just the addressing mode.  All three
    `make regen` pipelines (MUSCLE, Pyramid, Urban Champion) are
    unaffected and still reproduce prg.s byte for byte.

  * cdl_reclass.py left the original data-block header in place when the
    CDL split that block, so the file ended up with two region headers
    describing overlapping spans (20 of them here).  The header is now
    dropped with the block it describes; the promoted projects that use
    the tool are byte-for-byte unchanged.

The project README has the power-on flow analysis.  The most interesting
find is at $FC45: the reset path compares RAM $0100-$0107 against the
ASCII string "Kei Cros", and the copyright text at $FF1E reads
"MAIN: KEI CROSS.  SOUND: UHOHOY OHNOGI."  The OAM is at $0700, not
$0200 -- the NMI handler does LDA #$07 / STA $4014.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add explicit `make wip` / `make wipclean` targets for the _wip tier

The root Makefile's all / clean / <MAPPER> targets deliberately name only
the promoted mapper folders, so nothing in them reaches into _wip/.  Give
the pre-promotion tier its own opt-in targets instead of the manual
`for d in _wip/*/*/; do make -C "$d"; done` loop:

  make wip          # build + verify every _wip/<MAPPER>/<Project>/
  make wipclean     # clean the build artifacts in that tier

Both the Makefile and the four documents that describe the tier (README,
TODO, WIP, AGENTS) are updated together, so the tree stays consistent.

The same commit records Xevious (U) reaching Pass 2 in those documents:
all 487 labels named, every branch and call symbolic, zero Label_XXXX and
zero raw $XXXX operands left.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Update the _wip/ tier documentation; fix the make wip target and the mirroring bug

The WIP tier's four documents (WIP.md, README.md, TODO.md, AGENTS.md) had
drifted away from the tree.  Every number in them was re-derived from the
ROMs and the folder layout rather than carried forward.

Counts and structure
  - The tier holds 854 projects across 29 mapper folders, not "856 across
    30".  Xevious (U) and Zanac (U) were promoted on 2026-10-20 and their
    _wip/ folders were left behind as residue -- "git mv" moves only the
    tracked files, so the old folder survived with its build artifacts and a
    stale README.  Both are removed now; the promotion procedure documents
    the step so it does not recur.
  - _wip/MAPPER8/ holds a copy of the already-promoted Mermaids of
    Atlantis (U) and is excluded from every count and table.
  - Pass 0 skeletons: 852 (was 855).  Xevious (U) left the tier, so the
    two projects with a real disassembly are Golf (JU) and Volleyball (U).
  - PRG sizes 2 / 150 / 42 / 522 / 127 / 10 / 1; 226 projects with no CHR;
    38 keep the CPU-address linker layout; 37 are mapper 0.
  - Every inventory row was re-checked against its ROM: PRG/CHR size,
    mirroring, all three vectors, and the battery / four-screen / footer
    notes.  854 of 854 rows now match.

Fixes
  - The root Makefile used $(WIP) but never defined it, so "make wip" and
    "make wipclean" expanded to "for m in /*/" -- they walked the
    filesystem root instead of the tier.  "WIP := _wip" is now defined, and
    "make wip" builds and verifies the whole tier: 855 of 855 passed, 0
    failures.
  - make_skeleton.py masked header byte 6 with 0x03, so the battery bit
    (bit 1) leaked into the mirroring value and 56 battery-backed ROMs'
    READMEs claimed "Four-screen" while their own "Four-screen VRAM" row
    said "no".  The mask is now 0x01 and the 56 READMEs are corrected.
  - Golf (JU)'s README still described the pre-flatten copilot/ folder and
    the old PRG-only compare; it now describes the whole-ROM build.
  - Pyramid (U)'s audit comments are canonical again, so Golf (JU) is the
    only project in the tree still on the pre-canonical spelling.

New in WIP.md
  - Byte 7's lower nybble is non-zero on 58 of the ROMs -- 55 carry the
    literal "DiskDude!" watermark, two "DisNi0330", one "byKH+KEN".  Only
    the upper nybble carries mapper bits, so the noise is harmless.
  - The twelve ROMs that set the four-screen bit, split by whether their
    board actually supports four-screen VRAM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Promote Volleyball (U) from _wip/NROM/ to NROM/

Volleyball (U) reached Pass 3 before promotion, so this is a promotion of a
finished disassembly rather than a Pass 0 skeleton.

The disassembly
  - 11,395 instructions / 23,926 code bytes (73.0 % of the PRG image),
    8,842 data bytes, 0 unclassified.
  - The split needed the game-state dispatch table at $85E1 declared to the
    tracer (--table 85E1:11) -- without it only 66 % of the image traced --
    plus a Mesen Code Data Logger capture taken while a random-input bot
    played the attract demo, which promoted 1,757 bytes from data to code.
  - Pass 2 named all 1,989 labels: 730 read out of the code by hand, 1,221
    internal branch targets derived mechanically.  Zero Label_XXXX, zero raw
    $XXXX call or branch operands, zero SubC residue.
  - Pass 3 named the zero page, the $0500 ball structure and all 37 data
    blocks.  apply_symbols.py --report: 0 pending substitutions, 0
    multi-alias residue.  align_audit_comments.py: no column drift.
  - cdl_check.py cross-checks the finished prg.s against the capture: 0
    CDL-code bytes not covered by an instruction, 0 CDL-data bytes that
    start one.
  - 29 neutral ZpXX zero-page slots remain, each with its reason recorded
    in locals.inc, as the promotion rules allow.

The promotion
  - git mv "_wip/NROM/Volleyball (U)" "NROM/Volleyball (U)".  Mapper 0, so
    it lands in NROM/.  The whole-ROM build was already in place, so there
    was no build surgery; the project builds byte-identically at its new
    home and the old folder is gone (git mv moves the untracked build
    artifacts too, so no residue was left behind).
  - Root README.md: new inventory row 39 and a per-project detail section,
    plus the WIP-tier counts.  The detail sections for Xevious (U) and
    Zanac (U) were numbered one too high against the inventory rows and
    are now aligned; a leftover editing note in the same place is gone.
  - WIP.md: the Volleyball row leaves the inventory, and the tier is now
    853 projects across 29 mapper folders (36 in _wip/NROM/), 852 of them
    Pass 0 skeletons.  Golf (JU) is the only project left in the tier with
    a real disassembly.
  - TODO.md and AGENTS.md: the same counts, and the promoted tier is now
    40 projects (39 under NROM/).

Verified: make wip builds and verifies the whole tier (854 of 854 passed,
0 failures) and make builds the promoted tier (39 of 39 passed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Rename NROM/Pyramid (U) to NROM/Pyramid (U) (Sachen)

The promoted inventory row had read "Pyramid (U) (Sachen)" since the
project was promoted on 2026-10-20, but the folder was named "Pyramid (U)" --
so the row pointed at a folder that did not exist.  The folder now matches
the row, and the project's own README title, its ROM file name
("Pyramid (U) (Sachen).nes") and the parenthetical convention the rest of
the promoted tier already uses (Popeye (JU) (PRG 0/1), Raid on Bungeling
Bay (U) [a2], Pac-Man (Tengen) (U)).

What changed inside the project
  - The Makefile's regen target passes --project "Pyramid (U) (Sachen)", so
    the tracer writes the matching header line.  prg.s and locals.inc were
    updated to that header line and nothing else -- the disassembly is
    byte-identical to before.
  - make regen still reproduces prg.s and locals.inc byte for byte.
  - The project's README's build-verification line now names the new folder.

References updated: the root README.md (inventory prose, the row-33 detail
section, the build-status table, and the notable-observations bullet),
TODO.md (the promoted-projects table and the Pyramid notes) and AGENTS.md
(the recently-promoted list and the unlicensed-titles quirk).

Verified: make builds the promoted tier (39 of 39 passed) and the project
rebuilds byte-identically at its new path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Rebuild Mermaids of Atlantis (U): bank-aware Pass 1, Pass 2/3 names

The promoted project did not build: a Pass 2 rename had been applied
half-way, leaving eight undefined symbols and two jumps that pointed at
the wrong routine, while the root README already claimed the project was
byte identical.  The _wip copy still built, because it held the un-renamed
Pass 1 source.

The disassembly is now derived from the ROM itself, with the Mesen Code
Data Logger as the oracle for the code/data split:

* _tools/banked_dis.py traces every MMC3 bank separately, each with the CPU
  window it is visible at, seeded from a CDL capture so the code a static
  trace cannot reach is disassembled too.  The CDL is the authority: a
  decode the CPU never fetched is dropped, and so is the fall-through
  behind a call whose target never returns (the game's dispatchers have
  their jump table right behind the call).
* One segment per bank (PRG0..PRG7) with its own .org, so the assembler
  knows the CPU address of every label while the linker still writes the
  bytes to the bank's file offset.  A flat segment cannot do both: a call
  would encode the file offset instead of the CPU address.
* Pass 2 names every label - 494 routines, no Label_XXXX and no raw
  $XXXX branch or jump operand left.  Pass 3 applies 13 semantic zero-page
  aliases and writes the routine headers.
* 8,075 bytes of code (12.3% of the 64 KB PRG), up from 200 bytes.

The banking turned out to be unusual and is documented: the fixed bank is
the fourth 8 KB bank (file $6000-$7FFF), not the last one, and the image
carries the reset stub and the vector table twice.  The live NMI vector
is $808F - the frame handler - while the copy at the end of the image
names $8189, which is a BRK in the first half.  The game's text table
lives at CPU $F1D6, which is the fixed window, and that is what settles
the question.

Also: the leftover _wip/MAPPER8 folder is removed, as it is for every
other promotion.

The project builds byte identical:

    Byte-identical ROM verified.

Add CHANGELOG.md: the pass history of every project

A project goes through four milestones - Pass 1 (the code/data split),
Pass 2 (the labels replaced by names), Pass 3 (the semantic naming) and
Promotion (the move into the top-level mapper folder) - and the history
of those milestones was scattered over the commit log, the project
READMEs and the root README.

CHANGELOG.md gathers it: one section per project with a row for each
milestone, the date, and a sentence saying what the milestone produced.
The numbers are the ones the project README and the root README already
carry - instruction count, code coverage, named labels, semantic
zero-page aliases, named data tables - so a reader can see at a glance
which projects are done and which are not.  Golf (JU), the only project
still in _wip/ with a real disassembly, is listed with its open
milestones, and the other 852 _wip/ projects are covered as a tier.

The requirement is now part of the workflow contract: PROMPTS.md gains a
"Pass 3: Changelog entry" section and a ninth checklist item, and
AGENTS.md points at the file.  A milestone and its changelog row go in
the same commit, so the changelog and the source never disagree.

Balloon Fight (JU): repair the audit trail and complete the routine headers

The per-instruction audit trail was stale by two bytes from
ResetInitializationData ($C085) onward.  5,944 of the 6,007 comments named
the wrong CPU address and PRG offset, because the three-byte .byte "HAL"
signature at $C082-$C084 had been counted as one byte when the trail was
first generated - the routine headers and the linker labels were right all
along, so the source and its own headers disagreed with each other.  The
trail was rebuilt from the ca65 -l listing with
_tools/repair_audit_comments.py, and every rewritten comment was re-checked
against prg.bin.  The comments were also moved into the canonical column-35
field: they had sat two spaces after the mnemonic, with no column discipline
at all, unlike every other promoted project.

All 63 routine headers now carry a ; Clobbers: block, so the headers carry
the full template AGENTS.md describes.  The blocks are derived, not written by
hand.  The new _tools/derive_clobbers.py takes the opcode from the audit
comment's byte list, so the addressing mode is exact and a data directive is
never mistaken for an instruction; it resolves the operand through
locals.inc and symbols.inc to name what is written; it follows the real
control flow, so a jsr to another named entry is a call (the callee's
clobbers count) while a branch or a fall-through is not; and it models
register preservation, so NmiHandler reports *no register* - it saves and
restores A, X and Y around the frame - and FlushPpuCommandPreserved reports
only A, because X and Y come back unchanged.

Two tools changed.  derive_clobbers.py is new.  align_audit_comments.py had
an off-by-one: it tested the indented code length against 31 instead of
against 35, so code texts of 28-31 characters were never padded into the
column.

The change is comments only.  The code text of all 8,538 lines is unchanged,
make still prints "Byte-identical ROM verified.", the 6,007 audit comments
still re-check against prg.bin, all 63 routine headers agree with the
instructions they describe, and every label agrees with reconstructed.lbl.
ReflectObjectYVelocity and UpdateSpecialObjectSystem moved out of locals.inc
and became labels in prg.s at their real addresses.

CHANGELOG.md becomes the repository's running log: a Pending list of the
changes to be put in the next commit, a History of the commits already made,
and the per-project milestone index at the end.  The next commit's message is
written from the Pending entries, so the two cannot drift apart.  PROMPTS.md
and AGENTS.md describe the new shape.  WIP.md drops the _wip/MAPPER8/ note:
the folder held a copy of Mermaids of Atlantis (U) that had already been
promoted.

World Class Track Meet (U): Pass 1, the code/data split

Pass 1 is done.  The split is 8,400 instructions / 18,028 code bytes (55.0 %)
of the 32 KB PRG, with 884 routine entries, 55 data blocks and 102 zero-page
aliases.  The PRG is CNROM (mapper 3) and maps whole at $8000-$FFFF, so the
linker moved the PRG segment from a .nes file offset to $8000 and the build
still compares byte identical.

Four dispatch tables had to be seeded by hand.  The event table at $F796 (5
entries) is read at $F780 and jumped to through `jmp ($001F)`.  The three
push-pointer tables - $BFC4 (16 entries), $BFE4 (2) and $E5F2 (7) - push a
handler address and `rts` into it, and the 6502's `rts` adds one to the
address it pops, so each table stores the entry address minus one; the seeds
are the table values plus one.  The tracer runs with --undocumented: the code
uses undocumented NMOS opcodes as ordinary code (ANC $37 at $BFF8, and the
RRA/RLA/SLO and NOP forms elsewhere).  Seeding the tables takes the coverage
from 27.8 % to 55.0 %.

$C0BE-$C0C5 is forced back to data: the IRQ vector $C0C0 points into the
middle of the `jmp $C047` at $C0BE, and the game never executes CLI, so the
vector is dead.  `make pass1` regenerates prg.s and locals.inc from prg.bin
and reproduces the committed prg.s byte for byte.

The project README records the split, the four dispatch tables, the dead IRQ
vector, the wrong CHR size field in the header, and what is still data.  The
milestone index in CHANGELOG.md gains the project; WIP.md, README.md and
AGENTS.md follow the tier counts and the layout.

nesdis.py gains a --mapping-note option, so the header's "PRG mapped at" line
can describe a board that is not NROM - CNROM's whole 32 KB at $8000-$FFFF is
not "mirrored to the upper half".

Replaces runs of $00 and $FF with .res <run>, <byte>

Golf (JU) and World Class Track Meet (U): Pass 2/3 complete, promoted from _wip/

- World Class Track Meet (U) - Pass 2 complete: all 884 Label_<addr>
  definitions replaced (651 semantic routine names + 233 Sub_<addr>), all 731
  raw $XXXX jsr/jmp call operands resolved to symbolic labels, all branch
  targets given role-based names; routinemap.tsv added; make stays
  byte-identical.
- World Class Track Meet (U) - Pass 3 complete: 24 semantic zero-page
  aliases applied via apply_symbols.py (0 residue), 78 neutral ZpXX slots
  documented with reasons in locals.inc, all hardware-register operands use
  symbols.inc names, project README rewritten with Pass 2/Pass 3 sections and
  data-table inventory; promoted out of _wip/CNROM/.
- World Class Track Meet (U) - project README documents the five event-table
  entries at $F796 and marks the olympics mode as not yet proven.
- Golf (JU) - audit comments converted from pre-canonical (; C000: A9 00)
  to the canonical ; CPU:$C000 PRG:0000  A9 00 form at column 35; 4,156
  comments re-aligned, comment/whitespace only, whole-ROM build stays
  byte-identical.
- Golf (JU) - 339 internal branch-target Label_ labels given role-based names
  via _tools/derive_branch_names.py; 0 raw $XXXX branch/jump operands remain.
- Golf (JU) - 275 zero-page operands substituted with locals.inc aliases via
  apply_symbols.py; 0 residue; promoted out of _wip/NROM/.
- Xevious (U) - runs of $FF padding replaced with .res <count>, $FF in prg.s;
  whole-ROM build stays byte-identical.
- Both projects promoted via git mv; leftover _wip/ folders removed; root
  README.md inventory rows added; WIP.md rows removed.

World Class Track Meet (U): apply zero-page symbol substitutions

Run apply_symbols.py to substitute ZpXX zero-page operands with
semantic locals.inc aliases (SlotState, GameMode, etc.) at proven
sites; 0 residue, whole-ROM build stays byte-identical.

Xevious (U): re-split Pass 1 from the Mesen CDL and decode the data tables

- Xevious (U) - Pass 1 re-split from the Mesen CDL (xevious_mesen.cdl) with
  _tools/cdl_reclass.py: 1,158 bytes data->code, 48 code->data, 42 labels
  emitted and 3 branch targets labelled; code 11,321->12,455 bytes
  (34.5%->38.0%), instructions 5,513->6,002. The 45 recovered entry points
  are named Sub_XXXX, so no Label_XXXX remains; six CDL-silent raw call
  operands are documented as residue.
- Xevious (U) - the 4,448-byte block at $A74B-$B8AA is split into six proven
  tables: VramWritePtrTable (15 pointers), VramWriteData, DataStreamPtrTable
  (16), ObjTilePtrTable (99), ObjTileData and DataStreamData. The $D833
  pointer table is corrected to base $D833 (4 entries; the old ObjCodePtrTable
  at $D834 was one byte late). $F957-$FAF5 is split into ObjSpeedTable,
  ObjSpeedTable2, SpriteLayoutPtrTable (19) and SpriteLayoutData;
  $FEE6-$FF1C into PaletteData and OamAttrTable. $F5D8 is decoded as
  TextTileData (font tile indices: PLAYER, GAME OVER, READY, SOLVALOU, LEFT,
  PAUSE); $F53C becomes AttrAddrTable, $F89E HandlerPtrTable and $EF3B
  SpriteLayoutData2. Project README rewritten with the corrected stats, a
  32-entry table inventory and the Pass 1 residue.
- _tools/cdl_reclass.py - preserve the data label(s) between a data-block
  header and its first .byte row when the block stays data; the $DF78 split
  dropped Data_DF78 and the rebuild failed with an undefined symbol.
- _tools/resolve_raw_branches.py - handle negative relative offsets
  (beq *-$0B); the pattern only matched *+$NN and *$NN.

Docs: refresh the tier counts to 41 promoted / 851 _wip, add the WCTM row

- Docs - tier counts refreshed to the post-promotion state across AGENTS.md,
  root README.md, WIP.md, TODO.md and CHANGELOG.md: 41 promoted projects
  (38 under NROM/, plus CNROM/World Class Track Meet (U), UNROM/Zanac (U) and
  MAPPER8/Mermaids of Atlantis (U)) and 851 _wip/ projects, every one of them
  a Pass 0 skeleton. The stale "39/40 promoted", "853 _wip", "Golf (JU) is
  still in _wip/" and "Golf is the last pre-canonical project" statements are
  corrected.
- Root README.md - the promoted inventory gained the missing World Class Track
  Meet (U) row (row 41, mapper 3, 32 KB PRG / 32 KB CHR), and the "All 38 are
  mapper 0 / 16 KB PRG" sentence is replaced with the actual split (36 of the
  41 rows are mapper 0 / 16 KB; Xevious and Volleyball are 32 KB, World Class
  Track Meet is mapper 3, Zanac mapper 2 and Mermaids mapper 8).
This commit is contained in:
tmerritt committed 2026-10-21 11:36:32 -04:00
1 parent df11857075
commit 691b2bc23c
198 files changed
+84517 -25902
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

_tools/derive_clobbers.py
+598
View File
@@ -0,0 +1,598 @@
#!/usr/bin/env python3
"""Derive the `; Clobbers:` line of every routine header in a project's prg.s.
The repo's routine-header template (AGENTS.md) is "global + local description,
parameters, returns, clobbers". A routine's *clobbers* are what the caller
loses across the call: the registers the routine writes, the processor flags
it changes, and the memory it writes - plus the same for the routines it
calls, because a callee's damage survives the return.
Everything here is derived from the source itself, not guessed:
* the opcode comes from the audit comment's byte list, so the addressing
mode is exact (`sta $0400,x` is ABSX, `sta Zp00` is ZP) and a data
directive is never mistaken for an instruction;
* the operand text is resolved through `locals.inc`, `symbols.inc` and the
labels defined in `prg.s`, so a clobber can be named;
* the control flow comes from the instructions themselves. A *unit* is a
named entry point in the source (a label that is followed by code); the
code a unit reaches - the shared tails it branches into, the block it
falls through into - is its own, while a `jsr`/`jmp` to another unit is a
call, so the callee's clobbers count but its code is not merged in.
Register *preservation* is modelled too, so a routine that saves a register on
the stack and puts it back before returning (`tya`/`pha` ... `pla`/`tay`) is
not reported as clobbering it. A small abstract interpreter carries the value
of A, X, Y and of each processor flag through the unit: a register survives
when its value at the end is the value it had at the entry, and a flag
survives when a `php`/`plp` pair puts it back.
A call into code the project has not decoded (a forward reference in
`locals.inc` that lands in a data region) can damage anything, so it is
reported as a clobber of every register and every flag.
Usage:
python3 _tools/derive_clobbers.py "NROM/Balloon Fight (JU)" # report
python3 _tools/derive_clobbers.py "NROM/Balloon Fight (JU)" --write # apply
"""
import argparse
import os
import re
import sys
from pathlib import Path
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import nesdis # noqa: E402
# Registers written by each mnemonic.
REG_WRITE = {
"lda": "A", "pla": "A", "txa": "A", "tya": "A", "lax": "A",
"ldx": "X", "inx": "X", "dex": "X", "tax": "X", "txs": "X",
"ldy": "Y", "iny": "Y", "dey": "Y", "tay": "Y",
}
# Which processor flags each mnemonic sets or clears.
FLAG_BITS = {
"N": ("lda", "ldx", "ldy", "tax", "tay", "txa", "tya", "tsx", "cmp",
"cpx", "cpy", "adc", "and", "asl", "bit", "dec", "dex", "dey",
"eor", "inc", "inx", "iny", "lsr", "ora", "rol", "ror", "sbc"),
"Z": ("lda", "ldx", "ldy", "tax", "tay", "txa", "tya", "tsx", "cmp",
"cpx", "cpy", "adc", "and", "asl", "bit", "dec", "dex", "dey",
"eor", "inc", "inx", "iny", "lsr", "ora", "rol", "ror", "sbc"),
"C": ("adc", "cmp", "cpx", "cpy", "sbc", "sec", "asl", "lsr", "rol", "ror"),
"D": ("cld", "sed"),
"I": ("cli", "sei", "plp", "brk"),
"V": ("adc", "sbc", "bit", "plp"),
}
FLAGS = list(FLAG_BITS)
# Mnemonics that write their operand.
MEM_WRITE = {"sta", "stx", "sty", "inc", "dec", "asl", "lsr", "rol", "ror"}
# Mnemonics that only move a register around.
COPY = {"txa": ("X", "A"), "tya": ("Y", "A"), "tax": ("A", "X"),
"tay": ("A", "Y")}
# Instructions that end the routine.
TERMINATORS = {"rts", "rti", "brk"}
AUDIT_RE = re.compile(
r"^ ([A-Za-z@][A-Za-z0-9_]*)\s*(.*?)\s*; CPU:\$([0-9A-F]{4}) "
r"PRG:([0-9A-F]{4}) ((?:[0-9A-F]{2} ?)+)$")
LABEL_RE = re.compile(r"^([A-Za-z_@][A-Za-z0-9_]*):\s*(.*)$")
ENTRY = {r: ("entry", r) for r in ("A", "X", "Y")}
# Memory regions, named rather than listed address by address.
REGIONS = (
(0x0200, 0x02FF, "the OAM shadow"),
(0x4000, 0x4017, "the APU registers"),
)
class Insn:
__slots__ = ("addr", "length", "mn", "operand", "mode", "opcode", "bytes",
"succ", "calls", "unknown_call")
def __init__(self, addr, length, mn, operand, mode, opcode, raw):
self.addr = addr
self.length = length
self.mn = mn
self.operand = operand
self.mode = mode
self.opcode = opcode
self.bytes = raw
self.succ = [] # (kind, instruction index); kind "fall"/"branch"
self.calls = [] # instruction index of a called unit's entry
self.unknown_call = False
class Unit:
"""A named entry point in the source and the code it reaches."""
def __init__(self, name, entry, addr):
self.name = name
self.entry = entry
self.addr = addr
self.reach = set()
self.calls = set() # units called
self.unknown_call = False
self.touched = set()
self.restored = set()
self.flag_written = set()
self.flag_restored = set()
self.mem = []
# transitive results
self.eff_regs = set()
self.eff_flags = set()
self.eff_mem = set()
self.eff_calls = set()
class Block:
"""A routine header in prg.s."""
def __init__(self, name, hdr_lines, entry_addr):
self.name = name
self.hdr_lines = hdr_lines
self.entry_addr = entry_addr
self.unit = None
def load_symbols(proj):
"""Return (name -> address, address -> [names in file order])."""
syms = {}
order = {}
for fname in ("locals.inc", "symbols.inc"):
p = proj / fname
if not p.is_file():
continue
for raw in p.read_text().split("\n"):
line = raw.split(";", 1)[0].strip()
if "=" not in line:
continue
name, _, val = line.partition("=")
name = name.strip()
val = val.strip()
if not val.startswith("$"):
continue
try:
addr = int(val[1:], 16)
except ValueError:
continue
syms.setdefault(name, addr)
order.setdefault(addr, []).append(name)
return syms, order
class Project:
def __init__(self, proj):
self.dir = Path(proj)
self.src = (self.dir / "prg.s").read_text().split("\n")
self.syms, self.alias_order = load_symbols(self.dir)
self.insns = []
self.by_addr = {}
self.labels = {}
self.units = {} # cpu address -> Unit
self.blocks = []
self._parse()
def _parse(self):
for line in self.src:
m = AUDIT_RE.match(line)
if not m:
continue
mn = m.group(1).lower()
operand = m.group(2).strip()
by = m.group(5).split()
opcode = int(by[0], 16)
entry = nesdis.OPCODES.get(opcode) or nesdis.UNDOC_OPCODES.get(opcode)
mode = entry[1] if entry else nesdis.IMP
addr = int(m.group(3), 16)
ins = Insn(addr, len(by), mn, operand, mode, opcode, by)
self.by_addr[addr] = len(self.insns)
self.insns.append(ins)
# labels: the address of the next instruction after the label
for i, line in enumerate(self.src):
m = LABEL_RE.match(line)
if not m:
continue
for j in range(i, min(i + 4, len(self.src))):
a = AUDIT_RE.match(self.src[j])
if a:
self.labels.setdefault(m.group(1), int(a.group(3), 16))
break
# a unit for every label that is followed by code
for name, addr in self.labels.items():
if addr in self.by_addr and addr not in self.units:
self.units[addr] = Unit(name, self.by_addr[addr], addr)
# header blocks
sep = [i for i, l in enumerate(self.src) if l.startswith(";=====")]
for k in range(len(sep) - 1):
o, c = sep[k], sep[k + 1]
body = self.src[o + 1:c]
if not any(l.startswith("; Global:") for l in body):
continue
name = None
for h in body:
if h.startswith("; ") and not h.startswith("; "):
t = h[2:].strip()
if t and not t.endswith(":") and ":" not in t:
name = t
break
entry = self._entry_after(c)
blk = Block(name, (o, c), entry)
if entry is not None and entry not in self.units:
self.units[entry] = Unit(name, entry, entry)
blk.unit = self.units.get(entry)
self.blocks.append(blk)
def _entry_after(self, line):
for j in range(line, len(self.src)):
m = AUDIT_RE.match(self.src[j])
if m:
addr = int(m.group(3), 16)
return addr if addr in self.by_addr else None
return None
def resolve(self, operand):
"""(cpu address, name) of a branch/call operand, or (None, None)."""
op = operand.strip()
if op.startswith("("):
return None, None
if op.startswith("$"):
try:
return int(op[1:], 16), op
except ValueError:
return None, None
if op in self.labels:
return self.labels[op], op
if op in self.syms:
return self.syms[op], op
return None, None
def build_cfg(pj):
"""Wire up the control flow of every instruction."""
for ins in pj.insns:
mn = ins.mn
nxt = pj.by_addr.get(ins.addr + ins.length)
if mn in TERMINATORS:
continue
if mn == "jmp":
if ins.operand.strip().startswith("("):
continue # indirect dispatch
addr, _name = pj.resolve(ins.operand)
tgt = pj.by_addr.get(addr) if addr is not None else None
if tgt is None:
ins.unknown_call = True
elif addr in pj.units:
ins.calls.append(tgt) # a tail call to another unit
else:
ins.succ.append(("branch", tgt)) # a jump into a shared tail
continue
if mn == "jsr":
addr, _name = pj.resolve(ins.operand)
tgt = pj.by_addr.get(addr) if addr is not None else None
if tgt is None:
ins.unknown_call = True
elif addr in pj.units:
ins.calls.append(tgt) # a call to another unit
else:
ins.succ.append(("branch", tgt)) # a call into a shared tail
if nxt is not None:
ins.succ.append(("fall", nxt))
continue
if ins.mode == nesdis.REL:
rel = int(ins.bytes[1], 16)
if rel > 0x7F:
rel -= 0x100
tgt = pj.by_addr.get(ins.addr + 2 + rel)
if tgt is None:
ins.unknown_call = True
else:
ins.succ.append(("branch", tgt))
if nxt is not None:
ins.succ.append(("fall", nxt))
continue
if nxt is not None:
ins.succ.append(("fall", nxt))
def analyse(pj):
"""Fill in the per-unit clobber data."""
build_cfg(pj)
for unit in pj.units.values():
seen, stack = set(), [unit.entry]
while stack:
i = stack.pop()
if i in seen:
continue
seen.add(i)
ins = pj.insns[i]
for c in ins.calls:
called = pj.units.get(pj.insns[c].addr)
if called is not None and called is not unit:
unit.calls.add(called)
if ins.unknown_call:
unit.unknown_call = True
for _kind, s in ins.succ:
if s not in seen:
stack.append(s)
unit.reach = seen
unit.touched = set()
unit.flag_written = set()
unit.mem = []
for i in sorted(seen):
ins = pj.insns[i]
mn = ins.mn
if mn in REG_WRITE:
unit.touched.add(REG_WRITE[mn])
for bit, mns in FLAG_BITS.items():
if mn in mns:
unit.flag_written.add(bit)
if mn in MEM_WRITE:
mem = mem_target(pj, ins)
if mem:
unit.mem.append(mem)
# register / flag preservation, over the reached code in address order
val = {"A": ENTRY["A"], "X": ENTRY["X"], "Y": ENTRY["Y"]}
flags = {f: ENTRY for f in FLAGS}
stack = []
for i in sorted(seen):
mn = pj.insns[i].mn
if mn == "pha":
stack.append(("A", val["A"]))
elif mn == "php":
stack.append(("P", dict(flags)))
elif mn == "pla":
v = stack.pop() if stack else None
val["A"] = v[1] if v and v[0] == "A" else ("?",)
elif mn == "plp":
v = stack.pop() if stack else None
flags = (dict(v[1]) if v and v[0] == "P"
else {f: ("?",) for f in FLAGS})
elif mn in COPY:
s, d = COPY[mn]
val[d] = val[s]
else:
if mn in REG_WRITE:
val[REG_WRITE[mn]] = ("?",)
for bit, mns in FLAG_BITS.items():
if mn in mns:
flags[bit] = ("?",)
if mn in ("jsr", "jmp"):
val = {"A": ("?",), "X": ("?",), "Y": ("?",)}
flags = {f: ("?",) for f in FLAGS}
unit.restored = {k for k, v in val.items() if v == ENTRY[k]}
unit.flag_restored = {f for f, v in flags.items() if v == ENTRY}
def mem_target(pj, ins):
"""(name, address) of the location an instruction writes."""
base = ins.operand.strip()
if base.startswith("("):
base = base.strip("()").split(",")[0].strip()
base = re.sub(r",[xy]$", "", base).strip()
if base in ("a", "A"):
return None
if base.startswith("$"):
try:
addr = int(base[1:], 16)
except ValueError:
return None
return (base, addr)
addr = pj.syms.get(base, pj.labels.get(base))
if addr is None:
return None
return (base, addr)
def closure(pj):
"""Fold each unit's callees' clobbers into the caller's."""
state = {}
def reach(u, seen):
if id(u) in state:
return state[id(u)]
if id(u) in seen:
return (set(), set(), set(), set())
regs = set(u.touched) - set(u.restored)
flags = set(u.flag_written) - set(u.flag_restored)
mem = set(u.mem)
calls = set()
for other in u.calls:
cr, cf, cm, cc = reach(other, seen | {id(u)})
regs |= cr
flags |= cf
mem |= cm
calls |= cc
calls.add(other.name)
if u.unknown_call:
# code this project has not decoded can damage anything
regs |= {"A", "X", "Y"}
flags |= set(FLAGS)
regs -= set(u.restored)
flags -= set(u.flag_restored)
state[id(u)] = (regs, flags, mem, calls)
return state[id(u)]
for u in pj.units.values():
(u.eff_regs, u.eff_flags, u.eff_mem,
u.eff_calls) = reach(u, frozenset())
def mem_name(name, addr):
if name and not name.startswith("$"):
return name
for lo, hi, label in REGIONS:
if lo <= addr <= hi:
return label
return name or f"${addr:04X}"
def fmt_regs(regs):
order = [x for x in ("A", "X", "Y") if x in regs]
if not order:
return "no register"
if len(order) == 1:
return order[0]
return ", ".join(order[:-1]) + " and " + order[-1]
def fmt_flags(flags):
if not flags:
return None
order = [f for f in FLAGS if f in flags]
if len(order) == len(FLAGS):
return "all the processor flags"
if len(order) == 1:
return f"the {order[0]} flag"
return "the " + ", ".join(order[:-1]) + " and " + order[-1] + " flags"
def memory_summary(pj, u, limit):
"""A list of the memory the unit writes, or a summary when it is long.
Several names can share one address (`Temp12` and `RecordIndex` are both
$12), so the entries are folded per address and the first name in
`locals.inc` is used - the file lists the primary name first.
"""
by_addr = {}
for name, addr in u.eff_mem:
if addr not in by_addr:
by_addr[addr] = name
entries = []
for addr in sorted(by_addr):
name = by_addr[addr]
aliases = pj.alias_order.get(addr, [])
if name in aliases:
name = aliases[0]
n = mem_name(name, addr)
if n not in entries:
entries.append(n)
if not entries:
return None
if len(entries) <= limit:
return ", ".join(entries)
regions, rest = [], []
for n in entries:
(regions if n.startswith("the ") else rest).append(n)
parts = list(dict.fromkeys(regions))
if rest:
parts.append("the RAM fields it writes")
return " and ".join(parts)
def clobber_text(pj, u, limit=6):
parts = [fmt_regs(u.eff_regs)]
fl = fmt_flags(u.eff_flags)
if fl:
parts.append(fl)
mem = memory_summary(pj, u, limit)
if mem:
parts.append(mem)
if len(parts) == 1 and parts[0] == "no register":
return "nothing"
return ", ".join(parts)
def wrap(text, width=76, indent="; "):
"""Wrap a sentence into the '; ' comment lines of a header block."""
words = text.split()
lines, cur = [], ""
for w in words:
cand = (cur + " " + w) if cur else w
if cur and len(indent) + len(cand) + 1 > width:
lines.append(indent + cur)
cur = w
else:
cur = cand
if cur:
lines.append(indent + cur)
return lines
CLOBBER_HEAD = "; Clobbers:"
def clobber_block(pj, u, limit=6):
text = clobber_text(pj, u, limit)
if not text.endswith("."):
text += "."
return [CLOBBER_HEAD] + wrap(text)
def is_blank(line):
"""A header line that carries no text: empty, or just the ';' separator."""
s = line.strip()
return s == "" or s == ";"
def insert_clobbers(pj, limit=6):
"""Return the source with a Clobbers block in every routine header."""
plan = []
for b in pj.blocks:
o, c = b.hdr_lines
at = c
for i in range(o + 1, c):
if pj.src[i].startswith("; PRG offset"):
at = i
break
plan.append((b, at))
out, pos = [], 0
for b, at in plan:
out.extend(pj.src[pos:at])
# drop every Clobbers block already in the header (its head, its text
# and the blank ';' line that follows it) so the new one replaces them
while True:
j = len(out)
while j > 0 and is_blank(out[j - 1]):
j -= 1
if j > 0 and out[j - 1].startswith("; "):
k = j
while k > 0 and out[k - 1].startswith("; "):
k -= 1
if k > 0 and out[k - 1] == CLOBBER_HEAD:
out = out[:k - 1]
continue
elif j > 0 and out[j - 1] == CLOBBER_HEAD:
out = out[:j - 1]
continue
break
out.extend(clobber_block(pj, b.unit, limit))
out.append(";")
pos = at
out.extend(pj.src[pos:])
return out
def main():
ap = argparse.ArgumentParser()
ap.add_argument("project")
ap.add_argument("--write", action="store_true")
ap.add_argument("--limit", type=int, default=6)
args = ap.parse_args()
pj = Project(args.project)
analyse(pj)
closure(pj)
if args.write:
out = insert_clobbers(pj, args.limit)
(pj.dir / "prg.s").write_text("\n".join(out))
print(f"{len(pj.blocks)} routine headers given a Clobbers block")
return
print(f"{len(pj.blocks)} routine headers, {len(pj.units)} units")
for b in pj.blocks:
u = b.unit
print(f"--- {b.name} (hdr {b.hdr_lines[0]+1}-{b.hdr_lines[1]+1}, "
f"{len(u.reach) if u else 0} insns reached)")
print(f" clobbers: {clobber_text(pj, u, args.limit) if u else '-'}")
print(f" calls : {', '.join(sorted(x for x in u.eff_calls if x)) or '-'}"
f"{' [undecoded callee]' if u and u.unknown_call else ''}")
if __name__ == "__main__":
main()