A workflow run on a Bevy weaving feature exposed two compounding
failures:
1. @test wrote 8 structural-only Rust tests that never invoked
weave_enemies or trigger_weaving. Every test passed against the
stub-first @make pre-pass because none of them called the
stubbed symbols, so todo!() never fired. The body-pass committed
code that "passed" the suite and silently broke trigger_weaving
in special stages.
2. @check found the trigger_weaving regression at Phase 8 (final
review) and the orchestrator decided to "fix them directly"
rather than dispatching @make — taking the license offered by
the existing review-loop wording.
Test-quality fixes:
- Phase 3 Test Design now requires each behavior to be expressed as
an action + observable outcome. Structural facts ("enum has 3
variants", "struct has these fields") are explicitly disqualified.
- Phase 6 stub-first flow gains a mandatory Panic-coverage check:
after @test returns, the orchestrator re-runs the test command and
rejects the output unless every test panics on todo!() (i.e. every
test exercises at least one stubbed symbol). Any passing test is
structural-only and routes back to @test.
- Phase 6 decision table gets a "Stub-first run: tests pass with zero
todo!() panics" row covering the same case.
- @test's Test Philosophy gains an explicit Do-NOT-write list of
structural-only patterns (variant_count, type ascriptions,
Box::new(my_fn), struct-literal-only flows, all-pass-on-stubs)
plus a positive rule: every test must call a function and assert
on observable outcome, or return NOT_TESTABLE rather than pad the
suite.
Orchestrator boundary fix:
- Phase 8 review loop replaces "fix them directly (no need to
re-dispatch @make for small fixes)" with the principle "the
orchestrator does not write production code; @make does". BLOCK,
behavioral, correctness, and test-quality findings round-trip
through @make. Only AST-preserving cosmetic edits (typos in
comments, trailing newlines) may be applied directly. Compiler-
detected issues (unused imports, dead code) go through @make.
Harald Hoyer
91ba5bd272