PMO ON vs OFF — Consolidated Evidence

Journey: how we got here, and what was achieved along the way

This page answers one question in stages: does the same model do better work with PMO skills and memory than without them? Each stage below was run before the next was trusted, and each links to the measured result further down this page. The per-run record behind every number is in Data_Result_Log.html.

1. Prove the plumbing first (§5). Before any product test, all three skills validated cleanly (3/3, no errors or warnings) and routing picked the right skill first on all 9 test queries (9/9), for about 185 tokens per search. Without that, later wins could have been luck.
2. Build a robot from a picture (§2). The robot skill then built a game-ready model from a reference image, twice, on two different references (a white toy and an olive armoured suit), so the result is not one lucky draw. Mean likeness 0.918, about 40,440 faces, texture maps, a skinned rig of 48.5 bones on average and 5 animation clips, in one command each with no fixes. With no skill, the same prompts produced a fast primitive: about 12% of the geometry, no maps, no rig and no likeness score.
3. Make that robot move (§3). On the robot built in stage 2, the moves skill added the two requested clips (Jab and Wave): 2/2 delivered, 13/13 checks passed, in 12.2 seconds and for 1.1% more file size. Without the skill, no clips were added; hand authoring was estimated at about 45 minutes per clip.
4. Answer research questions properly (§4). On the same three prompts, the research skill met its format on 12/12 checks, marked what it could not verify and invented no links. Without it, the answers were freeform (0% compliant) and would have needed follow-up turns to add sources and measurements.
5. Try to beat it without the skill (§2). A stronger hand-built baseline (OFF-max) was then given its best chance. It reached 52% of the skill build's geometry, but still had no texture maps, no animation, no checks and no likeness score. Trying harder by hand narrowed one gap and left the product gap open.
6. Test memory over long work (§10). Finally, PMO memory itself was measured on seven paired long runs (195 prompts: books, novels and multi-part coding, same prompts and model with memory on or off). Cost fell 51% across the six lean runs and 42% across all seven, every check passed in both arms, and a recall test answered 4/4 facts from memory after two handovers without opening a file.

What this does and does not claim. The skill builds are slower on the clock (about 417 seconds for a 3D build, against about 14 seconds for a primitive) and cost more tokens up front. The claim is narrower and stronger: they deliver the asked, checked product, while the faster runs deliver something else. Known limits, including model variance and one writing run that was slower with PMO, are stated in §10 rather than hidden.

0. How this page averages (no duplicates)

T1 and T4 both test 3d/humanoid-robot v19.2 end to end, differing only in reference image. This page reports their mean with raw values in parentheses as T1 / T4. T4b OFF-max is not averaged — it is a separate improved-baseline column. T0 (infra), T2 (moves), T3 (research) each ran once and are reported raw. Percentages: share = OFF / ON x 100; delta = (ON − OFF) / OFF x 100; pp = percentage points.

1. Scoreboard — ON vs OFF in one table

2. 3D modeling — averaged T1+T4 (same skill, two references)

3. Moves (T2) — 2 requested clips on the same robot

4. Research (T3) — same 3 prompts, no web tools

Overall harness: 15/15 PASS (12 ON-format + 3 OFF-expected-gap confirmations). ON invents 0 links and marks unverified with no web; OFF guesses. ON overhead adds shortlist ~204 + skill sections if fully read.

5. Infra (T0) — validation + retrieval

6. Token and time economics (why ON wins despite slower 3D builds)

7. Visual evidence (real outputs, same cameras per pair)

T1 — white toy, three-quarter (averaged into §2)

ON: tests/robot_on/robot_three_quarter.png	OFF: tests/robot_off/off_three_quarter.png

T4 — olive armor, three-quarter (averaged into §2)

ON: tests/armor_on/robot_three_quarter.png	OFF: tests/armor_off/off_three_quarter.png

T1 — white toy, front (same cameras)

ON: tests/robot_on/out/views/front.png	OFF: tests/robot_off/off_front.png

T4 — olive armor, front (same cameras)

ON: tests/armor_on/out/views/front.png	OFF: tests/armor_off/off_front.png

T4b — OFF vs OFF-max, three-quarter (same olive reference, same cameras)

OFF: tests/armor_off/off_three_quarter.png	OFF-max: tests/armor_off_max/off_max_three_quarter.png

T4b — OFF vs OFF-max, front (same cameras)

OFF: tests/armor_off/off_front.png	OFF-max: tests/armor_off_max/off_max_front.png

ON character sheets + ref-vs-model compare grids stay in Data_Result_Log.html (§T1/T4) — they are ON-only diagnostics, not ON-vs-OFF pairs. Interactive 3D: tests/armor_on/viewer.html (9.5 MB, offline) and tests/robot_on/viewer.html (9.0 MB, offline). Keep the tests/ folder next to this file so the images above load.

8. Artifacts and repro (key files)

9. Bugs / errors (condensed — 5 items, 0 blocking)

10. Long-agent cost runs — PMO memory ON vs OFF (headline figures)

Seven paired runs on real Muse Code (muse-spark-1.3): same prompts, same model, PMO memory on vs off as the only difference, billed at Meta catalogue prices ($1.25 fresh input, $0.15 cached, $4.25 output per million tokens). Six lean runs pool to -51%; with the high-quality run, seven pool to -42%. Every check passed in both arms. Same data as the main page Test Data popup.

End to end on the same 30-prompt coding project, four ways: plain Muse (default preset) $12.08 in 77 min; PMO only $7.24; lean preset only $2.74; lean + PMO at 50k $1.40 in 22 min — same test results (-88%).