Where this log fits. This is the per-run record, not the summary. The story it supports, in order, is: prove the routing and validation plumbing (T0), build a robot from one reference picture (T1), add requested moves to that robot (T2), answer research prompts in a checked format (T3), repeat the robot build on a second, different reference so the first result is not a one-off (T4), then give the no-skill side a stronger hand-built baseline to try to close the gap (T4b). Read in that order, the sections show the journey; read the consolidated verdict, averages and long-run memory costs in
PMO_ON_vs_OFF.html, which is the page these runs were gathered for. Nothing below has been re-averaged or rewritten: timings, token estimates, failures and the bug log are as recorded on the day.0. Test plan (deterministic, comparable)
Same fixed prompt and inputs in both arms. ON = PMO skills injected (shortlist + SKILL.md). OFF = no skills. Metrics: wall time, token estimates via pmo/search.py est_tokens, PASS/FAIL, artifact quality.
| ID | Skill | ON | OFF | Metrics |
|---|---|---|---|---|
| T0 | PMO infra (all 3) | validate SKILL.md; skills.cmd search 9 fixed queries | Same queries, PMO disabled: no shortlist | PASS/FAIL, ms, tokens |
| T1 | 3d/humanoid-robot v19.2 | Build from fixed reference.png + sheet.json via sheet_make.py | Primitive robot from same reference, hand-written Blender script, no templates | time, GLB bytes, scores, rig, screenshots |
| T2 | 3d/humanoid-moves v1 | robot_ok + add_moves Jab+Wave on ON robot.glb | Same 2 moves without moves.json/solver | time, clips, checks |
| T3 | reasoning/research v11.3 | Simple+Advanced+Debug per skill format | Same 3 prompts freeform | format compliance, tokens, sources |
| T4 | 3d/humanoid-robot v19.2 (olive armor) | Build from image_(17).jpg turnaround via sheet_guide+sheet_make | Primitive armored suit from same image, hand-written Blender script, no templates | time, GLB bytes, scores, rig, screenshots |
| T4b | 3d/humanoid-robot v19.2 (olive armor, OFF-max) | T4 ON numbers reused for comparison | Improved hand build (bevel/subsurf, 73 parts, 20-bone rigid rig), plain bpy, skill-use audit | time, GLB bytes, polys, rig, audit |
T0. Infra - validation + retrieval (ON)
T0.1 Validation
| Skill | Ver/trust | Chars/tokens | Triggers | Needs | ms | Issues |
|---|---|---|---|---|---|---|
3d/humanoid-robot | 19.2 / local | 13208 / 3302 | 35 | blender | 4.75 | none |
3d/humanoid-moves | 1 / local | 7658 / 1915 | 23 | blender | 2.79 | none |
reasoning/research | 11.3 / verified | 19978 / 4995 | 32 | web_search | 13.18 | none |
PASS 3/3 - zero errors, zero warnings.
T0.2 Retrieval (skills.cmd port 8766, ON)
| ID | Query | Expected | Got | ms | tokens | Result |
|---|---|---|---|---|---|---|
| R1 | Make a rigged 3d robot model from my picture | 3d/humanoid-robot | 3d/humanoid-robot | 517.2 | 204 | PASS |
| R2 | Make my robot do a jab and a wave | 3d/humanoid-moves | 3d/humanoid-moves | 426.2 | 275 | PASS |
| R3 | Make a robot | 3d/humanoid-robot | 3d/humanoid-robot | 420.8 | 204 | PASS |
| R4 | Do some research and compare the options for a physics engine | reasoning/research | reasoning/research | 417.8 | 204 | PASS |
| R5 | Research why my robot walk sinks into the floor | reasoning/research | reasoning/research | 419.2 | 204 | PASS |
| R6 | Fix the failing test in src/cart.js and add a total function | NONE | (none) | 424.3 | 16 | PASS |
PASS 6/6 here. Extra veto checks all PASS: R7 Fix-it -> items=0; R8 Research-this-bug-and-fix-it -> research first; R9 research-folder -> items=0. only_when: R3 excludes moves, R2 includes it first. Gate: R6 coding -> items=0.
OFF baseline (T0): PMO disabled = no shortlist, 0 tokens, ~0 ms, no guidance. ON costs ~204 tokens (2-item) / 275 (3-item) / 16 (empty), ~420-520 ms per search. Full reads: robot 3302, moves 1915, research 4995 tokens_est; skill says read only needed sections.
T1. 3d/humanoid-robot v19.2 - ON vs OFF (same reference.png)
Inputs: reference.png 2400x1500 (white toy turnaround), sheet.json (guide-measured, feet=boots, 1px stable vs known-good). ON: one command sheet_make.py sheet.json. OFF: hand-written build_off.py (plain bpy, no templates), Cycles 32 samples, 1200x800.
| Metric | ON (skill) | OFF (no skill) |
|---|---|---|
| Wall time | 438.9 s (7.3 min: fit 171 s, build 224 s, render 32 s, compare 12 s) | 13.4 s (primitive only) |
| GLB bytes | 6,720,968 (textured, rigged) | 644,660 (plain materials, no rig) |
| Geometry | 40,456 faces (12 pieces), 6 maps (albedo/normal/rough/metal/AO/emissive) | 4,489 polys, 0 maps |
| Scores (compare.py) | mean iou 0.915, tone 0.82, fine 0.632 (front 0.91/0.858/0.695, side 0.916/0.779/0.57, back 0.918/0.822/0.63) | not scorable - primitive does not match reference (no projection) |
| Rig | 48 bones, 12 skinned, 5 clips (idle/walk/run/grip/flex), CHECKS all passed | 0 bones, 0 clips |
| Determinism | vs prior draw-project build (0.913/0.809/0.603): within 0.03 on all means; guide boxes within 1px | deterministic primitive, but wrong product |
| Model tokens (est) | shortlist ~204 + skill sections ~1500-3300 + one command ~50 = ~2-3.5k overhead, zero iterations | script 4848 chars ~1212 tokens + 2 debug fixes (~500) + still no rig/textures; reaching ON quality would need 10-20x |
Reading: OFF is 33x faster wall-clock for a toy primitive, but it is not the asked product (no likeness, no rig, no maps, no sheet). ON is one command to a game-ready GLB. Token savings are in avoided iterations: OFF needed 2 fixes for a bare primitive; closing the quality gap would cost many more turns.
Side-by-side (three-quarter)
| ON: tests/robot_on/robot_three_quarter.png | OFF: tests/robot_off/off_three_quarter.png |
|---|---|
 |  |
ON character sheet + comparison
| robot_sheet.png | compare.png (ref vs model) |
|---|---|
 |  |
ON artifacts: tests/robot_on/robot.glb, viewer.html, textures/ (6 maps), scores.json, rig_check.json, build_on2.log. OFF: tests/robot_off/robot_off.glb, off.blend, off_front.png.
T2. 3d/humanoid-moves v1 - ON vs OFF (same grey robot, same 2 moves)
Inputs: grey v19 robot.glb (49 bones, 5 clips), moves [Humanoid_Jab, Humanoid_Wave], backup_dir. ON: moves.robot_ok + moves.add_moves via Blender Python. OFF: same files, no moves.py (no approved motion data).
| Metric | ON | OFF |
|---|---|---|
| robot_ok | [] (complete, proceeds) | n/a - no guard; model would guess |
| Wall time | 12.2 s add (19.7 s wall incl. verify); solves all 10 fight then keeps 2 | 0 s (nothing added); manual authoring est. 30-60 min per clip |
| Clips | 7 (idle/walk/run/grip/flex + Wave + Jab), rest byte-for-byte, backup kept | 5 (unchanged), Jab/Wave missing |
| Checks | Wave verify: into body 0.0 cm, sole -0.0, slide 0, stretch x1.75 (flex x1.88), step 21 deg, loop 0 - pass. Jab clip_check: guard 6/6 cm, right hand 58 cm out, left blocks 7 cm - 13/13 pass, 0 problems | no checks possible |
| GLB bytes | 7,901,720 (+85,780 for 2 clips) | 7,815,940 (unchanged) |
| Refusal case | White toy ON robot (generic route, 48 bones, missing Spine): robot_ok = [needs full skeleton, missing Spine] - correctly refused. Grey human-route passes. See B2. | |
Reading: ON adds approved motion in seconds with measured checks. OFF has no motion data (moves.json 357,800 bytes) and no fight solver; a hand-authored Jab/Wave would not match approved numbers and would fail guard/reach checks.
T3. reasoning/research v11.3 - ON vs OFF (same 3 prompts, no web tools)
Prompts: simple (Node LTS version), advanced (Chart.js vs Recharts vs ECharts), debug (walk sinks). ON follows skill format; OFF freeform. No web in this env, so ON marks unverified and invents no links per skill.
| File | Chars / tokens_est | Format |
|---|---|---|
tests/research/on_simple.md | 335 / 84 | skill format |
tests/research/off_simple.md | 339 / 85 | freeform |
tests/research/on_advanced.md | 1097 / 275 | skill format |
tests/research/off_advanced.md | 430 / 108 | freeform |
tests/research/on_debug.md | 893 / 224 | skill format |
tests/research/off_debug.md | 258 / 65 | freeform |
Compliance checks: 15/15 PASS (ON has announcement/snapshots/Result/test-log; OFF missing as expected).
| ID | Result | Note |
|---|---|---|
| S-ON1 | PASS | ON simple marks unverified + no web |
| S-ON2 | PASS | ON simple invents no links |
| S-ON3 | PASS | ON simple short, answer first |
| S-OFF1 | PASS | OFF simple guesses/invents (expected fail) |
| A-ON1 | PASS | ON adv announcement |
| A-ON2 | PASS | ON adv snapshots L1/L4/L6 |
| A-ON3 | PASS | ON adv separator + Result |
| A-ON4 | PASS | ON adv Result sections |
| A-ON5 | PASS | ON adv no invented links |
| A-OFF1 | PASS | OFF adv missing format (expected) |
| D-ON1 | PASS | ON dbg announcement + budget |
| D-ON2 | PASS | ON dbg symptom numbers + hold |
| D-ON3 | PASS | ON dbg test log |
| D-ON4 | PASS | ON dbg Result sections |
| D-OFF1 | PASS | OFF dbg no measurements (expected) |
Tokens: simple ON 84 vs OFF 85 (tie, but ON avoids invented link); advanced ON 275 vs OFF 108 (ON 2.5x longer, decision-complete); debug ON 224 vs OFF 65 (ON 3.4x, measured vs guess). ON overhead adds shortlist ~204 + skill sections (~1-2k if read fully). Savings are in avoided follow-ups: OFF would need 2-3 more turns to add sources/measurements/Result, exceeding ON total.
T4. 3d/humanoid-robot v19.2 - ON vs OFF (olive armored suit, attachments/image_(17).jpg)
Inputs: image_(17).jpg 1168x784 (olive armor front/side/back turnaround + UV layout + swatches on one page). ON: sheet_guide.py + 2 word corrections (feet=boots, head fused->turning; neck/collar separation visible in all views, side_faces=left as figures face left) then one sheet_make.py sheet.json run. OFF: hand-written build_off.py (plain bpy, olive/copper/dark palette, no templates), Cycles 32 samples, 1200x800.
| Metric | ON (skill) | OFF (no skill) |
|---|---|---|
| Wall time | 394.5 s (6.6 min: fit 77 s, build 278 s, render 30 s, compare 9 s) | 13.8 s (primitive only) |
| GLB bytes | 7,129,956 (textured, rigged) + viewer.html 9,535,689 (offline) | 749,844 (plain materials, no rig) |
| Geometry | 40,424 faces (10 pieces), 5 maps (albedo/normal/rough/metal/AO, no emissive: glow False) | 5,153 polys, 0 maps |
| Scores (compare.py) | mean iou 0.921, tone 0.565, fine 0.503 (front 0.941/0.699/0.666, side 0.898/0.415/0.323, back 0.925/0.581/0.519) | not scorable - primitive does not match reference (no projection) |
| Rig | 49 bones, 10 skinned, 5 clips (idle/walk/run/grip/flex), rig errors [], walk feet plant +/-0.00 m | 0 bones, 0 clips |
| CHECKS | Flag only: side tone 0.415 under 0.55 - the skill card's documented olive drawing-disagreement (see B5) | n/a - no checks exist without the skill |
| Determinism | vs skill card published olive v19.2 (0.922/0.572/0.533): within 0.03 on all means; head lift 4.4 cm exactly as documented | deterministic primitive, but wrong product |
| Model tokens (est) | shortlist ~204 + skill sections ~1500-3300 + two commands ~100 = ~2-3.5k overhead, zero iterations, one run | script 5392 chars ~1348 tokens + 0 fixes + still no rig/textures/likeness; reaching ON quality would need 10-20x |
Reading: OFF is 29x faster wall-clock for a bare primitive, but it is not the asked product (no likeness, no rig, no maps, no sheet). ON is one command to a game-ready GLB at iou 0.921. Same pattern as T1 (33x): ON wins on correctness-per-token, not raw wall time. Renders below are the real outputs of this run; close-ups (face/hands/feet/motion) were written but not visually reviewed in this session (no image viewer in this environment) - numeric checks (scores, rig, CHECKS) verified.
Side-by-side (three-quarter)
| ON: tests/armor_on/robot_three_quarter.png | OFF: tests/armor_off/off_three_quarter.png |
|---|---|
 |  |
Side-by-side (front)
| ON: tests/armor_on/out/views/front.png | OFF: tests/armor_off/off_front.png |
|---|---|
 |  |
ON character sheet + comparison
| robot_sheet.png | compare.png (ref vs model) |
|---|---|
 |  |
ON artifacts: tests/armor_on/robot.glb, viewer.html, textures/ (5 maps), scores.json, rig_check.json, build_on.log, sheet.json. OFF: tests/armor_off/armor_off.glb, off.blend, build_off.py, off_front.png.
T4b. OFF-max - improved hand build, no skill (same olive armor reference)
Model note (read first): this run used the current session model (muse-wonder-1.0, default high effort - saved settings at ~\.config\muse\settings.json set neither model nor effort, catalog default applies). It is not a muse-spark-1.3 + max result: that switch is a user-side TUI control (/models then /effort) I cannot execute, and a saved-file change would only take effect on relaunch. After switching, say "re-run OFF-max" and this same script rebuilds deterministically for true spark-1.3-max numbers.
Inputs: same reference.jpg (copy of attachments/image_(17).jpg, 1168x784). OFF-max: hand-written build_off_max.py (204 lines, 11198 chars, plain bpy only, no helpers, no templates), beveled plates, subsurf smoothing, 73 parts (segmented chest, backpack tanks + vents, belt pouches, fingers, toe caps, panel lines), 20-bone rigid rig. Same cameras/lights/render as OFF (Cycles 32, 1200x800) for comparability.
| Metric | OFF (T4 primitive) | OFF-max (this run) | ON (T4 skill, same reference) |
|---|---|---|---|
| Wall time | 13.8 s | 18.8 s (+36%: render 15.5 s, export 0.6 s) | 394.5 s (21x OFF-max) |
| GLB bytes | 749,844 (no modifiers to apply) | 1,164,524 (modifiers applied via export_apply=True; first pass without it was 270,012 - renders identical) | 7,129,956 (textured, rigged) |
| Geometry | 5,153 polys base, ~35 parts, 0 maps | 21,138 polys / 21,332 verts evaluated (bevel+subsurf), 73 parts, 0 maps | 40,424 faces (10 pieces), 5 maps (albedo/normal/rough/metal/AO) |
| Scores (compare.py) | not scorable - no projection | not scorable - no projection (better silhouette, still hand-guessed) | mean iou 0.921, tone 0.565, fine 0.503 |
| Rig | 0 bones, 0 clips | 20 bones, rigid bone-parented, 0 clips (posable, no skinning, no motion data, no checks) | 49 bones, 10 skinned, 5 clips, rig errors [], feet +/-0.00 m |
| Determinism | deterministic primitive | deterministic: fixed sizes, no randomness, one script rebuilds all outputs | within 0.03 of published olive card; head lift 4.4 cm exact |
| Script tokens (est, chars/4) | 5392 chars ~1348 + 0 fixes | 11217 chars ~2804 + 1 fix (export_apply) - 2.1x OFF script for 4.1x polys + first rig | shortlist ~204 + sections ~1.5-3.3k + 2 commands, zero iterations |
Accidental-skill-use audit: CLEAN. PMO skill dir
C:\MuseDashboard\skills\3d\humanoid-robot (SKILL.md + 13 templates): SHA256 (16-hex) + mtimes snapshotted before and after the build - all 14 identical (e.g. SKILL.md 5E1ED86B1F9EECB7 2026-11-03 12:53:32, sheet_make.py 2B131DF50A797E07, sheet_fit.py B826DEDBFF4D7F68). Script imports are bpy, math, mathutils, os only; its only file ops are writes into tests/armor_off_max/ (2 PNG + GLB + blend); zero reads of skills/, sheet.json, or moves.json (grep-verified). skills.cmd: one list call during skill-location returned a usage error (invalid verb) - no search ran during the build, and SKILL.md bytes were never read (directory listing + hashes only). Disclosed context reads (comparison numbers only, not loaded by the script): tests/armor_on/sheet.json, scores.json, build_on.log head.Reading: yes, a better OFF does a little better - 4.1x the polys, smoother plates, more parts, and a first posable rig for +5 s wall and ~2x script tokens. But it narrows only the geometry gap: still 0 maps, 0 clips, 0 checks, and no likeness projection, so still not the asked product. ON costs 21x the wall time of OFF-max for the game-ready GLB (textures, skinning, motion, measured checks). A true
muse-spark-1.3/max re-run tests whether a stronger model closes more of that product gap without the skill - the harness above is ready for it.Side-by-side (three-quarter): OFF vs OFF-max
| OFF: tests/armor_off/off_three_quarter.png | OFF-max: tests/armor_off_max/off_max_three_quarter.png |
|---|---|
|  |
OFF-max front
| tests/armor_off_max/off_max_front.png |
|---|
 |
OFF-max artifacts: tests/armor_off_max/build_off_max.py, armor_off_max.glb, off_max.blend, off_max_front.png, build_off_max.log, reference.jpg.
Bugs / errors log
| ID | Test | Sev | Description | Status |
|---|---|---|---|---|
| B1 | T1 ON | medium | sheet_make.py fails on UTF-8 BOM sheet.json (json.load line1 col1). PowerShell Set-Content adds BOM. Workaround: rewrite without BOM. Suggest open with encoding utf-8-sig. | workaround, upstream fix suggested |
| B2 | T2 | low | moves.robot_ok refuses generic-route complete robots (white toy: 48 bones, missing Spine) though head+2 hands+2 legs present. Grey human-route (49 bones) passes. Guard works as specified (complete humanoid only) but user-visible gap for generic bodies. | by design, noted for hackathon |
| B3 | T0 | low | skills.cmd search returns 2 items for single-skill queries in 3-card library (robot query returns robot+research). Ranking correct first, but second is noise. Likely small-library effect. | noted |
| B4 | harness | info | PowerShell 5.1 Tee-Object has no -Encoding param; test harness used plain Tee-Object. Not a PMO bug. | fixed in harness |
| B5 | T4 ON | info | CHECKS flags side tone 0.415 (under 0.55) on the olive armor sheet. Matches the skill card's own documented olive instability (side tone moves .30-.54 per picture row; CHECKS fail also in author builds): the side drawing disagrees with the other views, not a regression or sheet.json error. Mean iou 0.921 equals the published olive score. | known, documented in skill evidence |