Scenario Refine Loop

Overview

Agents fail generation QA in two symmetric ways: accepting the first roll, or rewording the whole prompt and re-rolling until the budget dies. Both skip the same two artifacts, a written rubric and a diagnosis. The loop that converges: rubric before generating, a small batch, a recorded verdict per asset, the cheapest targeted fix per failure, a hard round cap. Connection and the core loop: see the scenario skill. Critic tool contracts: scenario-asset-analysis. Baseline discipline: scenario-consistency. If a sibling skill named here is missing from your available skills, ask the user to install it (npx skills add scenario-labs/skills --skill <name>); unattended, proceed from tool schemas and flag the gap.

Quick reference

A tier change is a new generation, not the keeper enlarged: re-run only the keeper's recipe at delivery tier and re-critique it, or keep what passed and upscale it with a fidelity upscaler found by search target="models", filters={"tags": ["image-upscale"]}, public=true (fidelity versus creative picks and sizing in scenario-image-editing, video upscaling in scenario-video).

When the bar is the configured brand brief rather than a task rubric, and the team's Quality Gate add-on is enabled, critique images with asset_quality_gate_run instead: its reasons and suggestions feed the fix table directly (scenario-quality-gate; where the gate is missing it degrades to this asset_analyze path).

Fix routing, cheapest first:

Change one variable per round. A round that swaps prompt, references, and model at once cannot attribute the improvement, so the next failure restarts from zero.

The two rules that keep the loop honest

• Verdicts cite the rubric, not taste. Instruct the critic to answer per image, in order: <index>: pass|fail, <the failed line>. "Could be better" is not a verdict; a loop chasing better instead of the brief sands off exactly what made the direction distinctive and converges on generic output.
• Regenerate from the baseline, never from the last attempt. Fixes re-run from the approved reference and prompt; chaining output to output compounds drift (scenario-consistency explains why).

Worked example: four icons against a brief

1. Rubric from the brief, five lines: single object, centered, plain field, palette #2A9D8F and #E9C46A only, no text.
2. Generate four, one model_run each per scenario-image, then jobs_wait.
3. One asset_analyze call with all four ids in images and the rubric-plus-shape instruction; answers land as text assets, asset_download them to read the verdicts.
4. Two fail. Icon 2 is off palette, which is rendered content rather than a finish: edit the delta clause by pinning the hex codes in the prompt and re-run from the baseline. Icon 4 has one smeared edge: masked inpaint of that corner. Icons 1 and 3 ship untouched.
5. jobs_wait the fix runs, then re-critique only the two new assets with the byte-identical instruction. Clean round: stop, file the keepers in a collection (scenario-asset-analysis).

Common mistakes

• Judging by glancing at asset_display in chat: unrecorded impressions do not accumulate; verdicts do.
• Asking asset_analyze to improve or fix the image: it returns text only; every fix is a new run.
• Re-rolling the whole batch because one item failed: route per item.
• Writing the rubric after seeing the batch: it inherits the batch's flaws as the standard.
• Running the loop uncapped: failed jobs are reimbursed, unsatisfying ones are not. Gate rounds on the costs the runs themselves report (dry_run prices the next round ahead); usage totals lag and answer the report after the run, not the mid-run gate.
• Retrying a criterion a third time on the same model: two misses under two different fixes is evidence about the model, not bad luck.