Commands

stella tune

Test one setting against another over two loop-bench runs, then apply the winner only if it clearly wins, or roll it back.

stella tune tunes stella's own settings using evidence instead of a gut feeling. It runs an A/B test on one setting over two loop-bench runs, scores each side using the reward from every trial, and applies the winning setting only when it clearly beats the current one. Either way, it saves a record so you can undo the change.

The first setting you can tune this way is the worker's reasoning effort. This command works offline: it reads two result files and your local record of past tests, and only writes new settings when you pass --promote. It needs no model provider and no API key.

Synopsis

stella tune effort --baseline <file> --candidate <file> \
    [--baseline-effort <effort>] [--candidate-effort <effort>] \
    [--promote] [--scope <project|user>] [--min-samples <n>]
stella tune rollback
stella tune status

What it does

Most "tuning" of an agent is someone changing a default because one run felt better. That is not something you can prove one way or the other. This command works differently: run the same set of tasks twice, once per setting, and let the numbers decide.

# 1. Run the same task list at each effort level, saving --json output
loop-bench --json > base.json      # worker effort: medium
loop-bench --json > cand.json      # worker effort: high

# 2. Score the two runs — this only reports, it changes nothing
stella tune effort --baseline base.json --candidate cand.json

# 3. Apply the winner, with a record you can undo later
stella tune effort --baseline base.json --candidate cand.json --promote

stella tune effort

Compares the worker reasoning-effort setting across two loop-bench --json result files and picks the winner.

--baseline <file>Required

loop-bench --json output for the current setting.

--candidate <file>Required

loop-bench --json output for the setting you're testing.

--baseline-effort <effort>

The worker effort level used for the baseline run.

Default medium

--candidate-effort <effort>

The worker effort level used for the candidate run.

Default high

--promote

Applies the winner to your settings. Leave this off and the command just reports the result and saves a record of the comparison, without changing anything.

Default off

--scope <project|user>

Which settings scope to write to when you use --promote.

Default project

--min-samples <n>

The minimum number of trials required in each run before stella will let you promote a winner.

Default 5

--min-samples only limits promotion, not reporting. A test with just two trials on each side still prints its comparison. It just won't be allowed to change your settings.

stella tune rollback

Undoes the last effort change, restoring the worker effort setting you had before. Every promotion saves the record needed to undo it, so a tuning decision is never permanent.

stella tune rollback

stella tune status

Shows your tuning history: every test that has run, and which setting is active right now.

stella tune status

Why promotion is cautious

A/B testing an agent is noisy. The same tasks with the same settings will not return the exact same reward score twice. If stella promoted a setting based on a simple average, your settings would drift around based on noise, and you would never see it happen. So a promotion only happens when the candidate clearly wins, --min-samples sets a hard minimum number of trials on both sides, and every change is saved with a way to undo it. Those three rules are what make it safe to leave this feature turned on.

--promote writes to your settings.json at the scope you chose, and it shows you the old value it replaced. Run the command without --promote first and read the comparison. Either way, the record is saved, so looking first costs you nothing.

One side effect is worth knowing about: if effort_auto was turned on, promoting a result turns it off. Otherwise the automatic effort selector would just override the value the A/B test just chose. stella tune rollback restores both the old effort level and the old effort_auto setting together, so you never end up stuck with the old value but the selector still disabled.

settings.json

Every key --promote can write, and how project and user scope are resolved.

Agent engine config

The reasoning-effort setting and the rest of the engine's options.

Determinism over intelligence

Why stella prefers a clear, repeatable check over a model's own opinion, wherever one is available.

stella usage

Report on, replicate, and prune the cross-project usage hub at ~/.stella/usage.db, the same cost numbers as stella stats, across every project.

stella dataset

Build a redacted, source-stamped training dataset from this workspace's own records — one JSONL record per accepted turn, plus a manifest stating the exact rule that selected them.