更新数据分析 Skill，优化多 Run 评估流程 (#194)

packages/docs/content/self-improvement.en.md

@@ -24,11 +24,11 @@ Before the first dispatch of every new or changed Case, the Builder checks that
 
 Before each calibration dispatch, the Builder predicts the result produced by the observed Trace strategy, the different result produced by the desired behavior, and the score range affected. Adding another public rule, exception, source, or check that the model can directly execute does not automatically increase difficulty. If both strategies still reach the same scored result, the Builder chooses another refinement.
 
-The Pilot score is a desired target: meeting it permits an early Freeze; otherwise the Builder completes the configured number of valid Pilot iterations and freezes the lowest-scoring valid revision. The Builder temporarily retains only the current lowest valid revision, then removes that copy and other calibration scaffolding after recording the Formal Baseline. Freeze is followed by a fresh complete Formal matrix. Every valid Formal Baseline is recorded even when its score misses the desired target.
+Every Pilot iteration runs each Case exactly once. The Pilot score is a desired target: meeting it permits an early Freeze; otherwise the Builder completes the configured number of valid Pilot iterations and freezes the lowest-scoring valid revision. The Builder temporarily retains only the current lowest valid revision and its complete result. After the final consistency review, it records that selected Pilot's one-Run result directly as the Formal Baseline without rerunning or backfilling more Runs, then removes the temporary copy and other calibration scaffolding. A Formal score that misses the desired target does not invalidate the Benchmark.
 
-After the user confirms that step is complete, they start the second top-level Session in a new conversation. The Optimizer checks the Benchmark and its first complete Formal Baseline before following `agent-optimization`:
+After the user confirms that step is complete, they start the second top-level Session in a new conversation and specify `runs` per Case for every Candidate. The Optimizer checks the Benchmark and its first complete Formal Baseline before following `agent-optimization`:
 
-1. orchestrate Evaluators in parallel through `run_subagent`, covering the Case × runs matrix;
+1. orchestrate Evaluators in parallel through `run_subagent`, covering the Case × user-specified `runs` matrix;
 2. use scores and linked Traces to propose one bounded Candidate;
 3. edit the Target Agent's editable state — `AGENTS.md`, Skills, config — to produce version N+1;
 4. keep the Candidate only when its Evaluation score strictly improves; otherwise roll it back;
@@ -36,7 +36,7 @@ After the user confirms that step is complete, they start the second top-level S
 
 Invalid evaluations and correction reruns do not count toward the round limit. On an execution failure, the Optimizer keeps the same Candidate and repairs only the missing cell; it keeps trying while each attempt follows a new diagnosis and applies a distinct safe repair. Both Builder and Optimizer validate that the complete Evaluator response is plain protocol YAML before reading status or score; if formatting is invalid, that same Evaluator resends from its existing result without rerunning the Target Agent.
 
-Every accepted Candidate is appended to and verified in the Scoreboard immediately. A strictly higher Evaluation score decides acceptance; whether the predicted Case behavior changed is reported separately so unrelated single-run variation is not presented as causal evidence. Agent optimization requires a complete Formal Baseline in the Scoreboard — without one there is no improvement to compare against.
+Every accepted Candidate is appended to and verified in the Scoreboard immediately. A strictly higher Evaluation score decides acceptance; the first comparison directly compares the Candidate's multi-Run average with the Formal Baseline's one-Run score without backfilling the Baseline. Whether the predicted Case behavior changed is reported separately so unrelated single-run variation is not presented as causal evidence. Agent optimization requires a complete Formal Baseline in the Scoreboard — without one there is no improvement to compare against.
 
 ## Benchmark storage
 
@@ -44,7 +44,7 @@ Benchmarks are stored per Agent under `benchmarks/<id>/`:
 
 ```text
 benchmarks/<id>/
-├── benchmark_config.toml       # Benchmark configuration (e.g. runs per Case)
+├── benchmark_config.toml       # Benchmark configuration (Builder runs is fixed at 1)
 ├── <case-id>/
 │   ├── statement/              # the task given to the Target Agent
 │   └── rubric/                 # private scoring rubric, isolated from the Target Agent

packages/docs/content/self-improvement.zh.md

@@ -24,11 +24,11 @@ Builder 和 Optimizer 在各自的顶层 Session 中直接遵循对应 Skill。E
 
 每轮校准都要在派发前预测：当前 Trace 中的策略会产生什么结果、期望行为会产生什么不同结果，以及会影响多少分。增加一条模型可以直接执行的公开规则、例外、来源或检查项并不会自动增加难度；如果两种策略仍会得到相同的计分结果，就应选择其他改法。
 
-Pilot 分数是期望目标：达到后可以提前 Freeze；未达到时完成设定数量的有效 Pilot iteration，并选择其中分数最低的有效版本 Freeze。Builder 在临时目录只保留当前最低有效版本，Formal Baseline 记录后清理该副本和校准脚手架。Freeze 后必须运行全新完整的 Formal matrix；只要 Formal 有效就记录 Baseline，分数没有达到期望也不会使 Benchmark 作废。
+每个 Pilot iteration 对每个 Case 固定只运行一次。Pilot 分数是期望目标：达到后可以提前 Freeze；未达到时完成设定数量的有效 Pilot iteration，并选择其中分数最低的有效版本 Freeze。Builder 在临时目录只保留当前最低有效版本及其完整结果。最终一致性检查通过后，直接把被选中 Pilot 的单次运行结果记录为 Formal Baseline，不再重新运行或补齐更多 Runs；记录后清理临时副本和校准脚手架。Formal 分数没有达到期望也不会使 Benchmark 作废。
 
-用户确认第一步完成后，在新对话中启动第二个顶层 Session。Optimizer 先检查 Benchmark 和第一条完整 Formal Baseline，再使用 `agent-optimization`：
+用户确认第一步完成后，在新对话中启动第二个顶层 Session，并指定每个 Candidate、每个 Case 的 `runs`。Optimizer 先检查 Benchmark 和第一条完整 Formal Baseline，再使用 `agent-optimization`：
 
-1. 通过 `run_subagent` 并行编排 Evaluator，覆盖 Case × 运行次数矩阵；
+1. 通过 `run_subagent` 并行编排 Evaluator，覆盖 Case × 用户指定 `runs` 的矩阵；
 2. 根据得分和关联 Trace 提出一个有界 Candidate；
 3. 编辑 Target Agent 的可编辑状态——`AGENTS.md`、Skills、配置——产出版本 N+1；
 4. Evaluation 分数严格提升才保留 Candidate，否则回滚；
@@ -36,7 +36,7 @@ Pilot 分数是期望目标：达到后可以提前 Freeze；未达到时完成
 
 无效评测和修复重跑不计入轮数。出现执行失败时，Optimizer 保持同一个 Candidate，只补齐失败单元；只要还能根据新诊断提出不同的安全修复，就继续尝试。Builder 和 Optimizer 都先验证 Evaluator 的完整响应是否为纯协议 YAML，再读取状态或分数；格式不合规时，由同一个 Evaluator 基于已有结果重发，不重新运行 Target Agent。
 
-每个 Accepted Candidate 立即写入并校验 Scoreboard。Evaluation 分数严格提高决定是否接受；假设是否在预期 Case 上得到支持单独报告，避免把单次运行中的无关波动解释为改动因果。Agent 优化要求 Scoreboard 中已有完整 Formal Baseline——没有基线，就没有可比较的提升。
+每个 Accepted Candidate 立即写入并校验 Scoreboard。Evaluation 分数严格提高决定是否接受；第一次比较允许直接用 Candidate 的多 Run 平均分比较 Formal Baseline 的单 Run 分数，不为 Baseline 补跑。假设是否在预期 Case 上得到支持单独报告，避免把单次运行中的无关波动解释为改动因果。Agent 优化要求 Scoreboard 中已有完整 Formal Baseline——没有基线，就没有可比较的提升。
 
 ## Benchmark 存储
 
@@ -44,7 +44,7 @@ Benchmark 按 Agent 存放在 `benchmarks/<id>/` 下：
 
 ```text
 benchmarks/<id>/
-├── benchmark_config.toml       # Benchmark 配置（如每个 Case 的运行次数 runs）
+├── benchmark_config.toml       # Benchmark 配置（Builder 的 runs 固定为 1）
 ├── <case-id>/
 │   ├── statement/              # 交给 Target Agent 的任务描述
 │   └── rubric/                 # 私有评分标准，对 Target Agent 隔离