docs(examples): make self-improvement demo genuinely self-evolving (#29)

This commit is contained in:
Zhang Jason authored and GitHub committed 2026-07-22 22:04:52 +08:00
1 parent 6d25adfef4
commit 81d8c0045c
8 files changed
+728 -102

No files matched your search

@@ -156,25 +156,25 @@ The third pillar builds on the same idea. Because agents are editable data and e
Design tenets are easy to claim. Here is an end-to-end run that exercises them, entirely on fully local, open-weight infrastructure with an AMD GPU — every token generated on-device, nothing sent to a cloud API.
**The setup.** An AMD GPU running ROCm, serving `qwen3:8b` — a popular, everyday local open-weight model (about 5 GB) — through Ollama's OpenAI-compatible endpoint. Ollama detected the AMD GPU natively (no architecture override needed) and loaded the model into GPU memory. This same path spans AMD's ROCm-supported lineup — from Radeon PRO workstation cards such as the W7900 (48 GB, RDNA3) up to datacenter Instinct accelerators. Wiring it into PenguinHarness took a single command, precisely because the core treats any OpenAI-compatible endpoint the same way:
**The setup.** An AMD GPU running ROCm, serving `qwen3.6:35b` — a capable local open-weight model (a ~24 GB MoE, 36B total parameters, Q4_K_M) — through Ollama's OpenAI-compatible endpoint. Ollama detected the AMD GPU natively (no architecture override needed) and loaded the model into GPU memory. This same path spans AMD's ROCm-supported lineup — from Radeon PRO workstation cards such as the W7900 (48 GB, RDNA3) up to datacenter Instinct accelerators. Wiring it into PenguinHarness took a single command, precisely because the core treats any OpenAI-compatible endpoint the same way:
```bash
penguin config model add \
--model-id qwen3:8b \
--model-id qwen3.6:35b \
--provider custom --client-type openai \
--base-url http://localhost:11434/v1 \
--api-key ollama --set-default
```
**The task, and how it's scored.** We gave it a constrained writing task in the style of the built-in `example-benchmark`: read a project notes file and produce a summary with an overview of at most 2 sentences, exactly 3 key facts, under 60 words total. The task carries a private rubric — a scoring checklist the agent never sees, so it cannot "teach to the test." The rubric turns each requirement into one point (file actually written · ≤2-sentence overview · exactly 3 bullets · under 60 words · facts accurate), for a score out of 5.
**The task, and how it's scored.** We gave it a task that looks trivial: read a project notes file and produce a summary with a 2-sentence overview and exactly 3 key facts — *and follow the team's standard report format.* That last clause is the whole point. The "team format" is an arbitrary house convention — a specific marker line, a `# Report: <subject>` title, a `Classification: INTERNAL` line, a `Reviewed-by: Aurora Team` footer — that lives *only in the agent's `AGENTS.md`* and cannot be inferred from the task. The task carries a private rubric — a checklist the agent never sees — with 10 points: 5 for content any capable model earns from the task alone, and 5 for the convention, which is knowable only from `AGENTS.md`.
**The baseline — and why it isn't perfect.** Running locally on the AMD GPU, the model read the file, composed a perfectly reasonable summary… and then never wrote it to disk. It emitted the answer as chat text instead of calling the write tool. No deliverable means the rubric scores it **0 / 5**. And because every step is in the Trace, this is not guesswork — you can open the run and see exactly where it went wrong.
**The baseline — and why it isn't perfect.** Running locally on the AMD GPU with a blank `AGENTS.md`, the model wrote a perfectly reasonable summary — and stably lost all 5 convention points, landing around **4.6 / 10**. It could not have done otherwise: nothing in the task reveals the house convention, so this is an *information gap, not a capability gap* — which is exactly why a stronger model can't just "figure it out." And because every step is in the Trace, this is not guesswork — you can open the run and see precisely which points were missed.
That is the honest starting point: on local hardware, an out-of-the-box model does *not* ace a constrained task. Which is exactly what makes the next section interesting — a measured, auditable failure is something you can systematically fix.
That is the honest starting point: on local hardware, an out-of-the-box agent does *not* ace a task whose rules live in files it hasn't learned yet. Which is exactly what makes the next section interesting — a measured, auditable, *reproducible* failure is something the agent can systematically fix by teaching itself.
## How self-improvement actually works — and a real before/after
The 0 / 5 above is an *evaluation* — a snapshot of the agent as it stands. The more interesting question is how PenguinHarness turns that snapshot into progress. This is the recursive self-improvement loop, and it is worth understanding as a mechanism, because there is no magic engine code behind it — it is ordinary agent machinery, orchestrated by Skills:
The 4.6 / 10 above is an *evaluation* — a snapshot of the agent as it stands. The more interesting question is how PenguinHarness turns that snapshot into progress. This is the recursive self-improvement loop, and it is worth understanding as a mechanism, because there is no magic engine code behind it — it is ordinary agent machinery, orchestrated by Skills:
1. **Benchmark** — define capability cases, each with a private rubric (as above).
2. **Evaluate** — run the agent over the cases and score against the rubrics. Each run is an ordinary, fully-traced Session.
@@ -182,22 +182,24 @@ The 0 / 5 above is an *evaluation* — a snapshot of the agent as it stands. The
4. **Edit the Agent's state** — the agent's behavior lives in editable files (`AGENTS.md`, Skills, config). You (or an Optimizer agent) change those to address the failure, producing version N+1.
5. **Snapshot & keep-or-roll-back** — snapshot before each round; keep N+1 only if the score strictly improves, otherwise roll back.
Let's improve the exact agent from the previous section — the same `qwen3:8b` on the same AMD GPU, the same task that just scored **0 / 5**.
Let's improve the exact agent from the previous section — the same `qwen3.6:35b` on the same AMD GPU, the same task that just scored **4.6 / 10**. And here is the crucial part: the *agent* does the diagnosing and the editing — we do not hand it the answer.
- **The diagnosis (from the Trace).** The run failed for one concrete reason: the model produced the summary as chat text and never called the tool to write `summary.md`. The Trace shows it — a `cat` to read the input, then prose, and no write. So the fix is not "make the model smarter"; it is "make the behavior reliable."
- **The edit (N → N+1).** We added a short *working-discipline* section to the agent's `AGENTS.md`: *read the source first, restate the task's constraints as a checklist, actually call the tool to write the file, and re-read the output to self-check before finishing.* That is the entire change — a few sentences in an editable text file the agent reads on every run. No retraining, no code.
- **The re-evaluation.** Same model, same task, run again: this time it wrote a correct, on-topic `summary.md` to disk, with a 2-sentence overview and exactly 3 key facts drawn from the source — **4 / 5** (the one remaining miss: it ran to 70 words, over the 60-word budget).
- **The diagnosis (by the agent).** We give the agent two files and nothing else: the report it just got rejected, and a *different* project's report that passed review. Nobody tells it the rules. It compares the two, and infers the reusable house convention — the marker, the title shape, the metadata line, the sign-off. This is the "read the Trace to find where points were lost" step, performed by the agent itself.
- **The edit (N → N+1), authored by the agent.** The agent writes the convention it just inferred into its own `AGENTS.md`. From a single example it correctly recovers the *structure*, but can't yet tell which tokens are fixed constants vs per-report fields — a single example is ambiguous — so it generalizes the marker to a placeholder. Re-evaluated, the score climbs to about **6.6 / 10**. No retraining, no code — the agent edited a text file it reads on every run.
- **The recursion (N+1 → N+2).** Now we show it *several* accepted reports from different projects that share the same marker and sign-off. The agent reasons that whatever is identical across all of them must be a fixed constant, reads its *own* N+1 `AGENTS.md`, and refines it — locking `<!-- ACME-DATA-PLATFORM -->` and `Reviewed-by: Aurora Team` to literals. Re-evaluated again: about **9.8 / 10**. That is recursion in the true sense: `state_{n+1} = agent.reflect(state_n, new_evidence)`.
That is a jump on the same model and the same task, achieved purely by editing a text file the agent reads. That is the loop in miniature: *what you can see, you (or an Optimizer agent) can improve* — and because the score comes from a rubric and links to a Trace, the improvement is measured, not vibes. This is exactly the "keep the change only if the score strictly improves" loop, one turn of the crank.
That is a **4.6 → 6.6 → 9.8** climb on the same model and task, achieved purely by the agent editing a text file it reads — with the harness keeping each round only because the score strictly improved. That is the loop, self-driven: *what the agent can see, the agent can improve* — measured against a rubric, linked to a Trace, not vibes.
**Run it yourself.** This whole loop is a self-contained, SDK-driven script in the repo:
[`examples/self-improving-agent/`](https://github.com/Prism-Shadow/penguin-harness/tree/main/examples/self-improving-agent).
It uses a deterministic, readable rubric (file written · ≤2 sentences · exactly 3 bullets · under
60 words · facts accurate) and — because a local model is nondeterministic — averages several runs
per version, which is exactly why real benchmarks use a `runs` count. In our runs the averaged
score moved from **0.00/5** (blank `AGENTS.md`) to **5.00/5** (with the working-discipline edit),
and the script keeps N+1 only because the score strictly improved. It runs on the same local
Ollama + qwen3:8b setup and uses a dedicated agent id, so your own agents are untouched.
It uses a deterministic, readable rubric (10 points: 5 for content, 5 for the house convention that
lives only in `AGENTS.md`) and — because a local model is nondeterministic — averages several runs
per version, which is exactly why real benchmarks use a `runs` count. Crucially, the script never
writes the convention itself: the *agent* diagnoses the gap from a passing example and edits its own
`AGENTS.md`. In our runs the averaged score moved **4.6 → 6.6 → 9.8** across two self-authored
rounds (structure learned, then constants locked), and the harness keeps each round only because the
score strictly improved. It runs on the same local Ollama + qwen3.6:35b setup and uses a dedicated
agent id, so your own agents are untouched.
## No AMD GPU yet? Free cloud compute from AMD + Fireworks
@@ -154,25 +154,25 @@ spurious user-facing errors without requiring manual retries.
设计信条说起来容易。下面是一次真正践行了这些信条的端到端运行,全程在完全本地、开源权重、AMD GPU 的环境上进行——每一个 Token 都在本机生成,不向任何云端 API 发送。
**环境。** 一块 AMD GPU 运行 ROCm,通过 Ollama 的 OpenAI 兼容端点提供 `qwen3:8b`——一个开发者常用的本地开源权重模型(约 5 GB)。Ollama 原生识别到该 AMD GPU(无需任何架构 override),并把模型加载进显存。这条路径覆盖 AMD 受 ROCm 支持的整个产品线——从 Radeon PRO 工作站显卡(如 W7900,48 GB,RDNA3)一直到数据中心的 Instinct 加速卡。把它接入 PenguinHarness 只需一条命令,正因为 core 对任何 OpenAI 兼容端点一视同仁:
**环境。** 一块 AMD GPU 运行 ROCm,通过 Ollama 的 OpenAI 兼容端点提供 `qwen3.6:35b`——一个能力不俗的本地开源权重模型(约 24 GB 的 MoE,36B 总参数,Q4_K_M 量化)。Ollama 原生识别到该 AMD GPU(无需任何架构 override),并把模型加载进显存。这条路径覆盖 AMD 受 ROCm 支持的整个产品线——从 Radeon PRO 工作站显卡(如 W7900,48 GB,RDNA3)一直到数据中心的 Instinct 加速卡。把它接入 PenguinHarness 只需一条命令,正因为 core 对任何 OpenAI 兼容端点一视同仁:
```bash
penguin config model add \
--model-id qwen3:8b \
--model-id qwen3.6:35b \
--provider custom --client-type openai \
--base-url http://localhost:11434/v1 \
--api-key ollama --set-default
```
**任务,以及如何评分。** 我们给它一个仿照内置 `example-benchmark` 风格的约束型写作任务:读一个项目 notes 文件,写一份摘要——概述至多 2 句、恰好 3 条要点、全文不超过 60 词。任务附带一份私有 rubric——一份 Agent 永远看不到的评分清单,这样它就无法"照着答案作答"。rubric 把每条要求变成一分(文件确实写出 · 概述 ≤2 句 · 恰好 3 条要点 · 不超过 60 词 · 事实准确),满分 5 分。
**任务,以及如何评分。** 我们给它一个看似平平无奇的任务:读一个项目 notes 文件,写一份含 2 句概述和恰好 3 条要点的摘要——*并遵守团队的标准报告格式。* 玄机全在最后这句:这个“团队格式”是一套任意的内部约定——一行特定 marker、一个 `# Report: <subject>` 标题、一行 `Classification: INTERNAL`、一行 `Reviewed-by: Aurora Team` 页脚——它*只存在于 Agent 的 `AGENTS.md` 里*,无法从任务本身推断。任务附带一份私有 rubric——一份 Agent 永远看不到的清单——共 10 分:5 分是任何有能力的模型仅凭任务就能拿到的内容分,另外 5 分是只能从 `AGENTS.md` 得知的约定分。
**基线——以及它为何不是满分。** 跑在本地 AMD GPU 上,这个模型读了文件、组织出了一份相当合理的摘要……然后却从没把它写到磁盘。它把答案当成对话文本吐了出来,而没有调用写文件工具。没有交付物 → rubric 判它 **0 / 5**。而因为每一步都在 Trace 里,这不是靠猜——你可以打开这次运行,精确看到它错在哪。
**基线——以及它为何不是满分。** 跑在本地 AMD GPU 上、`AGENTS.md` 为空时,这个模型写出了一份相当合理的摘要——却稳定丢掉全部 5 个约定分,落在 **4.6 / 10** 左右。它别无选择:任务里没有任何东西透露那套内部约定,所以这是*信息缺口,不是能力缺口*——也正是为什么更强的模型也没法“自己想出来”。而因为每一步都在 Trace 里,这不是靠猜——你可以打开这次运行,精确看到哪些分被丢了。
这就是诚实的起点:在本地硬件上,一个开箱即用的模型面对约束型任务并不能一次拿满分。而这恰恰让下一节变得有意思——一个可度量、可审计的失败,是可以被系统性修复的。
这就是诚实的起点:在本地硬件上,一个开箱即用的 Agent,面对规则藏在它尚未学过的文件里的任务,并不能一次拿满分。而这恰恰让下一节变得有意思——一个可度量、可审计、*可复现*的失败,Agent 可以通过自学来系统性修复。
## 自我进化到底是怎么运作的——附一次真实的前后对比
上面那个 0 / 5 是一次*评估*——是这个 Agent 当前状态的一张快照。更有意思的问题是:PenguinHarness 如何把这张快照变成进步。这就是递归自我进化循环,值得作为一种"机制"来理解,因为它背后没有任何魔法引擎代码——它就是由 Skills 编排的普通 Agent 机制:
上面那个 4.6 / 10 是一次*评估*——是这个 Agent 当前状态的一张快照。更有意思的问题是:PenguinHarness 如何把这张快照变成进步。这就是递归自我进化循环,值得作为一种"机制"来理解,因为它背后没有任何魔法引擎代码——它就是由 Skills 编排的普通 Agent 机制:
1. **Benchmark(定基准)**——定义能力 Case,每个都配一份私有 rubric(如上)。
2. **Evaluate(评估)**——让 Agent 跑这些 Case 并按 rubric 打分。每一次运行都是一个普通的、被完整追踪的 Session。
@@ -180,17 +180,17 @@ penguin config model add \
4. **编辑 Agent 的状态**——Agent 的行为活在可编辑文件里(`AGENTS.md`、Skills、config)。你(或一个 Optimizer Agent)针对失分点去改这些文件,产出版本 N+1。
5. **快照 & 保留或回滚**——每轮前先打快照;只有当分数*严格提升*时才保留 N+1,否则回滚。
我们就来改进上一节里那个 Agent——同一个 `qwen3:8b`、同一块 AMD GPU、同一个刚拿了 0 / 5 的任务。
我们就来改进上一节里那个 Agent——同一个 `qwen3.6:35b`、同一块 AMD GPU、同一个刚拿了 4.6 / 10 的任务。关键之处在于:诊断和编辑都是 *Agent* 自己做的——我们并不把答案递给它。
- **诊断(来自 Trace)。** 这次失败只有一个具体原因:模型把摘要当对话文本产出,却从没调用工具去写 `summary.md`。Trace 把它摆在眼前——一次 `cat` 读入、然后是正文、没有写入。所以要修的不是"让模型更聪明",而是"让行为更可靠"。
- **编辑(N → N+1)。** 我们往这个 Agent 的 `AGENTS.md` 里加了一小段*任务纪律*:*先读原文、把任务约束逐条列成清单、真正调用工具把文件写出来、结束前重新读一遍输出做自检。* 改动仅此而已——几句话,写进一份 Agent 每次运行都会读的可编辑文本文件。没有重新训练、没有改代码。
- **重新评估。** 同一个模型、同一个任务,再跑一次:这次它把一份准确、切题的 `summary.md` 写到了磁盘,有一段 2 句的概述、恰好 3 条取自原文的要点——**4 / 5**(唯一还差的一分:它写到了 70 词,超出 60 词的预算)。
- **诊断(由 Agent 完成)。** 我们只给 Agent 两个文件,别无其他:它刚被拒的那份报告,以及*另一个*项目里一份通过了评审的报告。没有人告诉它规则。它对比两者,自己推断出可复用的内部约定——marker、标题形态、元数据行、签名页脚。这就是"读 Trace 定位失分点"那一步,只不过由 Agent 亲自完成。
- **编辑(N → N+1),由 Agent 撰写。** Agent 把刚推断出的约定写进它自己的 `AGENTS.md`。从单一范例里它能正确还原出*结构*,但还分不清哪些 token 是固定常量、哪些是每份报告要替换的字段(单个范例本就有歧义),于是把 marker 泛化成了占位符。重新评估,分数升到约 **6.6 / 10**。没有重新训练、没有改代码——Agent 编辑的是一份它每次运行都会读的文本文件。
- **递归(N+1 → N+2)。** 现在我们再给它*多份*来自不同项目、却共享同一 marker 和签名的通过报告。Agent 推断出:凡是在所有样本里都完全相同的,必是固定常量;于是它读取自己*上一轮*写的 N+1 `AGENTS.md` 并精炼它——把 `<!-- ACME-DATA-PLATFORM -->` 和 `Reviewed-by: Aurora Team` 锁定为字面量。再评估一次:约 **9.8 / 10**。这才是真正意义上的递归:`state_{n+1} = agent.reflect(state_n, 新证据)`。
在同一个模型、同一个任务上,仅仅通过编辑一份 Agent 会读取的文本文件,分数就提升了。这就是循环的微缩版:*你能看到的,你(或一个 Optimizer Agent)就能改进*——而且因为分数来自一份 rubric、又关联着 Trace,这份改进是被度量出来的,不是拍脑袋。这正是"只有分数严格提升才保留"的循环,转动了一格。
在同一个模型、同一个任务上,仅仅由 Agent 自己编辑一份它会读取的文本文件,分数就走出了 **4.6 → 6.6 → 9.8** 的轨迹——而且 harness 每一轮都只因分数严格提升才保留。这就是自驱动的循环:*Agent 能看到的,Agent 就能改进*——用一份 rubric 度量、关联着 Trace,不是拍脑袋。
**你可以自己跑一遍。** 整个循环在仓库里有一个自包含、纯 SDK 驱动的脚本:
[`examples/self-improving-agent/`](https://github.com/Prism-Shadow/penguin-harness/tree/main/examples/self-improving-agent)。
它用一份确定性的、可读的 rubric(文件是否写出 · 概述 ≤2 句 · 恰好 3 条要点 · 不超过 60 词 · 事实准确),并且——因为本地模型有随机性——对每个版本跑多次取平均分,这正是真实 benchmark 里 `runs`(多次运行)的意义所在。在我们的运行中,平均分从 **0.00/5**(空 `AGENTS.md`)提升到 **5.00/5**(加入"任务纪律"后),脚本仅因分数严格提升才保留 N+1。它跑在同一套本地 Ollama + qwen3:8b 环境上,并使用一个专用 agent id,所以你自己的 Agent 不会被动到。
它用一份确定性的、可读的 rubric(10 分:5 分内容分、5 分只存在于 `AGENTS.md` 的内部约定分),并且——因为本地模型有随机性——对每个版本跑多次取平均分,这正是真实 benchmark 里 `runs`(多次运行)的意义所在。关键在于:脚本从不自己写那套约定;是 *Agent* 从一份通过范例里诊断出缺口,并编辑自己的 `AGENTS.md`。在我们的运行中,平均分沿着 **4.6 → 6.6 → 9.8** 走过两轮自撰改进(先学到结构、再锁定常量),harness 每一轮都只因分数严格提升才保留。它跑在同一套本地 Ollama + qwen3.6:35b 环境上,并使用一个专用 agent id,所以你自己的 Agent 不会被动到。
## 还没有 AMD GPU?来自 AMD + Fireworks 的免费云端算力