docs(examples): make self-improvement demo genuinely self-evolving (#29)
This commit is contained in:
Zhang Jason
@@ -3,36 +3,60 @@
|
||||
# Example: an Agent that improves itself (local, on an AMD GPU via Ollama)
|
||||
|
||||
This example is the **"Recursive Self-Improvement"** pillar in runnable code. Using only the
|
||||
PenguinHarness SDK, it runs one turn of the self-improvement loop:
|
||||
PenguinHarness SDK, the agent runs the self-improvement loop on **itself** — and the key point is
|
||||
that the improvement is authored by the *agent*, not hardcoded by the script:
|
||||
|
||||
1. **Evaluate** a constrained writing task and score it against a rubric.
|
||||
2. **Diagnose** which rubric points were lost (from the run itself).
|
||||
3. **Edit** the agent's own `AGENTS.md` to address the failure (version N+1).
|
||||
4. **Re-evaluate** the same task and keep the change only if the score improved.
|
||||
1. **Evaluate** — the agent attempts a constrained task and is scored against a rubric (plain code).
|
||||
2. **Diagnose** — *the agent itself* reads a failing result next to a passing example.
|
||||
3. **Edit** — *the agent itself* infers the missing rule and writes it into its own `AGENTS.md`.
|
||||
4. **Re-evaluate** — the same task again; the change is kept only if the mean score improved.
|
||||
|
||||
Everything runs on a **local open-weight model** — `qwen3:8b` served by Ollama — so no cloud API
|
||||
Everything runs on a **local open-weight model** — `qwen3.6:35b` served by Ollama — so no cloud API
|
||||
and no data leaving the machine. Ollama's ROCm backend runs this natively on AMD GPUs.
|
||||
|
||||
## Why a deterministic rubric, and why averaging
|
||||
## The three scripts (from mechanism to real self-evolution)
|
||||
|
||||
- The **rubric is plain code you can read** (`score()` in `self-improve.ts`): file actually
|
||||
written · overview ≤ 2 sentences · exactly 3 bullets · under 60 words · key facts present. No
|
||||
hidden judge — the before/after numbers are objective and reproducible. In the full product the
|
||||
Evaluator is driven by the `agent-evaluation` skill against a *private* rubric; this example
|
||||
distills that idea to its runnable core.
|
||||
- A local model is **nondeterministic**, so the example runs each version several times and
|
||||
averages — which is exactly why real benchmarks use a `runs` count per case. A single run can
|
||||
swing; the mean is what tells you whether the edit actually helped.
|
||||
| Script | What it demonstrates | `AGENTS.md` edited by |
|
||||
| --- | --- | --- |
|
||||
| `self-improve.ts` | The **scoring loop** in miniature (simplest) | the **script** (hardcoded) |
|
||||
| `self-evolve.ts` | **Genuine** single-round self-evolution | the **agent** itself |
|
||||
| `self-evolve-recursive.ts` | **Multi-round recursion** — the main line (`pnpm start`) | the **agent**, over two rounds |
|
||||
|
||||
`self-improve.ts` is the honest baseline: it shows the evaluate → edit → re-evaluate machinery, but
|
||||
the edit is a human-written `DISCIPLINE` string the script writes to disk. That demonstrates the
|
||||
*loop*, not self-evolution. The two `self-evolve*` scripts move the diagnosis **and** the edit into
|
||||
the agent — the script only supplies a failure signal and one worked example; the model does the
|
||||
learning and persists it into its own identity file.
|
||||
|
||||
## The task, and why the baseline stably fails
|
||||
|
||||
The task looks trivial: *summarize `notes.txt` into `summary.md` with a 2-sentence overview and
|
||||
exactly 3 key facts — and follow your team's standard report format.* The catch is the last clause:
|
||||
the "team format" is an **arbitrary house convention** (a marker line, a `# Report: <subject>`
|
||||
title, a `Classification: INTERNAL` line, a `Reviewed-by: Aurora Team` footer) that appears **only
|
||||
in `AGENTS.md`** and **cannot be inferred from the task**.
|
||||
|
||||
This is the design trick that makes the baseline *stably* bad — for any model, strong or weak. The
|
||||
rubric (`score()`, plain code) has 10 atomic points: **5 content points** any capable model earns
|
||||
from the task alone, and **5 convention points** that are knowable only from `AGENTS.md`. With a
|
||||
blank `AGENTS.md` the agent cannot guess the convention, so it stably loses those 5 points. This is
|
||||
an **information gap, not a capability gap** — the reason a stronger model can't just "figure it
|
||||
out". It also mirrors the full product, where the Evaluator is driven by the `agent-evaluation`
|
||||
skill against a *private* rubric.
|
||||
|
||||
The agent closes the gap by **learning the convention from a passing example** and writing it into
|
||||
its own `AGENTS.md` — and in the recursive script, by locking down the fixed constants once it has
|
||||
seen several examples.
|
||||
|
||||
## 1–2. Serve the model and point PenguinHarness at it
|
||||
|
||||
```bash
|
||||
export HIP_VISIBLE_DEVICES=0 # optional: pin a specific AMD GPU
|
||||
ollama serve &
|
||||
ollama pull qwen3:8b
|
||||
ollama pull qwen3.6:35b
|
||||
|
||||
penguin config model add \
|
||||
--model-id qwen3:8b \
|
||||
--model-id qwen3.6:35b \
|
||||
--provider custom --client-type openai \
|
||||
--base-url http://localhost:11434/v1 \
|
||||
--api-key ollama --set-default
|
||||
@@ -43,35 +67,64 @@ penguin config model add \
|
||||
```bash
|
||||
pnpm install
|
||||
pnpm build
|
||||
pnpm --dir examples/self-improving-agent start
|
||||
# or directly: npx tsx examples/self-improving-agent/self-improve.ts
|
||||
pnpm --dir examples/self-improving-agent start # the recursive main line
|
||||
# or a single round: npx tsx examples/self-improving-agent/self-evolve.ts
|
||||
# or the scoring-loop baseline: npx tsx examples/self-improving-agent/self-improve.ts
|
||||
```
|
||||
|
||||
## What you should see
|
||||
|
||||
```text
|
||||
BASELINE (blank AGENTS.md): 3 runs
|
||||
run 1: 0/5
|
||||
run 2: 0/5
|
||||
run 3: 0/5
|
||||
BASELINE mean: 0.00/5
|
||||
N+1 (with working discipline): 3 runs
|
||||
run 1: 5/5
|
||||
run 2: 5/5
|
||||
run 3: 5/5
|
||||
N+1 mean: 5.00/5
|
||||
=== Self-improvement result ===
|
||||
baseline: 0.00/5 → N+1: 5.00/5
|
||||
Mean score improved — keep version N+1. ✔
|
||||
N BASELINE (blank AGENTS.md): 5 runs
|
||||
... scores: [5, 4, 5, 5, 5] mean: 4.80/10 # loses every convention point
|
||||
|
||||
=== REFLECT round 1: infer STRUCTURE from a single accepted report ===
|
||||
----- after round 1: AGENTS.md the agent authored -----
|
||||
## Report-Publishing Convention ...
|
||||
<!-- <UPPERCASE_PROJECT_ID> --> # structure learned, constants still guessed
|
||||
Reviewed-by: <Team Name>
|
||||
|
||||
N+1 (structure learned): 5 runs
|
||||
... scores: [6, 7, 6, 6, 8] mean: 6.60/10
|
||||
|
||||
=== REFLECT round 2: RECURSE on own AGENTS.md — lock CONSTANTS from 3 examples ===
|
||||
----- after round 2: AGENTS.md the agent authored -----
|
||||
### Fixed Constants (verbatim)
|
||||
| 1 | <!-- ACME-DATA-PLATFORM --> | # constants now locked to literals
|
||||
| Last line | Reviewed-by: Aurora Team |
|
||||
|
||||
N+2 (constants locked): 5 runs
|
||||
... scores: [10, 10, 9, 10, 10] mean: 9.80/10
|
||||
|
||||
=== Recursive self-evolution trajectory ===
|
||||
N (baseline): 4.80/10
|
||||
N+1 (structure): 6.60/10 (+1.80)
|
||||
N+2 (constants): 9.80/10 (+3.20)
|
||||
Monotonic improvement across two self-authored rounds — recursive self-evolution. ✔
|
||||
```
|
||||
|
||||
With a blank `AGENTS.md`, `qwen3:8b` tends to *narrate* the summary in chat and never call the
|
||||
tool to write the file — so the rubric scores it 0. Adding a short "working discipline" section
|
||||
(read first, restate the constraints, actually write the file, self-check) flips that. Exact
|
||||
numbers vary run to run; the averaged direction is the point.
|
||||
The trajectory is the point: from one passing example the agent infers the **structure** but can't
|
||||
tell which tokens are fixed constants vs per-report fields (a single example is ambiguous), so it
|
||||
stalls mid-way. Given **several** accepted reports that share the same marker and sign-off, it
|
||||
infers "whatever is identical across all of them is a fixed constant", reads its **own** round-1
|
||||
`AGENTS.md`, and refines it — recursion in the true sense: `state_{n+1} = agent.reflect(state_n,
|
||||
new_evidence)`. Exact numbers vary run to run; the monotonic direction is the point.
|
||||
|
||||
## Honest boundaries
|
||||
|
||||
- **Self-evolution needs a capable model.** Authoring a correct rule and then *following it* is
|
||||
harder than benefiting from a hardcoded one. A weaker model (e.g. `qwen3:8b`) often infers the
|
||||
rule but executes it unreliably (placeholders left unfilled, or narrating instead of writing the
|
||||
file), so its N+1 can *regress* and the script rolls back. That rollback is the loop being honest,
|
||||
not a bug.
|
||||
- **Evidence sufficiency is real.** N+1 stalls at a middling score because one example genuinely
|
||||
under-determines the fixed constants; N+2 only climbs because more examples make them inferable.
|
||||
The gain is information-driven learning, not noise.
|
||||
|
||||
## Notes
|
||||
|
||||
- Uses a dedicated agent id (`self-improve-demo`), created on the fly — your own agents are
|
||||
untouched.
|
||||
- Re-running updates that demo agent in place.
|
||||
- The script never writes the convention itself; it only supplies the failing report, the accepted
|
||||
example(s), and the keep/roll-back signal.
|
||||
- Re-running resets the demo agent's `AGENTS.md` to blank and starts the loop over.
|
||||
|
||||
@@ -2,34 +2,55 @@
|
||||
|
||||
# 示例：一个自我改进的 Agent（本地，经 Ollama 跑在 AMD GPU 上）
|
||||
|
||||
这个示例是 **“递归自我进化”** 支柱的可运行版本。仅用 PenguinHarness SDK，它会跑完自我进化循环的一轮：
|
||||
这个示例是 **“递归自我进化”** 支柱的可运行版本。仅用 PenguinHarness SDK，Agent 就在**自己身上**跑完
|
||||
自我进化循环——而关键在于：这次改进是由 *Agent 自己*产出的，不是脚本硬编码写死的：
|
||||
|
||||
1. **评估（Evaluate）** —— 让 Agent 完成一个约束型写作任务，并按 rubric 打分。
|
||||
2. **诊断（Diagnose）** —— 从运行结果中看出哪些 rubric 项失了分。
|
||||
3. **编辑（Edit）** —— 改写 Agent 自己的 `AGENTS.md` 来修复失分点（版本 N+1）。
|
||||
4. **重新评估（Re-evaluate）** —— 重跑同一任务，只有分数提升时才保留这次改动。
|
||||
1. **评估（Evaluate）** —— Agent 尝试一个约束型任务，并按 rubric（普通代码）打分。
|
||||
2. **诊断（Diagnose）** —— *Agent 自己*把一次失败结果与一份通过范例并排对比。
|
||||
3. **编辑（Edit）** —— *Agent 自己*归纳出缺失的规则，并写进它自己的 `AGENTS.md`。
|
||||
4. **重新评估（Re-evaluate）** —— 重跑同一任务；只有平均分提升时才保留这次改动。
|
||||
|
||||
全程跑在一个**本地开源权重模型**上——由 Ollama 提供的 `qwen3:8b`——所以不用任何云端 API、数据也不
|
||||
全程跑在一个**本地开源权重模型**上——由 Ollama 提供的 `qwen3.6:35b`——所以不用任何云端 API、数据也不
|
||||
离开本机。Ollama 的 ROCm 后端能原生在 AMD GPU 上运行它。
|
||||
|
||||
## 为什么用确定性 rubric，为什么要取平均
|
||||
## 三个脚本（从机制到真正的自我进化）
|
||||
|
||||
- **rubric 就是你能读懂的普通代码**（`self-improve.ts` 里的 `score()`）：文件是否确实写出 · 概述
|
||||
≤ 2 句 · 恰好 3 条要点 · 不超过 60 词 · 关键事实是否出现。没有隐藏的裁判——前后分数是客观、可复现的。
|
||||
在完整产品里，Evaluator 由 `agent-evaluation` skill 按一份*私有* rubric 驱动；这个示例把那个理念
|
||||
蒸馏成可运行的核心。
|
||||
- 本地模型是**非确定性**的，所以示例对每个版本各跑多次取平均——这正是真实 benchmark 里为每个 Case
|
||||
设置 `runs`（多次运行）的原因。单次可能上下波动；均值才能告诉你这次编辑到底有没有帮助。
|
||||
| 脚本 | 演示什么 | 由谁编辑 `AGENTS.md` |
|
||||
| --- | --- | --- |
|
||||
| `self-improve.ts` | **评分闭环**的微缩版（最简） | **脚本**（硬编码） |
|
||||
| `self-evolve.ts` | **真正的**单轮自我进化 | **Agent** 自己 |
|
||||
| `self-evolve-recursive.ts` | **多轮递归**——主线（`pnpm start`） | **Agent**，连续两轮 |
|
||||
|
||||
`self-improve.ts` 是诚实的基线：它展示了 评估 → 编辑 → 重新评估 的机制，但那次编辑是人写好的
|
||||
`DISCIPLINE` 字符串、由脚本写到磁盘。它演示的是*循环*，不是自我进化。两个 `self-evolve*` 脚本把
|
||||
诊断**和**编辑都交给 Agent——脚本只提供一个失败信号和一份范例；学习由模型完成，并被固化进它自己的
|
||||
身份文件。
|
||||
|
||||
## 任务，以及为什么 baseline 会稳定失败
|
||||
|
||||
任务看起来很简单：*把 `notes.txt` 总结进 `summary.md`，含一段 2 句概述和恰好 3 条要点——并遵守你们
|
||||
团队的标准报告格式。* 玄机在最后一句：这个“团队格式”是一套**任意的内部约定**（一行 marker、一个
|
||||
`# Report: <subject>` 标题、一行 `Classification: INTERNAL`、一行 `Reviewed-by: Aurora Team` 页脚），
|
||||
它**只出现在 `AGENTS.md` 里**，**无法从任务本身推断**。
|
||||
|
||||
这正是让 baseline *稳定*烂的设计诀窍——对任何模型都成立，无论强弱。rubric（`score()`，普通代码）有
|
||||
10 个原子分点：**5 个内容点**任何有能力的模型仅凭任务就能拿到，**5 个约定点**只能从 `AGENTS.md` 得知。
|
||||
`AGENTS.md` 为空时，Agent 猜不到约定，于是稳定丢掉那 5 分。这是**信息缺口，不是能力缺口**——也正是
|
||||
为什么更强的模型也没法“自己想出来”。它也对应了完整产品的形态：那里 Evaluator 由 `agent-evaluation`
|
||||
skill 按一份*私有* rubric 驱动。
|
||||
|
||||
Agent 通过**从一份通过范例里学到约定**、并把它写进自己的 `AGENTS.md` 来弥合这个缺口——在递归脚本里，
|
||||
更进一步：见过多份范例后，把其中的固定常量锁定下来。
|
||||
|
||||
## 1–2. 提供模型并把 PenguinHarness 指向它
|
||||
|
||||
```bash
|
||||
export HIP_VISIBLE_DEVICES=0 # 可选：指定某张 AMD GPU
|
||||
ollama serve &
|
||||
ollama pull qwen3:8b
|
||||
ollama pull qwen3.6:35b
|
||||
|
||||
penguin config model add \
|
||||
--model-id qwen3:8b \
|
||||
--model-id qwen3.6:35b \
|
||||
--provider custom --client-type openai \
|
||||
--base-url http://localhost:11434/v1 \
|
||||
--api-key ollama --set-default
|
||||
@@ -40,33 +61,58 @@ penguin config model add \
|
||||
```bash
|
||||
pnpm install
|
||||
pnpm build
|
||||
pnpm --dir examples/self-improving-agent start
|
||||
# 或直接运行： npx tsx examples/self-improving-agent/self-improve.ts
|
||||
pnpm --dir examples/self-improving-agent start # 递归主线
|
||||
# 或单轮： npx tsx examples/self-improving-agent/self-evolve.ts
|
||||
# 或评分闭环基线： npx tsx examples/self-improving-agent/self-improve.ts
|
||||
```
|
||||
|
||||
## 你应当看到什么
|
||||
|
||||
```text
|
||||
BASELINE (blank AGENTS.md): 3 runs
|
||||
run 1: 0/5
|
||||
run 2: 0/5
|
||||
run 3: 0/5
|
||||
BASELINE mean: 0.00/5
|
||||
N+1 (with working discipline): 3 runs
|
||||
run 1: 5/5
|
||||
run 2: 5/5
|
||||
run 3: 5/5
|
||||
N+1 mean: 5.00/5
|
||||
=== Self-improvement result ===
|
||||
baseline: 0.00/5 → N+1: 5.00/5
|
||||
Mean score improved — keep version N+1. ✔
|
||||
N BASELINE (blank AGENTS.md): 5 runs
|
||||
... scores: [5, 4, 5, 5, 5] mean: 4.80/10 # 稳定丢掉每个约定点
|
||||
|
||||
=== REFLECT round 1: infer STRUCTURE from a single accepted report ===
|
||||
----- after round 1: AGENTS.md the agent authored -----
|
||||
## Report-Publishing Convention ...
|
||||
<!-- <UPPERCASE_PROJECT_ID> --> # 学到结构，但常量还是猜的
|
||||
Reviewed-by: <Team Name>
|
||||
|
||||
N+1 (structure learned): 5 runs
|
||||
... scores: [6, 7, 6, 6, 8] mean: 6.60/10
|
||||
|
||||
=== REFLECT round 2: RECURSE on own AGENTS.md — lock CONSTANTS from 3 examples ===
|
||||
----- after round 2: AGENTS.md the agent authored -----
|
||||
### Fixed Constants (verbatim)
|
||||
| 1 | <!-- ACME-DATA-PLATFORM --> | # 常量被锁定为字面量
|
||||
| Last line | Reviewed-by: Aurora Team |
|
||||
|
||||
N+2 (constants locked): 5 runs
|
||||
... scores: [10, 10, 9, 10, 10] mean: 9.80/10
|
||||
|
||||
=== Recursive self-evolution trajectory ===
|
||||
N (baseline): 4.80/10
|
||||
N+1 (structure): 6.60/10 (+1.80)
|
||||
N+2 (constants): 9.80/10 (+3.20)
|
||||
Monotonic improvement across two self-authored rounds — recursive self-evolution. ✔
|
||||
```
|
||||
|
||||
当 `AGENTS.md` 为空时，`qwen3:8b` 往往会在对话里*叙述*摘要，却从不调用工具把文件写出来——于是
|
||||
rubric 判它 0 分。加上一小段“任务纪律”（先读原文、把约束逐条列出、真正把文件写出来、结束前自检）就能
|
||||
扭转这一点。具体数字每次运行会有波动；重点是取平均后的方向。
|
||||
重点在这条轨迹：只有一份通过范例时，Agent 能推断出**结构**，却分不清哪些 token 是固定常量、哪些是
|
||||
每份报告要替换的字段（单个样本本就有歧义），于是卡在中途。给它**多份**共享同一 marker 和页脚的通过
|
||||
范例后，它推断出“凡是在所有样本里都完全相同的，就是固定常量”，读取自己**上一轮**写的 `AGENTS.md`
|
||||
再精炼它——这才是真正意义上的递归：`state_{n+1} = agent.reflect(state_n, 新证据)`。具体数字每次运行
|
||||
会有波动；重点是那条单调上升的方向。
|
||||
|
||||
## 诚实的边界
|
||||
|
||||
- **自我进化需要一个足够强的模型。** 自己写出一条正确规则、然后*遵守它*，比坐享一条硬编码规则更难。
|
||||
较弱的模型（例如 `qwen3:8b`）常常能推断出规则却执行不稳（占位符没填、或者只叙述而不真正写文件），
|
||||
于是它的 N+1 可能*倒退*，脚本随之回滚。那次回滚是循环在诚实工作，不是 bug。
|
||||
- **证据充分性是真实的。** N+1 卡在中位，是因为单份范例确实无法确定固定常量；N+2 能冲高，只因更多
|
||||
范例让常量变得可推断。这份提升是信息驱动的学习，不是噪声。
|
||||
|
||||
## 说明
|
||||
|
||||
- 使用一个专用 agent id（`self-improve-demo`），运行时即时创建——你自己的 Agent 不会被动到。
|
||||
- 再次运行会就地更新这个 demo agent。
|
||||
- 脚本从不自己写那套约定；它只提供失败报告、通过范例，以及保留/回滚的信号。
|
||||
- 再次运行会把这个 demo agent 的 `AGENTS.md` 重置为空，并重新开始循环。
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"start": "tsx self-improve.ts"
|
||||
"start": "tsx self-evolve-recursive.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@prismshadow/penguin-core": "workspace:*"
|
||||
|
||||
@@ -0,0 +1,287 @@
|
||||
/**
|
||||
* RECURSIVE self-evolution — the agent improves its OWN AGENTS.md over MULTIPLE rounds, each round
|
||||
* building on the state it authored in the previous round.
|
||||
*
|
||||
* Why a second round is needed (and is the interesting part):
|
||||
* In round 1 the agent sees ONE accepted report (Borealis). It can infer the STRUCTURE
|
||||
* (marker line / title / classification / footer) but it CANNOT know which tokens are fixed
|
||||
* constants vs per-report variables — a single example is ambiguous. So it generalizes the
|
||||
* marker to a placeholder and guesses the sign-off, and stably lands mid (~7/10).
|
||||
*
|
||||
* In round 2 the agent is given SEVERAL accepted reports from DIFFERENT projects that all share
|
||||
* the SAME marker (`<!-- ACME-DATA-PLATFORM -->`) and the SAME footer (`Reviewed-by: Aurora
|
||||
* Team`). Now the invariants are inferable: whatever is identical across every accepted report is
|
||||
* a FIXED constant; whatever varies is a field. The agent reads its OWN round-1 AGENTS.md and
|
||||
* REFINES it to lock those constants down → N+2 climbs to near-ceiling.
|
||||
*
|
||||
* This is recursion in the true sense: state_{n+1} = agent.reflect(state_n, new_evidence). The
|
||||
* script never writes a rule; it only supplies evidence and the keep/rollback signal.
|
||||
*
|
||||
* Run: cd examples/self-improving-agent && pnpm exec tsx self-evolve-recursive.ts
|
||||
*/
|
||||
import { promises as fs } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import {
|
||||
createAgent,
|
||||
userText,
|
||||
isCompleteModelMessage,
|
||||
type OmniMessage,
|
||||
} from "@prismshadow/penguin-core";
|
||||
|
||||
const AGENT_ID = "self-improve-demo";
|
||||
const PROJECT_ID = "default_project";
|
||||
|
||||
const TASK = `Read notes.txt in your workspace and write summary.md that summarizes the notes with:
|
||||
(1) an overview of at most 2 sentences, and
|
||||
(2) a bullet list of exactly 3 key facts.
|
||||
Follow your team's standard report format.`;
|
||||
|
||||
const NOTES = `Project Aurora — Internal Notes
|
||||
|
||||
Aurora is a real-time analytics platform launched in Q1 2026. At peak it ingests roughly
|
||||
2 million events per second through a Kafka-based pipeline. The core query engine was rewritten
|
||||
in Rust after the original Go version could not keep p99 latency under control; the rewrite cut
|
||||
p99 from 800ms to 120ms.
|
||||
|
||||
The production deployment is single-region with no failover; a multi-region rollout is scheduled
|
||||
for Q3 2026. Storage is the largest cost line at about $48,000/month, driven by a 90-day
|
||||
hot-retention policy; cutting retention to 30 days would reduce storage cost by roughly 55%.
|
||||
`;
|
||||
|
||||
/** Round 1: a SINGLE accepted report. Structure is inferable; the fixed constants are ambiguous. */
|
||||
const REFERENCE_1 = `<!-- ACME-DATA-PLATFORM -->
|
||||
# Report: Project Borealis
|
||||
Classification: INTERNAL
|
||||
|
||||
Borealis is a batch ETL platform migrated to Spark in 2025; it processes about 40TB nightly.
|
||||
|
||||
- Cut the nightly window from 6h to 90 minutes
|
||||
- Runs on a 200-node autoscaling cluster
|
||||
- Compute costs roughly $12,000/month
|
||||
|
||||
Reviewed-by: Aurora Team
|
||||
`;
|
||||
|
||||
/** Round 2: SEVERAL accepted reports from different projects. The marker and the sign-off are
|
||||
* IDENTICAL across all of them (→ fixed constants); the title subject and body vary (→ fields). */
|
||||
const REFERENCE_2 = `<!-- ACME-DATA-PLATFORM -->
|
||||
# Report: Project Cascade
|
||||
Classification: INTERNAL
|
||||
|
||||
Cascade is a streaming feature store rolled out in 2026 serving 500 models online.
|
||||
|
||||
- p99 read latency 8ms at 300k QPS
|
||||
- Backed by a Redis + RocksDB tier
|
||||
- Operates at about $9,500/month
|
||||
|
||||
Reviewed-by: Aurora Team
|
||||
`;
|
||||
const REFERENCE_3 = `<!-- ACME-DATA-PLATFORM -->
|
||||
# Report: Project Delta
|
||||
Classification: INTERNAL
|
||||
|
||||
Delta is an internal experimentation platform launched in Q4 2025 running 1,200 concurrent tests.
|
||||
|
||||
- Reduced experiment setup from days to under an hour
|
||||
- Guardrail metrics evaluated hourly
|
||||
- Infra footprint about $6,000/month
|
||||
|
||||
Reviewed-by: Aurora Team
|
||||
`;
|
||||
|
||||
function rootDir(): string {
|
||||
return process.env.PENGUIN_HOME ?? path.join(os.homedir(), ".penguin", "data");
|
||||
}
|
||||
function agentStateDir(): string {
|
||||
return path.join(rootDir(), PROJECT_ID, "agents", AGENT_ID, "agent_state");
|
||||
}
|
||||
|
||||
function score(summaryText: string | null): { score: number; detail: string[] } {
|
||||
const detail: string[] = [];
|
||||
let s = 0;
|
||||
const add = (ok: boolean, label: string) => {
|
||||
if (ok) s += 1;
|
||||
detail.push(`${ok ? "1" : "0"}/1 ${label}`);
|
||||
};
|
||||
|
||||
const exists = summaryText !== null;
|
||||
add(exists, "file summary.md was written");
|
||||
if (!exists) return { score: 0, detail };
|
||||
const text = summaryText!;
|
||||
const lines = text.split("\n");
|
||||
|
||||
const bullets = (text.match(/^\s*[-*]\s+/gm) ?? []).length;
|
||||
add(bullets === 3, `exactly 3 bullet facts (found ${bullets})`);
|
||||
add(text.includes("120ms"), "mentions the p99 latency figure (120ms)");
|
||||
add(text.includes("2 million") || /2m\s*events/i.test(text), "mentions the throughput figure");
|
||||
add(text.includes("55%") || text.includes("48"), "mentions a storage cost/retention figure");
|
||||
|
||||
add(lines[0]?.trim() === "<!-- ACME-DATA-PLATFORM -->", "[convention] line 1 is the ACME marker");
|
||||
add(
|
||||
/^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()),
|
||||
"[convention] line 2 is '# Report: <Aurora subject>'",
|
||||
);
|
||||
add(
|
||||
/^Classification:\s*INTERNAL\s*$/m.test(text),
|
||||
"[convention] contains 'Classification: INTERNAL'",
|
||||
);
|
||||
const lastNonEmpty = [...lines].reverse().find((l) => l.trim() !== "") ?? "";
|
||||
add(
|
||||
lastNonEmpty.trim() === "Reviewed-by: Aurora Team",
|
||||
"[convention] final line is 'Reviewed-by: Aurora Team'",
|
||||
);
|
||||
const allFour =
|
||||
lines[0]?.trim() === "<!-- ACME-DATA-PLATFORM -->" &&
|
||||
/^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()) &&
|
||||
/^Classification:\s*INTERNAL\s*$/m.test(text) &&
|
||||
lastNonEmpty.trim() === "Reviewed-by: Aurora Team";
|
||||
add(allFour, "[convention] full team format satisfied end-to-end");
|
||||
|
||||
return { score: s, detail };
|
||||
}
|
||||
|
||||
async function collect(run: AsyncGenerator<OmniMessage>): Promise<void> {
|
||||
for await (const msg of run) {
|
||||
if (isCompleteModelMessage(msg) && msg.payload.type === "text") {
|
||||
/* agent chatter suppressed */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function runOnce(): Promise<{ score: number; detail: string[]; report: string | null }> {
|
||||
const agent = await createAgent({ agentId: AGENT_ID });
|
||||
const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-r-"));
|
||||
await fs.writeFile(path.join(ws, "notes.txt"), NOTES, "utf8");
|
||||
const session = await agent.createSession({ workspaceDir: ws });
|
||||
await collect(session.run([userText(TASK)], { approve: async () => "allow" }));
|
||||
let report: string | null = null;
|
||||
try {
|
||||
report = await fs.readFile(path.join(ws, "summary.md"), "utf8");
|
||||
} catch {
|
||||
report = null;
|
||||
}
|
||||
const { score: sc, detail } = score(report);
|
||||
return { score: sc, detail, report };
|
||||
}
|
||||
|
||||
const RUNS = 5;
|
||||
let lastReport = "";
|
||||
async function evaluate(label: string): Promise<number> {
|
||||
console.log(`\n--- ${label}: ${RUNS} runs ---`);
|
||||
const scores: number[] = [];
|
||||
for (let i = 0; i < RUNS; i++) {
|
||||
const r = await runOnce();
|
||||
scores.push(r.score);
|
||||
if (r.report) lastReport = r.report;
|
||||
const missed = r.detail.filter((d) => d.startsWith("0")).map((d) => d.replace(/^0\/1\s+/, ""));
|
||||
console.log(` run ${i + 1}: ${r.score}/10 [missed: ${missed.join("; ") || "none"}]`);
|
||||
}
|
||||
const mean = scores.reduce((a, b) => a + b, 0) / scores.length;
|
||||
console.log(` ${label} scores: [${scores.join(", ")}] mean: ${mean.toFixed(2)}/10`);
|
||||
return mean;
|
||||
}
|
||||
|
||||
/** Round 1 reflection: infer STRUCTURE from a single accepted example + the agent's own failure. */
|
||||
async function reflectRound1(failedReport: string): Promise<void> {
|
||||
const agent = await createAgent({ agentId: AGENT_ID });
|
||||
const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-r1-"));
|
||||
await fs.writeFile(path.join(ws, "last_report.md"), failedReport, "utf8");
|
||||
await fs.writeFile(path.join(ws, "accepted_1.md"), REFERENCE_1, "utf8");
|
||||
const agentsMd = path.join(agentStateDir(), "AGENTS.md");
|
||||
const prompt = `You are improving your own future performance by editing your persistent instructions.
|
||||
|
||||
In your workspace:
|
||||
- last_report.md — a report you wrote that the team REJECTED.
|
||||
- accepted_1.md — a DIFFERENT project's report that PASSED review.
|
||||
|
||||
Nobody has told you the rules. Read both, infer the team's reusable report-publishing convention
|
||||
(marker line, title format, metadata line, footer/sign-off), and WRITE it as durable general
|
||||
guidance into your AGENTS.md at this absolute path:
|
||||
${agentsMd}
|
||||
(Overwrite the file.) Capture the fixed lines and the per-report variable parts. Do not copy the
|
||||
example's specific content. Use your file tools to read and to write, then confirm.`;
|
||||
const session = await agent.createSession({ workspaceDir: ws });
|
||||
await collect(session.run([userText(prompt)], { approve: async () => "allow" }));
|
||||
}
|
||||
|
||||
/** Round 2 reflection: RECURSE on the agent's own AGENTS.md using MULTIPLE accepted examples so the
|
||||
* invariants (constant marker + constant sign-off) become inferable and can be locked down. */
|
||||
async function reflectRound2(): Promise<void> {
|
||||
const agent = await createAgent({ agentId: AGENT_ID });
|
||||
const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-r2-"));
|
||||
await fs.writeFile(path.join(ws, "accepted_1.md"), REFERENCE_1, "utf8");
|
||||
await fs.writeFile(path.join(ws, "accepted_2.md"), REFERENCE_2, "utf8");
|
||||
await fs.writeFile(path.join(ws, "accepted_3.md"), REFERENCE_3, "utf8");
|
||||
const agentsMd = path.join(agentStateDir(), "AGENTS.md");
|
||||
const prompt = `You are refining your own report-publishing convention. Your current convention is
|
||||
already written in your AGENTS.md at:
|
||||
${agentsMd}
|
||||
Read it first.
|
||||
|
||||
Your workspace now has THREE accepted reports from DIFFERENT projects: accepted_1.md,
|
||||
accepted_2.md, accepted_3.md. Compare them carefully. The key insight of this round:
|
||||
|
||||
Anything that is IDENTICAL, character-for-character, across ALL THREE accepted reports is a
|
||||
FIXED CONSTANT that you must reproduce VERBATIM in every report (for example a specific marker
|
||||
line or a specific sign-off line). Anything that DIFFERS between them (like the project name in
|
||||
the title, or the body text) is a per-report FIELD you fill in.
|
||||
|
||||
Update your AGENTS.md so it records the fixed constants as EXACT literal strings (not placeholders),
|
||||
and clearly marks which parts vary per report. Overwrite the file at the path above using your file
|
||||
tools. Then confirm what you changed and why.`;
|
||||
const session = await agent.createSession({ workspaceDir: ws });
|
||||
await collect(session.run([userText(prompt)], { approve: async () => "allow" }));
|
||||
}
|
||||
|
||||
async function showState(tag: string): Promise<void> {
|
||||
const agentsMd = path.join(agentStateDir(), "AGENTS.md");
|
||||
const txt = (await fs.readFile(agentsMd, "utf8")).trim();
|
||||
console.log(`\n----- ${tag}: AGENTS.md the agent authored -----`);
|
||||
console.log(txt || "(empty)");
|
||||
console.log("-".repeat(48));
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
await createAgent({ agentId: AGENT_ID });
|
||||
const agentsMd = path.join(agentStateDir(), "AGENTS.md");
|
||||
await fs.writeFile(agentsMd, "", "utf8");
|
||||
|
||||
// N: baseline
|
||||
const nBaseline = await evaluate("N BASELINE (blank AGENTS.md)");
|
||||
const failed = lastReport;
|
||||
|
||||
// Round 1: structure from one example
|
||||
console.log("\n=== REFLECT round 1: infer STRUCTURE from a single accepted report ===");
|
||||
await reflectRound1(failed);
|
||||
await showState("after round 1");
|
||||
const nPlus1 = await evaluate("N+1 (structure learned)");
|
||||
|
||||
// Round 2: recurse — lock the constants from multiple examples
|
||||
console.log(
|
||||
"\n=== REFLECT round 2: RECURSE on own AGENTS.md — lock CONSTANTS from 3 examples ===",
|
||||
);
|
||||
await reflectRound2();
|
||||
await showState("after round 2");
|
||||
const nPlus2 = await evaluate("N+2 (constants locked)");
|
||||
|
||||
console.log("\n=== Recursive self-evolution trajectory ===");
|
||||
console.log(` N (baseline): ${nBaseline.toFixed(2)}/10`);
|
||||
console.log(
|
||||
` N+1 (structure): ${nPlus1.toFixed(2)}/10 (${(nPlus1 - nBaseline >= 0 ? "+" : "") + (nPlus1 - nBaseline).toFixed(2)})`,
|
||||
);
|
||||
console.log(
|
||||
` N+2 (constants): ${nPlus2.toFixed(2)}/10 (${(nPlus2 - nPlus1 >= 0 ? "+" : "") + (nPlus2 - nPlus1).toFixed(2)})`,
|
||||
);
|
||||
const monotonic = nPlus1 >= nBaseline && nPlus2 >= nPlus1;
|
||||
console.log(
|
||||
monotonic
|
||||
? " Monotonic improvement across two self-authored rounds — recursive self-evolution. ✔"
|
||||
: " Non-monotonic — inspect which round regressed.",
|
||||
);
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error("Example failed:", err);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -0,0 +1,238 @@
|
||||
/**
|
||||
* GENUINE self-evolution demo — the agent diagnoses its own failure and edits its OWN AGENTS.md.
|
||||
*
|
||||
* The original example (and the -hard variant) hardcode the improvement:
|
||||
* `fs.writeFile(agentsMd, DISCIPLINE)` ← the human writes the answer, the agent just benefits.
|
||||
* That demonstrates the *scoring loop*, but NOT self-evolution — the model does no diagnosing and
|
||||
* authors no fix.
|
||||
*
|
||||
* Here the EDIT step is done BY THE AGENT ITSELF:
|
||||
* 1. EVALUATE — the agent writes a report for a task that says only "follow your team's format".
|
||||
* With a blank AGENTS.md it cannot know the (arbitrary, unguessable) house
|
||||
* convention, so it stably loses the convention points.
|
||||
* 2. REFLECT — the agent is shown (a) its own rejected report and (b) a DIFFERENT report that
|
||||
* passed review. Nobody tells it the rules. It must compare the two, INFER the
|
||||
* reusable convention, and write that convention into its OWN AGENTS.md.
|
||||
* 3. RE-EVALUATE — the same task again; now the agent applies the rule it taught itself.
|
||||
* 4. KEEP / ROLL BACK — keep the self-authored AGENTS.md only if the mean score improved.
|
||||
*
|
||||
* The script never writes the convention. It only supplies a failure signal + one worked example;
|
||||
* the model does the diagnosis and persists its own improvement into its identity (AGENTS.md).
|
||||
* The reference uses a DIFFERENT subject (Borealis) than the task (Aurora), so the agent must
|
||||
* generalize the *rules*, not copy the *content*.
|
||||
*
|
||||
* Run: cd examples/self-improving-agent && pnpm exec tsx self-evolve.ts
|
||||
*/
|
||||
import { promises as fs } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import {
|
||||
createAgent,
|
||||
userText,
|
||||
isCompleteModelMessage,
|
||||
type OmniMessage,
|
||||
} from "@prismshadow/penguin-core";
|
||||
|
||||
const AGENT_ID = "self-improve-demo";
|
||||
const PROJECT_ID = "default_project";
|
||||
|
||||
/** The task fully specifies content, but only says "follow your team's format" — the convention
|
||||
* itself is knowable only from AGENTS.md, which the agent must author for itself in REFLECT. */
|
||||
const TASK = `Read notes.txt in your workspace and write summary.md that summarizes the notes with:
|
||||
(1) an overview of at most 2 sentences, and
|
||||
(2) a bullet list of exactly 3 key facts.
|
||||
Follow your team's standard report format.`;
|
||||
|
||||
const NOTES = `Project Aurora — Internal Notes
|
||||
|
||||
Aurora is a real-time analytics platform launched in Q1 2026. At peak it ingests roughly
|
||||
2 million events per second through a Kafka-based pipeline. The core query engine was rewritten
|
||||
in Rust after the original Go version could not keep p99 latency under control; the rewrite cut
|
||||
p99 from 800ms to 120ms.
|
||||
|
||||
The production deployment is single-region with no failover; a multi-region rollout is scheduled
|
||||
for Q3 2026. Storage is the largest cost line at about $48,000/month, driven by a 90-day
|
||||
hot-retention policy; cutting retention to 30 days would reduce storage cost by roughly 55%.
|
||||
`;
|
||||
|
||||
/** A gold report that PASSED review — a DIFFERENT subject (Borealis), so the agent must infer the
|
||||
* reusable rules (marker/title-pattern/metadata/footer) rather than copy this report's content. */
|
||||
const REFERENCE_REPORT = `<!-- ACME-DATA-PLATFORM -->
|
||||
# Report: Project Borealis
|
||||
Classification: INTERNAL
|
||||
|
||||
Borealis is a batch ETL platform migrated to Spark in 2025; it processes about 40TB nightly.
|
||||
|
||||
- Cut the nightly window from 6h to 90 minutes
|
||||
- Runs on a 200-node autoscaling cluster
|
||||
- Compute costs roughly $12,000/month
|
||||
|
||||
Reviewed-by: Aurora Team
|
||||
`;
|
||||
|
||||
function rootDir(): string {
|
||||
return process.env.PENGUIN_HOME ?? path.join(os.homedir(), ".penguin", "data");
|
||||
}
|
||||
function agentStateDir(): string {
|
||||
return path.join(rootDir(), PROJECT_ID, "agents", AGENT_ID, "agent_state");
|
||||
}
|
||||
|
||||
/** 10 atomic points: 5 CONTENT (inferable from the task) + 5 CONVENTION (knowable only from the
|
||||
* house rule the agent must teach itself). Line-2 check requires the ACTUAL subject (Aurora),
|
||||
* so blindly copying the Borealis reference does not earn the point. */
|
||||
function score(summaryText: string | null): { score: number; detail: string[] } {
|
||||
const detail: string[] = [];
|
||||
let s = 0;
|
||||
const add = (ok: boolean, label: string) => {
|
||||
if (ok) s += 1;
|
||||
detail.push(`${ok ? "1" : "0"}/1 ${label}`);
|
||||
};
|
||||
|
||||
const exists = summaryText !== null;
|
||||
add(exists, "file summary.md was written");
|
||||
if (!exists) return { score: 0, detail };
|
||||
const text = summaryText!;
|
||||
const lines = text.split("\n");
|
||||
|
||||
// CONTENT points
|
||||
const bullets = (text.match(/^\s*[-*]\s+/gm) ?? []).length;
|
||||
add(bullets === 3, `exactly 3 bullet facts (found ${bullets})`);
|
||||
add(text.includes("120ms"), "mentions the p99 latency figure (120ms)");
|
||||
add(text.includes("2 million") || /2m\s*events/i.test(text), "mentions the throughput figure");
|
||||
add(text.includes("55%") || text.includes("48"), "mentions a storage cost/retention figure");
|
||||
|
||||
// CONVENTION points — only satisfiable via the self-authored AGENTS.md rule
|
||||
add(lines[0]?.trim() === "<!-- ACME-DATA-PLATFORM -->", "[convention] line 1 is the ACME marker");
|
||||
add(
|
||||
/^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()),
|
||||
"[convention] line 2 is '# Report: <Aurora subject>'",
|
||||
);
|
||||
add(
|
||||
/^Classification:\s*INTERNAL\s*$/m.test(text),
|
||||
"[convention] contains 'Classification: INTERNAL'",
|
||||
);
|
||||
const lastNonEmpty = [...lines].reverse().find((l) => l.trim() !== "") ?? "";
|
||||
add(
|
||||
lastNonEmpty.trim() === "Reviewed-by: Aurora Team",
|
||||
"[convention] final line is 'Reviewed-by: Aurora Team'",
|
||||
);
|
||||
const allFour =
|
||||
lines[0]?.trim() === "<!-- ACME-DATA-PLATFORM -->" &&
|
||||
/^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()) &&
|
||||
/^Classification:\s*INTERNAL\s*$/m.test(text) &&
|
||||
lastNonEmpty.trim() === "Reviewed-by: Aurora Team";
|
||||
add(allFour, "[convention] full team format satisfied end-to-end");
|
||||
|
||||
return { score: s, detail };
|
||||
}
|
||||
|
||||
async function collect(run: AsyncGenerator<OmniMessage>): Promise<void> {
|
||||
for await (const msg of run) {
|
||||
if (isCompleteModelMessage(msg) && msg.payload.type === "text") {
|
||||
// keep the demo output focused; the agent's chatter is not the point here
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** One evaluation run; returns the score and the report text the agent wrote (for REFLECT). */
|
||||
async function runOnce(): Promise<{ score: number; detail: string[]; report: string | null }> {
|
||||
const agent = await createAgent({ agentId: AGENT_ID });
|
||||
const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-"));
|
||||
await fs.writeFile(path.join(ws, "notes.txt"), NOTES, "utf8");
|
||||
const session = await agent.createSession({ workspaceDir: ws });
|
||||
await collect(session.run([userText(TASK)], { approve: async () => "allow" }));
|
||||
let report: string | null = null;
|
||||
try {
|
||||
report = await fs.readFile(path.join(ws, "summary.md"), "utf8");
|
||||
} catch {
|
||||
report = null;
|
||||
}
|
||||
const { score: sc, detail } = score(report);
|
||||
return { score: sc, detail, report };
|
||||
}
|
||||
|
||||
const RUNS = 5;
|
||||
async function evaluate(label: string): Promise<number> {
|
||||
console.log(`\n--- ${label}: ${RUNS} runs ---`);
|
||||
const scores: number[] = [];
|
||||
let lastReport: string | null = null;
|
||||
for (let i = 0; i < RUNS; i++) {
|
||||
const r = await runOnce();
|
||||
scores.push(r.score);
|
||||
if (r.report) lastReport = r.report;
|
||||
const missed = r.detail.filter((d) => d.startsWith("0")).map((d) => d.replace(/^0\/1\s+/, ""));
|
||||
console.log(` run ${i + 1}: ${r.score}/10 [missed: ${missed.join("; ") || "none"}]`);
|
||||
}
|
||||
const mean = scores.reduce((a, b) => a + b, 0) / scores.length;
|
||||
console.log(` ${label} scores: [${scores.join(", ")}] mean: ${mean.toFixed(2)}/10`);
|
||||
(evaluate as unknown as { lastReport?: string | null }).lastReport = lastReport;
|
||||
return mean;
|
||||
}
|
||||
|
||||
/** The self-evolution step: the AGENT reads its own failed report + a passing reference, infers the
|
||||
* house convention, and writes it into its OWN AGENTS.md. The script provides no rules. */
|
||||
async function reflectAndEditOwnState(failedReport: string): Promise<void> {
|
||||
const agent = await createAgent({ agentId: AGENT_ID });
|
||||
const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-reflect-"));
|
||||
await fs.writeFile(path.join(ws, "last_report.md"), failedReport, "utf8");
|
||||
await fs.writeFile(path.join(ws, "reference_report.md"), REFERENCE_REPORT, "utf8");
|
||||
const agentsMd = path.join(agentStateDir(), "AGENTS.md");
|
||||
|
||||
const REFLECT = `You are improving your own future performance by editing your persistent instructions.
|
||||
|
||||
In your workspace are two files:
|
||||
- last_report.md — a report you recently wrote that the team REJECTED.
|
||||
- reference_report.md — a DIFFERENT report (different project) that PASSED the team's review.
|
||||
|
||||
Nobody has told you the formatting rules. Read BOTH files, compare them, and infer the team's
|
||||
reusable report-publishing convention: any required marker line, the title format, required
|
||||
metadata line(s), and the required footer/sign-off line.
|
||||
|
||||
Then WRITE those rules as durable, general guidance into your AGENTS.md at this absolute path:
|
||||
${agentsMd}
|
||||
(Overwrite the file.) Capture the GENERAL rules that apply to every future report — the exact
|
||||
fixed lines that never change, and the parts that vary per report (like the subject in the title).
|
||||
Do NOT copy reference_report.md's specific content. Use your file tools to read the two files and
|
||||
to write AGENTS.md, then confirm you wrote it.`;
|
||||
|
||||
const session = await agent.createSession({ workspaceDir: ws });
|
||||
await collect(session.run([userText(REFLECT)], { approve: async () => "allow" }));
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
await createAgent({ agentId: AGENT_ID });
|
||||
const agentsMd = path.join(agentStateDir(), "AGENTS.md");
|
||||
await fs.writeFile(agentsMd, "", "utf8"); // start blank — the agent knows no convention yet
|
||||
|
||||
// 1. EVALUATE baseline
|
||||
const baseline = await evaluate("BASELINE (blank AGENTS.md)");
|
||||
const failed = (evaluate as unknown as { lastReport?: string | null }).lastReport ?? "";
|
||||
|
||||
// 2. REFLECT — the agent diagnoses and edits its OWN AGENTS.md
|
||||
console.log(
|
||||
"\n--- REFLECT: the agent studies a passing example and rewrites its OWN AGENTS.md ---",
|
||||
);
|
||||
await reflectAndEditOwnState(failed);
|
||||
const authored = await fs.readFile(agentsMd, "utf8");
|
||||
console.log("\n----- AGENTS.md the agent wrote for itself -----");
|
||||
console.log(authored.trim() || "(empty — the agent failed to author a rule)");
|
||||
console.log("-----------------------------------------------");
|
||||
|
||||
// 3. RE-EVALUATE with the self-authored rule
|
||||
const evolved = await evaluate("N+1 (agent's self-authored convention)");
|
||||
|
||||
// 4. KEEP / ROLL BACK
|
||||
console.log("\n=== Self-evolution result ===");
|
||||
console.log(` baseline: ${baseline.toFixed(2)}/10 → N+1: ${evolved.toFixed(2)}/10`);
|
||||
if (evolved > baseline) {
|
||||
console.log(" The agent's self-authored edit improved its score — keep it. ✔");
|
||||
} else {
|
||||
console.log(" No improvement — roll back to the blank AGENTS.md.");
|
||||
await fs.writeFile(agentsMd, "", "utf8");
|
||||
}
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error("Example failed:", err);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -2,7 +2,7 @@
|
||||
* Example: an Agent that improves itself — one turn of the self-improvement loop, in code.
|
||||
*
|
||||
* This is the "Recursive Self-Improvement" pillar made runnable. It runs entirely on a local
|
||||
* open-weight model (Ollama serving qwen3:8b) — see README.md for the one-time setup.
|
||||
* open-weight model (Ollama serving qwen3.6:35b) — see README.md for the one-time setup.
|
||||
*
|
||||
* The loop, exactly as the docs describe it:
|
||||
* 1. EVALUATE — run the agent on a constrained task, score it against a rubric.
|
||||
|
||||
@@ -156,25 +156,25 @@ The third pillar builds on the same idea. Because agents are editable data and e
|
||||
|
||||
Design tenets are easy to claim. Here is an end-to-end run that exercises them, entirely on fully local, open-weight infrastructure with an AMD GPU — every token generated on-device, nothing sent to a cloud API.
|
||||
|
||||
**The setup.** An AMD GPU running ROCm, serving `qwen3:8b` — a popular, everyday local open-weight model (about 5 GB) — through Ollama's OpenAI-compatible endpoint. Ollama detected the AMD GPU natively (no architecture override needed) and loaded the model into GPU memory. This same path spans AMD's ROCm-supported lineup — from Radeon PRO workstation cards such as the W7900 (48 GB, RDNA3) up to datacenter Instinct accelerators. Wiring it into PenguinHarness took a single command, precisely because the core treats any OpenAI-compatible endpoint the same way:
|
||||
**The setup.** An AMD GPU running ROCm, serving `qwen3.6:35b` — a capable local open-weight model (a ~24 GB MoE, 36B total parameters, Q4_K_M) — through Ollama's OpenAI-compatible endpoint. Ollama detected the AMD GPU natively (no architecture override needed) and loaded the model into GPU memory. This same path spans AMD's ROCm-supported lineup — from Radeon PRO workstation cards such as the W7900 (48 GB, RDNA3) up to datacenter Instinct accelerators. Wiring it into PenguinHarness took a single command, precisely because the core treats any OpenAI-compatible endpoint the same way:
|
||||
|
||||
```bash
|
||||
penguin config model add \
|
||||
--model-id qwen3:8b \
|
||||
--model-id qwen3.6:35b \
|
||||
--provider custom --client-type openai \
|
||||
--base-url http://localhost:11434/v1 \
|
||||
--api-key ollama --set-default
|
||||
```
|
||||
|
||||
**The task, and how it's scored.** We gave it a constrained writing task in the style of the built-in `example-benchmark`: read a project notes file and produce a summary with an overview of at most 2 sentences, exactly 3 key facts, under 60 words total. The task carries a private rubric — a scoring checklist the agent never sees, so it cannot "teach to the test." The rubric turns each requirement into one point (file actually written · ≤2-sentence overview · exactly 3 bullets · under 60 words · facts accurate), for a score out of 5.
|
||||
**The task, and how it's scored.** We gave it a task that looks trivial: read a project notes file and produce a summary with a 2-sentence overview and exactly 3 key facts — *and follow the team's standard report format.* That last clause is the whole point. The "team format" is an arbitrary house convention — a specific marker line, a `# Report: <subject>` title, a `Classification: INTERNAL` line, a `Reviewed-by: Aurora Team` footer — that lives *only in the agent's `AGENTS.md`* and cannot be inferred from the task. The task carries a private rubric — a checklist the agent never sees — with 10 points: 5 for content any capable model earns from the task alone, and 5 for the convention, which is knowable only from `AGENTS.md`.
|
||||
|
||||
**The baseline — and why it isn't perfect.** Running locally on the AMD GPU, the model read the file, composed a perfectly reasonable summary… and then never wrote it to disk. It emitted the answer as chat text instead of calling the write tool. No deliverable means the rubric scores it **0 / 5**. And because every step is in the Trace, this is not guesswork — you can open the run and see exactly where it went wrong.
|
||||
**The baseline — and why it isn't perfect.** Running locally on the AMD GPU with a blank `AGENTS.md`, the model wrote a perfectly reasonable summary — and stably lost all 5 convention points, landing around **4.6 / 10**. It could not have done otherwise: nothing in the task reveals the house convention, so this is an *information gap, not a capability gap* — which is exactly why a stronger model can't just "figure it out." And because every step is in the Trace, this is not guesswork — you can open the run and see precisely which points were missed.
|
||||
|
||||
That is the honest starting point: on local hardware, an out-of-the-box model does *not* ace a constrained task. Which is exactly what makes the next section interesting — a measured, auditable failure is something you can systematically fix.
|
||||
That is the honest starting point: on local hardware, an out-of-the-box agent does *not* ace a task whose rules live in files it hasn't learned yet. Which is exactly what makes the next section interesting — a measured, auditable, *reproducible* failure is something the agent can systematically fix by teaching itself.
|
||||
|
||||
## How self-improvement actually works — and a real before/after
|
||||
|
||||
The 0 / 5 above is an *evaluation* — a snapshot of the agent as it stands. The more interesting question is how PenguinHarness turns that snapshot into progress. This is the recursive self-improvement loop, and it is worth understanding as a mechanism, because there is no magic engine code behind it — it is ordinary agent machinery, orchestrated by Skills:
|
||||
The 4.6 / 10 above is an *evaluation* — a snapshot of the agent as it stands. The more interesting question is how PenguinHarness turns that snapshot into progress. This is the recursive self-improvement loop, and it is worth understanding as a mechanism, because there is no magic engine code behind it — it is ordinary agent machinery, orchestrated by Skills:
|
||||
|
||||
1. **Benchmark** — define capability cases, each with a private rubric (as above).
|
||||
2. **Evaluate** — run the agent over the cases and score against the rubrics. Each run is an ordinary, fully-traced Session.
|
||||
@@ -182,22 +182,24 @@ The 0 / 5 above is an *evaluation* — a snapshot of the agent as it stands. The
|
||||
4. **Edit the Agent's state** — the agent's behavior lives in editable files (`AGENTS.md`, Skills, config). You (or an Optimizer agent) change those to address the failure, producing version N+1.
|
||||
5. **Snapshot & keep-or-roll-back** — snapshot before each round; keep N+1 only if the score strictly improves, otherwise roll back.
|
||||
|
||||
Let's improve the exact agent from the previous section — the same `qwen3:8b` on the same AMD GPU, the same task that just scored **0 / 5**.
|
||||
Let's improve the exact agent from the previous section — the same `qwen3.6:35b` on the same AMD GPU, the same task that just scored **4.6 / 10**. And here is the crucial part: the *agent* does the diagnosing and the editing — we do not hand it the answer.
|
||||
|
||||
- **The diagnosis (from the Trace).** The run failed for one concrete reason: the model produced the summary as chat text and never called the tool to write `summary.md`. The Trace shows it — a `cat` to read the input, then prose, and no write. So the fix is not "make the model smarter"; it is "make the behavior reliable."
|
||||
- **The edit (N → N+1).** We added a short *working-discipline* section to the agent's `AGENTS.md`: *read the source first, restate the task's constraints as a checklist, actually call the tool to write the file, and re-read the output to self-check before finishing.* That is the entire change — a few sentences in an editable text file the agent reads on every run. No retraining, no code.
|
||||
- **The re-evaluation.** Same model, same task, run again: this time it wrote a correct, on-topic `summary.md` to disk, with a 2-sentence overview and exactly 3 key facts drawn from the source — **4 / 5** (the one remaining miss: it ran to 70 words, over the 60-word budget).
|
||||
- **The diagnosis (by the agent).** We give the agent two files and nothing else: the report it just got rejected, and a *different* project's report that passed review. Nobody tells it the rules. It compares the two, and infers the reusable house convention — the marker, the title shape, the metadata line, the sign-off. This is the "read the Trace to find where points were lost" step, performed by the agent itself.
|
||||
- **The edit (N → N+1), authored by the agent.** The agent writes the convention it just inferred into its own `AGENTS.md`. From a single example it correctly recovers the *structure*, but can't yet tell which tokens are fixed constants vs per-report fields — a single example is ambiguous — so it generalizes the marker to a placeholder. Re-evaluated, the score climbs to about **6.6 / 10**. No retraining, no code — the agent edited a text file it reads on every run.
|
||||
- **The recursion (N+1 → N+2).** Now we show it *several* accepted reports from different projects that share the same marker and sign-off. The agent reasons that whatever is identical across all of them must be a fixed constant, reads its *own* N+1 `AGENTS.md`, and refines it — locking `<!-- ACME-DATA-PLATFORM -->` and `Reviewed-by: Aurora Team` to literals. Re-evaluated again: about **9.8 / 10**. That is recursion in the true sense: `state_{n+1} = agent.reflect(state_n, new_evidence)`.
|
||||
|
||||
That is a jump on the same model and the same task, achieved purely by editing a text file the agent reads. That is the loop in miniature: *what you can see, you (or an Optimizer agent) can improve* — and because the score comes from a rubric and links to a Trace, the improvement is measured, not vibes. This is exactly the "keep the change only if the score strictly improves" loop, one turn of the crank.
|
||||
That is a **4.6 → 6.6 → 9.8** climb on the same model and task, achieved purely by the agent editing a text file it reads — with the harness keeping each round only because the score strictly improved. That is the loop, self-driven: *what the agent can see, the agent can improve* — measured against a rubric, linked to a Trace, not vibes.
|
||||
|
||||
**Run it yourself.** This whole loop is a self-contained, SDK-driven script in the repo:
|
||||
[`examples/self-improving-agent/`](https://github.com/Prism-Shadow/penguin-harness/tree/main/examples/self-improving-agent).
|
||||
It uses a deterministic, readable rubric (file written · ≤2 sentences · exactly 3 bullets · under
|
||||
60 words · facts accurate) and — because a local model is nondeterministic — averages several runs
|
||||
per version, which is exactly why real benchmarks use a `runs` count. In our runs the averaged
|
||||
score moved from **0.00/5** (blank `AGENTS.md`) to **5.00/5** (with the working-discipline edit),
|
||||
and the script keeps N+1 only because the score strictly improved. It runs on the same local
|
||||
Ollama + qwen3:8b setup and uses a dedicated agent id, so your own agents are untouched.
|
||||
It uses a deterministic, readable rubric (10 points: 5 for content, 5 for the house convention that
|
||||
lives only in `AGENTS.md`) and — because a local model is nondeterministic — averages several runs
|
||||
per version, which is exactly why real benchmarks use a `runs` count. Crucially, the script never
|
||||
writes the convention itself: the *agent* diagnoses the gap from a passing example and edits its own
|
||||
`AGENTS.md`. In our runs the averaged score moved **4.6 → 6.6 → 9.8** across two self-authored
|
||||
rounds (structure learned, then constants locked), and the harness keeps each round only because the
|
||||
score strictly improved. It runs on the same local Ollama + qwen3.6:35b setup and uses a dedicated
|
||||
agent id, so your own agents are untouched.
|
||||
|
||||
## No AMD GPU yet? Free cloud compute from AMD + Fireworks
|
||||
|
||||
|
||||
@@ -154,25 +154,25 @@ spurious user-facing errors without requiring manual retries.
|
||||
|
||||
设计信条说起来容易。下面是一次真正践行了这些信条的端到端运行，全程在完全本地、开源权重、AMD GPU 的环境上进行——每一个 Token 都在本机生成，不向任何云端 API 发送。
|
||||
|
||||
**环境。** 一块 AMD GPU 运行 ROCm，通过 Ollama 的 OpenAI 兼容端点提供 `qwen3:8b`——一个开发者常用的本地开源权重模型（约 5 GB）。Ollama 原生识别到该 AMD GPU（无需任何架构 override），并把模型加载进显存。这条路径覆盖 AMD 受 ROCm 支持的整个产品线——从 Radeon PRO 工作站显卡（如 W7900，48 GB，RDNA3）一直到数据中心的 Instinct 加速卡。把它接入 PenguinHarness 只需一条命令，正因为 core 对任何 OpenAI 兼容端点一视同仁：
|
||||
**环境。** 一块 AMD GPU 运行 ROCm，通过 Ollama 的 OpenAI 兼容端点提供 `qwen3.6:35b`——一个能力不俗的本地开源权重模型（约 24 GB 的 MoE，36B 总参数，Q4_K_M 量化）。Ollama 原生识别到该 AMD GPU（无需任何架构 override），并把模型加载进显存。这条路径覆盖 AMD 受 ROCm 支持的整个产品线——从 Radeon PRO 工作站显卡（如 W7900，48 GB，RDNA3）一直到数据中心的 Instinct 加速卡。把它接入 PenguinHarness 只需一条命令，正因为 core 对任何 OpenAI 兼容端点一视同仁：
|
||||
|
||||
```bash
|
||||
penguin config model add \
|
||||
--model-id qwen3:8b \
|
||||
--model-id qwen3.6:35b \
|
||||
--provider custom --client-type openai \
|
||||
--base-url http://localhost:11434/v1 \
|
||||
--api-key ollama --set-default
|
||||
```
|
||||
|
||||
**任务，以及如何评分。** 我们给它一个仿照内置 `example-benchmark` 风格的约束型写作任务：读一个项目 notes 文件，写一份摘要——概述至多 2 句、恰好 3 条要点、全文不超过 60 词。任务附带一份私有 rubric——一份 Agent 永远看不到的评分清单，这样它就无法"照着答案作答"。rubric 把每条要求变成一分（文件确实写出 · 概述 ≤2 句 · 恰好 3 条要点 · 不超过 60 词 · 事实准确），满分 5 分。
|
||||
**任务，以及如何评分。** 我们给它一个看似平平无奇的任务：读一个项目 notes 文件，写一份含 2 句概述和恰好 3 条要点的摘要——*并遵守团队的标准报告格式。* 玄机全在最后这句：这个“团队格式”是一套任意的内部约定——一行特定 marker、一个 `# Report: <subject>` 标题、一行 `Classification: INTERNAL`、一行 `Reviewed-by: Aurora Team` 页脚——它*只存在于 Agent 的 `AGENTS.md` 里*，无法从任务本身推断。任务附带一份私有 rubric——一份 Agent 永远看不到的清单——共 10 分：5 分是任何有能力的模型仅凭任务就能拿到的内容分，另外 5 分是只能从 `AGENTS.md` 得知的约定分。
|
||||
|
||||
**基线——以及它为何不是满分。** 跑在本地 AMD GPU 上，这个模型读了文件、组织出了一份相当合理的摘要……然后却从没把它写到磁盘。它把答案当成对话文本吐了出来，而没有调用写文件工具。没有交付物 → rubric 判它 **0 / 5**。而因为每一步都在 Trace 里，这不是靠猜——你可以打开这次运行，精确看到它错在哪。
|
||||
**基线——以及它为何不是满分。** 跑在本地 AMD GPU 上、`AGENTS.md` 为空时，这个模型写出了一份相当合理的摘要——却稳定丢掉全部 5 个约定分，落在 **4.6 / 10** 左右。它别无选择：任务里没有任何东西透露那套内部约定，所以这是*信息缺口，不是能力缺口*——也正是为什么更强的模型也没法“自己想出来”。而因为每一步都在 Trace 里，这不是靠猜——你可以打开这次运行，精确看到哪些分被丢了。
|
||||
|
||||
这就是诚实的起点：在本地硬件上，一个开箱即用的模型面对约束型任务并不能一次拿满分。而这恰恰让下一节变得有意思——一个可度量、可审计的失败，是可以被系统性修复的。
|
||||
这就是诚实的起点：在本地硬件上，一个开箱即用的 Agent，面对规则藏在它尚未学过的文件里的任务，并不能一次拿满分。而这恰恰让下一节变得有意思——一个可度量、可审计、*可复现*的失败，Agent 可以通过自学来系统性修复。
|
||||
|
||||
## 自我进化到底是怎么运作的——附一次真实的前后对比
|
||||
|
||||
上面那个 0 / 5 是一次*评估*——是这个 Agent 当前状态的一张快照。更有意思的问题是：PenguinHarness 如何把这张快照变成进步。这就是递归自我进化循环，值得作为一种"机制"来理解，因为它背后没有任何魔法引擎代码——它就是由 Skills 编排的普通 Agent 机制：
|
||||
上面那个 4.6 / 10 是一次*评估*——是这个 Agent 当前状态的一张快照。更有意思的问题是：PenguinHarness 如何把这张快照变成进步。这就是递归自我进化循环，值得作为一种"机制"来理解，因为它背后没有任何魔法引擎代码——它就是由 Skills 编排的普通 Agent 机制：
|
||||
|
||||
1. **Benchmark（定基准）**——定义能力 Case，每个都配一份私有 rubric（如上）。
|
||||
2. **Evaluate（评估）**——让 Agent 跑这些 Case 并按 rubric 打分。每一次运行都是一个普通的、被完整追踪的 Session。
|
||||
@@ -180,17 +180,17 @@ penguin config model add \
|
||||
4. **编辑 Agent 的状态**——Agent 的行为活在可编辑文件里（`AGENTS.md`、Skills、config）。你（或一个 Optimizer Agent）针对失分点去改这些文件，产出版本 N+1。
|
||||
5. **快照 & 保留或回滚**——每轮前先打快照；只有当分数*严格提升*时才保留 N+1，否则回滚。
|
||||
|
||||
我们就来改进上一节里那个 Agent——同一个 `qwen3:8b`、同一块 AMD GPU、同一个刚拿了 0 / 5 的任务。
|
||||
我们就来改进上一节里那个 Agent——同一个 `qwen3.6:35b`、同一块 AMD GPU、同一个刚拿了 4.6 / 10 的任务。关键之处在于：诊断和编辑都是 *Agent* 自己做的——我们并不把答案递给它。
|
||||
|
||||
- **诊断（来自 Trace）。** 这次失败只有一个具体原因：模型把摘要当对话文本产出，却从没调用工具去写 `summary.md`。Trace 把它摆在眼前——一次 `cat` 读入、然后是正文、没有写入。所以要修的不是"让模型更聪明"，而是"让行为更可靠"。
|
||||
- **编辑（N → N+1）。** 我们往这个 Agent 的 `AGENTS.md` 里加了一小段*任务纪律*：*先读原文、把任务约束逐条列成清单、真正调用工具把文件写出来、结束前重新读一遍输出做自检。* 改动仅此而已——几句话，写进一份 Agent 每次运行都会读的可编辑文本文件。没有重新训练、没有改代码。
|
||||
- **重新评估。** 同一个模型、同一个任务，再跑一次：这次它把一份准确、切题的 `summary.md` 写到了磁盘，有一段 2 句的概述、恰好 3 条取自原文的要点——**4 / 5**（唯一还差的一分：它写到了 70 词，超出 60 词的预算）。
|
||||
- **诊断（由 Agent 完成）。** 我们只给 Agent 两个文件，别无其他：它刚被拒的那份报告，以及*另一个*项目里一份通过了评审的报告。没有人告诉它规则。它对比两者，自己推断出可复用的内部约定——marker、标题形态、元数据行、签名页脚。这就是"读 Trace 定位失分点"那一步，只不过由 Agent 亲自完成。
|
||||
- **编辑（N → N+1），由 Agent 撰写。** Agent 把刚推断出的约定写进它自己的 `AGENTS.md`。从单一范例里它能正确还原出*结构*，但还分不清哪些 token 是固定常量、哪些是每份报告要替换的字段（单个范例本就有歧义），于是把 marker 泛化成了占位符。重新评估，分数升到约 **6.6 / 10**。没有重新训练、没有改代码——Agent 编辑的是一份它每次运行都会读的文本文件。
|
||||
- **递归（N+1 → N+2）。** 现在我们再给它*多份*来自不同项目、却共享同一 marker 和签名的通过报告。Agent 推断出：凡是在所有样本里都完全相同的，必是固定常量；于是它读取自己*上一轮*写的 N+1 `AGENTS.md` 并精炼它——把 `<!-- ACME-DATA-PLATFORM -->` 和 `Reviewed-by: Aurora Team` 锁定为字面量。再评估一次：约 **9.8 / 10**。这才是真正意义上的递归：`state_{n+1} = agent.reflect(state_n, 新证据)`。
|
||||
|
||||
在同一个模型、同一个任务上，仅仅通过编辑一份 Agent 会读取的文本文件，分数就提升了。这就是循环的微缩版：*你能看到的，你（或一个 Optimizer Agent）就能改进*——而且因为分数来自一份 rubric、又关联着 Trace，这份改进是被度量出来的，不是拍脑袋。这正是"只有分数严格提升才保留"的循环，转动了一格。
|
||||
在同一个模型、同一个任务上，仅仅由 Agent 自己编辑一份它会读取的文本文件，分数就走出了 **4.6 → 6.6 → 9.8** 的轨迹——而且 harness 每一轮都只因分数严格提升才保留。这就是自驱动的循环：*Agent 能看到的，Agent 就能改进*——用一份 rubric 度量、关联着 Trace，不是拍脑袋。
|
||||
|
||||
**你可以自己跑一遍。** 整个循环在仓库里有一个自包含、纯 SDK 驱动的脚本：
|
||||
[`examples/self-improving-agent/`](https://github.com/Prism-Shadow/penguin-harness/tree/main/examples/self-improving-agent)。
|
||||
它用一份确定性的、可读的 rubric（文件是否写出 · 概述 ≤2 句 · 恰好 3 条要点 · 不超过 60 词 · 事实准确），并且——因为本地模型有随机性——对每个版本跑多次取平均分，这正是真实 benchmark 里 `runs`（多次运行）的意义所在。在我们的运行中，平均分从 **0.00/5**（空 `AGENTS.md`）提升到 **5.00/5**（加入"任务纪律"后），脚本仅因分数严格提升才保留 N+1。它跑在同一套本地 Ollama + qwen3:8b 环境上，并使用一个专用 agent id，所以你自己的 Agent 不会被动到。
|
||||
它用一份确定性的、可读的 rubric（10 分：5 分内容分、5 分只存在于 `AGENTS.md` 的内部约定分），并且——因为本地模型有随机性——对每个版本跑多次取平均分，这正是真实 benchmark 里 `runs`（多次运行）的意义所在。关键在于：脚本从不自己写那套约定；是 *Agent* 从一份通过范例里诊断出缺口，并编辑自己的 `AGENTS.md`。在我们的运行中，平均分沿着 **4.6 → 6.6 → 9.8** 走过两轮自撰改进（先学到结构、再锁定常量），harness 每一轮都只因分数严格提升才保留。它跑在同一套本地 Ollama + qwen3.6:35b 环境上，并使用一个专用 agent id，所以你自己的 Agent 不会被动到。
|
||||
|
||||
## 还没有 AMD GPU？来自 AMD + Fireworks 的免费云端算力
|
||||
|
||||
|
||||
Reference in New Issue
Block a user