From 81d8c0045c065cb94821179a27340caeac3896fb Mon Sep 17 00:00:00 2001 From: Zhang Jason Date: Wed, 22 Jul 2026 22:04:52 +0800 Subject: [PATCH] docs(examples): make self-improvement demo genuinely self-evolving (#29) --- examples/self-improving-agent/README.md | 127 +++++--- examples/self-improving-agent/README.zh.md | 114 ++++--- examples/self-improving-agent/package.json | 2 +- .../self-evolve-recursive.ts | 287 ++++++++++++++++++ examples/self-improving-agent/self-evolve.ts | 238 +++++++++++++++ examples/self-improving-agent/self-improve.ts | 2 +- .../blog/local-agents-on-amd-gpus.en.md | 36 +-- .../blog/local-agents-on-amd-gpus.zh.md | 24 +- 8 files changed, 728 insertions(+), 102 deletions(-) create mode 100644 examples/self-improving-agent/self-evolve-recursive.ts create mode 100644 examples/self-improving-agent/self-evolve.ts diff --git a/examples/self-improving-agent/README.md b/examples/self-improving-agent/README.md index 26a39e2..04d427a 100644 --- a/examples/self-improving-agent/README.md +++ b/examples/self-improving-agent/README.md @@ -3,36 +3,60 @@ # Example: an Agent that improves itself (local, on an AMD GPU via Ollama) This example is the **"Recursive Self-Improvement"** pillar in runnable code. Using only the -PenguinHarness SDK, it runs one turn of the self-improvement loop: +PenguinHarness SDK, the agent runs the self-improvement loop on **itself** — and the key point is +that the improvement is authored by the *agent*, not hardcoded by the script: -1. **Evaluate** a constrained writing task and score it against a rubric. -2. **Diagnose** which rubric points were lost (from the run itself). -3. **Edit** the agent's own `AGENTS.md` to address the failure (version N+1). -4. **Re-evaluate** the same task and keep the change only if the score improved. +1. **Evaluate** — the agent attempts a constrained task and is scored against a rubric (plain code). +2. **Diagnose** — *the agent itself* reads a failing result next to a passing example. +3. **Edit** — *the agent itself* infers the missing rule and writes it into its own `AGENTS.md`. +4. **Re-evaluate** — the same task again; the change is kept only if the mean score improved. -Everything runs on a **local open-weight model** — `qwen3:8b` served by Ollama — so no cloud API +Everything runs on a **local open-weight model** — `qwen3.6:35b` served by Ollama — so no cloud API and no data leaving the machine. Ollama's ROCm backend runs this natively on AMD GPUs. -## Why a deterministic rubric, and why averaging +## The three scripts (from mechanism to real self-evolution) -- The **rubric is plain code you can read** (`score()` in `self-improve.ts`): file actually - written · overview ≤ 2 sentences · exactly 3 bullets · under 60 words · key facts present. No - hidden judge — the before/after numbers are objective and reproducible. In the full product the - Evaluator is driven by the `agent-evaluation` skill against a *private* rubric; this example - distills that idea to its runnable core. -- A local model is **nondeterministic**, so the example runs each version several times and - averages — which is exactly why real benchmarks use a `runs` count per case. A single run can - swing; the mean is what tells you whether the edit actually helped. +| Script | What it demonstrates | `AGENTS.md` edited by | +| --- | --- | --- | +| `self-improve.ts` | The **scoring loop** in miniature (simplest) | the **script** (hardcoded) | +| `self-evolve.ts` | **Genuine** single-round self-evolution | the **agent** itself | +| `self-evolve-recursive.ts` | **Multi-round recursion** — the main line (`pnpm start`) | the **agent**, over two rounds | + +`self-improve.ts` is the honest baseline: it shows the evaluate → edit → re-evaluate machinery, but +the edit is a human-written `DISCIPLINE` string the script writes to disk. That demonstrates the +*loop*, not self-evolution. The two `self-evolve*` scripts move the diagnosis **and** the edit into +the agent — the script only supplies a failure signal and one worked example; the model does the +learning and persists it into its own identity file. + +## The task, and why the baseline stably fails + +The task looks trivial: *summarize `notes.txt` into `summary.md` with a 2-sentence overview and +exactly 3 key facts — and follow your team's standard report format.* The catch is the last clause: +the "team format" is an **arbitrary house convention** (a marker line, a `# Report: ` +title, a `Classification: INTERNAL` line, a `Reviewed-by: Aurora Team` footer) that appears **only +in `AGENTS.md`** and **cannot be inferred from the task**. + +This is the design trick that makes the baseline *stably* bad — for any model, strong or weak. The +rubric (`score()`, plain code) has 10 atomic points: **5 content points** any capable model earns +from the task alone, and **5 convention points** that are knowable only from `AGENTS.md`. With a +blank `AGENTS.md` the agent cannot guess the convention, so it stably loses those 5 points. This is +an **information gap, not a capability gap** — the reason a stronger model can't just "figure it +out". It also mirrors the full product, where the Evaluator is driven by the `agent-evaluation` +skill against a *private* rubric. + +The agent closes the gap by **learning the convention from a passing example** and writing it into +its own `AGENTS.md` — and in the recursive script, by locking down the fixed constants once it has +seen several examples. ## 1–2. Serve the model and point PenguinHarness at it ```bash export HIP_VISIBLE_DEVICES=0 # optional: pin a specific AMD GPU ollama serve & -ollama pull qwen3:8b +ollama pull qwen3.6:35b penguin config model add \ - --model-id qwen3:8b \ + --model-id qwen3.6:35b \ --provider custom --client-type openai \ --base-url http://localhost:11434/v1 \ --api-key ollama --set-default @@ -43,35 +67,64 @@ penguin config model add \ ```bash pnpm install pnpm build -pnpm --dir examples/self-improving-agent start -# or directly: npx tsx examples/self-improving-agent/self-improve.ts +pnpm --dir examples/self-improving-agent start # the recursive main line +# or a single round: npx tsx examples/self-improving-agent/self-evolve.ts +# or the scoring-loop baseline: npx tsx examples/self-improving-agent/self-improve.ts ``` ## What you should see ```text -BASELINE (blank AGENTS.md): 3 runs - run 1: 0/5 - run 2: 0/5 - run 3: 0/5 - BASELINE mean: 0.00/5 -N+1 (with working discipline): 3 runs - run 1: 5/5 - run 2: 5/5 - run 3: 5/5 - N+1 mean: 5.00/5 -=== Self-improvement result === - baseline: 0.00/5 → N+1: 5.00/5 - Mean score improved — keep version N+1. ✔ +N BASELINE (blank AGENTS.md): 5 runs + ... scores: [5, 4, 5, 5, 5] mean: 4.80/10 # loses every convention point + +=== REFLECT round 1: infer STRUCTURE from a single accepted report === +----- after round 1: AGENTS.md the agent authored ----- + ## Report-Publishing Convention ... + # structure learned, constants still guessed + Reviewed-by: + +N+1 (structure learned): 5 runs + ... scores: [6, 7, 6, 6, 8] mean: 6.60/10 + +=== REFLECT round 2: RECURSE on own AGENTS.md — lock CONSTANTS from 3 examples === +----- after round 2: AGENTS.md the agent authored ----- + ### Fixed Constants (verbatim) + | 1 | | # constants now locked to literals + | Last line | Reviewed-by: Aurora Team | + +N+2 (constants locked): 5 runs + ... scores: [10, 10, 9, 10, 10] mean: 9.80/10 + +=== Recursive self-evolution trajectory === + N (baseline): 4.80/10 + N+1 (structure): 6.60/10 (+1.80) + N+2 (constants): 9.80/10 (+3.20) + Monotonic improvement across two self-authored rounds — recursive self-evolution. ✔ ``` -With a blank `AGENTS.md`, `qwen3:8b` tends to *narrate* the summary in chat and never call the -tool to write the file — so the rubric scores it 0. Adding a short "working discipline" section -(read first, restate the constraints, actually write the file, self-check) flips that. Exact -numbers vary run to run; the averaged direction is the point. +The trajectory is the point: from one passing example the agent infers the **structure** but can't +tell which tokens are fixed constants vs per-report fields (a single example is ambiguous), so it +stalls mid-way. Given **several** accepted reports that share the same marker and sign-off, it +infers "whatever is identical across all of them is a fixed constant", reads its **own** round-1 +`AGENTS.md`, and refines it — recursion in the true sense: `state_{n+1} = agent.reflect(state_n, +new_evidence)`. Exact numbers vary run to run; the monotonic direction is the point. + +## Honest boundaries + +- **Self-evolution needs a capable model.** Authoring a correct rule and then *following it* is + harder than benefiting from a hardcoded one. A weaker model (e.g. `qwen3:8b`) often infers the + rule but executes it unreliably (placeholders left unfilled, or narrating instead of writing the + file), so its N+1 can *regress* and the script rolls back. That rollback is the loop being honest, + not a bug. +- **Evidence sufficiency is real.** N+1 stalls at a middling score because one example genuinely + under-determines the fixed constants; N+2 only climbs because more examples make them inferable. + The gain is information-driven learning, not noise. ## Notes - Uses a dedicated agent id (`self-improve-demo`), created on the fly — your own agents are untouched. -- Re-running updates that demo agent in place. +- The script never writes the convention itself; it only supplies the failing report, the accepted + example(s), and the keep/roll-back signal. +- Re-running resets the demo agent's `AGENTS.md` to blank and starts the loop over. diff --git a/examples/self-improving-agent/README.zh.md b/examples/self-improving-agent/README.zh.md index 1913d1a..2e68f68 100644 --- a/examples/self-improving-agent/README.zh.md +++ b/examples/self-improving-agent/README.zh.md @@ -2,34 +2,55 @@ # 示例：一个自我改进的 Agent（本地，经 Ollama 跑在 AMD GPU 上） -这个示例是 **“递归自我进化”** 支柱的可运行版本。仅用 PenguinHarness SDK，它会跑完自我进化循环的一轮： +这个示例是 **“递归自我进化”** 支柱的可运行版本。仅用 PenguinHarness SDK，Agent 就在**自己身上**跑完 +自我进化循环——而关键在于：这次改进是由 *Agent 自己*产出的，不是脚本硬编码写死的： -1. **评估（Evaluate）** —— 让 Agent 完成一个约束型写作任务，并按 rubric 打分。 -2. **诊断（Diagnose）** —— 从运行结果中看出哪些 rubric 项失了分。 -3. **编辑（Edit）** —— 改写 Agent 自己的 `AGENTS.md` 来修复失分点（版本 N+1）。 -4. **重新评估（Re-evaluate）** —— 重跑同一任务，只有分数提升时才保留这次改动。 +1. **评估（Evaluate）** —— Agent 尝试一个约束型任务，并按 rubric（普通代码）打分。 +2. **诊断（Diagnose）** —— *Agent 自己*把一次失败结果与一份通过范例并排对比。 +3. **编辑（Edit）** —— *Agent 自己*归纳出缺失的规则，并写进它自己的 `AGENTS.md`。 +4. **重新评估（Re-evaluate）** —— 重跑同一任务；只有平均分提升时才保留这次改动。 -全程跑在一个**本地开源权重模型**上——由 Ollama 提供的 `qwen3:8b`——所以不用任何云端 API、数据也不 +全程跑在一个**本地开源权重模型**上——由 Ollama 提供的 `qwen3.6:35b`——所以不用任何云端 API、数据也不 离开本机。Ollama 的 ROCm 后端能原生在 AMD GPU 上运行它。 -## 为什么用确定性 rubric，为什么要取平均 +## 三个脚本（从机制到真正的自我进化） -- **rubric 就是你能读懂的普通代码**（`self-improve.ts` 里的 `score()`）：文件是否确实写出 · 概述 - ≤ 2 句 · 恰好 3 条要点 · 不超过 60 词 · 关键事实是否出现。没有隐藏的裁判——前后分数是客观、可复现的。 - 在完整产品里，Evaluator 由 `agent-evaluation` skill 按一份*私有* rubric 驱动；这个示例把那个理念 - 蒸馏成可运行的核心。 -- 本地模型是**非确定性**的，所以示例对每个版本各跑多次取平均——这正是真实 benchmark 里为每个 Case - 设置 `runs`（多次运行）的原因。单次可能上下波动；均值才能告诉你这次编辑到底有没有帮助。 +| 脚本 | 演示什么 | 由谁编辑 `AGENTS.md` | +| --- | --- | --- | +| `self-improve.ts` | **评分闭环**的微缩版（最简） | **脚本**（硬编码） | +| `self-evolve.ts` | **真正的**单轮自我进化 | **Agent** 自己 | +| `self-evolve-recursive.ts` | **多轮递归**——主线（`pnpm start`） | **Agent**，连续两轮 | + +`self-improve.ts` 是诚实的基线：它展示了 评估 → 编辑 → 重新评估 的机制，但那次编辑是人写好的 +`DISCIPLINE` 字符串、由脚本写到磁盘。它演示的是*循环*，不是自我进化。两个 `self-evolve*` 脚本把 +诊断**和**编辑都交给 Agent——脚本只提供一个失败信号和一份范例；学习由模型完成，并被固化进它自己的 +身份文件。 + +## 任务，以及为什么 baseline 会稳定失败 + +任务看起来很简单：*把 `notes.txt` 总结进 `summary.md`，含一段 2 句概述和恰好 3 条要点——并遵守你们 +团队的标准报告格式。* 玄机在最后一句：这个“团队格式”是一套**任意的内部约定**（一行 marker、一个 +`# Report: ` 标题、一行 `Classification: INTERNAL`、一行 `Reviewed-by: Aurora Team` 页脚）， +它**只出现在 `AGENTS.md` 里**，**无法从任务本身推断**。 + +这正是让 baseline *稳定*烂的设计诀窍——对任何模型都成立，无论强弱。rubric（`score()`，普通代码）有 +10 个原子分点：**5 个内容点**任何有能力的模型仅凭任务就能拿到，**5 个约定点**只能从 `AGENTS.md` 得知。 +`AGENTS.md` 为空时，Agent 猜不到约定，于是稳定丢掉那 5 分。这是**信息缺口，不是能力缺口**——也正是 +为什么更强的模型也没法“自己想出来”。它也对应了完整产品的形态：那里 Evaluator 由 `agent-evaluation` +skill 按一份*私有* rubric 驱动。 + +Agent 通过**从一份通过范例里学到约定**、并把它写进自己的 `AGENTS.md` 来弥合这个缺口——在递归脚本里， +更进一步：见过多份范例后，把其中的固定常量锁定下来。 ## 1–2. 提供模型并把 PenguinHarness 指向它 ```bash export HIP_VISIBLE_DEVICES=0 # 可选：指定某张 AMD GPU ollama serve & -ollama pull qwen3:8b +ollama pull qwen3.6:35b penguin config model add \ - --model-id qwen3:8b \ + --model-id qwen3.6:35b \ --provider custom --client-type openai \ --base-url http://localhost:11434/v1 \ --api-key ollama --set-default @@ -40,33 +61,58 @@ penguin config model add \ ```bash pnpm install pnpm build -pnpm --dir examples/self-improving-agent start -# 或直接运行： npx tsx examples/self-improving-agent/self-improve.ts +pnpm --dir examples/self-improving-agent start # 递归主线 +# 或单轮： npx tsx examples/self-improving-agent/self-evolve.ts +# 或评分闭环基线： npx tsx examples/self-improving-agent/self-improve.ts ``` ## 你应当看到什么 ```text -BASELINE (blank AGENTS.md): 3 runs - run 1: 0/5 - run 2: 0/5 - run 3: 0/5 - BASELINE mean: 0.00/5 -N+1 (with working discipline): 3 runs - run 1: 5/5 - run 2: 5/5 - run 3: 5/5 - N+1 mean: 5.00/5 -=== Self-improvement result === - baseline: 0.00/5 → N+1: 5.00/5 - Mean score improved — keep version N+1. ✔ +N BASELINE (blank AGENTS.md): 5 runs + ... scores: [5, 4, 5, 5, 5] mean: 4.80/10 # 稳定丢掉每个约定点 + +=== REFLECT round 1: infer STRUCTURE from a single accepted report === +----- after round 1: AGENTS.md the agent authored ----- + ## Report-Publishing Convention ... + # 学到结构，但常量还是猜的 + Reviewed-by: + +N+1 (structure learned): 5 runs + ... scores: [6, 7, 6, 6, 8] mean: 6.60/10 + +=== REFLECT round 2: RECURSE on own AGENTS.md — lock CONSTANTS from 3 examples === +----- after round 2: AGENTS.md the agent authored ----- + ### Fixed Constants (verbatim) + | 1 | | # 常量被锁定为字面量 + | Last line | Reviewed-by: Aurora Team | + +N+2 (constants locked): 5 runs + ... scores: [10, 10, 9, 10, 10] mean: 9.80/10 + +=== Recursive self-evolution trajectory === + N (baseline): 4.80/10 + N+1 (structure): 6.60/10 (+1.80) + N+2 (constants): 9.80/10 (+3.20) + Monotonic improvement across two self-authored rounds — recursive self-evolution. ✔ ``` -当 `AGENTS.md` 为空时，`qwen3:8b` 往往会在对话里*叙述*摘要，却从不调用工具把文件写出来——于是 -rubric 判它 0 分。加上一小段“任务纪律”（先读原文、把约束逐条列出、真正把文件写出来、结束前自检）就能 -扭转这一点。具体数字每次运行会有波动；重点是取平均后的方向。 +重点在这条轨迹：只有一份通过范例时，Agent 能推断出**结构**，却分不清哪些 token 是固定常量、哪些是 +每份报告要替换的字段（单个样本本就有歧义），于是卡在中途。给它**多份**共享同一 marker 和页脚的通过 +范例后，它推断出“凡是在所有样本里都完全相同的，就是固定常量”，读取自己**上一轮**写的 `AGENTS.md` +再精炼它——这才是真正意义上的递归：`state_{n+1} = agent.reflect(state_n, 新证据)`。具体数字每次运行 +会有波动；重点是那条单调上升的方向。 + +## 诚实的边界 + +- **自我进化需要一个足够强的模型。** 自己写出一条正确规则、然后*遵守它*，比坐享一条硬编码规则更难。 + 较弱的模型（例如 `qwen3:8b`）常常能推断出规则却执行不稳（占位符没填、或者只叙述而不真正写文件）， + 于是它的 N+1 可能*倒退*，脚本随之回滚。那次回滚是循环在诚实工作，不是 bug。 +- **证据充分性是真实的。** N+1 卡在中位，是因为单份范例确实无法确定固定常量；N+2 能冲高，只因更多 + 范例让常量变得可推断。这份提升是信息驱动的学习，不是噪声。 ## 说明 - 使用一个专用 agent id（`self-improve-demo`），运行时即时创建——你自己的 Agent 不会被动到。 -- 再次运行会就地更新这个 demo agent。 +- 脚本从不自己写那套约定；它只提供失败报告、通过范例，以及保留/回滚的信号。 +- 再次运行会把这个 demo agent 的 `AGENTS.md` 重置为空，并重新开始循环。 diff --git a/examples/self-improving-agent/package.json b/examples/self-improving-agent/package.json index 90dc91f..38ad9e2 100644 --- a/examples/self-improving-agent/package.json +++ b/examples/self-improving-agent/package.json @@ -3,7 +3,7 @@ "private": true, "type": "module", "scripts": { - "start": "tsx self-improve.ts" + "start": "tsx self-evolve-recursive.ts" }, "dependencies": { "@prismshadow/penguin-core": "workspace:*" diff --git a/examples/self-improving-agent/self-evolve-recursive.ts b/examples/self-improving-agent/self-evolve-recursive.ts new file mode 100644 index 0000000..2404a55 --- /dev/null +++ b/examples/self-improving-agent/self-evolve-recursive.ts @@ -0,0 +1,287 @@ +/** + * RECURSIVE self-evolution — the agent improves its OWN AGENTS.md over MULTIPLE rounds, each round + * building on the state it authored in the previous round. + * + * Why a second round is needed (and is the interesting part): + * In round 1 the agent sees ONE accepted report (Borealis). It can infer the STRUCTURE + * (marker line / title / classification / footer) but it CANNOT know which tokens are fixed + * constants vs per-report variables — a single example is ambiguous. So it generalizes the + * marker to a placeholder and guesses the sign-off, and stably lands mid (~7/10). + * + * In round 2 the agent is given SEVERAL accepted reports from DIFFERENT projects that all share + * the SAME marker (``) and the SAME footer (`Reviewed-by: Aurora + * Team`). Now the invariants are inferable: whatever is identical across every accepted report is + * a FIXED constant; whatever varies is a field. The agent reads its OWN round-1 AGENTS.md and + * REFINES it to lock those constants down → N+2 climbs to near-ceiling. + * + * This is recursion in the true sense: state_{n+1} = agent.reflect(state_n, new_evidence). The + * script never writes a rule; it only supplies evidence and the keep/rollback signal. + * + * Run: cd examples/self-improving-agent && pnpm exec tsx self-evolve-recursive.ts + */ +import { promises as fs } from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { + createAgent, + userText, + isCompleteModelMessage, + type OmniMessage, +} from "@prismshadow/penguin-core"; + +const AGENT_ID = "self-improve-demo"; +const PROJECT_ID = "default_project"; + +const TASK = `Read notes.txt in your workspace and write summary.md that summarizes the notes with: +(1) an overview of at most 2 sentences, and +(2) a bullet list of exactly 3 key facts. +Follow your team's standard report format.`; + +const NOTES = `Project Aurora — Internal Notes + +Aurora is a real-time analytics platform launched in Q1 2026. At peak it ingests roughly +2 million events per second through a Kafka-based pipeline. The core query engine was rewritten +in Rust after the original Go version could not keep p99 latency under control; the rewrite cut +p99 from 800ms to 120ms. + +The production deployment is single-region with no failover; a multi-region rollout is scheduled +for Q3 2026. Storage is the largest cost line at about $48,000/month, driven by a 90-day +hot-retention policy; cutting retention to 30 days would reduce storage cost by roughly 55%. +`; + +/** Round 1: a SINGLE accepted report. Structure is inferable; the fixed constants are ambiguous. */ +const REFERENCE_1 = ` +# Report: Project Borealis +Classification: INTERNAL + +Borealis is a batch ETL platform migrated to Spark in 2025; it processes about 40TB nightly. + +- Cut the nightly window from 6h to 90 minutes +- Runs on a 200-node autoscaling cluster +- Compute costs roughly $12,000/month + +Reviewed-by: Aurora Team +`; + +/** Round 2: SEVERAL accepted reports from different projects. The marker and the sign-off are + * IDENTICAL across all of them (→ fixed constants); the title subject and body vary (→ fields). */ +const REFERENCE_2 = ` +# Report: Project Cascade +Classification: INTERNAL + +Cascade is a streaming feature store rolled out in 2026 serving 500 models online. + +- p99 read latency 8ms at 300k QPS +- Backed by a Redis + RocksDB tier +- Operates at about $9,500/month + +Reviewed-by: Aurora Team +`; +const REFERENCE_3 = ` +# Report: Project Delta +Classification: INTERNAL + +Delta is an internal experimentation platform launched in Q4 2025 running 1,200 concurrent tests. + +- Reduced experiment setup from days to under an hour +- Guardrail metrics evaluated hourly +- Infra footprint about $6,000/month + +Reviewed-by: Aurora Team +`; + +function rootDir(): string { + return process.env.PENGUIN_HOME ?? path.join(os.homedir(), ".penguin", "data"); +} +function agentStateDir(): string { + return path.join(rootDir(), PROJECT_ID, "agents", AGENT_ID, "agent_state"); +} + +function score(summaryText: string | null): { score: number; detail: string[] } { + const detail: string[] = []; + let s = 0; + const add = (ok: boolean, label: string) => { + if (ok) s += 1; + detail.push(`${ok ? "1" : "0"}/1 ${label}`); + }; + + const exists = summaryText !== null; + add(exists, "file summary.md was written"); + if (!exists) return { score: 0, detail }; + const text = summaryText!; + const lines = text.split("\n"); + + const bullets = (text.match(/^\s*[-*]\s+/gm) ?? []).length; + add(bullets === 3, `exactly 3 bullet facts (found ${bullets})`); + add(text.includes("120ms"), "mentions the p99 latency figure (120ms)"); + add(text.includes("2 million") || /2m\s*events/i.test(text), "mentions the throughput figure"); + add(text.includes("55%") || text.includes("48"), "mentions a storage cost/retention figure"); + + add(lines[0]?.trim() === "", "[convention] line 1 is the ACME marker"); + add( + /^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()), + "[convention] line 2 is '# Report: '", + ); + add( + /^Classification:\s*INTERNAL\s*$/m.test(text), + "[convention] contains 'Classification: INTERNAL'", + ); + const lastNonEmpty = [...lines].reverse().find((l) => l.trim() !== "") ?? ""; + add( + lastNonEmpty.trim() === "Reviewed-by: Aurora Team", + "[convention] final line is 'Reviewed-by: Aurora Team'", + ); + const allFour = + lines[0]?.trim() === "" && + /^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()) && + /^Classification:\s*INTERNAL\s*$/m.test(text) && + lastNonEmpty.trim() === "Reviewed-by: Aurora Team"; + add(allFour, "[convention] full team format satisfied end-to-end"); + + return { score: s, detail }; +} + +async function collect(run: AsyncGenerator): Promise { + for await (const msg of run) { + if (isCompleteModelMessage(msg) && msg.payload.type === "text") { + /* agent chatter suppressed */ + } + } +} + +async function runOnce(): Promise<{ score: number; detail: string[]; report: string | null }> { + const agent = await createAgent({ agentId: AGENT_ID }); + const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-r-")); + await fs.writeFile(path.join(ws, "notes.txt"), NOTES, "utf8"); + const session = await agent.createSession({ workspaceDir: ws }); + await collect(session.run([userText(TASK)], { approve: async () => "allow" })); + let report: string | null = null; + try { + report = await fs.readFile(path.join(ws, "summary.md"), "utf8"); + } catch { + report = null; + } + const { score: sc, detail } = score(report); + return { score: sc, detail, report }; +} + +const RUNS = 5; +let lastReport = ""; +async function evaluate(label: string): Promise { + console.log(`\n--- ${label}: ${RUNS} runs ---`); + const scores: number[] = []; + for (let i = 0; i < RUNS; i++) { + const r = await runOnce(); + scores.push(r.score); + if (r.report) lastReport = r.report; + const missed = r.detail.filter((d) => d.startsWith("0")).map((d) => d.replace(/^0\/1\s+/, "")); + console.log(` run ${i + 1}: ${r.score}/10 [missed: ${missed.join("; ") || "none"}]`); + } + const mean = scores.reduce((a, b) => a + b, 0) / scores.length; + console.log(` ${label} scores: [${scores.join(", ")}] mean: ${mean.toFixed(2)}/10`); + return mean; +} + +/** Round 1 reflection: infer STRUCTURE from a single accepted example + the agent's own failure. */ +async function reflectRound1(failedReport: string): Promise { + const agent = await createAgent({ agentId: AGENT_ID }); + const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-r1-")); + await fs.writeFile(path.join(ws, "last_report.md"), failedReport, "utf8"); + await fs.writeFile(path.join(ws, "accepted_1.md"), REFERENCE_1, "utf8"); + const agentsMd = path.join(agentStateDir(), "AGENTS.md"); + const prompt = `You are improving your own future performance by editing your persistent instructions. + +In your workspace: +- last_report.md — a report you wrote that the team REJECTED. +- accepted_1.md — a DIFFERENT project's report that PASSED review. + +Nobody has told you the rules. Read both, infer the team's reusable report-publishing convention +(marker line, title format, metadata line, footer/sign-off), and WRITE it as durable general +guidance into your AGENTS.md at this absolute path: + ${agentsMd} +(Overwrite the file.) Capture the fixed lines and the per-report variable parts. Do not copy the +example's specific content. Use your file tools to read and to write, then confirm.`; + const session = await agent.createSession({ workspaceDir: ws }); + await collect(session.run([userText(prompt)], { approve: async () => "allow" })); +} + +/** Round 2 reflection: RECURSE on the agent's own AGENTS.md using MULTIPLE accepted examples so the + * invariants (constant marker + constant sign-off) become inferable and can be locked down. */ +async function reflectRound2(): Promise { + const agent = await createAgent({ agentId: AGENT_ID }); + const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-r2-")); + await fs.writeFile(path.join(ws, "accepted_1.md"), REFERENCE_1, "utf8"); + await fs.writeFile(path.join(ws, "accepted_2.md"), REFERENCE_2, "utf8"); + await fs.writeFile(path.join(ws, "accepted_3.md"), REFERENCE_3, "utf8"); + const agentsMd = path.join(agentStateDir(), "AGENTS.md"); + const prompt = `You are refining your own report-publishing convention. Your current convention is +already written in your AGENTS.md at: + ${agentsMd} +Read it first. + +Your workspace now has THREE accepted reports from DIFFERENT projects: accepted_1.md, +accepted_2.md, accepted_3.md. Compare them carefully. The key insight of this round: + + Anything that is IDENTICAL, character-for-character, across ALL THREE accepted reports is a + FIXED CONSTANT that you must reproduce VERBATIM in every report (for example a specific marker + line or a specific sign-off line). Anything that DIFFERS between them (like the project name in + the title, or the body text) is a per-report FIELD you fill in. + +Update your AGENTS.md so it records the fixed constants as EXACT literal strings (not placeholders), +and clearly marks which parts vary per report. Overwrite the file at the path above using your file +tools. Then confirm what you changed and why.`; + const session = await agent.createSession({ workspaceDir: ws }); + await collect(session.run([userText(prompt)], { approve: async () => "allow" })); +} + +async function showState(tag: string): Promise { + const agentsMd = path.join(agentStateDir(), "AGENTS.md"); + const txt = (await fs.readFile(agentsMd, "utf8")).trim(); + console.log(`\n----- ${tag}: AGENTS.md the agent authored -----`); + console.log(txt || "(empty)"); + console.log("-".repeat(48)); +} + +async function main(): Promise { + await createAgent({ agentId: AGENT_ID }); + const agentsMd = path.join(agentStateDir(), "AGENTS.md"); + await fs.writeFile(agentsMd, "", "utf8"); + + // N: baseline + const nBaseline = await evaluate("N BASELINE (blank AGENTS.md)"); + const failed = lastReport; + + // Round 1: structure from one example + console.log("\n=== REFLECT round 1: infer STRUCTURE from a single accepted report ==="); + await reflectRound1(failed); + await showState("after round 1"); + const nPlus1 = await evaluate("N+1 (structure learned)"); + + // Round 2: recurse — lock the constants from multiple examples + console.log( + "\n=== REFLECT round 2: RECURSE on own AGENTS.md — lock CONSTANTS from 3 examples ===", + ); + await reflectRound2(); + await showState("after round 2"); + const nPlus2 = await evaluate("N+2 (constants locked)"); + + console.log("\n=== Recursive self-evolution trajectory ==="); + console.log(` N (baseline): ${nBaseline.toFixed(2)}/10`); + console.log( + ` N+1 (structure): ${nPlus1.toFixed(2)}/10 (${(nPlus1 - nBaseline >= 0 ? "+" : "") + (nPlus1 - nBaseline).toFixed(2)})`, + ); + console.log( + ` N+2 (constants): ${nPlus2.toFixed(2)}/10 (${(nPlus2 - nPlus1 >= 0 ? "+" : "") + (nPlus2 - nPlus1).toFixed(2)})`, + ); + const monotonic = nPlus1 >= nBaseline && nPlus2 >= nPlus1; + console.log( + monotonic + ? " Monotonic improvement across two self-authored rounds — recursive self-evolution. ✔" + : " Non-monotonic — inspect which round regressed.", + ); +} + +main().catch((err) => { + console.error("Example failed:", err); + process.exitCode = 1; +}); diff --git a/examples/self-improving-agent/self-evolve.ts b/examples/self-improving-agent/self-evolve.ts new file mode 100644 index 0000000..f4e8d83 --- /dev/null +++ b/examples/self-improving-agent/self-evolve.ts @@ -0,0 +1,238 @@ +/** + * GENUINE self-evolution demo — the agent diagnoses its own failure and edits its OWN AGENTS.md. + * + * The original example (and the -hard variant) hardcode the improvement: + * `fs.writeFile(agentsMd, DISCIPLINE)` ← the human writes the answer, the agent just benefits. + * That demonstrates the *scoring loop*, but NOT self-evolution — the model does no diagnosing and + * authors no fix. + * + * Here the EDIT step is done BY THE AGENT ITSELF: + * 1. EVALUATE — the agent writes a report for a task that says only "follow your team's format". + * With a blank AGENTS.md it cannot know the (arbitrary, unguessable) house + * convention, so it stably loses the convention points. + * 2. REFLECT — the agent is shown (a) its own rejected report and (b) a DIFFERENT report that + * passed review. Nobody tells it the rules. It must compare the two, INFER the + * reusable convention, and write that convention into its OWN AGENTS.md. + * 3. RE-EVALUATE — the same task again; now the agent applies the rule it taught itself. + * 4. KEEP / ROLL BACK — keep the self-authored AGENTS.md only if the mean score improved. + * + * The script never writes the convention. It only supplies a failure signal + one worked example; + * the model does the diagnosis and persists its own improvement into its identity (AGENTS.md). + * The reference uses a DIFFERENT subject (Borealis) than the task (Aurora), so the agent must + * generalize the *rules*, not copy the *content*. + * + * Run: cd examples/self-improving-agent && pnpm exec tsx self-evolve.ts + */ +import { promises as fs } from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { + createAgent, + userText, + isCompleteModelMessage, + type OmniMessage, +} from "@prismshadow/penguin-core"; + +const AGENT_ID = "self-improve-demo"; +const PROJECT_ID = "default_project"; + +/** The task fully specifies content, but only says "follow your team's format" — the convention + * itself is knowable only from AGENTS.md, which the agent must author for itself in REFLECT. */ +const TASK = `Read notes.txt in your workspace and write summary.md that summarizes the notes with: +(1) an overview of at most 2 sentences, and +(2) a bullet list of exactly 3 key facts. +Follow your team's standard report format.`; + +const NOTES = `Project Aurora — Internal Notes + +Aurora is a real-time analytics platform launched in Q1 2026. At peak it ingests roughly +2 million events per second through a Kafka-based pipeline. The core query engine was rewritten +in Rust after the original Go version could not keep p99 latency under control; the rewrite cut +p99 from 800ms to 120ms. + +The production deployment is single-region with no failover; a multi-region rollout is scheduled +for Q3 2026. Storage is the largest cost line at about $48,000/month, driven by a 90-day +hot-retention policy; cutting retention to 30 days would reduce storage cost by roughly 55%. +`; + +/** A gold report that PASSED review — a DIFFERENT subject (Borealis), so the agent must infer the + * reusable rules (marker/title-pattern/metadata/footer) rather than copy this report's content. */ +const REFERENCE_REPORT = ` +# Report: Project Borealis +Classification: INTERNAL + +Borealis is a batch ETL platform migrated to Spark in 2025; it processes about 40TB nightly. + +- Cut the nightly window from 6h to 90 minutes +- Runs on a 200-node autoscaling cluster +- Compute costs roughly $12,000/month + +Reviewed-by: Aurora Team +`; + +function rootDir(): string { + return process.env.PENGUIN_HOME ?? path.join(os.homedir(), ".penguin", "data"); +} +function agentStateDir(): string { + return path.join(rootDir(), PROJECT_ID, "agents", AGENT_ID, "agent_state"); +} + +/** 10 atomic points: 5 CONTENT (inferable from the task) + 5 CONVENTION (knowable only from the + * house rule the agent must teach itself). Line-2 check requires the ACTUAL subject (Aurora), + * so blindly copying the Borealis reference does not earn the point. */ +function score(summaryText: string | null): { score: number; detail: string[] } { + const detail: string[] = []; + let s = 0; + const add = (ok: boolean, label: string) => { + if (ok) s += 1; + detail.push(`${ok ? "1" : "0"}/1 ${label}`); + }; + + const exists = summaryText !== null; + add(exists, "file summary.md was written"); + if (!exists) return { score: 0, detail }; + const text = summaryText!; + const lines = text.split("\n"); + + // CONTENT points + const bullets = (text.match(/^\s*[-*]\s+/gm) ?? []).length; + add(bullets === 3, `exactly 3 bullet facts (found ${bullets})`); + add(text.includes("120ms"), "mentions the p99 latency figure (120ms)"); + add(text.includes("2 million") || /2m\s*events/i.test(text), "mentions the throughput figure"); + add(text.includes("55%") || text.includes("48"), "mentions a storage cost/retention figure"); + + // CONVENTION points — only satisfiable via the self-authored AGENTS.md rule + add(lines[0]?.trim() === "", "[convention] line 1 is the ACME marker"); + add( + /^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()), + "[convention] line 2 is '# Report: '", + ); + add( + /^Classification:\s*INTERNAL\s*$/m.test(text), + "[convention] contains 'Classification: INTERNAL'", + ); + const lastNonEmpty = [...lines].reverse().find((l) => l.trim() !== "") ?? ""; + add( + lastNonEmpty.trim() === "Reviewed-by: Aurora Team", + "[convention] final line is 'Reviewed-by: Aurora Team'", + ); + const allFour = + lines[0]?.trim() === "" && + /^#\s+Report:.*aurora/i.test((lines[1] ?? "").trim()) && + /^Classification:\s*INTERNAL\s*$/m.test(text) && + lastNonEmpty.trim() === "Reviewed-by: Aurora Team"; + add(allFour, "[convention] full team format satisfied end-to-end"); + + return { score: s, detail }; +} + +async function collect(run: AsyncGenerator): Promise { + for await (const msg of run) { + if (isCompleteModelMessage(msg) && msg.payload.type === "text") { + // keep the demo output focused; the agent's chatter is not the point here + } + } +} + +/** One evaluation run; returns the score and the report text the agent wrote (for REFLECT). */ +async function runOnce(): Promise<{ score: number; detail: string[]; report: string | null }> { + const agent = await createAgent({ agentId: AGENT_ID }); + const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-")); + await fs.writeFile(path.join(ws, "notes.txt"), NOTES, "utf8"); + const session = await agent.createSession({ workspaceDir: ws }); + await collect(session.run([userText(TASK)], { approve: async () => "allow" })); + let report: string | null = null; + try { + report = await fs.readFile(path.join(ws, "summary.md"), "utf8"); + } catch { + report = null; + } + const { score: sc, detail } = score(report); + return { score: sc, detail, report }; +} + +const RUNS = 5; +async function evaluate(label: string): Promise { + console.log(`\n--- ${label}: ${RUNS} runs ---`); + const scores: number[] = []; + let lastReport: string | null = null; + for (let i = 0; i < RUNS; i++) { + const r = await runOnce(); + scores.push(r.score); + if (r.report) lastReport = r.report; + const missed = r.detail.filter((d) => d.startsWith("0")).map((d) => d.replace(/^0\/1\s+/, "")); + console.log(` run ${i + 1}: ${r.score}/10 [missed: ${missed.join("; ") || "none"}]`); + } + const mean = scores.reduce((a, b) => a + b, 0) / scores.length; + console.log(` ${label} scores: [${scores.join(", ")}] mean: ${mean.toFixed(2)}/10`); + (evaluate as unknown as { lastReport?: string | null }).lastReport = lastReport; + return mean; +} + +/** The self-evolution step: the AGENT reads its own failed report + a passing reference, infers the + * house convention, and writes it into its OWN AGENTS.md. The script provides no rules. */ +async function reflectAndEditOwnState(failedReport: string): Promise { + const agent = await createAgent({ agentId: AGENT_ID }); + const ws = await fs.mkdtemp(path.join(os.tmpdir(), "self-evolve-reflect-")); + await fs.writeFile(path.join(ws, "last_report.md"), failedReport, "utf8"); + await fs.writeFile(path.join(ws, "reference_report.md"), REFERENCE_REPORT, "utf8"); + const agentsMd = path.join(agentStateDir(), "AGENTS.md"); + + const REFLECT = `You are improving your own future performance by editing your persistent instructions. + +In your workspace are two files: +- last_report.md — a report you recently wrote that the team REJECTED. +- reference_report.md — a DIFFERENT report (different project) that PASSED the team's review. + +Nobody has told you the formatting rules. Read BOTH files, compare them, and infer the team's +reusable report-publishing convention: any required marker line, the title format, required +metadata line(s), and the required footer/sign-off line. + +Then WRITE those rules as durable, general guidance into your AGENTS.md at this absolute path: + ${agentsMd} +(Overwrite the file.) Capture the GENERAL rules that apply to every future report — the exact +fixed lines that never change, and the parts that vary per report (like the subject in the title). +Do NOT copy reference_report.md's specific content. Use your file tools to read the two files and +to write AGENTS.md, then confirm you wrote it.`; + + const session = await agent.createSession({ workspaceDir: ws }); + await collect(session.run([userText(REFLECT)], { approve: async () => "allow" })); +} + +async function main(): Promise { + await createAgent({ agentId: AGENT_ID }); + const agentsMd = path.join(agentStateDir(), "AGENTS.md"); + await fs.writeFile(agentsMd, "", "utf8"); // start blank — the agent knows no convention yet + + // 1. EVALUATE baseline + const baseline = await evaluate("BASELINE (blank AGENTS.md)"); + const failed = (evaluate as unknown as { lastReport?: string | null }).lastReport ?? ""; + + // 2. REFLECT — the agent diagnoses and edits its OWN AGENTS.md + console.log( + "\n--- REFLECT: the agent studies a passing example and rewrites its OWN AGENTS.md ---", + ); + await reflectAndEditOwnState(failed); + const authored = await fs.readFile(agentsMd, "utf8"); + console.log("\n----- AGENTS.md the agent wrote for itself -----"); + console.log(authored.trim() || "(empty — the agent failed to author a rule)"); + console.log("-----------------------------------------------"); + + // 3. RE-EVALUATE with the self-authored rule + const evolved = await evaluate("N+1 (agent's self-authored convention)"); + + // 4. KEEP / ROLL BACK + console.log("\n=== Self-evolution result ==="); + console.log(` baseline: ${baseline.toFixed(2)}/10 → N+1: ${evolved.toFixed(2)}/10`); + if (evolved > baseline) { + console.log(" The agent's self-authored edit improved its score — keep it. ✔"); + } else { + console.log(" No improvement — roll back to the blank AGENTS.md."); + await fs.writeFile(agentsMd, "", "utf8"); + } +} + +main().catch((err) => { + console.error("Example failed:", err); + process.exitCode = 1; +}); diff --git a/examples/self-improving-agent/self-improve.ts b/examples/self-improving-agent/self-improve.ts index a052cce..b79aff8 100644 --- a/examples/self-improving-agent/self-improve.ts +++ b/examples/self-improving-agent/self-improve.ts @@ -2,7 +2,7 @@ * Example: an Agent that improves itself — one turn of the self-improvement loop, in code. * * This is the "Recursive Self-Improvement" pillar made runnable. It runs entirely on a local - * open-weight model (Ollama serving qwen3:8b) — see README.md for the one-time setup. + * open-weight model (Ollama serving qwen3.6:35b) — see README.md for the one-time setup. * * The loop, exactly as the docs describe it: * 1. EVALUATE — run the agent on a constrained task, score it against a rubric. diff --git a/packages/landing/content/blog/local-agents-on-amd-gpus.en.md b/packages/landing/content/blog/local-agents-on-amd-gpus.en.md index 4976add..583c85a 100644 --- a/packages/landing/content/blog/local-agents-on-amd-gpus.en.md +++ b/packages/landing/content/blog/local-agents-on-amd-gpus.en.md @@ -156,25 +156,25 @@ The third pillar builds on the same idea. Because agents are editable data and e Design tenets are easy to claim. Here is an end-to-end run that exercises them, entirely on fully local, open-weight infrastructure with an AMD GPU — every token generated on-device, nothing sent to a cloud API. -**The setup.** An AMD GPU running ROCm, serving `qwen3:8b` — a popular, everyday local open-weight model (about 5 GB) — through Ollama's OpenAI-compatible endpoint. Ollama detected the AMD GPU natively (no architecture override needed) and loaded the model into GPU memory. This same path spans AMD's ROCm-supported lineup — from Radeon PRO workstation cards such as the W7900 (48 GB, RDNA3) up to datacenter Instinct accelerators. Wiring it into PenguinHarness took a single command, precisely because the core treats any OpenAI-compatible endpoint the same way: +**The setup.** An AMD GPU running ROCm, serving `qwen3.6:35b` — a capable local open-weight model (a ~24 GB MoE, 36B total parameters, Q4_K_M) — through Ollama's OpenAI-compatible endpoint. Ollama detected the AMD GPU natively (no architecture override needed) and loaded the model into GPU memory. This same path spans AMD's ROCm-supported lineup — from Radeon PRO workstation cards such as the W7900 (48 GB, RDNA3) up to datacenter Instinct accelerators. Wiring it into PenguinHarness took a single command, precisely because the core treats any OpenAI-compatible endpoint the same way: ```bash penguin config model add \ - --model-id qwen3:8b \ + --model-id qwen3.6:35b \ --provider custom --client-type openai \ --base-url http://localhost:11434/v1 \ --api-key ollama --set-default ``` -**The task, and how it's scored.** We gave it a constrained writing task in the style of the built-in `example-benchmark`: read a project notes file and produce a summary with an overview of at most 2 sentences, exactly 3 key facts, under 60 words total. The task carries a private rubric — a scoring checklist the agent never sees, so it cannot "teach to the test." The rubric turns each requirement into one point (file actually written · ≤2-sentence overview · exactly 3 bullets · under 60 words · facts accurate), for a score out of 5. +**The task, and how it's scored.** We gave it a task that looks trivial: read a project notes file and produce a summary with a 2-sentence overview and exactly 3 key facts — *and follow the team's standard report format.* That last clause is the whole point. The "team format" is an arbitrary house convention — a specific marker line, a `# Report: ` title, a `Classification: INTERNAL` line, a `Reviewed-by: Aurora Team` footer — that lives *only in the agent's `AGENTS.md`* and cannot be inferred from the task. The task carries a private rubric — a checklist the agent never sees — with 10 points: 5 for content any capable model earns from the task alone, and 5 for the convention, which is knowable only from `AGENTS.md`. -**The baseline — and why it isn't perfect.** Running locally on the AMD GPU, the model read the file, composed a perfectly reasonable summary… and then never wrote it to disk. It emitted the answer as chat text instead of calling the write tool. No deliverable means the rubric scores it **0 / 5**. And because every step is in the Trace, this is not guesswork — you can open the run and see exactly where it went wrong. +**The baseline — and why it isn't perfect.** Running locally on the AMD GPU with a blank `AGENTS.md`, the model wrote a perfectly reasonable summary — and stably lost all 5 convention points, landing around **4.6 / 10**. It could not have done otherwise: nothing in the task reveals the house convention, so this is an *information gap, not a capability gap* — which is exactly why a stronger model can't just "figure it out." And because every step is in the Trace, this is not guesswork — you can open the run and see precisely which points were missed. -That is the honest starting point: on local hardware, an out-of-the-box model does *not* ace a constrained task. Which is exactly what makes the next section interesting — a measured, auditable failure is something you can systematically fix. +That is the honest starting point: on local hardware, an out-of-the-box agent does *not* ace a task whose rules live in files it hasn't learned yet. Which is exactly what makes the next section interesting — a measured, auditable, *reproducible* failure is something the agent can systematically fix by teaching itself. ## How self-improvement actually works — and a real before/after -The 0 / 5 above is an *evaluation* — a snapshot of the agent as it stands. The more interesting question is how PenguinHarness turns that snapshot into progress. This is the recursive self-improvement loop, and it is worth understanding as a mechanism, because there is no magic engine code behind it — it is ordinary agent machinery, orchestrated by Skills: +The 4.6 / 10 above is an *evaluation* — a snapshot of the agent as it stands. The more interesting question is how PenguinHarness turns that snapshot into progress. This is the recursive self-improvement loop, and it is worth understanding as a mechanism, because there is no magic engine code behind it — it is ordinary agent machinery, orchestrated by Skills: 1. **Benchmark** — define capability cases, each with a private rubric (as above). 2. **Evaluate** — run the agent over the cases and score against the rubrics. Each run is an ordinary, fully-traced Session. @@ -182,22 +182,24 @@ The 0 / 5 above is an *evaluation* — a snapshot of the agent as it stands. The 4. **Edit the Agent's state** — the agent's behavior lives in editable files (`AGENTS.md`, Skills, config). You (or an Optimizer agent) change those to address the failure, producing version N+1. 5. **Snapshot & keep-or-roll-back** — snapshot before each round; keep N+1 only if the score strictly improves, otherwise roll back. -Let's improve the exact agent from the previous section — the same `qwen3:8b` on the same AMD GPU, the same task that just scored **0 / 5**. +Let's improve the exact agent from the previous section — the same `qwen3.6:35b` on the same AMD GPU, the same task that just scored **4.6 / 10**. And here is the crucial part: the *agent* does the diagnosing and the editing — we do not hand it the answer. -- **The diagnosis (from the Trace).** The run failed for one concrete reason: the model produced the summary as chat text and never called the tool to write `summary.md`. The Trace shows it — a `cat` to read the input, then prose, and no write. So the fix is not "make the model smarter"; it is "make the behavior reliable." -- **The edit (N → N+1).** We added a short *working-discipline* section to the agent's `AGENTS.md`: *read the source first, restate the task's constraints as a checklist, actually call the tool to write the file, and re-read the output to self-check before finishing.* That is the entire change — a few sentences in an editable text file the agent reads on every run. No retraining, no code. -- **The re-evaluation.** Same model, same task, run again: this time it wrote a correct, on-topic `summary.md` to disk, with a 2-sentence overview and exactly 3 key facts drawn from the source — **4 / 5** (the one remaining miss: it ran to 70 words, over the 60-word budget). +- **The diagnosis (by the agent).** We give the agent two files and nothing else: the report it just got rejected, and a *different* project's report that passed review. Nobody tells it the rules. It compares the two, and infers the reusable house convention — the marker, the title shape, the metadata line, the sign-off. This is the "read the Trace to find where points were lost" step, performed by the agent itself. +- **The edit (N → N+1), authored by the agent.** The agent writes the convention it just inferred into its own `AGENTS.md`. From a single example it correctly recovers the *structure*, but can't yet tell which tokens are fixed constants vs per-report fields — a single example is ambiguous — so it generalizes the marker to a placeholder. Re-evaluated, the score climbs to about **6.6 / 10**. No retraining, no code — the agent edited a text file it reads on every run. +- **The recursion (N+1 → N+2).** Now we show it *several* accepted reports from different projects that share the same marker and sign-off. The agent reasons that whatever is identical across all of them must be a fixed constant, reads its *own* N+1 `AGENTS.md`, and refines it — locking `` and `Reviewed-by: Aurora Team` to literals. Re-evaluated again: about **9.8 / 10**. That is recursion in the true sense: `state_{n+1} = agent.reflect(state_n, new_evidence)`. -That is a jump on the same model and the same task, achieved purely by editing a text file the agent reads. That is the loop in miniature: *what you can see, you (or an Optimizer agent) can improve* — and because the score comes from a rubric and links to a Trace, the improvement is measured, not vibes. This is exactly the "keep the change only if the score strictly improves" loop, one turn of the crank. +That is a **4.6 → 6.6 → 9.8** climb on the same model and task, achieved purely by the agent editing a text file it reads — with the harness keeping each round only because the score strictly improved. That is the loop, self-driven: *what the agent can see, the agent can improve* — measured against a rubric, linked to a Trace, not vibes. **Run it yourself.** This whole loop is a self-contained, SDK-driven script in the repo: [`examples/self-improving-agent/`](https://github.com/Prism-Shadow/penguin-harness/tree/main/examples/self-improving-agent). -It uses a deterministic, readable rubric (file written · ≤2 sentences · exactly 3 bullets · under -60 words · facts accurate) and — because a local model is nondeterministic — averages several runs -per version, which is exactly why real benchmarks use a `runs` count. In our runs the averaged -score moved from **0.00/5** (blank `AGENTS.md`) to **5.00/5** (with the working-discipline edit), -and the script keeps N+1 only because the score strictly improved. It runs on the same local -Ollama + qwen3:8b setup and uses a dedicated agent id, so your own agents are untouched. +It uses a deterministic, readable rubric (10 points: 5 for content, 5 for the house convention that +lives only in `AGENTS.md`) and — because a local model is nondeterministic — averages several runs +per version, which is exactly why real benchmarks use a `runs` count. Crucially, the script never +writes the convention itself: the *agent* diagnoses the gap from a passing example and edits its own +`AGENTS.md`. In our runs the averaged score moved **4.6 → 6.6 → 9.8** across two self-authored +rounds (structure learned, then constants locked), and the harness keeps each round only because the +score strictly improved. It runs on the same local Ollama + qwen3.6:35b setup and uses a dedicated +agent id, so your own agents are untouched. ## No AMD GPU yet? Free cloud compute from AMD + Fireworks diff --git a/packages/landing/content/blog/local-agents-on-amd-gpus.zh.md b/packages/landing/content/blog/local-agents-on-amd-gpus.zh.md index 22eef79..4c69f47 100644 --- a/packages/landing/content/blog/local-agents-on-amd-gpus.zh.md +++ b/packages/landing/content/blog/local-agents-on-amd-gpus.zh.md @@ -154,25 +154,25 @@ spurious user-facing errors without requiring manual retries. 设计信条说起来容易。下面是一次真正践行了这些信条的端到端运行，全程在完全本地、开源权重、AMD GPU 的环境上进行——每一个 Token 都在本机生成，不向任何云端 API 发送。 -**环境。** 一块 AMD GPU 运行 ROCm，通过 Ollama 的 OpenAI 兼容端点提供 `qwen3:8b`——一个开发者常用的本地开源权重模型（约 5 GB）。Ollama 原生识别到该 AMD GPU（无需任何架构 override），并把模型加载进显存。这条路径覆盖 AMD 受 ROCm 支持的整个产品线——从 Radeon PRO 工作站显卡（如 W7900，48 GB，RDNA3）一直到数据中心的 Instinct 加速卡。把它接入 PenguinHarness 只需一条命令，正因为 core 对任何 OpenAI 兼容端点一视同仁： +**环境。** 一块 AMD GPU 运行 ROCm，通过 Ollama 的 OpenAI 兼容端点提供 `qwen3.6:35b`——一个能力不俗的本地开源权重模型（约 24 GB 的 MoE，36B 总参数，Q4_K_M 量化）。Ollama 原生识别到该 AMD GPU（无需任何架构 override），并把模型加载进显存。这条路径覆盖 AMD 受 ROCm 支持的整个产品线——从 Radeon PRO 工作站显卡（如 W7900，48 GB，RDNA3）一直到数据中心的 Instinct 加速卡。把它接入 PenguinHarness 只需一条命令，正因为 core 对任何 OpenAI 兼容端点一视同仁： ```bash penguin config model add \ - --model-id qwen3:8b \ + --model-id qwen3.6:35b \ --provider custom --client-type openai \ --base-url http://localhost:11434/v1 \ --api-key ollama --set-default ``` -**任务，以及如何评分。** 我们给它一个仿照内置 `example-benchmark` 风格的约束型写作任务：读一个项目 notes 文件，写一份摘要——概述至多 2 句、恰好 3 条要点、全文不超过 60 词。任务附带一份私有 rubric——一份 Agent 永远看不到的评分清单，这样它就无法"照着答案作答"。rubric 把每条要求变成一分（文件确实写出 · 概述 ≤2 句 · 恰好 3 条要点 · 不超过 60 词 · 事实准确），满分 5 分。 +**任务，以及如何评分。** 我们给它一个看似平平无奇的任务：读一个项目 notes 文件，写一份含 2 句概述和恰好 3 条要点的摘要——*并遵守团队的标准报告格式。* 玄机全在最后这句：这个“团队格式”是一套任意的内部约定——一行特定 marker、一个 `# Report: ` 标题、一行 `Classification: INTERNAL`、一行 `Reviewed-by: Aurora Team` 页脚——它*只存在于 Agent 的 `AGENTS.md` 里*，无法从任务本身推断。任务附带一份私有 rubric——一份 Agent 永远看不到的清单——共 10 分：5 分是任何有能力的模型仅凭任务就能拿到的内容分，另外 5 分是只能从 `AGENTS.md` 得知的约定分。 -**基线——以及它为何不是满分。** 跑在本地 AMD GPU 上，这个模型读了文件、组织出了一份相当合理的摘要……然后却从没把它写到磁盘。它把答案当成对话文本吐了出来，而没有调用写文件工具。没有交付物 → rubric 判它 **0 / 5**。而因为每一步都在 Trace 里，这不是靠猜——你可以打开这次运行，精确看到它错在哪。 +**基线——以及它为何不是满分。** 跑在本地 AMD GPU 上、`AGENTS.md` 为空时，这个模型写出了一份相当合理的摘要——却稳定丢掉全部 5 个约定分，落在 **4.6 / 10** 左右。它别无选择：任务里没有任何东西透露那套内部约定，所以这是*信息缺口，不是能力缺口*——也正是为什么更强的模型也没法“自己想出来”。而因为每一步都在 Trace 里，这不是靠猜——你可以打开这次运行，精确看到哪些分被丢了。 -这就是诚实的起点：在本地硬件上，一个开箱即用的模型面对约束型任务并不能一次拿满分。而这恰恰让下一节变得有意思——一个可度量、可审计的失败，是可以被系统性修复的。 +这就是诚实的起点：在本地硬件上，一个开箱即用的 Agent，面对规则藏在它尚未学过的文件里的任务，并不能一次拿满分。而这恰恰让下一节变得有意思——一个可度量、可审计、*可复现*的失败，Agent 可以通过自学来系统性修复。 ## 自我进化到底是怎么运作的——附一次真实的前后对比 -上面那个 0 / 5 是一次*评估*——是这个 Agent 当前状态的一张快照。更有意思的问题是：PenguinHarness 如何把这张快照变成进步。这就是递归自我进化循环，值得作为一种"机制"来理解，因为它背后没有任何魔法引擎代码——它就是由 Skills 编排的普通 Agent 机制： +上面那个 4.6 / 10 是一次*评估*——是这个 Agent 当前状态的一张快照。更有意思的问题是：PenguinHarness 如何把这张快照变成进步。这就是递归自我进化循环，值得作为一种"机制"来理解，因为它背后没有任何魔法引擎代码——它就是由 Skills 编排的普通 Agent 机制： 1. **Benchmark（定基准）**——定义能力 Case，每个都配一份私有 rubric（如上）。 2. **Evaluate（评估）**——让 Agent 跑这些 Case 并按 rubric 打分。每一次运行都是一个普通的、被完整追踪的 Session。 @@ -180,17 +180,17 @@ penguin config model add \ 4. **编辑 Agent 的状态**——Agent 的行为活在可编辑文件里（`AGENTS.md`、Skills、config）。你（或一个 Optimizer Agent）针对失分点去改这些文件，产出版本 N+1。 5. **快照 & 保留或回滚**——每轮前先打快照；只有当分数*严格提升*时才保留 N+1，否则回滚。 -我们就来改进上一节里那个 Agent——同一个 `qwen3:8b`、同一块 AMD GPU、同一个刚拿了 0 / 5 的任务。 +我们就来改进上一节里那个 Agent——同一个 `qwen3.6:35b`、同一块 AMD GPU、同一个刚拿了 4.6 / 10 的任务。关键之处在于：诊断和编辑都是 *Agent* 自己做的——我们并不把答案递给它。 -- **诊断（来自 Trace）。** 这次失败只有一个具体原因：模型把摘要当对话文本产出，却从没调用工具去写 `summary.md`。Trace 把它摆在眼前——一次 `cat` 读入、然后是正文、没有写入。所以要修的不是"让模型更聪明"，而是"让行为更可靠"。 -- **编辑（N → N+1）。** 我们往这个 Agent 的 `AGENTS.md` 里加了一小段*任务纪律*：*先读原文、把任务约束逐条列成清单、真正调用工具把文件写出来、结束前重新读一遍输出做自检。* 改动仅此而已——几句话，写进一份 Agent 每次运行都会读的可编辑文本文件。没有重新训练、没有改代码。 -- **重新评估。** 同一个模型、同一个任务，再跑一次：这次它把一份准确、切题的 `summary.md` 写到了磁盘，有一段 2 句的概述、恰好 3 条取自原文的要点——**4 / 5**（唯一还差的一分：它写到了 70 词，超出 60 词的预算）。 +- **诊断（由 Agent 完成）。** 我们只给 Agent 两个文件，别无其他：它刚被拒的那份报告，以及*另一个*项目里一份通过了评审的报告。没有人告诉它规则。它对比两者，自己推断出可复用的内部约定——marker、标题形态、元数据行、签名页脚。这就是"读 Trace 定位失分点"那一步，只不过由 Agent 亲自完成。 +- **编辑（N → N+1），由 Agent 撰写。** Agent 把刚推断出的约定写进它自己的 `AGENTS.md`。从单一范例里它能正确还原出*结构*，但还分不清哪些 token 是固定常量、哪些是每份报告要替换的字段（单个范例本就有歧义），于是把 marker 泛化成了占位符。重新评估，分数升到约 **6.6 / 10**。没有重新训练、没有改代码——Agent 编辑的是一份它每次运行都会读的文本文件。 +- **递归（N+1 → N+2）。** 现在我们再给它*多份*来自不同项目、却共享同一 marker 和签名的通过报告。Agent 推断出：凡是在所有样本里都完全相同的，必是固定常量；于是它读取自己*上一轮*写的 N+1 `AGENTS.md` 并精炼它——把 `` 和 `Reviewed-by: Aurora Team` 锁定为字面量。再评估一次：约 **9.8 / 10**。这才是真正意义上的递归：`state_{n+1} = agent.reflect(state_n, 新证据)`。 -在同一个模型、同一个任务上，仅仅通过编辑一份 Agent 会读取的文本文件，分数就提升了。这就是循环的微缩版：*你能看到的，你（或一个 Optimizer Agent）就能改进*——而且因为分数来自一份 rubric、又关联着 Trace，这份改进是被度量出来的，不是拍脑袋。这正是"只有分数严格提升才保留"的循环，转动了一格。 +在同一个模型、同一个任务上，仅仅由 Agent 自己编辑一份它会读取的文本文件，分数就走出了 **4.6 → 6.6 → 9.8** 的轨迹——而且 harness 每一轮都只因分数严格提升才保留。这就是自驱动的循环：*Agent 能看到的，Agent 就能改进*——用一份 rubric 度量、关联着 Trace，不是拍脑袋。 **你可以自己跑一遍。** 整个循环在仓库里有一个自包含、纯 SDK 驱动的脚本： [`examples/self-improving-agent/`](https://github.com/Prism-Shadow/penguin-harness/tree/main/examples/self-improving-agent)。 -它用一份确定性的、可读的 rubric（文件是否写出 · 概述 ≤2 句 · 恰好 3 条要点 · 不超过 60 词 · 事实准确），并且——因为本地模型有随机性——对每个版本跑多次取平均分，这正是真实 benchmark 里 `runs`（多次运行）的意义所在。在我们的运行中，平均分从 **0.00/5**（空 `AGENTS.md`）提升到 **5.00/5**（加入"任务纪律"后），脚本仅因分数严格提升才保留 N+1。它跑在同一套本地 Ollama + qwen3:8b 环境上，并使用一个专用 agent id，所以你自己的 Agent 不会被动到。 +它用一份确定性的、可读的 rubric（10 分：5 分内容分、5 分只存在于 `AGENTS.md` 的内部约定分），并且——因为本地模型有随机性——对每个版本跑多次取平均分，这正是真实 benchmark 里 `runs`（多次运行）的意义所在。关键在于：脚本从不自己写那套约定；是 *Agent* 从一份通过范例里诊断出缺口，并编辑自己的 `AGENTS.md`。在我们的运行中，平均分沿着 **4.6 → 6.6 → 9.8** 走过两轮自撰改进（先学到结构、再锁定常量），harness 每一轮都只因分数严格提升才保留。它跑在同一套本地 Ollama + qwen3.6:35b 环境上，并使用一个专用 agent id，所以你自己的 Agent 不会被动到。 ## 还没有 AMD GPU？来自 AMD + Fireworks 的免费云端算力