Changelog, dev startup, README, AgentHub 0.4.0, model catalog, and landing site (#7)

Branch-length batch covering tooling, the model layer, the Web App and the public
surfaces. Highlights:

- Changelog: a per-release `changelog/<version>/` tree, grouped by the surface each
  change touches, with a root CHANGELOG.md holding one line per release.
- Dev startup: `scripts/dev-prebuild.mjs` serializes the skills+core prebuild behind a
  lock and keeps `pnpm install` current; `pnpm dev` runs server+web together.
- AgentHub 0.3.3 -> 0.4.0: OmniMessage complete payloads carry one opaque `fidelity`
  object in place of item-level `signature`/`phase`, threaded verbatim through Trace,
  replay and resume; malformed classification adapted to the new error types.
- Model layer: a model is always referenced by an explicit `(provider, model_id)` pair.
  The provider is never inferred, guessed or defaulted -- both the catalog inference and
  the unique-match config resolution are gone, and CLI, SDK, server routes and
  run_subagent all require the complete pair. Catalog gains the Qwen Token Plan, Qwen
  Pay-As-You-Go and Fireworks AI gateways, plus an expanded OpenRouter group.
- Web App: catalog preset sync and per-group speed test on the Models page, positional
  slash commands, a markdown renderer, skill-library update reminders, and a vertically
  centred draft page whose upward menus size themselves to the room available.
- Public surfaces: restructured READMEs, the penguin.ooo landing site and blog, refreshed
  benchmark results for both suites, and the demo videos playing on the landing page.

Includes the fixes from a full review of the branch: 23 confirmed findings, among them a
provider-inference bug that could send one vendor's API key to another vendor's endpoint,
and an Escape handler that destroyed the composer's contents unrecoverably.

Verified on the branch head: pnpm test (1127 passing, 7 packages), pnpm typecheck and
pnpm format:check clean, Playwright e2e 14/14.
This commit is contained in:
Yaowei Zheng authored and GitHub committed 2026-07-21 17:43:31 +08:00
1 parent abf0a2f248
commit d4faee3a1e
214 files changed
+9406 -1688

No files matched your search

@@ -0,0 +1,84 @@
---
title: "Partnering with the AMD Developer Program: $50 in free Fireworks credits, wired into PenguinHarness"
date: 2026-07-20
category: news
excerpt: We are delighted to partner with the AMD AI Developer Program to bring you free Fireworks coupon codes — apply for the $50 credits with this guide, then set them up in PenguinHarness in three steps.
---
We are delighted to partner with the **AMD AI Developer Program** to bring you free Fireworks redemption codes: join the program, pass the review, and you receive a Coupon Code redeemable for **$50 in Fireworks AI Credits**. PenguinHarness ships a built-in Fireworks AI gateway group — OpenAI protocol, preset base URL, five preset models — so the credits are usable the moment they land.
> Page content, credit amounts, and expiration dates are subject to change. Please refer to the application page and approval email for the latest information.
## Step 1: Join the AMD AI Developer Program
Visit the official page: <https://developer.amd.com/ai-developer-program/>. In the **Join the AMD AI Developer Program** section:
- No AMD ADP account yet: click **Create Account** and enter your personal information;
- Already have one: click **Log In** and follow the on-screen instructions.
![The AMD AI Developer Program join page](https://github.com/user-attachments/assets/47a3055b-9a95-40a1-80c6-e3bac7a9ac49)
## Step 2: Request Cloud Credits under Member Perks
After registering and signing in:
1. Click **Member Perks** in the top navigation bar;
2. Find **Cloud Credit Options**;
3. Click **Request Cloud Credits** at the bottom of the page.
![Cloud Credit Options under Member Perks](https://github.com/user-attachments/assets/773f8cf1-d72f-4aa3-83b8-f31fc7c9ed9e)
## Step 3: Complete the application form
Fill in the requested personal information:
- Under **Product Needed**, select **Fireworks AI**;
- In the **Profile** section, provide at least one public profile for account verification: LinkedIn, GitHub, a portfolio, a company or school profile, WeChat, WeCom, and similar all work.
![Application form: select Fireworks AI under Product Needed](https://github.com/user-attachments/assets/12a49136-0956-4f29-9d47-f0e473615075)
Complete the other required fields marked with `*`, double-check your email, identity, product selection, and profile link, then submit.
## Step 4: Wait for the review
AMD verifies your account and application, usually within **2–3 business days** — actual time varies with application volume, information completeness, and holidays.
## Step 5: Receive and redeem the coupon code
Once approved, AMD emails a unique **Coupon Code** redeemable for **$50 in Fireworks AI Credits** to your application address. Keep it secure — do not share it publicly, forward it, or commit it to a repository.
![The Coupon Code in the approval email](https://github.com/user-attachments/assets/c53129c7-2c87-4510-a4e3-31f52598dc24)
Redeem it and create an API key:
1. Open <https://fireworks.ai/> and sign in;
2. Click **Redeem Promo** and enter the Coupon Code from the email to redeem the $50 credits;
3. Click **Create API Key** to generate your Fireworks API key.
![Redeeming and creating an API key in the Fireworks console](https://github.com/user-attachments/assets/051a1e69-db7f-4867-b899-89981df15142)
## Set it up in PenguinHarness
With the API key in hand, three steps:
**1. Install and launch**
```bash
curl -fsSL https://penguin.ooo/install.sh | sh
penguin web # opens http://127.0.0.1:7364 (first login: admin / penguin-2026)
```
**2. Configure a Fireworks model**
Open the Models page and find the **Fireworks AI** group, then use its bulk key button to paste the API key you just created. The group presets five models — GLM 5.2, Kimi K2.7 Code, DeepSeek V4 Pro, MiniMax M3, and DeepSeek V4 Flash — with base URLs and pricing pre-filled; set any of them as the default. You can also hit the group's speed-test button to measure real TTFT and TPS before choosing.
**3. Start working**
Head back to Chat and hand the Agent its first task — e.g. "Analyze data.csv and summarize quarterly sales".
## References
- [AMD AI Developer Program](https://developer.amd.com/ai-developer-program/)
- [Official AMD Cloud Credits application video tutorial](https://www.youtube.com/watch?v=masSW53JkTY)
- Application steps and screenshots are adapted from WhatGhost's guides ([中文](https://github.com/WhatGhost/whatghost_Notebooks/blob/main/other/AMD_AI_Developer_Program_Credits_%E7%94%B3%E8%AF%B7%E6%8C%87%E5%8D%97.md) / [English](https://github.com/WhatGhost/whatghost_Notebooks/blob/main/other/AMD_AI_Developer_Program_Credits_Application_Guide_EN.md)) — thanks to the original author
- [PenguinHarness model configuration docs](https://penguin.ooo/docs/models)
@@ -0,0 +1,84 @@
---
title: 携手 AMD 开发者计划:免费领取 $50 Fireworks 额度,直连 PenguinHarness
date: 2026-07-20
category: news
excerpt: 我们很高兴与 AMD AI Developer Program 合作,为大家带来 Fireworks 的免费兑换码——按本文申请 $50 Credits,再在 PenguinHarness 里三步用起来。
---
我们很高兴与 **AMD AI Developer Program**(AMD 开发者计划)合作,为大家带来 Fireworks 的免费兑换码:加入计划并通过审核,即可获得可兑换 **$50 Fireworks AI Credits** 的 Coupon Code。PenguinHarness 内置 Fireworks AI 网关分组——OpenAI 协议、预置 base URL 与五个模型,额度到手即刻可用。
> 页面内容、Credits 金额与有效期可能调整,请以申请时的页面及审批邮件为准。
## 第一步:加入 AMD AI Developer Program
访问官方入口:<https://developer.amd.com/ai-developer-program/>,在 **Join the AMD AI Developer Program** 区域:
- 没有 AMD ADP 账户:点击 **Create Account**,填写个人信息创建账户;
- 已有账户:点击 **Log In**,按页面提示登录。
![AMD AI Developer Program 加入页面](https://github.com/user-attachments/assets/47a3055b-9a95-40a1-80c6-e3bac7a9ac49)
## 第二步:在 Member Perks 申请 Cloud Credits
注册并登录后:
1. 点击顶部导航栏中的 **Member Perks**;
2. 找到 **Cloud Credit Options**;
3. 点击底部的 **Request Cloud Credits**。
![Member Perks 中的 Cloud Credit Options](https://github.com/user-attachments/assets/773f8cf1-d72f-4aa3-83b8-f31fc7c9ed9e)
## 第三步:填写申请表
进入申请表后填写个人信息:
- **Product Needed** 处选择 **Fireworks AI**;
- **Profile** 处提供至少一个公开资料用于账户验证:LinkedIn、GitHub、Portfolio、公司 / 学校主页、WeChat、WeCom 等均可。
![申请表:Product Needed 选择 Fireworks AI](https://github.com/user-attachments/assets/12a49136-0956-4f29-9d47-f0e473615075)
填完页面中其他带 `*` 的必填项,检查邮箱、身份、产品选项与公开资料链接无误后提交。
## 第四步:等待审核
AMD 会验证账户与申请资料,通常需要 **2–3 个工作日**;实际时间可能因申请量、资料完整度或节假日而变化。
## 第五步:收到兑换码并兑换
审核通过后,AMD 会向申请邮箱发送包含唯一 **Coupon Code** 的邮件,可兑换 **$50 Fireworks AI Credits**。请妥善保存,不要公开、转发或提交到代码仓库。
![审批邮件中的 Coupon Code](https://github.com/user-attachments/assets/c53129c7-2c87-4510-a4e3-31f52598dc24)
兑换与创建 API key:
1. 打开 <https://fireworks.ai/> 并登录;
2. 点击 **Redeem Promo**,输入邮件中的 Coupon Code,兑换 $50 Credits;
3. 点击 **Create API Key**,生成 Fireworks API key。
![在 Fireworks 控制台兑换并创建 API key](https://github.com/user-attachments/assets/051a1e69-db7f-4867-b899-89981df15142)
## 在 PenguinHarness 中用起来
拿到 API key 后,三步接入:
**1. 安装并启动**
```bash
curl -fsSL https://penguin.ooo/install.sh | sh
penguin web # 打开 http://127.0.0.1:7364(首次登录:admin / penguin-2026)
```
**2. 配置 Fireworks 模型**
进入「模型仓库」页,找到 **Fireworks AI** 分组,点击「统一配置 key」粘贴刚创建的 API key。分组预置了五个模型——GLM 5.2、Kimi K2.7 Code、DeepSeek V4 Pro、MiniMax M3、DeepSeek V4 Flash——base URL 与价格已填好,任选一个设为默认即可;也可以点组头的「测速」,实测各模型的 TTFT 与 TPS 再决定。
**3. 开始使用**
回到对话页,把第一个任务交给 Agent——例如「分析 data.csv,输出各季度销售额汇总」。
## 参考链接
- [AMD AI Developer Program](https://developer.amd.com/ai-developer-program/)
- [AMD 官方 Cloud Credits 申请视频演示](https://www.youtube.com/watch?v=masSW53JkTY)
- 申请步骤与截图整理自 WhatGhost 的申请指南([中文](https://github.com/WhatGhost/whatghost_Notebooks/blob/main/other/AMD_AI_Developer_Program_Credits_%E7%94%B3%E8%AF%B7%E6%8C%87%E5%8D%97.md) / [English](https://github.com/WhatGhost/whatghost_Notebooks/blob/main/other/AMD_AI_Developer_Program_Credits_Application_Guide_EN.md)),感谢原作者
- [PenguinHarness 模型配置文档](https://penguin.ooo/docs/models)
@@ -2,70 +2,114 @@
title: "Introducing PenguinHarness: agents that build agents"
date: 2026-07-17
category: news
excerpt: The first open-source harness with recursive self-improvement is here — lightweight, efficient and secure infrastructure covering everything from automatic agent construction to continuous self-evolution.
excerpt: We proved agents can self-evolve in our GDPevo Benchmark — now we are bringing that capability to everyone. The first open-source harness with recursive self-improvement covers everything from one-sentence agent construction to continuous self-evolution.
---
Today we are releasing **PenguinHarness** — an open-source harness built for constructing and evolving agents. Its purpose fits in one line:
Today we are releasing **PenguinHarness** — an open-source harness built for constructing and evolving agents: a zero-code Harness CLI and Web UI, connected to 1000+ models. The story it tells fits in one line:
> Efficient Self-Improving Harness for Everyone.
> With LangChain, you build agents by hand — at 1× speed. With PenguinHarness, agents build agents — at 100×.
## From GDPevo to PenguinHarness: why we built this
Before PenguinHarness, our team published the [GDPevo Benchmark](https://prism-shadow.github.io/GDPevo/). In GDPevo we systematically verified one thing: **agents can self-evolve** — an Agent can score its own performance, find where the points were lost, rewrite its own prompts and Skills, and climb version after version.
With the capability proven, the question became: how does everyone get to use it? Self-evolution should not stay a curve in a paper — it should be infrastructure that works out of the box on every developer's desk. **Bringing an efficient self-improving harness to everyone is why we built PenguinHarness** — and it is right there in the name: Efficient Self-Improving Harness for Everyone.
## Why PenguinHarness
Over the past year the way agent applications are built has been converging fast: what really decides quality is not a heavyweight framework but a simple, reliable, observable harness. PenguinHarness is rebuilt from the ground up — no dependency on any agent framework, a fully open-source self-developed kernel — and it brings three things to the open-source world first:
Three reasons, in deliberate order — from task quality, to how agents get built, to how they keep improving.
- **Simplest Is the Best**: a deliberately minimal toolset over clean low-level interfaces — fewer tool calls, fewer Tokens, complex tasks done efficiently.
- **Harness for Building Agents**: with the PenguinHarness SDK, an Agent builds complete Agent applications for you, autonomously, from scratch.
- **Harness for Recursive Self-Improvement**: with PenguinHarness Skills, an Agent evaluates and optimizes itself, improving recursively over time.
### 1. Better on complex tasks, at lower cost
For the latter two, PenguinHarness is the first open-source implementation in the industry.
A deliberately minimal toolset over clean low-level interfaces: fewer tool calls, fewer Tokens, deeply tuned for open models like DeepSeek. Each harness runs the model it is normally paired with — the comparison is between the products as people actually use them — head-to-head on two suites:
## Same model, equal or better quality, lower cost
![Benchmark: PenguinHarness leads the data-analysis suite and ties OpenAI Codex on coding, at a small fraction of both rivals' cost](/blog-assets/benchmark-light.svg)
All runs use the same DeepSeek V4 Pro model, head-to-head against Claude Code and OpenAI Codex on two suites (per-run means below).
Complex data analysis (15 tasks, single run; PenguinHarness and Codex at thinking xhigh, Claude Code at max):
Complex data analysis (15 tasks, single run):
| Framework | Model | Accuracy (%) | Tokens (M) | Cost ($) |
| -------------- | --------------- | -----------: | ---------: | -------: |
| PenguinHarness | DeepSeek V4 Pro | 66.67 | 18.04 | 0.55 |
| Claude Code | Claude Opus 4.8 | 53.33 | 22.20 | 38.48 |
| OpenAI Codex | GPT-5.5 | 53.33 | 13.72 | 19.41 |
| Framework | Model | Accuracy (%) | Tokens (M) | Cost ($) |
| --- | --- | ---: | ---: | ---: |
| PenguinHarness | DeepSeek V4 Pro | 66.7 | 18.04 | 0.552 |
| Claude Code | DeepSeek V4 Pro | 66.7 | 21.17 | 0.641 |
| OpenAI Codex | DeepSeek V4 Pro | 46.7 | 13.36 | 0.427 |
Coding tasks (40 tasks × 2 runs; accuracy is over all 80 outcomes):
Coding tasks (40 tasks × 2 runs averaged, thinking high, 30 min per-case timeout, CNY pricing converted at $1 = ¥7):
| Framework | Model | Accuracy (%) | Tokens (M) | Cost ($) |
| -------------- | --------------- | -----------: | ---------: | -------: |
| PenguinHarness | DeepSeek V4 Pro | 71.25 | 200.00 | 3.81 |
| Claude Code | Claude Opus 4.8 | 86.25 | 151.61 | 146.97 |
| OpenAI Codex | GPT-5.5 | 71.25 | 251.20 | 220.08 |
| Framework | Model | Accuracy (%) | Tokens (M) | Cost ($) |
| --- | --- | ---: | ---: | ---: |
| PenguinHarness | DeepSeek V4 Pro | 50.00 | 2.10 | 0.041 |
| Claude Code | DeepSeek V4 Pro | 48.75 | 2.00 | 0.048 |
| OpenAI Codex | DeepSeek V4 Pro | 42.50 | 2.65 | 0.043 |
Tokens and cost are suite totals, not per-run means. On data analysis we take the highest accuracy of the three — 66.67% against 53.33% for both — while spending 1/35 of what Codex spent and 1/70 of Claude Code's. On coding we tie Codex at 71.25% and trail Claude Code's 86.25%, but the whole suite cost us $3.81 against their $220.08 and $146.97: comparable work, one to two orders of magnitude apart on the bill.
On the data-analysis suite PenguinHarness ties Claude Code on accuracy and clearly beats OpenAI Codex while using 14.8% fewer Tokens at 13.8% lower cost; on the coding suite it scores highest of the three at the lowest per-run cost.
### 2. One sentence, and an Agent builds your Agent app
Type one sentence, and an Agent builds the complete Agent application for you — scaffold, code, and run instructions, end to end:
```text
Collect the docs from https://github.com/ericbuess/claude-code-docs and build a RAG app that answers Claude Code questions as a configuration expert, citing its sources.
```
And this is the finished product — a docs expert with retrieval, cited sources that link to the original files, and example questions built in:
![The generated RAG app: a Claude Code docs expert answering with cited, clickable sources and example questions](/blog-assets/rag-app-en-light.webp)
**And generating this entire RAG app burned just $0.02 (¥0.2) of tokens — on DeepSeek V4 Pro.**
### 3. Self-evolution: it gets stronger with use
With PenguinHarness Skills, an Agent evaluates and optimizes itself: the Optimizer orchestrates multiple Evaluators to score in parallel, uses the scores and run traces to find where points were lost, and upgrades the Agent from version N to N+1 — with a snapshot before every round, and every request replayable in the Trace view. A self-evolution demo video is coming soon.
## Evolution within bounds, security first
The biggest worry about self-improvement is losing control. PenguinHarness answers with a contract — CONTRACT.md:
The biggest worry about self-evolution is losing control. PenguinHarness answers with a contract (CONTRACT.md):
- Evolution is strictly confined to Workspace and Skills; the harness core security boundary is never modified.
- Tool calls run only after approval, and every approval is audited.
- Risky changes snapshot first — every step of evolution can be rolled back.
- Fully open source and locally deployable: data never leaves your machine, meeting enterprise security requirements.
- Evolution is strictly confined to Workspace and Skills — the harness core security boundary is never modified;
- Tool calls require approval first, and every approval leaves an audit record;
- Risky changes are preceded by version snapshots, so any round of evolution can be rolled back;
- Fully open source and locally deployed — data never leaves your machine, meeting enterprise data-security requirements.
## Get started now
## Supported models
Install with one command (Linux / macOS, x64 / arm64, bundled Node runtime):
| Model | Providers |
| ---------------- | -------------------------------------------------------------------------------- |
| DeepSeek V4 | DeepSeek, OpenRouter, Fireworks AI, SiliconFlow, Qwen Token Plan |
| Kimi K3 | Moonshot AI, OpenRouter, Qwen Pay-As-You-Go |
| GLM 5.2 | Z.AI, OpenRouter, Fireworks AI, SiliconFlow, Qwen Token Plan, Qwen Pay-As-You-Go |
| Hunyuan 3 | OpenRouter |
| Qwen 3.8 Max | Qwen Token Plan (preview) |
| GPT 5.5 | OpenAI, OpenRouter |
| Gemini 3.5 Flash | Google Gemini, OpenRouter |
| Claude Opus 4.8 | Anthropic, OpenRouter |
Any OpenAI-protocol endpoint is supported: pick a preset above, or point a custom endpoint at any of the 1000+ online and local models.
## How to use it
Install with one command (Linux / macOS, x64 / arm64, bundled Node runtime), then launch the Web UI:
```bash
curl -fsSL https://github.com/Prism-Shadow/penguin-harness/releases/latest/download/install.sh | sh
curl -fsSL https://penguin.ooo/install.sh | sh
penguin web # opens http://127.0.0.1:7364 (first login: admin / penguin-2026)
```
Configure a model (DeepSeek as an example), run your first task, or open the desktop-grade web interface with `penguin web`:
Open the Models page, paste an API key under the DeepSeek or OpenRouter group and set it as default; then head back to Chat and hand the Agent its first task — e.g. "Analyze data.csv and summarize quarterly sales".
```bash
penguin config model add --model-id deepseek-v4-pro --api-key sk-your-key --set-default
penguin run --approve allow-all --message "Analyze data.csv and summarize quarterly sales"
penguin web
```
## What's next
PenguinHarness supports 1000+ online and local models and multi-agent collaborative evolution, and runs on as little as a single CPU. Through continuous evolution it makes complex AI development ever simpler — a more efficient, more reliable, lower-hallucination and lower-cost Agent productivity engine.
- Public release of the benchmark suite;
- A desktop app;
- Windows support;
- More to come.
Follow us on [GitHub](https://github.com/Prism-Shadow/penguin-harness) and open your first issue.
## Join the community and build with us
A self-improving harness needs a community that improves with it. Come discuss, request, and contribute — your first Issue is the best way to start:
- [Discord](https://discord.gg/eFHKqqcU3D): chat with us and other developers in real time;
- [X (Twitter)](https://x.com/code_hiyouga): follow the latest updates;
- [WeChat group](https://github.com/Prism-Shadow/penguin-harness-community/blob/main/wechat/group.jpg): Chinese community discussions;
- [GitHub](https://github.com/Prism-Shadow/penguin-harness): stars, Issues, and PRs all welcome.
Self-evolving agent infrastructure, for everyone — starting today.
@@ -2,44 +2,64 @@
title: PenguinHarness 正式发布:让 Agent 为你构建 Agent
date: 2026-07-17
category: news
excerpt: 首个支持递归自我进化的开源 Harness 正式发布——以轻量、高效、安全的方式,提供从 Agent 自动构建到持续自我进化的完整基础设施。
excerpt: 我们在 GDPevo Benchmark 中验证了 Agent 自我进化的能力,现在把它带给所有人——首个支持递归自我进化的开源 Harness 正式发布,从一句话构建 Agent 到持续自我进化,一套基础设施全部覆盖。
---
今天,我们正式发布 **PenguinHarness**——一个为构建与进化 Agent 而生的开源 Harness。它的主旨只有一句话:
今天,我们正式发布 **PenguinHarness**——一个为构建与进化 Agent 而生的开源 Harness:零代码的 Harness CLI 与 Web UI,连接 1000+ 模型。它要讲的故事只有一句话:
> Efficient Self-Improving Harness for Everyone.
> 使用 LangChain,以 1 倍速度人工构建 Agent;使用 PenguinHarness,以 100 倍速度用 Agent 构建 Agent。
## 从 GDPevo 到 PenguinHarness:我们的初心
在 PenguinHarness 之前,我们团队发布了 [GDPevo Benchmark](https://prism-shadow.github.io/GDPevo/)。在 GDPevo 中,我们系统性地验证了一件事:**Agent 可以自我进化**——让 Agent 评估自己的表现、定位失分原因、改写自己的提示词与技能,分数随版本一路上升。
能力验证了,问题就变成了:怎么让每个人都用上它?自我进化不该只是论文里的曲线,而应该是每个开发者桌面上开箱即用的基础设施。**让每个人都能使用 Efficient Self-Improving Harness,这就是我们构建 PenguinHarness 的初心**——它也因此得名:Efficient Self-Improving Harness for Everyone.
## 为什么是 PenguinHarness
过去一年里,Agent 应用的开发范式在快速收敛:真正决定效果的不是庞大的框架,而是一个简洁、可靠、可观测的 Harness。PenguinHarness 从底层重构,不依赖任何 Agent 框架,开源自研 Harness 内核,并率先把三件事带入开源世界:
三个递进的理由——从任务效果,到构建方式,再到进化能力。
- **Simplest Is the Best**:坚持最小化工具集与简洁的底层接口,以更少的工具调用与 Token 消耗,高效完成复杂任务。
- **Harness for Building Agents**:通过 PenguinHarness SDK,让 Agent 从零自主完成 Agent 应用的构建。
- **Harness for Recursive Self-Improvement**:通过 PenguinHarness Skills,Agent 以自我评估与自我优化实现递归式自我提升。
### 1. 复杂任务表现更好,成本更低
后两项能力,PenguinHarness 是业内首个开源实现。
刻意精简的工具集配合干净的底层接口:更少的工具调用、更少的 Token,对 DeepSeek 等开放模型深度适配。每个产品搭配它常用的模型——比的是大家实际会怎么用——在两套题库上正面对比:
## 同一模型,同级效果,更低消耗
![Benchmark:PenguinHarness 在数据分析题库准确率最高、编程题库与 OpenAI Codex 持平,成本仅为两者的零头](/blog-assets/benchmark-light.svg)
全部使用同一 DeepSeek V4 Pro 模型,与 Claude Code、OpenAI Codex 在两套题库上正面对比(表中为单次运行均值)。
复杂数据分析(15 题,单次运行;PenguinHarness 与 Codex 为 thinking xhigh,Claude Code 为 max):
复杂数据分析(15 题,单次运行):
| 实验框架 | 模型名称 | 准确率(%) | Token 用量(M) | 成本($) |
| -------------- | --------------- | ----------: | --------------: | --------: |
| PenguinHarness | DeepSeek V4 Pro | 66.67 | 18.04 | 0.55 |
| Claude Code | Claude Opus 4.8 | 53.33 | 22.20 | 38.48 |
| OpenAI Codex | GPT-5.5 | 53.33 | 13.72 | 19.41 |
| 实验框架 | 模型名称 | 准确率(%) | Token 用量(M) | 成本($) |
| --- | --- | ---: | ---: | ---: |
| PenguinHarness | DeepSeek V4 Pro | 66.7 | 18.04 | 0.552 |
| Claude Code | DeepSeek V4 Pro | 66.7 | 21.17 | 0.641 |
| OpenAI Codex | DeepSeek V4 Pro | 46.7 | 13.36 | 0.427 |
代码任务(40 题 × 2 次运行,准确率取全部 80 次结果):
代码任务(40 题 × 2 runs 取均值,thinking high、单题 30 分钟超时,人民币计价按 $1 = ¥7 折算):
| 实验框架 | 模型名称 | 准确率(%) | Token 用量(M) | 成本($) |
| -------------- | --------------- | ----------: | --------------: | --------: |
| PenguinHarness | DeepSeek V4 Pro | 71.25 | 200.00 | 3.81 |
| Claude Code | Claude Opus 4.8 | 86.25 | 151.61 | 146.97 |
| OpenAI Codex | GPT-5.5 | 71.25 | 251.20 | 220.08 |
| 实验框架 | 模型名称 | 准确率(%) | Token 用量(M) | 成本($) |
| --- | --- | ---: | ---: | ---: |
| PenguinHarness | DeepSeek V4 Pro | 50.00 | 2.10 | 0.041 |
| Claude Code | DeepSeek V4 Pro | 48.75 | 2.00 | 0.048 |
| OpenAI Codex | DeepSeek V4 Pro | 42.50 | 2.65 | 0.043 |
Token 与成本均为全套题目合计,不是单次均值。数据分析套件我们准确率最高——66.67% 对另两者的 53.33%——花的钱是 Codex 的 1/35、Claude Code 的 1/70;代码套件与 Codex 同为 71.25%、低于 Claude Code 的 86.25%,但整套题目只花了 $3.81,对方分别是 $220.08 与 $146.97:活干得差不多,账单差出一到两个数量级。
数据分析套件与 Claude Code 准确率持平、显著超过 OpenAI Codex,同时 Token 消耗少 14.8%、成本低 13.8%;代码套件三者中准确率最高、单次成本最低。
### 2. 一句话,让 Agent 构建 Agent 应用
输入一句话,Agent 为你构建完整的 Agent 应用——脚手架、代码、运行说明,一步到位:
```text
收集 https://github.com/ericbuess/claude-code-docs 的文档,做一个化身 Claude Code 配置专家、回答带来源引用的 RAG 问答应用。
```
这是做出来的成品——一个文档专家:检索增强、引用可点击直达原文、内置示例问题:
![生成的 RAG 应用成品:Claude Code 配置专家,回答带可点击的来源引用与示例问题](/blog-assets/rag-app-zh-light.webp)
**而生成整个 RAG 应用,仅消耗了 0.2 元($0.02)的 token——使用 DeepSeek V4 Pro 模型。**
### 3. 自进化,越用越强
借助 PenguinHarness 技能库,Agent 自己评估、自己优化:Optimizer 组织多个 Evaluator 并行打分,依据分数与运行轨迹定位失分原因,把 Agent 从版本 N 优化到版本 N+1——每轮之前自动快照,每个请求都可在轨迹观测中回放。自进化演示视频即将上线。
## 进化有界,安全先行
@@ -50,22 +70,46 @@ excerpt: 首个支持递归自我进化的开源 Harness 正式发布——以
- 风险修改之前先留版本快照,任何一次进化都可回退;
- 完全开源、本地部署,数据不出域,满足企业级数据安全。
## 现在就可以开始
## 支持的模型
一行命令安装(Linux / macOS,x64 / arm64,内嵌 Node 运行时):
| 模型 | 可用供应商 |
| ---------------- | -------------------------------------------------------------------------------- |
| DeepSeek V4 | DeepSeek, OpenRouter, Fireworks AI, SiliconFlow, Qwen Token Plan |
| Kimi K3 | Moonshot AI, OpenRouter, Qwen Pay-As-You-Go |
| GLM 5.2 | Z.AI, OpenRouter, Fireworks AI, SiliconFlow, Qwen Token Plan, Qwen Pay-As-You-Go |
| Hunyuan 3 | OpenRouter |
| Qwen 3.8 Max | Qwen Token Plan(预览) |
| GPT 5.5 | OpenAI, OpenRouter |
| Gemini 3.5 Flash | Google Gemini, OpenRouter |
| Claude Opus 4.8 | Anthropic, OpenRouter |
只要是 OpenAI 协议的端点都可以接入:从上表选择预置,或用自定义端点连接 1000+ 在线与本地模型。
## 如何使用
一行命令安装(Linux / macOS,x64 / arm64,内嵌 Node 运行时),然后启动 Web 界面:
```bash
curl -fsSL https://github.com/Prism-Shadow/penguin-harness/releases/latest/download/install.sh | sh
curl -fsSL https://penguin.ooo/install.sh | sh
penguin web # 打开 http://127.0.0.1:7364(首次登录:admin / penguin-2026)
```
配置模型(以 DeepSeek 为例)后即可运行第一个任务,或用 `penguin web` 打开桌面级 Web 界面:
进入「模型仓库」页,在 DeepSeek 或 OpenRouter 分组里粘贴 API key 并设为默认;回到对话页,把第一个任务交给 Agent——例如「分析 data.csv,输出各季度销售额汇总」。
```bash
penguin config model add --model-id deepseek-v4-pro --api-key sk-your-key --set-default
penguin run --approve allow-all --message "分析 data.csv,输出各季度销售额汇总"
penguin web
```
## 发展计划
PenguinHarness 支持 1000 多种在线与本地模型、多智能体协作进化,最低单 CPU 即可运行。通过不断进化,它会让复杂的 AI 开发越来越简单——为你提供更高效、更可靠、更低幻觉、更低成本的 Agent 生产力引擎。
- Benchmark 套件正式发布;
- 推出桌面端(Desktop)应用;
- 支持 Windows 系统;
- 更多规划,敬请期待。
欢迎在 [GitHub](https://github.com/Prism-Shadow/penguin-harness) 上关注我们,提出你的第一个 Issue。
## 加入社区,一起共建
自我进化的 Harness,也需要一个不断进化的社区。欢迎加入讨论、提出需求、贡献代码——你的第一个 Issue 就是最好的开始:
- [Discord](https://discord.gg/eFHKqqcU3D):与我们和其他开发者实时交流;
- [X(Twitter)](https://x.com/code_hiyouga):关注最新动态;
- [微信群](https://github.com/Prism-Shadow/penguin-harness-community/blob/main/wechat/group.jpg):中文社区讨论;
- [GitHub](https://github.com/Prism-Shadow/penguin-harness):Star、Issue 与 PR 都欢迎。
让每个人都用上会自我进化的 Agent 基础设施——从今天开始。