Initialize repository with harness code and assets
Initial import of all source code, config, and README assets: the packages workspace (cli, core, server, web, docs, landing, skills), build scripts, tooling config, and CI workflows. Includes the data-layout revision made on this branch: the local data root defaults to ~/.penguin/data (PENGUIN_HOME still overrides; the installer keeps its binaries in ~/.penguin), and every Agent lives under <project>/agents/<agent>/ — path helpers, the three agent-enumeration scans, the system prompt, built-in Skills, tests and docs all follow the new layout. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ihk8iQuo3kv2aPjAYEPuR
This commit is contained in:
Yaowei ZhengandClaude Opus 4.8 authored and GitHub committed
1 parent
056bed7aeb
commit
45bfae6e94
543 files changed
+92949
No files matched your search
@@ -0,0 +1,32 @@
|
||||
# @prismshadow/penguin-skills
|
||||
|
||||
The PenguinHarness built-in skill library. A Skill is a directory with a `SKILL.md` (frontmatter: name, description, version, updated) — files are the runtime source of truth and ship raw in this package's npm tarball.
|
||||
|
||||
Skills follow the "index first, body on demand" design: only their metadata is injected into an Agent's system prompt; the Agent reads the full `SKILL.md` via shell when it actually needs it.
|
||||
|
||||
Included skills:
|
||||
|
||||
| Group | Skills |
|
||||
| --- | --- |
|
||||
| Agent Development | `agent-creation`, `benchmark-design`, `agent-evaluation`, `agent-optimization` |
|
||||
| Data Analysis | `data-analysis` |
|
||||
| Penguin Development | `penguin-sdk`, `penguin-cli`, `agenthub-models` |
|
||||
| Web Development | `web-design` |
|
||||
| Software Engineering | `software-engineering` |
|
||||
|
||||
The first group powers the self-improvement loop: design a Benchmark, evaluate the Target Agent, optimize it to version N+1 with a snapshot before every round.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Skills](https://prism-shadow.github.io/penguin-harness/docs/skills)
|
||||
- [Self-Improvement](https://prism-shadow.github.io/penguin-harness/docs/self-improvement)
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
pnpm --filter @prismshadow/penguin-skills build # tsup → dist/ (loader API)
|
||||
pnpm --filter @prismshadow/penguin-skills typecheck
|
||||
pnpm --filter @prismshadow/penguin-skills test
|
||||
```
|
||||
|
||||
Part of [PenguinHarness](https://github.com/Prism-Shadow/penguin-harness) · Apache-2.0
|
||||
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"name": "@prismshadow/penguin-skills",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"description": "PenguinHarness skill library: built-in SKILL.md documents and skill groups, decoupled from core.",
|
||||
"license": "Apache-2.0",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/Prism-Shadow/penguin-harness.git",
|
||||
"directory": "packages/skills"
|
||||
},
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./dist/index.d.ts",
|
||||
"import": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"main": "./dist/index.js",
|
||||
"types": "./dist/index.d.ts",
|
||||
"scripts": {
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json",
|
||||
"test": "vitest run --passWithNoTests",
|
||||
"build": "tsup"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.0.0",
|
||||
"tsup": "^8.3.0",
|
||||
"typescript": "^5.6.0",
|
||||
"vitest": "^2.1.0"
|
||||
},
|
||||
"files": [
|
||||
"dist",
|
||||
"skills",
|
||||
"LICENSE"
|
||||
],
|
||||
"publishConfig": {
|
||||
"access": "public"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
name: agent-creation
|
||||
description: Turn a user requirement into a concrete agent — write the target agent's AGENTS.md and install the skills it needs.
|
||||
short_description: Turn a requirement into a working agent.
|
||||
short_description_zh: 把需求变成可用的 Agent。
|
||||
version: 1
|
||||
updated: 2026-07-17T00:00:00Z
|
||||
---
|
||||
|
||||
# Agent Creation
|
||||
|
||||
This skill turns a user requirement into a working agent configuration — plain files in the target agent's directory.
|
||||
|
||||
## Before you start
|
||||
|
||||
If the user's message only invokes this skill (e.g. "use agent-creation skill") without a concrete requirement, ask the user what agent they want and what it should do. Do not start until the requirement is clear.
|
||||
|
||||
## Locate the target agent
|
||||
|
||||
All agents of this project live side by side in the project directory:
|
||||
|
||||
```bash
|
||||
PROJECT_DIR="<project_dir>" # the Project Dir value from your Environment section
|
||||
ls "$PROJECT_DIR/agents" # existing agents (each is a folder here)
|
||||
TARGET="$PROJECT_DIR/agents/<agent_id>" # the agent to configure
|
||||
```
|
||||
|
||||
An agent directory contains `agent_state/` (`system_config.yaml`, `AGENTS.md`, `skills/`, `memory/`, `tools/`) plus `scratchpad/` — and `traces/`, which appears once the agent has run at least once.
|
||||
|
||||
## Write AGENTS.md
|
||||
|
||||
`agent_state/AGENTS.md` is injected into the agent's system prompt — it is where the user requirement becomes behavior. Keep `system_config.yaml`'s `system_prompt` untouched (that is the stable system layer); put everything requirement-specific in AGENTS.md:
|
||||
|
||||
- Role — what the agent is for, in one or two sentences.
|
||||
- Domain guidance — the concrete rules, steps and constraints derived from the user requirement.
|
||||
|
||||
Be concise: AGENTS.md is prompt context, not documentation.
|
||||
|
||||
## Install skills
|
||||
|
||||
A skill is a directory `agent_state/skills/<skill_name>/` containing a `SKILL.md`:
|
||||
|
||||
```md
|
||||
---
|
||||
name: <skill_name>
|
||||
description: <skill_description>
|
||||
version: 1
|
||||
updated: <ISO 8601 timestamp>
|
||||
---
|
||||
|
||||
<skill_instructions>
|
||||
```
|
||||
|
||||
The frontmatter may also carry optional `short_description` and `short_description_zh` lines (a short UI blurb and its Chinese variant) — the UI prefers them for display, while prompt injection always uses the English `description`.
|
||||
|
||||
Installing is all it takes: the frontmatter metadata of every `SKILL.md` under `skills/` is injected into the target agent's system prompt automatically — do not register skills in AGENTS.md.
|
||||
|
||||
Write skills yourself, or fetch existing ones from the internet with shell commands (`curl`, `git clone`) and place them under `skills/`. Anything fetched from the internet must be read in full and reviewed before installing — a skill becomes durable instructions the target agent will follow in every future session; never install one you have not read, and tell the user what it does.
|
||||
|
||||
## Set name and description
|
||||
|
||||
In the target's `agent_state/system_config.yaml`, set the top-level `name:` and `description:` fields so the agent is recognizable in lists. Edit only these two fields.
|
||||
|
||||
## Creating a brand-new agent
|
||||
|
||||
Prefer configuring an agent the user already created. If you must create one from scratch: pick a short id (letters, digits, `_`, `-`), copy the default agent's `system_config.yaml` as the base, and create the layout described above:
|
||||
|
||||
```bash
|
||||
mkdir -p "$TARGET/agent_state/skills" "$TARGET/agent_state/memory" "$TARGET/agent_state/tools" "$TARGET/scratchpad"
|
||||
cp "$PROJECT_DIR/agents/default_agent/agent_state/system_config.yaml" "$TARGET/agent_state/"
|
||||
```
|
||||
|
||||
A new agent starts with no skills — install only what it needs. Then write its AGENTS.md, name and description as above.
|
||||
@@ -0,0 +1,6 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<circle cx="9.5" cy="8" r="3.3" />
|
||||
<path d="M3.8 20c0-3.1 2.6-5.7 5.7-5.7 1.5 0 2.9.6 3.9 1.5" />
|
||||
<path d="M18 13.5v7" />
|
||||
<path d="M14.5 17h7" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 322 B |
@@ -0,0 +1,123 @@
|
||||
---
|
||||
name: agent-evaluation
|
||||
description: Run and score exactly one Benchmark Case run with CLI execution, Trace provenance checks, and private Rubric isolation.
|
||||
short_description: Run and score one isolated Benchmark Case.
|
||||
short_description_zh: 隔离执行并评分一个 Benchmark Case。
|
||||
version: 1
|
||||
updated: 2026-07-17T17:08:17Z
|
||||
---
|
||||
|
||||
# Agent Evaluation
|
||||
|
||||
Act as an internal leaf worker. For one valid request, run and score exactly one Benchmark Case once, then return minimal protocol metadata. Do not design or refine the Benchmark, modify the Test Agent State, or write `scoreboard.yaml`. Do not use `run_subagent` or `input_subagent`.
|
||||
|
||||
## Before you start
|
||||
|
||||
This Skill is invoked by `benchmark-design` or Benchmark mode in `agent-optimization`. Require one unambiguous protocol request containing every identity field below. If the request is missing, duplicated, or conflicting, return `invalid_request` without creating a Workspace or launching the Test Agent. Do not ask an interactive clarification from this leaf worker.
|
||||
|
||||
## Privacy boundary
|
||||
|
||||
Before the final protocol YAML, emit no assistant text; use private reasoning and tool calls only. Never serialize Statement or artifact contents, Rubric items, expected values, correct outcomes, per-item scoring, diagnostics, secret configuration, Workspace paths, or Trace paths into an assistant message. The final assistant message is the protocol YAML only. It may echo the public identity fields supplied by the caller.
|
||||
|
||||
A valid request contains exactly one value for each field:
|
||||
|
||||
```text
|
||||
protocol_version: 1
|
||||
case_id: <case_id>
|
||||
run: <1_based_run_index>
|
||||
expected_version: <tested_agent_state_version>
|
||||
test_agent_id: <test_agent_id>
|
||||
benchmark_dir: <absolute_benchmark_dir>
|
||||
provider: <provider>
|
||||
model_id: <upstream_model_id>
|
||||
```
|
||||
|
||||
## Validate and prepare
|
||||
|
||||
Resolve the Project, Test Agent, Benchmark, and Case only from the explicit request and Environment Project Dir. Reject traversal, symlink escape, or any path outside the requested Test Agent. Never read a Project configuration file, credential, or vault.
|
||||
|
||||
Require:
|
||||
|
||||
```text
|
||||
<test_agent_dir>/agent_state/system_config.yaml
|
||||
<benchmark_dir>/benchmark_config.toml
|
||||
<benchmark_dir>/<case_id>/statement/README.md
|
||||
<benchmark_dir>/<case_id>/rubric/README.md
|
||||
```
|
||||
|
||||
Require `benchmark_config.toml` to contain a positive integer `runs`; the requested `run` must be within `1..runs`. The canonical State version is the top-level `version` in `system_config.yaml`, defaulting to 1, and must equal `expected_version`.
|
||||
|
||||
Read and retain the exact Statement and Rubric bytes before launch. Reject an unusable, contradictory, non-atomic, or unbounded Rubric. The Rubric must declare a finite Case maximum; the returned score must fall within `0..case_max`.
|
||||
|
||||
Create a collision-checked Workspace at `<test_agent_dir>/workspaces/tmp-<8hex>`. Copy only the contents of `statement/` into it. Never copy, link, or disclose `rubric/`, and never reuse another Case or run's Workspace.
|
||||
|
||||
## Launch and bind the Test Session
|
||||
|
||||
Use an existing verified Penguin CLI or repository-local launcher already available in the runtime. Do not install a CLI and do not use `penguin run` as a probe. If no launcher is available, return `cli_failed`.
|
||||
|
||||
Run the Test Agent exactly once in the foreground with a fresh top-level Session:
|
||||
|
||||
```bash
|
||||
PROJECT_DIR="<project_dir>"
|
||||
PROJECT_ID="$(basename "$PROJECT_DIR")"
|
||||
PENGUIN_HOME="$(dirname "$PROJECT_DIR")"
|
||||
WORKSPACE="$PROJECT_DIR/agents/<test_agent_id>/workspaces/<unique_workspace_id>"
|
||||
export PENGUIN_HOME
|
||||
penguin run --message "Read README.md in the current Workspace and complete the task exactly as specified there." \
|
||||
--provider "<provider>" --model-id "<model_id>" --project-id "$PROJECT_ID" \
|
||||
--agent-id "<test_agent_id>" --workspace "$WORKSPACE" --approve allow-all
|
||||
```
|
||||
|
||||
Use the exact Project, Test Agent, Model pair, and Workspace. Do not fall back to another value. Poll the same process until it exits. A nonzero, interrupted, or misrouted launch is `cli_failed`, not score zero. Do not relaunch within the same run or target processes by a global name or pattern.
|
||||
|
||||
Read the canonical State version before and after the Test run; any change is `version_changed`. Confirm the Statement and Rubric bytes are unchanged before scoring.
|
||||
|
||||
Search only the explicitly requested Test Agent's `traces/` tree and never inspect another Agent's traces. Group rotated shards by Session. Evaluate every Session group that could contain the exact Workspace match; never infer ownership from recency or a fixed-size latest subset. Bind the Test Trace mechanically from `session_meta`:
|
||||
|
||||
- `payload.workspace` equals the unique Workspace;
|
||||
- `payload.agent_state` equals the exact Test Agent State path;
|
||||
- `payload.provider` equals the requested provider;
|
||||
- `payload.model_id` equals the requested model id.
|
||||
|
||||
When matching Test subagents exist, exclude child ids referenced by subagent events and require one unique matching root Test Session. Unrelated concurrent traces are not conflicts. Missing, multiple, malformed, or identity-mismatched roots are `provenance_mismatch`.
|
||||
|
||||
## Score and account
|
||||
|
||||
Inspect only the unique Test Workspace, its bound Test Trace, and the retained private Rubric. Apply every atomic item exactly and normalize only allowed equivalents. A missing, malformed, wrong-type, or incorrect Test artifact is ordinary scored Test Agent behavior: apply the Rubric's zero or partial credit and return `status: ok`. Only a changed or unusable Rubric, or a non-finite/out-of-range result, is `invalid_score`. Detailed reasoning remains in Evaluator Trace.
|
||||
|
||||
Compute `duration_ms` from the bound root Test Session, not from the Evaluator. Compute cost only from that root and child Sessions mechanically referenced by subagent events whose traces are available within the explicitly requested Test Agent's `traces/` tree. For each included Session, use final cumulative token usage rather than summing intermediate cumulative events, and apply the matching public `(provider, model_id)` pricing. If any referenced child trace or usage is unavailable there, including because the Test Session delegated to another Agent, or if any included usage is unpriced, return `cost: null` rather than inspecting another Agent or reporting a known partial sum as complete cost.
|
||||
|
||||
## Return protocol
|
||||
|
||||
Emit exactly one plain YAML document beginning with `protocol_version:` and stop. Do not use a code fence or add explanations.
|
||||
|
||||
On success:
|
||||
|
||||
```text
|
||||
protocol_version: 1
|
||||
status: ok
|
||||
case_id: <case_id>
|
||||
run: <run>
|
||||
expected_version: <version>
|
||||
provider: <provider>
|
||||
model_id: <model_id>
|
||||
score: <0_to_case_max>
|
||||
cost: <number_or_null>
|
||||
duration_ms: <non_negative_integer>
|
||||
session_id: <test_session_id>
|
||||
```
|
||||
|
||||
On failure:
|
||||
|
||||
```text
|
||||
protocol_version: 1
|
||||
status: infrastructure_failure
|
||||
case_id: <case_id>
|
||||
run: <run>
|
||||
expected_version: <version>
|
||||
provider: <provider>
|
||||
model_id: <model_id>
|
||||
failure_code: <stable_failure_code>
|
||||
```
|
||||
|
||||
Stable codes are `invalid_request`, `invalid_statement`, `invalid_rubric`, `cli_failed`, `provenance_mismatch`, `version_changed`, and `invalid_score`. Do not include score, cost, duration, Session id, private data, or optimization advice on failure.
|
||||
@@ -0,0 +1,6 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M8 4h8" />
|
||||
<path d="M9 3h6v3H9z" />
|
||||
<rect x="5" y="5" width="14" height="16" rx="2" />
|
||||
<path d="m8.5 13 2.2 2.2 4.8-5" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 307 B |
@@ -0,0 +1,146 @@
|
||||
---
|
||||
name: agent-optimization
|
||||
description: Improve an Agent State from direct feedback or versioned multi-Case Benchmark scores and score-linked Traces.
|
||||
short_description: Improve an Agent from feedback or measured Benchmark results.
|
||||
short_description_zh: 根据反馈或 Benchmark 结果改进 Agent。
|
||||
version: 1
|
||||
updated: 2026-07-17T17:08:17Z
|
||||
---
|
||||
|
||||
# Agent Optimization
|
||||
|
||||
Improve an existing Agent State. Use one-shot feedback mode for a direct correction, or Benchmark optimization mode for a measured loop. Do not mix their execution paths. One-shot mode does not require Agent Evaluator. Benchmark mode evaluates every configured Case run through Agent Evaluator and never launches or scores the Test Agent directly.
|
||||
|
||||
## Before you start
|
||||
|
||||
If the request supplies neither concrete feedback nor an explicit Test Agent and Benchmark, ask what to improve. Determine the mode before editing anything.
|
||||
|
||||
Benchmark mode requires a top-level Session with `run_subagent`, a complete baseline series in `scoreboard.yaml`, and `agent-evaluation` installed on the current Agent. If any requirement is missing, stop and explain what the user must provide or install. Do not begin a partial optimization round.
|
||||
|
||||
## Pick the target Agent
|
||||
|
||||
A one-shot request normally names the target. A delegated request begins with `Caller agent: <agent_id>`, and an @-mention handoff contains `<handoff_from>`. When one-shot mode has no explicit target, use that caller or origin; if neither exists, ask. Benchmark mode always requires an explicit Test Agent and Benchmark.
|
||||
|
||||
Resolve paths from the Environment's Project Dir without recursively discovering the Project:
|
||||
|
||||
```text
|
||||
PROJECT_DIR = <project_dir>
|
||||
PROJECT_ID = <basename_of_project_dir>
|
||||
PENGUIN_HOME = <parent_of_project_dir>
|
||||
TARGET = <project_dir>/agents/<test_agent_id>
|
||||
STATE = <target>/agent_state
|
||||
TRACES = <target>/traces
|
||||
BENCHMARK = <target>/benchmarks/<benchmark_id>
|
||||
SCOREBOARD = <benchmark>/scoreboard.yaml
|
||||
```
|
||||
|
||||
Never read a Project configuration file, credential, vault, private Rubric, Agent Evaluator State, Evaluator Workspace, or Evaluator Trace.
|
||||
|
||||
## State editing policy
|
||||
|
||||
Make the smallest complete edit supported by evidence and preserve unrelated instructions and files.
|
||||
|
||||
- Behavioral, workflow, role, or domain guidance belongs in `agent_state/AGENTS.md` unless a relevant target-owned Skill already owns that reusable capability.
|
||||
- Update a relevant target-owned `SKILL.md` when the behavior is a reusable capability shared across tasks.
|
||||
- Create a narrowly named Skill only when the capability is reusable and no suitable Skill exists. Installing its directory is sufficient; do not register Skill metadata in AGENTS.md.
|
||||
- Runtime limits belong in safe `system_config.yaml` fields. Do not edit `system_prompt` unless the user explicitly asks.
|
||||
- Never modify a library-provided Skill such as `penguin-sdk` to carry target-specific behavior.
|
||||
|
||||
Every edit must generalize beyond the observed run. Do not encode Case ids, exact expected outputs, Benchmark-specific constants, private criteria, or a guessed answer. Keep a recovered mapping, threshold, or formula only when repeated public evidence supports it as a durable rule; otherwise encode the reasoning and validation method.
|
||||
|
||||
## Version and rollback discipline
|
||||
|
||||
Before changing Agent State, read its canonical top-level `version` from `system_config.yaml`, defaulting to 1. The system owns Agent State snapshot archives and exposes them through Web export and import. Do not create, import, extract, or replace snapshot archives yourself. Require `<target>/snapshots/v<version>.tar.gz` to exist; if it is missing, stop and ask the user to export the current Agent State from Agent settings before continuing.
|
||||
|
||||
For each file the edit will change, record its exact original bytes and whether it existed in a temporary directory outside `STATE`. Never include `.vault.toml`, an unrelated State file, or a snapshot archive. Write each candidate file through a temporary sibling, validate it, and rename it into place. Set the State version to `current + 1` exactly once before evaluation.
|
||||
|
||||
If the candidate is rejected or a valid comparison cannot complete, restore only those recorded files, remove files created by the candidate, and verify that the prior version is active. If the active version or a candidate-owned file no longer matches the value written by this round, treat it as a concurrent mutation and stop without overwriting it. If rollback cannot be verified, stop and ask the user to restore the system snapshot through Web import.
|
||||
|
||||
## One-shot feedback mode
|
||||
|
||||
Use the user's feedback and relevant recent Trace when available. Turn that evidence into the smallest targeted change:
|
||||
|
||||
- behavior, workflow, role, or domain guidance → update AGENTS.md or the relevant target-owned Skill;
|
||||
- a missing reusable capability → install or update a narrowly scoped Skill;
|
||||
- a runtime limit → adjust the relevant `system_config.yaml` field.
|
||||
|
||||
Require the current-version system snapshot and record the exact originals of files being changed before editing. Increment the version once after the complete edit. If the edit cannot complete, roll back only those files. Report the evidence, changed State surface, new version, and reason. Do not claim measured improvement because this mode has no Benchmark comparison.
|
||||
|
||||
## Benchmark optimization mode
|
||||
|
||||
Require `benchmark_config.toml` with a positive integer `runs` and a complete baseline evaluation. The baseline Case set is frozen for optimization. Each reference or candidate evaluation must include exactly `runs` uniquely numbered runs for every frozen Case.
|
||||
|
||||
Use the reference evaluation's `(provider, model_id)` pair unchanged so scores remain comparable. If the user wants a different Model, stop and ask for a new baseline series rather than comparing across Models.
|
||||
|
||||
Before any candidate edit, read the canonical top-level State version and select a reference evaluation in the existing `(provider, model_id)` series. The selected reference is valid only when its `version` equals the active State version and its Cases and runs form the complete frozen Case × `runs` matrix. Never compare a candidate against an older-version, incomplete, or mixed-Model evaluation.
|
||||
|
||||
If the active State has no such evaluation, measure it before optimizing: leave State unchanged, run the complete frozen matrix with the existing series' exact `(provider, model_id)` pair, and validate the results under the same rules used for a candidate. Retain the exact Scoreboard bytes and active State version before dispatch; append the completed no-edit reference through a temporary sibling, YAML validation, and atomic rename only if both remain unchanged. This appended evaluation becomes the reference. If any cell remains invalid after the allowed retry, provenance does not match, State or Scoreboard changes, or the atomic append cannot complete, stop before editing Agent State.
|
||||
|
||||
You may inspect the complete target `agent_state/`, public Case Statements, the Scoreboard, and all Test traces referenced by the Scoreboard runs. Use only those explicit Case and Session ids. Never edit the Benchmark, Test traces, Project configuration, or another Agent.
|
||||
|
||||
Scoreboard v2 uses this shape:
|
||||
|
||||
```yaml
|
||||
evaluations:
|
||||
- time: "2026-07-17T00:00:00Z"
|
||||
version: 2
|
||||
provider: deepseek
|
||||
model_id: deepseek-v4-pro
|
||||
summary_title: "Improved evidence validation"
|
||||
summary: "Added a reusable validation step; all Cases improved without new instability."
|
||||
score: 24
|
||||
cost: 0.04
|
||||
duration_ms: 60000
|
||||
cases:
|
||||
- case: CASE-001-example
|
||||
score: 24
|
||||
cost: 0.04
|
||||
duration_ms: 60000
|
||||
runs:
|
||||
- score: 23
|
||||
cost: 0.03
|
||||
duration_ms: 58000
|
||||
session_id: session-1
|
||||
- score: 24
|
||||
cost: 0.04
|
||||
duration_ms: 60000
|
||||
session_id: session-2
|
||||
- score: 25
|
||||
cost: 0.05
|
||||
duration_ms: 62000
|
||||
session_id: session-3
|
||||
```
|
||||
|
||||
Case metrics are the means of valid `runs`; evaluation totals are the sums of Case means. Omit cost when any contributing run has unknown cost. `summary_title` and `summary` may describe public State changes, gains, regressions, and instability, but must not reveal private Rubric or Gold content, expected answers, or private scoring reasoning.
|
||||
|
||||
## Optimization loop
|
||||
|
||||
For each round:
|
||||
|
||||
1. Reconfirm that the reference version equals the active top-level State version and that it contains the complete frozen Case × `runs` matrix. Then analyze its aggregate, Case scores, repeated runs, and every score-linked Test Trace. Use repeated runs to separate stable failure from variation; never select a convenient Trace.
|
||||
2. State one falsifiable behavioral hypothesis connecting public evidence to a minimal State change. If no credible hypothesis remains, stop.
|
||||
3. Confirm the current-version system snapshot exists, record the exact originals of every candidate-owned file, make the candidate edit, and set `version` to `current + 1` once.
|
||||
4. Retain the exact Scoreboard bytes and candidate version. Build the complete Case-run matrix before dispatch.
|
||||
5. Start one child per cell with `run_subagent`, and omit `agent_id` so the child reuses the current Agent. Each prompt begins with the caller identity and the sentence: Use the `agent-evaluation` Skill. Then it contains one request:
|
||||
|
||||
```text
|
||||
Caller agent: <current_agent_id>
|
||||
Use the `agent-evaluation` Skill. Return only its terminal protocol YAML.
|
||||
protocol_version: 1
|
||||
case_id: <case_id>
|
||||
run: <1_based_run_index>
|
||||
expected_version: <candidate_agent_state_version>
|
||||
test_agent_id: <test_agent_id>
|
||||
benchmark_dir: <absolute_benchmark_dir>
|
||||
provider: <reference_provider>
|
||||
model_id: <reference_upstream_model_id>
|
||||
```
|
||||
|
||||
For N Cases and R runs, emit all N × R independent calls in the same parallel tool-call group before waiting. Continue an active child through `input_subagent`; never duplicate it.
|
||||
6. Parse only each child's last terminal `protocol_version: 1` YAML mapping. Keep every identity-matched `status: ok` result. Retry invalid cells once, dispatching all retries together with identical State, Model, Benchmark, and run identities. Never retry a valid scored cell. If any retry remains invalid, reject the round and roll back the candidate files.
|
||||
7. Compute Case means and evaluation sums. Retain every score, cost when complete, duration, and Test Session id. Re-read State version and exact Scoreboard bytes, and verify that every candidate-owned file still matches the value written by this round. If State changed concurrently, stop without overwriting it. If only the Scoreboard changed, reject the round and roll back the candidate files.
|
||||
8. Accept only a score strictly higher than the comparable reference evaluation. On improvement, keep the candidate State and append one evaluation through a temporary sibling, YAML validation, and atomic rename. On an equal or lower score, roll back the candidate files and append nothing.
|
||||
|
||||
Each accepted round becomes the next reference. Stop when the user's target or round limit is met, no credible evidence-backed hypothesis remains, or infrastructure prevents another valid comparison. Do not mutate State as random search.
|
||||
|
||||
At the end, report the accepted score curve, State versions and changes, rejected hypotheses and rollbacks, Test Session ids, stop reason, and limitations. Distinguish the active tested State from any unscored State; never claim a Scoreboard score applies to a later untested edit.
|
||||
@@ -0,0 +1,7 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M5 15.5a7 7 0 0 1 14 0" />
|
||||
<path d="M12 8.5v1.8" />
|
||||
<path d="M7.05 10.55l1.27 1.27" />
|
||||
<path d="M12 15.5l3.8-3.8" />
|
||||
<circle cx="12" cy="15.5" r="1.1" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 341 B |
@@ -0,0 +1,123 @@
|
||||
---
|
||||
name: agenthub-models
|
||||
description: Call model APIs through @prismshadow/agenthub — streaming text generation, image generation, speech synthesis and embeddings with one client.
|
||||
short_description: Call model APIs with one AgentHub client.
|
||||
short_description_zh: 用一个 AgentHub 客户端调用模型 API。
|
||||
version: 1
|
||||
updated: 2026-07-17T00:00:00Z
|
||||
---
|
||||
|
||||
# AgentHub Model APIs
|
||||
|
||||
`@prismshadow/agenthub` is a unified TypeScript client for model APIs: streaming text, image generation, speech synthesis and embeddings behind one entry point.
|
||||
|
||||
```bash
|
||||
npm install @prismshadow/agenthub
|
||||
```
|
||||
|
||||
The only entry point is `AutoLLMClient`:
|
||||
|
||||
```ts
|
||||
import { AutoLLMClient } from "@prismshadow/agenthub";
|
||||
|
||||
const client = new AutoLLMClient({ model: "<model_id>", apiKey: "<key>", baseUrl: "<url>", clientType: "<type>" });
|
||||
```
|
||||
|
||||
`apiKey`, `baseUrl` and `clientType` are optional (see routing below).
|
||||
|
||||
## Before you start
|
||||
|
||||
If the user's message only invokes this skill (e.g. "use agenthub-models skill") without a concrete task, ask the user what they want to build. Do not write code until the requirement is clear.
|
||||
|
||||
## Model IDs
|
||||
|
||||
Use exact model ids. If an id is not in the table below and the user has not given one, ask the user to confirm the exact id before writing code.
|
||||
|
||||
| Family | Official IDs | Gateway variants |
|
||||
| ---------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Gemini 3 | `gemini-3.1-pro-preview`, `gemini-3.5-flash`, `gemini-3.1-flash-lite` | — |
|
||||
| Gemini 3 image | `gemini-3.1-flash-image-preview`, `gemini-3-pro-image-preview` | — |
|
||||
| Gemini 3 TTS | `gemini-3.1-flash-tts-preview` | — |
|
||||
| Gemini embedding | `gemini-embedding-2` | — |
|
||||
| Claude | `claude-sonnet-4-6`, `claude-opus-4-7`, `claude-opus-4-8` | — |
|
||||
| GPT | `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.5` | — |
|
||||
| OpenAI embedding | `text-embedding-3-small`, `text-embedding-3-large` | — |
|
||||
| Kimi K2.6 | `kimi-k2.6` | OpenRouter `moonshotai/kimi-k2.6`; SiliconFlow `Pro/moonshotai/Kimi-K2.6` |
|
||||
| DeepSeek V4 | `deepseek-v4-pro`, `deepseek-v4-flash` | OpenRouter `deepseek/deepseek-v4-pro`, `deepseek/deepseek-v4-flash`; SiliconFlow `deepseek-ai/DeepSeek-V4-Pro`, `deepseek-ai/DeepSeek-V4-Flash` |
|
||||
| GLM 5.1 | `glm-5.1` | OpenRouter `z-ai/glm-5.1`; SiliconFlow `Pro/zai-org/GLM-5.1` |
|
||||
|
||||
Gateway model lists can be queried online:
|
||||
|
||||
```bash
|
||||
curl https://openrouter.ai/api/v1/models
|
||||
curl --request GET --url https://api.siliconflow.cn/v1/models --header 'Authorization: Bearer <token>'
|
||||
```
|
||||
|
||||
## Routing and credentials
|
||||
|
||||
- Without `clientType`, the client auto-routes by model id substring: `gemini-3*`, `gemini-embedding`, `claude` 4-6/4-7/4-8, `gpt-5.4`/`gpt-5.5`, `glm-5`, `kimi-k2.5`/`kimi-k2.6`, `deepseek-v4`, `openai`+`embedding` (embeddings), `openai`. Ids matching none of these throw. The gateway variants in the table above hit the same substrings, so they route to the right family — just set `baseUrl` to the gateway endpoint.
|
||||
- For any other OpenAI chat-completion compatible model (e.g. Qwen series via OpenRouter or SiliconFlow), pass `clientType: "openai"` plus `baseUrl` (embeddings endpoints use a different client type — see Embeddings below).
|
||||
- API key: constructor parameter first, then the provider environment variable — `DEEPSEEK_API_KEY`, `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`, `ZAI_API_KEY`, `MOONSHOT_API_KEY`. Base URLs read the same names with `_BASE_URL`.
|
||||
|
||||
## Streaming text
|
||||
|
||||
```ts
|
||||
for await (const event of client.streamingResponseStateful({
|
||||
message: { role: "user", content_items: [{ type: "text", text: "Hello" }] },
|
||||
config: {},
|
||||
})) {
|
||||
for (const item of event.content_items) {
|
||||
if (item.type === "text") process.stdout.write(item.text);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- Each `event` is a `UniEvent`: `event_type` is `start` | `delta` | `stop`, and `content_items` carry the increments.
|
||||
- `config` accepts `max_tokens`, `temperature`, `system_prompt`, `thinking_level` (the `ThinkingLevel` enum, `NONE` to `XHIGH`) and `tools`.
|
||||
- `streamingResponseStateful` keeps conversation history inside the client; manage it with `getHistory()` / `setHistory(history)` / `clearHistory()`. The stateless variant is `streamingResponse({ messages, config })`.
|
||||
|
||||
## Image generation
|
||||
|
||||
Use a Gemini image model (see Model IDs) and set `config.image_config` (optional `aspect_ratio`, and `image_size` of `"1K"` | `"2K"`):
|
||||
|
||||
```ts
|
||||
import fs from "node:fs";
|
||||
|
||||
const client = new AutoLLMClient({ model: "gemini-3.1-flash-image-preview" });
|
||||
for await (const event of client.streamingResponseStateful({
|
||||
message: { role: "user", content_items: [{ type: "text", text: "A penguin on a glacier" }] },
|
||||
config: { image_config: { aspect_ratio: "16:9", image_size: "2K" } },
|
||||
})) {
|
||||
for (const item of event.content_items) {
|
||||
if (item.type === "inline_data") fs.writeFileSync("image.png", item.data);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Images arrive as `inline_data` content items (`data` is a Buffer, with `mime_type`).
|
||||
|
||||
## Speech synthesis
|
||||
|
||||
Use a Gemini TTS model (`gemini-3.1-flash-tts-preview`) and set `config.tts_config`:
|
||||
|
||||
```ts
|
||||
config: { tts_config: [{ voice: "Kore" }] }
|
||||
```
|
||||
|
||||
- One entry → single voice; two entries → multi-speaker, and each entry must also set `speaker`.
|
||||
- The `inline_data` output is raw PCM (24kHz 16-bit mono) — wrap it in a WAV header yourself before saving as `.wav`.
|
||||
|
||||
## Embeddings
|
||||
|
||||
Two routes:
|
||||
|
||||
- Gemini: a model whose id contains `gemini-embedding` auto-routes (`gemini-embedding-2`).
|
||||
- Any OpenAI-compatible embeddings endpoint: pass `clientType: "openai-embedding"` (plus `baseUrl` and `apiKey` as needed) — ids like `text-embedding-3-small` / `text-embedding-3-large` match no auto-route substring and would throw without it.
|
||||
|
||||
Optional `config.embedding_config`:
|
||||
|
||||
```ts
|
||||
config: { embedding_config: { dimensions: 768 } }
|
||||
```
|
||||
|
||||
The output arrives as `embedding` content items (`embedding` is a number array).
|
||||
@@ -0,0 +1,9 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<circle cx="12" cy="12" r="2.6" />
|
||||
<circle cx="5" cy="5" r="1.9" />
|
||||
<circle cx="19" cy="5" r="1.9" />
|
||||
<circle cx="12" cy="20" r="1.9" />
|
||||
<path d="M10.2 10.2 6.3 6.3" />
|
||||
<path d="M13.8 10.2 17.7 6.3" />
|
||||
<path d="M12 14.6v3.5" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 410 B |
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: benchmark-design
|
||||
description: Design and calibrate a multi-Case capability Benchmark with repeated independent evaluations and a traceable baseline.
|
||||
short_description: Design and calibrate an Agent capability Benchmark.
|
||||
short_description_zh: 设计并校准 Agent 能力评测 Benchmark。
|
||||
version: 1
|
||||
updated: 2026-07-17T17:08:17Z
|
||||
---
|
||||
|
||||
# Benchmark Design
|
||||
|
||||
Create and calibrate a multi-Case Benchmark that discovers a specified Test Agent's capability boundary. Own the public Statements, private Rubrics, Case set, configuration, and final baseline. Do not modify the Test Agent State, launch the Test Agent directly, or score a Case yourself.
|
||||
|
||||
## Before you start
|
||||
|
||||
Require a Test Agent and the capability to measure. If either is missing, ask the user. A Benchmark run also requires a top-level Session with `run_subagent` and the current Agent must have `agent-evaluation` installed. If the Skill is missing or this Session is already a subagent, stop and ask the user to install the Skill or start a top-level Session. Do not begin a partial Benchmark.
|
||||
|
||||
## Boundaries
|
||||
|
||||
Access only the explicit Test Agent and Benchmark paths. Do not inspect another Agent, Project configuration files, Agent Evaluator State, Evaluator Workspace, or Evaluator Trace. Consume only each Evaluator's terminal protocol response and the returned Test Session id.
|
||||
|
||||
Use the Environment's Project Dir and the explicit Test Agent id:
|
||||
|
||||
```text
|
||||
PROJECT_DIR = <project_dir>
|
||||
PROJECT_ID = <basename_of_project_dir>
|
||||
PENGUIN_HOME = <parent_of_project_dir>
|
||||
TEST_AGENT_DIR = <project_dir>/agents/<test_agent_id>
|
||||
BENCHMARK_DIR = <project_dir>/agents/<test_agent_id>/benchmarks/<benchmark_id>
|
||||
SCOREBOARD = <benchmark_dir>/scoreboard.yaml
|
||||
```
|
||||
|
||||
Derive a semantic Benchmark id when the user does not supply one. Require `agent_state/system_config.yaml`; its top-level `version` is the canonical State version and defaults to 1 when absent.
|
||||
|
||||
## Benchmark contract
|
||||
|
||||
Use this structure:
|
||||
|
||||
```text
|
||||
<benchmark_id>/
|
||||
├── benchmark_config.toml
|
||||
├── scoreboard.yaml
|
||||
└── CASE-<nnn>-<semantic-name>/
|
||||
├── statement/
|
||||
│ └── README.md
|
||||
└── rubric/
|
||||
└── README.md
|
||||
```
|
||||
|
||||
Both README files are required; either directory may contain supporting files. `statement/` is the complete public task and evidence. `rubric/` is private scoring material. Never mention private criteria or paths in the Statement.
|
||||
|
||||
Create `benchmark_config.toml` first. The Model is deliberately not stored here; each evaluation records the actual `(provider, model_id)` pair.
|
||||
|
||||
```toml
|
||||
title = "<benchmark_title>"
|
||||
description = "<capability_and_scope>"
|
||||
runs = 3
|
||||
```
|
||||
|
||||
Use `runs = 3` unless the user explicitly requests another positive integer. Initialize the Scoreboard with:
|
||||
|
||||
```yaml
|
||||
evaluations: []
|
||||
```
|
||||
|
||||
Every accepted evaluation follows scoreboard v2:
|
||||
|
||||
```yaml
|
||||
evaluations:
|
||||
- time: "2026-07-17T00:00:00Z"
|
||||
version: 1
|
||||
provider: deepseek
|
||||
model_id: deepseek-v4-pro
|
||||
summary_title: "Calibrated baseline"
|
||||
summary: "Abbreviated baseline schema for one Case with three independent runs."
|
||||
score: 18
|
||||
cost: 0.04
|
||||
duration_ms: 60000
|
||||
cases:
|
||||
- case: CASE-001-example
|
||||
score: 18
|
||||
cost: 0.04
|
||||
duration_ms: 60000
|
||||
runs:
|
||||
- score: 17
|
||||
cost: 0.03
|
||||
duration_ms: 58000
|
||||
session_id: session-1
|
||||
- score: 18
|
||||
cost: 0.04
|
||||
duration_ms: 60000
|
||||
session_id: session-2
|
||||
- score: 19
|
||||
cost: 0.05
|
||||
duration_ms: 62000
|
||||
session_id: session-3
|
||||
```
|
||||
|
||||
Case `score`, `cost`, and `duration_ms` are the means of their valid `runs`. Evaluation totals are the sums of the Case means. Omit a Case or evaluation `cost` when any contributing run has unknown cost; never treat unknown as zero. Rubric maxima across the complete Case set should total 100 points so the evaluation score remains interpretable on a 0–100 scale.
|
||||
|
||||
Builder writes one final baseline for the current Benchmark definition. A material change to a Statement, Rubric, Case set, or `runs` invalidates prior results: clear `evaluations`, recalibrate, and write a new baseline. Rejected candidates and provisional matrices remain only in Builder Trace. Evaluator never writes the Scoreboard.
|
||||
|
||||
The public `summary_title` and `summary` may describe the tested State, Case-level score patterns, and instability. They must not reveal Rubric or Gold content, expected answers, private scoring reasoning, mappings, thresholds, formulas, or rules.
|
||||
|
||||
## Design and calibrate
|
||||
|
||||
Before writing Cases, define the observable difference between an Agent that has the requested capability and one that does not. Each Case must make that capability causally necessary, not merely share its topic.
|
||||
|
||||
Run this counterfactual before accepting a Case: could a competent executor without the target capability complete it by mechanically following the Statement? If yes, reject or redesign it. A self-contained Statement specifies the task, available evidence, and required artifact without disclosing the reasoning, mapping, or rule the capability is supposed to recover. The evidence must still make the answer inferable.
|
||||
|
||||
Build several independent, realistic end-to-end Cases with distinct capability-relevant failure modes. Do not manufacture low scores through missing essential evidence, trivia, formatting traps, excessive workload, or unstable infrastructure.
|
||||
|
||||
Freeze every Rubric before evaluation. Use atomic observable conditions, exact points, reasonable equivalence rules, and meaningful partial credit. Test the Rubric mentally against full, partial, missing, malformed, wrong-type, and extra output. Never execute Test Agent-produced code while scoring.
|
||||
|
||||
Use valid evidence to calibrate toward a user-supplied target; otherwise aim near 60/100. Treat a near-ceiling candidate as uncalibrated when a credible structural refinement remains. Audit high scores for shortcuts or leakage and low scores for ambiguity, missing evidence, unrelated difficulty, or a defective Rubric. More items, steps, or workload alone are not structural refinement.
|
||||
|
||||
## Select the evaluation Model
|
||||
|
||||
Use a user-specified `(provider, model_id)` pair when supplied. Otherwise resolve the Project default with the supported CLI, never by reading the hidden Project configuration:
|
||||
|
||||
```bash
|
||||
penguin config model list --project-id "<project_id>" --root "<penguin_home>"
|
||||
```
|
||||
|
||||
The row marked `*` is the default. Keep the same pair through all candidate matrices for this calibration. The pair selects the Test Agent's CLI Session, not the Builder or Evaluator runtime model.
|
||||
|
||||
## Run the Case-run matrix
|
||||
|
||||
Read and retain the exact State version, Scoreboard bytes, configured positive `runs`, selected Model pair, and complete valid Case set. Build every unique Case-run cell before dispatch.
|
||||
|
||||
Start one child per cell with `run_subagent`, and omit `agent_id` so the child reuses the current Agent. Each prompt must begin with the caller identity and the sentence: Use the `agent-evaluation` Skill. Then provide exactly one request:
|
||||
|
||||
```text
|
||||
Caller agent: <current_agent_id>
|
||||
Use the `agent-evaluation` Skill. Return only its terminal protocol YAML.
|
||||
protocol_version: 1
|
||||
case_id: <case_id>
|
||||
run: <1_based_run_index>
|
||||
expected_version: <tested_agent_state_version>
|
||||
test_agent_id: <test_agent_id>
|
||||
benchmark_dir: <absolute_benchmark_dir>
|
||||
provider: <provider>
|
||||
model_id: <upstream_model_id>
|
||||
```
|
||||
|
||||
For N Cases and R runs, emit all N × R independent `run_subagent` calls in the same parallel tool-call group before waiting for results. Continue an active child through `input_subagent`; never duplicate it.
|
||||
|
||||
Parse only the last terminal `protocol_version: 1` YAML mapping. Keep every identity-matched `status: ok` result. Retry only invalid cells once, with all retry cells dispatched together and the same State version, Model, Benchmark, and run identities. Never retry a valid scored cell. If any retry remains invalid, abandon the matrix.
|
||||
|
||||
For a complete matrix, calculate Case means and evaluation sums using scoreboard v2. Retain every run's score, cost when known, duration, and Test Session id. Re-read State version and exact Scoreboard bytes; abandon the result if either changed.
|
||||
|
||||
Use each returned Test Session id to inspect the exact Test Trace and artifact. Analyze all repeated runs; disagreement is capability instability, not permission to select a convenient result. Accept a candidate only when the capability caused the scored difference, evidence was sufficient, the Rubric was sound, and useful headroom remains.
|
||||
|
||||
When calibration stops after a complete valid matrix, write the final baseline to a temporary sibling, parse it as YAML, then atomically rename it over `scoreboard.yaml`. Include the sorted Case set and sorted runs, a real UTC ISO-8601 time, the tested version and Model pair, and a privacy-safe summary. A near-ceiling final score is still recorded, but if the refinement budget ends without an acceptable candidate, report `calibration_failed` rather than calling the Benchmark ready. Leave `evaluations` empty only when no complete valid matrix exists.
|
||||
|
||||
Report the Benchmark path, aggregate and Case scores, Test Session ids, refinements, stop reason, and limitations.
|
||||
@@ -0,0 +1,7 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<rect x="4" y="4" width="16" height="16" rx="2" />
|
||||
<path d="M8 8h8" />
|
||||
<path d="M8 12h5" />
|
||||
<path d="M8 16h3" />
|
||||
<path d="m15 15 1.4 1.4L19 13.8" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 327 B |
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: data-analysis
|
||||
description: Complete data-analysis tasks with bounded evidence inspection, explicit answer-changing decisions, native artifact handling, and final output verification.
|
||||
short_description: Complete data analysis with a bounded, verified method.
|
||||
short_description_zh: 以有界、可验证的方法完成数据分析任务。
|
||||
version: 1
|
||||
updated: 2026-07-18T00:00:00Z
|
||||
---
|
||||
|
||||
# Data Analysis
|
||||
|
||||
## Before you start
|
||||
|
||||
Require a concrete data-analysis task, its available inputs, and the requested deliverable, location, and format. If the task or required materials are missing or ambiguous in an answer-changing way, ask before calculating or creating outputs. Otherwise proceed without forcing a fixed analysis template.
|
||||
|
||||
## Success criteria
|
||||
|
||||
- Derive the final result from one committed method: the selected evidence, answer-changing decisions, transformations, calculations, and judgments.
|
||||
- Produce every output the task asks for, in the requested location and format, and ensure it reflects the committed method.
|
||||
- Ground each answer-changing choice in the task materials rather than in a merely plausible nearby match.
|
||||
- Prefer a complete, simple, defensible deliverable over an elaborate or exhaustive analysis that risks not being delivered.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Use bounded probes first for large, unfamiliar, or expensive-to-read inputs. Narrow the scope, cap output, or use a timeout before expanding.
|
||||
- Before calculating or producing final outputs, identify the few decisions that can change the answer, such as inclusion or exclusion, matching, boundary choices, transformations, formulas, and ranking criteria. Adapt this check to the task; do not force a fixed analysis template.
|
||||
- Compare materially plausible alternatives only when they would change the final output. Use the smallest comparison needed to resolve the choice from the task materials, then commit to a method.
|
||||
- Once the evidence supports a defensible answer, create the requested output files promptly. Do not continue open-ended thinking, searching, or polishing over alternatives that would not change the delivered answer.
|
||||
- Use intermediate files only when they help compute or verify the result. Preserve the requested output format and structure unless the task asks for a change.
|
||||
- Satisfy the requested deliverable with the simplest sufficient artifact and implementation. Avoid optional structure, styling, helper code, or reimplementation that is not required by the task.
|
||||
- Keep final delivery steps short and robust. Avoid putting a long report, large dataset, or large script into one fragile streamed command when the deliverable can be created by shorter steps. Create the required artifact first, then refine only if the required outputs already exist.
|
||||
- For spreadsheet or Office deliverables, prefer the tool path that preserves the native artifact contract. If a task depends on Excel formula recalculation, data tables, workbook formatting, or Office export, first check whether native Excel automation such as `xlwings` is available before falling back to LibreOffice or a hand-rolled model. Record formulas or inputs that will be temporarily overwritten, restore them before finalizing, and verify that the requested workbook, deck, document, or PDF can be opened and contains the expected tables, sheets, slides, or sections. Do not dump or reimplement an entire workbook when bounded input changes and output reads can answer the task.
|
||||
- When creating spreadsheet tables, keep each table as one contiguous rectangle: title or caption, row-axis labels, column-axis labels, and data cells should be adjacent and inspectable together. Do not place row labels or key headers outside the visible table bounds or to the left of the title anchor. If you add decorative axis labels, keep the actual machine-readable row and column values inside the same table rectangle.
|
||||
|
||||
## Stop rules
|
||||
|
||||
- Before finalizing, inspect the requested outputs and check their location, format, shape, and content against the committed method and the task.
|
||||
- If a later finding changes the method, assumptions, selections, transformations, calculations, decisions, or coverage, regenerate the affected outputs before finalizing.
|
||||
- Once the requested outputs exist and the relevant checks pass, stop instead of continuing open-ended exploration.
|
||||
@@ -0,0 +1,5 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M4 4.5v15h16" />
|
||||
<path d="m7 15 3-3 3 2 4-6" />
|
||||
<path d="M7 18v1.5M10 18v1.5M13 18v1.5M17 18v1.5" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 284 B |
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: penguin-cli
|
||||
description: Manage model API keys, default models and per-agent vault secrets with the penguin CLI.
|
||||
short_description: Manage models and secrets with the penguin CLI.
|
||||
short_description_zh: 用 penguin CLI 管理模型与密钥。
|
||||
version: 1
|
||||
updated: 2026-07-17T00:00:00Z
|
||||
---
|
||||
|
||||
# Penguin CLI
|
||||
|
||||
The `penguin` CLI manages model credentials, default models and per-agent vault secrets. Configuration goes through the CLI only — never read or hand-edit the underlying hidden files.
|
||||
|
||||
## Before you start
|
||||
|
||||
If the user's message only invokes this skill (e.g. "use penguin-cli skill") without a concrete request, ask the user what they want to configure. Do not run any command until the goal is clear.
|
||||
|
||||
## Models
|
||||
|
||||
Add or update a model (upsert by the stored model id; re-run with more options to amend an entry):
|
||||
|
||||
```bash
|
||||
penguin config model add --model-id <upstream_id> [--provider <group>] [--api-key <key>] [--base-url <url>] \
|
||||
[--client-type <type>] [--context-window <n>] [--vision | --no-vision] \
|
||||
[--price-cache-read <n>] [--price-cache-write <n>] [--price-output <n>] \
|
||||
[--project-id <id>] [--root <dir>] [--set-default]
|
||||
```
|
||||
|
||||
- `--model-id` takes the provider's upstream model id (what the API expects). The stored id is always `<provider>/<upstream_id>`: `--provider` picks the provider group, and when omitted it is inferred from the built-in catalog (unrecognized ids fall back to `custom`). The upstream id is persisted automatically as the entry's request id, so nothing extra is needed for it to reach the API unchanged.
|
||||
- For any OpenAI chat-completion compatible endpoint use `--client-type openai --base-url <endpoint>`; omit `--client-type` to auto-route by model id.
|
||||
- Prices are USD per million tokens (cache read / cache write / output).
|
||||
- `--vision` / `--no-vision` mark whether the model accepts images; omitting both keeps the current value (default is vision-capable).
|
||||
- All `penguin config model ...` and `penguin config vault ...` commands accept `--root <dir>` to target another data root (default `PENGUIN_HOME`, then `~/.penguin/data`).
|
||||
|
||||
Other model commands:
|
||||
|
||||
```bash
|
||||
penguin config model default --model-id <upstream_id> --provider <group> [--root <dir>] # set the project default model
|
||||
penguin config model vision --model-id <upstream_id> --provider <group> [--root <dir>] # set the project vision model (reads images for text-only sessions)
|
||||
penguin config model list [--root <dir>] # list models; api_key is shown masked
|
||||
```
|
||||
|
||||
## Vault (per-agent secrets)
|
||||
|
||||
The vault holds an agent's environment-variable secrets (third-party API keys etc.); values are injected into that agent's shell subprocesses:
|
||||
|
||||
```bash
|
||||
penguin config vault set --key <NAME> --value <value> [--project-id <id>] [--agent-id <id>] [--root <dir>]
|
||||
penguin config vault list [--project-id <id>] [--agent-id <id>] [--root <dir>] # values are shown masked
|
||||
penguin config vault remove --key <NAME> [--project-id <id>] [--agent-id <id>] [--root <dir>]
|
||||
```
|
||||
|
||||
- `--project-id` defaults to `default_project`, `--agent-id` to `default_agent`.
|
||||
- Key names follow shell variable rules (letter or underscore first, then letters, digits and underscores); values are limited to 8192 characters.
|
||||
|
||||
## Language
|
||||
|
||||
```bash
|
||||
penguin config lang <en|zh> # persist the CLI language via PENGUIN_LANG in your shell rc
|
||||
```
|
||||
|
||||
## Running agents
|
||||
|
||||
`penguin run -m "<task>" [--model-id <id>] [--agent-id <id>] [--workspace <path>] [--approve <mode>]` runs one task; `penguin chat [--resume [session_id]]` starts or resumes an interactive chat with the same options.
|
||||
|
||||
## Storage
|
||||
|
||||
- `<project_dir>/.project_config.toml` — the project's single hidden config file: model list, settings and per-model credentials (`api_key` etc. inlined in each model entry). Configuration is CLI-only — never read, print or hand-edit this file.
|
||||
- `<project_dir>/agents/<agent_id>/agent_state/.vault.toml` — that agent's vault entries, hidden file; same rule, manage it with `penguin config vault`.
|
||||
@@ -0,0 +1,5 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<rect x="3" y="4.5" width="18" height="15" rx="2" />
|
||||
<path d="M7 9.5l3.2 2.7L7 14.9" />
|
||||
<path d="M12.7 15.5h4.3" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 290 B |
@@ -0,0 +1,88 @@
|
||||
---
|
||||
name: penguin-sdk
|
||||
description: Build AI apps on the Penguin Harness SDK — self-contained projects inside the Workspace, model configuration, and the createSession/run streaming loop.
|
||||
short_description: Build AI apps on the Penguin Harness SDK.
|
||||
short_description_zh: 基于 Penguin Harness SDK 构建 AI 应用。
|
||||
version: 1
|
||||
updated: 2026-07-17T00:00:00Z
|
||||
---
|
||||
|
||||
# Penguin Harness SDK
|
||||
|
||||
`@prismshadow/penguin-core` is the TypeScript SDK this agent itself runs on. Use it to build your own AI apps:
|
||||
|
||||
- An **Agent** loads its state (prompts, tools, skills) from `<root>/<project_id>/agents/<agent_id>/`. Creating an Agent whose directory is empty initializes it with defaults.
|
||||
- A **Session** is one conversation of an Agent inside a **Workspace** directory.
|
||||
- `session.run()` executes one task and streams every step (thinking, text, tool calls) as OmniMessages.
|
||||
|
||||
To have an agent perform a task, use the `run_subagent` tool — the SDK is for building applications, not for invoking agents.
|
||||
|
||||
## Before you start
|
||||
|
||||
If the user's message only invokes this skill (e.g. "use penguin-sdk skill") without a concrete app to build, ask the user what they want to build. Do not start until the requirement is clear.
|
||||
|
||||
## Project location
|
||||
|
||||
Create the app in the current workspace directory by default (the `CWD` value from your Environment section), as a self-contained project — do not place it under `<project_dir>` or depend on any path outside the project folder. Point the agent data root at a directory inside the project with `createAgent({ root })`, resolved from the source file so it stays relative:
|
||||
|
||||
```ts
|
||||
const agent = await createAgent({ root: path.resolve(import.meta.dirname, "penguin_data") });
|
||||
```
|
||||
|
||||
With every reference relative to the project, the user can move or copy the folder anywhere and it still runs.
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
npm install @prismshadow/penguin-core
|
||||
```
|
||||
|
||||
If the package is not yet available on your npm registry (it is developed in the PenguinHarness monorepo and may not be published), develop inside a checkout of the PenguinHarness repo instead: add your app as a workspace package under `packages/` and depend on `"@prismshadow/penguin-core": "workspace:*"`, then run `pnpm install && pnpm build` at the repo root. Tell the user which route you took.
|
||||
|
||||
A model must be configured for the app's data root. Two ways:
|
||||
|
||||
1. The penguin CLI, pointed at the project-local data directory:
|
||||
|
||||
```bash
|
||||
penguin config model add --root <data_dir> --model-id <id> --api-key <key> [--base-url <url>] [--client-type openai] --set-default
|
||||
```
|
||||
|
||||
Generally prefer the OpenAI protocol client (chat completion): `--client-type openai --base-url <endpoint>` works with any OpenAI-compatible endpoint. Use exact model ids — see the agenthub-models skill for the id table.
|
||||
|
||||
2. Environment variables as a fallback: without a configured credential the SDK reads the provider's env vars (e.g. `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `DEEPSEEK_API_KEY`).
|
||||
|
||||
Model config lives in a single hidden file under the data root's project directory: `.project_config.toml` (model list, settings and per-model credentials such as `api_key` inlined in each model entry). It is CLI-only — never read, print or edit it; the CLI above manages it.
|
||||
|
||||
## Minimal conversational app
|
||||
|
||||
```ts
|
||||
import path from "node:path";
|
||||
import readline from "node:readline/promises";
|
||||
import { createAgent, userText } from "@prismshadow/penguin-core";
|
||||
|
||||
const agent = await createAgent({ root: path.resolve(import.meta.dirname, "penguin_data") });
|
||||
const session = await agent.createSession({ workspaceDir: process.cwd() });
|
||||
|
||||
const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
|
||||
for (;;) {
|
||||
const line = await rl.question("> ");
|
||||
if (!line.trim()) break;
|
||||
// One run per user turn; the same Session keeps the conversation context.
|
||||
for await (const msg of session.run([userText(line)], {
|
||||
approve: async () => "allow", // demo only — a real app should ask its user ("deny" blocks the call)
|
||||
})) {
|
||||
const p = msg.payload;
|
||||
if (p.type === "partial_text" && p.event_type === "delta") process.stdout.write(p.text);
|
||||
}
|
||||
process.stdout.write("\n");
|
||||
}
|
||||
rl.close();
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- An Agent's behavior is edited in its `agent_state/` files (system_config.yaml, AGENTS.md, skills/), not in code.
|
||||
- `createSession({ workspaceDir, modelId })` — omit `workspaceDir` for a temporary workspace, omit `modelId` for the project default model.
|
||||
- `session.run(messages, { approve, signal })` is an async generator of OmniMessages; filter the payload types you care about (`partial_text` deltas carry the streamed answer).
|
||||
- The `approve` callback gates every tool call — auto-allow is for demos; a production app should prompt its user before returning `"allow"`.
|
||||
- Call `session.run()` again on the same Session for the next user turn — the context carries over.
|
||||
@@ -0,0 +1,6 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="m12 3 8 4.5v9L12 21l-8-4.5v-9z" />
|
||||
<path d="m4 7.5 8 4.5 8-4.5" />
|
||||
<path d="M12 12v9" />
|
||||
<path d="m8 5.25 8 4.5" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 301 B |
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: software-engineering
|
||||
description: Complete software-engineering tasks — investigate and review code, implement bug fixes, features and refactors with minimal scope, validate changes, and report verified outcomes.
|
||||
short_description: Complete software-engineering tasks.
|
||||
short_description_zh: 完成软件工程任务。
|
||||
version: 1
|
||||
updated: 2026-07-18T00:00:00Z
|
||||
---
|
||||
|
||||
# Software Engineering
|
||||
|
||||
This skill guides PenguinHarness through general software-engineering work, including code investigation, reviews, bug fixes, features, refactors, and verified handoff.
|
||||
|
||||
## Before you start
|
||||
|
||||
If the user's message only invokes this skill without a concrete software-engineering task, ask what they want investigated, reviewed, fixed, or implemented. Do not start until the task is clear.
|
||||
|
||||
## Workflow
|
||||
|
||||
- Match the work to the user's intent. For explanation, review, or planning requests, inspect and report without editing unless a change is also requested. For implementation tasks, carry the requested change through verification and handoff.
|
||||
- Use the current working directory as the target project and work in place unless the user directs otherwise.
|
||||
- Preserve worktree changes you did not make. Never revert unrelated user or collaborator work, and do not use destructive Git commands unless explicitly requested.
|
||||
- Act autonomously and use tools to understand the real code. Resolve minor ambiguity from existing behavior, tests, and repository conventions; ask only when different choices would materially change the requested behavior or scope.
|
||||
- Before making changes, read the applicable repository instructions (such as `AGENTS.md` or `CLAUDE.md`) and identify the repository-provided build, test, lint, and formatting commands.
|
||||
- Inspect the relevant implementation, callers, tests, and interfaces before making changes.
|
||||
- For bug reports, try to reproduce the issue or identify a failing test before editing when feasible. Use the reproduced behavior or failing test to verify the fix afterward.
|
||||
- Make the smallest coherent change that fully satisfies the request. Follow existing patterns and dependencies, preserve unrelated behavior, and update coupled tests, schemas, configuration, or generated artifacts only when the repository requires it. Never weaken a test to justify the implementation.
|
||||
- For implementation tasks, do not stop after analysis. Inspect failures, revise the approach, and continue until the change is verified or a concrete blocker remains. Do not repeat a failed action without changing the approach.
|
||||
- Validate code changes proportionally with repository-provided commands: run the most focused relevant check first and broaden only when useful. Preserve exit status, inspect relevant failure output, and never claim that an unobserved check passed.
|
||||
- Before finishing a code change, review `git diff` and repository status, remove temporary artifacts, and keep the change limited to the task. Do not commit unless the user or applicable repository instructions require it.
|
||||
|
||||
## Handoff
|
||||
|
||||
- Keep the final response concise: summarize the outcome or change, list checks actually run and their outcomes, and state any remaining limitation.
|
||||
@@ -0,0 +1,5 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<path d="M8.5 8 5 12l3.5 4" />
|
||||
<path d="M15.5 8 19 12l-3.5 4" />
|
||||
<path d="M13.3 6.2 10.7 17.8" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 272 B |
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: web-design
|
||||
description: Default visual language for generated web pages — minimal black-white-gray, square corners, hierarchy from weight and spacing; colors, radii and gradients only on explicit request.
|
||||
short_description: Minimal monochrome defaults for generated web pages.
|
||||
short_description_zh: 生成网页的极简黑白默认视觉规范。
|
||||
version: 1
|
||||
updated: 2026-07-17T00:00:00Z
|
||||
---
|
||||
|
||||
# Web Design
|
||||
|
||||
Default visual rules for every web page or frontend interface you generate. Apply them to any HTML/CSS you produce unless the user explicitly asks otherwise.
|
||||
|
||||
## Before you start
|
||||
|
||||
If the user's message only invokes this skill (e.g. "use web-design skill") without a concrete page or interface to build, ask the user what they want to build. Do not start until the requirement is clear.
|
||||
|
||||
## Core rules
|
||||
|
||||
- Monochrome only: black, white and grays. No accent colors, no gradients, no decorative shadows.
|
||||
- Square corners everywhere: `border-radius: 0` on buttons, cards, inputs, images and modals.
|
||||
- Hierarchy comes from font weight, font size, spacing and thin light-gray borders — never from colored blocks or backgrounds.
|
||||
- Generous whitespace: prefer more spacing over more dividers; let sections breathe.
|
||||
- Introduce colors, rounded corners or gradients **only when the user explicitly asks for them**, and only where asked — the rest of the page stays monochrome and square.
|
||||
|
||||
## Tokens
|
||||
|
||||
Base every stylesheet on a small monochrome token set:
|
||||
|
||||
```css
|
||||
:root {
|
||||
--fg: #111111; /* primary text */
|
||||
--fg-muted: #666666; /* secondary text */
|
||||
--bg: #ffffff; /* page background */
|
||||
--bg-subtle: #f5f5f5; /* raised surfaces */
|
||||
--border: #e2e2e2; /* hairline borders (1px) */
|
||||
}
|
||||
```
|
||||
|
||||
Tailwind equivalent: stick to `text-neutral-900` / `text-neutral-500` / `bg-white` / `bg-neutral-100` / `border-neutral-200` / `rounded-none`; do not use color utilities (`blue-*`, `emerald-*`, ...), `rounded-*` variants other than `rounded-none`, or `bg-gradient-*`.
|
||||
@@ -0,0 +1,9 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.7" stroke-linecap="round" stroke-linejoin="round">
|
||||
<rect x="3" y="4.5" width="18" height="15" />
|
||||
<path d="M3 9h18" />
|
||||
<path d="M6 6.75h.01" />
|
||||
<path d="M8.7 6.75h.01" />
|
||||
<path d="M9 9v10.5" />
|
||||
<path d="M12 12.5h6" />
|
||||
<path d="M12 15.5h3.5" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 374 B |
@@ -0,0 +1,212 @@
|
||||
/**
|
||||
* PenguinHarness Skill library: built-in SKILL.md docs and skill group manifest.
|
||||
*
|
||||
* The runtime source of truth for library content is the package's `skills/<skill_name>/SKILL.md`
|
||||
* files: frontmatter is parsed on read (same rules as installed Skills), so editing a file takes
|
||||
* effect immediately with no caching (files are small, calls are infrequent). Only the skill group
|
||||
* manifest (id, title, and member names) is hardcoded in code; install / uninstall / scan still live
|
||||
* in core's state layer.
|
||||
*
|
||||
* Docs: packages/docs/content/skills.{zh,en}.md (site path /docs/skills) documents the Skill
|
||||
* format and the built-in library.
|
||||
*/
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
/** Skill's frontmatter metadata (four fields: name / description / version / updated; description itself is English-only, may also carry description_zh and short_description(_zh)). */
|
||||
export interface SkillMetadata {
|
||||
/** Skill name (matches its containing directory name). */
|
||||
name: string;
|
||||
/** One-line description; injected into the model prompt via `{{SKILL_METADATA}}`. */
|
||||
description: string;
|
||||
/** UI short description (frontmatter `short_description`, optional): preferred in compact spots like cards, falls back to the full description if missing; not injected into the prompt. */
|
||||
shortDescription?: string;
|
||||
/** Chinese short description (frontmatter `short_description_zh`, optional). */
|
||||
shortDescriptionZh?: string;
|
||||
/** Version number (natural number); falls back to 1 on parse failure. */
|
||||
version: number;
|
||||
/** Update date (YYYY-MM-DD); defaults to "". */
|
||||
updated: string;
|
||||
}
|
||||
|
||||
/** A Skill in the library: metadata + full SKILL.md content (including frontmatter, written as-is on install). */
|
||||
export interface LibrarySkill extends SkillMetadata {
|
||||
content: string;
|
||||
/** Optional raw `icon.svg` content in the directory (custom icon, the file is the sole source, copied alongside SKILL.md on install); absent means none (frontend falls back to the default book icon). */
|
||||
icon?: string;
|
||||
}
|
||||
|
||||
/** Skill group manifest entry: group id, title (optionally with a Chinese title, displayed per UI language), and member Skill names. */
|
||||
export interface SkillGroupInfo {
|
||||
id: string;
|
||||
title: string;
|
||||
/** Chinese group title (optional, displayed per UI language). */
|
||||
titleZh?: string;
|
||||
/** Member Skill names (i.e., directory names under `skills/`). */
|
||||
skills: string[];
|
||||
}
|
||||
|
||||
/** Grouping result: group metadata + member Skills read from library files. */
|
||||
export interface ResolvedSkillGroup extends Omit<SkillGroupInfo, "skills"> {
|
||||
skills: LibrarySkill[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses the frontmatter at the start of SKILL.md: only recognizes `key: value` lines inside the
|
||||
* first `---` block (split on the first colon, value trimmed, values may themselves contain colons);
|
||||
* all fields are scalars, no YAML dependency needed.
|
||||
* Error tolerance: returns null if the `---` block or name is missing; version falls back to 1 if
|
||||
* it isn't a natural number; updated defaults to "".
|
||||
*/
|
||||
export function parseSkillFrontmatter(content: string): SkillMetadata | null {
|
||||
// Strip a possible UTF-8 BOM (may be introduced by editors when manually editing an installed SKILL.md); CRLF is handled by \r?\n.
|
||||
const match = /^---\r?\n([\s\S]*?)\r?\n---/.exec(content.replace(/^\uFEFF/, ""));
|
||||
if (!match) return null;
|
||||
const fields: Record<string, string> = {};
|
||||
for (const line of match[1]!.split(/\r?\n/)) {
|
||||
const idx = line.indexOf(":");
|
||||
if (idx <= 0) continue;
|
||||
const key = line.slice(0, idx).trim();
|
||||
if (key) fields[key] = line.slice(idx + 1).trim();
|
||||
}
|
||||
const name = fields["name"];
|
||||
if (!name) return null;
|
||||
const version = Number.parseInt(fields["version"] ?? "", 10);
|
||||
const shortDescription = fields["short_description"];
|
||||
const shortDescriptionZh = fields["short_description_zh"];
|
||||
return {
|
||||
name,
|
||||
description: fields["description"] ?? "",
|
||||
// short_description(_zh) is optional: omitted when absent (undefined keys aren't set).
|
||||
...(shortDescription !== undefined ? { shortDescription } : {}),
|
||||
...(shortDescriptionZh !== undefined ? { shortDescriptionZh } : {}),
|
||||
version: Number.isInteger(version) && version >= 1 ? version : 1,
|
||||
updated: fields["updated"] ?? "",
|
||||
};
|
||||
}
|
||||
|
||||
/** Root directory of library files: the package's `skills/` (both dist/ and src/ sit one level below the package root, so one level up reaches it). */
|
||||
const SKILLS_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "skills");
|
||||
|
||||
/** Character rule for Skill names (directory names): prevents path traversal. */
|
||||
const SKILL_NAME_PATTERN = /^[A-Za-z0-9_-]+$/;
|
||||
|
||||
/**
|
||||
* Reads a single library directory to construct a LibrarySkill; returns undefined if SKILL.md
|
||||
* doesn't exist. name is taken from the directory name (overriding frontmatter); falls back to
|
||||
* empty metadata if frontmatter parsing fails.
|
||||
* The optional icon.svg in the directory is read alongside it (as a raw string); the icon field
|
||||
* is omitted if missing.
|
||||
*/
|
||||
function readSkillDir(name: string): LibrarySkill | undefined {
|
||||
const dir = path.join(SKILLS_ROOT, name);
|
||||
let content: string;
|
||||
try {
|
||||
content = fs.readFileSync(path.join(dir, "SKILL.md"), "utf8");
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
let icon: string | undefined;
|
||||
try {
|
||||
icon = fs.readFileSync(path.join(dir, "icon.svg"), "utf8");
|
||||
} catch {
|
||||
// icon.svg is optional: no custom icon if missing.
|
||||
}
|
||||
const meta = parseSkillFrontmatter(content) ?? {
|
||||
name,
|
||||
description: "",
|
||||
version: 1,
|
||||
updated: "",
|
||||
};
|
||||
return { ...meta, name, content, ...(icon !== undefined ? { icon } : {}) };
|
||||
}
|
||||
|
||||
/** Reads all Skills in the library (one per subdirectory under `skills/`), sorted by name. */
|
||||
export function loadLibrarySkills(): LibrarySkill[] {
|
||||
const skills: LibrarySkill[] = [];
|
||||
for (const entry of fs.readdirSync(SKILLS_ROOT, { withFileTypes: true })) {
|
||||
if (!entry.isDirectory()) continue;
|
||||
const skill = readSkillDir(entry.name);
|
||||
if (skill) skills.push(skill);
|
||||
}
|
||||
return skills.sort((a, b) => a.name.localeCompare(b.name));
|
||||
}
|
||||
|
||||
/**
|
||||
* Skill group manifest; members are library directory names.
|
||||
* Docs: /docs/skills § "Built-in library".
|
||||
*/
|
||||
export const SKILL_GROUPS: SkillGroupInfo[] = [
|
||||
{
|
||||
id: "agent-development",
|
||||
title: "Agent Development",
|
||||
titleZh: "Agent 开发",
|
||||
skills: ["agent-creation", "benchmark-design", "agent-evaluation", "agent-optimization"],
|
||||
},
|
||||
{
|
||||
id: "data-analysis",
|
||||
title: "Data Analysis",
|
||||
titleZh: "数据分析",
|
||||
skills: ["data-analysis"],
|
||||
},
|
||||
{
|
||||
id: "penguin-development",
|
||||
title: "Penguin Development",
|
||||
titleZh: "Penguin 开发",
|
||||
skills: ["penguin-sdk", "penguin-cli", "agenthub-models"],
|
||||
},
|
||||
{
|
||||
id: "web-development",
|
||||
title: "Web Development",
|
||||
titleZh: "网页开发",
|
||||
skills: ["web-design"],
|
||||
},
|
||||
{
|
||||
id: "software-engineering",
|
||||
title: "Software Engineering",
|
||||
titleZh: "软件工程",
|
||||
skills: ["software-engineering"],
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
* Groups library Skills according to SKILL_GROUPS (a member name missing from `all` is skipped);
|
||||
* Skills not listed in any group are appended to an Other group (only appears if non-empty). A
|
||||
* pure function, the testable core of loadSkillGroups.
|
||||
*/
|
||||
export function groupSkills(all: LibrarySkill[]): ResolvedSkillGroup[] {
|
||||
const byName = new Map(all.map((skill) => [skill.name, skill]));
|
||||
const grouped = new Set<string>();
|
||||
const groups: ResolvedSkillGroup[] = SKILL_GROUPS.map((group) => {
|
||||
const members: LibrarySkill[] = [];
|
||||
for (const name of group.skills) {
|
||||
const skill = byName.get(name);
|
||||
if (!skill) continue;
|
||||
members.push(skill);
|
||||
grouped.add(name);
|
||||
}
|
||||
return { ...group, skills: members };
|
||||
});
|
||||
const others = all.filter((skill) => !grouped.has(skill.name));
|
||||
if (others.length > 0) {
|
||||
groups.push({
|
||||
id: "other",
|
||||
title: "Other",
|
||||
titleZh: "其他",
|
||||
skills: others,
|
||||
});
|
||||
}
|
||||
return groups;
|
||||
}
|
||||
|
||||
/** Reads library files and groups them: SKILL_GROUPS order comes first, ungrouped Skills are appended to an Other group (only appears if non-empty). */
|
||||
export function loadSkillGroups(): ResolvedSkillGroup[] {
|
||||
return groupSkills(loadLibrarySkills());
|
||||
}
|
||||
|
||||
/** Reads a single library Skill by name; returns undefined if the name contains illegal characters (path traversal guard) or doesn't exist. */
|
||||
export function librarySkill(name: string): LibrarySkill | undefined {
|
||||
if (!SKILL_NAME_PATTERN.test(name)) return undefined;
|
||||
return readSkillDir(name);
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
/**
|
||||
* Tests for the Skill library file source of truth and its parser: loadLibrarySkills reading
|
||||
* files into a manifest, loadSkillGroups grouping, groupSkills' Other group and missing-member
|
||||
* tolerance, librarySkill's traversal-name rejection, doc conventions (`## Before you start` is
|
||||
* mandatory), and parseSkillFrontmatter's error tolerance.
|
||||
*/
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
SKILL_GROUPS,
|
||||
groupSkills,
|
||||
librarySkill,
|
||||
loadLibrarySkills,
|
||||
loadSkillGroups,
|
||||
parseSkillFrontmatter,
|
||||
type LibrarySkill,
|
||||
} from "../src/index.js";
|
||||
|
||||
const skillsRoot = path.resolve(import.meta.dirname, "../skills");
|
||||
|
||||
/** Minimal LibrarySkill for groupSkills unit tests. */
|
||||
const fakeSkill = (name: string): LibrarySkill => ({
|
||||
name,
|
||||
description: `Do ${name}.`,
|
||||
version: 1,
|
||||
updated: "2026-07-17T00:00:00Z",
|
||||
content: `---\nname: ${name}\n---\nBody`,
|
||||
});
|
||||
|
||||
describe("loadLibrarySkills", () => {
|
||||
it("按 name 排序读出技能，metadata 齐全（含中文描述与短描述）", async () => {
|
||||
const skills = loadLibrarySkills();
|
||||
const names = skills.map((skill) => skill.name);
|
||||
expect(names).toEqual([...names].sort());
|
||||
for (const skill of skills) {
|
||||
expect(skill.description, skill.name).toBeTruthy();
|
||||
// Short description (UI display): both languages present, and clearly shorter than the full description.
|
||||
expect(skill.shortDescription, skill.name).toBeTruthy();
|
||||
expect(skill.shortDescriptionZh, skill.name).toBeTruthy();
|
||||
expect(skill.shortDescription!.length, skill.name).toBeLessThan(skill.description.length);
|
||||
expect(skill.shortDescriptionZh!.length, skill.name).toBeLessThan(skill.description.length);
|
||||
// Pre-release, version is always 1.
|
||||
expect(skill.version).toBe(1);
|
||||
expect(skill.updated).toMatch(/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z$/);
|
||||
// content is the full SKILL.md text including frontmatter (written as-is on install).
|
||||
expect(skill.content.startsWith("---\n")).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("每个技能都带定制 icon.svg（原文读入，站内线稿风，无脚本）", async () => {
|
||||
for (const skill of loadLibrarySkills()) {
|
||||
const raw = await fs.readFile(path.join(skillsRoot, skill.name, "icon.svg"), "utf8");
|
||||
// The icon field is the raw icon.svg content in the directory (the file is the sole source).
|
||||
expect(skill.icon, skill.name).toBe(raw);
|
||||
expect(skill.icon, skill.name).toContain('viewBox="0 0 24 24"');
|
||||
expect(skill.icon, skill.name).toContain('stroke="currentColor"');
|
||||
expect(skill.icon, skill.name).toContain('fill="none"');
|
||||
// Security baseline: no scripts or event attributes (frontend also sanitizes before inline rendering).
|
||||
expect(skill.icon, skill.name).not.toContain("<script");
|
||||
expect(skill.icon, skill.name).not.toMatch(/\son[a-z]+=/i);
|
||||
}
|
||||
});
|
||||
|
||||
it("name 以目录名为准，content 与 skills/ 下 SKILL.md 原文一致", async () => {
|
||||
const dirs = (await fs.readdir(skillsRoot, { withFileTypes: true }))
|
||||
.filter((entry) => entry.isDirectory())
|
||||
.map((entry) => entry.name)
|
||||
.sort();
|
||||
const skills = loadLibrarySkills();
|
||||
expect(skills.map((s) => s.name)).toEqual(dirs);
|
||||
for (const skill of skills) {
|
||||
const raw = await fs.readFile(path.join(skillsRoot, skill.name, "SKILL.md"), "utf8");
|
||||
expect(skill.content).toBe(raw);
|
||||
// The library file's own frontmatter name should match its directory name (content quality constraint).
|
||||
expect(skill.content).toContain(`name: ${skill.name}`);
|
||||
}
|
||||
});
|
||||
|
||||
it("每个技能正文都有 `## Before you start` 一节（无具体需求时先反问）", () => {
|
||||
for (const skill of loadLibrarySkills()) {
|
||||
expect(skill.content, skill.name).toContain("## Before you start");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("loadSkillGroups / groupSkills", () => {
|
||||
it("按 Skill 组清单解组，成员齐全且带中文组名，无 Other 组", () => {
|
||||
const groups = loadSkillGroups();
|
||||
expect(groups.map((g) => g.id)).toEqual([
|
||||
"agent-development",
|
||||
"data-analysis",
|
||||
"penguin-development",
|
||||
"web-development",
|
||||
"software-engineering",
|
||||
]);
|
||||
expect(groups[0]!.skills.map((s) => s.name)).toEqual([
|
||||
"agent-creation",
|
||||
"benchmark-design",
|
||||
"agent-evaluation",
|
||||
"agent-optimization",
|
||||
]);
|
||||
expect(groups[1]!.skills.map((s) => s.name)).toEqual(["data-analysis"]);
|
||||
expect(groups[1]!.title).toBe("Data Analysis");
|
||||
expect(groups[1]!.titleZh).toBe("数据分析");
|
||||
expect(groups[2]!.skills.map((s) => s.name)).toEqual([
|
||||
"penguin-sdk",
|
||||
"penguin-cli",
|
||||
"agenthub-models",
|
||||
]);
|
||||
expect(groups[3]!.skills.map((s) => s.name)).toEqual(["web-design"]);
|
||||
expect(groups[3]!.title).toBe("Web Development");
|
||||
expect(groups[3]!.titleZh).toBe("网页开发");
|
||||
expect(groups[4]!.skills.map((s) => s.name)).toEqual(["software-engineering"]);
|
||||
expect(groups[4]!.title).toBe("Software Engineering");
|
||||
expect(groups[4]!.titleZh).toBe("软件工程");
|
||||
for (const group of groups) {
|
||||
expect(group.title).toBeTruthy();
|
||||
expect(group.titleZh).toBeTruthy();
|
||||
// Groups no longer carry a description (group header is just title + skill count).
|
||||
expect("description" in group).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it("groupSkills：未列入任何组的技能追加 Other 组（含中英文组名）", () => {
|
||||
const stray = fakeSkill("stray-skill");
|
||||
const groups = groupSkills([fakeSkill("agent-creation"), stray]);
|
||||
expect(groups.map((g) => g.id)).toEqual([
|
||||
"agent-development",
|
||||
"data-analysis",
|
||||
"penguin-development",
|
||||
"web-development",
|
||||
"software-engineering",
|
||||
"other",
|
||||
]);
|
||||
const other = groups[5]!;
|
||||
expect(other.title).toBe("Other");
|
||||
expect(other.titleZh).toBe("其他");
|
||||
expect(other.skills).toEqual([stray]);
|
||||
});
|
||||
|
||||
it("groupSkills：成员名缺失则跳过；全部入组时不出现 Other 组", () => {
|
||||
const groups = groupSkills([fakeSkill("penguin-cli")]);
|
||||
expect(groups.map((g) => g.id)).toEqual([
|
||||
"agent-development",
|
||||
"data-analysis",
|
||||
"penguin-development",
|
||||
"web-development",
|
||||
"software-engineering",
|
||||
]);
|
||||
expect(groups[0]!.skills).toEqual([]);
|
||||
expect(groups[1]!.skills).toEqual([]);
|
||||
expect(groups[2]!.skills.map((s) => s.name)).toEqual(["penguin-cli"]);
|
||||
expect(groups[3]!.skills).toEqual([]);
|
||||
expect(groups[4]!.skills).toEqual([]);
|
||||
});
|
||||
|
||||
it("SKILL_GROUPS 清单硬编码为成员名（库文件之外唯一的组信息真源）", () => {
|
||||
expect(SKILL_GROUPS.map((g) => ({ id: g.id, skills: g.skills }))).toEqual([
|
||||
{
|
||||
id: "agent-development",
|
||||
skills: ["agent-creation", "benchmark-design", "agent-evaluation", "agent-optimization"],
|
||||
},
|
||||
{ id: "data-analysis", skills: ["data-analysis"] },
|
||||
{ id: "penguin-development", skills: ["penguin-sdk", "penguin-cli", "agenthub-models"] },
|
||||
{ id: "web-development", skills: ["web-design"] },
|
||||
{ id: "software-engineering", skills: ["software-engineering"] },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("librarySkill", () => {
|
||||
it("按名称读单个技能，未知名称返回 undefined", () => {
|
||||
expect(librarySkill("penguin-sdk")?.name).toBe("penguin-sdk");
|
||||
expect(librarySkill("no-such-skill")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("非法字符名一律拒绝（防路径穿越），不触达文件系统", () => {
|
||||
for (const name of ["../penguin-sdk", "..", "penguin-sdk/SKILL.md", "a/../b", ".", ""]) {
|
||||
expect(librarySkill(name), name).toBeUndefined();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseSkillFrontmatter", () => {
|
||||
it("解析 name/description/version/updated，值允许含冒号", () => {
|
||||
const meta = parseSkillFrontmatter(
|
||||
"---\nname: demo\ndescription: How to use x: y and z\nversion: 3\nupdated: 2026-07-16\n---\n\nBody",
|
||||
);
|
||||
expect(meta).toEqual({
|
||||
name: "demo",
|
||||
description: "How to use x: y and z",
|
||||
version: 3,
|
||||
updated: "2026-07-16",
|
||||
});
|
||||
});
|
||||
|
||||
it("short_description_zh 可选：有则解析，缺省不带该字段", () => {
|
||||
const withZh = parseSkillFrontmatter(
|
||||
"---\nname: demo\ndescription: Do x\nshort_description_zh: 做 x\n---\nBody",
|
||||
);
|
||||
expect(withZh?.shortDescriptionZh).toBe("做 x");
|
||||
const withoutZh = parseSkillFrontmatter("---\nname: demo\ndescription: Do x\n---\nBody");
|
||||
expect(withoutZh).not.toBeNull();
|
||||
expect(withoutZh && "shortDescriptionZh" in withoutZh).toBe(false);
|
||||
});
|
||||
|
||||
it("short_description(_zh) 可选：有则解析为 shortDescription(Zh)，缺省不带该字段", () => {
|
||||
const withShort = parseSkillFrontmatter(
|
||||
"---\nname: demo\ndescription: Do x in detail\nshort_description: Do x\nshort_description_zh: 做 x\n---\nBody",
|
||||
);
|
||||
expect(withShort?.shortDescription).toBe("Do x");
|
||||
expect(withShort?.shortDescriptionZh).toBe("做 x");
|
||||
const without = parseSkillFrontmatter("---\nname: demo\ndescription: Do x\n---\nBody");
|
||||
expect(without && "shortDescription" in without).toBe(false);
|
||||
expect(without && "shortDescriptionZh" in without).toBe(false);
|
||||
});
|
||||
|
||||
it("UTF-8 BOM 与 CRLF 换行照常解析（手改文件的编辑器可能引入）", () => {
|
||||
const bom = parseSkillFrontmatter("\uFEFF---\nname: demo\ndescription: Do x\n---\nBody");
|
||||
expect(bom?.name).toBe("demo");
|
||||
const crlf = parseSkillFrontmatter("---\r\nname: demo\r\ndescription: Do x\r\n---\r\nBody");
|
||||
expect(crlf?.description).toBe("Do x");
|
||||
});
|
||||
|
||||
it("缺 --- 块或缺 name 返回 null", () => {
|
||||
expect(parseSkillFrontmatter("# No frontmatter")).toBeNull();
|
||||
expect(parseSkillFrontmatter("---\ndescription: only desc\n---\nBody")).toBeNull();
|
||||
// A block that isn't at the start doesn't count as frontmatter either.
|
||||
expect(parseSkillFrontmatter("Body\n---\nname: x\n---")).toBeNull();
|
||||
});
|
||||
|
||||
it("version 非自然数回退 1，updated 缺省空串", () => {
|
||||
expect(parseSkillFrontmatter("---\nname: a\nversion: zero\n---")?.version).toBe(1);
|
||||
expect(parseSkillFrontmatter("---\nname: a\nversion: 0\n---")?.version).toBe(1);
|
||||
expect(parseSkillFrontmatter("---\nname: a\n---")).toEqual({
|
||||
name: "a",
|
||||
description: "",
|
||||
version: 1,
|
||||
updated: "",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"extends": "../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "."
|
||||
},
|
||||
"include": ["src", "test"]
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
import { defineConfig } from "tsup";
|
||||
|
||||
export default defineConfig({
|
||||
entry: ["src/index.ts"],
|
||||
format: ["esm"],
|
||||
target: "node20",
|
||||
dts: true,
|
||||
clean: true,
|
||||
sourcemap: true,
|
||||
});
|
||||
Reference in new issue
Block a user