penguin-harness/packages/docs/content/sessions-and-traces.en.md

title, description

title	description
Sessions & Traces	The six-level run model, local data directory layout, Trace file design, and Session recovery.

All PenguinHarness runtime data lives on the local file system: configuration is editable files, history is append-only Traces. This page defines each level of the run model and explains how the Trace serves as history, recovery source, and statistics source at once.

Run model

Six levels: Project → Agent → Workspace → Session → Task → Request.

Concept	Definition
Project	Top-level unit organizing Agents; owns the model and credential configuration; in the multi-user Web setup, users and Projects are many-to-many
Agent	The executing subject; has exactly one Agent State (a persistent directory); one Agent can serve many Workspaces
Workspace	The working directory of one run — the only file scope the model sees; an explicit workspaceDir must already exist, otherwise a temporary Workspace workspaces/tmp-<8hex> is created
Session	A continuous conversation under one (Agent, Workspace); model and Workspace are locked at Session creation; ids look like session-YYYY-MM-DD-HH-mm-ss-<8hex>
Task	One execution goal started by one Prompt; consists of one or more consecutive Requests
Request	One LLM API call: context and tool definitions in, streamed output out

See the Architecture page for how the levels cooperate, and the Agent Loop for how Requests advance within a Task.

Data layout

The data root is the PENGUIN_HOME environment variable, defaulting to ~/.penguin/data. The layout is defined in one place, packages/core/src/state/paths.ts:

<root>/<project>/
├── .project_config.toml          # Project-level models & credentials (hidden file, 0600)
└── agents/
    └── <agent>/
        ├── agent_state/              # system_config.yaml, AGENTS.md, .vault.toml,
        │                             # tools/, memory/, skills/, schedule/
        ├── traces/
        │   └── <yyyy-mm-dd>/<sessionId>_<index3>.jsonl
        ├── scratchpad/               # temp files, one subdirectory per Session id (e.g. pasted images)
        ├── shared_env/               # shared interpreter/tool environments (virtualenvs, pipx,
        │                             # caches) the Agent creates on demand — a system prompt
        │                             # convention, not a path the code creates, so tooling is
        │                             # installed once for any task; project dependencies stay
        │                             # in the project
        ├── workspaces/               # temporary Workspaces (tmp-<8hex>)
        ├── benchmarks/               # capability Benchmark cases and scores
        └── snapshots/                # Agent State version snapshots

See the Configuration Reference for the fields of each config file.

Trace design

A Trace is an append-only JSON Lines file; each line is one OmniMessage envelope (see the OmniMessage Protocol). History is only ever appended, never modified in place.

• One Trace file corresponds to one complete model context. When compaction produces a new context segment, the writer rotates to a new file — _002, _003, … — with an incrementing index.
• Recorded: session_meta, complete model_msg, and all event_msg.
• Not recorded: streaming partial_* fragments (the producer appends the complete message once the segment ends), and nested messages tagged with origin — a subagent's messages go to the child Session's own Trace, while the parent Trace keeps a single subagent pointer event at the spawn site recording the child Session id.
• request_begin and request_end(status) come in pairs delimiting one Request; replay uses request_end.status === "completed" as the commit criterion for that turn.

See packages/core/src/trace/writer.ts for the implementation.

The head of a Trace (illustrative; one OmniMessage envelope per line):

{"timestamp":"2026-07-18T03:10:22.531Z","type":"session_meta","payload":{"session_id":"session-2026-07-18-11-10-22-3f8a1c2d","provider":"deepseek","model_id":"deepseek-v4-pro","model_context_window":1000000,"system_prompt":"…","tools":[…],"agent_state":"/home/u/.penguin/data/default_project/agents/default_agent/agent_state","workspace":"/home/u/work"}}
{"timestamp":"…","type":"event_msg","payload":{"type":"request_begin"}}
{"timestamp":"…","type":"model_msg","payload":{"type":"text","role":"user","text":"Create hello.txt"}}
{"timestamp":"…","type":"model_msg","payload":{"type":"tool_call","role":"assistant","name":"exec_command","arguments":"{\"cmd\":\"printf hi > hello.txt\"}","tool_call_id":"call_0"}}
{"timestamp":"…","type":"event_msg","payload":{"type":"approval_decision","decision":"allow","tool_call_id":"call_0"}}
{"timestamp":"…","type":"model_msg","payload":{"type":"tool_call_output","role":"user","output":"[no output]","tool_call_id":"call_0"}}
{"timestamp":"…","type":"event_msg","payload":{"type":"request_end","status":"completed"}}
{"timestamp":"…","type":"event_msg","payload":{"type":"token_usage","session":{…},"request":{…}}}

When tool output exceeds maxOutputLength, Trace records the same bounded head, truncation marker, and absolute Session recovery path seen by Web/CLI and the model; it does not separately duplicate the archived text. The path exposes the host data-root layout but remains valid across Tasks and Session resume because the unredacted recovery file lives in that Session's scratchpad. The existing explicit Session-deletion path removes the scratchpad and recovery file together. Trace replay therefore faithfully restores both what the model saw and a usable pointer for later follow-up.

Session recovery

The Trace is the single source of truth for recovery — there is no separate session database to keep in sync. resumeSession works as follows:

1. Locate the highest-index Trace file of the Session;
2. Read the runtime configuration from its session_meta — model, system prompt, Workspace — all immutable for the lifetime of the Session;
3. Replay the committed history into a fresh LLM context;
4. Reconstruct the carry-over (undelivered tool outputs, interruption markers) plus turn and Token counters;
5. Continue appending to the same Trace file.

Recovery requires that the Workspace and the model still exist. What recovery guarantees is structural legality: only committed turns are replayed, with tool_call / tool_call_output pairing intact; incomplete model output (thinking, text) is allowed to be lost. A truncated last line left by an abnormal process exit is tolerated and ignored. See packages/core/src/trace/resume.ts.

Special case: if the latest Trace file ends with a completed compaction, that context is closed as a whole — resume starts from an empty context; in summarize mode the [context_summary] is reconstructed and prepended to the first input after resume (old Traces using the earlier angle-bracket <summary> form are still understood).

Model switch (/model)

The Web's /model command changes models the way the /agent handoff does: picking a model stages it in the composer, and sending creates a new Session under the same Agent via the ordinary session-creation API (the chosen model, the source session's Workspace — so files stay reachable), and the first message opens with a [model_switch_from] source block — the source session id, the absolute path of its latest Trace file, the Workspace, and the previous model pair — followed by whatever the user typed. The history is not injected into the new context: some models require thinking payloads and fidelity byte-for-byte when history is replayed, which cannot cross models — instead the model reads the source Trace file itself (JSONL, one message envelope per line) when it needs the earlier context. The source session and its Trace are untouched.

Field fidelity

Each content message's opaque provider fidelity payload (thinking signatures, phase labels, encrypted reasoning, …) is preserved verbatim in the Trace and sent back verbatim — some models require it byte-for-byte on history replay, and any rewriting would break compatibility. This is one reason the Trace stores raw OmniMessage envelopes rather than a post-processed format.

Observability

Every approval decision (approval_decision), abort (abort), compaction (compaction_begin / compaction_end), and token_usage lands in the Trace as an event. The Web Trace view and the usage/cost statistics are both derived from this same data — there is no second source of truth; see the Web App Guide. The approval mechanism itself is covered in Tools & Approval. Trace files can also be moved across deployments from the Web Traces page: any file can be exported (downloaded verbatim as JSONL) and imported back under an Agent — an import whose session id already exists under that Agent is rejected, so an imported file always becomes index 001 of a new Session.