docs(changelog): truncated-output recovery and unified installer entries (#149)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

changelog/unreleased/2026-07-31-backward-compatibility.md

@@ -0,0 +1,27 @@
+# Backward compatibility in this batch
+
+Per the repo rule, every compatibility decision of the batch is recorded here once; the feature entries reference this file instead of re-telling it.
+
+## Installers keep a legacy program-archive path for pre-0.1.6 releases
+
+From this batch on, the canonical Release artifact is a flat installer bundle whose payload sits inside as `payload.tar.gz` / `payload.zip`. Releases up to v0.1.5 shipped the program tree directly (top-level `penguin/`) under the same asset names, and users can still pin those versions or hold such files locally.
+
+**Old shape tolerated:** any archive without a top-level `payload.*` is treated as a program archive. `--version` pins at pre-0.1.6 releases download the old raw asset and install through this path; `--archive` / `-ArchivePath` accepts a legacy raw archive (adjacent `.sha256` still required, renamed files still fall back to the canonical asset checksum plus the manifest rule) as well as the new bundle or its inner payload.
+
+**Scope:** the shape probe and the program-archive branch in `install.sh` and `install.ps1` only. Release packaging, CI and docs describe exclusively the new shape; no on-disk user data is involved (`~/.penguin/data` was never part of install or upgrade).
+
+**User action:** none for the documented flows.
+
+**Removal:** the legacy branch should stay while pre-0.1.6 releases remain plausible `--version` targets. Whoever prepares the release that drops support for installing pre-0.1.6 versions deletes the program-archive branch and the `tar -tzf` / zip-entry probes with it; the sites in both installers reference this file.
+
+## Installer scripts saved from releases up to v0.1.5 cannot install newer assets
+
+This is a deliberate incompatibility, not a handled one: the old installers download the asset under the same name, pass the outer checksum, then fail with their "unexpected archive layout: top-level penguin/ missing" error because the new artifact is a bundle. The documented install commands are unaffected — the penguin.ooo forwarders fetch the installer attached to the latest Release at run time — and the fix for a saved old script is to re-fetch it. Old releases' own assets are untouched: an old installer pinned at an old version keeps working forever.
+
+## The Windows payload drops its penguin.ps1 launcher
+
+This is a deliberate removal with a self-cleaning upgrade path, not a tolerated old shape. Typing `penguin` is unaffected — PowerShell and cmd.exe both resolve `bin\penguin.cmd`, which is exempt from the execution policy that used to block the `.ps1` launcher on default client Windows. Only direct `.\penguin.ps1` invocations need to switch to `penguin` / `penguin.cmd`. Installs made before this change keep their old launcher until the next upgrade: the installer swaps `bin\` wholesale, so re-running it removes the stale `penguin.ps1` along with the rest of the old `bin\` — which is also the fix for machines already showing the "running scripts is disabled" error. Nothing reads or regenerates the file anymore; no removal follow-up is needed.
+
+## The recover-truncated-tool-output batch needs no handling
+
+`EnvironmentConfig.sessionScratchpadDir` is additive and optional: standalone SDK embedders that never pass it keep the exact previous truncation behavior, existing Traces replay unchanged, and no stored format changes shape. Recorded here only to state that the check was made.

changelog/unreleased/2026-07-31-recover-truncated-tool-output.md

@@ -0,0 +1,15 @@
+# Core: truncated tool output remains recoverable within its Session
+
+Tool output still obeys each entry's `maxOutputLength` exactly as before: the Web/CLI stream, the complete `tool_call_output`, Trace, and the next model input remain byte-for-byte aligned, with truncation and terminal markers staying outside the cap.
+
+When a tool call in an Agent Session actually exceeds that cap, Environment now also saves the text it received to the Session scratchpad (`scratchpad/<sessionId>/truncated-tool-output/`, created only on actual truncation, private permissions where the platform has them) and appends the recovery path to the same tool result seen by the frontend and the model. No model-facing tool or schema is added: the model inspects the file with the existing `read_file` (`offset`/`limit`) or targeted `rg`/`tail` commands.
+
+The wiring is one generic Environment parameter rather than a feature-specific manager: `EnvironmentConfig.sessionScratchpadDir` names the Session-scoped storage root, Environment derives `truncated-tool-output/` from it internally, and the new `sessionScratchpadDir()` path helper becomes the single definition of `scratchpad/<sessionId>` (input images and the goal file already lived there and now derive from the same helper). Agent composition passes it automatically; standalone SDK embedders omit it and keep the previous truncation-only behavior, or opt in by passing a directory — no archive class crosses the public config surface.
+
+The model-visible path is a plain absolute path, always the last element of the note — and the spelling rule now covers every path core composes for the model, through one `modelVisiblePath` helper exported from the package barrel: the system prompt's App Data Dir and CWD lines, `[attached image: …]` lines, the server's `[attached file: …]` lines, and the goal-file line. On Windows that spelling uses forward slashes: `exec_command` runs through (Git) Bash and Node's fs APIs accept them, so the model can re-emit the same spelling into JSON tool arguments and shell commands without backslash-escaping mistakes; harness paths are ordinary absolute paths (never `\\?\`-prefixed), so the swap is lossless, and POSIX paths pass through unchanged.
+
+File tools stay compatible with Windows spellings on the consuming side: `path.resolve` accepts `C:\…` and `C:/…` alike (pinned by win32-only tests), tool messages echo the caller's own spelling, and the one system-composed path in tool output — `read_file`'s not-found workspace hint — uses the model-visible spelling too. The Web App's file cards follow suit: workspace-prefix stripping previously required the assistant's separators to match the Workspace's exactly, so on a Windows Workspace the forward-slash spelling produced no card at all; it now compares both sides on `/` with a case-insensitive drive letter, while POSIX Workspaces still never touch backslashes (legal filename characters there).
+
+Each call archives at most 8 MiB — one byte under `read_file`'s 8 MiB scan cap, so the tool's final zero-byte read can still confirm end of file. Larger calls keep UTF-8-safe head/tail windows with an explicit `[archive middle truncated]` gap. Capture memory is bounded (Buffer chunks while exact, a fixed-capacity ring tail after promotion), surrogate pairs split across stream deltas are rejoined, and this is a per-call bound only: a Session has no aggregate archive quota. Recovery preserves what Environment received; text a producer already dropped upstream (e.g. an `[..., N chars of earlier output dropped ...]` marker from a bounded unread buffer) is not recoverable.
+
+Recovery files contain unredacted tool text and live and die with the Session scratchpad — the existing explicit Session-deletion path removes them; no separate cleanup lifecycle is added, which can increase local at-rest retention of accidentally read sensitive data. Trace stores the same truncated result and absolute path, not a second copy of the archived bytes. An archive-write failure never changes the tool's `stop_reason`; the visible note and stderr warning carry a short errno code, not the path or raw error.

changelog/unreleased/2026-07-31-unified-installer-artifact.md

@@ -0,0 +1,15 @@
+# Tooling: one canonical install bundle per target, for online and offline alike
+
+Each Release now attaches exactly one artifact per target: `penguin-{linux,darwin}-{x64,arm64}.tar.gz`, `penguin-universal.tar.gz` and `penguin-win32-x64.zip` are flat installer bundles, each sealing the native installer, the program payload (`payload.tar.gz` / `payload.zip`) and the payload's SHA256 checksum. Raw program archives and the five `*-offline` wrappers introduced in 0.1.5 are no longer published; `SHA256SUMS` covers exactly the six bundles.
+
+Both installation modes consume the same file. Online, `install.sh` / `install.ps1` download the bundle, verify it against its published `.sha256` — checksums are now always mandatory; the online warn-and-skip fallback is gone — then open the flat outer layer and verify the payload checksum sealed inside before staging. Offline, the user transfers that one file, extracts it once and runs the bundled `./install.sh` (or double-clicks `install.cmd`); the installer finds the sibling payload by itself and verifies the same sealed checksum with no network access, so no separate checksum file needs to be carried.
+
+Archive shape is probed by content, not filename: a top-level `payload.*` means bundle, anything else is treated as a program archive with a top-level `penguin/`. That single probe replaces the previous online/offline split and keeps pinned pre-0.1.6 versions and legacy local archives installable (see the batch's [backward-compatibility notes](2026-07-31-backward-compatibility.md)). On Windows, the flat outer layer means the initial extraction never creates deep paths — PowerShell 5.1's 260-character limit stops being reachable from the download directory — and the zip shape probe reads the entry list without extracting; only the installer expands the payload, straight into its short staging directory.
+
+The canonical installers travel byte-identical inside the bundles — the generated POSIX wrapper entry (`penguin-installer.sh` + stub `install.sh`) is gone, replaced by sibling-payload detection in the one real installer. That detection is hardened to keep 0.1.5's "never trust archives beside a temporary script" property: it engages only for a real file actually named `install.sh`, and the penguin.ooo forwarder now stages its download in a private `mktemp -d` directory rather than a bare file in shared `/tmp`. The Windows forwarder already runs the installer as an in-memory script block and keeps its sibling detection offline-only.
+
+In-place upgrades stop failing on filesystems that pin in-use directories. `penguin update` re-runs the installer while the CLI process is still executing out of `lib/`; on overlayfs under Docker the plain `mv lib .old.<pid>/lib` failed with `Device or resource busy` and rolled back. Directory moves now degrade through three strategies — whole-directory rename, per-entry renames into a fresh or existing destination, then per-entry copy with the source cleared (POSIX keeps an unlinked-but-open file valid for the process using it) — with a pinned, empty directory husk reused by the incoming move, rollback following the same rules, and a failure message that says to stop running penguin processes. A hermetic test stubs `mv`/`rmdir` to refuse directory renames touching the installed `lib/` and verifies the upgrade completes cleanly.
+
+Two Windows papercuts are fixed at the root. The payload no longer ships a `penguin.ps1` launcher: PowerShell prefers `.ps1` over `.cmd` on PATH and client Windows defaults to the Restricted execution policy, so a fresh install's plain `penguin` command failed with "running scripts is disabled"; batch files are policy-exempt, both PowerShell and cmd.exe resolve `penguin.cmd`, and since upgrades swap `bin\` wholesale, re-running the installer also removes the old shim (see the batch's [backward-compatibility notes](2026-07-31-backward-compatibility.md)). And after appending to the user Path registry value, the installer now broadcasts `WM_SETTINGCHANGE` — a raw registry write alone left Explorer, and every terminal launched from it, on the stale Path until the next logon, which is why a brand-new PowerShell window still could not find `penguin`; the completion note now says to open a new terminal window rather than a new tab.
+
+`scripts/package-release-bundles.sh` replaces `package-offline-bundles.sh`, and the release workflow now builds payloads as intermediates, packages the six bundles, and validates the real outputs before upload: outer checksum, exact flat member set, byte-identical installer and payload, and a passing sealed checksum. `scripts/test-installer.sh` and `scripts/test-installer.ps1` replace the offline-only test scripts and hermetically cover bundle layout, offline installs with network access stubbed to fail, corrupted-payload rejection, upgrade rollback, both checksum layers, no-fallback download failures and pinned legacy versions, wired into the existing Linux and Windows CI jobs. README, the docs Installation page and the landing install copy describe the single-artifact flow.

changelog/unreleased/2026-08-02-update-check-feedback.md

@@ -0,0 +1,3 @@
+# Web App: the manual update check reports every outcome
+
+The sidebar user menu's "Check for updates" row used to notify only when nothing changed visibly; finding a newer release changed the row silently. The manual check now gives explicit feedback for every outcome: a spinner and busy label while the check runs, a success toast when already up to date, and a success toast naming the release when a newer one is found — at which point the same row turns into the persistent "New version vX available" entry that opens the update dialog (release-notes link, admin-only self-update). Failed lookups and disabled checks keep their error/info toasts. The outcome mapping is a pure, unit-tested classifier: disabled beats failed beats found, and "found" must name a version so neither the toast nor the row is ever blank.

changelog/unreleased/README.md

@@ -2,6 +2,14 @@
 
 Changes since v0.1.5. The version number is assigned at release, when this folder is renamed.
 
+- [2026-08-02] Web App: the manual "Check for updates" row reports every outcome — busy spinner while checking, success toasts for both up-to-date and update-found (naming the release, with the row itself becoming the update entry), and the existing failure/disabled notices — via a unit-tested outcome classifier. ([details](2026-08-02-update-check-feedback.md))
+
+- [2026-07-31] Tooling: each Release now attaches exactly one artifact per target — a flat installer bundle sealing the native installer, the program payload and its checksum — serving online and offline installation from the same file, with mandatory checksums at both layers, no more raw archives or `*-offline` wrappers, and hermetic installer tests in CI; in-place upgrades survive filesystems that pin in-use directories (the `penguin update` overlayfs `Device or resource busy` failure), the Windows payload drops its policy-blocked `penguin.ps1` launcher, and the installer broadcasts the user-Path change so new terminal windows find `penguin`. ([details](2026-07-31-unified-installer-artifact.md))
+
+- [2026-07-31] Backward compatibility: the batch's compat decisions in one place — installers keep a content-probed legacy path so pre-0.1.6 releases and old local archives stay installable (with its removal owner named), old saved installer scripts break loudly against new assets by design, the dropped `penguin.ps1` launcher cleans itself up on upgrade, and the truncated-output batch is additive with nothing to migrate. ([details](2026-07-31-backward-compatibility.md))
+
+- [2026-07-31] Core: tool output that exceeds `maxOutputLength` in an Agent Session is now saved to the Session scratchpad and the recovery path appended to the same truncated result the model and frontends see — no new tool, no change to the visible cap or the stream-equals-complete contract — wired through one generic `EnvironmentConfig.sessionScratchpadDir` parameter, with per-call 8 MiB bounds and UTF-8-safe head/tail for larger calls; every model-visible path core composes (prompt Environment lines, attachment lines, goal file, tool messages, recovery notes) now shares one forward-slash spelling on Windows via `modelVisiblePath`, file tools keep accepting both Windows spellings, and the Web App's file cards match Windows paths in any separator spelling. ([details](2026-07-31-recover-truncated-tool-output.md))
+
 - [2026-07-31] Evaluation Center: Case details separate Target Agent task materials from project-member-visible scoring rubrics, Score charts use a padded dynamic axis without discarding stored values, and the benchmark Skills write YAML-safe Scoreboard summaries. ([details](2026-07-31-evaluation-center-case-details.md))
 
 - [2026-07-30] Web App: the main conversation's file summary moves to the completed Task boundary — one card per Task scanning all of its assistant text, nested agent conversations keep their per-message summaries, and file-existence caching stops retaining negative results so a later Task can surface a newly created path. ([details](2026-07-30-file-summary-task-boundary.md))