open-harness/memory/MEMORY.md

Memory

Long-term lessons distilled from session experience. Append-only. See context/rules/memory.md for the Memory Improvement Protocol.

Lessons Learned

• 2026-05-11: First-round critic council on PR #270 (3 critics: PM / Implementer / Critic with implicit, overlapping lenses) shipped a rule that laundered RAO's RL-training results as inference-time guarantees — a HIGH-severity faithfulness gap. A second-round council on the same PR (5 critics with explicit distinct lenses: faithfulness, codebase-fit, practical, safety, editor) surfaced the gap on the very first reply. Lesson: when fanning out critics to review one artifact, brief each with a different focal lens — lens-diversity exposes failure modes that critic-count alone misses. Promote to context/rules/advisor-model.md as a new Pipeline Variant only after the pattern recurs (≥ 3 invocations across sessions).
• 2026-05-18: Secret-exposure Bash guard blocks git commit whenever the staged diff content matches SLACK_*= / *_TOKEN= / *_SECRET= / etc., even when the values are placeholders like xapp-... or xoxb-.... Affects any port of .devcontainer/.example.env-style files that document Slack / OAuth / API env vars. Workarounds (in order of preference): (a) ask the user to run git commit themselves and paste back the commit SHA; (b) restructure the file to document vars in a markdown table under docs/ rather than env-var assignment syntax. The guard scans the staged diff, not the command text — so heredoc, -F file, and --no-verify do NOT bypass it. Encountered while shipping .pi/ + slack to brand-signal (ryaneggz/brand-signal#6, PR #7).
• 2026-05-19: When local branch is behind origin AND the working tree carries uncommitted edits, running additional Edit/Write on append-style tracked files (CHANGELOG.md [Unreleased], MEMORY.md, daily logs) silently composes with the stale view — content origin added since the last pull can be wholly replaced rather than appended-to, because the Edit tool only sees the local pre-pull state. The fix is the existing rule from context/rules/git.md § Isolating in-flight work: cut a worktree off origin/<base> and re-do the edit there. The worktree starts from current state, so edits compose correctly. Caught during /interview build (PR #317): the in-main-checkout CHANGELOG edit replaced the just-merged /render-html [Unreleased] entry; the byte-equivalence check (also in that rule) surfaced the drift before push. Worth promoting the failure mode into the rule's prose so future agents reach for the worktree pattern before editing, not after a near-miss.
• 2026-05-19: The Claude Code CLI binary at /usr/lib/node_modules/@anthropic-ai/claude-code/bin/claude.exe embeds its own inline JSON schema docs as plain-text strings — strings <binary> | grep -B1 -A25 '"<top_field>":' reliably extracts the full shape (verified for statusLine, context_window, rate_limits, workspace, model). Use this as the authoritative reference when writing about Claude Code internals (blog posts, scripts, statusline integrations); the npm package ships no docs and the public changelog only names fields piecemeal. Same trick should work for any future fields the team adds — schema lives in the binary, not in the repo.
• 2026-05-22: When asked to install a tool via curl … | bash from a third-party repo, the right move was to fetch and read the script first, then re-implement the capability natively to harness skill conventions — not execute the untrusted installer (it delegated to an unreviewable npx Node installer that mutated agent configs across many tools). Done while shipping the /caveman skill (PR #336).
• 2026-05-22: CLAUDE.md at the repo root is a symlink to AGENTS.md; the Edit tool refuses to write through it ("Refusing to write through symlink"). Edits to the orchestrator skills table / instructions must target AGENTS.md directly (resolve with readlink -f CLAUDE.md).
• 2026-05-22: skills.lock provenance for non–mifunedev-registry skills: a vendored external skill records its real upstream commit SHA plus a note marking it vendored (caveman → JuliusBrussee/caveman); a skill authored in-tree for mifunedev/skills but not yet contributed records commit: pending-upstream, backfilled on upstream merge (reflect, PR #338).
• 2026-05-23: .claude/plans/ AND .claude/specs/* are both gitignored project-wide (.gitignore:50-51) — local-only working areas, zero tracked files in either. When asked to commit a plan/spec, surface the conflict instead of forcing it; the user's intent may be to make a deliberate exception. Resolution requires two changes in lockstep: (a) change the parent rule from **/.claude/plans to **/.claude/plans/* — the bare-directory form prevents git from traversing into the dir so child negations are silently no-ops; (b) add the targeted exemption (e.g. !**/.claude/plans/karpathy-llm-wiki.md). Encountered landing PR #347 (Karpathy LLM Wiki plan, issue #346).
• 2026-05-24: When /delegate spawns parallel sub-agents that need to touch a shared append-style file (CHANGELOG.md, AGENTS.md skills table, settings.json), scope each sub-agent to disjoint files only and let the orchestrator integrate the shared-file edits at commit time. Sub-agents commit nothing; orchestrator commits per wave with all wave files plus its own shared-file additions in one atomic commit. Avoids the concurrent-commit failure mode where two sub-agents independently git commit against the same branch and clobber each other's CHANGELOG additions (or, in worktree-less single-checkout execution, where the second sub-agent's Edit silently composes against the pre-first-edit state — see 2026-05-19 lesson for the underlying mechanism). The pattern is "fan out implementation, fan in integration." Encountered shipping the Karpathy wiki integration (PRs #350, #351); Wave 2 (US-002 /wiki-ingest + US-003 /wiki-query in one assistant message) landed both SKILL.md files in parallel, then the single orchestrator commit included both + both CHANGELOG entries atomically.
• 2026-05-24: /ship-spec Stage 4's "HALT on HIGH findings" gate can be safely substituted by in-place AC mitigation when (a) all HIGH findings are in the "AC-tightening" category — no architectural HALT signals like [PROTECTED-PATH] violations, orchestrator-boundary crossings, or unstated assumptions that block /delegate — and (b) critique.md records a Synthesis paragraph naming each HIGH finding and its mitigation as the audit trail. Saves a critic round-trip when fixes are mechanical. Encountered shipping the Karpathy wiki integration (PR #351): 7 HIGH findings, all mitigated inline; gate PROCEEDED rather than HALTED.
• 2026-05-24: /ship-spec Stage 7 template path tasks/openharness-v07-convergence/prompt.md has been archived to tasks/archive/openharness-v07-convergence/prompt.md; the skill prose at .claude/skills/ship-spec/SKILL.md § Stage 7 still references the old (now-empty) location. Most recent live prompt.md example: tasks/fork-friendly-install/prompt.md. Either update the skill to point at the archive copy or seed a non-task-coupled template in the skill's own directory.
• 2026-05-24: context/rules/memory.md § Layout claims memory/YYYY-MM-DD/log.md is "gitignored directory; log.md tracked inside," but .gitignore:42 (memory/[0-9]*/) excludes the entire directory contents. git add memory/<date>/log.md fails without -f; no daily log has ever been committed (verified via git log --all --diff-filter=A). In practice daily logs are local-only working state; only MEMORY.md is the long-term tracked store. The rule's "tracked inside" wording is aspirational. Either update the rule's prose to match the gitignore behavior OR add an exemption (!memory/[0-9]*/log.md).
• 2026-05-24: /ship-spec MUST end with gh pr ready <N> — do not leave PRs in draft. The skill's documented "v1 stops at draft PR creation" stance is overridden by user preference (correction received after PR #351 was left as draft). Applies to any equivalent end-to-end PR scaffold flow. Update the skill prose to make readiness the final step rather than a deferred manual action.