Skip to content

Fix trail's save-a-learning prompts - #739

Open
anirudhkumar-nanonets wants to merge 2 commits into
mainfrom
claude/fix-save-learning-prompts-84888d
Open

anirudhkumar-nanonets wants to merge 2 commits into
mainfrom
claude/fix-save-learning-prompts-84888d

Conversation

@anirudhkumar-nanonets

@anirudhkumar-nanonets anirudhkumar-nanonets commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Issue: In Claude Desktop, a user's correction never led to a "save a learning" ask, and after a push the Stop hook said no learning went with the work when one had.
Why: Desktop puts a <system-reminder> status block before every typed message and the hook's typed() dropped anything starting with <; its advice also sent business reasoning to the repo, and it counted writes outside the repo as code edits.
Fix: The hook now reads typed words through the shared transcript filter, knows more "keep it out of the repo" phrasings, says business reasoning goes in --personal, and ignores writes outside the repo or pushes whose branch already adds a learning.

Moves: share of user corrections that end in a saved learning. Measured by the new note_asked event in PostHog (Metabase db 11, posthog_event_full_view): kind = corrected with saved in (repo, personal) over all kind = corrected. Value today: none, nothing measured it before this PR (Metabase checked 2026-10-29).

What changed

  • typed() uses typedText (src/trace/claude.ts), the filter every other learning path already uses. It strips <system-reminder> and IDE wrappers and keeps the words after them, and it never counts compaction summaries (isCompactSummary, or text starting "This session is being continued"), task notices or other sessions' messages (origin.kind other than human) as typed. A queued message typed while Claude was busy now counts too.
  • Artifact comments: ownText opens "[Artifact comment sent to Claude]" to its quoted comment between the BEGIN/END lines. Before, the whole harness wrapper counted as typed on every learning path.
  • Correction words: adds na, need not / needn't go|be, shouldn't go, keep it out of the repo, not in the repo (not when the sentence is a question). Tests cover look-alikes that must stay quiet: "is this not in the repo yet?", "you need not ask…", "nan values…", "naming is fine".
  • Repo vs personal advice: session start (format.ts), the trail skill (skill-template.ts) and the corrected and shipped asks now say business, growth, sign-up, pricing or competitor reasoning and internal numbers go in --personal, never the repo, "even if you also keep your own copy" (the answer to why Claude's own memory got the rule and trail didn't). The words test bans "memory" in agent-facing text, hence "your own copy".
  • False "no learning went with it": shellEdit returns no edit when a command starts with cd outside the repo, or when every full path it names is outside the repo (a scratch PR body via $P, Claude's memory index). A write it can't name from inside the repo still counts. The shipped sign is skipped when the branch's commits already add a .trail/learnings/ file (branchAddsLearning in src/cloud/diff.ts, next to branchDiff).
  • Telemetry: note_asked {kind, saved: repo/personal/no}, sent on the stop after an ask (the stop_hook_active stop). askAnswer only counts it when the last Stop hook feedback is trail's own ask, so another hook's block isn't counted. Listed in contract.ts, its allowed values, TELEMETRY.md and the pinned event list.

Choices made

trail check

Exits 1 with 4 hits, all word overlaps with unrelated learnings, not conflicts: two about hooks going quiet under a sandboxed claude -p (matched on "claude", "stop", "hook"), one about the hook shim with a real HOME (matched on "copy"), one about replaces: holding ids (matched on "file name").

Tests

  • test/trail-note-ask.test.ts, test/trace-typed.test.ts, test/telemetry-contract.test.ts, test/trail-words.test.ts, test/claude-stop.test.ts: 61 pass.
  • Full suite with a scratch HOME: 142 failing on this branch vs 143 on origin/main (dd0e6cb), the same parser/extraction set from the worktree resolving the main checkout's node_modules. Nothing new fails.
  • After merging main (a44da3d, Fix npm downloads showing 0 after the rename #723 and Install trail without sudo #734): no conflicts. skill-template.ts and the skill fixture keep both this PR's --personal lines and Install trail without sudo #734's no-sudo lines. Typecheck is clean, and the tests above plus cli-meta, node-version, claude-shim-resolve, trail-notes and skills-fixtures pass (130).

🤖 Generated with Claude Code

github-actions Bot added a commit that referenced this pull request Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

🌱 graft blast radius

4 areas changed → 11 areas can be affected. 39 dependent symbols, depth 2.
Tests: no test reaches Trace Claude, Branch Learning Diff; 1 area updated its tests.
Tag: @shhdwi — 14 of 15 areas · @afeddersen — Telemetry Tracking

flowchart TB
  A0(("Hook Initialization<br/>15 symbols"))
  A1(("Session Replay<br/>7 symbols"))
  A2(("Memory Notes<br/>3 symbols"))
  A3(("Cloud Activity<br/>3 symbols"))
  A4(("CLI Skills<br/>2 symbols"))
  AX(("6 smaller areas<br/>9 symbols"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2,A3,A4 reached;
  classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
  class AX tail;
Loading
Can be affected Symbols Nearest hop Reached from
Hook Initialization 15 src/claude/hooks.ts:L474-L484 orientationAtStart — calls, depth 1 Claude Formatting, Telemetry Contract
Session Replay 7 src/init/keep.ts:L1-L387 keep.ts — imports, depth 1 Telemetry Contract, Trace Claude
Memory Notes 3 src/notes/memory-notes.ts:L1-L280 memory-notes.ts — imports, depth 1 Telemetry Contract, Trace Claude
Cloud Activity 3 src/cloud/nudge.ts:L1-L505 nudge.ts — imports, depth 1 Telemetry Contract
CLI Skills 2 src/skills/cli-skills.ts:L1-L376 cli-skills.ts — imports, depth 2 Telemetry Contract, Trace Claude
Host Registry 2 src/hosts/antigravity.ts:L43-L61 installAntigravitySkill — calls, depth 2 Claude Formatting
5 smaller areas 7 Learning Sessions, Telemetry Tracking, CLI Entry Point, … see below
Who knows this code — 2 people across 15 areas
Area Who knows it
Telemetry Contract · changed @shhdwi — 1 commit, last 10d ago
Trace Claude · changed @shhdwi — 1 commit, last 11d ago
Claude Formatting · changed @shhdwi — 18 commits, last 6d ago
Hook Initialization · affected @shhdwi — 24 commits, last 6d ago
Session Replay · affected @shhdwi — 12 commits, last 6d ago
Memory Notes · affected @shhdwi — 5 commits, last 6d ago
Cloud Activity · affected @shhdwi — 22 commits, last 6d ago
CLI Skills · affected @shhdwi — 3 commits, last 7d ago
…7 further areas

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 39 dependent symbols, grouped by area

Hook Initialization — 15 symbols in 12 files

  • src/claude/hooks.ts:L474-L484 — orientationAtStart (calls, depth 1)
    482: return learnings.kept ? `${tag()} ${teamNotesDirective(learnings.count, true)}${learnings.extra}` : learnings.extra;
  • src/claude/hooks.ts:L1-L580 — hooks.ts (imports, depth 1)
    6: import { formatBlastRadius, formatOrientation, notesOnTouch, standingDirective, teamNotesDirective, type Touch } from './format.js';
  • src/claude/session-metrics.ts:L1-L215 — session-metrics.ts (imports, depth 1)
    121: * record (kind null and no savings), so a host can call it unconditionally after
  • src/claude/skills-wiring.ts:L1-L116 — skills-wiring.ts (imports, depth 1)
    46: /** 3.1, after the learnings the start text already has. */
  • src/claude/state.ts:L1-L165 — state.ts (imports, depth 1)
    30: * nothing the person typed is kept (spec section 6). Optional so files
  • src/claude/hook-io.ts:L1-L140 — hook-io.ts (imports, depth 2)
    36: /** The graft child's JSON answer, or null. `input` goes to it on stdin: for anything the person typed, which argv would show to `ps`. */
  • src/claude/hooks.ts:L495-L579 — main (calls, depth 2)
    538: // The skills context comes after the learnings; its notice is for the person.
  • src/claude/init.ts:L133-L181 — runInit (calls, depth 2)
    135: opts: { build?: boolean; cliPath?: string; statusline?: boolean; global?: boolean; home?: string; brand?: Brand } = {},
  • src/claude/init.ts:L194-L245 — runTrailInit (calls, depth 2)
    196: opts: { build?: boolean; cliPath?: string; statusline?: boolean; global?: boolean; home?: string },
  • src/claude/prompt-hook.ts:L1-L307 — prompt-hook.ts (imports, depth 2)
    2: * The prompt hook: what the agent reads alongside what the person just typed.
  • src/claude/prompt-skills.ts:L1-L327 — prompt-skills.ts (imports, depth 2)
    4: * skill a prompt brings back. `handlePrompt` calls it right after the
  • src/claude/skill-use.ts:L1-L80 — skill-use.ts (imports, depth 2)
    15: import { repoPlace } from "../notes/home.js";
  • src/claude/stats.ts:L1-L27 — stats.ts (imports, depth 2)
  • src/claude/statusline.ts:L1-L39 — statusline.ts (imports, depth 2)
    6: import { noteCount } from '../notes/home.js';
  • src/claude/sync-run.ts:L1-L48 — sync-run.ts (imports, depth 2)

Session Replay — 7 symbols in 6 files

  • src/init/keep.ts:L1-L387 — keep.ts (imports, depth 1)
    13: * committed `.trail/learnings/` after `r`.
  • src/init/run.ts:L1-L445 — run.ts (imports, depth 1)
    13: * the agent's line (B9). `--yes` there keeps only the person's own typed
  • src/init/keep-list.ts:L161-L203 — afterList (calls, depth 2)
    165: let typed: AfterList['typed'] = null;
  • src/init/keep-list.ts:L210-L224 — typedLine (calls, depth 2)
  • src/init/replay-runs.ts:L445-L478 — typedAt (calls, depth 2)
  • src/init/review-screen.ts:L1-L316 — review-screen.ts (imports, depth 2)
    3: * relay an agent or a script gets, and the lines after a keep. Pure: no
  • src/init/transcript.ts:L171-L223 — scanTranscript (calls, depth 2)
    187: // A script's run, unless a lab home has the switch (D99, D136 (2)), as the hooks read it.

Memory Notes — 3 symbols in 3 files

  • src/notes/memory-notes.ts:L1-L280 — memory-notes.ts (imports, depth 1)
    23: import { ensureRepoHome } from "./home.js";
  • src/notes/pr-reviews.ts:L1-L577 — pr-reviews.ts (imports, depth 1)
    32: import { ensureRepoHome } from "./home.js";
  • src/notes/always-follow-gate.ts:L39-L63 — typedLines (calls, depth 2)

Cloud Activity — 3 symbols in 3 files

  • src/cloud/nudge.ts:L1-L505 — nudge.ts (imports, depth 1)
    26: import { keepsNotes, knownRepos, noteCount, repoName, repoPlace, trailHome } from "../notes/home.js";
  • src/cloud/lookup-activity.ts:L1-L60 — lookup-activity.ts (imports, depth 2)
    7: import { repoPlace } from "../notes/home.js";
  • src/cloud/pair.ts:L1-L368 — pair.ts (imports, depth 2)
    19: import { repoName, repoPlace } from "../notes/home.js";

CLI Skills — 2 symbols in 2 files

  • src/skills/cli-skills.ts:L1-L376 — cli-skills.ts (imports, depth 2)
    15: import { repoPlace } from '../notes/home.js';
  • src/skills/offer-lines.ts:L33-L50 — typedLineAt (calls, depth 2)
    43: if (use && use.dialect.host !== 'claude') {

Host Registry — 2 symbols in 2 files

  • src/hosts/antigravity.ts:L43-L61 — installAntigravitySkill (calls, depth 2)
    43: export function installAntigravitySkill(home: string, name: Brand = 'graft'): ConfigWrite[] {
  • src/hosts/registry.ts:L1-L186 — registry.ts (references, depth 2)
    17: home: string;

Learning Sessions — 2 symbols in 1 file

  • src/learn/sessions.ts:L1-L310 — sessions.ts (imports, depth 1)
    6: * typed, Claude's replies, the questions the person answered, plans, and the
  • src/learn/sessions.ts:L57-L110 — sessionLines (calls, depth 2)
    81: const typed = typedText(r);

Telemetry Tracking — 2 symbols in 2 files

  • src/telemetry/sessions.ts:L1-L113 — sessions.ts (imports, depth 1)
    51: opts: { host?: AgentHost; home?: string; env?: NodeJS.ProcessEnv } = {},
  • src/telemetry/track.ts:L1-L162 — track.ts (imports, depth 1)
    19: * key's values (`VALUES`), a value outside them is dropped too.

CLI Entry Point — 1 symbol in 1 file

  • src/cli.ts:L1-L2844 — cli.ts (imports, depth 2)
    28: import { checkoutRoot, ensureRepoHome, keepsNotes, noteCount, rememberRepo, repoName, repoPlace, shownPath } from "./notes/home.js";

MCP Tools — 1 symbol in 1 file

  • src/mcp/tools.ts:L1-L337 — tools.ts (imports, depth 2)

Trace Dialect — 1 symbol in 1 file

  • src/trace/dialect.ts:L114-L121 — typedOnLine (calls, depth 2)
    118: if (ev.kind === 'typed' && use.dialect.typedByPerson(ev, use.st, env)) return ev.text;
Test signal per changed area — 1 ✓ · 2 ✗ · 1 –

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • – Telemetry Contract — 1 test file changed here: test/telemetry-contract.test.ts
  • ✗ Trace Claude — 0 of 1 reached · no test file reaches it
    • not reached: ownText
  • ✓ Claude Formatting — 4 of 10 reached · 1 test file changed here: test/trail-note-ask.test.ts
    • not reached: home, typed, outside, shellEdit, trailSkill, askForNote
  • ✗ Branch Learning Diff — 0 of 1 reached · no test file reaches it
    • not reached: branchAddsLearning
116 test suites also reference this code

117 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/agent-trust-merge-1b.test.ts
  • test/claude-copy-dashes.test.ts
  • test/claude-format.test.ts
  • test/claude-hooks-1b.test.ts
  • test/claude-hooks-perf.test.ts
  • test/claude-hooks-skills.test.ts
  • test/claude-hooks.test.ts
  • test/claude-host-standdown.test.ts
  • test/claude-post-bash-walls.test.ts
  • test/claude-prompt-learnings.test.ts
  • test/claude-prompt-private.test.ts
  • test/claude-session-start-learnings.test.ts
  • test/claude-skill-template.test.ts
  • test/claude-skills-journey-fixtures.ts
  • test/claude-start-skills-1b.test.ts
  • test/claude-start-skills-2.test.ts
  • test/claude-state.test.ts
  • test/claude-statusline.test.ts
  • test/claude-stop-moments.test.ts
  • test/claude-stop-trace.test.ts
  • …96 more

⚠️ 3 changed files not in the graph (.trail/learnings/2026-10-29-the-stop-hook-reads-what-the-anirudhkumar.md, TELEMETRY.md, test/fixtures/skills/trail/SKILL.md) — no parser claims the extension, or the index predates the file.

graft blast · origin/main...HEAD · depth 2 · 13 changed files

Open the interactive graph → — click an area to see its dependent symbols at file:line.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant