Flag learnings that name code that's gone by anirudhkumar-nanonets · Pull Request #731 · trailhq/Trail-Code

anirudhkumar-nanonets · 2026-10-29T06:22:35Z

Issue: committed learnings name files and functions that were deleted since, and Claude was told to follow them as written.
Why: trail showed every learning as saved; nothing compared the code a learning names with the code that's there now.
Fix: each learning about to be shown has its touches: checked against the code in the checkout (no LLM call); one that names gone code is marked "may be stale", with what's gone and trail note --replaces <id>. The same list goes to Trail Cloud at sync.

Moves: learnings reused, measured as query events whose notes is 1 or 2 (a trail ask that showed learnings), PostHog via Metabase db 11 (posthog_event_full_view): 136,557 from 1,074 people in the 7 days to 2026-10-29 12:15 IST, out of 193,148 asks in repos that keep learnings. This PR adds stale (0/1/2) next to notes, so from the next release the same event counts how many shown learnings were stale. (learning_reused isn't used: it left the code in #658.)

What Claude sees

from Priya's learning, a teammate's · Sep 2 · Retries to the card gateway go through withJitter, never a fixed delay
  may be stale  src/retry.ts#withJitter is gone from the code. Check it against the code before you follow it; if it no longer holds, save what's true now with trail note --replaces l-5e1f20aa
  decided     Every retry to the card gateway waits `withJitter(attempt)` (src/retry.ts): exponential…

Shown in: trail ask, MCP trail_find_code, the prompt hook, the file-touch hook, and the standing learnings at session start (as an indented may be stale: line).

The rule (one place; Trail Cloud shows what the CLI sends)

Only a learning's touches: is judged, entry by entry:

path is gone when the checkout has no file or folder there.
path#Symbol is gone when its path is, or when that file's text no longer has the name as a whole word (last part of Type.method, ~N stripped).
One gone entry marks the learning. A learning on a teammate's unmerged branch is never judged (its code may only be on that branch).

Trail Cloud: it has no copy of the code, so PUT /api/cli/learnings now carries missing on each HEAD learning: the gone touches verbatim, [] when all are there, left out for branch copies and skills. The sync fingerprint includes missing, so a deleted file re-sends unchanged learnings (every repo re-syncs once after upgrade, since the fingerprint changed). Today's server ignores the field (Go encoding/json, no DisallowUnknownFields), so this is safe before the trail-cloud side ships. The rule was agreed with the trail-cloud "Stale-learning cleanup" session.

What I chose, and why

touches: only, not names in the text. Checking backticked names in the body against code-map symbols flagged 10 of 38 real learnings (Trail-Code + assign), and 9 of those names were still in the code: properties (costUsd), module variables (livePair), Node built-ins (execFileSync). touches: only ever holds what existed when the learning was saved, so a miss there is real: on this repo's main checkout, 3 of 29 learnings are flagged, all for the deleted src/ask/jev-ask.ts.
The file's text, not the code map. The task said "check against the code map". I built that first, then switched: loading the map costs a hook about 0.4 s here, a fresh clone has none, and Trail Cloud can't run it. Reading the one touched file flags the same 3 learnings in about 2 ms, and both sides now state one rule.
Telemetry on query, not learning_reused. stale sits beside notes; allowlisted in src/telemetry/contract.ts and TELEMETRY.md.

trail check

Exits 1 with 2 conflicts on src/notes/notes.ts:419, both word matches:

"brings back replace file name": the new line passes the learning's id to --replaces, which is what that learning decided.
"brings back check name": the Oct 21 learning about the Stop hook keying on a check's name; unrelated, matched on the words "check" and "name".

The learning this PR first added describes the code-map rule; the second one replaces it (--replaces l-d70a9e57), since learnings aren't edited.

Tests

New test/trail-stale-learnings.test.ts (8 tests): file and folder present vs gone, symbol as a whole word vs gone, unmerged-branch learning, the ask line, the session-start line, the sync payload's missing and the re-sync when it changes, the telemetry key.
Sync tests (trail-upload-safety, trail-cloud, workspace-sync-skills) pass.
npm test (scratch HOME): 4,143 tests, 4,131 pass, 3 fail. All 3 also fail on origin/main (19865ff) on this machine: two graft-to-trail moves and the "machine without trail" install line, which depend on whether trail is installed here.

Follow-ups

trail note records only files in touches:, so a learning that names a function only in its text is flagged when the file goes, not when the function does. Recording file#Symbol for backticked names the graph knows (as codeAnchors already does for init) closes that.
Session start could say "N of your learnings may be stale" on the count line.
Should a stale learning still be able to fail trail check?

🤖 Generated with Claude Code

A learning whose touches: lists a file the checkout no longer has, or a file#Symbol no code-map symbol is named, is shown as "may be stale" in trail ask, MCP find_code, the prompt and file-touch hooks, and the session-start standing list. query gets a stale count next to notes. trail check flags notes.ts:419 twice; both are word matches: the line passes the learning's id to --replaces (what the replaces learning decided), and the Stop-hook check-name learning is unrelated.

trailhq-graft · 2026-10-29T06:23:04Z

🌱 graft blast radius

5 areas changed → 10 areas can be affected. 31 dependent symbols, depth 2.
Tests: no test reaches Workspace Sync; 2 areas updated their tests.
Tag: @shhdwi — 15 of 15 areas · @qoole — Engine Core · @afeddersen — Event Telemetry

flowchart TB
  A0(("Session State Management<br/>10 symbols"))
  A1(("Cloud Collaboration<br/>7 symbols"))
  A2(("Token Query Interface<br/>3 symbols"))
  A3(("MCP Tool Invocation<br/>2 symbols"))
  A4(("Contextual Notes<br/>2 symbols"))
  AX(("5 smaller areas<br/>7 symbols"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2,A3,A4 reached;
  classDef tail fill:#EEF2F3,stroke:#9AA4A9,stroke-width:1px,color:#3A4247;
  class AX tail;

Can be affected	Symbols	Nearest hop	Reached from
Session State Management	10	`src/claude/session-metrics.ts:L1-L215` session-metrics.ts — imports, depth 1	Telemetry Contract
Cloud Collaboration	7	`src/cloud/nudge.ts:L1-L503` nudge.ts — imports, depth 1	Workspace Sync, Telemetry Contract
Token Query Interface	3	`src/ask/ask.ts:L1474-L1526` renderAsk — calls, depth 1	Note Attachment
MCP Tool Invocation	2	`src/mcp/tools.ts:L250-L336` callSingleTool — calls, depth 2	Note Attachment, Telemetry Contract
Contextual Notes	2	`src/notes/memory-notes.ts:L1-L280` memory-notes.ts — imports, depth 1	Telemetry Contract
Initialization	2	`src/init/run.ts:L1-L447` run.ts — imports, depth 1	Telemetry Contract
4 smaller areas	5	Event Telemetry, Engine Core, CLI Skills, …	see below

Who knows this code — 3 people across 15 areas

Area	Who knows it
Telemetry Contract · changed	@shhdwi — 1 commit, last 6d ago
Note Attachment · changed	@shhdwi — 3 commits, last 3d ago
Workspace Sync · changed	@shhdwi — 16 commits, last 2d ago
Prompt Directives · changed	@shhdwi — 21 commits, last 2d ago
CLI · changed	@shhdwi — 51 commits, last yesterday
Session State Management · affected	@shhdwi — 15 commits, last 6d ago
Cloud Collaboration · affected	@shhdwi — 22 commits, last 2d ago
Token Query Interface · affected	@shhdwi — 18 commits, last 7d ago
…7 further areas

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 31 dependent symbols, grouped by area

Session State Management — 10 symbols in 10 files

src/claude/session-metrics.ts:L1-L215 — session-metrics.ts (imports, depth 1)
67: * "graft" (or a repo working on graft itself) must not count as a graft read —
src/claude/skills-wiring.ts:L1-L116 — skills-wiring.ts (imports, depth 1)
16: import { hookMsBucket } from '../telemetry/contract.js';
src/claude/state.ts:L1-L165 — state.ts (imports, depth 1)
33: /** The repo skills this session used, by name, to when it last went to the workspace as a run (skill-use.ts). */
src/claude/hook-io.ts:L1-L140 — hook-io.ts (imports, depth 2)
17: /** The env a hook's graft child runs with. In a repo on trail with no map yet, the
src/claude/prompt-skills.ts:L1-L327 — prompt-skills.ts (imports, depth 2)
4: * skill a prompt brings back. `handlePrompt` calls it right after the
src/claude/skill-use.ts:L1-L80 — skill-use.ts (imports, depth 2)
2: * A coding agent using one of the repo's skills, as a run on the workspace.
src/claude/stats.ts:L1-L27 — stats.ts (imports, depth 2)
src/claude/statusline.ts:L1-L39 — statusline.ts (imports, depth 2)
2: import { renderStatusline, renderSubagent } from './format.js';
src/claude/stop.ts:L1-L216 — stop.ts (imports, depth 2)
11: * to save a learning (note-signals.ts). An offer or notice that wins leaves
src/claude/sync-run.ts:L1-L48 — sync-run.ts (imports, depth 2)
17: // unconfigured repo's rebuild sees byte-identical argv to before this existed.

Cloud Collaboration — 7 symbols in 4 files

src/cloud/nudge.ts:L1-L503 — nudge.ts (imports, depth 1)
5: * sharing, `trail ask`, `trail note` and `trail check` hand the agent a link
src/cloud/lookup-activity.ts:L1-L60 — lookup-activity.ts (imports, depth 2)
8: import { noteAuthor } from "../notes/git-facts.js";
src/cloud/other-repos.ts:L102-L135 — setUpPicked (calls, depth 2)
src/cloud/pair.ts:L209-L242 — finish (calls, depth 2)
209: finish(repo: string, found: FoundSkill[] = []): Promise<void> {
src/cloud/pair.ts:L137-L155 — start (calls, depth 2)
137: static async start(repo: string, code: string, opts: { agent: string; version: string; fetchImpl?: typeof fetch }): Promise<PairRun | { error: string; status?: number }> {
src/cloud/pair.ts:L308-L367 — waitAndUpload (calls, depth 2)
308: async function waitAndUpload(state: string, repo: string, opts: FinishOptions): Promise<Waited> {
src/cloud/pair.ts:L1-L368 — pair.ts (imports, depth 2)
12: * review, and uploads the repo the usual way, which leaves a dropped learning

Token Query Interface — 3 symbols in 1 file

src/ask/ask.ts:L1474-L1526 — renderAsk (calls, depth 1)
1476: // The note prints as its own prominent line(s) right under the header —
src/ask/ask.ts:L1543-L1546 — askSavedTokens (calls, depth 2)
src/ask/ask.ts:L1534-L1539 — formatAskWithSaved (calls, depth 2)

MCP Tool Invocation — 2 symbols in 1 file

src/mcp/tools.ts:L250-L336 — callSingleTool (calls, depth 2)
274: return { text: formatSkeleton(r), isError: !r.entries.length && !!r.note };
src/mcp/tools.ts:L1-L337 — tools.ts (imports, depth 2)
49: 'Query the repo context graph in plain words. Returns ranked nodes with exact file:line spans and the relevant source inlined — usually the full answer, no file reads needed.',

Contextual Notes — 2 symbols in 2 files

src/notes/memory-notes.ts:L1-L280 — memory-notes.ts (imports, depth 1)
2: * Learnings from the notes Claude Code keeps for this repo on this machine
src/notes/pr-reviews.ts:L1-L577 — pr-reviews.ts (imports, depth 1)
2: * Learnings from a repo's pull requests, for `trail init` to offer on day one

Initialization — 2 symbols in 2 files

src/init/run.ts:L1-L447 — run.ts (imports, depth 1)
3: * no-model look-back in a worker while init wires the repo, then show what
src/init/keep.ts:L1-L386 — keep.ts (imports, depth 2)
9: * - A guardrail: the catalog's template (`guardrailLearning`), every repo

Event Telemetry — 2 symbols in 2 files

src/telemetry/sessions.ts:L1-L113 — sessions.ts (imports, depth 1)
21: * must never touch the network (see the note atop upkeep.ts) and this doesn't:
src/telemetry/track.ts:L1-L162 — track.ts (imports, depth 1)
41: repo?: string;

Engine Core — 1 symbol in 1 file

src/engine.ts:L112-L122 — ask (calls, depth 1)
121: return attachNotes(r, dir);

CLI Skills — 1 symbol in 1 file

src/skills/cli-skills.ts:L1-L376 — cli-skills.ts (imports, depth 2)
118: /** A personal skill by name (any repo), or a name the person turned off. */

Learning Sessions — 1 symbol in 1 file

src/learn/sessions.ts:L1-L310 — sessions.ts (imports, depth 1)
2: * Learnings from a repo's past Claude Code sessions, for `trail init`: what

Test signal per changed area — 2 ✓ · 1 ✗ · 2 –

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

– Telemetry Contract — 1 test file changed here: test/trail-stale-learnings.test.ts
✓ Note Attachment — 4 of 7 reached · 1 test file changed here: test/trail-stale-learnings.test.ts
- not reached: staleLine, hasWord, symbolName
✗ Workspace Sync — 0 of 2 reached · no test file reaches it
- not reached: workspaceLearnings, fingerprint
✓ Prompt Directives — 3 of 5 reached · 1 test file changed here: test/trail-stale-learnings.test.ts
- not reached: learningsOnTouch, learningsAtStart
– CLI — no function, method or class changed here

60 test suites also reference this code

63 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

test/claude-copy-dashes.test.ts
test/claude-format.test.ts
test/claude-hook-timeouts.test.ts
test/claude-hooks-1b.test.ts
test/claude-hooks-perf.test.ts
test/claude-hooks-skills.test.ts
test/claude-hooks.test.ts
test/claude-host-standdown.test.ts
test/claude-post-bash-walls.test.ts
test/claude-prompt-learnings.test.ts
test/claude-prompt-private.test.ts
test/claude-session-start-learnings.test.ts
test/claude-skills-journey-fixtures.ts
test/claude-start-skills-1b.test.ts
test/claude-state.test.ts
test/claude-statusline.test.ts
test/claude-stop-trace.test.ts
test/claude-stop.test.ts
test/claude-tally.test.ts
test/hosts-codex-hooks.test.ts
…40 more

⚠️ 3 changed files not in the graph (.trail/learnings/2026-10-29-the-stale-learning-check-judges-touches-anirudhkumar.md, .trail/learnings/2026-10-29-the-stale-learning-check-reads-the-anirudhkumar.md, TELEMETRY.md) — no parser claims the extension, or the index predates the file.

_{graft blast · origin/main...HEAD · depth 2 · 14 changed files}

Open the interactive graph → — click an area to see its dependent symbols at file:line.

Trail Cloud has no code, so the CLI does the check and sends the gone touches per learning (`missing`, [] when all are there); the sync goes again when that list changes. The symbol check reads the touched file's text instead of the code map, the rule both sides now state the same way, and drops the 0.4s map load from hooks.

github-actions Bot added a commit that referenced this pull request Oct 29, 2026

viz: PR #731

fc82c26

github-actions Bot added a commit that referenced this pull request Oct 29, 2026

viz: PR #731

f89f761

This was referenced Oct 29, 2026

Fix trail's save-a-learning prompts #739

Open

Show file learnings in worktree sessions #741

Closed

Get overlap warnings ready to go live #738

Open

Navigation Menu

Repository navigation

Flag learnings that name code that's gone - #731

Flag learnings that name code that's gone#731
anirudhkumar-nanonets wants to merge 2 commits into
mainfrom
claude/flag-stale-learnings-99579b

anirudhkumar-nanonets commented Oct 29, 2026 •

edited

Loading

Uh oh!

trailhq-graft Bot commented Oct 29, 2026 •

edited by github-actions Bot

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

anirudhkumar-nanonets commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Claude sees

The rule (one place; Trail Cloud shows what the CLI sends)

What I chose, and why

trail check

Tests

Follow-ups

Uh oh!

trailhq-graft Bot commented Oct 29, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌱 graft blast radius

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

anirudhkumar-nanonets commented Oct 29, 2026 •

edited

Loading

trailhq-graft Bot commented Oct 29, 2026 •

edited by github-actions Bot

Loading