Fix headlines cut off mid-word in Key Signals (+ mashed Mastodon titles) - #416
Open
runyourempire wants to merge 1 commit into
Open
Fix headlines cut off mid-word in Key Signals (+ mashed Mastodon titles)#416runyourempire wants to merge 1 commit into
runyourempire wants to merge 1 commit into
Conversation
The Key Signals card read "…Infects More Than 400 npm Pa" and "…encryptor wi". `signals.rs::generate_action` interpolated the headline through a flat `title.chars().take(60)` — a hard character cut with no word boundary and no ellipsis. 9,983 of 17,341 live titles (58%) are long enough to hit it. Three linked defects, one root: nothing in the codebase truncated text *for a human to read*. Every helper was a byte/char cut built for embedding and prompt budgets. 1. `utils::text::truncate_display` — the canonical display truncator. Breaks on a word boundary, strips dangling punctuation, marks the cut with one `…` (U+2026), and counts that ellipsis against the caller's budget so the EvidenceItem schema caps (title ≤ 120, relevance_note ≤ 200) hold without call-site arithmetic. Falls back to a hard cut for scripts that do not space their words, so the 12 shipped locales cannot degenerate to a bare ellipsis. `truncate_utf8` stays, now documented as the machine truncator. 2. `signals.rs` uses it at 120 chars. 120 matches the cap the Mastodon adapter already applies, so a Mastodon title interpolated into an action is never truncated twice and never grows a second ellipsis. 2.3% of the corpus now truncates, down from 58% — and never mid-word. 3. `mastodon.rs::derive_title` takes the first paragraph as the title when it already reads as a headline, instead of welding every `<p>` into one run and cutting at 120. That flattening is why the card's second line read "…Major Software Supply Chain Attack A large-scale…", and why three postings of the same DeadLock story carried three different titles that de-duplication could not collapse. The paragraph wins only if it is a headline in its own right — ≥ 5 words and ≥ 40 chars (rejects digest headers like "New updates · 3 Aug 9") and not ending in a connector (rejects lead-ins like "Lots of new stuff in pip 26.2:"). Otherwise the whole body is used, exactly as before. Security-id hoisting still reads the full body, so a CVE below the fold still leads the title. Measured on the live corpus (4,616 Mastodon rows): 2,049 headlines cleaned, 49 more duplicate items collapse, zero empty titles, one over-trim. Phase 104 re-derives stored titles so existing rows heal without waiting for re-ingest (same shape as Phases 93/94/102). Swept the same defect off every other user-visible surface: blind-spot and knowledge-decay titles/notes, dependency-match reasons (which truncated mid-word with no ellipsis at all), decision-window descriptions (ASCII "..." that overflowed its own cap), deep-scan narration, scoring progress, and NL search previews. Left the internal budgets alone — embedding text, LLM prompts and log lines are not read by humans — and left `fallback.rs` untouched because that cut feeds a `source_id`, not a display string. SignalRow clamps the action to two lines with the full text on hover, so a long headline cannot go ragged on a narrow window. Mastodon tests move to a sibling `mastodon_tests.rs` (the `windows_tests.rs` convention) — the new cases pushed the module past the 1000-line limit. Gates: 4,435 Rust tests pass, clippy -D warnings clean, fmt clean, file sizes clean, tsc clean, 55 signals frontend tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kdmy8EJv79fcbRWNJpwRGN
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
runyourempire
enabled auto-merge (squash)
August 11, 2026 17:40
runyourempire
added a commit
that referenced
this pull request
Aug 13, 2026
… text (#422) `&s[a..b]` on a `&str` panics when a bound lands mid-UTF-8-sequence. 4DA ingests arbitrary internet text, so every byte index derived from arithmetic was a live crash. Audits ALL char-boundary-panicking APIs (indexing, split_at, truncate, replace_range, split_off, drain, insert/remove, get_unchecked, multi-line forms) and fixes all 23 unguarded sites across 18 files. Includes a distinct index-desync class: to_lowercase()/to_uppercase() are Unicode-aware and change byte length, so four sites that searched a case-folded copy and sliced the original could panic or emit byte-shifted output. In preemption::extract_advisory_id that silently corrupted the cross-tier dedup key. All four now use the ASCII-only folding variants. Does not touch utils/text.rs (owned by #416). 18 regression tests added, each verified to fail on the pre-fix code. clippy -D warnings (default + experimental): clean. cargo test --lib: 4437 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the operator saw
The Key Signals card rendered
Review security implications: Self-Propagating ChainDrop Worm Infects More Than 400 npm Pa— cut mid-word, no ellipsis. Three near-identical DeadLock cards sat below it.Root cause
signals.rs::generate_actioninterpolated the headline through a flattitle.chars().take(60). Both screenshot titles measured exactly 60 chars at the cut. 9,983 of 17,341 live titles (58%) are long enough to hit it.The deeper finding: nothing in the codebase truncated text for a human to read. Every helper (
truncate_utf8, three privatetruncate_titles,blind_spots,knowledge_decay,dep_linker,decision_advantage) was a byte/char cut designed for embedding and prompt budgets.Changes
1.
utils::text::truncate_display— the canonical display truncator. Word boundary, dangling-punctuation strip, one…(U+2026), and the ellipsis is counted against the budget soEvidenceItemschema caps (title ≤ 120, note ≤ 200) hold without call-site arithmetic. Hard-cut fallback for unspaced scripts so the 12 shipped locales can't degenerate to a bare ellipsis.2.
signals.rsuses it at 120 chars — matching the Mastodon adapter's own cap, so a Mastodon title is never truncated twice. Truncation rate drops 58% → 2.3%, and never mid-word.3.
mastodon.rs::derive_titletakes the first paragraph as the title when it already reads as a headline, instead of welding every<p>into one run. That flattening is why the card's second line read…Supply Chain Attack A large-scale…and why one story produced three uncollapsible titles.The paragraph wins only if it stands alone: ≥ 5 words and ≥ 40 chars (rejects digest headers —
New updates · 3 Aug 9), and not ending in a connector (rejects lead-ins —Lots of new stuff in pip 26.2:). Otherwise the full body is used, exactly as before. Security-id hoisting still reads the whole body, so a CVE below the fold still leads the title.Thresholds were measured, not guessed — prototyped against all 4,616 live Mastodon rows before any Rust was written:
Phase 104 re-derives stored titles so existing rows heal without waiting for re-ingest (same shape as Phases 93/94/102;
contentis preserved, so titles remain fully re-derivable).4. Swept the same defect off every other user-visible surface: blind-spot and knowledge-decay titles/notes, dependency-match reasons (mid-word with no ellipsis at all), decision-window descriptions (ASCII
...that overflowed its own cap), deep-scan narration, scoring progress, NL search previews.Deliberately not touched: internal budgets (embedding text, LLM prompts, log lines — not read by humans), and
sources/fallback.rs:299, whose cut feeds asource_ididentity key, not a display string. Changing that would orphan existing items.5.
SignalRowclamps the action to two lines with full text on hover.Mastodon tests move to a sibling
mastodon_tests.rs(thewindows_tests.rsconvention) — the new cases pushed the module past the 1000-line limit.Verification
cargo test --libcargo clippy -- -D warningscargo fmt --checkcheck-file-sizestsc --noEmitThree
decision_advantagetests were updated: they asserted the old defect (ASCII..., and a result that exceeded its own cap —truncate("hello world", 5)returned 8 chars).Live in-app verification pending a rebuild + Phase 104 run.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Kdmy8EJv79fcbRWNJpwRGN