Found while profiling bolt lint (branch autoresearch/bolt-lint-perf). The fix is on claude/bolt-lint-performance-3jytgy.
Shape
syntax/lex.bend step, which runs once per char of every file:
Bool.pick(St, continues(mode, cls),
St{after(mode, cls), kind, line2, col2, sl, sc, cc <> buf, toks},
St{begin.mode(cls), begin.kind(cls), line2, col2, line, col, [cc], flush(kind, sl, sc, buf, toks)})
Bool.pick is eager, so flush ran on every char, including chars that just continue a token. flush reverses the buffer into a word (List.reverse, String.from_list), refines its kind against the keyword list (List.contains ×16, String.contains, String.eq) and allocates a Tok. That's O(token length) work per char. Switching to Lazy.either took lex on a 1.4 MB file from 2.44 s to 0.89 s, and the whole run from 31.3 s to 26.6 s.
Why eager misses it
eager counts a branch call only to "a def of this file that loops (it calls itself, or reaches something that does)", on the grounds that "a Base call and a one-line accessor are everywhere and cost nothing". flush doesn't loop in this file, but everything it calls is a Base walk over the buffer and the keyword list. The pick also isn't inside a recursive def: step is the body of run's loop.
Proposal
- Treat a Base walk as looping:
List.reverse, List.append, List.contains, String.from_list, String.to_list, String.contains, String.eq/String.cmp on non-literals, List.map, and so on. That makes a this-file def that reaches one count as "loops". This may need a narrower allowlist to stay quiet.
- Also count a pick in a def whose only caller calls it once per step of a loop (
run(t, step(c, st))), not just a pick inside the recursive def itself.
Found while profiling bolt lint (branch
autoresearch/bolt-lint-perf). The fix is onclaude/bolt-lint-performance-3jytgy.Shape
syntax/lex.bendstep, which runs once per char of every file:Bool.pickis eager, soflushran on every char, including chars that just continue a token.flushreverses the buffer into a word (List.reverse,String.from_list), refines its kind against the keyword list (List.contains×16,String.contains,String.eq) and allocates aTok. That's O(token length) work per char. Switching toLazy.eithertook lex on a 1.4 MB file from 2.44 s to 0.89 s, and the whole run from 31.3 s to 26.6 s.Why
eagermisses iteagercounts a branch call only to "a def of this file that loops (it calls itself, or reaches something that does)", on the grounds that "a Base call and a one-line accessor are everywhere and cost nothing".flushdoesn't loop in this file, but everything it calls is a Base walk over the buffer and the keyword list. The pick also isn't inside a recursive def:stepis the body ofrun's loop.Proposal
List.reverse,List.append,List.contains,String.from_list,String.to_list,String.contains,String.eq/String.cmpon non-literals,List.map, and so on. That makes a this-file def that reaches one count as "loops". This may need a narrower allowlist to stay quiet.run(t, step(c, st))), not just a pick inside the recursive def itself.