[FBumann]

Prompt: Yes!

Note

The following content was generated by AI.
The prompt answered a proposal: "Collapse and sort the matrix once, when the model is frozen, instead of per block."

Stacked on #2022. A constraint's terms are summed, pruned and ordered once for the whole model rather than once per constraint. The fixed cost per constraint falls from about 6 to 3.4–4 ms; models of a million variables build in the same time.

What changed, method, measurements, mutation table, what was tried and dropped

Why. A profile of a model with 128 variables and 128 constraints, each over 100 coordinates, put 22% of its build in each constraint's own _collapsed and _sorted. Each constraint ran its order and repeat probes and its sort as separate small polars queries.

What.

• _block keeps a constraint's linear terms as they arrive. It reads which of its rows have any term with _rows_with_terms, which marks them in one pass over the block's own rows from 0. It drops its rows with no terms there.
• Assembly.run stacks every block's terms on the model's rows and sums repeated cells, drops zeros and orders by (row, col) in one _collapsed call, under the build's own engine choice.
• Each constraint's coefficient range is read off its slice of that summed matrix (_per_block_ranges), so the range is the one the per-block collapse gave.
• Bounds, right-hand sides and the objective keep their per-block _magnitude_range.
• _matrix_share and _ordered_rows are gone.

Tried and dropped. Reading every block's range in one pass, with numpy or a grouped polars query, cut the cost per constraint to about 2.6 ms. But it made dispatch/m and network/m 1.2–1.3× slower: one pass over 4M values on one thread against the per-block polars probe on all threads. A first version of this PR also found the rows with terms by unique(), which hashed a million unsorted rows and cost the 1M rungs about 10%.

Method. Engine.build from tidied sources, the median of 5–15 builds per process, alternating process runs per arm, started below a 1-minute load of 1. 4-vCPU cloud container. Base: #2022's head, a9fdd89.

At this branch's head, five runs per arm:

model	#2022 ms	this ms	ratio
dispatch/3xs, dispatch/xs	13.5, 15.1	12.4, 15.0	0.92, 0.99
nodal/2xs, nodal/xs	16.5, 19.2	15.2, 18.9	0.92, 0.98
genx_piecewise_fuel (8 variables, 15 constraints)	117.6	86.5	0.74
dispatch/m (1M)	100.1	92.2	0.92
network/m (1M)	276.3	271.3	0.98

At the commit before _rows_with_terms, which only adds speed: N variables and N constraints over 100 coordinates, two runs per arm.

N	#2022 ms	this ms	ratio
8	58.9	42.1	0.72
32	206.7	143.9	0.70
128	796.8	527.5	0.66
128, masked	723.4	444.3	0.61

The slope from N = 8 to 128 falls from 6.15 to 4.05 ms per pair, and from 5.61 to 3.37 masked. At that commit the port models were 0.77–0.97×: genx 0.77, pypsa_ac_dc 0.81, pypsa_unit_commitment 0.83.

Mutation table, taken by hand: tools.mutate cannot express an edited expression. Clean tree, restored with git checkout --, __pycache__ dropped before and after each run.

mutation	result
each constraint's coefficient range read off the next one's slice	caught, 41 failed
the matrix left uncollapsed (stacked = terms)	caught, 920 failed
the null-coefficient refusal moved into _block (→ pass)	caught, 7 failed
every row marked as having terms (seen[:] = True)	caught, 52 failed

Gates. pixi run lint, pyrefly check on src/, ruff format --check ., pixi run test: 5593 passed. test-bench not run: bench/ is untouched.

Not done. The bound, right-hand-side and objective ranges are still one polars query per block, which is about 1.2 ms per pair at N = 128. Reading them only when diagnostics ask would remove that at every size. But Measured keeps them past close(), so that is a design question, not part of this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_015YXphRT4nkmgh8LgdYMgCj

---

Generated by Claude Code

[read-the-docs-community]

Documentation build overview

📚 specsolve | 🛠️ Build #35115861 | 📁 Comparing 611d936 against latest (f37252b)

  🔍 Preview build  

3 files changed

± index.html
± about/benchmarks/index.html
± about/changelog/index.html

[FBumann]

Note

The following content was generated by AI.

Folded into #2022. These commits are now on #2022's branch, claude/batched-build, as a fast-forward, and #2022's description covers both.

---

Generated by Claude Code

[codspeed]

Merging this PR will degrade performance by 22.15%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 12 regressed benchmarks
✅ 92 untouched benchmarks
⏩ 307 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_window[fleet-s-specsolve-highs-values]	10.4 MB	14.9 MB	-30.29%
❌	test_window[fleet-s-specsolve-highs-one]	10.4 MB	14.9 MB	-30.29%
❌	test_window[commitment-s-specsolve-highs-values]	1.2 MB	1.6 MB	-29.09%
❌	test_window[commitment-s-specsolve-highs-one]	1.2 MB	1.6 MB	-29.09%
❌	test_window[storage-s-specsolve-highs-one]	11.7 MB	15.8 MB	-25.96%
❌	test_window[storage-s-specsolve-highs-values]	11.7 MB	15.8 MB	-25.96%
❌	test_window[network-s-specsolve-highs-one]	14.8 MB	18.5 MB	-19.69%
❌	test_window[network-s-specsolve-highs-values]	14.8 MB	18.5 MB	-19.69%
❌	test_window[milp-s-specsolve-highs-one]	21.9 MB	27.2 MB	-19.67%
❌	test_window[milp-s-specsolve-highs-values]	21.9 MB	27.2 MB	-19.67%
❌	test_window[transport-s-specsolve-highs-one]	13.1 MB	13.9 MB	-5.69%
❌	test_window[transport-s-specsolve-highs-values]	13.1 MB	13.9 MB	-5.67%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/work-once-at-freeze (611d936) with claude/batched-build (a9fdd89)²}

Footnotes

1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on claude/batched-build (611d936) during the generation of this report, so cbfca83 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩