[FBumann]

Prompt: It’s super worth!! Let’s do it. And use the same linopy version as the oracle for now. As another stacked pr

Scope added later: "Why not move them over? Or call them from check() ?"

Note

The following content was generated by AI.

Model.to_linopy() returns a linopy.Model. Each variable and each constraint keeps its declared name and dims, so m.variables['p'].sel(generator='wind') works. Coordinates the build did not produce are masked out. Model.check('linopy') refuses what the export refuses, without importing linopy. It needs the new [linopy] extra, pinned to the oracle's linopy, 0.10.0.

Stacked on #1793

The base is feat/pyomo-model (#1793, which is itself on #1795). This PR reads the same Declared from sinks/handoff.py: which declaration owns each column, row and SOS set, and at which coordinate. Merge #1795, then #1793, then this one.

What the model holds, and what it refuses

• Coordinates: every label of each dim, in ordinal order, taken from the attached dimension frames. Each variable and constraint is masked to the coordinates the build produced. A label keeps its type: int, str and datetime stay as they are.
• Rows: the build's numbers. Each row's terms run along _term and are built through linopy's public LinearExpression(Dataset, model). Shorter rows are padded with linopy's absent variable, -1. A masked entry carries bounds 0 and 1 and a right-hand side of 0. linopy refuses a NaN constant, and a binary bound other than 0 or 1, even where the mask hides it.
• SOS: linopy's own add_sos_constraints(variable, sos_type, sos_dim), which matches mathspec's sos: exactly (a variable and the dim it is ordered along).
• Names: linopy keeps variables and constraints in separate containers, so a variable and a constraint can share a name. The pyomo export refuses that clash and asks for a rename; this export needs no rename.
• Refused, because linopy has no form for it:
  • a quadratic constraint. linopy is an entry in the capability table, so this is refused as every sink refuses one, naming the sinks that take it (.lp, gurobi, pyomo). The message no longer names the constraint and its coordinate, as no table refusal does;
  • an objective constant. osemosys_utopia has one. The oracle has the same gap, already an xfail citing eager lane: an objective constant refuses a model the relational lane solves #894;
  • a row the build kept that has no term and that no point satisfies, such as 0 >= 10. Probed on this linopy: it drops a row without terms, even with a +0 placeholder term, so an infeasible model solved as optimal. A termless row that every point satisfies is dropped without changing the answer.
• A constraint with no term at all is left out, because linopy refuses one.

The check

• EXPORTS gains 'linopy': LINOPY_CAPABILITIES (every capability but a quadratic constraint). Another sink's refusal now lists linopy among the takers where it takes the model.
• The two refusals the table cannot express, the objective constant and the unmet termless row, move unchanged from specsolve/linopy.py to relational/sinks/linopy_ingest.py. They read only the handoff and Declared, with numpy and polars, so the engine may hold them: test_engine_is_isolated forbids importing linopy or xarray, not knowing what linopy drops. The engine's check('linopy') calls them; Model.to_linopy calls that check, then builds. One path, and a test asserts check and to_linopy give equal messages for each of the three refusals.
• The module is not called linopy.py: LAZY_ORACLE_ALLOWED is keyed on that file name, so a second one would be allow-listed by accident.
• check('linopy') needs no linopy installed; to_linopy still refuses without the extra.

Policy and packaging decisions

• Hard rule 3 relaxed. The rule was "linopy is the test oracle only". It is now: the engine never imports linopy, and the export may import it when called.
  • The export lives in specsolve/linopy.py, outside relational/, because test_engine_is_isolated forbids the engine subpackage from importing linopy, even lazily.
  • It is the one entry in LAZY_ORACLE_ALLOWED, with its reason.
  • docs/about/linopy.md now describes three relationships: not a dependency, an export, the oracle. The architecture page says why the module sits outside a fence.
  • Two architecture sentences that said nothing under src/ imports linopy are corrected.
• The oracle stays independent. tests/linopy_lane does not read the export, so the differential comparison is still a check.
• The [linopy] extra pins linopy==0.10.0, the oracle's linopy, as asked. Since docs: the tests, the benchmarks and every reference model run on the linopy 0.10.0 release #1945 moved the oracle to that release, the extra names no git URL, hatch needs no allow-direct-references, and a release can be uploaded to PyPI.
• The PyPSA parity job installs ".[linopy]". On main it greps the oracle's pin out of pyproject.toml. The extra writes the pin a second time, so that grep would return two lines, and uv would refuse them. The local command in differential/pypsa/parity.py changes with it.
• SetRun.sos_type is now program.SosType (Literal[1, 2]), mathspec's own type, instead of int. linopy's signature takes the literal.

Verification

• 2521241, after main was merged in through the stack (feat(api): a written LP or MPS file can name each variable and constraint by its declaration and coordinate #1795 → feat(api): a built model can be handed to pyomo as a ConcreteModel indexed by its labels #1793 → this PR): with uv (pixi is not installed in the session): ruff check ., ruff format --check ., pyrefly check on src/, and the full suite (5892 passed, 566 skipped, 4 xfailed). The merge changed one test here: activity is now read only when asked for (outputs={'activity'}), and a printed term coordinate names its dims (snapshot=1, generator=wind). The same two changes were made to the row tests of feat(api): a written LP or MPS file can name each variable and constraint by its declaration and coordinate #1795 and feat(api): a built model can be handed to pyomo as a ConcreteModel indexed by its labels #1793. Not run on this commit: docs-build, docs-test (the docstring-link test ran in the suite), test-floors, the PyPSA parity job, the mutation table.
• 21ad24d: pixi run check (4833 passed, 522 skipped, 1 xfailed). pixi run docs-build ran on 1d8dfd9, before the rebase onto main and the check, and was not run again.
• Parity fix (b057163): the workflow's own uv run line, with ".[linopy]", run locally against a mathspec v0.2.0 checkout. It resolved linopy master, and every rung matched PyPSA. Not run again since.
• The branch's first commit: pixi run test-floors (2843 passed; linopy is absent there, so the new tests skip) and pixi run docs-test, both clean.
• tests/test_linopy_model.py, solved through linopy with HiGHS, and with Gurobi for SOS:
  • Every linear fixture and the quadratic objective reach the engine's optimum. Every port except osemosys_utopia reaches its published objective to 1e-6. osemosys_utopia is refused for its objective constant.
  • Every row of every linear fixture holds the terms, right-hand side and sign that Model.row reads at that coordinate. The masked count shows that the constraint holds nothing else.
  • On dispatch, each variable's solution at a coordinate is the engine's primal value there.
  • Masking, named coordinates and datetime labels.
  • The empty-row refusal and the drop, for each of >=, <= and ==.
  • check('linopy') refuses each of the three in the export's words, and a model linopy takes checks clean.
  • SOS1 and SOS2, including a set over a one-dim variable.
  • A variable and a constraint that share a name, and the message when linopy is missing.

Mutation table (full suite each time; the tree came back clean after every row). The last two rows were taken on 21ad24d. The rest were taken before the refusals moved to linopy_ingest.py; they moved unchanged, except the quadratic-row refusal, which the capability table replaced. None was taken again after the merge of main:

mutation	result
the objective-constant refusal (tools.mutate, then linopy.py:83-87)	caught
the quadratic-row refusal (tools.mutate, then linopy.py:88-93; now a table entry)	caught
the refusal of a termless row that no point satisfies (tools.mutate, then linopy.py:100-107)	caught
leaving out a constraint with no term (tools.mutate, linopy.py:186-187)	caught
the missing-extra message (by hand)	caught (1 failed)
masked bounds left infinite (by hand)	caught (3 failed)
masked right-hand sides left NaN (by hand)	caught (21 failed)
the at-most arm of the unmet test (by hand)	caught (1 failed)
the at-least arm of the unmet test (by hand)	caught (1 failed)
the equality arm of the unmet test (by hand)	caught (1 failed)
the linopy refusal in check (tools.mutate, engine.py:158-159)	caught
to_linopy asking the check first (tools.mutate, api.py:453)	caught

Not done

• No memory measurement. linopy holds each variable and constraint dense over the product of its dims, and pads each row to the longest in its constraint. On very sparse models this can take much more memory than specsolve or the pyomo export. This has not been measured.
• No timing. The export has not been timed.
• The internal CSR expression is not used. This linopy also has a CSR-backed expression, but it is private (_from_csr), so the export stays on the public dense path.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CM4oeTkzkk5G5zE6zgikSE
https://claude.ai/code/session_017HhMcpuGcaXgttYMf9Jg2q
https://claude.ai/code/session_01DSYMFAGbSZpZ4ECVdXGMxc

[read-the-docs-community]

Documentation build overview

📚 specsolve | 🛠️ Build #34915863 | 📁 Comparing 495e567 against latest (af1779e)

  🔍 Preview build  

75 files changed · + 43 added · ± 32 modified

+ Added

• reference/internals/index.html
• examples/pypsa_ladder/rung_17_process/index.html
• examples/pypsa_ladder/rung_18_transformer/index.html
• examples/pypsa_ladder/rung_19_losses_secants/index.html
• examples/pypsa_ladder/rung_20_phase_shifter/index.html
• examples/pypsa_ladder/rung_21_carrier_growth/index.html
• examples/pypsa_ladder/rung_22_transformer_losses/index.html
• examples/pypsa_ladder/rung_23_transformer_losses_secants/index.html
• examples/pypsa_ladder/rung_24_must_stay_down/index.html
• examples/pypsa_ladder/rung_25_committable_link/index.html
• and 33 more...

± Modified

• index.html
• sweep/index.html
• tables/index.html
• about/architecture/index.html
• about/changelog/index.html
• about/linopy/index.html
• examples/pypsa_ladder/index.html
• howto/archiving/index.html
• howto/debug/index.html
• howto/pandas-xarray/index.html
• and 22 more...

[codspeed]

Merging this PR will degrade performance by 67.75%

❌ 1 regressed benchmark
✅ 92 untouched benchmarks
⏩ 307 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_read[commitment-s-specsolve-parquet]	220.7 KB	684.3 KB	-67.75%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing feat/linopy-model (628ffea) with feat/pyomo-model (9671da2)}

Footnotes

1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩