Repository navigation
Score WolfXL calc results from calculate(); rank Community 2.0.7 and Commercial 2.3.0 - #49
Merged
Merged
Conversation
A local WolfXL wheel carried a release version string while running different code, so the calc lane could not show which build produced a score. Each engine now reports its version; Python packages installed from a local build, editable checkout, or VCS are labeled as such without recording paths or URLs.
…ately WolfXL returns calculated values from its API. Whether save() writes them into the file as cached values depends on the build, so scoring only the saved file reported 0 for builds whose calculations were correct. The calc lane now scores the returned values and records how many matching values the saved file carries in a separate Saved to file column.
WolfXL Commercial and Community both install as wolfxl, so the Commercial build runs in the interpreter named by EXCELBENCH_WOLFXL_COMMERCIAL_PYTHON and reports its version, install source, and calculated values over JSON. The calc lane ranks both builds in one run. The calc fallback-contract fakes gain the version() method the suite has required since engine versions were recorded.
Community 2.0.7 from PyPI returns 133/133 from calculate() and writes no calculated values on save (0/133 saved). Commercial 2.3.0 returns and saves 133/133. Engine notes now render below the report table instead of splitting it.
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Version probing can abort the suite, while public and architectural documentation contradicts the new results and semantics.
Review effort: Balanced
Findings: 1
Open (3)
What changed in this PR
Updates the calc lane to score API-returned values separately from persisted values and adds isolated WolfXL Commercial execution.
Changes:
- Adds version/source reporting and Commercial subprocess support.
- Separates calculated and saved-value scores.
- Refreshes WolfXL 2.0.7/2.3.0 results and tests.
| File | Description |
|---|---|
uv.lock |
Upgrades WolfXL to 2.0.7. |
src/excelbench/harness/calc.py |
Implements scoring, versions, and Commercial engine. |
src/excelbench/results/calc_renderer.py |
Adds version and saved-value columns. |
tests/test_calc_suite.py |
Tests new scoring and subprocess behavior. |
tests/test_env_fallback_contracts.py |
Updates fake engine protocol. |
results-2026-10-02/calc/results.json |
Records refreshed results. |
results-2026-10-02/calc/README.md |
Publishes refreshed calc report. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
One engine now carries one version across every lane of a snapshot.
A calc-only snapshot needs its own run metadata; the calc lane previously borrowed both from the xlsx lane.
A failing version probe now records None instead of aborting the suite. README, snapshot manifest, architecture, and DEC-028 describe the calc rerun, API-result scoring, and the Commercial interpreter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
calculate()returns and reports separately how many correct valuessave()writes into the workbook (new "Saved to file" column).wolfxl-commercialengine runs the Commercial build in a separate interpreter named byEXCELBENCH_WOLFXL_COMMERCIAL_PYTHON, because both editions install aswolfxl.uv.lockmoves to WolfXL Community 2.0.7.results-2026-10-02/calcrerun: Community 2.0.7 133/133 computed, 0/133 saved; Commercial 2.3.0 133/133 computed and saved; LibreOffice 133/133; Aspose.Cells FOSS 25/133; Zavora 0/133.Verification
tests/test_calc_suite.pyandtests/test_env_fallback_contracts.pypass; the new scoring and report-layout tests fail before their fixes.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.