Score WolfXL calc results from calculate(); rank Community 2.0.7 and Commercial 2.3.0

METHODOLOGY.md

@@ -135,6 +135,12 @@ oracle, oracle version pinned in the artifact).
 - Status semantics are strict: `passed` requires a full match against the
   oracle; any wrong or missing value reports `failed` with matched/total and
   per-cell mismatch details.
+- Engines with a calculation API are scored on the values that API returns.
+  How many correct values the engine's save writes into the workbook is
+  reported separately as "Saved to file". Engines that calculate only on save,
+  such as LibreOffice, are scored on the saved file (DEC-028).
+- Every engine result records the engine version; builds not installed from a
+  package registry are labeled as local, editable, or VCS builds.
 - Formulas avoid `IFS` and `XLOOKUP` because the LibreOffice oracle build
   returns `#NAME?` for them on this path; nested `IF` and INDEX/MATCH are used
   instead. This constrains function coverage and is documented per artifact.

README.md

@@ -116,7 +116,7 @@ Use this when the question is: which engine actually computes a 133-formula fina
 - **Speed vs fidelity tradeoff is measurable**: use the perf snapshot together with the fidelity matrix rather than quoting one without the other
 - **Optimization modes have clear costs**: openpyxl-readonly loses 13 green features for streaming speed
 - **Cross-language context is now strong too**: `Apache POI` and `Excelize` land at `18/18` in the scored write lane; `zavora-xlsx` 0.1.2 writes fast but corrupts hyperlink relationships on mutate and cannot recalculate
-- **Calculation is a differentiator**: only LibreOffice computes the full 133-formula DAG (2026-10-02 snapshot); aspose-cells-foss covers 25/133; WolfXL 2.0.5 scores 0/133 because `Workbook.calculate()` runs but the saved workbook carries no cached formula values
+- **Calculation is a differentiator**: in the 2026-10-02 calc rerun, LibreOffice, WolfXL Community 2.0.7, and WolfXL Commercial 2.3.0 compute all 133 formulas; aspose-cells-foss covers 25/133 and zavora-xlsx 0/133. Community's `save()` writes no calculated values (0/133 saved), while Commercial saves all 133
 
 
 See the [release snapshot dashboard](results-release-2026-04-28/DASHBOARD.md) for the fresh wheel-backed combined view, or the [historical dashboard](results/DASHBOARD.md) for the older public baseline.

architecture.md

@@ -83,7 +83,10 @@ Most-touched top-level directories:
   - `harness/adapters/json_model_adapter.py`: base for adapters driven through
     an external JSON helper (SheetJS, ExcelJS, LibreOffice)
   - `harness/calc.py`: formula-recalculation tier engines (wolfxl, LibreOffice,
-    aspose-cells-foss, zavora oracle) against the cache-free fixture
+    aspose-cells-foss, zavora oracle) against the cache-free fixture; the
+    wolfxl-commercial engine runs a second `wolfxl` build in the interpreter
+    named by `EXCELBENCH_WOLFXL_COMMERCIAL_PYTHON` and returns its values as JSON
+    (see DEC-028)
   - `modifiable.py`: engine registry for mutation-capable adapters
   - `harness/external_fixture_specs/`: tool-specific external oracle fixture
     definitions imported by the fixture-pack generator

decisions.md

@@ -40,6 +40,45 @@ Skip logging for routine bug fixes, refactors, or incremental test additions.
 
 ## Decisions
 
+### DEC-028 — Calc tier scores API results; saved values and engine identity are recorded beside them (2026-10-02)
+
+**Context**: DEC-026 scored each engine from the cached values in the
+workbook it saved. For WolfXL that measured `save()`, not calculation:
+`Workbook.calculate()` returned values that the saved file never carried, so
+WolfXL scored 0/133 while computing them. The results also named no engine
+version, and a local WolfXL build was reported as if it were the PyPI
+release with the same version string. WolfXL Commercial and Community both
+install as `wolfxl`, so one interpreter cannot load both.
+
+**Decision**:
+
+- Engines that calculate through an API are scored on the values the API
+  returns. How many correct values `save()` writes into the workbook is
+  recorded separately as `saved_matched` and shown in a "Saved to file"
+  column. Engines that only calculate on save (LibreOffice) keep their
+  saved-file score.
+- Each engine result records `version()`. Non-registry installs carry their
+  PEP 610 source (local build, editable build, VCS commit) and never an install
+  path or URL. A failing version probe records `None`.
+- `wolfxl-commercial` runs in the interpreter named by
+  `EXCELBENCH_WOLFXL_COMMERCIAL_PYTHON` and reports values over JSON; without
+  that variable it is unavailable.
+- Calc results carry `metadata.run_date` and `metadata.platform` so a calc
+  run can be published as its own snapshot.
+
+**Alternatives considered**:
+
+1. Keep scoring saved caches - rejected; it reports a persistence gap as a
+   calculation failure.
+2. Install both editions side by side under different names - rejected; the
+   Commercial wheel is the product users install, unmodified.
+
+**Consequences**:
+
+- WolfXL Community 2.0.7 scores 133/133 calculated and 0/133 saved; WolfXL
+  Commercial 2.3.0 scores 133/133 on both. DEC-026's cache-freedom rule still
+  holds: no engine can pass by copying input caches.
+
 ### DEC-027 — External-helper adapters are scored; mutation preservation compares parsed content (2026-10-02)
 
 **Context**: DEC-021 kept external helpers out of `get_all_adapters()` until a

results-2026-10-02/README.md

@@ -4,14 +4,15 @@
 | --- | --- | --- |
 | Fidelity (17 adapters) | [xlsx/](xlsx/README.md) | `excelbench benchmark -t fixtures/excel -o results-2026-10-02/xlsx -a openpyxl -a xlsxwriter -a python-calamine -a aspose-cells-foss -a wolfxl -a pylightxl -a xlrd -a pyexcel -a xlwt -a pandas -a xlsxwriter-constmem -a openpyxl-readonly -a polars -a tablib -a sheetjs -a exceljs -a libreoffice` then `excelbench heatmap -i results-2026-10-02/xlsx/results.json -o results-2026-10-02/xlsx` |
 | Template mutation | [mutation/](mutation/README.md) | `excelbench mutation -o results-2026-10-02/mutation --repeats 3` |
-| Formula recalculation | [calc/](calc/README.md) | `excelbench calc -o results-2026-10-02/calc` |
+| Formula recalculation | [calc/](calc/README.md) | `EXCELBENCH_WOLFXL_COMMERCIAL_PYTHON=<commercial venv>/bin/python excelbench calc -o results-2026-10-02/calc` (rerun later the same day; see Versions) |
 | Cross-language context | [cross-language/](cross-language/README.md) | `EXCELBENCH_ORACLE_DOCKER_CONTEXT=pc excelbench cross-language-context -t fixtures/excel -o results-2026-10-02/cross-language` |
 
 ## Versions
 
 | Library | Version | Runtime |
 | --- | --- | --- |
-| wolfxl | 2.0.5 (latest PyPI release when run) | Python wheel |
+| wolfxl | 2.0.5 (latest PyPI release when run); the calc rerun used 2.0.7 | Python wheel |
+| wolfxl-commercial (calc only) | 2.3.0 from SynthGL's authenticated index | Python wheel, separate interpreter |
 | openpyxl / openpyxl-readonly | 3.1.5 | Python |
 | aspose-cells-foss | 26.7.0 | Python |
 | xlsxwriter / xlsxwriter-constmem | 3.2.9 | Python |

results-2026-10-02/calc/README.md

@@ -2,18 +2,17 @@
 
 Oracle: libreoffice/LibreOffice 26.8.0.3 bce0998afefdbc355585ca324285661a2170ba77
 
-Formula values are compared with absolute tolerance 1e-6 or relative tolerance 1e-9 for numbers; strings and booleans compare exactly.
+Formula values are compared with absolute tolerance 1e-6 or relative tolerance 1e-9 for numbers; strings and booleans compare exactly. Saved to file counts the matching values a save writes into the workbook, for engines that return results through an API.
 
-| Engine | Status | Matched/total | First mismatches |
-| --- | --- | --- | --- |
-| wolfxl | failed | 0/133 | Schedule!B10: expected 0.0, actual None<br>Schedule!B11: expected -308872.0, actual None<br>Schedule!B2: expected 100000.0, actual None<br>Schedule!B3: expected 38000.0, actual None<br>Schedule!B4: expected 62000.0, actual None<br>Schedule!B5: expected 12000.0, actual None<br>Schedule!B6: expected 488.0, actual None<br>Schedule!B7: expected 719.0, actual None<br>Schedule!B8: expected 400872.0, actual None<br>Schedule!B9: expected -300872.0, actual None |
-
-wolfxl: 133/133 formula values wrong or missing (first mismatch batch: 10 shown, 10 missing, 0 wrong)
-| libreoffice | passed | 133/133 | — |
-| aspose_cells_foss | failed | 25/133 | Schedule!B11: expected -308872.0, actual None<br>Schedule!B2: expected 100000.0, actual None<br>Schedule!B3: expected 38000.0, actual None<br>Schedule!B4: expected 62000.0, actual None<br>Schedule!B6: expected 488.0, actual None<br>Schedule!B7: expected 719.0, actual None<br>Schedule!B8: expected 400872.0, actual None<br>Schedule!B9: expected -300872.0, actual None<br>Schedule!C11: expected -306442.0, actual None<br>Schedule!C2: expected 104500.0, actual None |
+| Engine | Version | Status | Matched/total | Saved to file | First mismatches |
+| --- | --- | --- | --- | --- | --- |
+| wolfxl | 2.0.7 | passed | 133/133 | 0/133 | — |
+| wolfxl-commercial | 2.3.0 | passed | 133/133 | 133/133 | — |
+| libreoffice | 26.8.0.3 (build bce0998afefdbc355585ca324285661a2170ba77) | passed | 133/133 | — | — |
+| aspose_cells_foss | 26.7.0 | failed | 25/133 | — | Schedule!B11: expected -308872.0, actual None<br>Schedule!B2: expected 100000.0, actual None<br>Schedule!B3: expected 38000.0, actual None<br>Schedule!B4: expected 62000.0, actual None<br>Schedule!B6: expected 488.0, actual None<br>Schedule!B7: expected 719.0, actual None<br>Schedule!B8: expected 400872.0, actual None<br>Schedule!B9: expected -300872.0, actual None<br>Schedule!C11: expected -306442.0, actual None<br>Schedule!C2: expected 104500.0, actual None |
+| zavora | unknown | failed | 0/133 | — | Schedule!B10: expected 0.0, actual None<br>Schedule!B11: expected -308872.0, actual None<br>Schedule!B2: expected 100000.0, actual None<br>Schedule!B3: expected 38000.0, actual None<br>Schedule!B4: expected 62000.0, actual None<br>Schedule!B5: expected 12000.0, actual None<br>Schedule!B6: expected 488.0, actual None<br>Schedule!B7: expected 719.0, actual None<br>Schedule!B8: expected 400872.0, actual None<br>Schedule!B9: expected -300872.0, actual None |
 
 aspose_cells_foss: FOSS FormulaEvaluator returned no value for 107 formulas: Schedule!B2, Schedule!C2, Schedule!D2, Schedule!E2, Schedule!F2
-| zavora | failed | 0/133 | Schedule!B10: expected 0.0, actual None<br>Schedule!B11: expected -308872.0, actual None<br>Schedule!B2: expected 100000.0, actual None<br>Schedule!B3: expected 38000.0, actual None<br>Schedule!B4: expected 62000.0, actual None<br>Schedule!B5: expected 12000.0, actual None<br>Schedule!B6: expected 488.0, actual None<br>Schedule!B7: expected 719.0, actual None<br>Schedule!B8: expected 400872.0, actual None<br>Schedule!B9: expected -300872.0, actual None |
 
 zavora: 133/133 formula values wrong or missing (first mismatch batch: 10 shown, 10 missing, 0 wrong)
 
@@ -31,6 +30,7 @@ one are **not applicable** here, not scored 0:
 | excelize 2.10.1 | Not run | Excelize ships `CalcCellValue`, but the ExcelBench Excelize helper has no evaluate operation, so there is no harness path to score it yet |
 
 The `libreoffice` row above is LibreOffice 26.8.0.3 headless recalculation (it is also the oracle that produced `fixtures/calc/expected_values.json`).
-`wolfxl` is WolfXL 2.0.5 from PyPI: `Workbook.calculate()` runs, but the saved workbook carries no cached formula values, so every scored cell reads back empty.
+`wolfxl` is WolfXL Community 2.0.7 from PyPI. Its score counts the values `Workbook.calculate()` returns; its `save()` writes no calculated values, shown as Saved to file 0/133.
+`wolfxl-commercial` is WolfXL Commercial 2.3.0 from SynthGL's authenticated package index. Both editions install as `wolfxl`, so it ran in a separate interpreter named by `EXCELBENCH_WOLFXL_COMMERCIAL_PYTHON`.
 
-Command: `excelbench calc -o results-2026-10-02/calc` (defaults: `fixtures/calc/financial_model.xlsx`, `fixtures/calc/expected_values.json`).
+Command: `EXCELBENCH_WOLFXL_COMMERCIAL_PYTHON=<commercial venv>/bin/python excelbench calc -o results-2026-10-02/calc` (defaults: `fixtures/calc/financial_model.xlsx`, `fixtures/calc/expected_values.json`).

results-2026-10-02/calc/results.json

@@ -55,73 +55,37 @@
         }
       ],
       "reason": "FOSS FormulaEvaluator returned no value for 107 formulas: Schedule!B2, Schedule!C2, Schedule!D2, Schedule!E2, Schedule!F2",
+      "saved_matched": null,
       "status": "failed",
-      "total": 133
+      "total": 133,
+      "version": "26.7.0"
     },
     "libreoffice": {
       "matched": 133,
       "mismatched_cells": [],
       "reason": null,
+      "saved_matched": null,
       "status": "passed",
-      "total": 133
+      "total": 133,
+      "version": "26.8.0.3 (build bce0998afefdbc355585ca324285661a2170ba77)"
     },
     "wolfxl": {
-      "matched": 0,
-      "mismatched_cells": [
-        {
-          "actual": null,
-          "cell": "Schedule!B10",
-          "expected": 0.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B11",
-          "expected": -308872.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B2",
-          "expected": 100000.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B3",
-          "expected": 38000.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B4",
-          "expected": 62000.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B5",
-          "expected": 12000.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B6",
-          "expected": 488.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B7",
-          "expected": 719.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B8",
-          "expected": 400872.0
-        },
-        {
-          "actual": null,
-          "cell": "Schedule!B9",
-          "expected": -300872.0
-        }
-      ],
-      "reason": "133/133 formula values wrong or missing (first mismatch batch: 10 shown, 10 missing, 0 wrong)",
-      "status": "failed",
-      "total": 133
+      "matched": 133,
+      "mismatched_cells": [],
+      "reason": null,
+      "saved_matched": 0,
+      "status": "passed",
+      "total": 133,
+      "version": "2.0.7"
+    },
+    "wolfxl-commercial": {
+      "matched": 133,
+      "mismatched_cells": [],
+      "reason": null,
+      "saved_matched": 133,
+      "status": "passed",
+      "total": 133,
+      "version": "2.3.0"
     },
     "zavora": {
       "matched": 0,
@@ -178,10 +142,16 @@
         }
       ],
       "reason": "133/133 formula values wrong or missing (first mismatch batch: 10 shown, 10 missing, 0 wrong)",
+      "saved_matched": null,
       "status": "failed",
-      "total": 133
+      "total": 133,
+      "version": null
     }
   },
   "fixture": "fixtures/calc/financial_model.xlsx",
+  "metadata": {
+    "platform": "Darwin-arm64",
+    "run_date": "2026-10-02T11:37:18.885621+00:00"
+  },
   "oracle": "libreoffice/LibreOffice 26.8.0.3 bce0998afefdbc355585ca324285661a2170ba77"
 }