Repository navigation
Speed up long-form object measurement spreadsheet folding - #195
Merged
Merged
Conversation
trissim
marked this pull request as ready for review
September 29, 2026 13:35
Record unprofiled stage timings and an exact-byte dense writer prototype to bound follow-up work for #194.
trissim
changed the base branch from
codex/perf-plate-input-transport-pr
to
main
September 29, 2026 18:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and result
ExportToSpreadsheetfolds millions of long-form object measurements for the ImagingFlow plate. The generic accumulator recomputes ownership, source scope, and subject maps for every cell even when the table contract fixes object scope and has one image identity and one object ID. This change keepsWideMeasurementRowAccumulatoras the sole folding authority and uses a direct loop for that declared shape; other shapes retain the existing path. Duplicate-value conflicts, feature projection, row order, and CSV values are preserved. Fixes #194.A saved 8-well batch replay improved from about 25.1 to 20.2 seconds with identical 92 MB CSV bytes. In warmed ordinary ZMQ 16-well ImagingFlow runs, execution and total both fell by about 10.5 seconds:
The warmed official30 one-well suite changed from 80.329 to 79.350 seconds execution and 214.708 to 213.524 seconds total. All 30 cases succeeded and all 110 generated files matched byte for byte; the exact-stack 16-well CSV also matched its parent. Seventy-two focused unit tests passed on this PR branch. Black, changed-line Ruff E/F, and diff checks passed.
The report and fresh figure include raw timings and a native CellProfiler comparison. The earlier 985.625-second native 16-well value is a 16× cold-launch projection. PR #197 measures native CP at 59.680 seconds for one well and 388.431 seconds for 16 wells/four jobs, making the matched warmed execution ratios 3.16× and 3.35×. The report labels the historical projection separately.
This PR is stacked on #193. The separate #170/#175 changes are present only in the integration checkout, so comparisons are branch-matched and absolute runtimes differ across the two checkouts.