Skip to content

Choose Containers for export and statistics through Container selection (ADR 0011) - #101

Open
m-murphy wants to merge 94 commits into
docs/adr-coding-standardsfrom
feat/container-selection
Open

m-murphy wants to merge 94 commits into
docs/adr-coding-standardsfrom
feat/container-selection

Conversation

@m-murphy

@m-murphy m-murphy commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Summary

Container export and the statistics dashboard choose Containers through one Container selection module (ADR 0011).

 runSingleStudyContainerExport*  (#96)
-  buildContainerQuery → filterContainerIdsByType → enrichContainerData(type filter)
+  selectContainers({ kind: 'study' | 'subjects' }) → enrichContainerData
 runMultiStudy / runBarcode      (#97)
-  buildMultiStudyContainerQuery (one query per Study, fake MULTI study)
+  selectContainers({ kind: 'subjects' | 'micronixBarcodes' })
 getDashboardStatistics          (#98)
-  inline all-tags intersection, unreachable location-only branch
+  selectContainers({ kind: 'study' | 'database' })
 GET /containers?tag_ids         (#99)
-  any selected Tag
+  containerHasAllTags(tagIds)

Also: GET /export/available-types selects through the module; JSON POST container exports return their rows again (they returned data: []); buildContainerQuery, lib/export/filter.ts, lib/container-tag-filter.ts and lib/statistics/location-filter.ts are deleted; enrichContainerData(database, containers) reads study fields from the Source.

Evidence

  • Before: a subject list of 500 Subjects × 80 dates → expected 14964 values, received 80500; available-types for a Study with 70,000 Subjects → 500; a POST export with format: 'json' → data: [].
    After: all three pass. Container selection, export, statistics and containers tests: 178 pass against the in-memory database; bun run ci:verify passes.

Merge Danger

Door: two-way

Revertible per commit; no schema change.

Blast Radius: wire-semantics

GET /containers?tag_ids changes from any-Tag to all-Tags for external clients. GET study export returns 404 when every Container is filtered out by type. The dashboard's "Created" filter is relabelled "Specimen created".

Closes #94
Closes #95
Closes #96
Closes #97
Closes #98
Closes #99
Closes #100

🤖 Generated with Claude Code

m-murphy and others added 30 commits October 6, 2026 16:47
…0007).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e entry point.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The repo's docs spell it "behaviour" (7 uses in 4 files against 1
"behavior"), which CODING_STANDARDS.md already follows.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…onalDatabase.

check-integrity.ts opens a backup with new Database(..., { readonly: true })
on purpose, so the rule now names that exception and leaves room for others.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…dule (ADR 0006).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The API test README offered loginAndGetCookie and friends with no caveat,
against the rule to use setupAuthenticatedRouteTest. Logging in by hand is
now limited to tests of the login flow, and the client wrapper is shown on
top of the adapter's cookie, matching how the tests use them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The db README defers to CODING_STANDARDS.md for the transaction rule, but the
move dropped why: Drizzle's bun-sqlite transaction commits at the first await.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The old API and web test READMEs asked for edge-case tests, loading and
error state tests, and waitFor or findBy for async work. The rewrite cut
them instead of moving them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The web test README restated it; it now shows how createMockedDomainModule
works and points to the rule.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AGENTS.md now points to the paragraph instead of restating it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the Checks workflow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The old pointer covered nearly every task, so it routed nowhere.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…types with the migrated schema.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The 90% target and what each threshold does were spread across README.md,
packages/web/vitest.config.ts and the commented-out bunfig.toml threshold.
The configs now point to a Coverage section; no config values change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It is the only place the README defines the CURRENT that the evolution table
compares against, and support needs it to read a schema_version value.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…per parent type.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…arent.

The comparison against the first row whose parent resolves (71c1919) had
no test: with the old `i === 0` check, an invalid first row switched the
warning off for the whole file.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each rule now applies to new or changed code, keeps its ADR's strength,
scope and exemptions, and names the paths still migrating. Adds the
missing rules from ADRs 0001, 0004, 0005, 0006 and 0008.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…try.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tests too.

The selector only caught async function literals written inline. A typed local rule
also catches a named async function and a sync callback that returns a Promise.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
specimen-types.ts drops its own copy of chunkArray.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
withRolledBackTransaction left BEGIN open across awaits on the one
bun:sqlite connection. A request that wrote while a validate was open
got "within a transaction" from its own BEGIN, ran inside the validate's
transaction, and lost its writes when the validate rolled back, after it
had already reported success. Two overlapping withWriteTransaction calls
shared one transaction the same way.

The open transaction now lives in an AsyncLocalStorage, and a lock per
connection serializes transactions. A call from inside the same async
context runs in a savepoint; a call from any other context waits for the
lock and opens its own transaction. Nesting no longer depends on the
SQLite error text.

A nested withWriteTransaction that throws now rolls back its own writes,
so a per_subject import run inside an open transaction drops the failed
subject's Subject row too. ADR 0010 says how per_subject gets that.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The db README covered only withWriteTransaction. It now lists both
helpers, how the per-connection lock and AsyncLocalStorage decide
between waiting, a savepoint, and a new BEGIN, and the writes the lock
does not cover: plain writes and Drizzle's sync transactions still land
in whatever transaction is open, and are lost if it rolls back.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…helper.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
m-murphy and others added 29 commits October 6, 2026 17:12
# Conflicts:
#	packages/web/src/contexts/UserContext.tsx
only(rows, what) returned the row an INSERT … RETURNING gave back, or
threw. The name now says that.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…port say "your file".

Registration errors named CSV columns (plate_name, box_name, sheet_name)
and said a value repeated "in your file", though add-container,
POST /specimens, /subjects/with-specimens, and the specimen entry form
send JSON. The module now says "in this request" and describes missing
names without column names. RegistrationOptions.inputName lets a caller
name its input; the bulk combined import, which takes a CSV, passes
"your file".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…t the difference.

The parity fixtures ran only full_file imports. Each fixture now also
runs per_subject and checks the weaker property that mode allows:
validate passes exactly when every subject imports, and every subject
validate flags is one the import rejects. One more test pins the case
where they differ: a subject that relies on a plate an earlier, failing
subject would create is rejected by the import but not flagged by
validate. validateBulkCombinedPayload and ADR 0010 describe this.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tion

# Conflicts:
#	packages/api/src/db/README.md
#	packages/api/src/lib/bulk-combined-import.ts
#	packages/api/src/lib/container-creation.ts
#	packages/api/src/lib/container-write-placement.ts
#	packages/api/src/lib/registration-orchestrator.ts
…tructure

# Conflicts:
#	packages/api/src/__tests__/README.md
#	packages/api/src/db/README.md
#	packages/web/src/__tests__/README.md
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Breaking: tag_ids now lists only Containers carrying all the selected
Tags, using containerHasAllTags from Container selection (ADR 0011).
Pagination and the total stay in SQL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion.

`GET /export/containers` now picks its Containers with `selectContainers`
(ADR 0011), so the container type filter runs once, in the selection. A
Study whose Containers are all of other types now returns 404 "No
containers found for this study" instead of an empty file. `created_from`
and `created_to` map to Container created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The statistics created_from/created_to filter has always matched the
Specimen's created date. The filter fields and their chips now say so,
in the wording ADR 0011 uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
formatContainerExport returned the JSON rows as `json`, but
buildPostExportEnvelope reads `jsonData`, so every POST container export
in JSON format sent an empty `data` array. New executor tests for
multi-study and barcode export pin the rows, the per-Study summary and
the barcode summary before those paths move to Container selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion.

`POST /export/containers` now builds a subject-list selection from the resolved subject names and their per-subject dates, and selects with `selectContainers` (ADR 0011). Count-only and full export share the one selection, so neither calls `filterContainerIdsByType` any more. The summary is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…election.

getDashboardStatistics now asks selectContainers for its Containers and
Specimens. created_from/created_to map to Specimen created and
specimen_type_id to a one-item list. The inline all-Tags intersection,
the unreachable location- and tag-only branch, the statistics location
helpers and buildDateFilter are gone. The location filter no longer needs
the raw SQLite handle, so the statistics routes drop it.

The wire shape is unchanged. A filter that matches no Specimen returns
the all-zero statistics, without the storage summary, as an unknown Study
or Tag already did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ainer selection.

A multi-study upload is now one subject-list selection instead of one
buildContainerQuery call per Study, followed by a container type filter
and enrichment per Study. The executor groups the enriched rows by Study
for the Export summary. The barcode export selects the found tubes with
a micronix barcode selection and applies no other filters; found and not
found still come from resolveMicronixBarcodesToContainers, since the
selection carries no barcodes.

buildMultiStudyContainerQuery, buildContainerQueryByMicronixBarcodes and
MultiStudyExportResult had no other callers and are deleted. The
multi-visit test moves to the executor tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	packages/api/src/lib/export/container-export.ts
… the old selection code.

enrichContainerData now takes only the Containers. Each subject row's
study fields come from its Source's Study (ADR 0007) and control rows
have none, so the type filter, the unused specimen list, the study
argument and the subject-to-Study map are gone, along with
loadStudiesBySubject. The MULTI study record is now used only for JSON
export metadata, including the barcode export's.

buildContainerQuery, filterContainerIdsByType (lib/export/filter.ts) and
lib/container-tag-filter.ts had no callers outside their tests and are
deleted with them; Container selection's tests cover their cases.
query.ts keeps resolveMicronixBarcodesToContainers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every export and statistics path now selects through
src/lib/container-selection/, so the rule lists no unmigrated paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Container selection now owns the rules its callers re-derived:

- containerTypes is typed as ContainerSubtype. containerTypesFilter
  narrows wire values and drops unrecognised ones, so a list of only
  unrecognised values still matches nothing; 'unknown' matches
  Containers with no subtype row, as ADR 0011 now says.
- hasContainerLevelFilter replaces dashboard-stats' own rule, which
  left out Container created.
- SpecimenSourceType and specimenSourceType replace the ternary in the
  database selection and the check in statisticsSelection.

Export and statistics look up a Study through identifier-resolution
(resolveStudyRecordByShortCode backs resolveStudyByShortCode) instead of
their own queries. Export inverts subject name maps with one helper,
and multi-study JSON metadata uses MULTI_STUDY_RECORD, as barcode export
does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /export/available-types picked a Study's Specimens and Containers
with unchunked IN lists that grew with the Study, and failed once its
Subjects outnumbered SQLite's bound-parameter limit. It now calls
selectContainers with a study selection and reads the specimen types and
containerTypes from the result. The response is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
createTestStorageContainer takes a created override, createTestSpecimen
takes collectionDate and created, and createTestContainerTags links
Tags to a Container. The Container selection tests use them instead of
raw inserts and updates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
subjectListScopes chunked by Subject, but each dated Subject added a
condition per date to one OR, so 500 Subjects with 80 dates each bound
80,500 parameters and the query failed. Each listed date is now its own
condition, and a query takes at most SQL_IN_CHUNK_SIZE of them; undated
Subjects get their own query. The list is named subjectDateConditions,
since it no longer mixes in the undated condition. Scopes can now
overlap, so selectContainers collects Containers by id.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The filter table already ran each filter on study, subject-list and
whole-database selections. It now also runs them on a whole-database
selection narrowed to Subjects, and a subject-list test applies the
other filters to a Subject's own dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@m-murphy
m-murphy marked this pull request as ready for review October 23, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant