Repository navigation
MILAB-7194: column value references and block dependencies - #1911
Open
AStaroverov wants to merge 30 commits into
Open
AStaroverov wants to merge 30 commits into
AStaroverov wants to merge 30 commits into
Conversation
🦋 Changeset detectedLatest commit: 7d52947 The changes in this PR will be included in the next version bump. This PR includes changesets to release 25 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1911 +/- ##
==========================================
+ Coverage 57.57% 58.59% +1.02%
==========================================
Files 445 458 +13
Lines 23457 23994 +537
Branches 5288 5478 +190
==========================================
+ Hits 13505 14059 +554
+ Misses 8402 8363 -39
- Partials 1550 1572 +22 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
AStaroverov
marked this pull request as ready for review
October 19, 2026 14:21
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
AStaroverov
changed the title
Collaborator
Author
Collaborator
Author
This was referenced Oct 21, 2026
| host: undefined | OpenBlockHost, | ||
| ): undefined | ((blockId: string) => Promise<void>) { | ||
| if (host === undefined || host.apiVersion !== 3) return undefined; | ||
| const openBlock = host.openBlock?.bind(host); |
Contributor
There was a problem hiding this comment.
Is there a simpler way maybe that doesn't involve .bind()?
kevindetry-milaboratories
approved these changes
Oct 21, 2026
…dation
Column value references. `ColumnValueId` names one value of a column a block
picked; template relocation repoints the producer block inside it. In
workflows, `workflow.column-value` reads such ids back into plain values, and
`wf.body` / `wf.prepare` take `{ unwrapValues: true }`. Models build value
pickers with `readColumnValues` / `deriveColumnValueOptions`.
Block dependencies. The middle layer indexes which blocks reference which
outputs from args (`ctx.getOutputDependencies()`, `ctx.blockId`). The
`blockDependencies` plugin and `PlBlockDependencies` show links in both
directions with statuses checked against the producer's last run;
`PlBlockName` and `useOpenBlock` open another block of the project.
Args validation. `.argsValidation(({ args, data }) => ArgsIssue[])` and its
prerun counterpart report run readiness with severity instead of throwing in
args, so incomplete args are still kept as a draft for dependency tracking.
The model runtime has no Intl, so creating a collator at module load broke every block model bundle. Natural order is now compared by hand.
A workflow turns a picked column value id into its value with valueOf or valuesOf where it uses it. wf.body, wf.prepare and wf.preparePreRun no longer take an unwrapValues option, and the deep walk over args that replaced ids, map keys included, is gone with it.
The args and prerun args derive callbacks return the three states the middle layer tells apart. Args not ready for no reason given are notReady; args held back by errors are blocked, with the draft when args() returned one. A prerun without its own prerunArgs is notReady while its args are not ready or blocked: an older middle layer skips staging for that as well, so it needs no error of its own.
A project field created and then removed within one transaction stays on the backend, pointing at a value the backend has deleted; the next load of the project fails reading it. Adding a block whose args() throws and then setting data that makes them valid does exactly that with currentArgsIssues. The mutator now creates fields the project did not have when save() runs, so such a field is never sent.
Block dependencies are read from a block's current args, else from the args of its last run;
prerun args are no longer read. This removes argsValidation / prerunArgsValidation, args issues
in the UI, overview and MCP, the draft, issues and staging args fields in the middle layer, and
the argsField runtime API. Args derivation callbacks are back to their { error } / { value }
results.
PlBlockDependencies drops the links panel and its header button: it shows only the alerts for links that need attention. Blocks using data this block no longer has are a warning, with advice to add the data to this block or reconfigure those blocks, instead of an error. PlBlockName no longer takes onOpen, which only closed the panel.
A PColumnData/JsonPartitioned part is a JSON file, not a resource holding `{ data }`, and the
model reads files only asynchronously. readColumnValues gives undefined for such a column, and
picked values in it are unchecked as unreadable, as for binary and parquet data.
PlDropdown and PlDropdownMulti show a selected column value id by the value it picks: as is while the options load, and as "<value> — not available" when the options lack it. PlValueLabel shows a value as text the same way, for lists and other places outside a dropdown. columnValueLabel names a picked value to the user without reading any data.
A column counts as exported once its spec is in the producer's last run. Columns marked pl7.app/hideDataFromUi reach block UIs without their data, so data presence said nothing about the export and such columns, like the datasets of samples-and-data, were reported as not exported. Picked values are checked against the pl7.app/discreteValues annotation when the spec has it, else against the data, read only then. The dependents output reports hasUnrunChanges, whether the block has changes its last run does not include. While it does, the PlBlockDependencies warning says the links are checked against the last run, and that running the block shows whether the changes fix them.
Dropdowns, ref dropdowns and value labels take missingValueHint, a tooltip explaining a missing value. The ui-vue exports of these components pass one when the block has the blockDependencies plugin: the block that was to provide the column, as a link, and what went wrong. Ref dropdowns show missing values as "Value not available" like the other dropdowns, and links inside tooltips use the tooltip link color.
…el plugin The middle layer computes each block's column links on its own: its dependencies read its args and only the referenced columns of the blocks above it, its dependents read the args of the blocks below it and its own last run. They reach the app as `columnLinks` of the block state, so a run of one block no longer recomputes the links of blocks that do not reference it. The status logic moves from the blockDependencies plugin to pl-model-common (block_links) with its tests; the plugin, getOutputDependencies and the output dependency index are removed. A link to a block that has not been run yet is `notRun`, decided by whether the block has a last run, and raises both alerts, one line per block. ui-vue: PlBlockDependencies reads the links from the app and takes no props; the value pickers and PlDatasetSelector explain a missing selection by them.
… do not load the app
AStaroverov
force-pushed
the
MILAB-7194_column-value-refs
branch
from
October 22, 2026 07:13
744ef42 to
0e839dc
Compare
A value of a number column is picked as its decimal string, so the same value picked from a number cell and from a text cell is the same ColumnValueId. createColumnValueId refuses a value that is not a string, and the workflow's valueOf reads back strings only.
…th runIssues args() and prerunArgs() build args even when the configuration is incomplete, so the middle layer sees which columns a block references; runIssues() and prerunIssues() say why the block may not run yet. The middle layer writes the issues found in the current args to the block's inputsValid field together with a fingerprint of those args, and the project overview lets a block run only when that record matches its current args and holds no errors. A missing or foreign record is repaired by deriving the args again when the project is opened or rendered. The SDK sets the requiresRunIssues feature flag on every block it builds, so an app that would run incomplete args refuses the block. A block whose results are up to date no longer keeps its dependents from running when its own args have run errors: only a block that needs to run and cannot holds them back.
…rame Value pickers read the picked column's values with getUniqueValues, as blocks did before, so a column of any storage format can be picked from; reading inline JSON in the model left a column stored any other way unreadable, and a check over its values open forever. The model gives createColumnValuesSource(ctx, column) as an output: a frame of the column alone, resolving while it or its frame is, absent once the column is gone, each with the pick it was made for. The UI reads it with readColumnValueOptions, or in Vue with useColumnValueOptions, which follows the pick and tells loading, absent, read or failed. readColumnValues is removed.
runIssues read as issues a run came back with; the issues are found in args, before a run, so the builder methods are argsIssues and prerunArgsIssues, next to args and prerunArgs. RunIssue becomes ArgsIssue, the requiresRunIssues flag requiresArgsIssues, and the middle layer's run errors args errors.
Args without an args errors record of their own - a project saved before the record, or args a middle layer unaware of it wrote - record no errors and run as before, instead of being derived again when the project is opened or rendered: such a middle layer refuses every block whose argsIssues could find errors, so only blocks of earlier SDKs, whose args() throws instead, end up that way. Opening a project no longer rewrites or re-derives anything. The rule that an up-to-date block keeps feeding its dependents despite errors in its args holds only for blocks whose inputsValid comes from their args; a model API v1 block that is not valid, or a block whose code cannot run here, keeps the blocks below it from running, as before.
…PlColumnValueDropdown
A value picker is made like a data table: the model gives
createColumnValueOptions({ valueColumn, labelColumn? }, { ctx? }) as an output,
a frame of the value column and its label column, and the UI offers it with
PlColumnValueDropdown / PlColumnValueDropdownMulti, or reads it with
usePlColumnValueOptions where the whole list is needed. A label column labels
each value by its row, while the value column's value is what is stored.
Replaces createColumnValuesSource and useColumnValueOptions.
… slots usePlColumnValueOptions is no longer exported: the pair of createColumnValueOptions and PlColumnValueDropdown(Multi) is for a dropdown over the values of a column only, and a block needing the values for anything else reads its own frame. The dropdowns take exclude, values not offered, and pass their slots on.
usePlColumnValueSettings({ source, valueColumn }) makes the settings of
PlColumnValueDropdown(Multi) out of the block output and the column picked now:
none, pending until the output is computed for that pick, absent, or present.
It reads no values; the dropdowns take the settings and read the values
themselves, through a reader that is not exported.
…and missingValueTooltip
…, marked once it changes
dbolotin
force-pushed
the
MILAB-7194_column-value-refs
branch
from
October 31, 2026 20:54
435eea5 to
7d52947
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Blocks that pick values of another block's column (conditions, antigens, sample groups) say so through their args, and the platform tracks who uses whose data. Nothing is declared inside a block: references are found structurally in args.
Column value references (
pl-model-common,sdk/model,workflow-tengo)ColumnValueId— canonical id of one value of one column:{ __isColumnValue: true, column: ColumnUniversalId, value }. The column's leaf names the producer block; template relocation repoints it.collectColumnReferences(args)finds column ids and column value ids anywhere in args, including map keys and stringified JSON.readColumnValues(recipe)(cells of a column stored inline asPColumnData/Json, each with itsColumnValueId) andderiveColumnValueOptions(entries)(distinct{ value: ColumnValueId, label }options, natural order).workflow.column-valuewithisColumnValueId,valueOf,valuesOf. Blocks unwrap picked values explicitly inwf.body.Column links (
pl-middle-layer,pl-model-common)columnLinks(BlockStateV3);undefinedin app versions that do not compute them. Blocks add nothing to their model.dependencies— outputs of other blocks the block's args reference (its current args, else those of its last run);dependents— blocks whose args reference the block's outputs.ok,pending,notRun,unchecked,broken,missingValues. Picked values are checked againstpl7.app/discreteValues, else the column data.pl-model-common/block_links(computeDependencies,computeDependents,linkSections,linkAlerts,describeMissingValue), with their tests.UI (
ui-vue,uikit)PlDropdown,PlDropdownMulti,PlDropdownRef,PlDropdownMultiRef,PlValueLabelandPlDatasetSelectorfrom@platforma-sdk/ui-vueexplain a missing pick in a tooltip: the block that was to provide it, as a link, and what went wrong. uikit's versions take the hint as amissingValueHintprop.<PlBlockDependencies />(no props), optional: alerts for links that need attention — a warning when blocks use data this block does not have or has not produced yet, an error when an output this block uses is unavailable or lacks picked values.PlBlockNameanduseOpenBlock()open another block of the project;PlatformaV3.openBlockis optional and absent in app versions that cannot do it.Removed
blockDependenciesmodel plugin,ctx.getOutputDependencies(),ctx.blockIdand the middle layer's output dependency index, all added earlier on this branch.Docs
docs/column-identity.md,docs/column-access-api.md,migrations/2026-05-20-new-column-access-mechanism.md(corrections).Tests
pl-model-common(block_links),pl-middle-layer(column_links, offline, including invalidation scope),sdk/model,ui-vueunit tests;tests/drivers-ml-blocks-integrationgains a live test that links reach the block state.Consumers (branches
MILAB-7194_column-value-refs)ColumnValueId: MILAB-7194: track condition and antigen picks as column value references platforma-open/clonotype-enrichment#118, MILAB-7194: track compared groups as column value references, drop deprecated column APIs platforma-open/differential-clonotype-abundance#60, MILAB-7194: track compared groups as column value references, drop deprecated column APIs platforma-open/tcr-disco#21, MILAB-7194: store picked columns as ids and timepoints as column value references platforma-open/clonotype-distribution#17, MILAB-7194: store picked columns as ids and gates and conditions as column value references platforma-open/sort-seq-analysis#16, MILAB-7194: store picked columns as ids and expected values as column value references platforma-open/clonotype-convergence#20.<PlBlockDependencies />placed by MILAB-7194: warn about blocks missing data of this block platforma-open/samples-and-data#137 and MILAB-7194: warn about blocks missing data of this block platforma-open/import-vdj-data#55.openBlockand passescolumnLinksto block UIs.
The PR appears safe to merge; no new actionable defects were found.Summary
Adds column-value references and moves dependency checks into block state. Pickers can explain missing selections and link to the block that provides them.
Important touched terms:
Previous-thread context: AStaroverov confirmed that partitioned JSON is intentionally unsupported by these synchronous readers because its parts are files. Greptile accepted that explanation; the current readers preserve that limit.
No new actionable issues were found. Tests and browser checks were not run.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A["Block args"] --> B["Collect column references"] B --> C["Check producer's last run"] C --> D["BlockColumnLinks"] D --> E["Block state"] E --> F["Dependency alerts"] E --> G["Missing-selection hints"] G --> H["Open producer block"]Reviews (3) · Last reviewed commit: "MILAB-7194: compute column links in the ..." · Reviewed by Greptile