Skip to content

fix(tasks): keep parallel task responses apart with variableKey and a slot-aware merge - #1072

Merged
yilmaztayfun merged 12 commits into
masterfrom
feature/task-variable-key
Oct 5, 2026
Merged

yilmaztayfun merged 12 commits into
masterfrom
feature/task-variable-key

Conversation

@yilmaztayfun

Copy link
Copy Markdown
Contributor

Summary

  • Fixes InvalidOperationException: Parallel tasks produced conflicting output for key 'subprocessTaskSendNotification' (ScriptContext.MergeDictionary). Entries at one order run in parallel and are merged by response slot ToVariableName(task.key); the same task listed twice at one order (e.g. two SubProcess starts) wrote one slot with different payloads (new instance id, ExecutionDurationMs), so the merge threw on practically every run.
  • Task entries accept an optional variableKey; the effective slot is variableKey ?? ToVariableName(task.key), used verbatim. TaskCoordinator threads it into the existing TaskEngineExecutionOptions.ResponseVariableKey, which every executor now honors (ExternalHttp did not).
  • Publish rejects a same-order slot collision and a malformed variableKey (WorkflowValidator; FunctionComponentValidator compares effective slots across all onExecutionTasks). Definitions without variableKey behave exactly as before.
  • The parallel merge is now slot-aware (ScriptContext.MergeParallelBranches): one round is merged by the slots each branch actually wrote (reference inequality against the parent — branches container-copy the dictionaries). A later order re-writing an earlier slot overwrites it, also when it runs as a parallel group (this used to throw too); only two branches of one round writing one slot differently is a conflict, detected before anything is applied.

Changes

  • Definitions/Tasks/OnExecuteTask.cs: VariableKey, computed ResponseVariableKey ([JsonIgnore]), IsValidVariableKey.
  • Definitions/Validators/WorkflowValidator.cs (ValidateTaskCollection), Definitions/Validators/FunctionComponentValidator.cs: slot and format rules.
  • Tasks/Coordinator/TaskCoordinator.cs: ResolveGroupEngineOptions threads VariableKey (caller/extension key still wins); duplicate-slot warning groups by slot; round merge call.
  • Scripting/Models.cs: MergeParallelBranches (two-pass), MergeParallelBranch as a one-branch round; MergeDictionary/MergeMetadata removed.
  • Tasks/Executors/Http/ExternalHttpTaskExecutor.cs: passes ResponseVariableKey. Functions/FunctionAppService.cs: single-task slot uses the effective key.
  • Logging/WorkflowLogs.cs, docs/runtime/script-context-and-engine.md, docs/domain/fan-out-task.md, vnext-meta/features.json (taskVariableKey, since 0.0.99).
  • Tests: OnExecuteTaskVariableKeyTests, WorkflowValidatorTests, FunctionComponentValidatorTests, TaskCoordinatorGroupEngineOptionsTests, TaskCoordinatorDuplicateTaskKeyTests, ExternalHttpTaskExecutorTests, ScriptContextCowBranchTests (the three pins of the old inherited-slot comparison are rewritten as two-branch conflicts; new: overwrite, untouched-by-reference, conflict atomicity, metadata).

Test Plan

  • Domain: ScriptContext*, WorkflowValidatorTests, OnExecuteTaskVariableKeyTests — 232/236; the 4 reds are the DefaultAutoTransition tests, also red on master cddab84c.
  • Application: validators, coordinator, executors, FanOut, CacheAside, extensions, functions, correlation tree — 253/253 after merging master.
  • dotnet build vnext.sln 0 errors.

Integration test evidence

  • Scenario: vnext-example Tests/ParallelVariableKeyLab (--filter FullyQualifiedName~ParallelVariableKeyLab), flows pvk-parent (same SubProcess task twice at one order with variableKey, plus a legacy-slot entry) and pvk-reuse (a parallel group re-writing an earlier order's slot), plus three publish-rejection probes.
  • Runtime: commit 8046dbd6 and, after merging master, 8f11acfc, at VNEXT_BASE_URL=http://localhost:4201 (run-docker.sh up core, local build).
  • Result: 5/5 green (merged base: ParallelVariableKeyLab + SubflowOrchestration + FanOut 50/50). Distinguishing run on master cddab84c: 0/5 — pvk-reuse Faulted with Parallel tasks produced conflicting output for key 'pvkSpawnChild', publish probes answered 200.
  • Verified: OpenObserve trace 4f5c8cc95f9f65cb2de3fbdf4b067abc — order-2 spawns ran in parallel (51.7 / 46.8 ms, OK); 0 "conflicting output" logs after rebuild; Postgres: three distinct child ids in the parent's latest data row, children waiting.
  • Full suite (FullyQualifiedName!~CrossDomainLab) on 8046dbd6: 357 passed / 38 failed / 9 skipped (404). All 29 distinct failing tests also fail on master cddab84c (RoleMatrixLab, HumanTaskChain — partner/credit not up, DataIntegrityLab, two role tests, TaskInvocationLab DaprService); no new red.
  • TEST-SCENARIOS.md row added/updated: yes (parallel-variable-key-lab).

Notes

  • Not breaking for existing definitions; a definition with the same task twice at one order and no variableKey is now rejected at publish — that shape crashed at run time on most runs before (it could pass only when both payloads happened to be identical).
  • Behavior change: a parallel group re-writing a slot an earlier order left behind now overwrites it instead of throwing. In-place mutation of an inherited value inside a parallel task is still not seen by the merge (documented).
  • Schema: feat(task): add optional variableKey to task entries vnext-schema#149 (needed for variableKey in function definitions; workflows already accept it). Scenario PR in burgan-tech/vnext-example follows.
  • This PR includes a one-line test fix (577110a7): master's InstanceQueryAppServiceCorrelationTreeTests missed the currentUser constructor argument and the Application test project did not compile.

🤖 Generated with Claude Code

yilmaztayfun and others added 12 commits October 4, 2026 20:13
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parallel entries were merged by ToVariableName(task.key); two runs of
the same task (e.g. two SubProcess starts) carried different payloads
and the merge threw 'Parallel tasks produced conflicting output'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…duplicate-slot warning

ExternalHttpTaskExecutor wrote its response without the context's
ResponseVariableKey, so a variableKey was ignored for that task type.
The duplicate warning now names every distinct task key in the shared
slot, and its docs match the variableKey behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…notes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A parallel group re-writing a slot an earlier order left behind was
compared against the inherited value and threw 'Parallel tasks
produced conflicting output'. Only two branches of one round writing
one slot differently is a conflict now; a single writer overwrites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ge is slot-aware

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The InstanceQueryAppService constructor gained an ICurrentUser parameter;
this test did not, so the Application test project stopped compiling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yilmaztayfun
yilmaztayfun requested review from a team October 5, 2026 06:57
@coldtea-pr-lens

coldtea-pr-lens Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Nothing flagged · reviewed 577110a


Architecture

Architecture diagram for burgan-tech/vnext at 577110a

Play the walkthrough


Inside the changed components — 2 views

Component view — Definition & validation

Validation rules for task variableKey slots across workflows and functions

Architecture view of Component view — Definition & validation in burgan-tech/vnext

Component view — Parallel execution & branch merge

Task coordination, per-slot response assignment, and two-pass parallel branch merging

Architecture view of Component view — Parallel execution & branch merge in burgan-tech/vnext

Data flow

Data flow diagram for burgan-tech/vnext at 577110a

Follow each request


View

  • Architecture lens
  • Data flow lens
  • Expand every detail

Tip

Untick Architecture lens or Data flow lens under View to hide a diagram, or tick Expand every detail to open every section. The comment redraws in a few seconds

🪧 More tips
  • Run npx skills add coldteadotai/pr-lens, then tell your coding agent: "Diagram the change you just made with PR Lens and attach it to the pull request."
  • Run npx @coldtea/pr-lens-cli analyze --base origin/main on a branch, then npx @coldtea/pr-lens-cli render .pr-lens/graph.json. Same lenses, your own model key, before the pull request exists
  • Click the link under each diagram to open it on a canvas you can zoom, pan and step through
  • The diagrams are links. Click one to open it on the canvas, then press W or click play to walk through the change
  • Open a diagram on the canvas, then press W or click play to walk through the change one step at a time
  • The CLI's render reads .github/pr-lens.yml and applies your renames, exclusions and lane pins at draw time
  • Set github.comment.collapsed: true in .github/pr-lens.yml to fold the comment behind one View architecture and data flow row. Drawing still runs as before
  • Set github.draw: on-demand in .github/pr-lens.yml and PR Lens stops drawing pull requests on its own. Comment @pr-lens draw on a pull request when you want that one drawn
  • Add .github/workflows/pr-lens.yml with coldteadotai/pr-lens/packages/action@v0 and your model provider's key as its api-key to run PR Lens from your own CI. Any /chat/completions endpoint works
  • Push a commit and the drawing stays, with a note that it is out of date. Tick Redraw in the note to draw the new head
  • Switch GitHub to dark mode and the diagrams follow. The moving dots are this pull request's data in motion

Thanks for using PR Lens! It's built by Coldtea, free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4a0047bb-5f83-4606-b08b-90f3b6e9df84
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@deepsource-io

deepsource-io Bot commented Oct 5, 2026

Copy link
Copy Markdown

DeepSource Code Review

We reviewed changes in 0819f06...577110a on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade   Security  

Reliability  

Complexity  

Hygiene  

Code Review Summary

Analyzer Status Updated (UTC) Details
C# Oct 5, 2026 6:57a.m. Review ↗

Important

AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.

@yilmaztayfun yilmaztayfun self-assigned this Oct 5, 2026
@yilmaztayfun yilmaztayfun added this to the v0.0.99 milestone Oct 5, 2026
@yilmaztayfun
yilmaztayfun merged commit 1104d8a into master Oct 5, 2026
5 of 6 checks passed
@sonarqubecloud

sonarqubecloud Bot commented Oct 5, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)
B Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant