Skip to content

fix: read config through scalo and drop refused DLQ entries - #369

Merged
catinspace-au merged 1 commit into
mainfrom
fix/scalo-2-14-33-rollouts
Nov 21, 2026
Merged

catinspace-au merged 1 commit into
mainfrom
fix/scalo-2-14-33-rollouts

Conversation

@catinspace-au

Copy link
Copy Markdown
Contributor

When --config is unset, scalo 2.14.33 reads the file DFE_LOADER_CONFIG names. It reports DLQ size refusals as DlqError::Refused.

  • The watcher uses CommonArgs::config_path. Remove the loader's resolve_config_path.
  • place_dead_letters drops the entries a Refused error lists and counts them as too_large. It writes the other entries. It still retries other DLQ errors.
  • Both scalo minimum versions are 2.14.33.

Orchestrator unit tests check which entries are dropped and how many are counted. They use a predicate in place of RefusedEntries, which only scalo can build. A binary test checks DFE_LOADER_CONFIG. scalo tests refusal from a broker.

--config still wins. Without it, scalo 2.14.33 reads the file DFE_LOADER_CONFIG names. scalo ignores a blank DFE_LOADER_CONFIG.

The hot-reload watcher uses CommonArgs::config_path. The loader's resolve_config_path is gone.

On DlqError::Refused, place_dead_letters drops the entries the error lists. The loader counts them in pipeline_dead_letters_dropped_total with reason too_large. The rest of the write counts as placed.

When the broker refuses a dead letter, the batch can move on before the retry expires. Every other DLQ error keeps its retry.

Both scalo minimum versions are 2.14.33 to pick up DlqError::Refused.
@catinspace-au
catinspace-au merged commit 5709299 into main Nov 21, 2026
26 checks passed
@catinspace-au
catinspace-au deleted the fix/scalo-2-14-33-rollouts branch November 21, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant