Skip to content

fix: deduplicate concurrent upload parse dispatch - #351

Open
Frankie-Xu wants to merge 2 commits into
Ontos-AI:mainfrom
Frankie-Xu:fix/Frankie-Xu/dedupe-parse-dispatch
Open

fix: deduplicate concurrent upload parse dispatch#351
Frankie-Xu wants to merge 2 commits into
Ontos-AI:mainfrom
Frankie-Xu:fix/Frankie-Xu/dedupe-parse-dispatch

Conversation

@Frankie-Xu

Copy link
Copy Markdown

Summary

  • Dispatch parsing only when this handoff owns the successful waiting-file -> pending transition.
  • Treat a CAS retry that succeeds as pending -> pending as an idempotent no-op.
  • Add direct and HTTP-interleaving regression coverage for the S3 notification / confirm-upload race.

Closes #286

Verification

  • uv run pytest apps/api/tests/contract/test_s3_event_contract.py -q — 13 passed
  • Focused concurrent HTTP and ordered-delivery cases — 3 passed
  • make lint — passed
  • make typecheck — 0 errors, 0 warnings
  • uv run pytest apps/api/tests/contract -q — 222 passed, 1 unrelated environment-specific failure in test_should_reject_a_url_source_when_file_type_detection_redirects_to_a_private_host; this environment resolves example.com to reserved 198.18.0.92, so URL validation rejects it before the test fake HEAD client is called. The failing test and URL-validation path are unchanged by this PR.

Deployment notes

  • No API, schema, migration, queue, storage, or environment-variable changes.
  • Existing worker lock and terminal-state gate remain unchanged.
  • During a rolling deploy, old API replicas can still double-dispatch until replaced; worker defenses continue to prevent duplicate processing and billing.

Scope

Changed files only:

  • apps/api/app/services/document_ingestion/handoff_service.py
  • apps/api/tests/contract/test_s3_event_contract.py

@Frankie-Xu

Copy link
Copy Markdown
Author

This is a first-time fork workflow and GitHub currently reports no checks. The focused regression, concurrent HTTP cases, affected contract file, lint, and typecheck pass locally; the full API contract result and its unrelated environment-specific failure are documented in the PR body. Please approve the workflow run when convenient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid duplicate parse dispatch when S3 notification races confirm-upload

1 participant