Skip to content

Optimize Spark PDF translation throughput - #20

Merged
SunChJ merged 1 commit into
mainfrom
codex/spark-translation-throughput
Aug 17, 2026
Merged

Optimize Spark PDF translation throughput#20
SunChJ merged 1 commit into
mainfrom
codex/spark-translation-throughput

Conversation

@SunChJ

@SunChJ SunChJ commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace the fixed 12-item BabelDOC cap with an estimated character budget
  • use a 3,000-character default selected from repeated local Spark measurements
  • recover malformed structured output by bisecting only the failed document batch
  • pace Spark background turns and retry capacity errors without applying the policy to other models
  • add an opt-in reproducible PDF package benchmark

Local benchmark

18,006 source characters from 2603.05890v1.pdf, Spark with low reasoning:

  • 3,000 chars: 4/4 successful, 51.3 s median, 12 turns median
  • 2,600 chars: 4/4 successful, 63.1 s median, 14 turns median

Validation

  • swift test --filter LoopbackServerTests (9 passed)
  • swift test --filter CodexAppServerClientTests (24 passed)
  • swift test --filter TranslationDispatchCenterTests (5 passed)
  • live Spark package benchmark (11 measured runs across screening and finalists)

@SunChJ
SunChJ merged commit 4d8c41d into main Aug 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant