Repository navigation
fix(server): parse shutdown, KMP flag, uvicorn redaction - #578
Merged
Merged
Conversation
Cancelling a parse task never stopped the thread doing the work. The row was written "failed: cancelled" at once while rendering carried on and could still succeed, a queued parse got no row at all, and the default executor's thread join at loop teardown kept the process open until the render finished. Each parse now runs on its own daemon thread. A first cancel asks the thread to skip rendering and keeps waiting, so the row follows the thread: complete if it finished, failed with MinerU's error if it was terminated, "rendering was skipped" if MinerU finished late. Shutdown waits a bounded time, then cancels again and records the parse as abandoned; a queued parse is recorded as never started.
The whole suite ran under KMP_DUPLICATE_LIB_OK=TRUE, a flag no server process ever has, so a reintroduced duplicate OpenMP load could never fail a test. conftest now sets it only on macOS, where the faiss-cpu and PyTorch wheels each bring a runtime, and the rest of the suite runs the way production does. A new test boots the app in a child process with the flag removed on every platform and checks /healthz and /readyz. Off macOS a second one loads faiss and then torch in one flag-free process.
Only the console script and the desktop child installed the Threat-8 RedactionFilter. Served as `uvicorn server.main:app`, nothing did: uvicorn's access logger kept its own unfiltered handler and FastMCP's basicConfig put an unfiltered RichHandler on root, so a record logged with `query` printed it in the clear. The module-level app now calls ensure_configured before building the app. With no root handler yet, it configures logging fully on a stderr handler, which also leaves FastMCP's basicConfig with nothing to do. With logging already set up, as on the CLI and desktop paths, it only adds the filter where it is missing and keeps the existing level and format. configure() now filters the handlers of every existing logger, not only root's.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three server-runtime findings: parse cancellation, the test-only KMP flag, and log redaction on the uvicorn launch path.
#305: record what a cancelled parse actually did, and bound shutdown
#376: set
KMP_DUPLICATE_LIB_OKon macOS onlytests/conftest.pynow sets it only on macOS, where the faiss-cpu and PyTorch wheels each bring their own OpenMP runtime./healthzand/readyz. Off macOS, a second test loads faiss and then torch in one process without the flag.#321: redact logs on the
uvicorn server.main:apppathRedactionFilter. Served asuvicorn server.main:app, nothing did:basicConfigput an unfilteredRichHandleron root;queryprinted it in the clear.ensure_configuredbefore building the app.basicConfigwith nothing to do.configure()now filters the handlers of every existing logger, not only root's.Closes #305
Closes #376
Closes #321
Verification
ruff check .is clean.5ac66b2: 5593 passed, 182 skipped, 1 xfailed.