Repository navigation
ci: run doctests in ci and make test-rust - #1014
Open
dragonknightk wants to merge 1 commit into
Open
dragonknightk wants to merge 1 commit into
dragonknightk wants to merge 1 commit into
Conversation
## Description CI and `make test-rust` only run `cargo test --all-targets`, which skips doctests, so the examples in rustdoc comments (including the crate docs docs.rs renders for `hudi` and `hudi-datafusion`) could break after an API change without any check failing. This adds a `make test-rust-doc` target that runs them, calls it from `make test-rust`, and runs it as a step in the `rust-tests` CI job. Closes issue 970 - The step sits in the existing tarpaulin job because tarpaulin runs lib, bin and integration test targets only, and doctest coverage there would need a nightly toolchain. It costs one extra compile, since tarpaulin's instrumented artifacts cannot be reused for doctests; keeping it in this job avoids another runner setup and keeps all Rust test results in one place. - `hudi-cpp` is excluded. Its lib target is named `hudi`, the same as its `hudi` dependency, so rustdoc receives two `--extern hudi` candidates and fails with E0464 before running anything. The crate has no rustdoc examples. Setting `doctest = false` on it does not help, because an explicit `--doc` overrides that flag. - Every example outside `ignore` blocks already compiled and passed. `ignore` blocks are not compiled at all, so this change also converts the 8 `ignore` examples in hudi-core. Seven become `no_run`, with hidden lines for the async main, imports and placeholder paths, and their compile errors are fixed against the current API: two imported from `hudi`, which is not a dependency of hudi-core, `FileGroupReader::read_file_slice_stream` passed the `Result` from `with_batch_size` without `?`, and `HFileReader::open` iterated a reader not bound as `mut`. The eighth, on `HoodieMergedLogRecordReader`, documents a type that is not reachable outside the crate (its module is `pub(crate)`), so a doctest cannot use it, and it becomes a `text` block. No `ignore` doctests remain. - AGENTS.md lists the new target. crates/AGENTS.md asks that a new example needing a real table be written as `no_run` with hidden setup lines, and that a sketch on an item not reachable outside the crate use a `text` block. ## How are the changes test-covered - [ ] N/A - [x] Automated tests (unit and/or integration tests) - [x] Manual tests - [x] Details are described below The new CI step is itself the coverage: from now on every rustdoc example is compiled, and run unless marked `no_run`, on each pull request. Manual run of `make test-rust-doc` on Rust 1.94.1: - `hudi`: 6 pass - `hudi-core`: 16 pass (9 run, 7 compile-only `no_run`), 0 ignored - `hudi-datafusion`: 2 pass (compile-only, `no_run`) - Without `--exclude hudi-cpp`, the same command fails on `hudi-cpp` with E0464. ## AI Disclosure Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change. All submitted content was reviewed by a human before submission. Signed-off-by: dragonknightk <37928252+dragonknightk@users.noreply.github.com>
Author
|
@yihua Thanks for raising this! This is my first PR in hudi-rs, so the workflows are waiting for approval to run. Could you approve them when you have a moment? (´。• ᵕ •。`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
CI and
make test-rustonly runcargo test --all-targets, which skips doctests, so the examples in rustdoc comments (including the crate docs docs.rs renders forhudiandhudi-datafusion) could break after an API change without any check failing. This adds amake test-rust-doctarget that runs them, calls it frommake test-rust, and runs it as a step in therust-testsCI job.Closes #970
hudi-cppis excluded. Its lib target is namedhudi, the same as itshudidependency, so rustdoc receives two--extern hudicandidates and fails with E0464 before running anything. The crate has no rustdoc examples. Settingdoctest = falseon it does not help, because an explicit--docoverrides that flag.ignoreblocks already compiled and passed.ignoreblocks are not compiled at all, so this change also converts the 8ignoreexamples in hudi-core. Seven becomeno_run, with hidden lines for the async main, imports and placeholder paths, and their compile errors are fixed against the current API: two imported fromhudi, which is not a dependency of hudi-core,FileGroupReader::read_file_slice_streampassed theResultfromwith_batch_sizewithout?, andHFileReader::openiterated a reader not bound asmut. The eighth, onHoodieMergedLogRecordReader, documents a type that is not reachable outside the crate (its module ispub(crate)), so a doctest cannot use it, and it becomes atextblock. Noignoredoctests remain.no_runwith hidden setup lines, and that a sketch on an item not reachable outside the crate use atextblock.How are the changes test-covered
The new CI step is itself the coverage: from now on every rustdoc example is compiled, and run unless marked
no_run, on each pull request.Manual run of
make test-rust-docon Rust 1.94.1:hudi: 6 passhudi-core: 16 pass (9 run, 7 compile-onlyno_run), 0 ignoredhudi-datafusion: 2 pass (compile-only,no_run)--exclude hudi-cpp, the same command fails onhudi-cppwith E0464.AI Disclosure
Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change. All submitted content was reviewed by a human before submission.
(๑•̀ㅂ•́)و✧