Repository navigation
refactor(core): re-arm the dead-code gate - #1023
Open
dragonknightk wants to merge 1 commit into
Open
dragonknightk wants to merge 1 commit into
dragonknightk wants to merge 1 commit into
Conversation
## Description Closes issue 683 `hudi-core` carried 68 `#[allow(dead_code)]` attributes, 6 of them module-wide. The `dead_code` lint is the check that an item has a production caller, so each allow switched that check off. This change removes all of them. Every item the lint then reports is either deleted or kept with a written reason that the compiler enforces. On current main the issue's command reports 38 warnings with default features and 43 without `spill-rocksdb`, not 27. Many of the allows suppressed nothing and are simply dropped. Unused does not mean delete, so the reported items are handled four ways: - **Delete** only an item with no Java method or field counterpart (counting the getters Lombok generates), or a second item mirroring a Java member that another item already mirrors. No Java counterpart: `DeleteContext::from_reader_schema` (Java's closest is `DeleteContext.fromRecordSchema(Properties, HoodieSchema)`, with different parameters), `RecordContext::delete_batch_to_keys`, `ReaderContext::partition_path`, `slice_row` and `FileGroupMergeStream::current_in_memory_bytes`. Second mirrors: the five `HoodieMergedLogRecordReader` forwarders to the base reader's getters, which Java defines once on `BaseHoodieLogRecordReader`. - **Keep with `#[expect(dead_code, reason = "...")]`**, naming the Java counterpart or the missing caller. Examples: the four Lombok getters `get_progress` and `get_valid_block_instants` on the base log reader (`getProgress`, `getValidBlockInstants`), `HoodieFileGroupReader::valid_block_instants` (`getValidBlockInstants`) and `DataBlock::data` (`HFileBlock.getByteBuff`), plus `FileGroupRecordBuffer::take_next`, `BufferType::Unmerged` and the unread HFile block header fields. - **Test-only items** are `#[cfg(test)]`, such as `TimelineLoader::storage`, `ReaderContext::empty`, `HoodieFileGroupReader::builder` and the merged reader's stats getters. Where `#[cfg(test)]` does not fit, such as struct fields, the builder impl that also holds the Java-parity setters, and the pull-style trait methods, the item uses `#[cfg_attr(not(test), expect(dead_code, reason = "..."))]` instead: the lib test build uses it, so a plain `expect` would be unfulfilled there. - **Compile only with `spill-rocksdb`** the constants, types and test helpers that only the RocksDB disk tier uses. The module-wide allow in `buffer` hid that they are dead without the feature. An `expect`ed item counts as live, so the items it calls are not reported, and only the root of each unused call graph carries an attribute. Three trait methods, `contains_log_record`, `current_in_memory_bytes` and `set_reader_schema`, are `#[cfg(test)]` on the trait and both implementations instead. The position-based buffer forwards each of them to its inner key-based buffer, and rustc counts that call as a use of the trait method, so an `expect` on the trait method is never fulfilled. With the forwarding calls removed, the same `expect` is fulfilled. Scope stays crate-internal. `TimelineView::start_timestamp` is kept rather than deleted, because removing it changes the public `TimelineView::new`. A field that is only assigned to is not reported, because rustc counts an assignment as a use; only a field that is never touched after construction is reported. No new tooling is added: re-adding a module-wide `#![allow(dead_code)]` is caught only by the new rule in `crates/AGENTS.md`, which `AGENTS.md` asks for when a coding convention changes. No runtime behavior changes. ## How are the changes test-covered - [ ] N/A - [x] Automated tests (unit and/or integration tests) - [ ] Manual tests - [ ] Details are described below No new tests: the lint is the check. CI checks the lib with and without default features, and the lib test build with all features only. `make check-rust` uses `--all-features`. An item used only with `spill-rocksdb` is caught by CI's `cargo clippy -p hudi-core --lib --no-default-features -- -D warnings`. Locally, `cargo clippy -p hudi-core` with `--lib` and with `--all-targets`, each with default features and with `--no-default-features`, reports no dead code and no unfulfilled expectation. The existing `hudi-core` lib tests cover the edited code and pass with default features and with `--no-default-features`. ## AI Disclosure Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change, including the code, tests, and description. All submitted content was reviewed by a human before submission. Signed-off-by: dragonknightk <37928252+dragonknightk@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes #683
hudi-corecarried 68#[allow(dead_code)]attributes, 6 of them module-wide. Thedead_codelint is the check that an item has a production caller, so each allow switched that check off. This change removes all of them. Every item the lint then reports is either deleted or kept with a written reason that the compiler enforces.On current main the issue's command reports 38 warnings with default features and 43 without
spill-rocksdb, not 27. Many of the allows suppressed nothing and are simply dropped. Unused does not mean delete, so the reported items are handled four ways:DeleteContext::from_reader_schema(Java's closest isDeleteContext.fromRecordSchema(Properties, HoodieSchema), with different parameters),RecordContext::delete_batch_to_keys,ReaderContext::partition_path,slice_rowandFileGroupMergeStream::current_in_memory_bytes. Second mirrors: the fiveHoodieMergedLogRecordReaderforwarders to the base reader's getters, which Java defines once onBaseHoodieLogRecordReader.#[expect(dead_code, reason = "...")], naming the Java counterpart or the missing caller. Examples: the four Lombok gettersget_progressandget_valid_block_instantson the base log reader (getProgress,getValidBlockInstants),HoodieFileGroupReader::valid_block_instants(getValidBlockInstants) andDataBlock::data(HFileBlock.getByteBuff), plusFileGroupRecordBuffer::take_next,BufferType::Unmergedand the unread HFile block header fields.#[cfg(test)], such asTimelineLoader::storage,ReaderContext::empty,HoodieFileGroupReader::builderand the merged reader's stats getters. Where#[cfg(test)]does not fit, such as struct fields, the builder impl that also holds the Java-parity setters, and the pull-style trait methods, the item uses#[cfg_attr(not(test), expect(dead_code, reason = "..."))]instead: the lib test build uses it, so a plainexpectwould be unfulfilled there.spill-rocksdbthe constants, types and test helpers that only the RocksDB disk tier uses. The module-wide allow inbufferhid that they are dead without the feature.An
expected item counts as live, so the items it calls are not reported, and only the root of each unused call graph carries an attribute. Three trait methods,contains_log_record,current_in_memory_bytesandset_reader_schema, are#[cfg(test)]on the trait and both implementations instead. The position-based buffer forwards each of them to its inner key-based buffer, and rustc counts that call as a use of the trait method, so anexpecton the trait method is never fulfilled. With the forwarding calls removed, the sameexpectis fulfilled.Scope stays crate-internal.
TimelineView::start_timestampis kept rather than deleted, because removing it changes the publicTimelineView::new. A field that is only assigned to is not reported, because rustc counts an assignment as a use; only a field that is never touched after construction is reported. No new tooling is added: re-adding a module-wide#![allow(dead_code)]is caught only by the new rule incrates/AGENTS.md, whichAGENTS.mdasks for when a coding convention changes. No runtime behavior changes.How are the changes test-covered
No new tests: the lint is the check. CI checks the lib with and without default features, and the lib test build with all features only.
make check-rustuses--all-features. An item used only withspill-rocksdbis caught by CI'scargo clippy -p hudi-core --lib --no-default-features -- -D warnings. Locally,cargo clippy -p hudi-corewith--liband with--all-targets, each with default features and with--no-default-features, reports no dead code and no unfulfilled expectation. The existinghudi-corelib tests cover the edited code and pass with default features and with--no-default-features.AI Disclosure
Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change, including the code, tests, and description. All submitted content was reviewed by a human before submission.
(๑˃ᴗ˂)ﻭ