Skip to content

refactor(core): re-arm the dead-code gate - #1023

Open
dragonknightk wants to merge 1 commit into
apache:mainfrom
dragonknightk:chore/rearm-dead-code-gate
Open

dragonknightk wants to merge 1 commit into
apache:mainfrom
dragonknightk:chore/rearm-dead-code-gate

Conversation

@dragonknightk

Copy link
Copy Markdown

Description

Closes #683

hudi-core carried 68 #[allow(dead_code)] attributes, 6 of them module-wide. The dead_code lint is the check that an item has a production caller, so each allow switched that check off. This change removes all of them. Every item the lint then reports is either deleted or kept with a written reason that the compiler enforces.

On current main the issue's command reports 38 warnings with default features and 43 without spill-rocksdb, not 27. Many of the allows suppressed nothing and are simply dropped. Unused does not mean delete, so the reported items are handled four ways:

  • Delete only an item with no Java method or field counterpart (counting the getters Lombok generates), or a second item mirroring a Java member that another item already mirrors. No Java counterpart: DeleteContext::from_reader_schema (Java's closest is DeleteContext.fromRecordSchema(Properties, HoodieSchema), with different parameters), RecordContext::delete_batch_to_keys, ReaderContext::partition_path, slice_row and FileGroupMergeStream::current_in_memory_bytes. Second mirrors: the five HoodieMergedLogRecordReader forwarders to the base reader's getters, which Java defines once on BaseHoodieLogRecordReader.
  • Keep with #[expect(dead_code, reason = "...")], naming the Java counterpart or the missing caller. Examples: the four Lombok getters get_progress and get_valid_block_instants on the base log reader (getProgress, getValidBlockInstants), HoodieFileGroupReader::valid_block_instants (getValidBlockInstants) and DataBlock::data (HFileBlock.getByteBuff), plus FileGroupRecordBuffer::take_next, BufferType::Unmerged and the unread HFile block header fields.
  • Test-only items are #[cfg(test)], such as TimelineLoader::storage, ReaderContext::empty, HoodieFileGroupReader::builder and the merged reader's stats getters. Where #[cfg(test)] does not fit, such as struct fields, the builder impl that also holds the Java-parity setters, and the pull-style trait methods, the item uses #[cfg_attr(not(test), expect(dead_code, reason = "..."))] instead: the lib test build uses it, so a plain expect would be unfulfilled there.
  • Compile only with spill-rocksdb the constants, types and test helpers that only the RocksDB disk tier uses. The module-wide allow in buffer hid that they are dead without the feature.

An expected item counts as live, so the items it calls are not reported, and only the root of each unused call graph carries an attribute. Three trait methods, contains_log_record, current_in_memory_bytes and set_reader_schema, are #[cfg(test)] on the trait and both implementations instead. The position-based buffer forwards each of them to its inner key-based buffer, and rustc counts that call as a use of the trait method, so an expect on the trait method is never fulfilled. With the forwarding calls removed, the same expect is fulfilled.

Scope stays crate-internal. TimelineView::start_timestamp is kept rather than deleted, because removing it changes the public TimelineView::new. A field that is only assigned to is not reported, because rustc counts an assignment as a use; only a field that is never touched after construction is reported. No new tooling is added: re-adding a module-wide #![allow(dead_code)] is caught only by the new rule in crates/AGENTS.md, which AGENTS.md asks for when a coding convention changes. No runtime behavior changes.

How are the changes test-covered

  • N/A
  • Automated tests (unit and/or integration tests)
  • Manual tests
    • Details are described below

No new tests: the lint is the check. CI checks the lib with and without default features, and the lib test build with all features only. make check-rust uses --all-features. An item used only with spill-rocksdb is caught by CI's cargo clippy -p hudi-core --lib --no-default-features -- -D warnings. Locally, cargo clippy -p hudi-core with --lib and with --all-targets, each with default features and with --no-default-features, reports no dead code and no unfulfilled expectation. The existing hudi-core lib tests cover the edited code and pass with default features and with --no-default-features.

AI Disclosure

Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change, including the code, tests, and description. All submitted content was reviewed by a human before submission.

(๑˃ᴗ˂)ﻭ

## Description

Closes issue 683

`hudi-core` carried 68 `#[allow(dead_code)]` attributes, 6 of them module-wide. The `dead_code` lint is the check that an item has a production caller, so each allow switched that check off. This change removes all of them. Every item the lint then reports is either deleted or kept with a written reason that the compiler enforces.

On current main the issue's command reports 38 warnings with default features and 43 without `spill-rocksdb`, not 27. Many of the allows suppressed nothing and are simply dropped. Unused does not mean delete, so the reported items are handled four ways:

- **Delete** only an item with no Java method or field counterpart (counting the getters Lombok generates), or a second item mirroring a Java member that another item already mirrors. No Java counterpart: `DeleteContext::from_reader_schema` (Java's closest is `DeleteContext.fromRecordSchema(Properties, HoodieSchema)`, with different parameters), `RecordContext::delete_batch_to_keys`, `ReaderContext::partition_path`, `slice_row` and `FileGroupMergeStream::current_in_memory_bytes`. Second mirrors: the five `HoodieMergedLogRecordReader` forwarders to the base reader's getters, which Java defines once on `BaseHoodieLogRecordReader`.
- **Keep with `#[expect(dead_code, reason = "...")]`**, naming the Java counterpart or the missing caller. Examples: the four Lombok getters `get_progress` and `get_valid_block_instants` on the base log reader (`getProgress`, `getValidBlockInstants`), `HoodieFileGroupReader::valid_block_instants` (`getValidBlockInstants`) and `DataBlock::data` (`HFileBlock.getByteBuff`), plus `FileGroupRecordBuffer::take_next`, `BufferType::Unmerged` and the unread HFile block header fields.
- **Test-only items** are `#[cfg(test)]`, such as `TimelineLoader::storage`, `ReaderContext::empty`, `HoodieFileGroupReader::builder` and the merged reader's stats getters. Where `#[cfg(test)]` does not fit, such as struct fields, the builder impl that also holds the Java-parity setters, and the pull-style trait methods, the item uses `#[cfg_attr(not(test), expect(dead_code, reason = "..."))]` instead: the lib test build uses it, so a plain `expect` would be unfulfilled there.
- **Compile only with `spill-rocksdb`** the constants, types and test helpers that only the RocksDB disk tier uses. The module-wide allow in `buffer` hid that they are dead without the feature.

An `expect`ed item counts as live, so the items it calls are not reported, and only the root of each unused call graph carries an attribute. Three trait methods, `contains_log_record`, `current_in_memory_bytes` and `set_reader_schema`, are `#[cfg(test)]` on the trait and both implementations instead. The position-based buffer forwards each of them to its inner key-based buffer, and rustc counts that call as a use of the trait method, so an `expect` on the trait method is never fulfilled. With the forwarding calls removed, the same `expect` is fulfilled.

Scope stays crate-internal. `TimelineView::start_timestamp` is kept rather than deleted, because removing it changes the public `TimelineView::new`. A field that is only assigned to is not reported, because rustc counts an assignment as a use; only a field that is never touched after construction is reported. No new tooling is added: re-adding a module-wide `#![allow(dead_code)]` is caught only by the new rule in `crates/AGENTS.md`, which `AGENTS.md` asks for when a coding convention changes. No runtime behavior changes.

## How are the changes test-covered

- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
  - [ ] Details are described below

No new tests: the lint is the check. CI checks the lib with and without default features, and the lib test build with all features only. `make check-rust` uses `--all-features`. An item used only with `spill-rocksdb` is caught by CI's `cargo clippy -p hudi-core --lib --no-default-features -- -D warnings`. Locally, `cargo clippy -p hudi-core` with `--lib` and with `--all-targets`, each with default features and with `--no-default-features`, reports no dead code and no unfulfilled expectation. The existing `hudi-core` lib tests cover the edited code and pass with default features and with `--no-default-features`.

## AI Disclosure

Human-led contribution. Grok Bot, working for dragonknightk, assisted with this change, including the code, tests, and description. All submitted content was reviewed by a human before submission.

Signed-off-by: dragonknightk <37928252+dragonknightk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-arm the dead-code gate: 27 items in hudi-core have no production caller

1 participant