Skip to content

feat: enhance batch matching with expected record keys - #70

Merged
nil-malh merged 2 commits into
mainfrom
feat/batch-keys-matching
Oct 18, 2026
Merged

nil-malh merged 2 commits into
mainfrom
feat/batch-keys-matching

Conversation

@nil-malh

@nil-malh nil-malh commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

📝 Summary

Add support for expectedRecordKeys in batch assertion steps (Then expected records from files and Then expected records from files based on schema):

  • Context & Transport (ktestify-core):
    • Add expectedRecordKeys (List<String>) to ConsumerContext and matchKeys (List<String>) to MatchContext (wired in AbstractKafkaConsumer.buildMatchContext()).
    • Update KafkaRecordFetcher.passesKeyFilter() so that when expectedRecordKeys is non-empty, only records matching at least one expected key (using the configured keyMatchStrategy) are claimed during batch polling.
  • Batch Matchers (ktestify-core):
    • Update FileRecordMatcher and AvroFileRecordMatcher to validate matchKeys.size() == matchFilePaths.size() when matchKeys is provided (throwing ComparisonException on count mismatch).
    • POSITIONAL mode: assert that record at index i matches both expectedRecordKeys[i] (via keyMatchStrategy) and files[i].
    • LAZY mode: perform order-independent 1-to-1 matching without replacement where a record claims index j only when both its key matches expectedRecordKeys[j] and its value matches files[j].
  • Cucumber Runner & Docs (ktestify-cucumber & ktestify-doc):
    • Add DATA_TABLE_EXPECTED_RECORD_KEYS = "expectedRecordKeys" in DataTableUtils.Constants and parse comma-separated keys in ConsumerValidationService.validateRawBatch and validateAvroBatch.
    • Document expectedRecordKeys across batch-assertions.mdx, step-reference.mdx, batch-testing.mdx, core-concepts.mdx, built-in-matchers.mdx, assertion-steps.mdx, complete.mdx, and README.md.

Closes #

🔄 Type of change

  • 🐛 Bug fix (fix): non-breaking change that fixes an issue
  • ✨ Feature (feat): non-breaking change that adds functionality
  • 💥 Breaking change (!): changes existing behaviour, public API, Gherkin steps or config keys
  • ♻️ Refactor (refactor): no functional change
  • ⚡ Performance (perf)
  • 📖 Documentation (docs)
  • ⬆️ Dependencies (build(deps))
  • 🔧 CI / build (ci / build): workflows, pom.xml, tooling

✅ Checklist

Code

  • Formatting passes mvn spotless:check (run mvn spotless:apply to fix it and add the Apache 2.0 headers)
  • Public types and methods have up-to-date Javadoc

Tests

  • New / changed behaviour is covered by tests
  • mvn verify passes locally, and JaCoCo line coverage stays ≥ 70 %
  • Testcontainers-based tests are isolated and clean up after themselves

Compatibility & docs

  • New or changed HOCON keys have a default in reference.conf and a documented env-var override
  • New or changed Gherkin steps are documented, with an example scenario
  • New SPI implementations are registered in META-INF/services/
  • User-facing changes are documented (docs PR: )

🧪 How to test

mvn spotless:check
mvn verify

# ktestify-cucumber / plugins: also run the integration suite
mvn verify -Pintegration-tests -Dcucumber.it.tags="@integration"

📸 Logs / screenshots

Output

💬 Notes for reviewers

Added support for expected record keys in batch matchers, allowing
assertions to validate both record keys and values against expected
files. This change introduces new validation logic to ensure that
the number of expected keys matches the number of files, and
enhances logging to provide clearer feedback on key mismatches.

This update affects the behavior of both positional and lazy batch
matching strategies, improving the flexibility and accuracy of
record matching in batch processing scenarios.
@github-actions

Copy link
Copy Markdown
Contributor

📋 Unreleased Changelog Preview

This is what the next release notes will look like based on commits in this PR.

Changelog

All notable changes to this project will be documented in this file.

[Unreleased]

♻️ Refactoring

✨ Features

  • Enhance batch matching with expected record keys

Generated by git-cliff


🔄 Run #191 · Sun, 18 Oct 2026 15:04:23 GMT

@nil-malh
nil-malh merged commit 6e37c93 into main Oct 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant