[Core] Validate coverage for declared partitioned inputs in framework-native training

[jiangxt2]

Description

Add a public alpha Framework Native input route contract and validate declared input-role coverage before a run reports success or publishes a required Bundle. Partitioned roles are checked by summing worker counts; replicated roles are checked against the canonical row count on each worker. The runtime reuses InputCoverageContract, counts routed datasets before trainer construction, and validates staged and cross-fit evidence. The legacy single-train plan remains compatible.

Cross-fit worker evidence aggregates row counts across folds and identifies that scope in state details. The generated API reference and manual Ray Jobs suite inventory are updated.

Related issues

Fixes #121

Additional information

• Tests: 58 focused algorithm-contract tests passed. The repository precheck completed successfully, including Python 3.12/3.13 CI parity and documentation gates; it reported one non-blocking style warning for an inline requests import in the integration fixture.
• Integration: the full manual Docker Ray Jobs suite is not confirmed as passing for this pull request, so no full-suite pass is claimed.
• Performance: counting a routed Ray Dataset can execute lazy operations or scan input data when the source does not provide row-count metadata.
• Dependencies: no dependency changes.

[greptile-apps]

Confidence Score: 4/5

[Medium impact] Adds input role validation to the framework-native training runtime.

The PR appears safe to merge, with a non-blocking gap in the new integration test.

Findings

1.  Coverage gate skips real training ▶

Fix with agent prompt

### Issue 1
tests/training/jobs/framework_native_input_coverage_job.py:61-74
The job builds both worker records on the driver, invents their node IDs, and calls the private coverage helpers directly. It never runs `FrameworkNativeRuntime.execute` or a trainer, so it can pass even if the runtime stops checking coverage after a real fit. Run a small framework-native trainer and collect counts and row IDs inside its workers, then check the resulting evidence.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds declared input routes for framework-native training.

• Framework Native runs check the coverage of each declared input role.
• Cross-fit evidence now shows role counts accumulated across folds.

The new integration job checks helper functions with locally created worker records rather than exercising a real trainer.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Prepared inputs] --> B[Bind datasets]
  B --> C[Check declared routes and count rows]
  C --> D[Build trainer and fit]
  D --> E[Collect worker evidence]
  E --> F[Check each stage or fold]
  F --> G{Coverage matches?}
  G -->|No| H[Fail before export]
  G -->|Yes| I[Return success or export Bundle]

Loading

_{Reviews (1) · Last reviewed commit: "feat(algorithms): validate framework-nat..." · Reviewed by Greptile}

[greptile-apps]

Coverage gate skips real training

The job builds both worker records on the driver, invents their node IDs, and calls the private coverage helpers directly. It never runs FrameworkNativeRuntime.execute or a trainer, so it can pass even if the runtime stops checking coverage after a real fit. Run a small framework-native trainer and collect counts and row IDs inside its workers, then check the resulting evidence.

Prompt To Fix With AI

This is a comment left during a code review.
Path: tests/training/jobs/framework_native_input_coverage_job.py
Line: 61-74

Comment:
**Coverage gate skips real training**

The job builds both worker records on the driver, invents their node IDs, and calls the private coverage helpers directly. It never runs `FrameworkNativeRuntime.execute` or a trainer, so it can pass even if the runtime stops checking coverage after a real fit. Run a small framework-native trainer and collect counts and row IDs inside its workers, then check the resulting evidence.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.