Problem
The artifact's missing-value policy is "baseline" (median imputation) or "reject". In credit data, missingness is signal: no trade line, no inquiry history, thin file. Median-imputing it destroys that signal and silently maps a thin-file applicant onto the score of a median-file applicant.
Standard practice in scorecard development is to give missing its own bin. We currently cannot express that.
Related: compileml/compile/extract.py already detects XGBoost's missing-value branches and warns that the compiled artifact has no equivalent — then discards the information. That warning is the ticket for this issue.
Proposed approach
Add a third policy, missing_as_bin:
- Spec (
docs/ARTIFACT_SPEC.md §8): define how a node routes an explicitly-missing input. Simplest expressible form is a per-node missing_left: bool, preserving the current two-way structure and integer determinism.
- Compile: capture the teacher's missing direction when available (XGBoost gives it directly); for sklearn, require the caller to declare it.
- Runtime: route NaN/None per the recorded direction instead of imputing.
- Exports: SQL needs explicit
IS NULL handling; COBOL needs a validity flag convention.
Acceptance criteria
Files
docs/ARTIFACT_SPEC.md, src/compileml/runtime/decide.py, src/compileml/runtime/score.py, src/compileml/compile/extract.py, src/compileml/export/*
Scope
Substantial, and it is a spec change — the schema version goes to 3. Worth designing in an issue thread before opening a PR.
Problem
The artifact's missing-value policy is
"baseline"(median imputation) or"reject". In credit data, missingness is signal: no trade line, no inquiry history, thin file. Median-imputing it destroys that signal and silently maps a thin-file applicant onto the score of a median-file applicant.Standard practice in scorecard development is to give missing its own bin. We currently cannot express that.
Related:
compileml/compile/extract.pyalready detects XGBoost's missing-value branches and warns that the compiled artifact has no equivalent — then discards the information. That warning is the ticket for this issue.Proposed approach
Add a third policy,
missing_as_bin:docs/ARTIFACT_SPEC.md§8): define how a node routes an explicitly-missing input. Simplest expressible form is a per-nodemissing_left: bool, preserving the current two-way structure and integer determinism.IS NULLhandling; COBOL needs a validity flag convention.Acceptance criteria
"baseline"and"reject"policies unchangedFiles
docs/ARTIFACT_SPEC.md,src/compileml/runtime/decide.py,src/compileml/runtime/score.py,src/compileml/compile/extract.py,src/compileml/export/*Scope
Substantial, and it is a spec change — the schema version goes to 3. Worth designing in an issue thread before opening a PR.