Skip to content

Treat missingness as informative: add a missing_as_bin policy #11

Description

@orgoca

Problem

The artifact's missing-value policy is "baseline" (median imputation) or "reject". In credit data, missingness is signal: no trade line, no inquiry history, thin file. Median-imputing it destroys that signal and silently maps a thin-file applicant onto the score of a median-file applicant.

Standard practice in scorecard development is to give missing its own bin. We currently cannot express that.

Related: compileml/compile/extract.py already detects XGBoost's missing-value branches and warns that the compiled artifact has no equivalent — then discards the information. That warning is the ticket for this issue.

Proposed approach

Add a third policy, missing_as_bin:

  • Spec (docs/ARTIFACT_SPEC.md §8): define how a node routes an explicitly-missing input. Simplest expressible form is a per-node missing_left: bool, preserving the current two-way structure and integer determinism.
  • Compile: capture the teacher's missing direction when available (XGBoost gives it directly); for sklearn, require the caller to declare it.
  • Runtime: route NaN/None per the recorded direction instead of imputing.
  • Exports: SQL needs explicit IS NULL handling; COBOL needs a validity flag convention.

Acceptance criteria

  • Spec section written before implementation — this changes the artifact contract and needs a schema version bump
  • Runtime routes missing deterministically; a test asserts a missing value and a median value produce different scores
  • SQL export parity test includes NULL rows
  • Existing "baseline" and "reject" policies unchanged

Files

docs/ARTIFACT_SPEC.md, src/compileml/runtime/decide.py, src/compileml/runtime/score.py, src/compileml/compile/extract.py, src/compileml/export/*

Scope

Substantial, and it is a spec change — the schema version goes to 3. Worth designing in an issue thread before opening a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    credit-riskDomain requirement from regulated lending practiceenhancementNew feature or requestspec-changeRequires an ARTIFACT_SPEC.md revision

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions