Redact multiline and duplicate-key JSON atomically #908

Outcome

A complete JSON document is redacted atomically before evidence is split into lines, while preserving valid JSON layout, duplicate-key order, scalar types, result markers, and bounded processing.

Current defect

The current redaction path splits text into lines before attempting structural JSON handling. A valid document such as:

{
  "password"
  :
  "<opaque-credential>"
}

can separate the sensitive key from its value, so line-local handling never associates them. Conventional json.loads also collapses duplicate object keys, which cannot prove duplicate-key or layout preservation.

This is distinct from the wrapper-recursion defect tracked for Draft #906.

Required repair

• recognize and process a complete bounded JSON document before line splitting;
• preserve whitespace/layout, duplicate-key order, scalar types, and stable result-marker/schema fields;
• redact values selected by sensitive keys and supported opaque credential semantics even when key, colon, and value span lines;
• cover escaped strings, arrays/objects, duplicate sensitive and benign keys, and multiple sensitive occurrences;
• keep invalid/non-JSON text on the bounded general redaction path without leaking raw parser diagnostics;
• retain depth/input-size/performance bounds and fail closed when a safe structural context cannot be established;
• do not normalize/re-serialize evidence in a way that destroys diagnostic layout or silently drops duplicate keys.

Security design constraints

• Use a bounded iterative token/span rewriter before line splitting. Preserve every untouched source slice; do not transform raw text with whole-value json.loads/dict/json.dumps or object_pairs_hook.
• Process complete JSON and bounded JSON spans surrounded by prefixes/suffixes or multiple diagnostic records. A malformed structural candidate that can bridge key, colon, and value across lines fails closed rather than leaking through line-local fallback.
• Decode key/string tokens only for classification. Preserve unchanged escape spelling, whitespace, CR/LF boundaries, duplicate-key order/count, punctuation, and stable result-marker/schema fields.
• Preserve value categories under sensitive keys: string → "[REDACTED]"; integer/number → a zero value of the same JSON number category; boolean → false; null remains null; arrays/objects retain shape and recursively scrub scalar leaves.
• Apply explicit input-byte, depth, token, string, replacement, and cumulative work limits. Limit breach or unsafe parser state returns bounded fail-closed evidence without raw input or exception text.
• Keep redact_json_value() for already-materialized trusted result objects; raw-text layout preservation belongs only to the text path.
• Test JSON command fields containing nested wrappers and shell operands containing JSON so neither parser assumes the other already handled the credential.
• Do not claim end-to-end output-memory closure: unbounded PIPE/service-file capture remains separate Issue Bound sandbox subprocess output and long-running service logs #766.
• Implement on a separate stacked Draft from exact fix(sandbox): replay redaction after fixture-history contamination #906: hosted test-only RED, then bounded GREEN, then Secret Scan/full-history and exact 100% coverage before convergence.

Acceptance

• fail-first fixtures reproduce multiline key/value separation and duplicate-key loss;
• GREEN tests prove layout/order/type/marker preservation and complete credential removal;
• malformed, excessive-depth, high-volume, and mixed text/JSON cases remain bounded;
• no fixed credential-shaped literal is committed;
• Sandbox Log Redaction Quality CI passes at the exact head with 100% statement/branch coverage for the three owned production modules;
• full suite, Secret Scan, security/supply-chain gates, current source review, and independent approval pass;
• protected-main synthetic stdout/stderr/timeout/service-tail evidence is required after integration.

Related: #906.