Skip to content

Specify and test ambiguous execution classification #16

Description

@ovrsr

This is a policy issue, not a beginner security fix

Do not open a PR against this until the policy questions below are answered here. Adding more regex patterns is not a close-out.

Observed behaviour (source-level)

In packages/enforcement-core/src/risk-classifier.ts:

  1. Composition is first-match, not highest-risk. classifyToolCall() builds a list [classifyFilesystem, classifyExec, classifyHttp, classifyMessage, classifyCodePatch, classifyInternalHeartbeat, classifyInternalRead, classifyGatewayTool] and returns the first non-null result. The file header says the classifier “returns the highest applicable risk class” when unsure. A tool whose name matches more than one family (for example a name containing both write and shell, with both path and command parameters) is judged by the filesystem classifier alone; the exec classifier never sees the command.
  2. Recognized exec with no matched pattern allows. classifyExec() returns exec.benign / allow for any tool whose name matches /exec|shell|bash|sh|run|command/i and whose command matched none of the credential, outbound, gateway, install, publish, system-modify, or delete patterns. The README documents exec.benign as an explicit exception to “unrecognized tool calls do not default to allow”, so this is documented behaviour, but it means a command the heuristics cannot read (encoded, indirected, or in an unfamiliar syntax) is allowed rather than escalated.
  3. Opaque command shapes. command is taken from params.command ?? params.cmd ?? params.argv and stringified. An argv array becomes a comma-joined string; a command passed under another key is treated as no command and falls through to unknown.unclassified (approval), which is safe but inconsistent with (2).

Nothing here contradicts docs/CAPABILITY_STATUS.md, which labels dispatcher enforcement PARTIAL with a heuristic classifier. The gap is that the policy for overlapping and opaque cases is implicit in code order rather than stated and tested.

Policy questions for the maintainer

  • Should composition select the most restrictive decision among all non-null classifier results (block > approval > allow), the first match (current), or the first match with a documented family precedence? State the rule and the reason.
  • Should an exec command that matches no pattern remain exec.benign / allow, or become a distinct classification (for example exec.unrecognized) whose default decision is approval in operator-present mode and abstain unattended, with exec.benign reserved for an explicit allowlist of shapes? What does that do to false-positive rates on the benign corpus?
  • How should non-string command shapes (arrays, objects, nested args) be normalized before matching, and should un-normalizable shapes be treated as opaque (approval) rather than empty (fall-through)?
  • Which of these are v1.x behaviour changes requiring a CAPABILITY_STATUS.md row update and a README caveat edit in the same PR?

What closes this issue

  1. The answers above recorded in this issue and, once decided, in a short policy note (docs/governance/ or the classifier header) that the code cites.
  2. Regression fixtures in test/fixtures/classifier-adversarial.json and classifier-benign.json covering: overlapping tool-name families; an exec command with no matched pattern; an argv array; a command under a non-standard key; a base64-wrapped command. Each fixture asserts the decided classification and decision.
  3. Unit tests in packages/enforcement-core/src/risk-classifier.test.ts for the composition rule itself.
  4. npm run test:corpus, npm run self-test, and npm run test -w @fides-anima/fpp-enforcement-core pass; coverage floors in the plugin .c8rc.json files are not lowered.
  5. If behaviour changed: docs/CAPABILITY_STATUS.md and the README “Honest Caveats” bullet on enforcement coverage updated in the same PR.

Paths

packages/enforcement-core/src/risk-classifier.ts, packages/enforcement-core/src/risk-classifier.test.ts, test/fixtures/classifier-*.json, scripts/run-classifier-corpus.ts, scripts/self-test.ts, docs/CAPABILITY_STATUS.md, README.md.

Non-goals

  • Adding more regex patterns as the sole change.
  • Claiming that any outcome here makes the classifier non-bypassable; it remains a heuristic and the capability matrix should keep saying so.
  • Changing the disposition engine or mandate paths; this issue is about classification inputs and composition only.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    design-neededA maintainer decision is outstanding; a PR is prematureenforcement-corepackages/enforcement-core classifier or dispositionpolicyClassifier or enforcement policy; needs an explicit decision

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions