Skip to content

fix(prow-jobs): let the Tekton validator see every document and kind - #5570

Merged
ti-chi-bot[bot] merged 1 commit into
mainfrom
fix/tekton-validator-multidoc
Nov 13, 2026
Merged

ti-chi-bot[bot] merged 1 commit into
mainfrom
fix/tekton-validator-multidoc

Conversation

@wuhuizuo

Copy link
Copy Markdown
Contributor

Problem

pull-verify-tekton-yaml reads .apiVersion, .kind and .metadata.name once
per file. A file holding two documents (_/git-create-branch.yaml holds two
Triggers) makes yq print Trigger\n---\nTrigger, which the kind allowlist
rejects:

unknown or missing kind: 'Trigger
---
Trigger'

So the check fails on a valid file the moment that file is touched — which is
what happened in #5569.

Two kinds of file it could never accept either:

  • kustomization.yaml — not a Tekton manifest, yet it was flagged for
    Kustomization + empty metadata.name;
  • StepAction — a Tekton resource missing from the allowlist, so every
    tekton/v1/step-actions/* change failed.

Change

Evaluate every document and report the ones that are not a named Tekton
resource; skip kustomization.yaml; add StepAction to the allowlist. A file's
document count no longer matters.

Verification

  • Ran the check against every Tekton manifest in the repository
    (PULL_BASE_SHA=<empty tree>): 119 files, exit 0.
  • Negative case: a fixture whose first document is a valid Trigger and whose
    second is kind: NotATektonThing reports
    invalid document: apiVersion="v1" kind="NotATektonThing" name="bad" and exits 1
    (and the valid document produces no noise).

The offline structural check read `.apiVersion`, `.kind` and `.metadata.name`
once per file. A file holding two documents — two `Trigger`s, for example — made
`yq` print `Trigger\n---\nTrigger`, which the kind allowlist rejected, so the
check failed on a perfectly valid file as soon as one was touched.

Evaluate every document instead and report the ones that are not a named Tekton
resource, so a file's document count no longer matters.

While here, two kinds of file the check could never accept:

- `kustomization.yaml` is not a Tekton manifest; skip it explicitly instead of
  flagging `Kustomization` and an empty `metadata.name`;
- `StepAction` is a Tekton resource and was missing from the allowlist
  (`tekton/v1/step-actions/*` all failed).

Verified locally against every Tekton manifest in the repository
(`PULL_BASE_SHA=<empty tree>`): 119 files, exit 0. A fixture with a valid first
document and an invalid second one is reported as
`invalid document: apiVersion="v1" kind="NotATektonThing" name="bad"` and exits 1.
@wuhuizuo

Copy link
Copy Markdown
Contributor Author

/approve

@ti-chi-bot

ti-chi-bot Bot commented Nov 13, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wuhuizuo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Nov 13, 2026

@ti-chi-bot ti-chi-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have already done a preliminary review for you, and I hope to help you do a better job.

Summary
This PR improves the Tekton YAML validation step in the CI by modifying how .apiVersion, .kind, and .metadata.name fields are checked. Instead of reading these fields once per file (which fails on multi-document files), the script now parses every YAML document within a file and validates them individually. It also skips kustomization.yaml files and adds StepAction to the allowed kinds. Overall, this is a solid fix addressing a concrete bug, and the new logic is clear and more robust.


Critical Issues

  • None found. The new approach correctly handles multi-document YAML files and known edge cases.

Code Improvements

  • YQ Expression Complexity (prow-jobs/pingcap-qe/ci/presubmits.yaml, lines ~327-345):
    The current yq expression is quite complex and mixes logical checks with string concatenation. This makes it somewhat hard to maintain or extend.

    Suggestion:
    Break the logic into multiple steps or use more explicit variable assignments if possible. If yq version supports it, consider emitting JSON and using jq or simpler filters outside yq. Alternatively, add comments explaining the select conditions more verbosely.

  • Hardcoded Allowed Kinds List (same file, lines ~335-340):
    The allowed kinds list is embedded directly in the test() regex inside the script.

    Suggestion:
    Consider defining the allowed kinds as a shell variable or a separate config file to ease future updates. For example:

    ALLOWED_KINDS="Task|TaskRun|Pipeline|PipelineRun|StepAction|Trigger|TriggerTemplate|TriggerBinding|ClusterTriggerBinding|EventListener|Interceptor"
    ...
    bad=$(yq e "
      select(
        (((.kind // \"\") | test(\"^($ALLOWED_KINDS)$\")) | not)
        or ...
    " "$f")
  • Error Output Handling (lines ~337-343):
    The script captures bad output including both stdout and stderr (2>&1), but the comment suggests YAML parsing errors should be reported separately.

    Suggestion:
    Separate YAML parsing errors from validation errors for clearer diagnostics. For instance:

    bad=$(yq e '...' "$f" 2>/dev/null) || {
      echo "  not valid YAML"
      yq e '.' "$f"  # or just print the error message captured elsewhere
      failed=1
      continue
    }
  • Consistent Exit Status:
    The script sets failed=1 on any validation failure but does not explicitly show the final exit status.

    Suggestion:
    At the script end, ensure the exit status reflects the validation result, e.g.:

    exit $failed

Best Practices

  • Documentation (same file, lines ~330-335):
    The added comment about reading multiple documents is helpful, but more detailed explanation of the regex and validation logic would aid future maintainers.

  • Testing:
    The PR description mentions manual verification but does not include automated tests or fixtures.

    Suggestion:
    Add unit tests or CI integration tests that include multi-document YAML files with both valid and invalid kinds to prevent regressions.

  • Shell Script Style:
    Minor style improvements could be made, for example:

    • Use [[ ... ]] for test conditions in bash.
    • Quote variable expansions consistently, e.g., "$changed_files" in loops to handle filenames with spaces (if applicable).
  • Naming:
    Variable bad is somewhat ambiguous.

    Suggestion:
    Rename to invalid_docs or validation_errors for clarity.


Summary of Suggested Changes

# Define allowed kinds once
ALLOWED_KINDS="Task|TaskRun|Pipeline|PipelineRun|StepAction|Trigger|TriggerTemplate|TriggerBinding|ClusterTriggerBinding|EventListener|Interceptor"

# Skip kustomization.yaml files upfront
changed_files=$(git diff --name-only --diff-filter=d "${PULL_BASE_SHA:?}" -- 'tekton/v1/**/*.yaml' \
  | grep -v -E '(^|/)kustomization\.yaml$' | LC_ALL=C sort)

# Loop over files
for f in $changed_files; do
  echo "Validating: $f"
  # Validate each document separately
  validation_errors=$(yq e "
    select(
      (((.kind // \"\") | test(\"^($ALLOWED_KINDS)$\")) | not)
      or (((.apiVersion // \"\") | length) == 0)
      or (((.metadata.name // \"\") | length) == 0)
    ) | \"  invalid document: apiVersion=\\\"\" + (.apiVersion // \"\") + \"\\\" kind=\\\"\" + (.kind // \"\") + \"\\\" name=\\\"\" + (.metadata.name // \"\") + \"\\\"\"  
  " "$f" 2>/dev/null) || {
    echo "  not valid YAML"
    failed=1
    continue
  }
  if [ -n "$validation_errors" ]; then
    printf '%s\n' "$validation_errors"
    failed=1
  fi
done
exit $failed

This would improve readability, configurability, and maintainability.


Overall, this PR is an important fix for a real validation bug and is a sound improvement. Addressing the above improvements will reduce technical debt and improve clarity for future contributors.

@ti-chi-bot ti-chi-bot Bot added the size/M label Nov 13, 2026
@ti-chi-bot
ti-chi-bot Bot merged commit eedc060 into main Nov 13, 2026
10 checks passed
@ti-chi-bot
ti-chi-bot Bot deleted the fix/tekton-validator-multidoc branch November 13, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

1 participant