Repository navigation
fix(prow-jobs): let the Tekton validator see every document and kind - #5570
Conversation
The offline structural check read `.apiVersion`, `.kind` and `.metadata.name` once per file. A file holding two documents — two `Trigger`s, for example — made `yq` print `Trigger\n---\nTrigger`, which the kind allowlist rejected, so the check failed on a perfectly valid file as soon as one was touched. Evaluate every document instead and report the ones that are not a named Tekton resource, so a file's document count no longer matters. While here, two kinds of file the check could never accept: - `kustomization.yaml` is not a Tekton manifest; skip it explicitly instead of flagging `Kustomization` and an empty `metadata.name`; - `StepAction` is a Tekton resource and was missing from the allowlist (`tekton/v1/step-actions/*` all failed). Verified locally against every Tekton manifest in the repository (`PULL_BASE_SHA=<empty tree>`): 119 files, exit 0. A fixture with a valid first document and an invalid second one is reported as `invalid document: apiVersion="v1" kind="NotATektonThing" name="bad"` and exits 1.
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: wuhuizuo The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
I have already done a preliminary review for you, and I hope to help you do a better job.
Summary
This PR improves the Tekton YAML validation step in the CI by modifying how .apiVersion, .kind, and .metadata.name fields are checked. Instead of reading these fields once per file (which fails on multi-document files), the script now parses every YAML document within a file and validates them individually. It also skips kustomization.yaml files and adds StepAction to the allowed kinds. Overall, this is a solid fix addressing a concrete bug, and the new logic is clear and more robust.
Critical Issues
- None found. The new approach correctly handles multi-document YAML files and known edge cases.
Code Improvements
-
YQ Expression Complexity (
prow-jobs/pingcap-qe/ci/presubmits.yaml, lines ~327-345):
The currentyqexpression is quite complex and mixes logical checks with string concatenation. This makes it somewhat hard to maintain or extend.Suggestion:
Break the logic into multiple steps or use more explicit variable assignments if possible. Ifyqversion supports it, consider emitting JSON and using jq or simpler filters outsideyq. Alternatively, add comments explaining theselectconditions more verbosely. -
Hardcoded Allowed Kinds List (same file, lines ~335-340):
The allowed kinds list is embedded directly in thetest()regex inside the script.Suggestion:
Consider defining the allowed kinds as a shell variable or a separate config file to ease future updates. For example:ALLOWED_KINDS="Task|TaskRun|Pipeline|PipelineRun|StepAction|Trigger|TriggerTemplate|TriggerBinding|ClusterTriggerBinding|EventListener|Interceptor" ... bad=$(yq e " select( (((.kind // \"\") | test(\"^($ALLOWED_KINDS)$\")) | not) or ... " "$f")
-
Error Output Handling (lines ~337-343):
The script capturesbadoutput including both stdout and stderr (2>&1), but the comment suggests YAML parsing errors should be reported separately.Suggestion:
Separate YAML parsing errors from validation errors for clearer diagnostics. For instance:bad=$(yq e '...' "$f" 2>/dev/null) || { echo " not valid YAML" yq e '.' "$f" # or just print the error message captured elsewhere failed=1 continue }
-
Consistent Exit Status:
The script setsfailed=1on any validation failure but does not explicitly show the final exit status.Suggestion:
At the script end, ensure the exit status reflects the validation result, e.g.:exit $failed
Best Practices
-
Documentation (same file, lines ~330-335):
The added comment about reading multiple documents is helpful, but more detailed explanation of the regex and validation logic would aid future maintainers. -
Testing:
The PR description mentions manual verification but does not include automated tests or fixtures.Suggestion:
Add unit tests or CI integration tests that include multi-document YAML files with both valid and invalid kinds to prevent regressions. -
Shell Script Style:
Minor style improvements could be made, for example:- Use
[[ ... ]]for test conditions in bash. - Quote variable expansions consistently, e.g., "$changed_files" in loops to handle filenames with spaces (if applicable).
- Use
-
Naming:
Variablebadis somewhat ambiguous.Suggestion:
Rename toinvalid_docsorvalidation_errorsfor clarity.
Summary of Suggested Changes
# Define allowed kinds once
ALLOWED_KINDS="Task|TaskRun|Pipeline|PipelineRun|StepAction|Trigger|TriggerTemplate|TriggerBinding|ClusterTriggerBinding|EventListener|Interceptor"
# Skip kustomization.yaml files upfront
changed_files=$(git diff --name-only --diff-filter=d "${PULL_BASE_SHA:?}" -- 'tekton/v1/**/*.yaml' \
| grep -v -E '(^|/)kustomization\.yaml$' | LC_ALL=C sort)
# Loop over files
for f in $changed_files; do
echo "Validating: $f"
# Validate each document separately
validation_errors=$(yq e "
select(
(((.kind // \"\") | test(\"^($ALLOWED_KINDS)$\")) | not)
or (((.apiVersion // \"\") | length) == 0)
or (((.metadata.name // \"\") | length) == 0)
) | \" invalid document: apiVersion=\\\"\" + (.apiVersion // \"\") + \"\\\" kind=\\\"\" + (.kind // \"\") + \"\\\" name=\\\"\" + (.metadata.name // \"\") + \"\\\"\"
" "$f" 2>/dev/null) || {
echo " not valid YAML"
failed=1
continue
}
if [ -n "$validation_errors" ]; then
printf '%s\n' "$validation_errors"
failed=1
fi
done
exit $failedThis would improve readability, configurability, and maintainability.
Overall, this PR is an important fix for a real validation bug and is a sound improvement. Addressing the above improvements will reduce technical debt and improve clarity for future contributors.
Problem
pull-verify-tekton-yamlreads.apiVersion,.kindand.metadata.nameonceper file. A file holding two documents (
_/git-create-branch.yamlholds twoTriggers) makesyqprintTrigger\n---\nTrigger, which the kind allowlistrejects:
So the check fails on a valid file the moment that file is touched — which is
what happened in #5569.
Two kinds of file it could never accept either:
kustomization.yaml— not a Tekton manifest, yet it was flagged forKustomization+ emptymetadata.name;StepAction— a Tekton resource missing from the allowlist, so everytekton/v1/step-actions/*change failed.Change
Evaluate every document and report the ones that are not a named Tekton
resource; skip
kustomization.yaml; addStepActionto the allowlist. A file'sdocument count no longer matters.
Verification
(
PULL_BASE_SHA=<empty tree>): 119 files, exit 0.Triggerand whosesecond is
kind: NotATektonThingreportsinvalid document: apiVersion="v1" kind="NotATektonThing" name="bad"and exits 1(and the valid document produces no noise).