Repository navigation
fix(prow-jobs): validate Tekton manifests offline instead of against a cluster - #5553
Conversation
…a cluster The validator ran `kubectl apply --dry-run=server`, but the presubmit pod has no kubeconfig: kubectl dials localhost:8080 and the step fails with "connect: connection refused". That is why the original `kubectl … | sed` pipeline could only ever report success — the failure it was meant to catch never became the step's status, and if it had, the job would have failed on every Tekton change for a reason unrelated to the manifest. Check what can be checked without a cluster: the file parses as YAML, its `kind` is a Tekton kind, and it sets `apiVersion` and `metadata.name`. That is a real gate (a typo, a missing field or a non-Tekton document fails it) and it needs no credentials.
There was a problem hiding this comment.
I have already done a preliminary review for you, and I hope to help you do a better job.
Summary
This PR replaces the previous cluster-dependent kubectl apply --dry-run=server validation of Tekton manifests with an offline, structural validation using yq to parse YAML and check required fields and kinds. This approach removes the need for kubeconfig or API server access in the presubmit job, fixing the flaky validation and ensuring failures are surfaced correctly. The implementation is concise and well-scoped, improving reliability without impacting existing Tekton resources, as verified locally.
Critical Issues
- None found. The new offline validation approach correctly addresses the root cause of flaky validation due to lack of kubeconfig and connection refusal errors.
Code Improvements
-
Repeated
yqinvocations on the same file (prow-jobs/pingcap-qe/ci/presubmits.yaml, lines ~327-342)
Currently,yqis called three times per file to extract.apiVersion,.kind, and.metadata.name. This causes unnecessary parsing overhead and potential inconsistency if the file changes between calls (unlikely here but still better practice to parse once).
Suggestion: Parse the file once into variables, e.g.:api=$(yq e '.apiVersion // ""' "$f" 2>/dev/null) || { echo " not valid YAML"; failed=1; continue; } kind=$(yq e '.kind // ""' "$f" 2>/dev/null) name=$(yq e '.metadata.name // ""' "$f" 2>/dev/null)
Can be replaced by:
# Parse all needed fields at once read api kind name < <(yq e '.apiVersion + " " + .kind + " " + .metadata.name' "$f" 2>/dev/null) || { echo " not valid YAML" failed=1 continue }
This reduces command invocations and improves performance and readability.
-
Use of
casefor kind validation is good, but consider extracting the list of valid kinds into a variable for easier maintenance, e.g.:valid_kinds="Task TaskRun Pipeline PipelineRun Trigger TriggerTemplate TriggerBinding ClusterTriggerBinding EventListener Interceptor" if ! echo "$valid_kinds" | grep -qw "$kind"; then echo " unknown or missing kind: '${kind}'" failed=1 fi
This makes adding/removing kinds simpler.
-
Error messages: Add quotes around variables in messages to avoid ambiguity if fields contain spaces or are empty, e.g.:
echo " unknown or missing kind: '${kind}'" echo " missing apiVersion" echo " missing metadata.name"
-
Fail-fast behavior: Currently the script continues validating all files after encountering errors. This is fine for reporting all errors in one run, but consider whether failing immediately on the first error is preferable in your context to save time.
Best Practices
-
Comments:
- The added inline comments explaining why offline validation is used are clear and helpful.
- Consider adding a short comment above the kind validation listing the valid kinds, to help future maintainers.
-
Testing coverage:
- The PR description mentions local manual testing. Consider adding automated tests (if possible in your CI framework) for this validation step, e.g., testing a sample valid and invalid manifest file and verifying the exit status and output.
-
Exit status and output:
- It is good that the failure exit status is properly propagated and that the output is indented similarly to previous validation for readability.
-
Shell portability:
- The script assumes
yqis installed and available, which is fine if guaranteed in the job image. - The use of
readwith process substitution is POSIX compatible in bash but may fail insh-only environments; confirm shell compatibility in your CI image.
- The script assumes
-
Code style:
- Consistent indentation and quoting are used.
- Consider quoting all variable expansions to avoid word splitting, e.g.,
"$changed_files"in the for loop if filenames may contain spaces. - The for loop
for f in $changed_files; docan break on filenames with spaces. Use a safer method if filenames with spaces are possible, e.g.:Or use an array.IFS=$'\n' for f in $changed_files; do
Overall, the PR effectively fixes the flaky offline validation problem with a clean and maintainable approach. Improving the efficiency of YAML parsing and slightly enhancing maintainability around the kind validation would further strengthen it. Adding automated tests for this validation step is recommended for future stability.
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: wuhuizuo The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Problem
#5550 fixed the shell (the job ran bash on an alpine image and could not start),
and the canary then showed the second half:
kubectl apply --dry-run=servercannot work in this pod — no kubeconfig is mounted, so kubectl dials
localhost:8080and fails withconnect: connection refused.That also explains the original defect: the
kubectl … | sedpipeline reportedsed's status, so a validation failure was masked. Had it not been masked, the job
would have failed on every Tekton change for a reason unrelated to the manifest.
Change
Validate what can be validated without a cluster:
kindis a Tekton kind (Task,TaskRun,Pipeline,PipelineRun,Trigger,TriggerTemplate,TriggerBinding,ClusterTriggerBinding,EventListener,Interceptor);apiVersionandmetadata.nameare set.This is a real gate — a typo, a missing field or a non-Tekton document fails it —
and it needs no credentials. The step now also uses the same capture-then-format
pattern as #5550, so a failure actually reaches the exit status.
Verification
kind: NotATektonKindfails withunknown or missing kindand exit 1.tekton/on purpose: the presubmit reads its definitionfrom the base branch, so a Tekton change here would run the old body. A canary
PR follows once this is merged.