Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/upstream-sql-tests-label.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Upstream SQL tests by label

# Runs upstream-sql-tests.yml on a pull request when a maintainer adds the tests:run-core-sql label. Use it for a
# pull request that does not start that workflow itself, for example a change in a client. The run removes the
# label when it ends, so to run the tests again, add the label again.
on:
# Not pull_request_target: the tests need no secrets, and pull_request runs the code of a fork with a read-only
# token. The `paths` filter of the pull_request trigger in upstream-sql-tests.yml also applies to a `labeled`
# event, so the label needs this workflow of its own.
pull_request:
types: [labeled]

permissions: {}

jobs:
upstream-sql-tests:
name: Upstream SQL tests
if: github.event.label.name == 'tests:run-core-sql'
uses: $/.github/workflows/upstream-sql-tests.yml
permissions:
contents: read

remove-label:
name: Remove the tests:run-core-sql label
needs: upstream-sql-tests
# The token of a pull request from a fork cannot write: remove the label by hand there.
if: >-
always() && github.event.label.name == 'tests:run-core-sql' &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
timeout-minutes: 5
# A separate job, so that the pull request code in upstream-sql-tests never gets a token that can write.
permissions:
pull-requests: write
steps:
- name: Remove the label
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
NUMBER: ${{ github.event.pull_request.number }}
run: gh api -X DELETE "repos/$REPO/issues/$NUMBER/labels/tests%3Arun-core-sql" || true
3 changes: 3 additions & 0 deletions .github/workflows/upstream-sql-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@ name: Upstream SQL tests
# Runs a curated subset of the ClickHouse core SQL tests (tests/queries/0_stateless) through the HTTP client
# (ClickHouse.Driver) and the native TCP client (ClickHouse.Driver.Tcp). See
# tests/clickhouse-test-runner/README.md.
#
# upstream-sql-tests-label.yml also calls it, for the tests:run-core-sql label on a pull request.

permissions:
contents: read

on:
workflow_call:
workflow_dispatch:
inputs:
clickhouse_version:
Expand Down
4 changes: 3 additions & 1 deletion tests/clickhouse-test-runner/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@ Do not hide it.
test in its test project (read the `AGENTS.md` of the client first, as the root `AGENTS.md` says). Do not
fix a client bug in a pull request that changes the harness: one change per pull request. Some logic
exists in both clients (see the root `AGENTS.md`), so check the other client too. When someone files an
issue for a group, put it on the `Issue:` line of the group.
issue for a group, put it on the `Issue:` line of the group. Such a pull request does not start the
workflow of the harness: to check that the fix breaks no test of the allowlist, add the
`tests:run-core-sql` label (see the README, CI).
3. **The default for a shim bug is a fix in the shim.** Make the shim copy `clickhouse-client` more exactly.
Do not change the shim to hide a client bug, for example by another read method, a type-name workaround
or a correction of a wrong value. The shim must use each client as a user does: `http` reads each value
Expand Down
6 changes: 6 additions & 0 deletions tests/clickhouse-test-runner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,12 @@ of more than 7 elements (`[tcp-tuple-over-7-elements]`).
manually, and on pushes and pull requests that change this directory or the workflow. A pull request that
changes only the clients does not start it; the nightly run tests the clients of `main`.

To run it on such a pull request, add the `tests:run-core-sql` label
([`upstream-sql-tests-label.yml`](../../.github/workflows/upstream-sql-tests-label.yml)). The run tests the
pull request as it is when the label is added, and removes the label when it ends: to test a later push, add
the label again. The token of a pull request from a fork cannot remove the label, so remove it by hand before
you add it again.

- The `build` job builds the shim once (`scripts/build.sh`), and the test jobs use it. The `harness-tests` job
runs the unit and live tests of the shim.
- The matrix of the test jobs is `{protocol: http, tcp} × {shard: 1..8}`, at `-j 4` on 4-vCPU runners.
Expand Down
Loading