Skip to content

feat(client-node): pluggable Connection — createClient({ connection }) injection point - #879

Merged
peter-leonov-ch merged 3 commits into
ClickHouse:mainfrom
ShawnChen-Sirius:feat/pluggable-connection
Jun 23, 2026
Merged

peter-leonov-ch merged 3 commits into
ClickHouse:mainfrom
ShawnChen-Sirius:feat/pluggable-connection

Conversation

@ShawnChen-Sirius

Copy link
Copy Markdown
Contributor

What

A single, additive hook on the Node client: createClient({ connection }) lets a caller plug an externally-built Connection<Stream.Readable> into createClient instead of letting the package build its default HTTP connection.

import { createClient } from '@clickhouse/client';
import { createChdbConnection } from 'chdb/connection';

const client = createClient({
  connection: createChdbConnection({ path: ':memory:' }),
});

client.query / insert / command / exec / ping / close then route through the injected Connection's existing contract. When connection is omitted (the default), behavior is byte-identical to today.

Tracking issue: #865.
Companion PR (the chdb-node side): chdb-io/chdb-node#52.

Shape

3 lines of actual code: make_connection factory override when connection is provided, spread of NodeConfigImpl otherwise. No HTTP/HTTPS code touched. No new public types added. The new field is @experimental so we can iterate on shape without semver pressure.

See docs/design/pluggable-connection.md for the full rationale and the asymmetric upstream-clean design (zero chdb code, zero chdb tests, zero chdb CI in this repo — all chdb-side logic lives in chdb-node).

Why this shape

Two design pressures, in tension:

  1. chDB's reason for existing is in-process zero-copy I/O. Hiding chdb behind a loopback HTTP server (or any wire-format boundary) erases that.
  2. @clickhouse/client's public surface is intentionally slim. Bolting a second client family onto it — one mirroring the entire public API for chdb — is a maintenance trap.

The compromise:

  • One public client API stays createClient from @clickhouse/client.
  • The Connection<Stream> interface behind it is unchanged — only injectable.
  • The backend implementation lives in the backend's repo. chdb-node ships ChdbConnection from chdb/connection; all chdb-side data, skip list, CI matrix that runs this suite against ChdbConnection stays in chdb-node's repo.

Verification (chdb-node side)

The companion chdb-node PR carries tests/clickhouse-js/runner.mjs, which clones this repo's integration suite and runs all 232 tests against an in-process ChdbConnection. Currently 202 / 232 pass. The 30 skips are documented in chdb-node's skip_list.json with reasons — every one is either an HTTP-only behavior (no socket in-process) or a chdb engine gap (default timezone, etc.).

This is the first real-world consumer of the injection point; if shape needs to change based on what we learn here, the @experimental tag is the room to do that without semver pressure.

Backwards compatibility

Default code path (connection omitted) is unchanged — same NodeConfigImpl, same make_connection, byte-identical behavior. No existing user of createClient sees a difference.

Test plan

  • Existing test suite green (no behavior change when connection is omitted)
  • TypeScript check clean
  • End-to-end verification via chdb-node companion PR: 202 / 232 integration tests pass against ChdbConnection

Generated with Claude Code
via Happy

ShawnChen-Sirius and others added 2 commits June 23, 2026 14:18
…oint

Add an optional `connection` field on `NodeClickHouseClientConfigOptions`
that lets a caller plug an externally-built `Connection<Stream.Readable>`
into `createClient` instead of letting the package build its default HTTP
connection from `tls` / `keep_alive` / `http_agent` / etc. options.

  const client = createClient({
    connection: createChdbConnection({ path: ':memory:' }),
  })

This is the single hook needed for an external backend (e.g. chdb-node's
in-process `ChdbConnection`) to plug into the public client API: the
existing `Connection<Stream>` interface stays unchanged — the backend
implements that exact contract and gets dropped in. When `connection`
is omitted (the default), behavior is byte-identical to today: the
bundled HTTP factory is used.

Implementation is a 3-line wrap of `NodeConfigImpl`: when `connection`
is provided, spread `NodeConfigImpl` and override only its
`make_connection` factory to return the injected connection verbatim;
otherwise pass `NodeConfigImpl` through unchanged. No HTTP/HTTPS code
is touched, and the public `Connection<Stream>` interface is unchanged.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Document the single new hook on createClient (Connection injection) and
make explicit what this PR deliberately does NOT do:

  - no chdb-specific code paths in this repo;
  - no per-feature test markers;
  - no skip-profile loader; backends own their blacklists;
  - no dependency on chdb;
  - test parity is owned by the backend's repo (chdb-node clones
    THIS repo's suite, runs it with ChdbConnection injected, and
    applies its OWN tests/clickhouse-js/skip_list.json).

The sync policy is described as: chdb-node tests against this PR
branch until merged, then against main, then against the latest
released tag — re-running parity only when this repo cuts a new
release.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an additive injection point to the Node.js client so callers can supply a pre-built backend Connection<Stream.Readable> via createClient({ connection }), bypassing the default HTTP(S) connection factory. This enables pluggable backends (e.g. in-process chDB) while keeping the existing HTTP path unchanged when connection is omitted.

Changes:

  • Extends NodeClickHouseClientConfigOptions with an experimental connection?: Connection<Stream.Readable> option.
  • Updates createClient to override make_connection to return the injected connection when provided.
  • Adds a design doc describing the rationale, boundaries, and intended usage (docs/design/pluggable-connection.md).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
packages/client-node/src/config.ts Adds the experimental connection config option and accompanying JSDoc.
packages/client-node/src/client.ts Implements the createClient({ connection }) injection behavior by overriding make_connection.
docs/design/pluggable-connection.md Documents the design rationale and usage pattern for pluggable connections.

Comment thread packages/client-node/src/client.ts Resolved
Comment thread packages/client-node/src/client.ts Resolved
Comment thread packages/client-node/src/config.ts Resolved
Comment thread packages/client-node/src/config.ts Resolved
Comment thread packages/client-node/src/config.ts Resolved
…est, abs URL, Connection re-export

Single follow-up commit addressing all 4 line-level review comments on
PR ClickHouse#879, on top of the original feat + docs commits.

### `Connection` contract types now re-exported from `@clickhouse/client`

`createClient({ connection })` is typed as `Connection<Stream.Readable>`,
but the `Connection` type and its companions (`ConnBaseQueryParams`,
`ConnQueryResult`, `ConnInsertParams`, `ConnExecParams`, `ConnPingParams`,
`ConnPingResult`, `ConnInsertResult`, `ConnExecResult`,
`ConnCommandResult`, `ConnOperation`, `ConnBaseResult`,
`ClickHouseSummary`, `WithClickHouseSummary`) were only reachable via
the deprecated `@clickhouse/client-common`. Third-party backends
implementing the injection point now don't need to deep-import; they
can `import type { Connection } from "@clickhouse/client"` directly.

### Absolute GitHub URL in `connection?` JSDoc

The JSDoc on `NodeClickHouseClientConfigOptions.connection` pointed at
`docs/design/pluggable-connection.md` (repo-relative). Matches the
existing convention used in other JSDoc references throughout the
codebase (e.g. `connection/stream.ts`, `connection/socket_pool.ts`)
and renders correctly from IDE hovers / generated API docs.

### Unit test for the injection path

New file: `packages/client-node/__tests__/unit/node_create_client_with_connection.test.ts`.
8 tests covering the two contract guarantees:

  1. `NodeConnectionFactory.create` is NOT invoked when `connection` is
     provided (`vi.spyOn` + `not.toHaveBeenCalled()`).
  2. The default HTTP path IS used when `connection` is omitted (sanity
     check that the override is conditional, not unconditional).
  3-8. `client.query` / `insert` / `command` / `exec` / `ping` /
       `close` all delegate to the injected stub's matching method,
       with the right `ConnBaseQueryParams` / `ConnInsertParams` shape
       on the way in.

Verified: 8/8 pass, full client-node unit suite has no new failures
(`node_config` + `node_create_connection` 21/21 still pass; the two
pre-existing test-infra failures in `node_client.test.ts` and
`node_exception_header.test.ts` are unrelated `@test/utils/simple_client`
import errors that existed before this PR).

### CHANGELOG.md entry

Added a `1.23.0 > New features` bullet for `(Node.js, @experimental)
createClient({ connection })`. Calls out:

- the additive nature (default path byte-identical when omitted),
- the implementation shape (3-line `make_connection` factory override),
- the intended consumer (chdb-node's `chdb/connection` subpath export),
- the `Connection` contract types now being re-exported from this
  package,
- a pointer to `docs/design/pluggable-connection.md`,
- and the `@experimental` tag rationale (iterate shape based on what
  the first real backend learns, without semver pressure).

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
@peter-leonov-ch
peter-leonov-ch merged commit c2e28b9 into ClickHouse:main Jun 23, 2026
59 checks passed
peter-leonov-ch added a commit that referenced this pull request Jun 23, 2026
…w-up to #879) (#880)

## What this does

A small follow-up to #879. It dials the pluggable `connection` option
back from a *public plugin surface* to a **deliberately narrow, internal
`@experimental` hook** whose only intended consumer today is the chDB
integration.

Concretely:

- **Removes the public re-exports** of the `Connection<Stream>` contract
(`Connection`, `ConnBaseQueryParams`, `ConnQueryResult`,
`ConnInsertParams`, `ConnPingResult`, `ClickHouseSummary`, …) from the
`@clickhouse/client` (Node) entrypoint.
- **Deletes `docs/design/pluggable-connection.md`**, which described a
fuller pluggable-backend system we are intentionally not committing to
yet.
- **Trims the `CHANGELOG` and `connection?` JSDoc** to state plainly
that the field exists to unblock chDB and may change.
- **Moves the rationale into the `connection?` JSDoc itself**, so it
lives right where anyone touching the field will read it.

The runtime behavior is unchanged: the `make_connection` factory
override still works exactly as merged in #879. This is purely a
framing/surface-area change.

## Why

Two pressures, gently in tension:

1. We genuinely want to **unblock chDB**
([chdb-io/chdb-node#52](chdb-io/chdb-node#52))
with an in-process backend, today.
2. We do **not** want to turn the Node client into the common package
replacement — a public pluggable-backend framework implies a stable
third-party contract and invites a second client family mirroring the
entire public API, which is a real long-term maintenance burden.

Keeping the `Connection` types out of the public entrypoint is what
*enforces* this distinction. A backend has to deep-import or
structurally match the shape — and that small friction is by design: it
signals "unsupported, may change" rather than promising an API we are
not ready to own. Nothing internal depends on the re-exports (internal
code imports these types from `./common`), and `npm run typecheck`
passes.

## Next steps (not in this PR)

1. **Verify it works and is actually needed in production** with the
real chDB backend before hardening anything.
2. If validated, **build a specialized client** that drops the HTTP
connection-pooling burden entirely and leans into the in-process
transport — leaving the higher-level query API as the surface we invest
in and improve, rather than retrofitting it onto the HTTP-shaped client.

Feedback very welcome — this is intentionally the smallest step that
unblocks chDB while keeping our options open.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants