diff --git a/README.md b/README.md index 00d366c..2aa3995 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,8 @@ For production environments, prefer the official [DamengDB/dmPython](https://git - **Supported build targets**: macOS 14+ ARM64 with CPython 3.9–3.13. Database behavior is supported only where integration tests have evidence. - **Best-effort**: Extended scenarios not currently covered by CI. - **Not guaranteed**: Production SLA commitments, vendor-certified compatibility guarantees, and closed-source component support contracts. -- **Connection security**: `ssl_path` supports encrypted DM8 connections with client certificate and key. The directory must contain `ca-cert.pem`, `client-cert.pem`, and `client-key.pem`; bundled legacy server certificates without a SAN also require an exact `server-cert.pem` pin. A plain server is rejected when `ssl_path` is set. Non-empty `ssl_pwd`, `ukey_name`, and `ukey_pin` remain unsupported. MPP and read/write separation settings are passed through, but cluster routing needs a cluster regression environment. +- **Connection security**: `ssl_path` supports encrypted DM8 connections with client certificate and key. The directory must contain `ca-cert.pem`, `client-cert.pem`, and `client-key.pem`; bundled legacy server certificates without a SAN also require an exact `server-cert.pem` pin. A plain server is rejected when `ssl_path` is set. Non-empty `ssl_pwd`, `ukey_name`, and `ukey_pin` remain unsupported. +- **Primary/standby routing**: Read/write separation modes 1 and 4 were verified against a local DM8 primary/standby pair with autocommit enabled. MPP cluster routing still needs a cluster regression environment. ## Roadmap & Status diff --git a/docs/README_zh.md b/docs/README_zh.md index bcf40b5..9d2d44d 100644 --- a/docs/README_zh.md +++ b/docs/README_zh.md @@ -22,7 +22,8 @@ dmPython 是达梦数据库（DM8）的原生 Python 驱动程序，遵循 [Pyth - **构建支持范围**：macOS 14+ ARM64、CPython 3.9–3.13。数据库行为仅以已有集成测试证据为准。 - **Best-effort（尽力支持）**：尚未纳入 CI 覆盖的扩展使用场景。 - **Not guaranteed（不保证）**：生产 SLA 承诺、厂商认证兼容性与闭源组件支持协议。 -- **连接安全**：`ssl_path` 支持使用客户端证书与私钥连接启用加密的 DM8。目录需包含 `ca-cert.pem`、`client-cert.pem`、`client-key.pem`；服务端证书没有 SAN 的旧版本还需提供与服务端完全一致的 `server-cert.pem`。指定 `ssl_path` 时，未协商加密的连接会报错。非空的 `ssl_pwd`、`ukey_name`、`ukey_pin` 仍不支持；MPP 与读写分离参数已传递到底层驱动，集群路由效果仍需集群环境回归。 +- **连接安全**：`ssl_path` 支持使用客户端证书与私钥连接启用加密的 DM8。目录需包含 `ca-cert.pem`、`client-cert.pem`、`client-key.pem`；服务端证书没有 SAN 的旧版本还需提供与服务端完全一致的 `server-cert.pem`。指定 `ssl_path` 时，未协商加密的连接会报错。非空的 `ssl_pwd`、`ukey_name`、`ukey_pin` 仍不支持。 +- **主备路由**：读写分离模式 1 和 4 已在本机 DM8 主备环境、自动提交模式下验证。MPP 集群路由仍需对应环境回归。 ## 路线图与状态 diff --git a/docs/api-reference.md b/docs/api-reference.md index e22a96d..6905f9e 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -61,7 +61,7 @@ dmPython.connect( - `host` 与 `server` 互斥（只允许设置一个）。 - IPv6 地址使用方括号，例如 `server="[::1]"`；`dsn` 可写为 `"[::1]:5236"`。 - `dmsvc_path` 指向包含 `dm_svc.conf` 的目录；连接时可把 `server` 设为配置文件中的服务名。 -- `mpp_login` 接受 `DSQL_MPP_LOGIN_GLOBAL` 或 `DSQL_MPP_LOGIN_LOCAL`；`rwseparate` 接受 `DSQL_RWSEPARATE_OFF`、`DSQL_RWSEPARATE_ON` 或 `DSQL_RWSEPARATE_ON2`，`rwseparate_percent` 范围为 0–100。这些选项在建连时传给底层驱动，当前仅验证了单机实例上的 MPP LOCAL 握手和参数传递，读写路由仍需主备环境验证。 +- `mpp_login` 接受 `DSQL_MPP_LOGIN_GLOBAL` 或 `DSQL_MPP_LOGIN_LOCAL`；`rwseparate` 接受 `DSQL_RWSEPARATE_OFF`、`DSQL_RWSEPARATE_ON` 或 `DSQL_RWSEPARATE_ON2`，`rwseparate_percent` 范围为 0–100。这些选项在建连时传给底层驱动。读写分离模式 1 和 4 已在本机 DM8 主备环境中验证自动提交模式下的查询路由；MPP 只验证了单机 LOCAL 握手，集群路由尚待验证。 - `ssl_path` 指向含 `ca-cert.pem`、`client-cert.pem`、`client-key.pem` 的目录。服务端证书没有 SAN 时还需提供准确的 `server-cert.pem`，用于证书固定校验；有 SAN 的证书按 CA 链和主机名校验。设置后若服务端未协商加密，连接失败。非空的 `ssl_pwd`、`ukey_name`、`ukey_pin` 暂不支持。 - `user` 支持 `user/password@server:port[/schema][?catalog=...]` 形式。 - `login_timeout` 以毫秒为单位，默认 5000，限制首次建连握手；设为 0 表示不限制。`connection_timeout` 以秒为单位，默认 0 不限制，限制 SQL 执行时间。 diff --git a/docs/test-results/2026-09-27-ha-routing.md b/docs/test-results/2026-09-27-ha-routing.md new file mode 100644 index 0000000..e014f63 --- /dev/null +++ b/docs/test-results/2026-09-27-ha-routing.md @@ -0,0 +1,46 @@ +# Real DM8 primary/standby routing (2026-09-27) + +An isolated two-node DM8 watcher group and monitor ran in local Orb, using the +2025-09-24 ARM image. `V$INSTANCE` reported `PRIMARY, OPEN` and `STANDBY, OPEN` +with the same OGUID. Both endpoints accepted direct Python connections. + +The image's `DOCKER_DMWATCHER` startup script omitted the `TIME_ZONE` template +substitution. The temporary containers replaced `ENV_21` with `+08:00` before +running the image startup script. This workaround was confined to the local +test containers. + +In the cluster network, a Go driver probe found that `rwSeparate=1` sent a +read-only query to the standby, while `rwSeparate=4` failed by dialing `:0`. +The server did not provide a standby address in the mode 4 login response. The +driver now falls back to its existing valid-standby metadata query when that +address is absent. + +After the fix, `scripts/verify_dm_ha.py` built and ran with the ARM Linux +Python extension. With autocommit enabled and 0% primary reads: + +| Mode | Query destination | +| --- | --- | +| `rwseparate=0` | Primary (`GRP453932_DW1`) | +| `rwseparate=1` | Standby (`GRP453932_DW2`) | +| `rwseparate=4` | Standby (`GRP453932_DW2`) | + +The script then inserted `12345678901234567890.12345678` into a +`DECIMAL(30,8)` column through mode 4. A direct standby connection read the +same value after replication, and the script removed its test table. + +To repeat the check, provide `DM_HA_PRIMARY_HOST`, `DM_HA_STANDBY_HOST`, +`DM_HA_USER`, and `DM_HA_PASSWORD` to `scripts/verify_dm_ha.py`. Both hosts +must be reachable from the process running the driver. Optional +`DM_HA_PRIMARY_PORT` and `DM_HA_STANDBY_PORT` default to 5236. The account +needs permission to create and drop a table on the primary. + +This HA check currently runs locally: the GitHub hosted runners cannot reach +the user's Orb network. The regular GitHub ARM real-database matrix still +checks single-node behavior and build compatibility. MPP routing and UKey +authentication remain unverified. + +An additional stop-primary trial did not prove automatic takeover. The +monitor's confirmation mode remained active; after roughly 35 seconds, the +second node still reported `STANDBY, OPEN`. The replicated high-precision +value remained readable there. Automatic promotion and reconnect after +promotion need a separately configured failover regression. diff --git a/dpi_bridge/third_party/chunanyong_dm/PATCHES.md b/dpi_bridge/third_party/chunanyong_dm/PATCHES.md index 7d15d21..422bd51 100644 --- a/dpi_bridge/third_party/chunanyong_dm/PATCHES.md +++ b/dpi_bridge/third_party/chunanyong_dm/PATCHES.md @@ -146,6 +146,19 @@ in two schemas and reads both a foreign object and an array of those objects from tables owned by the login user. +## Patch: discover standby for apply-wait read/write separation + +- File: `zx.go` +- Some DM8 servers do not include the standby host and port in the login + response for `rwSeparate=4`. Query the primary's valid standby metadata in + that case, as the other read/write separation modes do, instead of dialing + `:0`. The apply-wait mode remains enabled on the connection. +- A failed metadata query may return no statement or rows. Close only handles + that exist before retrying the alternate metadata view. +- Regression: `scripts/verify_dm_ha.py` checks modes 0, 1, and 4 against a + real primary/standby pair, then writes `DECIMAL(30,8)` through mode 4 and + verifies the replicated value on the standby. + ## Patch: resolve service names with the bridge's default port - Files: `n.go`, `zzm.go` diff --git a/dpi_bridge/third_party/chunanyong_dm/zx.go b/dpi_bridge/third_party/chunanyong_dm/zx.go index 03823a1..555203b 100644 --- a/dpi_bridge/third_party/chunanyong_dm/zx.go +++ b/dpi_bridge/third_party/chunanyong_dm/zx.go @@ -166,7 +166,8 @@ func (RWUtil rwUtil) chooseValidStandby(connection *DmConnection) (*ep, error) { var err error if connection.dmConnector.rwSeparate == RW_SEPARATE_USER_DEFINED { return RWUtil.chooseStandbyUserDefined(connection), nil - } else if connection.dmConnector.rwSeparate == RW_SEPARATE_DB_APPLY_WAIT { + } else if connection.dmConnector.rwSeparate == RW_SEPARATE_DB_APPLY_WAIT && + connection.StandbyHost != "" && connection.StandbyPort > 0 { return newEP(connection.StandbyHost, connection.StandbyPort), nil } else if connection.dmConnector.rwSeparate == RW_SEPARATE_EP_GROUP { epStr := "" @@ -200,8 +201,13 @@ func (RWUtil rwUtil) chooseValidStandby(connection *DmConnection) (*ep, error) { }() if err != nil { - rs.close() - stmt.close() + if rs != nil { + rs.close() + } + if stmt != nil { + stmt.close() + } + rs, stmt = nil, nil if connection.Malini2 { stmt, rs, err = connection.driverQuery(SQL_SELECT_STANDBY2 + filter) diff --git a/scripts/verify_dm_ha.py b/scripts/verify_dm_ha.py new file mode 100644 index 0000000..092774b --- /dev/null +++ b/scripts/verify_dm_ha.py @@ -0,0 +1,97 @@ +"""Verify read/write routing against a real DM primary and standby. + +Set DM_HA_PRIMARY_HOST, DM_HA_STANDBY_HOST, DM_HA_USER, and DM_HA_PASSWORD. +The account must be able to query V$INSTANCE and create/drop a test table. +""" + +from __future__ import annotations + +import os +import time +import uuid +from decimal import Decimal + +import dmPython + + +def connect(host: str, port: int, **options): + return dmPython.connect( + user=os.environ["DM_HA_USER"], + password=os.environ["DM_HA_PASSWORD"], + server=host, + port=port, + autoCommit=dmPython.DSQL_AUTOCOMMIT_ON, + login_timeout=2000, + **options, + ) + + +def instance(conn): + with conn.cursor() as cur: + cur.execute("SELECT INSTANCE_NAME, MODE$, STATUS$ FROM V$INSTANCE") + return cur.fetchone() + + +def main(): + primary_host = os.environ["DM_HA_PRIMARY_HOST"] + standby_host = os.environ["DM_HA_STANDBY_HOST"] + primary_port = int(os.environ.get("DM_HA_PRIMARY_PORT", "5236")) + standby_port = int(os.environ.get("DM_HA_STANDBY_PORT", "5236")) + amount = Decimal("12345678901234567890.12345678") + table = "DMPY_HA_" + uuid.uuid4().hex[:8].upper() + + with connect(primary_host, primary_port) as primary, connect( + standby_host, standby_port + ) as standby: + primary_name, primary_mode, primary_status = instance(primary) + standby_name, standby_mode, standby_status = instance(standby) + assert (primary_mode, primary_status) == ("PRIMARY", "OPEN") + assert (standby_mode, standby_status) == ("STANDBY", "OPEN") + + for mode, expected in ( + (dmPython.DSQL_RWSEPARATE_OFF, primary_name), + (dmPython.DSQL_RWSEPARATE_ON, standby_name), + (dmPython.DSQL_RWSEPARATE_ON2, standby_name), + ): + with connect( + primary_host, primary_port, rwseparate=mode, rwseparate_percent=0 + ) as routed: + actual = instance(routed)[0] + assert actual == expected, (mode, actual, expected) + print(f"rwseparate={mode}: {actual}") + + with primary.cursor() as cur: + cur.execute(f"CREATE TABLE {table} (ID INT, AMOUNT DECIMAL(30,8))") + try: + with connect( + primary_host, + primary_port, + rwseparate=dmPython.DSQL_RWSEPARATE_ON2, + rwseparate_percent=0, + ) as routed: + with routed.cursor() as cur: + cur.execute(f"INSERT INTO {table} VALUES (?, ?)", (1, amount)) + + deadline = time.monotonic() + 10 + while True: + try: + with standby.cursor() as cur: + cur.execute(f"SELECT CAST(AMOUNT AS VARCHAR(80)) FROM {table} WHERE ID=1") + row = cur.fetchone() + if row is not None: + assert Decimal(row[0]) == amount + print("standby replicated DECIMAL(30,8) exactly") + break + except dmPython.Error: + # The table may not be visible until the DDL reaches the standby. + pass + if time.monotonic() >= deadline: + raise AssertionError("write did not reach standby within 10 seconds") + time.sleep(0.2) + finally: + with primary.cursor() as cur: + cur.execute(f"DROP TABLE {table}") + + +if __name__ == "__main__": + main()