From a6fd55f80e2e9337113c3c88059c8df77f30e409 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 20 Nov 2026 23:34:22 +0000 Subject: [PATCH 1/2] chore(release): bump HyperDX app/package versions --- .changeset/adhoc-filters-ga.md | 5 - .changeset/alert-history-lookback.md | 10 -- .changeset/bump-clickhouse-go-brotli.md | 8 -- .changeset/chart-table-edited-order-by.md | 5 - .../classify-clickhouse-proxy-failures.md | 9 -- .changeset/dedupe-map-keys-reads.md | 10 -- .changeset/faster-exact-filter-values.md | 9 -- .changeset/filter-keys-fetch-limit-max.md | 10 -- .changeset/filter-type-switch-validation.md | 7 - .changeset/json-array-paths.md | 14 -- .changeset/json-filter-keys-reescape.md | 11 -- .changeset/map-key-index-median-parts.md | 15 --- .changeset/mcp-hastoken-separator-hint.md | 12 -- .changeset/mcp-search-partial-results.md | 14 -- .changeset/metric-group-by-time-column.md | 14 -- .changeset/minmax-datetime-keep-type.md | 13 -- .changeset/patterns-mask-dated-lines.md | 19 --- .changeset/red-metrics-trace-search.md | 5 - .../rename-queried-values-filter-type.md | 7 - ...uire-password-on-connection-host-change.md | 8 -- .changeset/save-temporary-dashboard.md | 5 - .changeset/sidebar-expand-icons.md | 5 - .../sql-query-dashboard-filter-source.md | 9 -- .changeset/sql-query-filter-autocomplete.md | 8 -- .changeset/team-default-filter-values-mode.md | 7 - .../text-index-dictionary-block-size.md | 21 --- .changeset/theme-field-background-tokens.md | 7 - .changeset/topk-text-index-values.md | 27 ---- .changeset/trace-search-duration-heatmap.md | 5 - .changeset/v2-alert-silence.md | 5 - .env | 4 +- packages/api/CHANGELOG.md | 100 ++++++++++++++ packages/api/package.json | 4 +- packages/app/CHANGELOG.md | 122 ++++++++++++++++++ packages/app/package.json | 4 +- packages/cli/CHANGELOG.md | 18 +++ packages/cli/package.json | 4 +- packages/common-utils/CHANGELOG.md | 99 ++++++++++++++ packages/common-utils/package.json | 2 +- packages/otel-collector/CHANGELOG.md | 27 ++++ packages/otel-collector/package.json | 2 +- yarn.lock | 8 +- 42 files changed, 380 insertions(+), 318 deletions(-) delete mode 100644 .changeset/adhoc-filters-ga.md delete mode 100644 .changeset/alert-history-lookback.md delete mode 100644 .changeset/bump-clickhouse-go-brotli.md delete mode 100644 .changeset/chart-table-edited-order-by.md delete mode 100644 .changeset/classify-clickhouse-proxy-failures.md delete mode 100644 .changeset/dedupe-map-keys-reads.md delete mode 100644 .changeset/faster-exact-filter-values.md delete mode 100644 .changeset/filter-keys-fetch-limit-max.md delete mode 100644 .changeset/filter-type-switch-validation.md delete mode 100644 .changeset/json-array-paths.md delete mode 100644 .changeset/json-filter-keys-reescape.md delete mode 100644 .changeset/map-key-index-median-parts.md delete mode 100644 .changeset/mcp-hastoken-separator-hint.md delete mode 100644 .changeset/mcp-search-partial-results.md delete mode 100644 .changeset/metric-group-by-time-column.md delete mode 100644 .changeset/minmax-datetime-keep-type.md delete mode 100644 .changeset/patterns-mask-dated-lines.md delete mode 100644 .changeset/red-metrics-trace-search.md delete mode 100644 .changeset/rename-queried-values-filter-type.md delete mode 100644 .changeset/require-password-on-connection-host-change.md delete mode 100644 .changeset/save-temporary-dashboard.md delete mode 100644 .changeset/sidebar-expand-icons.md delete mode 100644 .changeset/sql-query-dashboard-filter-source.md delete mode 100644 .changeset/sql-query-filter-autocomplete.md delete mode 100644 .changeset/team-default-filter-values-mode.md delete mode 100644 .changeset/text-index-dictionary-block-size.md delete mode 100644 .changeset/theme-field-background-tokens.md delete mode 100644 .changeset/topk-text-index-values.md delete mode 100644 .changeset/trace-search-duration-heatmap.md delete mode 100644 .changeset/v2-alert-silence.md diff --git a/.changeset/adhoc-filters-ga.md b/.changeset/adhoc-filters-ga.md deleted file mode 100644 index a8bdcaeb6c..0000000000 --- a/.changeset/adhoc-filters-ga.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -feat: Removing the NEXT_PUBLIC_ENABLE_ADHOC_FILTERS flag diff --git a/.changeset/alert-history-lookback.md b/.changeset/alert-history-lookback.md deleted file mode 100644 index 7127a6158a..0000000000 --- a/.changeset/alert-history-lookback.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -'@hyperdx/api': patch ---- - -perf: bound the check-alerts previous-history lookup by the alert interval - -The check-alerts task no longer scans 7 days of alert history per alert on -every tick. A one-row lookup per alert skips alerts already evaluated in the -current window. Non-grouped alerts take their previous state from that row, -and grouped alerts scan a window sized to their interval (1 to 7 days). diff --git a/.changeset/bump-clickhouse-go-brotli.md b/.changeset/bump-clickhouse-go-brotli.md deleted file mode 100644 index 963695f99c..0000000000 --- a/.changeset/bump-clickhouse-go-brotli.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -'@hyperdx/otel-collector': patch ---- - -fix(security): bump `clickhouse-go/v2` to v2.51.0 in the migration tool - -Pulls in `github.com/andybalholm/brotli` v1.2.7, which fixes GO-2026-6888 -(CVE-2026-111497), a denial of service in the brotli decompressor. diff --git a/.changeset/chart-table-edited-order-by.md b/.changeset/chart-table-edited-order-by.md deleted file mode 100644 index e1072d8094..0000000000 --- a/.changeset/chart-table-edited-order-by.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -fix: run an edited Order By on a chart table instead of the last column header sort diff --git a/.changeset/classify-clickhouse-proxy-failures.md b/.changeset/classify-clickhouse-proxy-failures.md deleted file mode 100644 index 0bb5e8e840..0000000000 --- a/.changeset/classify-clickhouse-proxy-failures.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -'@hyperdx/api': patch ---- - -feat: Classify failed ClickHouse proxy requests on the server span and the -`clickhouse_proxy.query` SLI (`error_category`), including errors ClickHouse -writes after a 200 response. The span also records the upstream status, the -ClickHouse error code and type, and the body of upstream errors that carry no -ClickHouse code. diff --git a/.changeset/dedupe-map-keys-reads.md b/.changeset/dedupe-map-keys-reads.md deleted file mode 100644 index 8459fcefc5..0000000000 --- a/.changeset/dedupe-map-keys-reads.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -'@hyperdx/common-utils': patch ---- - -perf: share one Map key read between concurrent callers - -The search page's filter panel, search bar and group-by each load the same Map -keys when the page opens. Each sent its own text index read, which on large -tables can use several GiB of memory. Concurrent requests for the same keys now -share one query. diff --git a/.changeset/faster-exact-filter-values.md b/.changeset/faster-exact-filter-values.md deleted file mode 100644 index 017dae5324..0000000000 --- a/.changeset/faster-exact-filter-values.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -'@hyperdx/common-utils': patch -'@hyperdx/app': patch ---- - -perf: load search filter values faster in exact mode by reading several keys on -one Map column with a single `groupUniqArrayMap`, and by querying plain columns -and each Map or JSON column separately so slow attribute maps don't hold up the -rest diff --git a/.changeset/filter-keys-fetch-limit-max.md b/.changeset/filter-keys-fetch-limit-max.md deleted file mode 100644 index b05b2bd862..0000000000 --- a/.changeset/filter-keys-fetch-limit-max.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -'@hyperdx/app': patch -'@hyperdx/common-utils': patch ---- - -feat: allow up to 10,000 in the team "Filter Keys Fetch Limit" setting - -The setting's input in Team settings stopped at 1,000 keys per column; it now -accepts up to 10,000. Map key discovery returns up to the setting's value per -Map column, and never fewer than 1,000. diff --git a/.changeset/filter-type-switch-validation.md b/.changeset/filter-type-switch-validation.md deleted file mode 100644 index 9c8b8de032..0000000000 --- a/.changeset/filter-type-switch-validation.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -fix: stop the previous filter type's validation from blocking a save after switching types - -Switching a dashboard filter from "PromQL label values" to another type no longer fails with "Select a PromQL source". diff --git a/.changeset/json-array-paths.md b/.changeset/json-array-paths.md deleted file mode 100644 index 6e34a0785a..0000000000 --- a/.changeset/json-array-paths.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -fix: address JSON array elements by index in row-panel filters - -"Add to Filters" on a value inside an array in a native JSON column now uses -1-based array subscripts, for example `events[1]`, instead of quoted keys such -as `events.0`, so the filter matches rows instead of silently returning none. - -The same applies to arrays inside a JSON string (for example a String `Body` -column), which now pass 1-based integer indexes to `JSONExtract*`. JSON strings -stored in a native JSON column are wrapped in `toString()` so `JSONExtract*` -accepts them. diff --git a/.changeset/json-filter-keys-reescape.md b/.changeset/json-filter-keys-reescape.md deleted file mode 100644 index 3a71e54acc..0000000000 --- a/.changeset/json-filter-keys-reescape.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -fix: keep JSON-column filter keys in JSON form when another filter is added - -Adding a second filter, or clicking "Load more" on a filter, no longer rewrites -an earlier filter on a key inside a native JSON column to `col['key']`, which -ClickHouse rejects (`Illegal type Dynamic of argument of function in`, or -`First argument for function 'arrayElement' must be array, got 'JSON'` on older -versions). diff --git a/.changeset/map-key-index-median-parts.md b/.changeset/map-key-index-median-parts.md deleted file mode 100644 index 62e41e1ad7..0000000000 --- a/.changeset/map-key-index-median-parts.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -'@hyperdx/common-utils': patch -'@hyperdx/api': patch -'@hyperdx/app': patch ---- - -perf: read only 20 median-sized parts for text index key and value lookups - -Map key discovery and "all values" lookups on text indexes read the index -dictionary of every part in the time range, which is slow on tables with many -parts. They now read the 20 parts closest to the median size. Log and trace -sources can change the count with the new "Map key index parts" setting -(`mapKeyIndexMaxParts`), and which parts are read with "Map key index part size" -(`mapKeyIndexPartSelection`: median, smallest, or largest). "Load more" on a -single filter key still reads every part in the time range. diff --git a/.changeset/mcp-hastoken-separator-hint.md b/.changeset/mcp-hastoken-separator-hint.md deleted file mode 100644 index 647db60442..0000000000 --- a/.changeset/mcp-hastoken-separator-hint.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -'@hyperdx/api': patch ---- - -fix(mcp): explain hasToken() separator errors in query tools - -When a SQL filter calls `hasToken()` with a needle that contains `_`, `-`, `.`, -`@`, `:` or spaces, ClickHouse fails with "Needle must not contain whitespace or -separator characters". MCP query tools now append a hint pointing to a Lucene -term, or in SQL to `position()` for the exact term (with `hasAllTokens()` as an -optional text-index speed-up), and the `whereLanguage` description warns about -it. diff --git a/.changeset/mcp-search-partial-results.md b/.changeset/mcp-search-partial-results.md deleted file mode 100644 index d312664106..0000000000 --- a/.changeset/mcp-search-partial-results.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -'@hyperdx/api': patch -'@hyperdx/common-utils': patch -'@hyperdx/app': patch ---- - -fix: return partial rows from `clickstack_search` instead of timing out - -The MCP search tool now scans newest-first over the same progressive time -windows as the app's search page, under a 25s budget. When time runs out it -returns the rows found so far with a `partial` block (requested range, the range -fully covered, and the window that was cut off) and a hint for the next call, -instead of failing after 30s. The time-window helpers moved from the app to -`@hyperdx/common-utils/core/searchWindows`; the app's behavior is unchanged. diff --git a/.changeset/metric-group-by-time-column.md b/.changeset/metric-group-by-time-column.md deleted file mode 100644 index 66d282ae22..0000000000 --- a/.changeset/metric-group-by-time-column.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -'@hyperdx/common-utils': patch -'@hyperdx/api': patch -'@hyperdx/app': patch ---- - -fix: allow grouping gauge and sum metric charts by an expression over the time column - -A table, pie or bar chart on a gauge or sum metric grouped by a time expression -such as `toStartOfHour(TimeUnix)` or `toDate(TimeUnix)` failed with -`Unknown identifier TimeUnix`, because the bucketed CTE the outer query reads -did not carry the time column. When the group-by references it, `TimeUnix` now -resolves to each metric bucket's start, so values are the same as the chart -without the time group-by. diff --git a/.changeset/minmax-datetime-keep-type.md b/.changeset/minmax-datetime-keep-type.md deleted file mode 100644 index 9537ba2a06..0000000000 --- a/.changeset/minmax-datetime-keep-type.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -'@hyperdx/common-utils': patch -'@hyperdx/api': patch -'@hyperdx/app': patch ---- - -fix: return the actual earliest/latest time for min/max on a date or time column - -`min` and `max` on a Date, DateTime, or DateTime64 column (for example -`Timestamp`) returned 0, because every aggregation cast its value to Float64 -first. Tables, number tiles, and the MCP `clickstack_table` tool now return the -timestamp itself. Line, bar, pie, and other charts, ratios, formulas, and number -tiles with several series keep the numeric cast. diff --git a/.changeset/patterns-mask-dated-lines.md b/.changeset/patterns-mask-dated-lines.md deleted file mode 100644 index 39e2886669..0000000000 --- a/.changeset/patterns-mask-dated-lines.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -'@hyperdx/common-utils': patch -'@hyperdx/api': patch -'@hyperdx/cli': patch ---- - -fix: MCP `event_patterns`, `emerging_signals` and search denoise, and the CLI, -no longer split the same log line by its date. Before mining, the Drain miner -masks the date of a timestamp (weekday, day, month, year) to ``, so dated -lines from different days, months or years get the same template instead of one -"new" and one "gone". Times and ids are not masked. Within one day lines group -as before, except that klog lines with different severity letters are no longer -merged. No mask changes a line's token count: the `event_patterns` drill-down -predicate matches `` as one or more non-whitespace characters, also inside -a token (the new `masks` option of the pattern predicate builders, which only -the MCP drill-down sets), and search denoise masks bodies the same way before -matching. Masking instructions get an optional `perWord` flag (default off, as -in drain3). The web app's Event patterns view uses its own drain3 config and is -unchanged. diff --git a/.changeset/red-metrics-trace-search.md b/.changeset/red-metrics-trace-search.md deleted file mode 100644 index 12632044e9..0000000000 --- a/.changeset/red-metrics-trace-search.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@hyperdx/app': minor ---- - -Show RED metrics (Throughput, Errors, Duration) above the trace search results instead of the single count histogram. The three charts share a synced hover cursor, Errors toggles between rate and volume, and clicking a status series on the volume chart filters the results. Logs and other sources keep the existing histogram. diff --git a/.changeset/rename-queried-values-filter-type.md b/.changeset/rename-queried-values-filter-type.md deleted file mode 100644 index 620be022ae..0000000000 --- a/.changeset/rename-queried-values-filter-type.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -feat: rename the "Queried values" dashboard filter type to "SQL expression values" - -The new name separates it from "SQL query values" and "PromQL label values". diff --git a/.changeset/require-password-on-connection-host-change.md b/.changeset/require-password-on-connection-host-change.md deleted file mode 100644 index 67c11e971b..0000000000 --- a/.changeset/require-password-on-connection-host-change.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -'@hyperdx/api': patch -'@hyperdx/app': patch ---- - -fix: require the password when changing a connection's host - -Updating a connection's host without sending its password used to keep the stored password, so it would be sent to the new host on the next query. Both `PUT /connections/:id` and `PUT /api/v2/connections/:id` now return 400 in that case. Prometheus endpoints never use the stored password, so it is cleared instead. The connection form asks for the password when the host is edited and shows the API's error message. diff --git a/.changeset/save-temporary-dashboard.md b/.changeset/save-temporary-dashboard.md deleted file mode 100644 index ba2db86643..0000000000 --- a/.changeset/save-temporary-dashboard.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -feat: save a temporary dashboard with its content intact diff --git a/.changeset/sidebar-expand-icons.md b/.changeset/sidebar-expand-icons.md deleted file mode 100644 index 3b5fce2495..0000000000 --- a/.changeset/sidebar-expand-icons.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -Use the sidebar layout icons for showing and hiding search filters diff --git a/.changeset/sql-query-dashboard-filter-source.md b/.changeset/sql-query-dashboard-filter-source.md deleted file mode 100644 index a981db1fad..0000000000 --- a/.changeset/sql-query-dashboard-filter-source.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -'@hyperdx/common-utils': patch -'@hyperdx/app': patch -'@hyperdx/api': patch ---- - -feat: let SQL query dashboard filters use an optional source - -A SQL query filter can now pick a source on its connection, or a data source variable, and can do so on a connection variable too. With one, its SQL can use `$__sourceTable` and `$__filters`, which applies the dashboard's filter selections for that source, the same way raw SQL tiles do. The external API and MCP read and write it as `sourceId`. diff --git a/.changeset/sql-query-filter-autocomplete.md b/.changeset/sql-query-filter-autocomplete.md deleted file mode 100644 index 2dab816720..0000000000 --- a/.changeset/sql-query-filter-autocomplete.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -feat: autocomplete table and column names in the SQL query filter editor - -The SQL query dashboard filter's editor now suggests the tables and columns of -the selected connection's sources, like the Raw SQL tile editor. diff --git a/.changeset/team-default-filter-values-mode.md b/.changeset/team-default-filter-values-mode.md deleted file mode 100644 index 803b2f91ff..0000000000 --- a/.changeset/team-default-filter-values-mode.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -'@hyperdx/common-utils': patch -'@hyperdx/api': patch -'@hyperdx/app': patch ---- - -feat: Add a team-wide default (All or Exact) for the search filters "Show All Values" setting diff --git a/.changeset/text-index-dictionary-block-size.md b/.changeset/text-index-dictionary-block-size.md deleted file mode 100644 index e6331b58b2..0000000000 --- a/.changeset/text-index-dictionary-block-size.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -'@hyperdx/otel-collector': patch ---- - -perf: use 8,192-token text index dictionary blocks on the seeded tables - -On ClickHouse 26.6 and later, the collector's schema migration now sets -`text_index_dictionary_block_size = 8192` (the default is 512) on `otel_logs`, -`otel_traces` and `hyperdx_sessions`, with `ALTER TABLE … MODIFY SETTING` after -the seed. Each part's text index sparse index is 16x smaller. Reading filter -values with `mergeTreeTextIndex()` loads that index for every part on every -read, and on a 537M-row table 4,096-token blocks already made those reads about -10x cheaper per part, made searches for a rare attribute value 3-7x faster, and -made searches for a common value 10-30% faster. - -Because it's an `ALTER`, existing installs get it too: new parts use 8,192-token -blocks, and existing parts keep theirs until they're merged. A table that -already sets `text_index_dictionary_block_size` is left alone, and operators who -manage these tables themselves can turn the change off with -`HYPERDX_OTEL_EXPORTER_SET_TEXT_INDEX_BLOCK_SIZE=false`. ClickHouse 26.2–26.5 -doesn't have the setting, so those servers keep 512-token blocks. diff --git a/.changeset/theme-field-background-tokens.md b/.changeset/theme-field-background-tokens.md deleted file mode 100644 index d49446ac38..0000000000 --- a/.changeset/theme-field-background-tokens.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -'@hyperdx/app': patch ---- - -In dark mode, the field background now has opacity so it works on different -backgrounds, such as popovers. It is set directly in the theme tokens instead of -through a Click UI token that resolved to a solid color. diff --git a/.changeset/topk-text-index-values.md b/.changeset/topk-text-index-values.md deleted file mode 100644 index 3565b3d1c8..0000000000 --- a/.changeset/topk-text-index-values.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -'@hyperdx/common-utils': patch ---- - -perf: read large-block Map text index values per key, most frequent first - -"All values" lookups on Map key=value text indexes with large dictionary blocks -(2048 or more, e.g. `text_index_dictionary_block_size = 8192`, which the -collector's migration now sets on ClickHouse 26.6+) -now read each key separately, in parallel queries of at most 10 keys, and stop -after 20,000 tokens per key, so a key with a value per row (request IDs, UUIDs) -can't hold up the others. Values come from `topKWeighted(limit)` weighted by -each token's row count, so a capped list holds the most common values, most -common first. These reads no longer apply `max_rows_to_read`: the index is read -in sorted token order, so the cap left every key that sorts after a -high-cardinality key with no values. - -Indexes with smaller blocks, including ClickHouse's default of 512, keep the -previous query shape (one read for all of a query's keys, the alphabetically -first values, and the `max_rows_to_read` cap), now in parallel queries of at -most 10 keys, so the cap's cut-off affects at most 9 other keys. Every read loads -each part's sparse index, which is large with small blocks, so one read per key -would multiply that cost. The block size comes from the index definition, then -the table's `text_index_dictionary_block_size`, then the server's. - -Text indexes on plain columns now return the most frequent values with -`topKWeighted`, in parallel queries of at most 10 columns, without the row cap. diff --git a/.changeset/trace-search-duration-heatmap.md b/.changeset/trace-search-duration-heatmap.md deleted file mode 100644 index 0d3278cc02..0000000000 --- a/.changeset/trace-search-duration-heatmap.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@hyperdx/app': minor ---- - -Add a RED/Heatmap switch to the trace search results view. Heatmap shows the duration distribution over time as the same tile the dashboard renders, and the choice is kept in the URL so a reload or a shared link reopens the same view. diff --git a/.changeset/v2-alert-silence.md b/.changeset/v2-alert-silence.md deleted file mode 100644 index e0504abd6b..0000000000 --- a/.changeset/v2-alert-silence.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@hyperdx/api': minor ---- - -feat: Silence and unsilence alerts through the external API with POST and DELETE /api/v2/alerts/{id}/silenced diff --git a/.env b/.env index a127d0c604..c03b05aa62 100644 --- a/.env +++ b/.env @@ -9,8 +9,8 @@ NEXT_ALL_IN_ONE_IMAGE_NAME_DOCKERHUB=clickhouse/clickstack-all-in-one ALL_IN_ONE_IMAGE_NAME_DOCKERHUB=hyperdx/hyperdx-all-in-one NEXT_OTEL_COLLECTOR_IMAGE_NAME_DOCKERHUB=clickhouse/clickstack-otel-collector OTEL_COLLECTOR_IMAGE_NAME_DOCKERHUB=hyperdx/hyperdx-otel-collector -CODE_VERSION=2.46.0 -IMAGE_VERSION_SUB_TAG=.46.0 +CODE_VERSION=2.47.0 +IMAGE_VERSION_SUB_TAG=.47.0 IMAGE_VERSION=2 IMAGE_NIGHTLY_TAG=2-nightly IMAGE_LATEST_TAG=latest diff --git a/packages/api/CHANGELOG.md b/packages/api/CHANGELOG.md index 4ed95e894b..9323bb2190 100644 --- a/packages/api/CHANGELOG.md +++ b/packages/api/CHANGELOG.md @@ -1,5 +1,105 @@ # @hyperdx/api +## 2.47.0 + +### Minor Changes + +- 7905721bc: feat: Silence and unsilence alerts through the external API with POST and DELETE /api/v2/alerts/{id}/silenced + +### Patch Changes + +- 22377dfcf: perf: bound the check-alerts previous-history lookup by the alert interval + + The check-alerts task no longer scans 7 days of alert history per alert on + every tick. A one-row lookup per alert skips alerts already evaluated in the + current window. Non-grouped alerts take their previous state from that row, + and grouped alerts scan a window sized to their interval (1 to 7 days). + +- b129ec1ac: feat: Classify failed ClickHouse proxy requests on the server span and the + `clickhouse_proxy.query` SLI (`error_category`), including errors ClickHouse + writes after a 200 response. The span also records the upstream status, the + ClickHouse error code and type, and the body of upstream errors that carry no + ClickHouse code. +- 86acf27c0: perf: read only 20 median-sized parts for text index key and value lookups + + Map key discovery and "all values" lookups on text indexes read the index + dictionary of every part in the time range, which is slow on tables with many + parts. They now read the 20 parts closest to the median size. Log and trace + sources can change the count with the new "Map key index parts" setting + (`mapKeyIndexMaxParts`), and which parts are read with "Map key index part size" + (`mapKeyIndexPartSelection`: median, smallest, or largest). "Load more" on a + single filter key still reads every part in the time range. + +- e80fc1ba2: fix(mcp): explain hasToken() separator errors in query tools + + When a SQL filter calls `hasToken()` with a needle that contains `_`, `-`, `.`, + `@`, `:` or spaces, ClickHouse fails with "Needle must not contain whitespace or + separator characters". MCP query tools now append a hint pointing to a Lucene + term, or in SQL to `position()` for the exact term (with `hasAllTokens()` as an + optional text-index speed-up), and the `whereLanguage` description warns about + it. + +- 2f61137e2: fix: return partial rows from `clickstack_search` instead of timing out + + The MCP search tool now scans newest-first over the same progressive time + windows as the app's search page, under a 25s budget. When time runs out it + returns the rows found so far with a `partial` block (requested range, the range + fully covered, and the window that was cut off) and a hint for the next call, + instead of failing after 30s. The time-window helpers moved from the app to + `@hyperdx/common-utils/core/searchWindows`; the app's behavior is unchanged. + +- 64a55d5b7: fix: allow grouping gauge and sum metric charts by an expression over the time column + + A table, pie or bar chart on a gauge or sum metric grouped by a time expression + such as `toStartOfHour(TimeUnix)` or `toDate(TimeUnix)` failed with + `Unknown identifier TimeUnix`, because the bucketed CTE the outer query reads + did not carry the time column. When the group-by references it, `TimeUnix` now + resolves to each metric bucket's start, so values are the same as the chart + without the time group-by. + +- e8d1d9add: fix: return the actual earliest/latest time for min/max on a date or time column + + `min` and `max` on a Date, DateTime, or DateTime64 column (for example + `Timestamp`) returned 0, because every aggregation cast its value to Float64 + first. Tables, number tiles, and the MCP `clickstack_table` tool now return the + timestamp itself. Line, bar, pie, and other charts, ratios, formulas, and number + tiles with several series keep the numeric cast. + +- d99be7650: fix: MCP `event_patterns`, `emerging_signals` and search denoise, and the CLI, + no longer split the same log line by its date. Before mining, the Drain miner + masks the date of a timestamp (weekday, day, month, year) to ``, so dated + lines from different days, months or years get the same template instead of one + "new" and one "gone". Times and ids are not masked. Within one day lines group + as before, except that klog lines with different severity letters are no longer + merged. No mask changes a line's token count: the `event_patterns` drill-down + predicate matches `` as one or more non-whitespace characters, also inside + a token (the new `masks` option of the pattern predicate builders, which only + the MCP drill-down sets), and search denoise masks bodies the same way before + matching. Masking instructions get an optional `perWord` flag (default off, as + in drain3). The web app's Event patterns view uses its own drain3 config and is + unchanged. +- e58329c87: fix: require the password when changing a connection's host + + Updating a connection's host without sending its password used to keep the stored password, so it would be sent to the new host on the next query. Both `PUT /connections/:id` and `PUT /api/v2/connections/:id` now return 400 in that case. Prometheus endpoints never use the stored password, so it is cleared instead. The connection form asks for the password when the host is edited and shows the API's error message. + +- 2cec94f7d: feat: let SQL query dashboard filters use an optional source + + A SQL query filter can now pick a source on its connection, or a data source variable, and can do so on a connection variable too. With one, its SQL can use `$__sourceTable` and `$__filters`, which applies the dashboard's filter selections for that source, the same way raw SQL tiles do. The external API and MCP read and write it as `sourceId`. + +- 7afa31c71: feat: Add a team-wide default (All or Exact) for the search filters "Show All Values" setting +- Updated dependencies [e36cec3a2] +- Updated dependencies [1a4c116b2] +- Updated dependencies [351f19400] +- Updated dependencies [86acf27c0] +- Updated dependencies [2f61137e2] +- Updated dependencies [64a55d5b7] +- Updated dependencies [e8d1d9add] +- Updated dependencies [d99be7650] +- Updated dependencies [2cec94f7d] +- Updated dependencies [7afa31c71] +- Updated dependencies [f11b9f58b] + - @hyperdx/common-utils@0.36.1 + ## 2.46.0 ### Minor Changes diff --git a/packages/api/package.json b/packages/api/package.json index a46ff850dc..e655a31edf 100644 --- a/packages/api/package.json +++ b/packages/api/package.json @@ -1,6 +1,6 @@ { "name": "@hyperdx/api", - "version": "2.46.0", + "version": "2.47.0", "license": "MIT", "private": true, "engines": { @@ -23,7 +23,7 @@ "@clickhouse/clickstack-node": "^1.0.2", "@clickhouse/client-common": "1.23.0-head.fae5998.1", "@esm2cjs/p-queue": "^7.3.0", - "@hyperdx/common-utils": "^0.36.0", + "@hyperdx/common-utils": "^0.36.1", "@hyperdx/passport-local-mongoose": "^9.0.1", "@modelcontextprotocol/sdk": "^1.32.0", "@opentelemetry/api": "~1.9.1", diff --git a/packages/app/CHANGELOG.md b/packages/app/CHANGELOG.md index 0204c3250a..b3df4a8b07 100644 --- a/packages/app/CHANGELOG.md +++ b/packages/app/CHANGELOG.md @@ -1,5 +1,127 @@ # @hyperdx/app +## 2.47.0 + +### Minor Changes + +- c6657aa0c: Show RED metrics (Throughput, Errors, Duration) above the trace search results instead of the single count histogram. The three charts share a synced hover cursor, Errors toggles between rate and volume, and clicking a status series on the volume chart filters the results. Logs and other sources keep the existing histogram. +- 81000d0ce: Add a RED/Heatmap switch to the trace search results view. Heatmap shows the duration distribution over time as the same tile the dashboard renders, and the choice is kept in the URL so a reload or a shared link reopens the same view. + +### Patch Changes + +- 6e7ee8366: feat: Removing the NEXT_PUBLIC_ENABLE_ADHOC_FILTERS flag +- 6e7c77247: fix: run an edited Order By on a chart table instead of the last column header sort +- 1a4c116b2: perf: load search filter values faster in exact mode by reading several keys on + one Map column with a single `groupUniqArrayMap`, and by querying plain columns + and each Map or JSON column separately so slow attribute maps don't hold up the + rest +- 351f19400: feat: allow up to 10,000 in the team "Filter Keys Fetch Limit" setting + + The setting's input in Team settings stopped at 1,000 keys per column; it now + accepts up to 10,000. Map key discovery returns up to the setting's value per + Map column, and never fewer than 1,000. + +- 0c77d52a3: fix: stop the previous filter type's validation from blocking a save after switching types + + Switching a dashboard filter from "PromQL label values" to another type no longer fails with "Select a PromQL source". + +- 661a5a139: fix: address JSON array elements by index in row-panel filters + + "Add to Filters" on a value inside an array in a native JSON column now uses + 1-based array subscripts, for example `events[1]`, instead of quoted keys such + as `events.0`, so the filter matches rows instead of silently returning none. + + The same applies to arrays inside a JSON string (for example a String `Body` + column), which now pass 1-based integer indexes to `JSONExtract*`. JSON strings + stored in a native JSON column are wrapped in `toString()` so `JSONExtract*` + accepts them. + +- 3d12509ef: fix: keep JSON-column filter keys in JSON form when another filter is added + + Adding a second filter, or clicking "Load more" on a filter, no longer rewrites + an earlier filter on a key inside a native JSON column to `col['key']`, which + ClickHouse rejects (`Illegal type Dynamic of argument of function in`, or + `First argument for function 'arrayElement' must be array, got 'JSON'` on older + versions). + +- 86acf27c0: perf: read only 20 median-sized parts for text index key and value lookups + + Map key discovery and "all values" lookups on text indexes read the index + dictionary of every part in the time range, which is slow on tables with many + parts. They now read the 20 parts closest to the median size. Log and trace + sources can change the count with the new "Map key index parts" setting + (`mapKeyIndexMaxParts`), and which parts are read with "Map key index part size" + (`mapKeyIndexPartSelection`: median, smallest, or largest). "Load more" on a + single filter key still reads every part in the time range. + +- 2f61137e2: fix: return partial rows from `clickstack_search` instead of timing out + + The MCP search tool now scans newest-first over the same progressive time + windows as the app's search page, under a 25s budget. When time runs out it + returns the rows found so far with a `partial` block (requested range, the range + fully covered, and the window that was cut off) and a hint for the next call, + instead of failing after 30s. The time-window helpers moved from the app to + `@hyperdx/common-utils/core/searchWindows`; the app's behavior is unchanged. + +- 64a55d5b7: fix: allow grouping gauge and sum metric charts by an expression over the time column + + A table, pie or bar chart on a gauge or sum metric grouped by a time expression + such as `toStartOfHour(TimeUnix)` or `toDate(TimeUnix)` failed with + `Unknown identifier TimeUnix`, because the bucketed CTE the outer query reads + did not carry the time column. When the group-by references it, `TimeUnix` now + resolves to each metric bucket's start, so values are the same as the chart + without the time group-by. + +- e8d1d9add: fix: return the actual earliest/latest time for min/max on a date or time column + + `min` and `max` on a Date, DateTime, or DateTime64 column (for example + `Timestamp`) returned 0, because every aggregation cast its value to Float64 + first. Tables, number tiles, and the MCP `clickstack_table` tool now return the + timestamp itself. Line, bar, pie, and other charts, ratios, formulas, and number + tiles with several series keep the numeric cast. + +- 0c77d52a3: feat: rename the "Queried values" dashboard filter type to "SQL expression values" + + The new name separates it from "SQL query values" and "PromQL label values". + +- e58329c87: fix: require the password when changing a connection's host + + Updating a connection's host without sending its password used to keep the stored password, so it would be sent to the new host on the next query. Both `PUT /connections/:id` and `PUT /api/v2/connections/:id` now return 400 in that case. Prometheus endpoints never use the stored password, so it is cleared instead. The connection form asks for the password when the host is edited and shows the API's error message. + +- a0732b175: feat: save a temporary dashboard with its content intact +- 67e2eb36e: Use the sidebar layout icons for showing and hiding search filters +- 2cec94f7d: feat: let SQL query dashboard filters use an optional source + + A SQL query filter can now pick a source on its connection, or a data source variable, and can do so on a connection variable too. With one, its SQL can use `$__sourceTable` and `$__filters`, which applies the dashboard's filter selections for that source, the same way raw SQL tiles do. The external API and MCP read and write it as `sourceId`. + +- 0c77d52a3: feat: autocomplete table and column names in the SQL query filter editor + + The SQL query dashboard filter's editor now suggests the tables and columns of + the selected connection's sources, like the Raw SQL tile editor. + +- 7afa31c71: feat: Add a team-wide default (All or Exact) for the search filters "Show All Values" setting +- 6a5fd1127: In dark mode, the field background now has opacity so it works on different + backgrounds, such as popovers. It is set directly in the theme tokens instead of + through a Click UI token that resolved to a solid color. +- Updated dependencies [22377dfcf] +- Updated dependencies [b129ec1ac] +- Updated dependencies [e36cec3a2] +- Updated dependencies [1a4c116b2] +- Updated dependencies [351f19400] +- Updated dependencies [86acf27c0] +- Updated dependencies [e80fc1ba2] +- Updated dependencies [2f61137e2] +- Updated dependencies [64a55d5b7] +- Updated dependencies [e8d1d9add] +- Updated dependencies [d99be7650] +- Updated dependencies [e58329c87] +- Updated dependencies [2cec94f7d] +- Updated dependencies [7afa31c71] +- Updated dependencies [f11b9f58b] +- Updated dependencies [7905721bc] + - @hyperdx/api@2.47.0 + - @hyperdx/common-utils@0.36.1 + ## 2.46.0 ### Minor Changes diff --git a/packages/app/package.json b/packages/app/package.json index 6fd972dcd1..1107255659 100644 --- a/packages/app/package.json +++ b/packages/app/package.json @@ -1,6 +1,6 @@ { "name": "@hyperdx/app", - "version": "2.46.0", + "version": "2.47.0", "private": true, "license": "MIT", "engines": { @@ -40,7 +40,7 @@ "@dnd-kit/utilities": "^3.2.2", "@hookform/resolvers": "^3.9.0", "@hyperdx/api": "workspace:*", - "@hyperdx/common-utils": "^0.36.0", + "@hyperdx/common-utils": "^0.36.1", "@mantine/core": "^9.0.0", "@mantine/dates": "^9.0.0", "@mantine/dropzone": "^9.0.0", diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.md index 1b46fa44b4..2007317350 100644 --- a/packages/cli/CHANGELOG.md +++ b/packages/cli/CHANGELOG.md @@ -1,5 +1,23 @@ # @hyperdx/cli +## 0.9.2 + +### Patch Changes + +- d99be7650: fix: MCP `event_patterns`, `emerging_signals` and search denoise, and the CLI, + no longer split the same log line by its date. Before mining, the Drain miner + masks the date of a timestamp (weekday, day, month, year) to ``, so dated + lines from different days, months or years get the same template instead of one + "new" and one "gone". Times and ids are not masked. Within one day lines group + as before, except that klog lines with different severity letters are no longer + merged. No mask changes a line's token count: the `event_patterns` drill-down + predicate matches `` as one or more non-whitespace characters, also inside + a token (the new `masks` option of the pattern predicate builders, which only + the MCP drill-down sets), and search denoise masks bodies the same way before + matching. Masking instructions get an optional `perWord` flag (default off, as + in drain3). The web app's Event patterns view uses its own drain3 config and is + unchanged. + ## 0.9.1 ### Patch Changes diff --git a/packages/cli/package.json b/packages/cli/package.json index f922b75f7d..1fdb14859e 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "@hyperdx/cli", - "version": "0.9.1", + "version": "0.9.2", "license": "MIT", "type": "module", "repository": { @@ -41,7 +41,7 @@ }, "devDependencies": { "@clickhouse/client": "1.23.0-head.fae5998.1", - "@hyperdx/common-utils": "^0.36.0", + "@hyperdx/common-utils": "^0.36.1", "@jest/globals": "^30.2.0", "@types/crypto-js": "^4.2.2", "@types/jest": "^29.5.14", diff --git a/packages/common-utils/CHANGELOG.md b/packages/common-utils/CHANGELOG.md index 5670ab628a..8829505b6b 100644 --- a/packages/common-utils/CHANGELOG.md +++ b/packages/common-utils/CHANGELOG.md @@ -1,5 +1,104 @@ # @hyperdx/common-utils +## 0.36.1 + +### Patch Changes + +- e36cec3a2: perf: share one Map key read between concurrent callers + + The search page's filter panel, search bar and group-by each load the same Map + keys when the page opens. Each sent its own text index read, which on large + tables can use several GiB of memory. Concurrent requests for the same keys now + share one query. + +- 1a4c116b2: perf: load search filter values faster in exact mode by reading several keys on + one Map column with a single `groupUniqArrayMap`, and by querying plain columns + and each Map or JSON column separately so slow attribute maps don't hold up the + rest +- 351f19400: feat: allow up to 10,000 in the team "Filter Keys Fetch Limit" setting + + The setting's input in Team settings stopped at 1,000 keys per column; it now + accepts up to 10,000. Map key discovery returns up to the setting's value per + Map column, and never fewer than 1,000. + +- 86acf27c0: perf: read only 20 median-sized parts for text index key and value lookups + + Map key discovery and "all values" lookups on text indexes read the index + dictionary of every part in the time range, which is slow on tables with many + parts. They now read the 20 parts closest to the median size. Log and trace + sources can change the count with the new "Map key index parts" setting + (`mapKeyIndexMaxParts`), and which parts are read with "Map key index part size" + (`mapKeyIndexPartSelection`: median, smallest, or largest). "Load more" on a + single filter key still reads every part in the time range. + +- 2f61137e2: fix: return partial rows from `clickstack_search` instead of timing out + + The MCP search tool now scans newest-first over the same progressive time + windows as the app's search page, under a 25s budget. When time runs out it + returns the rows found so far with a `partial` block (requested range, the range + fully covered, and the window that was cut off) and a hint for the next call, + instead of failing after 30s. The time-window helpers moved from the app to + `@hyperdx/common-utils/core/searchWindows`; the app's behavior is unchanged. + +- 64a55d5b7: fix: allow grouping gauge and sum metric charts by an expression over the time column + + A table, pie or bar chart on a gauge or sum metric grouped by a time expression + such as `toStartOfHour(TimeUnix)` or `toDate(TimeUnix)` failed with + `Unknown identifier TimeUnix`, because the bucketed CTE the outer query reads + did not carry the time column. When the group-by references it, `TimeUnix` now + resolves to each metric bucket's start, so values are the same as the chart + without the time group-by. + +- e8d1d9add: fix: return the actual earliest/latest time for min/max on a date or time column + + `min` and `max` on a Date, DateTime, or DateTime64 column (for example + `Timestamp`) returned 0, because every aggregation cast its value to Float64 + first. Tables, number tiles, and the MCP `clickstack_table` tool now return the + timestamp itself. Line, bar, pie, and other charts, ratios, formulas, and number + tiles with several series keep the numeric cast. + +- d99be7650: fix: MCP `event_patterns`, `emerging_signals` and search denoise, and the CLI, + no longer split the same log line by its date. Before mining, the Drain miner + masks the date of a timestamp (weekday, day, month, year) to ``, so dated + lines from different days, months or years get the same template instead of one + "new" and one "gone". Times and ids are not masked. Within one day lines group + as before, except that klog lines with different severity letters are no longer + merged. No mask changes a line's token count: the `event_patterns` drill-down + predicate matches `` as one or more non-whitespace characters, also inside + a token (the new `masks` option of the pattern predicate builders, which only + the MCP drill-down sets), and search denoise masks bodies the same way before + matching. Masking instructions get an optional `perWord` flag (default off, as + in drain3). The web app's Event patterns view uses its own drain3 config and is + unchanged. +- 2cec94f7d: feat: let SQL query dashboard filters use an optional source + + A SQL query filter can now pick a source on its connection, or a data source variable, and can do so on a connection variable too. With one, its SQL can use `$__sourceTable` and `$__filters`, which applies the dashboard's filter selections for that source, the same way raw SQL tiles do. The external API and MCP read and write it as `sourceId`. + +- 7afa31c71: feat: Add a team-wide default (All or Exact) for the search filters "Show All Values" setting +- f11b9f58b: perf: read large-block Map text index values per key, most frequent first + + "All values" lookups on Map key=value text indexes with large dictionary blocks + (2048 or more, e.g. `text_index_dictionary_block_size = 8192`, which the + collector's migration now sets on ClickHouse 26.6+) + now read each key separately, in parallel queries of at most 10 keys, and stop + after 20,000 tokens per key, so a key with a value per row (request IDs, UUIDs) + can't hold up the others. Values come from `topKWeighted(limit)` weighted by + each token's row count, so a capped list holds the most common values, most + common first. These reads no longer apply `max_rows_to_read`: the index is read + in sorted token order, so the cap left every key that sorts after a + high-cardinality key with no values. + + Indexes with smaller blocks, including ClickHouse's default of 512, keep the + previous query shape (one read for all of a query's keys, the alphabetically + first values, and the `max_rows_to_read` cap), now in parallel queries of at + most 10 keys, so the cap's cut-off affects at most 9 other keys. Every read loads + each part's sparse index, which is large with small blocks, so one read per key + would multiply that cost. The block size comes from the index definition, then + the table's `text_index_dictionary_block_size`, then the server's. + + Text indexes on plain columns now return the most frequent values with + `topKWeighted`, in parallel queries of at most 10 columns, without the row cap. + ## 0.36.0 ### Minor Changes diff --git a/packages/common-utils/package.json b/packages/common-utils/package.json index acf485e4ab..8ff8db1e30 100644 --- a/packages/common-utils/package.json +++ b/packages/common-utils/package.json @@ -1,7 +1,7 @@ { "name": "@hyperdx/common-utils", "description": "Common utilities for HyperDX application", - "version": "0.36.0", + "version": "0.36.1", "license": "MIT", "private": true, "files": [ diff --git a/packages/otel-collector/CHANGELOG.md b/packages/otel-collector/CHANGELOG.md index 0dcba467cf..95bbf5096d 100644 --- a/packages/otel-collector/CHANGELOG.md +++ b/packages/otel-collector/CHANGELOG.md @@ -1,5 +1,32 @@ # @hyperdx/otel-collector +## 2.47.0 + +### Patch Changes + +- d945d7ea8: fix(security): bump `clickhouse-go/v2` to v2.51.0 in the migration tool + + Pulls in `github.com/andybalholm/brotli` v1.2.7, which fixes GO-2026-6888 + (CVE-2026-111497), a denial of service in the brotli decompressor. + +- f11b9f58b: perf: use 8,192-token text index dictionary blocks on the seeded tables + + On ClickHouse 26.6 and later, the collector's schema migration now sets + `text_index_dictionary_block_size = 8192` (the default is 512) on `otel_logs`, + `otel_traces` and `hyperdx_sessions`, with `ALTER TABLE … MODIFY SETTING` after + the seed. Each part's text index sparse index is 16x smaller. Reading filter + values with `mergeTreeTextIndex()` loads that index for every part on every + read, and on a 537M-row table 4,096-token blocks already made those reads about + 10x cheaper per part, made searches for a rare attribute value 3-7x faster, and + made searches for a common value 10-30% faster. + + Because it's an `ALTER`, existing installs get it too: new parts use 8,192-token + blocks, and existing parts keep theirs until they're merged. A table that + already sets `text_index_dictionary_block_size` is left alone, and operators who + manage these tables themselves can turn the change off with + `HYPERDX_OTEL_EXPORTER_SET_TEXT_INDEX_BLOCK_SIZE=false`. ClickHouse 26.2–26.5 + doesn't have the setting, so those servers keep 512-token blocks. + ## 2.46.0 ### Minor Changes diff --git a/packages/otel-collector/package.json b/packages/otel-collector/package.json index 95b7888158..8803d84487 100644 --- a/packages/otel-collector/package.json +++ b/packages/otel-collector/package.json @@ -1,7 +1,7 @@ { "name": "@hyperdx/otel-collector", "description": "HyperDX OpenTelemetry Collector configuration and Docker image", - "version": "2.46.0", + "version": "2.47.0", "license": "MIT", "private": true } diff --git a/yarn.lock b/yarn.lock index 4c27b757f6..8a10f2afa6 100644 --- a/yarn.lock +++ b/yarn.lock @@ -4094,7 +4094,7 @@ __metadata: "@clickhouse/clickstack-node": "npm:^1.0.2" "@clickhouse/client-common": "npm:1.23.0-head.fae5998.1" "@esm2cjs/p-queue": "npm:^7.3.0" - "@hyperdx/common-utils": "npm:^0.36.0" + "@hyperdx/common-utils": "npm:^0.36.1" "@hyperdx/passport-local-mongoose": "npm:^9.0.1" "@modelcontextprotocol/sdk": "npm:^1.32.0" "@opentelemetry/api": "npm:~1.9.1" @@ -4189,7 +4189,7 @@ __metadata: "@dnd-kit/utilities": "npm:^3.2.2" "@hookform/resolvers": "npm:^3.9.0" "@hyperdx/api": "workspace:*" - "@hyperdx/common-utils": "npm:^0.36.0" + "@hyperdx/common-utils": "npm:^0.36.1" "@mantine/core": "npm:^9.0.0" "@mantine/dates": "npm:^9.0.0" "@mantine/dropzone": "npm:^9.0.0" @@ -4308,7 +4308,7 @@ __metadata: resolution: "@hyperdx/cli@workspace:packages/cli" dependencies: "@clickhouse/client": "npm:1.23.0-head.fae5998.1" - "@hyperdx/common-utils": "npm:^0.36.0" + "@hyperdx/common-utils": "npm:^0.36.1" "@jest/globals": "npm:^30.2.0" "@types/crypto-js": "npm:^4.2.2" "@types/jest": "npm:^29.5.14" @@ -4335,7 +4335,7 @@ __metadata: languageName: unknown linkType: soft -"@hyperdx/common-utils@npm:^0.36.0, @hyperdx/common-utils@workspace:packages/common-utils": +"@hyperdx/common-utils@npm:^0.36.1, @hyperdx/common-utils@workspace:packages/common-utils": version: 0.0.0-use.local resolution: "@hyperdx/common-utils@workspace:packages/common-utils" dependencies: From 5024100f66f5ae77bcf5af013565d5052c468640 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Fri, 20 Nov 2026 23:36:24 +0000 Subject: [PATCH 2/2] chore(release): update root CHANGELOG.md --- CHANGELOG.md | 214 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 214 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index c61bfe31b6..1f077ca83c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,220 @@ PR — keep the `hyperdx-release-notes` comment marker intact when editing so yo edits survive regeneration. Per-package detail lives in each `packages/*/CHANGELOG.md`. +## v2.47.0 — 2026-11-20 + + + +**RED metrics and a duration heatmap for traces** + +Trace search now opens on RED metrics — Throughput, Errors and Duration — in +place of the single count histogram, so you can see how a service is behaving +before you read a single span. A RED/Heatmap switch sits alongside them, swapping +the three charts for the duration distribution over time, and whichever view you +pick is kept in the URL so a reload or a shared link reopens it. Loading filter +keys and values on large tables is a great deal faster and lighter on memory, +and the collector's schema migration now gives the seeded tables much larger text +index dictionary blocks on ClickHouse 26.6 and later. A SQL query dashboard +filter can also now take a source, so its SQL can use `$__sourceTable` and +`$__filters` the way raw SQL tiles do. Note before upgrading that changing a +ClickHouse connection's host now requires you to supply its password again. + +### 💥 Breaking Changes + +- **`NEXT_PUBLIC_ENABLE_ADHOC_FILTERS` has been removed**: ad-hoc filters are now + generally available and always on, and the flag no longer has any effect. + Remove it from your deployment configuration; if you were leaving it unset to + keep ad-hoc filters hidden, they will now be visible to everyone. +- **Changing a connection's host requires its password**: updating a + connection's host without sending its password used to keep the stored + password, so that password was then sent to the new host on the next query. + Both `PUT /connections/:id` and `PUT /api/v2/connections/:id` now return 400 in + that case, so anything that edits a connection's host — a provisioning script, + say — has to send the password with it. The connection form asks for the + password when you edit the host and shows the API's error message, and + Prometheus endpoints, which never use the stored password, have it cleared + instead. + +### ✨ New Features + +- **RED metrics above trace search results**: Throughput, Errors and Duration + charts replace the count histogram on trace searches. The three share a synced + hover cursor, Errors toggles between rate and volume, and clicking a status + series on the volume chart filters the results. Logs and other sources keep the + existing histogram. +- **A duration heatmap for trace search**: a RED/Heatmap switch on the trace + search results view shows the duration distribution over time as the same tile + your dashboards render, so you can spot a slow tail without building a chart + for it. The choice is kept in the URL, so a reload or a shared link reopens the + same view. +- **SQL query dashboard filters can take a source**: a SQL query filter can now + pick a source on its connection, or a data source variable, and can do so on a + connection variable too. With one chosen, its SQL can use `$__sourceTable` and + `$__filters`, which applies the dashboard's filter selections for that source + just as raw SQL tiles do. The external API and MCP read and write it as + `sourceId`. +- **Saving a temporary dashboard keeps its content**: a temporary dashboard now + saves with its content intact, so work you started on an ad hoc dashboard + carries over when you decide to keep it. +- **Silence and unsilence alerts through the external API**: `POST` and + `DELETE /api/v2/alerts/{id}/silenced` now silence an alert and lift that + silence again, so a planned maintenance window can quieten an alert from a + script instead of someone clicking through the UI. + +### 🔧 Improvements + +- **Larger text index dictionary blocks on the seeded tables**: on ClickHouse + 26.6 and later, the collector's schema migration now sets + `text_index_dictionary_block_size = 8192` on `otel_logs`, `otel_traces` and + `hyperdx_sessions`, where the default is 512, making each part's text index + sparse index 16x smaller. Reading filter values loads that index for every part + on every read, and on a 537M-row table even 4,096-token blocks made those reads + about 10x cheaper per part, searches for a rare attribute value 3-7x faster and + searches for a common value 10-30% faster. Existing installs get it too — new + parts use the larger blocks and existing parts keep theirs until they are + merged — while a table that already sets the setting is left alone. If you + manage these tables yourself, turn the change off with + `HYPERDX_OTEL_EXPORTER_SET_TEXT_INDEX_BLOCK_SIZE=false`; ClickHouse 26.2–26.5 + has no such setting and keeps 512-token blocks. +- **Filter values arrive most frequent first on large-block text indexes**: for a + Map key=value text index with dictionary blocks of 2,048 or more — what the + collector's migration now sets — "all values" lookups read each key separately, + in parallel queries of at most 10 keys, and stop after 20,000 tokens per key, + so a key with a value per row such as a request ID or UUID can no longer hold + up the others. A capped list now holds the most common values, most common + first, rather than the alphabetically first ones. Indexes with smaller blocks, + including ClickHouse's default of 512, keep the previous query shape but also + run in parallel queries of at most 10 keys, so a cut-off affects at most nine + other keys, and text indexes on plain columns likewise return their most + frequent values. +- **Text index lookups read 20 parts rather than every part**: map key discovery + and "all values" lookups read the index dictionary of every part in the time + range, which is slow on tables with many parts, and now read the 20 parts + closest to the median size instead. Log and trace sources can change the count + with the new "Map key index parts" setting (`mapKeyIndexMaxParts`) and which + parts are read with "Map key index part size" (`mapKeyIndexPartSelection`: + median, smallest or largest). "Load more" on a single filter key still reads + every part in the time range. +- **The search page loads its filter keys and values faster**: the filter panel, + search bar and group-by each sent their own read for the same Map keys when the + page opened, which on large tables can use several GiB of memory apiece; + concurrent requests for the same keys now share one query. In exact mode, + several keys on one Map column are read with a single `groupUniqArrayMap`, and + plain columns and each Map or JSON column are queried separately so a slow + attribute map doesn't hold up the rest. +- **"Filter Keys Fetch Limit" goes up to 10,000**: the team setting's input + stopped at 1,000 keys per column and now accepts up to 10,000. Map key + discovery returns up to the setting's value per Map column, and never fewer + than 1,000. +- **A team-wide default for the search filters' "Show All Values" setting**: a + team can now set the default mode — All or Exact — for the search filters, + so everyone starts from whichever suits your tables instead of changing it + themselves on each visit. +- **Autocomplete in the SQL query filter editor**: the editor for a SQL query + dashboard filter now suggests the tables and columns of the selected + connection's sources, the same way the Raw SQL tile editor does, so you can + write a values query without going to look up a column name. +- **A clearer name for the "Queried values" filter type**: the dashboard filter + type formerly called "Queried values" is now "SQL expression values", which + separates it from "SQL query values" and "PromQL label values". +- **Sidebar layout icons for the search filters panel**: showing and hiding the + search filters now uses the sidebar layout icons, so the control reads as a + panel toggle at a glance. +- **Field backgrounds sit properly on dark popovers**: in dark mode the field + background now carries opacity, so it works over whatever is behind it — a + popover, for instance — rather than resolving to a solid colour through a + Click UI token. It is set directly in the theme tokens. +- **Alert checks read far less history on each tick**: the check-alerts task no + longer scans seven days of alert history for every alert on every tick — a + one-row lookup skips alerts already evaluated in the current window and gives a + non-grouped alert its previous state, and a grouped alert scans only a window + the size of its own interval. Deployments carrying a long alert history should + see each tick do noticeably less work. +- **MCP query tools explain `hasToken()` separator errors**: a SQL filter calling + `hasToken()` with a needle containing `_`, `-`, `.`, `@`, `:` or a space fails + in ClickHouse with "Needle must not contain whitespace or separator + characters". The query tools now append a hint — reach for a Lucene term, or in + SQL `position()` for the exact term, with `hasAllTokens()` as an optional + text-index speed-up — and the `whereLanguage` description warns about it, so an + assistant can correct the query rather than relaying an opaque failure. +- **Failed ClickHouse proxy requests are categorised in HyperDX's own + telemetry**: a request that fails through the ClickHouse proxy now carries an + `error_category` on its server span and on the `clickhouse_proxy.query` SLI, + including the errors ClickHouse writes after it has already returned a 200. The + span also records the upstream status, the ClickHouse error code and type, and + the body of upstream errors that carry no ClickHouse code, so proxy failures + can be told apart when you monitor your own deployment instead of arriving as + one undifferentiated error count. + +### 🐛 Bug Fixes + +- **Filters on native JSON column keys survive a second filter**: adding another + filter, or clicking "Load more" on a filter, no longer rewrites an earlier + filter on a key inside a native JSON column to `col['key']` — a form ClickHouse + rejects with `Illegal type Dynamic of argument of function in`, or + `First argument for function 'arrayElement' must be array, got 'JSON'` on older + versions. +- **"Add to Filters" matches values inside JSON arrays**: filtering on a value in + an array in a native JSON column now uses 1-based array subscripts such as + `events[1]` rather than quoted keys such as `events.0`, so the filter matches + rows instead of silently returning none. Arrays inside a JSON string — a String + `Body` column, for example — are handled the same way. +- **Changing a dashboard filter's type no longer blocks the save**: switching a + filter from "PromQL label values" to another type used to fail with "Select a + PromQL source", because the old type's validation still applied. The save now + goes through. +- **An edited Order By on a chart table is the one that runs**: a chart in table + form now orders by the Order By you edited, instead of falling back to the last + column header you sorted by. +- **Gauge and sum metric charts can be grouped by a time expression**: a table, + pie or bar chart on a gauge or sum metric grouped by an expression over the + time column, such as `toStartOfHour(TimeUnix)` or `toDate(TimeUnix)`, failed + with `Unknown identifier TimeUnix`, because the bucketed CTE the outer query + reads did not carry the time column. `TimeUnix` now resolves to each metric + bucket's start when the group-by references it, so the values match the same + chart without the time group-by. +- **`min` and `max` on a time column return the real earliest and latest time**: + taking `min` or `max` of a Date, DateTime or DateTime64 column such as + `Timestamp` returned 0, because every aggregation cast its value to Float64 + first. Tables, number tiles and the MCP `clickstack_table` tool now give you the + timestamp itself, while line, bar, pie and other charts, ratios, formulas and + number tiles with several series keep the numeric cast. +- **`clickstack_search` returns the rows it found instead of timing out**: the + MCP search tool now scans newest-first over the same progressive time windows + as the search page, under a 25 second budget. When the budget runs out you get + the rows found so far plus a `partial` block — the range you asked for, the + range fully covered, and the window that was cut off — and a hint for the next + call, rather than a failure after 30 seconds. +- **The same log line is no longer split into one pattern per day**: the MCP + `event_patterns` and `emerging_signals` tools, search denoise and the CLI now + mask the date part of a timestamp — weekday, day, month and year — to `` + before mining, so a dated line recurring across days, months or years lands in + one template rather than appearing as one "new" pattern and one that has gone. + Times and ids are left unmasked and lines within a single day group as they did + before, except that klog lines with differing severity letters are no longer + merged, and the `event_patterns` drill-down still matches the events its + pattern describes. The web app's Event patterns view uses its own + configuration and is unchanged. + +### 📦 Build / Packaging + +- **Security fix in the collector's migration tool**: the migration tool now + builds against `clickhouse-go/v2` v2.51.0, which pulls in + `github.com/andybalholm/brotli` v1.2.7 and fixes GO-2026-6888 + (CVE-2026-111497), a denial of service in the brotli decompressor. + + + +### 📦 Package changelogs + +- `@hyperdx/api` 2.46.0 → 2.47.0 — [changelog](https://github.com/ClickHouse/ClickStack/blob/main/packages/api/CHANGELOG.md#2470) +- `@hyperdx/app` 2.46.0 → 2.47.0 — [changelog](https://github.com/ClickHouse/ClickStack/blob/main/packages/app/CHANGELOG.md#2470) +- `@hyperdx/cli` 0.9.1 → 0.9.2 — [changelog](https://github.com/ClickHouse/ClickStack/blob/main/packages/cli/CHANGELOG.md#092) +- `@hyperdx/common-utils` 0.36.0 → 0.36.1 — [changelog](https://github.com/ClickHouse/ClickStack/blob/main/packages/common-utils/CHANGELOG.md#0361) +- `@hyperdx/otel-collector` 2.46.0 → 2.47.0 — [changelog](https://github.com/ClickHouse/ClickStack/blob/main/packages/otel-collector/CHANGELOG.md#2470) + + + ## v2.46.0 — 2026-11-18