Repository navigation
Format float and double values with std::to_chars, not std::to_string - #634
Open
polyglotAI-bot wants to merge 2 commits into
Open
polyglotAI-bot wants to merge 2 commits into
polyglotAI-bot wants to merge 2 commits into
Conversation
std::to_string is printf("%f"): fixed notation with 6 decimal places.
A bound SQL_C_DOUBLE or SQL_C_FLOAT parameter was sent to the server
as this text, so 1e-7 was stored as 0. With
RowBinaryWithNamesAndTypes, SQL_C_CHAR and SQL_C_WCHAR of a Float32 or
Float64 column returned the same text ("0.000000", "42.000000").
Write the shortest text that converts back to the same value
(std::to_chars), with the notation of the server: fixed for
1e-6 <= |x| < 1e21, scientific for other values. A float is
formatted as a float. std::to_chars does not depend on the locale.
SQL_C_NUMERIC of a Float32 or Float64 value parses this text with the
Decimal/Numeric parser, which does not accept an exponent, so that
conversion uses the fixed notation for all values.
Fixes: #632
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Auto-approved by the AI review gate on 690ed54. Details in the approver's comment on this pull request.
AI review gate — auto-approved
History (latest 5 heads, newest first)
|
The parameter round-trip tests compare the text that comes back from
the server. Server 25.6 (the CI server in test/docker-compose.yml)
parses "9.99e-07" as 9.989999999999999e-7, and "3.4028235e+38" as the
Float32 3.4028233e38, also for SELECT toFloat64('9.99e-07'). Server
26.10 parses both exactly. The driver sends the correct text.
Use "9.5e-07" and "3e+38", which the server parses exactly. They test
the same notation branches, and both fail with std::to_string.
type_conversion_ut still checks the driver text of "9.99e-07" and
"3.4028235e+38" without a server.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Auto-approved by the AI review gate on 1426297. Details in the approver's comment on this pull request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #632.
from_value<float>::to_value<std::string>andfrom_value<double>::to_value<std::string>indriver/utils/type_info.husedstd::to_string, which isprintf("%f"): fixed notation with 6 decimal places. Two paths use these functions:Statement::prepareHttpRequest()converts a boundSQL_C_DOUBLEorSQL_C_FLOATvalue to this text.1e-7was sent as0.000000and stored as0.RowBinaryWithNamesAndTypes.SQL_C_CHAR,SQL_C_WCHARandSQL_C_BINARYof aFloat32orFloat64value go throughconvert_via_proxy<std::string>to the same functions (0.000000,42.000000).ODBCDriver2keeps the text of the server and does not use them.The functions now write the shortest text that converts back to the same value, with
std::to_chars. Afloatis formatted as afloat(0.1fgives0.1).std::to_charsdoes not depend on the locale;std::to_stringusedLC_NUMERICof the application.One difference from the fix suggested in the issue.
std::to_charswith no format selects the shorter notation, so100000.0gives1e+05and1700000000.0gives1.7e+09. The server rejects that text for integer andDateTime64parameters, and it accepted the old text1700000000.000000forDateTime64(9). So the notation is the notation of the server: fixed for 1e-6 <= |x| < 1e21, scientific for other values. In this range the text is the text that the server writes for the same value, soRowBinaryWithNamesAndTypesandODBCDriver2give the same text. Outside it only the exponent is different (1e-07, server1e-7). If you prefer the plainstd::to_chars(first, last, src), it is a one-line change infloatToChars.SQL_C_NUMERIC.
SQL_C_NUMERICof aFloat32orFloat64value goes through the same text to the Decimal/Numeric parser, which does not accept an exponent. Two newto_value<SQL_NUMERIC_STRUCT>specializations give that parser the fixed notation for all values. The results for the values that worked before stay the same (1000000,-2.5→-2,1e-7→0at scale 0). Two behavior changes: a scale above 6 now keeps the digits (1e-7at scale 9 was0, now0.000000100), and the fractional digits are truncated, not rounded to 6 decimals first (0.9999999at scale 0 was1, now0, likeODBCDriver2).Changes
driver/utils/type_info.h: newfloatToChars(std::to_charswith the server notation, or a given notation); the float and doubleto_value<std::string>use it; new float and doubleto_value<SQL_NUMERIC_STRUCT>with the fixed notation.driver/test/statement_parameter_binding_ut.cpp: the parameter text of boundSQL_C_DOUBLE/SQL_C_FLOATvalues (Float64, Float32, Int32, Decimal parameter types).driver/test/type_conversion_ut.cpp: text ↔ float/double round trips (both notations and their limits,-0,inf), and float/double →SQL_NUMERIC_STRUCT. These tests do not need a server, so they also run on macOS and Windows.driver/test/statement_parameters_it.cpp:SQL_C_DOUBLE/SQL_C_FLOATparameter → server →SQLGetDataround trips.driver/test/misc_it.cpp:SQLGetDataofFloat32/Float64asSQL_C_CHAR/SQL_C_WCHARand asSQL_C_NUMERIC, in both formats. TheODBCDriver2rows and theSQL_C_NUMERICrows pin the output from before the change.Test
Server 26.10.4.207, Debug build with Clang 18, unixODBC. The program
repro.cfrom the issue now gives1e-7,123.456789123,0.3333333333333333,2.5e-10for the bound values, and1e-07,123.456789123,1e-40,42forSQL_C_CHARwithRowBinaryWithNamesAndTypes.master, all new tests fail except the rows that pin the old output (theODBCDriver2text, theSQL_C_NUMERICrows,inf,-inf, and0.000001, which%fgives correctly). With this change they pass.std::to_charsand noSQL_NUMERIC_STRUCTspecializations, theSQL_C_NUMERICrows fail (Cannot interpret '1e+06' as Decimal/Numeric).ctest -E "-dsn-[45]$": 16/16 passed.pytest: 41/41 passed with the ANSI and with the Unicode DSN.test/docker-compose.yml):ctest -E "-dsn-[45]$"16/16,pytest41/41 with each DSN. This server parses some float texts with an error of 1 ULP:9.99e-07gives9.989999999999999e-7, and3.4028235e+38asFloat32gives3.4028233e38, also inSELECT toFloat64('9.99e-07'). Server 26.10 parses them exactly, and the driver sends the correct text. So the round trips instatement_parameters_it.cppuse values that 25.6 parses exactly (9.5e-07,3e+38).type_conversion_utchecks the driver text of9.99e-07and3.4028235e+38without a server.curlandparam_x: the server accepts the new texts forFloat32,Float64,Decimal(P, S), integer,StringandDateTime64(9)parameters (1e+20intoDecimal(15, 10)fails, as the old text did, because the value does not fit).Floating-point
std::to_charsneeds macOS 13.3 in Apple's libc++. That is the value ofCMAKE_OSX_DEPLOYMENT_TARGET.Pre-PR validation gate
repro.cfrom the issue, and the new unit tests)🤖 Generated with Claude Code
Note
Medium Risk
Changes core float-to-text conversion used for ODBC parameters and result fetch paths; behavior shifts for edge magnitudes, Numeric binding, and locales, though coverage is broad in new tests.
Overview
Replaces
std::to_string(fixed six-decimalprintfstyle) forfloatanddoubleintype_info.hwith locale-independentstd::to_charsvia newfloatToChars, using fixed notation when 1e-6 ≤ |x| < 1e21 and scientific otherwise so parameter text matches what the ClickHouse server accepts andRowBinaryWithNamesAndTypesreads align withODBCDriver2.Adds
to_value<SQL_NUMERIC_STRUCT>for floats that routes through fixed decimal text (no exponents) for the Decimal/Numeric parser, which changes behavior for high scale and truncation vs the old six-digit round-trip path.Tests lock the new formatting: bound
SQL_C_FLOAT/SQL_C_DOUBLEparameter strings, string↔float unit tests (includinginf), server round-trips for Float32/Float64, andSQLGetDataas char/wchar/numeric across wire formats.Reviewed by Cursor Bugbot for commit 1426297. Bugbot is set up for automated code reviews on this repo. Configure here.