Add Support for Vector Data Type in SQL for GraphQL#3714
Open
RubenCerna2079 wants to merge 44 commits into
Open
Add Support for Vector Data Type in SQL for GraphQL#3714RubenCerna2079 wants to merge 44 commits into
RubenCerna2079 wants to merge 44 commits into
Conversation
Joint runtime + driver bump that prerequisites issue #2768 (MSSQL JSON data type support). Behavior-preserving — no production logic changes. WHY BOTH BUMPS ARE REQUIRED TOGETHER ==================================== SqlDbType.Json (numeric value 35) is a BCL enum value in System.Data, added in .NET 9 and present in .NET 10. It is NOT a Microsoft.Data.SqlClient symbol — SqlClient does not get to add values to a BCL enum it does not own. On .NET 8, the SqlDbType enum stops at DateTimeOffset = 34, so Enum.TryParse<SqlDbType>("json", ignoreCase: true, out _) in TypeHelper.GetSystemTypeFromSqlDbType returns false regardless of which SqlClient version is installed. The companion feature PR for #2768 plans to add a single dictionary entry [SqlDbType.Json] = typeof(string) which simply will not compile on net8.0. Both upgrades must therefore land together as a single prerequisite. WHY .NET 10 AND NOT .NET 9 ========================== - .NET 10 is the current Long-Term Support release (Nov 2025+). - .NET 9 is Standard-Term Support, EOL May 2026. - DAB's current .NET 8 line reaches EOL November 2026. Going straight to LTS avoids a second forced upgrade inside 12 months. SCOPE OF CHANGES (NO BEHAVIOR CHANGE INTENDED) ============================================== Runtime / SDK pin: - global.json: 8.0.420 -> 10.0.301 Target framework (all 11 src/**/*.csproj): - net8.0 -> net10.0 NuGet packages (Directory.Packages.props): - Microsoft.Data.SqlClient 5.2.3 -> 6.0.2 - Microsoft.Extensions.Caching.Memory 8.0.1 -> 10.0.0 - Microsoft.Extensions.Caching.Abstractions 9.0.0 -> 10.0.0 - Microsoft.Extensions.Primitives 9.0.0 -> 10.0.0 - Microsoft.Extensions.Configuration.Binder 9.0.0 -> 10.0.0 - Microsoft.Extensions.Configuration.Json 9.0.0 -> 10.0.0 - Microsoft.Extensions.Caching.StackExchangeRedis 9.0.3 -> 10.0.0 The Microsoft.Extensions.* bumps are required because OpenTelemetry's transitive chain (Microsoft.Extensions.Logging.Configuration 10.0.0) demands Configuration.Binder >= 10.0.0 — staying on 9.0.0 produces NU1605 package-downgrade errors (warning-as-error). Service.csproj (Microsoft.NET.Sdk.Web) — drop now-redundant PackageReferences (NU1510, warning-as-error): - Microsoft.Extensions.Configuration.Binder (in shared framework) - Microsoft.Extensions.Configuration.Json (in shared framework) Azure DevOps pipelines (9 occurrences across 8 files): - UseDotNet@2 version: 8.0.x -> 10.0.x Container base images (Dockerfile): - mcr.microsoft.com/dotnet/sdk:8.0-cbl-mariner2.0 -> 10.0-azurelinux3.0 - mcr.microsoft.com/dotnet/aspnet:8.0-cbl-mariner2.0 -> 10.0-azurelinux3.0 (cbl-mariner2.0 tag does not exist for .NET 9+; Azure Linux 3.0 is Microsoft's documented successor.) Aspire AppHost (src/Aspire.AppHost/AppHost.cs): - WithArgs("-f", "net8.0") -> WithArgs("-f", "net10.0") (mssql + pg) Build/publish scripts: - scripts/publish.ps1: $dotnetTargetFrameworks net8.0 -> net10.0 - scripts/create-manifest-file.ps1: $dotnetTargetFrameworks + hashtable keys net8.0_{rid} -> net10.0_{rid} (TODO marker: release-engineering to confirm download URLs/hashes resolve) License file rename: - external_licenses/Microsoft.Data.SqlClient.SNI.5.2.0.License.txt -> external_licenses/Microsoft.Data.SqlClient.SNI.6.0.0.License.txt - scripts/notice-generation.ps1: path updated to match - TODO marker at top of license file: contents still 5.2.0 text; release-engineering to refresh from upstream before merge. SUPPRESSED ASPDEPR008 (warning-as-error) — DELIBERATE ===================================================== ASP.NET Core 10 deprecates IWebHostBuilder / IWebHost in favor of WebApplicationBuilder / IHost. Two locations in this repo still consume the obsolete types and would block compilation under the repo's TreatWarningsAsErrors=true setting: - src/Service/Program.cs (2 sites): test-only helpers CreateWebHostBuilder + CreateWebHostFromInMemoryUpdatableConfBuilder are consumed by the existing TestServer fixture which takes IWebHostBuilder. - src/Service.Tests/Configuration/ConfigurationTests.cs (3 sites): the consumer of those helpers. Migrating from WebHost to HostBuilder/WebApplicationBuilder is a behavior-affecting refactor and explicitly OUT OF SCOPE for this no-behavior-change prerequisite PR. Suppression is added at the project level (NoWarn=ASPDEPR008 on Service.csproj + Service.Tests.csproj) with an inline TODO comment pointing at this branch and a follow-up issue link placeholder. VALIDATION ========== dotnet --version : 10.0.301 dotnet restore --nologo : 11/11 projects restored (8.05s avg) dotnet build -c Debug : 0 warnings, 0 errors, 10.32s Multi-engine test categories (MsSql, PostgreSql, MySql, CosmosDb_NoSql, DwSql) were NOT run locally; they MUST run green on the Azure DevOps multi-engine matrix before this PR merges. That matrix is the gate. NEXT STEPS BEFORE MERGE ======================= 1. Release-engineering: refresh external_licenses/Microsoft.Data.SqlClient.SNI.6.0.0.License.txt from upstream for the SNI 6.0.0 version (currently still 5.2.0 text with a TODO marker at top). 2. Release-engineering: confirm scripts/create-manifest-file.ps1 net10.0_{linux,win,osx}-x64 download URLs and SHA hashes resolve once the .NET 10 publish cycle runs. 3. Multi-engine CI matrix runs green (MsSql, PostgreSql, MySql, CosmosDb_NoSql, DwSql) — that is the merge gate. 4. File follow-up issue for ASPDEPR008 migration (WebHost -> WebApplicationBuilder / IHost) so the NoWarn suppressions can be removed in a future behavior-changing PR.
- Fix UseDotNet displayName v8.0.x -> v10.0.x across pipeline files
- Replace internal branch-path TODOs with durable follow-up notes
- Rename SNI license to 6.0.2 (actual runtime version) and fix notice-generation reference
- Bump Microsoft.AspNetCore.{TestHost,Authorization,Authentication.JwtBearer,Mvc.Testing} to 10.0.0
Aspire.Hosting pulls MessagePack 2.5.192 transitively (via KubernetesClient), which has high-severity advisory GHSA-hv8m-jj95-wg3x (CVE-2026-48109). The repo treats NU1903 as error, failing restore. Pin to patched 2.5.301 via CPM and add a direct reference in Aspire.AppHost to force the transitive upgrade.
…or change SqlClient 6.x changed the internal SqlError 9-parameter constructor: the win32ErrorCode parameter is now Int32 (was UInt32 in 5.x), and there are now multiple 9-parameter overloads. Select the constructor by exact parameter types and pass (int)0 for win32ErrorCode so CreateSqlException works across SqlClient versions.
Starting with .NET 10, setting an environment variable to an empty string preserves it as an empty value instead of deleting it (which previously surfaced here as null). This caused ValidateAspNetCoreUrls to treat an empty ASPNETCORE_URLS as invalid and exit with error, a regression from .NET 8 where an empty value was equivalent to unset. Explicitly treat an empty value as valid (Kestrel falls back to default URLs), preserving the prior behavior. Whitespace-only values remain invalid.
These three files were modified by an earlier merge in a way unrelated to the .NET 10 / SqlClient 6.x upgrade, reverting recent main formatting/cleanup: - CLRtoJsonValueTypeUnitTests.cs: restore the DBTYPE_RESOLUTION_ERROR constant that was inadvertently removed. - SqlUpdateQueryStructure.cs: restore main's formatting of the update-fields block. - MutationEngineFactory.cs: restore main's version. Restoring them to match main keeps this PR scoped to the upgrade and addresses reviewer feedback (constant removal and formatting).
Base automatically changed from
dev/rubencerna/first-changes-vector-type
to
main
July 13, 2026 21:18
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end GraphQL support for SQL Server vector/array-backed columns by (1) coercing leaf-list results into HotChocolate runtime scalar types, and (2) ensuring list-valued mutation inputs for array columns are treated as scalars (not relationships) and correctly parameterized for SQL.
Changes:
- Add leaf-list result coercion (
CoerceJsonLeafValueToRuntimeType) so GraphQL can return list-of-scalar fields (e.g.,vectorexposed as[Single]) without HotChocolate leaf coercion errors. - Add
ISqlMetadataProvider.TryGetUnderlyingFieldKindand plumb it through mutation validation/parsing so array columns aren’t misclassified as relationship inputs. - Add MSSQL integration tests + enable GraphQL for the VectorType test entity, plus stringify list inputs for SQL parameters.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Service.Tests/SqlTests/GraphQLQueryTests/MsSqlGraphQLVectorTypesTests.cs | Adds GraphQL integration coverage for vector read/query + create/update/delete mutations. |
| src/Service.Tests/dab-config.MsSql.json | Enables GraphQL for the VectorType entity in MSSQL test config. |
| src/Core/Services/TypeHelper.cs | Adds SystemType→SyntaxKind mapping helper used for array column kind detection. |
| src/Core/Services/MultipleMutationInputValidator.cs | Uses underlying kind detection to avoid treating array scalar lists as relationship fields. |
| src/Core/Services/MetadataProviders/SqlMetadataProvider.cs | Implements TryGetUnderlyingFieldKind for SQL-backed metadata providers. |
| src/Core/Services/MetadataProviders/ISqlMetadataProvider.cs | Adds the TryGetUnderlyingFieldKind API to the provider contract. |
| src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs | Implements the new API as “not supported” for Cosmos. |
| src/Core/Services/ExecutionHelper.cs | Refactors scalar coercion and adds list-of-leaf coercion to runtime types for HotChocolate. |
| src/Core/Resolvers/SqlMutationEngine.cs | Plumbs metadata provider into mutation input parsing to distinguish array columns from relationships. |
| src/Core/Resolvers/Sql Query Structures/SqlUpdateQueryStructure.cs | Uses GetStringifiedValue so list values are correctly converted before parameter parsing. |
| src/Core/Resolvers/Sql Query Structures/SqlInsertQueryStructure.cs | Uses GetStringifiedValue for insert parameter values. |
| src/Core/Resolvers/Sql Query Structures/BaseSqlQueryStructure.cs | Adds GetStringifiedValue to serialize list inputs into JSON array strings for parsing. |
| src/Core/Resolvers/MultipleCreateOrderHelper.cs | Uses underlying kind detection to treat array scalar lists as scalars. |
| config-generators/mssql-commands.txt | Updates generator command to enable GraphQL for VectorType. |
| // Licensed under the MIT License. | ||
|
|
||
| using System.Net; | ||
| using System.Text.Json; |
| using Azure.DataApiBuilder.Core.Services; | ||
| using Azure.DataApiBuilder.Service.Exceptions; | ||
| using Azure.DataApiBuilder.Service.GraphQLBuilder.Mutations; | ||
| using HotChocolate.Language; |
Comment on lines
+400
to
+414
| if (TryGetBackingColumn(entityName, fieldName, out string? columnName)) | ||
| { | ||
| SourceDefinition sourceDefinition = GetSourceDefinition(entityName); | ||
| ColumnDefinition column = sourceDefinition.Columns[columnName]; | ||
|
|
||
| // If the column is an array type, we need to get the syntax kind from the element system type. | ||
| if (column.IsArrayType && TypeHelper.TryGetSyntaxKindFromSystemType(column.ElementSystemType!, out fieldKind)) | ||
| { | ||
| return true; | ||
| } | ||
| } | ||
|
|
||
| fieldKind = 0; | ||
| return false; | ||
| } |
Comment on lines
+255
to
+265
| /// <summary> | ||
| /// | ||
| /// </summary> | ||
| /// <param name="entityName"></param> | ||
| /// <param name="fieldName"></param> | ||
| /// <param name="fieldKind"></param> | ||
| /// <returns></returns> | ||
| public bool TryGetUnderlyingFieldKind( | ||
| string entityName, | ||
| string fieldName, | ||
| [NotNullWhen(true)] out SyntaxKind fieldKind); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why make this change?
What is this change?
[string]we moved the section that parses the scalar values to types hot chocolate can use in order output them to the user. This newCoerceJsonLeafValueToRuntimeTypemethod is then used in a loop to allow support for arrays inside theExecutionHelper.csfile.TryGetUnderlyingFieldKindinISqlMetadataProvider.csthat checks if the value is a list and it comes from a row that has theIsArrayType. If that condition is met, then it means the list value we have should be processed as a regular ScalarType and not as a List/Object that is a relationship to a different entity. This method is used in the following files:MultipleMutationInputValidator.cs,MultipleCreateOrderHelper.cs,SqlMutationEnginge.cs.GetStringifiedValuefunction that checks if the value that we are going to parse to the data base is a list or a regular value and transforms it into a string accordingly. Since we set all values as strings, this is something that cannot be directly done with a list as the.ToString()function will return a useless value. This value is then parsed before being used as a parameter in the data base query.How was this tested?
Added tests that query (read) from the data base as well as mutation (change) to the data base such as
create,update, anddelete.Sample Request(s)
Query
Create
Update
Delete