Skip to content

Latest commit

 

History

74 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

esql-dotnet

CI

Write LINQ, get ES|QL, execute against Elasticsearch. Type-safe, compile-time field resolution, Native AOT ready.

Packages

Package Description
Elastic.Esql LINQ-to-ES|QL translation engine -- no HTTP dependencies, pure query generation
Elastic.Clients.Esql EsqlClient that connects the translation engine to a real cluster via Elastic.Transport

Quick Start

1. Define your domain types

Clean POCOs with System.Text.Json attributes:

public class LogEntry
{
    [JsonPropertyName("@timestamp")]
    public DateTime Timestamp { get; set; }

    [JsonPropertyName("log.level")]
    public string Level { get; set; }

    public string Message { get; set; }

    [JsonPropertyName("service.name")]
    public string ServiceName { get; set; }

    public long Duration { get; set; }
}

Field names are resolved from [JsonPropertyName] attributes, or via the configured JsonNamingPolicy (defaults to camelCase).

2. Write LINQ, get ES|QL

var esql = new EsqlQueryable<LogEntry>()
    .From("logs-*")
    .Where(l => l.Level == "ERROR" && l.Duration > 1000)
    .OrderByDescending(l => l.Timestamp)
    .Take(50)
    .ToString();

Produces:

FROM logs-*
| WHERE (log.level == "ERROR" AND duration > 1000)
| SORT @timestamp DESC
| LIMIT 50

3. Execute against Elasticsearch

var transport = new DistributedTransport(
    new TransportConfiguration(new Uri(url), new ApiKey(apiKey)));

var settings = new EsqlClientSettings(transport);
using var client = new EsqlClient(settings);

var errors = await client.CreateQuery<LogEntry>()
    .From("logs-*")
    .Where(l => l.Level == "ERROR")
    .OrderByDescending(l => l.Timestamp)
    .Take(10)
    .ToListAsync();

How It Works

  Your C# code                          Runtime
┌──────────────┐    ┌──────────────────────────────────────┐
│  POCO types  │    │                                      │
│  + STJ attrs │    │  LINQ expression tree                │
└──────┬───────┘    │         │                            │
       │            │         v                            │
       │            │  ┌──────────────────┐                │
       └───────────>│  │   Elastic.Esql   │  ES|QL string  │
                    │  │  LINQ-to-ES|QL   │────────┐       │
                    │  └──────────────────┘        │       │
                    │                              v       │
                    │  ┌──────────────────┐  ┌───────────┐ │
                    │  │ Elastic.Clients  │  │   HTTP     │ │
                    │  │     .Esql        │─>│ execution  │ │
                    │  └──────────────────┘  │ → results  │ │
                    │                        └───────────┘ │
                    └──────────────────────────────────────┘

What Translates?

C# LINQ ES|QL
.Where(l => l.Level == "ERROR") WHERE log.level == "ERROR"
.OrderByDescending(l => l.Timestamp) SORT @timestamp DESC
.Take(50) LIMIT 50
.Select(l => new { l.Message }) KEEP message
.Select(l => new { Secs = l.Duration / 1000 }) EVAL secs = (duration / 1000)
.GroupBy(...).Select(g => new { Count = g.Count() }) STATS count = COUNT(*) BY ...
.Where(l => l.Message.Contains("timeout")) WHERE message LIKE "*timeout*"
.Where(l => l.Timestamp.Year == 2025) WHERE DATE_EXTRACT("year", @timestamp) == 2025
.Where(l => Math.Abs(l.Delta) > 0.5) WHERE ABS(delta) > 0.5
EsqlFunctions.Match(l.Message, "error") MATCH(message, "error")
.Keep(l => l.Message, l => l.Timestamp) KEEP message, @timestamp
.Drop("duration", "host") DROP duration, host
.LeftJoin(...) / .LookupJoin(...) LOOKUP JOIN index ON field
.Completion(l => l.Message, endpoint) COMPLETION col = message WITH {...}
.Row(() => new { prompt = "..." }) ROW prompt = "..."
.From("books", MetadataField.Score) FROM books METADATA _score
_ => EsqlMetadata.Score (in any lambda) _score
EsqlFunctions.Knn(field, queryVec, opts) KNN(field, [...], { ... })
EsqlFunctions.TextEmbedding(text, id) TEXT_EMBEDDING("text", "id")
EsqlFunctions.VCosine(a, b) etc. V_COSINE(a, b) etc.
.Fork(b1, b2, ...).Fuse() FORK ( ... ) ( ... ) | FUSE

See the Elastic.Esql README for the full list including string methods, DateTime arithmetic, and ES|QL-specific functions.

Key Features

Full LINQ Translation

Translates .Where(), .Select(), .GroupBy(), .OrderBy(), .Take(), and more into ES|QL commands: WHERE, EVAL, KEEP, DROP, STATS...BY, SORT, LIMIT, RENAME, ROW, COMPLETION, LOOKUP JOIN, FORK, FUSE, and FROM ... METADATA.

RawEsql Fragment Append

For expert scenarios, append raw ES|QL fragments inline with .RawEsql(...) while keeping the existing typed execution pipeline:

var rows = client.Query<LogEntry>(q => q
    .From("logs-*")
    .RawEsql("WHERE log.level == \"ERROR\"")
    .RawEsql("| LIMIT 25"));

You can also switch the downstream result type with RawEsql<TSource, TNext>(...).

80+ ES|QL Functions

Math (ABS, SQRT, ROUND, ...), string (TRIM, CONCAT, REPLACE, ...), date/time (DATE_EXTRACT, DATE_TRUNC, NOW, ...), search (MATCH, KQL, QSTR, ...), IP (CIDR_MATCH, IP_PREFIX), cast operators (::integer, ::keyword, ...), grouping (BUCKET, CATEGORIZE), aggregation (PERCENTILE, MEDIAN, STD_DEV, VALUES, ...), and dense vector (KNN, TEXT_EMBEDDING, V_COSINE, V_DOT_PRODUCT, V_HAMMING, V_L1_NORM, V_L2_NORM).

Vector and Hybrid Search

Run KNN, exact similarity, and hybrid (lexical + semantic) search using DenseVector<T> vector parameters (with implicit conversion from T[] and ReadOnlyMemory<T>) and the Fork / Fuse extensions. Use DenseVector<float> for element_type: "float" and DenseVector<byte> for both element_type: "byte" and element_type: "bit". Document metadata fields are exposed via the MetadataField flags enum and the EsqlMetadata static marker class:

var queryVec = new float[] { 0.12f, -0.03f, 0.98f /* ... */ };

await client.CreateQuery<Book>()
    .From("books", MetadataField.Id | MetadataField.Index | MetadataField.Score)
    .Fork(
        b => b.Where(x => EsqlFunctions.Match(x.Title, "shakespeare")).Take(50),
        b => b.Where(x => EsqlFunctions.Knn(x.TitleVec, queryVec)).Take(50))
    .Fuse(method: FuseMethod.Linear, normalizer: ScoreNormalizer.MinMax, weights: [0.7, 0.3])
    .OrderByDescending(_ => EsqlMetadata.Score)
    .Take(10)
    .ToListAsync();

See the vector and hybrid search docs for the full surface.

Async Query Execution

Submit long-running queries asynchronously with ToAsyncQueryAsync(). Poll for completion, stream results, and auto-cleanup on dispose:

await using var asyncQuery = await client.CreateQuery<LogEntry>()
    .From("logs-*")
    .Where(l => l.Level == "ERROR")
    .ToAsyncQueryAsync(new EsqlAsyncQueryOptions
    {
        WaitForCompletionTimeout = TimeSpan.FromSeconds(5),
        KeepAlive = TimeSpan.FromMinutes(10)
    });

var results = await asyncQuery.ToListAsync();

Raw Response Formats

Get the raw, server-formatted bytes (Csv, Tsv, Txt, Json, Arrow, Smile, Cbor, Yaml) instead of materialised POCO rows. Useful when piping to a file, feeding columnar consumers, or doing zero-copy decoding:

using var stream = await client.CreateQuery<LogEntry>()
    .From("logs-*")
    .Where(l => l.Level == "ERROR")
    .ToStreamAsync(EsqlFormat.Csv);

await stream.CopyToAsync(File.Create("errors.csv"));

Server-side async with a non-JSON format. The query is best-effort DELETEd on disposal:

await using var q = await client.CreateQuery<LogEntry>()
    .From("logs-*")
    .Where(l => l.Level == "ERROR")
    .ToAsyncQueryAsync(EsqlFormat.Arrow);

await q.WaitForCompletionAsync();

using var stream = q.GetResponseStream();
using var reader = new ArrowStreamReader(stream); // Apache.Arrow.Ipc — separate NuGet
while (await reader.ReadNextRecordBatchAsync() is { } batch)
    Console.WriteLine($"Batch: {batch.Length} rows, {batch.ColumnCount} cols");

A ToPipeReaderAsync(format) overload is available on .NET 10+ for zero-copy consumers.

COMPLETION (LLM Inference)

Run LLM inference directly in ES|QL pipelines or as standalone prompts using preconfigured inference endpoints:

// RAG pipeline
client.CreateQuery<LogEntry>()
    .From("logs-*")
    .Where(l => l.Level == "ERROR")
    .Completion(l => l.Message, InferenceEndpoints.OpenAi.Gpt41, column: "analysis")

// Standalone
client.CreateQuery<Result>()
    .Row(() => new { prompt = "Summarize Elasticsearch" })
    .Completion("prompt", InferenceEndpoints.Anthropic.Claude46Opus, column: "answer")

LOOKUP JOIN

Correlate data across indices with LeftJoin or LookupJoin:

query.LookupJoin<Order, Customer, string, OrderWithCustomer>(
    "customers",
    o => o.CustomerId,
    c => c.Id,
    (o, c) => new OrderWithCustomer { Order = o, CustomerName = c.Name }
)

Per-Query Options

Override client defaults on individual queries with .WithOptions():

var results = await client.CreateQuery<LogEntry>()
    .WithOptions(new EsqlQueryOptions { TimeZone = "America/New_York", Locale = "en-US" })
    .From("logs-*")
    .Where(l => l.Level == "ERROR")
    .ToListAsync();

Named Parameters

Extract captured variables as named ?param placeholders instead of inlining them:

var minStatus = 400;
var esql = query
    .Where(l => l.StatusCode >= minStatus)
    .ToEsqlString(inlineParameters: false);
// WHERE statusCode >= ?minStatus

Streaming Results

Stream query results with IAsyncEnumerable<T>:

await foreach (var entry in client.QueryAsync<LogEntry>(q =>
    q.From("logs-*").Where(l => l.Level == "ERROR").Take(100)))
{
    Console.WriteLine(entry.Message);
}

Native AOT

The entire pipeline is AOT compatible. Pass a source-generated JsonSerializerContext and field names, serialization, and queries all derive from the same compile-time source of truth with zero reflection at runtime.

var provider = new EsqlQueryProvider(MyJsonContext.Default);
var query = new EsqlQueryable<LogEntry>(provider);

Multi-Target

Supports netstandard2.0, net8.0, and net10.0 with polyfills for older targets.

Building

dotnet build esql-dotnet.slnx

# Run tests
dotnet test --project tests/Elastic.Esql.Tests/Elastic.Esql.Tests.csproj

# Or run the TUnit runner directly
dotnet run --project tests/Elastic.Esql.Tests

# Or use the build script
./build.sh

Examples

The examples/ directory contains working applications:

  • esql-aot-smoketest -- Native AOT smoke test for query translation and execution

License

Apache 2.0. See LICENSE for details.

About

ESQL dotnet client (LINQ provider for ESQL)

Resources

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages