diff --git a/packages/google_cloud_firestore/CHANGELOG.md b/packages/google_cloud_firestore/CHANGELOG.md index 0a9e4f2b..3bdf99aa 100644 --- a/packages/google_cloud_firestore/CHANGELOG.md +++ b/packages/google_cloud_firestore/CHANGELOG.md @@ -3,6 +3,19 @@ - Added `CollectionReference.listDocumentsPages()`, which lists a collection's documents, including missing documents, one `DocumentReferencePage` at a time. It takes an optional `pageSize` and a `pageToken` to resume from a previous page's `nextPageToken`, so large collections can be walked without holding every reference in memory. - Fixed `CollectionReference.listDocuments()` returning only the first page of results. It now follows `nextPageToken` until the collection is exhausted, matching the Node Admin SDK, so large collections and missing documents past the first page are no longer silently dropped. - Fixed `DocumentReference.listCollections()` and `Firestore.listCollections()` returning only the first page of collection IDs. They now follow `nextPageToken` until every collection has been returned. +- **Breaking:** `PipelineFunctions.join` and `PipelineFunctions.split` now require their delimiter, matching `PipelineExpression` and the Node Admin SDK. A one-argument call was always rejected by the backend with `INVALID_ARGUMENT`. +- **Breaking:** `PipelineFunctions.mapRemove` and `PipelineExpression.mapRemove` take a single key (a `String` or an expression) and send `map_remove(map, key)`, like the Node Admin SDK, instead of an `Iterable` of keys. Chain `.mapRemove('a').mapRemove('b')` to remove several keys. +- Fixed `PipelineFunctions.arrayMaximum`, `arrayMaximumN`, `arrayMinimum`, `arrayMinimumN` and `arraySum` sending function names (`array_maximum`, ...) that the backend rejects. They now send `maximum`, `maximum_n`, `minimum`, `minimum_n` and `sum`, like their `PipelineExpression` counterparts. +- Fixed `Pipeline.addFields` sending one `alias` function per field instead of a single map keyed by alias, which the backend rejected. +- Fixed `PipelineSource.collectionGroup()`, and `createFrom()` on a collection group query, omitting the leading root ancestor argument of the `collection_group` stage. +- Fixed `PipelineValueType.double` sending `'double'` instead of `'float64'`, which made `isType` fail. Added the `int32`, `decimal128`, `maxKey`, `minKey`, `objectId` and `regex` value types. +- Fixed `cosineDistance`, `dotProduct`, `euclideanDistance` and `Pipeline.findNearest` sending a plain list of numbers as an array instead of a vector. +- Fixed Pipeline functions such as `equalAny`, `notEqualAny`, `arrayContainsAll`, `arrayContainsAny` and `mapMerge` failing when a `List` or `Map` argument holds expressions. Collection arguments are now sent as `array(...)` / `map(...)` functions, like the Node Admin SDK, and the static and fluent forms encode identically. +- Fixed `PipelineResult.get` ignoring nested paths. It now accepts a `String` or a `FieldPath` and resolves dot-separated paths such as `'metadata.lang'`, like `DocumentSnapshot.get`. +- `Pipeline.select`, `addFields`, `aggregate` and `distinct` now throw an `ArgumentError` on a duplicate field name or alias instead of silently keeping the last one, like the Node Admin SDK. +- `PipelineExpression.substring` and `substringLiteral` now take `(position, [length])` like `PipelineFunctions.substring`: the second argument is a length, not an end index, and can be omitted. +- `PipelineExpression.round` now accepts the optional `decimalPlaces` argument. +- Exported the top-level `variable()` Pipeline helper. ## 0.5.5 diff --git a/packages/google_cloud_firestore/README.md b/packages/google_cloud_firestore/README.md index e1476eec..12182e5a 100644 --- a/packages/google_cloud_firestore/README.md +++ b/packages/google_cloud_firestore/README.md @@ -307,6 +307,10 @@ proportion between 0 and 1. Exactly one of the two must be given. ) ``` +`queryVector` takes a `VectorValue`, a plain list of numbers, or an +expression. So does the second argument of `cosineDistance`, `dotProduct` and +`euclideanDistance`; either way it is sent as a vector. + **`rawStage`** — escape hatch for preview stages this SDK does not yet wrap. `search` is a thin wrapper over the same mechanism. @@ -343,8 +347,16 @@ field('title').toUpperCase(); PipelineFunctions.toUpper('title'); ``` -`Expression.field` / `Expression.constant` are aliases for the top-level -`field` / `constant`, for callers who prefer a namespaced entry point. +`Expression.field` / `Expression.constant` / `Expression.variable` are aliases +for the top-level `field` / `constant` / `variable`, for callers who prefer a +namespaced entry point. + +`variable` references a name bound by the enclosing expression, such as the +element alias of `arrayFilter` / `arrayTransform`: + +```dart +field('tags').arrayFilter('tag', variable('tag').notEqual('draft')); +``` **Field arguments vs value arguments.** A `String` in a *field* position means a field reference; in a *value* position it stays a string literal. The field @@ -358,6 +370,15 @@ PipelineFunctions.startsWith('title', 'Harry'); PipelineFunctions.startsWith('title', field('prefix')); ``` +A `List` or `Map` argument may mix literals and expressions. It is sent as an +`array(...)` / `map(...)` function so the backend evaluates the expressions +inside it; wrap it in `constant` to send a literal value instead: + +```dart +PipelineFunctions.equalAny('rating', [field('score'), 5]); +field('metadata').mapMerge([{'reviewer': field('editor')}]); +``` + Selected expressions must be aliased with `as` (or `alias`): ```dart @@ -375,7 +396,7 @@ available on `PipelineFunctions`; most also exist as a fluent method. | Logical | `and`, `or`, `xor`, `nor`, `not`, `conditional`, `ifNull`, `coalesce`, `switchOn`, `equalAny`, `notEqualAny` | `and`, `or`, `xor`, `nor`, `not`, `conditional`, `if_null`, `coalesce`, `switch_on`, `equal_any`, `not_equal_any` | | Aggregate | `count`, `countAll`, `countIf`, `countDistinct`, `sum`, `average`, `minimum`, `maximum`, `first`, `last`, `arrayAgg`, `arrayAggDistinct` | `count`, `count_if`, `count_distinct`, `sum`, `average`, `minimum`, `maximum`, `first`, `last`, `array_agg`, `array_agg_distinct` | | Arithmetic | `add`, `subtract`, `multiply`, `divide`, `mod`, `abs`, `ceil`, `floor`, `round`, `trunc`, `sqrt`, `pow`, `exp`, `ln`, `log`, `log10`, `rand`, `logicalMinimum`, `logicalMaximum` | `add`, `subtract`, `multiply`, `divide`, `mod`, `abs`, `ceil`, `floor`, `round`, `trunc`, `sqrt`, `pow`, `exp`, `ln`, `log`, `log10`, `rand`, `minimum`, `maximum` | -| Array | `array`, `arrayConcat`, `arrayContains`, `arrayContainsAll`, `arrayContainsAny`, `arrayFilter`, `arrayGet`, `arrayLength`, `arrayReverse`, `arrayFirst`, `arrayFirstN`, `arrayLast`, `arrayLastN`, `arrayIndexOf`, `arrayIndexOfAll`, `arrayLastIndexOf`, `arraySlice`, `arrayTransform`, `arrayMaximum`, `arrayMaximumN`, `arrayMinimum`, `arrayMinimumN`, `arraySum`, `maximumN`, `minimumN`, `join` | `array`, `array_concat`, `array_contains`, `array_contains_all`, `array_contains_any`, `array_filter`, `array_get`, `array_length`, `array_reverse`, `array_first`, `array_first_n`, `array_last`, `array_last_n`, `array_index_of`, `array_index_of_all`, `array_index_of`, `array_slice`, `array_transform`, `array_maximum`, `array_maximum_n`, `array_minimum`, `array_minimum_n`, `array_sum`, `maximum_n`, `minimum_n`, `join` | +| Array | `array`, `arrayConcat`, `arrayContains`, `arrayContainsAll`, `arrayContainsAny`, `arrayFilter`, `arrayGet`, `arrayLength`, `arrayReverse`, `arrayFirst`, `arrayFirstN`, `arrayLast`, `arrayLastN`, `arrayIndexOf`, `arrayIndexOfAll`, `arrayLastIndexOf`, `arraySlice`, `arrayTransform`, `arrayMaximum`, `arrayMaximumN`, `arrayMinimum`, `arrayMinimumN`, `arraySum`, `maximumN`, `minimumN`, `join` | `array`, `array_concat`, `array_contains`, `array_contains_all`, `array_contains_any`, `array_filter`, `array_get`, `array_length`, `array_reverse`, `array_first`, `array_first_n`, `array_last`, `array_last_n`, `array_index_of`, `array_index_of_all`, `array_index_of`, `array_slice`, `array_transform`, `maximum`, `maximum_n`, `minimum`, `minimum_n`, `sum`, `maximum_n`, `minimum_n`, `join` | | String | `byteLength`, `charLength`, `startsWith`, `endsWith`, `like`, `regexContains`, `regexMatch`, `regexFind`, `regexFindAll`, `stringConcat`, `stringContains`, `stringIndexOf`, `toUpper`, `toLower`, `substring`, `stringReverse`, `stringRepeat`, `stringReplaceAll`, `stringReplaceOne`, `trim`, `ltrim`, `rtrim`, `split` | `byte_length`, `char_length`, `starts_with`, `ends_with`, `like`, `regex_contains`, `regex_match`, `regex_find`, `regex_find_all`, `string_concat`, `string_contains`, `string_index_of`, `to_upper`, `to_lower`, `substring`, `string_reverse`, `string_repeat`, `string_replace_all`, `string_replace_one`, `trim`, `ltrim`, `rtrim`, `split` | | Generic | `length`, `reverse`, `concat` | `length`, `reverse`, `concat` | | Map | `map`, `mapGet`, `getField`, `mapSet`, `mapRemove`, `mapMerge`, `mapKeys`, `mapValues`, `mapEntries` | `map`, `map_get`, `get_field`, `map_set`, `map_remove`, `map_merge`, `map_keys`, `map_values`, `map_entries` | @@ -393,7 +414,8 @@ maps; `charLength`/`stringReverse`/`stringConcat` and `logicalMaximum` to compare several operands element-wise. Anything not yet wrapped is reachable via `PipelineFunctions.raw` or -`pipelineFunction`: +`pipelineFunction`. Their arguments are sent as-is, so build a collection that +holds expressions with `PipelineFunctions.array` / `PipelineFunctions.map`: ```dart PipelineFunctions.raw('some_new_function', [field('x'), 42]); @@ -457,10 +479,11 @@ metadata, its identity: ```dart for (final result in snapshot.results) { - print(result.data()); // all decoded fields - print(result.get('title')); // a single field - print(result.ref?.path); // null when a projection dropped metadata - print(result.id); // the document ID, or null + print(result.data()); // all decoded fields + print(result.get('title')); // a single field + print(result.get('metadata.lang')); // a nested field (or pass a FieldPath) + print(result.ref?.path); // null when a projection dropped metadata + print(result.id); // the document ID, or null print(result.createTime); print(result.updateTime); } diff --git a/packages/google_cloud_firestore/lib/google_cloud_firestore.dart b/packages/google_cloud_firestore/lib/google_cloud_firestore.dart index 1c3d2526..9b49eed8 100644 --- a/packages/google_cloud_firestore/lib/google_cloud_firestore.dart +++ b/packages/google_cloud_firestore/lib/google_cloud_firestore.dart @@ -109,7 +109,8 @@ export 'src/firestore.dart' or, pipelineFunction, score, - sum; + sum, + variable; export 'src/firestore_exception.dart' show FirestoreClientErrorCode, FirestoreException; export 'src/status_code.dart' show StatusCode; diff --git a/packages/google_cloud_firestore/lib/src/pipeline.dart b/packages/google_cloud_firestore/lib/src/pipeline.dart index 2a94a822..fc8d7d65 100644 --- a/packages/google_cloud_firestore/lib/src/pipeline.dart +++ b/packages/google_cloud_firestore/lib/src/pipeline.dart @@ -72,6 +72,10 @@ typedef Selectable = PipelineExpression; typedef PipelineAggregateFunction = PipelineExpression; /// Firestore Pipeline backend value types used with [PipelineExpression.isType]. +/// +/// Each member encodes as the type name the backend expects, matching the +/// Node.js SDK's `Type` union. [PipelineExpression.isType] also accepts a raw +/// type name string for backend types not listed here. enum PipelineValueType { /// Null values. nullValue('null'), @@ -82,11 +86,17 @@ enum PipelineValueType { /// Any numeric value. number('number'), - /// Integer numeric values. + /// 32-bit integer values. + int32('int32'), + + /// 64-bit integer values. int64('int64'), - /// Double numeric values. - double('double'), + /// 64-bit floating point values, sent to the backend as `float64`. + double('float64'), + + /// 128-bit decimal values. + decimal128('decimal128'), /// Timestamp values. timestamp('timestamp'), @@ -110,7 +120,19 @@ enum PipelineValueType { map('map'), /// Vector values. - vector('vector'); + vector('vector'), + + /// Max key values. + maxKey('max_key'), + + /// Min key values. + minKey('min_key'), + + /// Object ID values. + objectId('object_id'), + + /// Regular expression values. + regex('regex'); const PipelineValueType(this.value); @@ -179,6 +201,10 @@ final class PipelineExplainOptions { } /// Creates a raw Pipeline function expression. +/// +/// [args] are sent as-is: a [List] or [Map] is a literal value, which cannot +/// hold expressions. Build those with [PipelineFunctions.array] or +/// [PipelineFunctions.map]. PipelineExpression pipelineFunction( String name, Iterable args, { @@ -222,7 +248,7 @@ PipelineBooleanExpression _comparison( Object? left, Object? right, ) { - return _PipelineBooleanExpression(name, [_fieldOrExpression(left), right]); + return PipelineFunctions._bool(name, [_fieldOrExpression(left), right]); } /// Interprets a [String] in a field position as a field reference. @@ -246,6 +272,61 @@ List _fieldOrExpressionFirst(Iterable values) { return list; } +/// Interprets a list of numbers in a vector position as a [VectorValue]. +/// +/// Mirrors the Node SDK's `vectorToExpr`: the vector distance functions and +/// the `find_nearest` stage take a [VectorValue], an expression, or a plain +/// list of numbers. Left to [_encodePipelineValue], a list would encode as an +/// `ARRAY`, which the backend rejects where it expects a `Vector`. +Object? _vectorOrExpression(Object? value, String name) { + if (value is! Iterable) return value; + return FieldValue.vector([ + for (final element in value) + switch (element) { + num() => element.toDouble(), + _ => throw ArgumentError.value( + value, + name, + 'Expected a VectorValue, a list of numbers, or an expression.', + ), + }, + ]); +} + +/// Converts a Dart collection in a value position to the function that builds +/// it. +/// +/// Mirrors the Node SDK's `valueToDefaultExpr`: an [Iterable] becomes an +/// `array(...)` function and a [Map] a `map(...)` function, with their entries +/// converted the same way. The backend rejects expressions nested inside a +/// literal array or map value, so this is what lets a collection such as +/// `[field('a'), 1]` hold expressions. Other values are returned unchanged; +/// wrap a collection in [constant] to send it as a literal value instead. +Object? _valueToDefaultExpr(Object? value) { + return switch (value) { + Uint8List() => value, + Iterable() => PipelineFunctions.array(value), + Map() => PipelineFunctions.map([ + for (final entry in value.entries) ...[entry.key.toString(), entry.value], + ]), + _ => value, + }; +} + +/// Whether [value] is, or holds, an expression the backend has to evaluate. +/// +/// Constants already encode to plain values, so they don't count. +bool _containsExpression(Object? value) { + return switch (value) { + _PipelineConstant() || _PipelineProtoValue() => false, + PipelineExpression() || Pipeline() => true, + Uint8List() => false, + Iterable() => value.any(_containsExpression), + Map() => value.values.any(_containsExpression), + _ => false, + }; +} + /// Creates a logical AND expression. PipelineBooleanExpression and(Iterable expressions) { return _PipelineBooleanExpression('and', expressions.toList()); @@ -277,18 +358,46 @@ PipelineBooleanExpression documentMatches(Object? rquery) { /// These helpers encode to the backend function names documented in the /// Firestore Pipeline functions reference. String arguments are encoded as /// string literals; use [field] when you want to reference a document field. +/// +/// A [List] or [Map] argument may hold expressions, as in `[field('a'), 1]`: +/// it is sent as an [array] or [map] function so the backend evaluates them. +/// Wrap a collection in [constant] to send it as a literal value instead. abstract final class PipelineFunctions { static PipelineExpression _expr(String name, Iterable args) { - return pipelineFunction(name, args); + return pipelineFunction(name, args.map(_valueToDefaultExpr)); } static PipelineBooleanExpression _bool(String name, Iterable args) { - return _PipelineBooleanExpression(name, args.toList()); + return _PipelineBooleanExpression(name, [...args.map(_valueToDefaultExpr)]); + } + + /// Tests [target] against the values in [searchSpace]. + /// + /// Like the Node SDK, a list of plain values is sent as a literal array + /// value. A list holding expressions is built with [array] instead, since the + /// backend rejects expressions nested inside a literal array value. + static PipelineBooleanExpression _searchSpaceFunction( + String name, + Object? target, + Object? searchSpace, + ) { + final values = switch (searchSpace) { + Iterable() when !_containsExpression(searchSpace) => searchSpace.toList(), + _ => _valueToDefaultExpr(searchSpace), + }; + return _PipelineBooleanExpression(name, [ + _valueToDefaultExpr(_fieldOrExpression(target)), + values, + ]); } /// Creates a raw Pipeline function expression. + /// + /// Unlike the other helpers, [args] are sent as-is: a [List] or [Map] is a + /// literal value, which cannot hold expressions. Build those with [array] + /// or [map]. static PipelineExpression raw(String name, Iterable args) { - return _expr(name, args); + return pipelineFunction(name, args); } /// COUNT aggregate function. @@ -409,6 +518,9 @@ abstract final class PipelineFunctions { } /// ROUND arithmetic function. + /// + /// Rounds to [decimalPlaces] decimal places when given, otherwise to the + /// nearest integer. static PipelineExpression round(Object? fieldName, [Object? decimalPlaces]) { return _expr('round', [ _fieldOrExpression(fieldName), @@ -417,6 +529,9 @@ abstract final class PipelineFunctions { } /// TRUNC arithmetic function. + /// + /// Truncates to [decimalPlaces] decimal places when given, otherwise to an + /// integer. static PipelineExpression trunc(Object? fieldName, [Object? decimalPlaces]) { return _expr('trunc', [ _fieldOrExpression(fieldName), @@ -458,6 +573,8 @@ abstract final class PipelineFunctions { static PipelineExpression rand() => _expr('rand', const []); /// ARRAY construction function. + /// + /// [values] may mix literals and expressions, as in `[field('a'), 1]`. static PipelineExpression array(Iterable values) { return _expr('array', values); } @@ -473,25 +590,23 @@ abstract final class PipelineFunctions { } /// ARRAY_CONTAINS_ALL function. + /// + /// [searchValues] is a list of values or an array expression. static PipelineBooleanExpression arrayContainsAll( Object? array, Object? searchValues, ) { - return _bool('array_contains_all', [ - _fieldOrExpression(array), - searchValues, - ]); + return _searchSpaceFunction('array_contains_all', array, searchValues); } /// ARRAY_CONTAINS_ANY function. + /// + /// [searchValues] is a list of values or an array expression. static PipelineBooleanExpression arrayContainsAny( Object? array, Object? searchValues, ) { - return _bool('array_contains_any', [ - _fieldOrExpression(array), - searchValues, - ]); + return _searchSpaceFunction('array_contains_any', array, searchValues); } /// ARRAY_FILTER function. @@ -532,30 +647,30 @@ abstract final class PipelineFunctions { return _expr('array_first_n', [_fieldOrExpression(array), n]); } - /// ARRAY_MAXIMUM function. - static PipelineExpression arrayMaximum(Object? array) { - return _expr('array_maximum', [_fieldOrExpression(array)]); - } + /// MAXIMUM function over the elements of [array]. + /// + /// The backend has no `array_maximum`; this emits `maximum`, like [maximum]. + static PipelineExpression arrayMaximum(Object? array) => maximum(array); - /// ARRAY_MAXIMUM_N function. + /// MAXIMUM_N function over the elements of [array]; same as [maximumN]. static PipelineExpression arrayMaximumN(Object? array, Object? n) { - return _expr('array_maximum_n', [_fieldOrExpression(array), n]); + return maximumN(array, n); } - /// ARRAY_MINIMUM function. - static PipelineExpression arrayMinimum(Object? array) { - return _expr('array_minimum', [_fieldOrExpression(array)]); - } + /// MINIMUM function over the elements of [array]. + /// + /// The backend has no `array_minimum`; this emits `minimum`, like [minimum]. + static PipelineExpression arrayMinimum(Object? array) => minimum(array); - /// ARRAY_MINIMUM_N function. + /// MINIMUM_N function over the elements of [array]; same as [minimumN]. static PipelineExpression arrayMinimumN(Object? array, Object? n) { - return _expr('array_minimum_n', [_fieldOrExpression(array), n]); + return minimumN(array, n); } - /// ARRAY_SUM function. - static PipelineExpression arraySum(Object? array) { - return _expr('array_sum', [_fieldOrExpression(array)]); - } + /// SUM function over the elements of [array]. + /// + /// The backend has no `array_sum`; this emits `sum`, like [sum]. + static PipelineExpression arraySum(Object? array) => sum(array); /// COUNT function over every input, without inspecting a field. static PipelineAggregateFunction countAll() => _expr('count', const []); @@ -622,12 +737,10 @@ abstract final class PipelineFunctions { return _expr('minimum_n', [_fieldOrExpression(array), n]); } - /// JOIN function. - static PipelineExpression join(Object? array, [Object? separator]) { - return _expr('join', [ - _fieldOrExpression(array), - ..._optionalArg(separator), - ]); + /// JOIN function: joins the elements of [array] into a string separated by + /// [delimiter]. + static PipelineExpression join(Object? array, Object? delimiter) { + return _expr('join', [_fieldOrExpression(array), delimiter]); } /// EQUAL comparison function. @@ -784,22 +897,28 @@ abstract final class PipelineFunctions { } /// EQUAL_ANY logical function. + /// + /// [searchSpace] is a list of values or an array expression. static PipelineBooleanExpression equalAny( Object? fieldName, Object? searchSpace, ) { - return _bool('equal_any', [_fieldOrExpression(fieldName), searchSpace]); + return _searchSpaceFunction('equal_any', fieldName, searchSpace); } /// NOT_EQUAL_ANY logical function. + /// + /// [searchSpace] is a list of values or an array expression. static PipelineBooleanExpression notEqualAny( Object? fieldName, Object? searchSpace, ) { - return _bool('not_equal_any', [_fieldOrExpression(fieldName), searchSpace]); + return _searchSpaceFunction('not_equal_any', fieldName, searchSpace); } /// MAP construction function. + /// + /// [keyValues] alternates keys and values; values may be expressions. static PipelineExpression map(Iterable keyValues) { return _expr('map', keyValues); } @@ -822,8 +941,22 @@ abstract final class PipelineFunctions { } /// MAP_REMOVE function. - static PipelineExpression mapRemove(Object? map, Iterable keys) { - return _expr('map_remove', [_fieldOrExpression(map), ...keys]); + /// + /// Removes [key] from [map]. A [String] [map] names a field; [key] is the + /// key itself (a [String] literal) or an expression producing it. Each call + /// removes exactly one key, so chain calls to remove several: + /// `mapRemove(mapRemove('address', 'city'), 'zip')`. + /// + /// Throws an [ArgumentError] when [key] is an [Iterable]. + static PipelineExpression mapRemove(Object? map, Object? key) { + if (key is Iterable) { + throw ArgumentError.value( + key, + 'key', + 'Must be a single key. Chain mapRemove calls to remove several keys.', + ); + } + return _expr('map_remove', [_fieldOrExpression(map), key]); } /// MAP_MERGE function. @@ -967,14 +1100,18 @@ abstract final class PipelineFunctions { } /// SUBSTRING string function. + /// + /// Returns [length] characters (or bytes, for a bytes value) of [fieldName] + /// starting at index [position]. [length] is a count, not an end index; when + /// omitted the substring runs to the end of the input. static PipelineExpression substring( Object? fieldName, - Object? offset, [ + Object? position, [ Object? length, ]) { return _expr('substring', [ _fieldOrExpression(fieldName), - offset, + position, ..._optionalArg(length), ]); } @@ -1040,11 +1177,11 @@ abstract final class PipelineFunctions { } /// SPLIT string function. - static PipelineExpression split(Object? fieldName, [Object? delimiter]) { - return _expr('split', [ - _fieldOrExpression(fieldName), - ..._optionalArg(delimiter), - ]); + /// + /// Splits [fieldName] on [delimiter]. The delimiter is required, as in the + /// Node SDK; a string [delimiter] is sent as a literal, not a field. + static PipelineExpression split(Object? fieldName, Object? delimiter) { + return _expr('split', [_fieldOrExpression(fieldName), delimiter]); } /// CURRENT_TIMESTAMP function. @@ -1158,18 +1295,33 @@ abstract final class PipelineFunctions { } /// COSINE_DISTANCE vector function. + /// + /// [right] is a [VectorValue], a list of numbers, or an expression. static PipelineExpression cosineDistance(Object? left, Object? right) { - return _expr('cosine_distance', [_fieldOrExpression(left), right]); + return _expr('cosine_distance', [ + _fieldOrExpression(left), + _vectorOrExpression(right, 'right'), + ]); } /// DOT_PRODUCT vector function. + /// + /// [right] is a [VectorValue], a list of numbers, or an expression. static PipelineExpression dotProduct(Object? left, Object? right) { - return _expr('dot_product', [_fieldOrExpression(left), right]); + return _expr('dot_product', [ + _fieldOrExpression(left), + _vectorOrExpression(right, 'right'), + ]); } /// EUCLIDEAN_DISTANCE vector function. + /// + /// [right] is a [VectorValue], a list of numbers, or an expression. static PipelineExpression euclideanDistance(Object? left, Object? right) { - return _expr('euclidean_distance', [_fieldOrExpression(left), right]); + return _expr('euclidean_distance', [ + _fieldOrExpression(left), + _vectorOrExpression(right, 'right'), + ]); } /// VECTOR_LENGTH vector function. @@ -1228,7 +1380,13 @@ final class PipelineSource { 'Invalid collectionId "$collectionId". Collection IDs must not contain "/".', ); } - return _start('collection_group', [collectionId]); + // The backend stage is `collection_group(ancestor, collection_id)`. An + // empty reference names the database root as the ancestor, matching the + // Node SDK's `CollectionGroupSource`. + return _start('collection_group', [ + _PipelineProtoValue(firestore_v1.Value(referenceValue: '')), + collectionId, + ]); } /// Starts a Pipeline over every document in the database. @@ -1334,16 +1492,30 @@ final class Pipeline { /// /// Entries may be [String] field names, [PipelineField] references, /// [PipelineExpression] instances, or [PipelineAliasedExpression] values. + /// + /// Throws an [ArgumentError] when two [selections] land on the same field + /// name or alias. Pipeline select(Iterable selections) { return rawStage('select', [_projectionMap(selections)]); } /// Adds or overwrites fields on the inputs. + /// + /// Each expression is written to the field named by its alias, replacing + /// any existing value. Like [select], the fields are sent to the backend as + /// a single map keyed by alias. + /// + /// Throws an [ArgumentError] when two [fields] share an alias. Pipeline addFields(Iterable fields) { - return rawStage('add_fields', fields); + return rawStage('add_fields', [ + _projectionMap(fields, argumentName: 'fields'), + ]); } /// Aggregates inputs using aliased aggregate expressions. + /// + /// Throws an [ArgumentError] when two [accumulators], or two [groups], land + /// on the same field name or alias. Pipeline aggregate( Iterable accumulators, { Iterable groups = const [], @@ -1357,8 +1529,8 @@ final class Pipeline { ); } return rawStage('aggregate', [ - _projectionMap(values), - _projectionMap(groups), + _projectionMap(values, argumentName: 'accumulators'), + _projectionMap(groups, argumentName: 'groups'), ]); } @@ -1366,12 +1538,17 @@ final class Pipeline { /// /// Entries may be [String] field names, [PipelineField] references, or /// [PipelineAliasedExpression] values. + /// + /// Throws an [ArgumentError] when two [groups] land on the same field name + /// or alias. Pipeline distinct(Iterable groups) { final values = groups.toList(); if (values.isEmpty) { throw ArgumentError.value(groups, 'groups', 'Must not be empty.'); } - return rawStage('distinct', [_projectionMap(values)]); + return rawStage('distinct', [ + _projectionMap(values, argumentName: 'groups'), + ]); } /// Removes fields from the inputs. @@ -1475,6 +1652,8 @@ final class Pipeline { } /// Performs vector nearest-neighbor search. + /// + /// [queryVector] is a [VectorValue], a list of numbers, or an expression. Pipeline findNearest({ required Object vectorField, required Object queryVector, @@ -1487,7 +1666,7 @@ final class Pipeline { 'find_nearest', [ if (vectorField is String) field(vectorField) else vectorField, - queryVector, + _vectorOrExpression(queryVector, 'queryVector'), distanceMeasure.value.toLowerCase(), ], options: _compactOptions({ @@ -1993,8 +2172,21 @@ final class PipelineResult { /// in which case this is empty rather than `null`. DocumentData data() => _data; - /// Returns the decoded value at [fieldName], or `null` when absent. - Object? get(String fieldName) => _data[fieldName]; + /// Returns the decoded value at [field], or `null` when absent. + /// + /// [field] is a [String] or a [FieldPath], validated as in + /// [DocumentSnapshot.get]. A dot-separated string such as `'metadata.lang'` + /// reads a nested map field; use a [FieldPath] when a segment itself + /// contains a dot. Returns `null` when any segment is missing or traverses + /// a non-map value. + Object? get(Object field) { + Object? value = _data; + for (final segment in FieldPath.from(field).segments) { + if (value is! Map) return null; + value = value[segment]; + } + return value; + } /// Whether [other] refers to the same document with the same fields. /// @@ -2096,52 +2288,52 @@ sealed class PipelineExpression { /// Creates an equality expression. PipelineBooleanExpression equal(Object? other) { - return _PipelineBooleanExpression('equal', [this, other]); + return PipelineFunctions.equal(this, other); } /// Creates a not-equal expression. PipelineBooleanExpression notEqual(Object? other) { - return _PipelineBooleanExpression('not_equal', [this, other]); + return PipelineFunctions.notEqual(this, other); } /// Creates a less-than expression. PipelineBooleanExpression lessThan(Object? other) { - return _PipelineBooleanExpression('less_than', [this, other]); + return PipelineFunctions.lessThan(this, other); } /// Creates a less-than-or-equal expression. PipelineBooleanExpression lessThanOrEqual(Object? other) { - return _PipelineBooleanExpression('less_than_or_equal', [this, other]); + return PipelineFunctions.lessThanOrEqual(this, other); } /// Creates a greater-than expression. PipelineBooleanExpression greaterThan(Object? other) { - return _PipelineBooleanExpression('greater_than', [this, other]); + return PipelineFunctions.greaterThan(this, other); } /// Creates a greater-than-or-equal expression. PipelineBooleanExpression greaterThanOrEqual(Object? other) { - return _PipelineBooleanExpression('greater_than_or_equal', [this, other]); + return PipelineFunctions.greaterThanOrEqual(this, other); } /// Creates an addition expression. PipelineExpression add(Object? other) { - return _PipelineFunctionExpression('add', [this, other], const {}); + return PipelineFunctions.add(this, other); } /// Creates a subtraction expression. PipelineExpression subtract(Object? other) { - return _PipelineFunctionExpression('subtract', [this, other], const {}); + return PipelineFunctions.subtract(this, other); } /// Creates a multiplication expression. PipelineExpression multiply(Object? other) { - return _PipelineFunctionExpression('multiply', [this, other], const {}); + return PipelineFunctions.multiply(this, other); } /// Creates a division expression. PipelineExpression divide(Object? other) { - return _PipelineFunctionExpression('divide', [this, other], const {}); + return PipelineFunctions.divide(this, other); } /// Returns the absolute value of this expression. @@ -2156,14 +2348,20 @@ sealed class PipelineExpression { /// Returns the floor of this expression. PipelineExpression floor() => PipelineFunctions.floor(this); - /// Returns the rounded value of this expression. - PipelineExpression round() => PipelineFunctions.round(this); + /// Rounds this expression to [decimalPlaces] decimal places, or to the + /// nearest integer when [decimalPlaces] is omitted. + /// + /// [decimalPlaces] may be a number or an expression. + PipelineExpression round([Object? decimalPlaces]) { + return PipelineFunctions.round(this, decimalPlaces); + } - /// Truncates this expression. - PipelineExpression trunc([Object? decimals]) { - return decimals == null - ? PipelineFunctions.trunc(this) - : PipelineFunctions.raw('trunc', [this, decimals]); + /// Truncates this expression to [decimalPlaces] decimal places, or to an + /// integer when [decimalPlaces] is omitted. + /// + /// [decimalPlaces] may be a number or an expression. + PipelineExpression trunc([Object? decimalPlaces]) { + return PipelineFunctions.trunc(this, decimalPlaces); } /// Returns the square root of this expression. @@ -2253,10 +2451,7 @@ sealed class PipelineExpression { /// Checks if this array contains all [values]. PipelineBooleanExpression arrayContainsAll(Iterable values) { - return PipelineFunctions.arrayContainsAll( - this, - PipelineFunctions.array(values), - ); + return PipelineFunctions.arrayContainsAll(this, values); } /// Checks if this array contains all values from [arrayExpression]. @@ -2266,10 +2461,7 @@ sealed class PipelineExpression { /// Checks if this array contains any [values]. PipelineBooleanExpression arrayContainsAny(Iterable values) { - return PipelineFunctions.arrayContainsAny( - this, - PipelineFunctions.array(values), - ); + return PipelineFunctions.arrayContainsAny(this, values); } /// Filters this array expression. @@ -2318,18 +2510,18 @@ sealed class PipelineExpression { } /// Returns the maximum element of this array expression. - PipelineExpression arrayMaximum() => PipelineFunctions.maximum(this); + PipelineExpression arrayMaximum() => PipelineFunctions.arrayMaximum(this); /// Returns the largest [n] elements of this array expression. PipelineExpression arrayMaximumN(Object? n) => - PipelineFunctions.maximumN(this, n); + PipelineFunctions.arrayMaximumN(this, n); /// Returns the minimum element of this array expression. - PipelineExpression arrayMinimum() => PipelineFunctions.minimum(this); + PipelineExpression arrayMinimum() => PipelineFunctions.arrayMinimum(this); /// Returns the smallest [n] elements of this array expression. PipelineExpression arrayMinimumN(Object? n) => - PipelineFunctions.minimumN(this, n); + PipelineFunctions.arrayMinimumN(this, n); /// Reverses this array expression. PipelineExpression arrayReverse() => PipelineFunctions.arrayReverse(this); @@ -2342,7 +2534,7 @@ sealed class PipelineExpression { } /// Returns the sum of numeric elements in this array expression. - PipelineExpression arraySum() => PipelineFunctions.sum(this); + PipelineExpression arraySum() => PipelineFunctions.arraySum(this); /// Transforms this array expression. PipelineExpression arrayTransform(String elementAlias, Object? transform) { @@ -2443,9 +2635,15 @@ sealed class PipelineExpression { /// Returns this map expression's entries. PipelineExpression mapEntries() => PipelineFunctions.mapEntries(this); - /// Removes [keys] from this map expression. - PipelineExpression mapRemove(Iterable keys) { - return PipelineFunctions.mapRemove(this, keys); + /// Removes [key] from this map expression. + /// + /// [key] is the key itself (a [String] literal) or an expression producing + /// it. Each call removes exactly one key, so chain calls to remove several: + /// `field('address').mapRemove('city').mapRemove('zip')`. + /// + /// Throws an [ArgumentError] when [key] is an [Iterable]. + PipelineExpression mapRemove(Object? key) { + return PipelineFunctions.mapRemove(this, key); } /// Merges this map expression with [maps]. @@ -2562,14 +2760,23 @@ sealed class PipelineExpression { return stringReplaceOne(find, replacement); } - /// Extracts a substring from this string expression. - PipelineExpression substring(Object? start, Object? end) { - return PipelineFunctions.substring(this, start, end); + /// Extracts [length] characters of this string (or bytes) expression, + /// starting at index [position]. + /// + /// Unlike [String.substring], the second argument is a length, not an end + /// index: `substring(2, 3)` returns three characters starting at index 2. + /// When [length] is omitted the substring runs to the end of the input. + PipelineExpression substring(Object? position, [Object? length]) { + return PipelineFunctions.substring(this, position, length); } - /// Extracts a substring from this string expression. - PipelineExpression substringLiteral(int start, int end) { - return substring(start, end); + /// Extracts [length] characters of this string (or bytes) expression, + /// starting at literal index [position]. + /// + /// See [substring]: [length] is a count, not an end index, and when omitted + /// the substring runs to the end of the input. + PipelineExpression substringLiteral(int position, [int? length]) { + return substring(position, length); } /// Checks if this string expression starts with [prefix]. @@ -2678,16 +2885,22 @@ sealed class PipelineExpression { } /// Computes cosine distance between this vector and [other]. + /// + /// [other] is a [VectorValue], a list of numbers, or an expression. PipelineExpression cosineDistance(Object? other) { return PipelineFunctions.cosineDistance(this, other); } /// Computes dot product between this vector and [other]. + /// + /// [other] is a [VectorValue], a list of numbers, or an expression. PipelineExpression dotProduct(Object? other) { return PipelineFunctions.dotProduct(this, other); } /// Computes Euclidean distance between this vector and [other]. + /// + /// [other] is a [VectorValue], a list of numbers, or an expression. PipelineExpression euclideanDistance(Object? other) { return PipelineFunctions.euclideanDistance(this, other); } @@ -3000,8 +3213,24 @@ Map _compactOptions(Map options) { return Map.fromEntries(options.entries.where((entry) => entry.value != null)); } -Map _projectionMap(Iterable selections) { - return Map.fromEntries(selections.map(_projectionEntry)); +/// Keys each selection's expression by the field name or alias it lands on. +/// +/// Throws an [ArgumentError] on a repeated key instead of silently keeping the +/// last entry. A [String] or [PipelineField] lands on its own path, so it +/// collides with an alias of the same name, matching the Node SDK. +Map _projectionMap( + Iterable selections, { + String argumentName = 'selections', +}) { + final result = {}; + for (final selection in selections) { + final MapEntry(:key, :value) = _projectionEntry(selection); + if (result.containsKey(key)) { + throw ArgumentError("Duplicate alias or field '$key'.", argumentName); + } + result[key] = value; + } + return result; } /// Splits a selectable into the expression it computes and the alias it lands diff --git a/packages/google_cloud_firestore/test/e2e/pipeline_e2e_test.dart b/packages/google_cloud_firestore/test/e2e/pipeline_e2e_test.dart index 9b68dc44..ce052acb 100644 --- a/packages/google_cloud_firestore/test/e2e/pipeline_e2e_test.dart +++ b/packages/google_cloud_firestore/test/e2e/pipeline_e2e_test.dart @@ -174,6 +174,15 @@ void main() { expect(metadataSnapshot.results.single.createTime, isNotNull); expect(metadataSnapshot.results.single.updateTime, isNotNull); expect(metadataSnapshot.results.single.ref, isNotNull); + // get resolves nested dot-paths and FieldPaths, as in Node. + expect(metadataSnapshot.results.single.get('metadata.lang'), 'dart'); + expect( + metadataSnapshot.results.single.get( + FieldPath(const ['metadata', 'category']), + ), + 'sdk', + ); + expect(metadataSnapshot.results.single.get('metadata.missing'), isNull); }); test('executes aggregate pipeline stages', () async { @@ -194,6 +203,45 @@ void main() { expect(aggregateSnapshot.results.single.get('bookCount'), 2); }); + test('executes an addFields stage', () async { + final snapshot = await firestore + .pipeline() + .collection(_collectionPath) + .where( + _runFilter( + runId, + Expression.field('title').equal('Dart Pipelines'), + ), + ) + .addFields([ + Expression.field('rating').as('copiedRating'), + Expression.constant(true).as('annotated'), + ]) + .execute(); + + expect(snapshot.results, hasLength(1)); + final result = snapshot.results.single; + expect(result.get('copiedRating'), 5); + expect(result.get('annotated'), true); + // Existing fields are kept alongside the added ones. + expect(result.get('title'), 'Dart Pipelines'); + + // A single field must still be sent as a map, not an alias function. + final single = await firestore + .pipeline() + .collection(_collectionPath) + .where( + _runFilter( + runId, + Expression.field('title').equal('Dart Pipelines'), + ), + ) + .addFields([Expression.field('title').toUpperCase().as('upper')]) + .execute(); + + expect(single.results.single.get('upper'), 'DART PIPELINES'); + }); + test('requests explain stats via typed options', () async { final snapshot = await firestore .pipeline() @@ -243,6 +291,66 @@ void main() { ]); }); + test('filters with a search space holding expressions', () async { + // Regression: the list was sent as a literal array value, which the + // backend rejected with "Value type is not supported: + // FIELD_REFERENCE_VALUE". + final snapshot = await firestore + .pipeline() + .collection(_collectionPath) + .where( + _runFilter( + runId, + PipelineFunctions.equalAny('discount', [ + Expression.field('flags'), + 2, + ]), + ), + ) + .sort([Expression.field('price').ascending()]) + .select([Expression.field('title')]) + .execute(); + + // Book 1 matches the literal 2, book 2 its own `flags` (3). + expect(snapshot.results.map((result) => result.get('title')), [ + 'Dart Pipelines', + 'Firestore Admin', + ]); + }); + + test('executes a collection group source stage', () async { + final snapshot = await firestore + .pipeline() + .collectionGroup(_collectionPath) + .where(_runFilter(runId, Expression.field('active').equal(true))) + .sort([Expression.field('price').ascending()]) + .select([Expression.field('title')]) + .execute(); + + expect(snapshot.results.map((result) => result.get('title')), [ + 'Dart Pipelines', + 'Firestore Admin', + ]); + }); + + test('executes a pipeline created from a collection group query', () async { + final snapshot = await firestore + .pipeline() + .createFrom( + firestore + .collectionGroup(_collectionPath) + .where('runId', WhereFilter.equal, runId) + .where('active', WhereFilter.equal, true) + .orderBy('price'), + ) + .execute(); + + expect(snapshot.results.map((result) => result.get('title')), [ + 'Dart Pipelines', + 'Firestore Admin', + ]); + }); + group('function catalog', () { for (final scenario in _functionScenarios) { test(scenario.name, () async { @@ -319,6 +427,24 @@ void main() { expect(vectorSnapshot.results.single.get('title'), 'Dart Pipelines'); expect(vectorSnapshot.results.single.get('distance'), isNotNull); }); + + test('executes vector nearest-neighbor stage with a list', () async { + final vectorSnapshot = await firestore + .pipeline() + .collection(_collectionPath) + .where(Expression.field('runId').equal(runId)) + .findNearest( + vectorField: 'embedding', + queryVector: const [1.0, 0.0, 0.0], + distanceMeasure: DistanceMeasure.cosine, + limit: 1, + distanceResultField: 'distance', + ) + .execute(); + + expect(vectorSnapshot.results, hasLength(1)); + expect(vectorSnapshot.results.single.get('title'), 'Dart Pipelines'); + }); }); } @@ -399,6 +525,16 @@ final _functionScenarios = <_FunctionScenario>[ _FunctionExpectation('floor', Expression.constant(12.8).floor(), 12), _FunctionExpectation('round', Expression.constant(12.6).round(), 13), _FunctionExpectation('trunc', Expression.constant(12.8).trunc(), 12), + _FunctionExpectation( + 'roundDecimalPlaces', + Expression.constant(12.68).round(1), + closeTo(12.7, 0.0001), + ), + _FunctionExpectation( + 'truncDecimalPlaces', + Expression.constant(12.89).trunc(1), + closeTo(12.8, 0.0001), + ), _FunctionExpectation('pow', PipelineFunctions.pow(2, 3), 8), _FunctionExpectation('sqrt', Expression.constant(9).sqrt(), 3), _FunctionExpectation( @@ -537,21 +673,64 @@ final _functionScenarios = <_FunctionScenario>[ ), [3, 2, 4, 6], ), + _FunctionExpectation( + 'arrayMaximum', + Expression.field('numbers').arrayMaximum(), + 3, + ), _FunctionExpectation( 'maximumN', Expression.field('numbers').arrayMaximumN(2), [3, 3], ), + _FunctionExpectation( + 'arrayMinimum', + Expression.field('numbers').arrayMinimum(), + 1, + ), _FunctionExpectation( 'minimumN', Expression.field('numbers').arrayMinimumN(2), [1, 2], ), + _FunctionExpectation('arraySum', Expression.field('numbers').arraySum(), 9), + // The static helpers must emit the same backend names as the fluent + // forms (`maximum`, not `array_maximum`). + _FunctionExpectation( + 'staticArrayMaximum', + PipelineFunctions.arrayMaximum('numbers'), + 3, + ), + _FunctionExpectation( + 'staticArrayMaximumN', + PipelineFunctions.arrayMaximumN('numbers', 2), + [3, 3], + ), + _FunctionExpectation( + 'staticArrayMinimum', + PipelineFunctions.arrayMinimum('numbers'), + 1, + ), + _FunctionExpectation( + 'staticArrayMinimumN', + PipelineFunctions.arrayMinimumN('numbers', 2), + [1, 2], + ), + _FunctionExpectation( + 'staticArraySum', + PipelineFunctions.arraySum('numbers'), + 9, + ), _FunctionExpectation( 'join', Expression.field('words').joinLiteral('-'), 'dart-firebase', ), + _FunctionExpectation( + 'joinStatic', + PipelineFunctions.join('words', ', '), + 'dart, firebase', + ), ]), _FunctionScenario('comparison functions', [ _FunctionExpectation('equal', Expression.field('price').equal(10), true), @@ -689,9 +868,19 @@ final _functionScenarios = <_FunctionScenario>[ ), _FunctionExpectation( 'mapRemove', - Expression.field('metadata').mapRemove(['lang']), + Expression.field('metadata').mapRemove('lang'), isNot(contains('lang')), ), + _FunctionExpectation( + 'mapRemoveExpressionKey', + Expression.field('metadata').mapRemove(Expression.constant('lang')), + isNot(contains('lang')), + ), + _FunctionExpectation( + 'mapRemoveChained', + Expression.field('metadata').mapRemove('lang').mapRemove('category'), + isEmpty, + ), _FunctionExpectation( 'mapMerge', Expression.field('metadata').mapMerge([ @@ -720,6 +909,83 @@ final _functionScenarios = <_FunctionScenario>[ isA>(), ), ]), + // Collections that hold expressions must be built with array(...) / + // map(...); the backend rejects them inside a literal array or map value. + _FunctionScenario('collections holding expressions', [ + _FunctionExpectation( + 'equalAny', + PipelineFunctions.equalAny('rating', [Expression.field('score'), 5]), + true, + ), + _FunctionExpectation( + 'equalAnyMatchesExpression', + Expression.field( + 'price', + ).equalAny([Expression.field('rating').multiply(2), 3]), + true, + ), + _FunctionExpectation( + 'notEqualAny', + PipelineFunctions.notEqualAny('rating', [Expression.field('price'), 4]), + true, + ), + _FunctionExpectation( + 'arrayContainsAll', + PipelineFunctions.arrayContainsAll('tags', [ + Expression.field('metadata').mapGetLiteral('lang'), + 'firebase', + ]), + true, + ), + _FunctionExpectation( + 'arrayContainsAny', + Expression.field('tags').arrayContainsAny([ + Expression.field('metadata').mapGetLiteral('lang'), + 'missing', + ]), + true, + ), + _FunctionExpectation( + 'arrayConcat', + Expression.field('tags').arrayConcat([Expression.field('title')]), + ['dart', 'firebase', 'Dart Pipelines'], + ), + _FunctionExpectation( + 'mapMerge', + Expression.field('metadata').mapMerge([ + {'title': Expression.field('title')}, + ]), + containsPair('title', 'Dart Pipelines'), + ), + _FunctionExpectation( + 'equal', + Expression.field( + 'tags', + ).equal([Expression.field('metadata').mapGetLiteral('lang'), 'firebase']), + true, + ), + _FunctionExpectation( + 'nestedArray', + Expression.array([ + 1, + [Expression.field('price')], + ]), + [ + 1, + [10], + ], + ), + _FunctionExpectation( + 'nestedMap', + PipelineFunctions.map([ + 'nested', + {'price': Expression.field('price')}, + ]), + { + 'nested': {'price': 10}, + }, + ), + ]), _FunctionScenario('string functions', [ _FunctionExpectation( 'byteLength', @@ -788,6 +1054,17 @@ final _functionScenarios = <_FunctionScenario>[ Expression.field('title').substringLiteral(0, 4), 'Dart', ), + // The second argument is a length, not an end index. + _FunctionExpectation( + 'substringLength', + Expression.field('title').substring(5, 3), + 'Pip', + ), + _FunctionExpectation( + 'substringToEnd', + Expression.field('title').substring(5), + 'Pipelines', + ), _FunctionExpectation( 'stringReverse', PipelineFunctions.stringReverse(Expression.constant('Dart')), @@ -816,6 +1093,11 @@ final _functionScenarios = <_FunctionScenario>[ 'firebase', 'admin', ]), + _FunctionExpectation('splitStatic', PipelineFunctions.split('csv', ','), [ + 'dart', + 'firebase', + 'admin', + ]), ]), _FunctionScenario('timestamp and type functions', [ _FunctionExpectation( @@ -892,6 +1174,21 @@ final _functionScenarios = <_FunctionScenario>[ Expression.field('price').isType(PipelineValueType.number), true, ), + _FunctionExpectation( + 'isTypeInt64', + Expression.field('price').isType(PipelineValueType.int64), + true, + ), + _FunctionExpectation( + 'isTypeFloat64', + Expression.field('score').isType(PipelineValueType.double), + true, + ), + _FunctionExpectation( + 'isTypeNotFloat64', + Expression.field('price').isType(PipelineValueType.double), + false, + ), ]), _FunctionScenario('vector functions', [ _FunctionExpectation( @@ -919,6 +1216,24 @@ final _functionScenarios = <_FunctionScenario>[ 3, ), ]), + // A plain list of numbers must reach the backend as a vector, not an array. + _FunctionScenario('vector functions with list arguments', [ + _FunctionExpectation( + 'cosineDistance', + Expression.field('embedding').cosineDistance(const [1.0, 0.0, 0.0]), + closeTo(0, 0.0001), + ), + _FunctionExpectation( + 'dotProduct', + PipelineFunctions.dotProduct('embedding', const [1, 0, 0]), + closeTo(1, 0.0001), + ), + _FunctionExpectation( + 'euclideanDistance', + Expression.field('embedding').euclideanDistance(const [1.0, 0.0, 0.0]), + closeTo(0, 0.0001), + ), + ]), ]; final class _FunctionScenario { diff --git a/packages/google_cloud_firestore/test/pipeline_test.dart b/packages/google_cloud_firestore/test/pipeline_test.dart index b5430194..08d553e4 100644 --- a/packages/google_cloud_firestore/test/pipeline_test.dart +++ b/packages/google_cloud_firestore/test/pipeline_test.dart @@ -181,6 +181,65 @@ void main() { expect(snapshot.pipeline, isA()); }); + test('result get resolves nested field paths', () async { + when( + () => + mockClient.v1>(any()), + ).thenAnswer((invocation) async { + final callback = + invocation.positionalArguments.single + as Future> + Function(firestore_v1.Firestore api, String projectId); + + final api = FakeFirestore( + executePipeline: (_) { + return Stream.value( + firestore_v1.ExecutePipelineResponse( + results: [ + firestore_v1.Document( + fields: { + 'title': firestore.serializer.encodeValue('Dart')!, + 'metadata': firestore.serializer.encodeValue({ + 'lang': 'dart', + 'stats': {'pages': 42, 'note': null}, + })!, + 'a.b': firestore.serializer.encodeValue('dotted')!, + }, + ), + ], + ), + ); + }, + ); + + return callback(api, _projectId); + }); + + final snapshot = await firestore.pipeline().collection('books').execute(); + final result = snapshot.results.single; + + expect(result.get('title'), 'Dart'); + expect(result.get('metadata.lang'), 'dart'); + expect(result.get('metadata.stats.pages'), 42); + expect(result.get('metadata.stats'), {'pages': 42, 'note': null}); + expect(result.get(FieldPath(const ['metadata', 'lang'])), 'dart'); + // A FieldPath segment may contain a dot; a string is split on dots. + expect(result.get(FieldPath(const ['a.b'])), 'dotted'); + expect(result.get('a.b'), isNull); + + // Missing fields and paths through non-map values resolve to null. + expect(result.get('missing'), isNull); + expect(result.get('metadata.missing.deeper'), isNull); + expect(result.get('title.length'), isNull); + expect(result.get('metadata.stats.note'), isNull); + + // Invalid paths are rejected, as in DocumentSnapshot.get. + expect(() => result.get('metadata..lang'), throwsArgumentError); + expect(() => result.get('.metadata'), throwsArgumentError); + expect(() => result.get(''), throwsArgumentError); + expect(() => result.get(42), throwsArgumentError); + }); + test('results compare by reference and fields, not read time', () async { firestore_v1.ExecutePipelineResponse chunk({ required String path, @@ -652,6 +711,63 @@ void main() { expect(whereFunction.args[1].functionValue!.name, 'is_type'); }); + test('join always sends the array and the delimiter', () async { + firestore_v1.ExecutePipelineRequest? capturedRequest; + + when( + () => + mockClient.v1>(any()), + ).thenAnswer((invocation) async { + final callback = + invocation.positionalArguments.single + as Future> + Function(firestore_v1.Firestore api, String projectId); + + final api = FakeFirestore( + executePipeline: (firestore_v1.ExecutePipelineRequest request) { + capturedRequest = request; + return const Stream.empty(); + }, + ); + + return callback(api, _projectId); + }); + + // The backend only accepts `join(array, delimiter)`: a one-argument + // `join` is rejected with INVALID_ARGUMENT, so the delimiter is required. + await firestore.pipeline().collection('books').select([ + PipelineFunctions.join('tags', ', ').as('staticJoin'), + PipelineFunctions.join( + field('tags'), + field('separator'), + ).as('expressionJoin'), + field('tags').join(', ').as('fluentJoin'), + ]).execute(); + + final fields = capturedRequest! + .structuredPipeline! + .pipeline! + .stages[1] + .args + .single + .mapValue! + .fields; + + for (final alias in ['staticJoin', 'fluentJoin']) { + final join = fields[alias]!.functionValue!; + expect(join.name, 'join', reason: alias); + expect(join.args, hasLength(2), reason: alias); + expect(join.args[0].fieldReferenceValue, 'tags', reason: alias); + expect(join.args[1].stringValue, ', ', reason: alias); + } + + final expressionJoin = fields['expressionJoin']!.functionValue!; + expect(expressionJoin.name, 'join'); + expect(expressionJoin.args, hasLength(2)); + expect(expressionJoin.args[0].fieldReferenceValue, 'tags'); + expect(expressionJoin.args[1].fieldReferenceValue, 'separator'); + }); + test('serializes FlutterFire-style expression and stage APIs', () async { firestore_v1.ExecutePipelineRequest? capturedRequest; @@ -790,7 +906,7 @@ void main() { Expression.field('path').referenceSlice(0, 2).as('slice'), Expression.field('metadata').mapKeys().as('keys'), Expression.field('metadata').mapValues().as('values'), - Expression.field('metadata').mapRemove(['draft']).as('removed'), + Expression.field('metadata').mapRemove('draft').as('removed'), Expression.field('metadata') .mapMerge([ {'lang': 'dart'}, @@ -879,6 +995,76 @@ void main() { }, ); + test('isType encodes PipelineValueType as backend type names', () async { + firestore_v1.ExecutePipelineRequest? capturedRequest; + + when( + () => + mockClient.v1>(any()), + ).thenAnswer((invocation) async { + final callback = + invocation.positionalArguments.single + as Future> + Function(firestore_v1.Firestore api, String projectId); + + final api = FakeFirestore( + executePipeline: (firestore_v1.ExecutePipelineRequest request) { + capturedRequest = request; + return const Stream.empty(); + }, + ); + + return callback(api, _projectId); + }); + + // The names the backend accepts for `is_type` (and the Node SDK's `Type` + // union). Covers every member so a new one can't ship unchecked. + const expected = { + PipelineValueType.nullValue: 'null', + PipelineValueType.boolean: 'boolean', + PipelineValueType.number: 'number', + PipelineValueType.int32: 'int32', + PipelineValueType.int64: 'int64', + PipelineValueType.double: 'float64', + PipelineValueType.decimal128: 'decimal128', + PipelineValueType.timestamp: 'timestamp', + PipelineValueType.string: 'string', + PipelineValueType.bytes: 'bytes', + PipelineValueType.reference: 'reference', + PipelineValueType.geoPoint: 'geo_point', + PipelineValueType.array: 'array', + PipelineValueType.map: 'map', + PipelineValueType.vector: 'vector', + PipelineValueType.maxKey: 'max_key', + PipelineValueType.minKey: 'min_key', + PipelineValueType.objectId: 'object_id', + PipelineValueType.regex: 'regex', + }; + expect(expected.keys, unorderedEquals(PipelineValueType.values)); + + await firestore.pipeline().collection('books').select([ + for (final type in PipelineValueType.values) + Expression.field('value').isType(type).as(type.name), + ]).execute(); + + final fields = capturedRequest! + .structuredPipeline! + .pipeline! + .stages[1] + .args + .single + .mapValue! + .fields; + for (final MapEntry(key: type, value: wireName) in expected.entries) { + final function = fields[type.name]!.functionValue!; + expect(function.name, 'is_type'); + expect(function.args.last.stringValue, wireName, reason: type.name); + } + // Regression: `double` used to encode as 'double', which the backend + // rejects with INVALID_ARGUMENT. + expect(fields['double']!.functionValue!.args.last.stringValue, 'float64'); + }); + // Golden encodings, asserted arg-by-arg against the canonical Node SDK // stage definitions (`dev/src/pipelines/stage.ts`). These catch wire-format // drift without needing an Enterprise database. @@ -914,6 +1100,33 @@ void main() { Pipeline base() => firestore.pipeline().collection('books'); + group('collection_group', () { + test('sends the root ancestor before the collection id', () async { + await capture(firestore.pipeline().collectionGroup('books')); + + expect(stage.name, 'collection_group'); + // The backend stage is `collection_group(ancestor, collection_id)` + // and rejects a lone collection id. + expect(stage.args, hasLength(2)); + expect(stage.args[0].referenceValue, ''); + expect(stage.args[1].stringValue, 'books'); + expect(stage.options, isEmpty); + }); + + test('keeps the empty root reference on the wire', () async { + await capture(firestore.pipeline().collectionGroup('books')); + + expect(stage.args[0].toJson(), {'referenceValue': ''}); + }); + }); + + test('database sends no arguments', () async { + await capture(firestore.pipeline().database()); + + expect(stage.name, 'database'); + expect(stage.args, isEmpty); + }); + group('unnest', () { test('sends the array expression and its alias', () async { await capture(base().unnest(field('tags').as('tag'))); @@ -1001,12 +1214,212 @@ void main() { }); }); + group('add_fields', () { + test('sends a single map argument keyed by alias', () async { + await capture( + base().addFields([ + field('rating').as('copiedRating'), + constant(true).as('annotated'), + ]), + ); + + expect(stage.name, 'add_fields'); + // The backend stage takes exactly one MapValue argument; one arg per + // field is rejected with "takes [1..1] argument(s)". + expect(stage.args, hasLength(1)); + final fields = stage.args.single.mapValue!.fields; + expect(fields.keys, ['copiedRating', 'annotated']); + expect(fields['copiedRating']!.fieldReferenceValue, 'rating'); + expect(fields['annotated']!.booleanValue, isTrue); + expect(stage.options, isEmpty); + }); + + test('wraps a single field in a map, not an alias function', () async { + await capture( + base().addFields([field('title').toUpperCase().as('upper')]), + ); + + expect(stage.args, hasLength(1)); + expect(stage.args.single.functionValue, isNull); + final fields = stage.args.single.mapValue!.fields; + expect(fields.keys, ['upper']); + expect(fields['upper']!.functionValue!.name, 'to_upper'); + }); + }); + + group('substring', () { + test('sends position and length, in that order', () async { + await capture( + base().select([ + field('title').substring(2, 3).as('fluent'), + field('title').substringLiteral(2, 3).as('literal'), + PipelineFunctions.substring('title', 2, 3).as('static'), + ]), + ); + + final fields = stage.args.single.mapValue!.fields; + for (final alias in ['fluent', 'literal', 'static']) { + final function = fields[alias]!.functionValue!; + expect(function.name, 'substring', reason: alias); + expect(function.args, hasLength(3), reason: alias); + expect(function.args[0].fieldReferenceValue, 'title'); + expect(function.args[1].integerValue, 2, reason: alias); + // A length (as in Node), not an end index like String.substring. + expect(function.args[2].integerValue, 3, reason: alias); + } + }); + + test('omits the length when it is not given', () async { + await capture( + base().select([ + field('title').substring(2).as('fluent'), + field('title').substringLiteral(2).as('literal'), + PipelineFunctions.substring('title', 2).as('static'), + ]), + ); + + final fields = stage.args.single.mapValue!.fields; + for (final alias in ['fluent', 'literal', 'static']) { + final function = fields[alias]!.functionValue!; + expect(function.name, 'substring', reason: alias); + // Regression: the fluent forms required a second `end` argument, + // so "to the end of the input" could not be expressed. + expect(function.args, hasLength(2), reason: alias); + expect(function.args[1].integerValue, 2, reason: alias); + } + }); + + test('accepts expressions for position and length', () async { + await capture( + base().select([ + field( + 'title', + ).substring(field('start'), field('count')).as('fromFields'), + ]), + ); + + final function = + stage.args.single.mapValue!.fields['fromFields']!.functionValue!; + expect(function.args[1].fieldReferenceValue, 'start'); + expect(function.args[2].fieldReferenceValue, 'count'); + }); + }); + test('select and aggregate use the same projection map', () async { await capture(base().select(['title', field('rating')])); expect(stage.args, hasLength(1)); expect(stage.args.single.mapValue!.fields.keys, ['title', 'rating']); }); + + test('top-level variable() encodes a variable reference', () async { + // Uses the barrel's top-level `variable`, so dropping it from the + // public exports fails compilation here. + final kept = variable('tag').notEqual('draft'); + final aliased = Expression.variable('tag').notEqual('draft'); + await capture( + base().select([ + field('tags').arrayFilter('tag', kept).as('kept'), + field('tags').arrayFilter('tag', aliased).as('aliased'), + ]), + ); + + final fields = stage.args.single.mapValue!.fields; + final filter = fields['kept']!.functionValue!; + expect(filter.name, 'array_filter'); + expect(filter.args[0].fieldReferenceValue, 'tags'); + expect(filter.args[1].stringValue, 'tag'); + final predicate = filter.args[2].functionValue!; + expect(predicate.name, 'not_equal'); + expect(predicate.args[0].variableReferenceValue, 'tag'); + expect(predicate.args[0].fieldReferenceValue, isNull); + expect(predicate.args[1].stringValue, 'draft'); + + // `variable` and `Expression.variable` are interchangeable. + expect(fields['aliased']!.toJson(), fields['kept']!.toJson()); + }); + }); + + // Mirrors the Node SDK's `selectablesToObject` / `aliasedAggregateToMap`, + // which throw rather than let a later entry overwrite an earlier one. + group('duplicate aliases or fields', () { + Pipeline base() => firestore.pipeline().collection('books'); + + Matcher duplicateError(String key, String argumentName) { + return throwsA( + isA() + .having((e) => e.message, 'message', contains("'$key'")) + .having((e) => e.message, 'message', contains('Duplicate')) + .having((e) => e.name, 'name', argumentName), + ); + } + + test('select rejects a repeated alias', () { + expect( + () => base().select([constant(1).as('x'), constant(2).as('x')]), + duplicateError('x', 'selections'), + ); + }); + + test('select rejects a repeated field name', () { + expect( + () => base().select(['title', field('title')]), + duplicateError('title', 'selections'), + ); + }); + + test('select rejects an alias that collides with a field name', () { + expect( + () => base().select(['title', constant('x').as('title')]), + duplicateError('title', 'selections'), + ); + }); + + test('addFields rejects a repeated alias', () { + expect( + () => base().addFields([constant(1).as('x'), constant(2).as('x')]), + duplicateError('x', 'fields'), + ); + }); + + test('aggregate rejects a repeated accumulator alias', () { + expect( + () => base().aggregate([ + PipelineFunctions.countAll().as('n'), + field('pages').sum().as('n'), + ]), + duplicateError('n', 'accumulators'), + ); + }); + + test('aggregate rejects a repeated group', () { + expect( + () => base().aggregate( + [PipelineFunctions.countAll().as('n')], + groups: ['genre', PipelineFunctions.toLower('genre').as('genre')], + ), + duplicateError('genre', 'groups'), + ); + }); + + test('aggregate checks accumulators and groups separately', () { + // The Node SDK builds the two maps independently, so a group and an + // accumulator sharing a name is left for the backend to judge. + expect( + () => base().aggregate( + [PipelineFunctions.countAll().as('genre')], + groups: ['genre'], + ), + returnsNormally, + ); + }); + + test('distinct rejects a repeated group', () { + expect( + () => base().distinct(['genre', field('genre')]), + duplicateError('genre', 'groups'), + ); + }); }); group('String arguments in a field position', () { @@ -1092,6 +1505,35 @@ void main() { expect(fields['lang']!.functionValue!.args[1].stringValue, 'lang'); }); + test('split always sends the field and its delimiter', () async { + await run( + firestore.pipeline().collection('books').select([ + PipelineFunctions.split('csv', ',').as('static'), + field('csv').split(',').as('fluent'), + PipelineFunctions.split('csv', null).as('nullDelimiter'), + ]), + ); + + final fields = stages[1].args.single.mapValue!.fields; + for (final alias in ['static', 'fluent']) { + final function = fields[alias]!.functionValue!; + expect(function.name, 'split'); + expect(function.args, hasLength(2), reason: alias); + expect(function.args[0].fieldReferenceValue, 'csv', reason: alias); + expect(function.args[1].stringValue, ',', reason: alias); + } + + // Regression: the static form made the delimiter optional and dropped + // a null one, emitting a one-argument `split` the backend rejects. + // Like Node, a null delimiter is now sent as a null constant. + final nullDelimiter = fields['nullDelimiter']!.functionValue!; + expect(nullDelimiter.args, hasLength(2)); + expect( + nullDelimiter.args[1].nullValue, + protobuf_v1.NullValue.nullValue, + ); + }); + test('leave value positions and variadic tails alone', () async { await run( firestore.pipeline().collection('books').select([ @@ -1122,6 +1564,502 @@ void main() { 'books/book-1', ); }); + + test('map_remove sends exactly one key per call', () async { + await run( + firestore.pipeline().collection('books').select([ + PipelineFunctions.mapRemove('metadata', 'lang').as('static'), + field('metadata').mapRemove(constant('lang')).as('expressionKey'), + field('metadata').mapRemove('lang').mapRemove('draft').as('chain'), + ]), + ); + + final fields = stages[1].args.single.mapValue!.fields; + + // Regression: the key used to be an Iterable spread into a variadic + // `map_remove(map, ...keys)`; the backend contract is (map, key). + final static = fields['static']!.functionValue!; + expect(static.name, 'map_remove'); + expect(static.args, hasLength(2)); + expect(static.args[0].fieldReferenceValue, 'metadata'); + // A String key is a literal, not a field reference, as in Node. + expect(static.args[1].stringValue, 'lang'); + expect(static.args[1].fieldReferenceValue, isNull); + + final expressionKey = fields['expressionKey']!.functionValue!; + expect(expressionKey.args, hasLength(2)); + expect(expressionKey.args[1].stringValue, 'lang'); + + final outer = fields['chain']!.functionValue!; + expect(outer.name, 'map_remove'); + expect(outer.args, hasLength(2)); + expect(outer.args[1].stringValue, 'draft'); + final inner = outer.args[0].functionValue!; + expect(inner.name, 'map_remove'); + expect(inner.args, hasLength(2)); + expect(inner.args[0].fieldReferenceValue, 'metadata'); + expect(inner.args[1].stringValue, 'lang'); + }); + + test('map_remove rejects an Iterable of keys', () { + expect( + () => field('metadata').mapRemove(['lang', 'draft']), + throwsArgumentError, + ); + expect( + () => PipelineFunctions.mapRemove('metadata', ['lang']), + throwsArgumentError, + ); + }); + }); + + group('vector arguments', () { + late List stages; + + Future run(Pipeline pipeline) async { + firestore_v1.ExecutePipelineRequest? capturedRequest; + + when( + () => mockClient.v1>( + any(), + ), + ).thenAnswer((invocation) async { + final callback = + invocation.positionalArguments.single + as Future> + Function(firestore_v1.Firestore api, String projectId); + + final api = FakeFirestore( + executePipeline: (firestore_v1.ExecutePipelineRequest request) { + capturedRequest = request; + return const Stream.empty(); + }, + ); + + return callback(api, _projectId); + }); + + await pipeline.execute(); + stages = capturedRequest!.structuredPipeline!.pipeline!.stages; + } + + Pipeline base() => firestore.pipeline().collection('books'); + + void expectVector(firestore_v1.Value value, List expected) { + expect(value.arrayValue, isNull, reason: 'a vector is not an ARRAY'); + final fields = value.mapValue!.fields; + expect(fields['__type__']!.stringValue, '__vector__'); + expect( + fields['value']!.arrayValue!.values.map((v) => v.doubleValue), + expected, + ); + } + + Future> selectAll( + Object? Function() vector, + ) async { + await run( + base().select([ + PipelineFunctions.cosineDistance( + 'embedding', + vector(), + ).as('cosine'), + PipelineFunctions.dotProduct('embedding', vector()).as('dot'), + PipelineFunctions.euclideanDistance( + 'embedding', + vector(), + ).as('euclidean'), + field('embedding').cosineDistance(vector()).as('fluentCosine'), + field('embedding').dotProduct(vector()).as('fluentDot'), + field( + 'embedding', + ).euclideanDistance(vector()).as('fluentEuclidean'), + ]), + ); + return stages[1].args.single.mapValue!.fields; + } + + test('encode a List as a vector', () async { + // Regression: a plain list used to encode as an ARRAY, which the + // backend rejects with "requires `Vector` but got `ARRAY`". + final fields = await selectAll(() => const [1.0, 0.0, 0.5]); + + expect(fields, hasLength(6)); + for (final entry in fields.entries) { + final function = entry.value.functionValue!; + expect(function.args[0].fieldReferenceValue, 'embedding'); + expectVector(function.args[1], [1.0, 0.0, 0.5]); + } + }); + + test('encode a List as a vector of doubles', () async { + final fields = await selectAll(() => const [1, 2, 3]); + + for (final function in fields.values) { + expectVector(function.functionValue!.args[1], [1.0, 2.0, 3.0]); + } + }); + + test('encode an untyped list of numbers as a vector', () async { + // As decoded from JSON, for instance. + final fields = await selectAll(() => [1, 2.5]); + + for (final function in fields.values) { + expectVector(function.functionValue!.args[1], [1.0, 2.5]); + } + }); + + test('keep a VectorValue as a vector', () async { + final fields = await selectAll(() => FieldValue.vector([1, 2, 3])); + + for (final function in fields.values) { + expectVector(function.functionValue!.args[1], [1.0, 2.0, 3.0]); + } + }); + + test('pass expressions through', () async { + final fieldFunctions = await selectAll(() => field('other')); + for (final function in fieldFunctions.values) { + expect(function.functionValue!.args[1].fieldReferenceValue, 'other'); + } + + final constantFunctions = await selectAll( + () => Expression.vector([1, 2, 3]), + ); + for (final function in constantFunctions.values) { + expectVector(function.functionValue!.args[1], [1.0, 2.0, 3.0]); + } + }); + + test('reject a list that is not all numbers', () { + expect( + () => PipelineFunctions.cosineDistance('embedding', [1, 'two']), + throwsA(isA()), + ); + expect( + () => field('embedding').dotProduct([field('x')]), + throwsA(isA()), + ); + expect( + () => base().findNearest( + vectorField: 'embedding', + queryVector: ['1'], + distanceMeasure: DistanceMeasure.cosine, + ), + throwsA(isA()), + ); + }); + + group('findNearest', () { + Future queryVectorOf(Object queryVector) async { + await run( + base().findNearest( + vectorField: 'embedding', + queryVector: queryVector, + distanceMeasure: DistanceMeasure.euclidean, + ), + ); + final stage = stages.last; + expect(stage.name, 'find_nearest'); + expect(stage.args[0].fieldReferenceValue, 'embedding'); + expect(stage.args[2].stringValue, 'euclidean'); + return stage.args[1]; + } + + test('encodes a List as a vector', () async { + expectVector(await queryVectorOf(const [1.0, 2.0]), [1.0, 2.0]); + }); + + test('encodes a List as a vector of doubles', () async { + expectVector(await queryVectorOf(const [1, 2]), [1.0, 2.0]); + }); + + test('keeps a VectorValue as a vector', () async { + expectVector(await queryVectorOf(FieldValue.vector([1, 2])), [ + 1.0, + 2.0, + ]); + }); + + test('passes expressions through', () async { + expectVector(await queryVectorOf(Expression.vector([1, 2])), [ + 1.0, + 2.0, + ]); + expect( + (await queryVectorOf(field('target'))).fieldReferenceValue, + 'target', + ); + }); + }); + }); + + // The backend rejects expressions nested inside a literal array or map + // value ("Value type is not supported: FIELD_REFERENCE_VALUE"), so + // collections holding them must be built with array(...) / map(...). + group('Collections holding expressions', () { + late Map fields; + + Future run(Iterable selections) async { + firestore_v1.ExecutePipelineRequest? capturedRequest; + + when( + () => mockClient.v1>( + any(), + ), + ).thenAnswer((invocation) async { + final callback = + invocation.positionalArguments.single + as Future> + Function(firestore_v1.Firestore api, String projectId); + + final api = FakeFirestore( + executePipeline: (firestore_v1.ExecutePipelineRequest request) { + capturedRequest = request; + return const Stream.empty(); + }, + ); + + return callback(api, _projectId); + }); + + await firestore + .pipeline() + .collection('books') + .select(selections) + .execute(); + final stages = capturedRequest!.structuredPipeline!.pipeline!.stages; + fields = stages[1].args.single.mapValue!.fields; + } + + firestore_v1.Function$ function(String alias) { + return fields[alias]!.functionValue!; + } + + /// Expects [value] to be `array(field(score), 5)`. + void expectMixedArray(firestore_v1.Value value) { + final array = value.functionValue!; + expect(array.name, 'array'); + expect(array.args, hasLength(2)); + expect(array.args[0].fieldReferenceValue, 'score'); + expect(array.args[1].integerValue, 5); + expect(value.arrayValue, isNull); + } + + test('equalAny and notEqualAny build an array function', () async { + final values = [field('score'), 5]; + await run([ + PipelineFunctions.equalAny('rating', values).as('static'), + field('rating').equalAny(values).as('fluent'), + PipelineFunctions.notEqualAny('rating', values).as('notStatic'), + field('rating').notEqualAny(values).as('notFluent'), + ]); + + for (final alias in ['static', 'fluent']) { + expect(function(alias).name, 'equal_any'); + expect(function(alias).args[0].fieldReferenceValue, 'rating'); + expectMixedArray(function(alias).args[1]); + } + for (final alias in ['notStatic', 'notFluent']) { + expect(function(alias).name, 'not_equal_any'); + expect(function(alias).args[0].fieldReferenceValue, 'rating'); + expectMixedArray(function(alias).args[1]); + } + }); + + test( + 'arrayContainsAll and arrayContainsAny build an array function', + () async { + final values = [field('score'), 5]; + await run([ + PipelineFunctions.arrayContainsAll('tags', values).as('allStatic'), + field('tags').arrayContainsAll(values).as('allFluent'), + PipelineFunctions.arrayContainsAny('tags', values).as('anyStatic'), + field('tags').arrayContainsAny(values).as('anyFluent'), + ]); + + for (final alias in ['allStatic', 'allFluent']) { + expect(function(alias).name, 'array_contains_all'); + expectMixedArray(function(alias).args[1]); + } + for (final alias in ['anyStatic', 'anyFluent']) { + expect(function(alias).name, 'array_contains_any'); + expectMixedArray(function(alias).args[1]); + } + }, + ); + + test('search spaces of plain values stay literal arrays', () async { + // Matches the Node SDK, which sends these as a literal array value. + await run([ + PipelineFunctions.equalAny('genre', ['fiction', 'poetry']).as('eq'), + field('genre').notEqualAny(['fiction']).as('notEq'), + PipelineFunctions.arrayContainsAll('tags', ['a', 'b']).as('all'), + field('tags').arrayContainsAny(['a', constant('b')]).as('any'), + field('genre').equalAny(field('genres')).as('expression'), + ]); + + for (final alias in ['eq', 'notEq', 'all', 'any']) { + final searchSpace = function(alias).args[1]; + expect(searchSpace.functionValue, isNull, reason: alias); + expect(searchSpace.arrayValue, isNotNull, reason: alias); + } + expect( + function('any').args[1].arrayValue!.values.map((v) => v.stringValue), + ['a', 'b'], + ); + expect(function('expression').args[1].fieldReferenceValue, 'genres'); + }); + + test('static and fluent forms encode identically', () async { + final values = [field('score'), 5, 'x']; + await run([ + PipelineFunctions.equalAny('rating', values).as('a'), + field('rating').equalAny(values).as('b'), + PipelineFunctions.arrayContainsAll('tags', values).as('c'), + field('tags').arrayContainsAll(values).as('d'), + PipelineFunctions.arrayContainsAll('tags', ['x']).as('e'), + field('tags').arrayContainsAll(['x']).as('f'), + ]); + + expect(fields['a']!.toJson(), fields['b']!.toJson()); + expect(fields['c']!.toJson(), fields['d']!.toJson()); + expect(fields['e']!.toJson(), fields['f']!.toJson()); + }); + + test('mapMerge builds a map function', () async { + await run([ + PipelineFunctions.mapMerge([ + 'metadata', + {'reviewer': field('editor'), 'lang': 'dart'}, + ]).as('static'), + field('metadata') + .mapMerge([ + {'reviewer': field('editor'), 'lang': 'dart'}, + ]) + .as('fluent'), + field('metadata').mapMerge([{}]).as('empty'), + ]); + + for (final alias in ['static', 'fluent']) { + final merge = function(alias); + expect(merge.name, 'map_merge'); + expect(merge.args[0].fieldReferenceValue, 'metadata'); + final map = merge.args[1].functionValue!; + expect(map.name, 'map'); + expect(map.args.map((arg) => arg.toJson()), [ + {'stringValue': 'reviewer'}, + {'fieldReferenceValue': 'editor'}, + {'stringValue': 'lang'}, + {'stringValue': 'dart'}, + ]); + } + + final empty = function('empty').args[1].functionValue!; + expect(empty.name, 'map'); + expect(empty.args, isEmpty); + }); + + test('nested collections are built recursively', () async { + await run([ + PipelineFunctions.array([ + 1, + [field('title')], + {'published': field('published')}, + ]).as('array'), + PipelineFunctions.map([ + 'nested', + {'price': field('price')}, + ]).as('map'), + ]); + + final array = function('array'); + expect(array.name, 'array'); + expect(array.args[0].integerValue, 1); + final inner = array.args[1].functionValue!; + expect(inner.name, 'array'); + expect(inner.args.single.fieldReferenceValue, 'title'); + final innerMap = array.args[2].functionValue!; + expect(innerMap.name, 'map'); + expect(innerMap.args[0].stringValue, 'published'); + expect(innerMap.args[1].fieldReferenceValue, 'published'); + + final map = function('map'); + expect(map.args[0].stringValue, 'nested'); + final nested = map.args[1].functionValue!; + expect(nested.name, 'map'); + expect(nested.args[1].fieldReferenceValue, 'price'); + }); + + test('apply to every value position', () async { + await run([ + PipelineFunctions.arrayConcat([ + 'tags', + [field('title')], + ]).as('concat'), + field('tags').arrayConcat([field('title')]).as('fluentConcat'), + field('tags').arrayContains([field('title')]).as('contains'), + equal('tags', [field('title')]).as('topLevelEqual'), + PipelineFunctions.equal('tags', [field('title')]).as('staticEqual'), + field('tags').equal([field('title')]).as('fluentEqual'), + field( + 'metadata', + ).mapSet('reviewer', {'name': field('editor')}).as('mapSet'), + PipelineFunctions.conditional(field('active'), [ + field('title'), + ], const []).as('conditional'), + field('tags').ifAbsent([field('title')]).as('ifAbsent'), + PipelineFunctions.logicalMaximum('tags', [field('title')]).as('max'), + ]); + + for (final alias in [ + 'concat', + 'fluentConcat', + 'contains', + 'topLevelEqual', + 'staticEqual', + 'fluentEqual', + 'ifAbsent', + 'max', + ]) { + final array = function(alias).args[1].functionValue; + expect(array?.name, 'array', reason: alias); + expect(array!.args.single.fieldReferenceValue, 'title'); + } + + final mapSet = function('mapSet'); + expect(mapSet.args[1].stringValue, 'reviewer'); + expect(mapSet.args[2].functionValue!.name, 'map'); + + final conditional = function('conditional'); + expect(conditional.args[1].functionValue!.name, 'array'); + expect(conditional.args[2].functionValue!.name, 'array'); + expect(conditional.args[2].functionValue!.args, isEmpty); + }); + + test('constant and raw keep collections literal', () async { + await run([ + field('tags').equal(constant(['a', 'b'])).as('constant'), + PipelineFunctions.raw('array_length', [ + ['a', 'b'], + ]).as('raw'), + field('bytes').equal(Uint8List.fromList([1, 2])).as('bytes'), + ]); + + expect( + function( + 'constant', + ).args[1].arrayValue!.values.map((v) => v.stringValue), + ['a', 'b'], + ); + expect( + function( + 'raw', + ).args.single.arrayValue!.values.map((v) => v.stringValue), + ['a', 'b'], + ); + expect(function('bytes').args[1].bytesValue, isNotNull); + }); }); test('serializes newly added expressions', () async { @@ -1250,15 +2188,173 @@ void main() { expect(fields['inactive']!.functionValue!.name, 'not'); expect(fields['activeCount']!.functionValue!.name, 'count_if'); expect(fields['label']!.functionValue!.name, 'conditional'); - expect(fields['maxNumber']!.functionValue!.name, 'array_maximum'); - expect(fields['minNumber']!.functionValue!.name, 'array_minimum'); - expect(fields['top2']!.functionValue!.name, 'array_maximum_n'); - expect(fields['bottom2']!.functionValue!.name, 'array_minimum_n'); - expect(fields['total']!.functionValue!.name, 'array_sum'); + expect(fields['maxNumber']!.functionValue!.name, 'maximum'); + expect(fields['minNumber']!.functionValue!.name, 'minimum'); + expect(fields['top2']!.functionValue!.name, 'maximum_n'); + expect(fields['bottom2']!.functionValue!.name, 'minimum_n'); + expect(fields['total']!.functionValue!.name, 'sum'); expect(fields['rows']!.functionValue!.name, 'count'); expect(fields['rows']!.functionValue!.args, isEmpty); }); + test('round and trunc forward optional decimal places', () async { + firestore_v1.ExecutePipelineRequest? capturedRequest; + + when( + () => + mockClient.v1>(any()), + ).thenAnswer((invocation) async { + final callback = + invocation.positionalArguments.single + as Future> + Function(firestore_v1.Firestore api, String projectId); + + final api = FakeFirestore( + executePipeline: (request) { + capturedRequest = request; + return const Stream.empty(); + }, + ); + + return callback(api, _projectId); + }); + + await firestore.pipeline().collection('books').select([ + field('price').round().as('round'), + // Regression: the fluent form used to take no argument at all. + field('price').round(2).as('round2'), + field('price').round(field('places')).as('roundExpr'), + field('price').trunc().as('trunc'), + field('price').trunc(2).as('trunc2'), + field('price').trunc(field('places')).as('truncExpr'), + PipelineFunctions.round('price').as('staticRound'), + PipelineFunctions.round('price', 2).as('staticRound2'), + PipelineFunctions.round( + 'price', + Expression.constant(2), + ).as('staticRoundExpr'), + PipelineFunctions.trunc('price').as('staticTrunc'), + PipelineFunctions.trunc('price', 2).as('staticTrunc2'), + ]).execute(); + + final fields = capturedRequest! + .structuredPipeline! + .pipeline! + .stages[1] + .args + .single + .mapValue! + .fields; + + for (final MapEntry(key: alias, value: value) in fields.entries) { + final function = value.functionValue!; + expect( + function.name, + alias.toLowerCase().contains('round') ? 'round' : 'trunc', + reason: alias, + ); + expect(function.args[0].fieldReferenceValue, 'price', reason: alias); + } + + // Without decimal places only the operand is sent, matching Node. + for (final alias in ['round', 'trunc', 'staticRound', 'staticTrunc']) { + expect(fields[alias]!.functionValue!.args, hasLength(1), reason: alias); + } + + for (final alias in [ + 'round2', + 'trunc2', + 'staticRound2', + 'staticTrunc2', + ]) { + final args = fields[alias]!.functionValue!.args; + expect(args, hasLength(2), reason: alias); + expect(args[1].integerValue, 2, reason: alias); + } + + for (final alias in ['roundExpr', 'truncExpr']) { + final args = fields[alias]!.functionValue!.args; + expect(args, hasLength(2), reason: alias); + expect(args[1].fieldReferenceValue, 'places', reason: alias); + } + + final staticRoundExpr = fields['staticRoundExpr']!.functionValue!.args; + expect(staticRoundExpr, hasLength(2)); + expect(staticRoundExpr[1].integerValue, 2); + }); + + test('array aggregation helpers encode like their fluent forms', () async { + firestore_v1.ExecutePipelineRequest? capturedRequest; + + when( + () => + mockClient.v1>(any()), + ).thenAnswer((invocation) async { + final callback = + invocation.positionalArguments.single + as Future> + Function(firestore_v1.Firestore api, String projectId); + + final api = FakeFirestore( + executePipeline: (request) { + capturedRequest = request; + return const Stream.empty(); + }, + ); + + return callback(api, _projectId); + }); + + final numbers = field('numbers'); + await firestore.pipeline().collection('books').select([ + PipelineFunctions.arrayMaximum('numbers').as('max'), + numbers.arrayMaximum().as('maxFluent'), + PipelineFunctions.arrayMaximumN('numbers', 2).as('maxN'), + numbers.arrayMaximumN(2).as('maxNFluent'), + PipelineFunctions.arrayMinimum('numbers').as('min'), + numbers.arrayMinimum().as('minFluent'), + PipelineFunctions.arrayMinimumN('numbers', 2).as('minN'), + numbers.arrayMinimumN(2).as('minNFluent'), + PipelineFunctions.arraySum('numbers').as('sum'), + numbers.arraySum().as('sumFluent'), + ]).execute(); + + final fields = capturedRequest! + .structuredPipeline! + .pipeline! + .stages[1] + .args + .single + .mapValue! + .fields; + + // Regression: the static helpers used to emit `array_maximum`, + // `array_maximum_n`, `array_minimum`, `array_minimum_n` and `array_sum`, + // which the backend rejects with "The function 'array_maximum' does not + // exist, did you mean 'maximum'?". Node emits the unprefixed names. + const expectedNames = { + 'max': 'maximum', + 'maxN': 'maximum_n', + 'min': 'minimum', + 'minN': 'minimum_n', + 'sum': 'sum', + }; + for (final MapEntry(key: alias, value: name) in expectedNames.entries) { + final helper = fields[alias]!.functionValue!; + final fluent = fields['${alias}Fluent']!.functionValue!; + + expect(helper.name, name, reason: alias); + expect(helper.args.first.fieldReferenceValue, 'numbers', reason: alias); + expect( + helper.toJson(), + fluent.toJson(), + reason: '$alias should encode exactly like its fluent form', + ); + } + expect(fields['maxN']!.functionValue!.args[1].integerValue, 2); + expect(fields['minN']!.functionValue!.args[1].integerValue, 2); + }); + group('createFrom', () { late List stages; @@ -1434,7 +2530,10 @@ void main() { await run(firestore.collectionGroup('books')); expect(stages.first.name, 'collection_group'); - expect(stages.first.args.single.stringValue, 'books'); + // The root ancestor comes first, as in the Node SDK. + expect(stages.first.args, hasLength(2)); + expect(stages.first.args[0].referenceValue, ''); + expect(stages.first.args[1].stringValue, 'books'); }); test('converts orderBy, limit and offset', () async { @@ -1546,6 +2645,12 @@ void main() { final findNearest = stages.last; expect(findNearest.args[0].fieldReferenceValue, 'embedding'); + final queryVector = findNearest.args[1].mapValue!.fields; + expect(queryVector['__type__']!.stringValue, '__vector__'); + expect( + queryVector['value']!.arrayValue!.values.map((v) => v.doubleValue), + [1.0, 2.0, 3.0], + ); // Pipelines take a lowercase distance measure even though the // DistanceMeasure enum keeps the uppercase proto values. expect(findNearest.args[2].stringValue, 'cosine');