diff --git a/CHANGELOG.md b/CHANGELOG.md index 263821a..0c6cc7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,7 @@ number says nothing about any other port. See the Releases section of - **A decoder rejects `null` on a field with `(pxf.required)` or `@required`** (#131, decided by the owner; trendvidia/protowire#569, protowire-go#268). **This narrows what decodes.** A required field holds a value, so `null` is now rejected as absence is, whatever the field's type and presence. Before, `null` counted as present, so `r = null` on `optional int32 r [(pxf.required) = true]` decoded and left a required field with no value. The error is `null is not allowed on required field "r": a required field holds a value`, in a body, a nested body, a `@dataset` cell and a dotted cell, from `Unmarshal` as well as `UnmarshalFull`. A plain scalar with `(pxf.required)`, which already rejected `null`, now names the field as required instead of suggesting `optional`. A zero value still satisfies the field. A `_null` mask that names an unset required field no longer makes `AppendDatasetRow` write `null` for it: the writer refuses it, as it does without the mask. - **`pxf::Marshal` writes an unset field that has `(pxf.default)` or `(pxf.required)` so that it reads back** (#129, protowire-go#259; draft -01 § Common Schema Layer, "`decode_PXF(encode_PXF(v)) == v`"), as protowire-go does. This changes what Marshal writes for annotated fields, and narrows what encodes. Marshal left out every unset field, so a reader that applies defaults filled it in: `int32 n = 1 [(pxf.default) = "5"]` holding 0 read back as 5, and a required scalar holding 0 made the full decode refuse the document. A proto3 scalar without presence is now written as its zero value, a required list or map as `[]` or `{}`, and a message-typed field with a default, such as a wrapper or a Timestamp, as `null`. An unset `optional` or oneof scalar with a default is still left out and reads back as its default, the owner's choice in protowire-go#267. An unset `optional` field that is `(pxf.required)` is now an error, `pxf: is unset, and the field is (pxf.required)`; with `emit_defaults` it is written as zero, as before. An Any whose body has such a field stays as `type_url` and `value`. Inside a `@dataset` value cell, the same error now names the field by its full name. Unannotated fields are unchanged. +- **`sbe::Codec::New` refuses a schema the SBE header cannot describe, and `Marshal` a group it cannot count** (#132, protowire-go#250). This narrows what binds and what encodes. Every SBE header field is a 16-bit number, and the codec kept the low 16 bits of a larger one. `bytes b = 1 [(sbe.length) = 65537]` bound as a 1-byte field. A block longer than 65535 bytes, in a message, a group entry or a composite, bound with a wrapped length and offsets. A `(sbe.template_id)`, `(sbe.schema_id)` or `(sbe.version)` of 65537 was written as 1. `Codec::New` now refuses each of these with protowire-go's text, naming the field, message or file, such as `sbe: field m.v1.M.b: (sbe.length) = 65537 is past 65535, the largest blockLength SBE can state`; 65535 still binds. `Marshal` refuses a repeated field with more than 65535 entries, with `sbe: field m.v1.M.e has 65536 entries, past 65535, the largest numInGroup SBE can state`: 65536 entries used to be written with a count of 0, and the message read back empty with no error. Every other bind error of a template now has protowire-go's text as well, prefixed with the field, group or composite it is about, such as `sbe: field m.v1.M.s: string field requires (sbe.length) annotation` where it used to be `string/bytes field requires (sbe.length)`. - **`sbe` tells two schemas' template ids apart** (#134, protowire-go#253 and #257; draft -01 § SBE Annotations, "sbe.template_id MUST be unique among messages within a schema", and § Header and Block Length). This narrows what binds and what decodes. `Codec::New` refuses two different messages with one `(sbe.template_id)` in one schema, with `sbe: messages A and B share (sbe.template_id) = 7 in schema 1; it must be unique within a schema`; the second used to replace the first, and `NewView` then read one message's bytes as the other's. A file passed twice still binds, and so do two schemas that each use a template id. `NewView` now finds a template by the header's schema id and template id together, and refuses an unknown pair with `sbe: unknown template ID 9 in schema 2`, even if another schema has that template id; it used to say `unknown template id 9`. `Unmarshal` and `UnmarshalDescriptor` refuse a header whose schema id is not the target message's, with `sbe: schema ID mismatch: got 1, want 2 for template 7`; the version is not checked. A caller that builds headers by hand must now set the schema id. - **`pxf::Marshal` refuses a message nested past `kMaxNestingDepth`** (#107, protowire#502), as protowire-go does (protowire-go#236), with `pxf: nesting depth exceeds MaxNestingDepth=100`. Depth is counted as a decoder counts it: the root is depth 0, and each `{` or `[` is one level, so a list of lists stops at 50. It used to write a document that a reader at the same limit refused. **This narrows what encodes.** `AppendDatasetRow` refuses too. An Any whose body would be too deep stays as `type_url` and `value`, so an Any's contents never cause a refusal; only an Any that itself sits past the limit does. An Any's value is also no longer cut off at libprotobuf's 100-level parse limit when the writer's limit is raised. `pb::Marshal` cannot refuse yet (#93). - **A schema that redefines a well-known type with the wrong shape is refused at bind time** (draft -01 §wkt-shapes, protowire#487, protowire-go#218). This narrows what binds. A message named, say, `pxf.Secret` or `google.protobuf.Timestamp` without the fields the codec reads now gives a violation of the new kind `ViolationKind::kWellKnownShape`. Copies of the canonical schemas are unaffected, and field numbers are not checked. The check also reaches a schema's own file at a `google/protobuf/` path, which the closure walk used to skip. diff --git a/src/sbe/codec.cc b/src/sbe/codec.cc index 3a44c17..9bf6cfd 100644 --- a/src/sbe/codec.cc +++ b/src/sbe/codec.cc @@ -5,6 +5,7 @@ #include #include +#include "header_bounds.h" #include "protowire/sbe/annotations.h" namespace protowire::sbe { @@ -31,6 +32,16 @@ StatusOr Codec::New(std::vector files, CodecOp " missing (sbe.schema_id) option"); } auto version = GetFileUint32Option(fd, kExtVersion).value_or(0); + if (*schema > static_cast(kMaxUint16)) { + return Status::Error("sbe: file " + std::string(fd->name()) + + ": (sbe.schema_id) = " + std::to_string(*schema) + " is past " + + std::to_string(kMaxUint16) + ", the largest schemaId SBE can state"); + } + if (version > static_cast(kMaxUint16)) { + return Status::Error("sbe: file " + std::string(fd->name()) + + ": (sbe.version) = " + std::to_string(version) + " is past " + + std::to_string(kMaxUint16) + ", the largest version SBE can state"); + } for (int i = 0; i < fd->message_type_count(); ++i) { Status st = c.RegisterMessage( fd->message_type(i), static_cast(*schema), static_cast(version)); diff --git a/src/sbe/header_bounds.h b/src/sbe/header_bounds.h new file mode 100644 index 0000000..3a4626f --- /dev/null +++ b/src/sbe/header_bounds.h @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 TrendVidia, LLC. +#pragma once + +namespace protowire::sbe { + +// kMaxUint16 is the largest value of every SBE header field: blockLength, +// templateId, schemaId and version in the message header, blockLength and +// numInGroup in a group header. A schema or a message needing a larger value +// cannot be written: narrowing it would write a header that disagrees with +// the bytes after it (protowire-go#250). +inline constexpr int kMaxUint16 = 65535; + +} // namespace protowire::sbe diff --git a/src/sbe/marshal.cc b/src/sbe/marshal.cc index 7e22d33..76ea190 100644 --- a/src/sbe/marshal.cc +++ b/src/sbe/marshal.cc @@ -5,8 +5,11 @@ #include "protowire/sbe.h" #include +#include #include +#include "header_bounds.h" + namespace protowire::sbe { namespace pb = google::protobuf; @@ -156,6 +159,13 @@ StatusOr> Codec::Marshal(const pb::Message& msg) const { for (const GroupTemplate& gt : tmpl->groups) { const pb::Reflection* r = msg.GetReflection(); int n = r->FieldSize(msg, gt.fd); + // numInGroup is a uint16: a larger count would be written as its low 16 + // bits, and a reader would take the entries past them for trailing bytes. + if (n > kMaxUint16) { + return Status::Error("sbe: field " + std::string(gt.fd->full_name()) + " has " + + std::to_string(n) + " entries, past " + std::to_string(kMaxUint16) + + ", the largest numInGroup SBE can state"); + } size_t header_off = buf.size(); buf.resize(header_off + 4, 0); WriteU16(buf.data() + header_off + 0, gt.block_length); diff --git a/src/sbe/template.cc b/src/sbe/template.cc index 214a1b7..cd7f5e7 100644 --- a/src/sbe/template.cc +++ b/src/sbe/template.cc @@ -7,6 +7,8 @@ #include #include +#include "header_bounds.h" +#include "protowire/detail/go_quote.h" #include "protowire/sbe/annotations.h" namespace protowire::sbe { @@ -29,6 +31,28 @@ std::vector SortedFields(const pb::Descriptor* md) { return out; } +// "pkg.Msg.field", as Go's "%s.%s" of a message's full name and a field's +// name writes it. +std::string FieldPath(const pb::Descriptor* md, const pb::FieldDescriptor* fd) { + return std::string(md->full_name()) + "." + std::string(fd->name()); +} + +Status Wrap(const std::string& prefix, const Status& st) { + return Status::Error(prefix + ": " + st.message()); +} + +// EndOfField returns the offset after a field of size bytes placed at +// offset, refusing a field that ends past what a uint16 blockLength can +// state. Every offset in the block is then a uint16 as well. +StatusOr EndOfField(int offset, uint16_t size) { + const int end = offset + size; + if (end > kMaxUint16) { + return Status::Error("ends at byte " + std::to_string(end) + " of its block, past " + + std::to_string(kMaxUint16) + ", the largest blockLength SBE can state"); + } + return end; +} + StatusOr> FieldEncodingSize(const pb::FieldDescriptor* fd) { if (auto enc = GetFieldStringOption(fd, kExtEncoding); enc.has_value()) { const std::string& e = *enc; @@ -36,7 +60,7 @@ StatusOr> FieldEncodingSize(const pb::FieldDesc if (e == kEncInt16 || e == kEncUint16) return std::make_pair(e, uint16_t{2}); if (e == kEncInt32 || e == kEncUint32 || e == kEncFloat) return std::make_pair(e, uint16_t{4}); if (e == kEncInt64 || e == kEncUint64 || e == kEncDouble) return std::make_pair(e, uint16_t{8}); - return Status::Error("unknown encoding \"" + e + "\""); + return Status::Error("unknown encoding " + detail::GoQuote(e)); } switch (fd->cpp_type()) { case pb::FieldDescriptor::CPPTYPE_BOOL: @@ -58,12 +82,18 @@ StatusOr> FieldEncodingSize(const pb::FieldDesc case pb::FieldDescriptor::CPPTYPE_STRING: { auto len = GetFieldUint32Option(fd, kExtLength); if (!len.has_value()) { - return Status::Error("string/bytes field requires (sbe.length)"); + return Status::Error(std::string(fd->type_name()) + + " field requires (sbe.length) annotation"); + } + if (*len > static_cast(kMaxUint16)) { + return Status::Error("(sbe.length) = " + std::to_string(*len) + " is past " + + std::to_string(kMaxUint16) + + ", the largest blockLength SBE can state"); } return std::make_pair(std::string(kEncChar), static_cast(*len)); } default: - return Status::Error("unsupported proto type for SBE"); + return Status::Error("unsupported proto kind " + std::string(fd->type_name())); } } @@ -73,35 +103,41 @@ StatusOr> BuildCompositeFields(const pb::Descriptor* StatusOr> BuildCompositeFields(const pb::Descriptor* md, uint16_t* total_size) { std::vector out; - uint16_t offset = 0; + int offset = 0; for (const pb::FieldDescriptor* fd : SortedFields(md)) { + const std::string where = "composite field " + FieldPath(md, fd); if (fd->is_repeated() || fd->is_map()) { - return Status::Error("composite contains list/map field"); + return Status::Error("composite " + std::string(md->full_name()) + + " contains list/map field " + std::string(fd->name())); } if (fd->cpp_type() == pb::FieldDescriptor::CPPTYPE_MESSAGE) { uint16_t sub_size = 0; auto subs = BuildCompositeFields(fd->message_type(), &sub_size); if (!subs.ok()) return subs.status(); + auto end = EndOfField(offset, sub_size); + if (!end.ok()) return Wrap(where, end.status()); FieldTemplate ft; ft.fd = fd; - ft.offset = offset; + ft.offset = static_cast(offset); ft.size = sub_size; ft.composite = std::move(subs).consume(); out.push_back(std::move(ft)); - offset += sub_size; + offset = *end; continue; } auto es = FieldEncodingSize(fd); - if (!es.ok()) return es.status(); + if (!es.ok()) return Wrap(where, es.status()); + auto end = EndOfField(offset, es->second); + if (!end.ok()) return Wrap(where, end.status()); FieldTemplate ft; ft.fd = fd; - ft.offset = offset; + ft.offset = static_cast(offset); ft.encoding = es->first; ft.size = es->second; out.push_back(ft); - offset += ft.size; + offset = *end; } - *total_size = offset; + *total_size = static_cast(offset); return out; } @@ -109,35 +145,46 @@ StatusOr BuildGroupTemplate(const pb::FieldDescriptor* fd) { const pb::Descriptor* md = fd->message_type(); GroupTemplate gt; gt.fd = fd; - uint16_t offset = 0; + int offset = 0; for (const pb::FieldDescriptor* f : SortedFields(md)) { - if (f->is_map() || f->is_repeated()) { - return Status::Error("group field cannot be repeated/map"); + if (f->is_map()) { + return Status::Error("sbe: map field in group " + std::string(md->full_name()) + + " not supported"); + } + if (f->is_repeated()) { + return Status::Error("sbe: nested repeated field in group " + std::string(md->full_name()) + + " not supported"); } if (f->cpp_type() == pb::FieldDescriptor::CPPTYPE_MESSAGE) { + const std::string where = "sbe: composite in group " + FieldPath(md, f); uint16_t sub_size = 0; auto subs = BuildCompositeFields(f->message_type(), &sub_size); - if (!subs.ok()) return subs.status(); + if (!subs.ok()) return Wrap(where, subs.status()); + auto end = EndOfField(offset, sub_size); + if (!end.ok()) return Wrap(where, end.status()); FieldTemplate ft; ft.fd = f; - ft.offset = offset; + ft.offset = static_cast(offset); ft.size = sub_size; ft.composite = std::move(subs).consume(); gt.fields.push_back(std::move(ft)); - offset += sub_size; + offset = *end; continue; } + const std::string where = "sbe: group field " + FieldPath(md, f); auto es = FieldEncodingSize(f); - if (!es.ok()) return es.status(); + if (!es.ok()) return Wrap(where, es.status()); + auto end = EndOfField(offset, es->second); + if (!end.ok()) return Wrap(where, end.status()); FieldTemplate ft; ft.fd = f; - ft.offset = offset; + ft.offset = static_cast(offset); ft.encoding = es->first; ft.size = es->second; gt.fields.push_back(ft); - offset += ft.size; + offset = *end; } - gt.block_length = offset; + gt.block_length = static_cast(offset); return gt; } @@ -146,19 +193,25 @@ StatusOr BuildGroupTemplate(const pb::FieldDescriptor* fd) { StatusOr BuildTemplate(const pb::Descriptor* md, uint16_t schema_id, uint16_t version) { + const std::string name(md->full_name()); auto tid = GetMessageUint32Option(md, kExtTemplateID); if (!tid.has_value()) { - return Status::Error(std::string(md->full_name()) + " missing (sbe.template_id)"); + return Status::Error("sbe: message " + name + " missing (sbe.template_id)"); + } + if (*tid > static_cast(kMaxUint16)) { + return Status::Error("sbe: message " + name + ": (sbe.template_id) = " + std::to_string(*tid) + + " is past " + std::to_string(kMaxUint16) + + ", the largest templateId SBE can state"); } MessageTemplate tmpl; tmpl.template_id = static_cast(*tid); tmpl.schema_id = schema_id; tmpl.version = version; - uint16_t offset = 0; + int offset = 0; for (const pb::FieldDescriptor* fd : SortedFields(md)) { if (fd->is_map()) { - return Status::Error("map field not supported in SBE"); + return Status::Error("sbe: map field " + FieldPath(md, fd) + " not supported"); } if (fd->is_repeated() && fd->cpp_type() == pb::FieldDescriptor::CPPTYPE_MESSAGE) { auto gt = BuildGroupTemplate(fd); @@ -167,32 +220,39 @@ StatusOr BuildTemplate(const pb::Descriptor* md, continue; } if (fd->is_repeated()) { - return Status::Error("repeated scalar not supported in SBE"); + return Status::Error("sbe: repeated scalar field " + FieldPath(md, fd) + + " not supported; wrap in a message"); } if (fd->cpp_type() == pb::FieldDescriptor::CPPTYPE_MESSAGE) { + const std::string where = "sbe: composite " + FieldPath(md, fd); uint16_t sub_size = 0; auto subs = BuildCompositeFields(fd->message_type(), &sub_size); - if (!subs.ok()) return subs.status(); + if (!subs.ok()) return Wrap(where, subs.status()); + auto end = EndOfField(offset, sub_size); + if (!end.ok()) return Wrap(where, end.status()); FieldTemplate ft; ft.fd = fd; - ft.offset = offset; + ft.offset = static_cast(offset); ft.size = sub_size; ft.composite = std::move(subs).consume(); tmpl.fields.push_back(std::move(ft)); - offset += sub_size; + offset = *end; continue; } + const std::string where = "sbe: field " + FieldPath(md, fd); auto es = FieldEncodingSize(fd); - if (!es.ok()) return es.status(); + if (!es.ok()) return Wrap(where, es.status()); + auto end = EndOfField(offset, es->second); + if (!end.ok()) return Wrap(where, end.status()); FieldTemplate ft; ft.fd = fd; - ft.offset = offset; + ft.offset = static_cast(offset); ft.encoding = es->first; ft.size = es->second; tmpl.fields.push_back(ft); - offset += ft.size; + offset = *end; } - tmpl.block_length = offset; + tmpl.block_length = static_cast(offset); return tmpl; } diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 1cc89dc..7ea43f1 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -64,7 +64,7 @@ endif() if(TARGET protowire_sbe) set(_sbe_test_srcs "") foreach(f sbe_codec_test sbe_convert_test sbe_navigation_test sbe_stream_test - sbe_template_id_test) + sbe_template_id_test sbe_header_bounds_test) if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${f}.cc) list(APPEND _sbe_test_srcs ${f}.cc) endif() diff --git a/test/sbe_header_bounds_test.cc b/test/sbe_header_bounds_test.cc new file mode 100644 index 0000000..d61c468 --- /dev/null +++ b/test/sbe_header_bounds_test.cc @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) 2026 TrendVidia, LLC. +// +// Every SBE header field is a uint16, so each value a schema declares, and +// each block it lays out, must fit in 65535. Ported from protowire-go's +// encoding/sbe/header_bounds_test.go (protowire-go#250). + +#include + +#include +#include +#include +#include + +#include "protowire/sbe.h" +#include "sbe_files.h" + +namespace { + +namespace pb = google::protobuf; +using protowire::sbe::Codec; +using protowire_test::SbeFiles; + +bool BoundsFile(SbeFiles* files, + const std::string& schema_id, + const std::string& version, + const std::string& body) { + return files->Compile( + {{"bounds.proto", + "syntax = \"proto3\";\npackage bounds.v1;\nimport \"sbe/annotations.proto\";\noption " + "(sbe.schema_id) = " + + schema_id + ";\noption (sbe.version) = " + version + ";\n" + body + "\n"}}); +} + +uint16_t Le16(const std::vector& b, size_t at) { + return static_cast(b[at] | (b[at + 1] << 8U)); +} + +// TestNewCodecHeaderBounds pairs each schema the SBE header cannot describe +// with the same shape at the bound, which must bind and round-trip. +TEST(SbeHeaderBounds, NewCodec) { + struct Case { + const char* name; + std::string schema_id; + std::string version; + std::string body; + std::string reject; // a substring of the error; empty means it must bind + uint16_t block_size = 0; // the root blockLength when it binds + }; + const std::vector cases = { + {"length past 65535", + "", + "", + "message M { option (sbe.template_id) = 1; bytes b = 1 [(sbe.length) = 65537]; uint32 t = " + "2; }", + "field bounds.v1.M.b: (sbe.length) = 65537 is past 65535"}, + {"length at 65535", + "", + "", + "message M { option (sbe.template_id) = 1; bytes b = 1 [(sbe.length) = 65535]; }", + "", + 65535}, + {"string length past 65535", + "", + "", + "message M { option (sbe.template_id) = 1; string s = 1 [(sbe.length) = 70000]; }", + "field bounds.v1.M.s: (sbe.length) = 70000 is past 65535"}, + {"root block past 65535", + "", + "", + "message M { option (sbe.template_id) = 1; bytes a = 1 [(sbe.length) = 30000]; bytes b = " + "2 [(sbe.length) = 30000]; bytes c = 3 [(sbe.length) = 30000]; uint32 t = 4; }", + "field bounds.v1.M.c: ends at byte 90000 of its block, past 65535"}, + {"root block one byte past 65535", + "", + "", + "message M { option (sbe.template_id) = 1; bytes a = 1 [(sbe.length) = 65532]; uint32 t = " + "2; }", + "field bounds.v1.M.t: ends at byte 65536 of its block"}, + {"root block at 65535", + "", + "", + "message M { option (sbe.template_id) = 1; bytes a = 1 [(sbe.length) = 65531]; uint32 t = " + "2; }", + "", + 65535}, + {"group block past 65535", + "", + "", + "message E { bytes a = 1 [(sbe.length) = 40000]; bytes b = 2 [(sbe.length) = 40000]; } " + "message M { option (sbe.template_id) = 1; repeated E e = 1; }", + "group field bounds.v1.E.b: ends at byte 80000 of its block"}, + {"group block at 65535", + "", + "", + "message E { bytes a = 1 [(sbe.length) = 40000]; bytes b = 2 [(sbe.length) = 25535]; } " + "message M { option (sbe.template_id) = 1; repeated E e = 1; }", + ""}, + {"composite past 65535", + "", + "", + "message C { bytes a = 1 [(sbe.length) = 40000]; bytes b = 2 [(sbe.length) = 40000]; } " + "message M { option (sbe.template_id) = 1; C c = 1; }", + "composite bounds.v1.M.c: composite field bounds.v1.C.b: ends at byte 80000"}, + {"composite fits, block past 65535", + "", + "", + "message C { bytes a = 1 [(sbe.length) = 40000]; } message M { option (sbe.template_id) = " + "1; C c = 1; C d = 2; }", + "composite bounds.v1.M.d: ends at byte 80000"}, + {"composite in group past 65535", + "", + "", + "message C { bytes a = 1 [(sbe.length) = 40000]; } message E { C c = 1; C d = 2; } message " + "M { option (sbe.template_id) = 1; repeated E e = 1; }", + "composite in group bounds.v1.E.d: ends at byte 80000"}, + {"nested composite past 65535", + "", + "", + "message D { bytes a = 1 [(sbe.length) = 40000]; } message C { D x = 1; D y = 2; } message " + "M { option (sbe.template_id) = 1; C c = 1; }", + "composite field bounds.v1.C.y: ends at byte 80000"}, + {"template_id past 65535", + "", + "", + "message M { option (sbe.template_id) = 65537; uint32 t = 1; }", + "message bounds.v1.M: (sbe.template_id) = 65537 is past 65535"}, + {"template_id at 65535", + "", + "", + "message M { option (sbe.template_id) = 65535; uint32 t = 1; }", + "", + 4}, + {"schema_id past 65535", + "65536", + "", + "message M { option (sbe.template_id) = 1; uint32 t = 1; }", + "file bounds.proto: (sbe.schema_id) = 65536 is past 65535"}, + {"schema_id at 65535", + "65535", + "", + "message M { option (sbe.template_id) = 1; uint32 t = 1; }", + "", + 4}, + {"version past 65535", + "", + "65536", + "message M { option (sbe.template_id) = 1; uint32 t = 1; }", + "file bounds.proto: (sbe.version) = 65536 is past 65535"}, + {"version at 65535", + "", + "65535", + "message M { option (sbe.template_id) = 1; uint32 t = 1; }", + "", + 4}, + }; + for (const Case& tc : cases) { + SCOPED_TRACE(tc.name); + const std::string schema_id = tc.schema_id.empty() ? "1" : tc.schema_id; + const std::string version = tc.version.empty() ? "0" : tc.version; + SbeFiles files; + if (!BoundsFile(&files, schema_id, version, tc.body)) { + ADD_FAILURE() << files.error(); + continue; + } + auto codec = Codec::New(files.files()); + if (!tc.reject.empty()) { + if (codec.ok()) { + ADD_FAILURE() << "bound; want " << tc.reject; + continue; + } + EXPECT_NE(codec.status().message().find(tc.reject), std::string::npos) + << codec.status().message(); + continue; + } + if (!codec.ok()) { + ADD_FAILURE() << codec.status().ToString(); + continue; + } + + const pb::Descriptor* md = files.Message("bounds.v1.M"); + auto msg = files.New(md); + if (const pb::FieldDescriptor* e = md->FindFieldByName("e"); e != nullptr) { + msg->GetReflection()->AddMessage(msg.get(), e); + } + auto data = codec->Marshal(*msg); + if (!data.ok()) { + ADD_FAILURE() << data.status().ToString(); + continue; + } + + // The header states exactly what the schema declared. + if (data->size() < 8) { + ADD_FAILURE() << "short header"; + continue; + } + if (tc.block_size != 0) EXPECT_EQ(Le16(*data, 0), tc.block_size) << "blockLength"; + EXPECT_EQ(std::to_string(Le16(*data, 4)), schema_id) << "schemaId"; + EXPECT_EQ(std::to_string(Le16(*data, 6)), version) << "version"; + + auto back = codec->UnmarshalDescriptor(*data, md); + EXPECT_TRUE(back.ok()) << back.status().ToString(); + } +} + +// TestMarshalGroupCountBound: numInGroup is a uint16. Before the fix 65536 +// entries were written with a count of 0, and the message read back with no +// entries and no error. +TEST(SbeHeaderBounds, MarshalGroupCount) { + SbeFiles files; + ASSERT_TRUE(BoundsFile( + &files, + "1", + "0", + "message E { uint32 a = 1; } message M { option (sbe.template_id) = 1; repeated E e = 1; }")) + << files.error(); + auto codec = Codec::New(files.files()); + ASSERT_TRUE(codec.ok()) << codec.status().ToString(); + const pb::Descriptor* md = files.Message("bounds.v1.M"); + const pb::FieldDescriptor* efd = md->FindFieldByName("e"); + + const auto with_entries = [&](int n) { + auto msg = files.New(md); + for (int i = 0; i < n; ++i) msg->GetReflection()->AddMessage(msg.get(), efd); + return msg; + }; + + auto data = codec->Marshal(*with_entries(65535)); + ASSERT_TRUE(data.ok()) << data.status().ToString(); + EXPECT_EQ(Le16(*data, 8 + 2), 65535) << "numInGroup"; + auto back = codec->UnmarshalDescriptor(*data, md); + ASSERT_TRUE(back.ok()) << back.status().ToString(); + EXPECT_EQ((*back)->GetReflection()->FieldSize(**back, efd), 65535); + + auto past = codec->Marshal(*with_entries(65536)); + ASSERT_FALSE(past.ok()); + EXPECT_NE(past.status().message().find("field bounds.v1.M.e has 65536 entries, past 65535"), + std::string::npos) + << past.status().message(); +} + +} // namespace