Support for human-readable formats?

[fjarri]

While this crate helps serialize bytes in binary formats efficiently, human-readable formats will still have them as vectors of integers. Is efficient serialization in human-readable formats something potentially fitting for this crate, or should it be implemented elsewhere?

That may include:

• support for hex/base64 encoding
• multibase support
• selection of the specific representation for the encoding (e.g. prefix or no prefix, or lowercase or uppercase for hex)

[jonasbb]

This crate is just about the serde internal data format and does not control how the format exposes it. Many human-readable formats only have the option of using a list of integers, since using base64 would make the value a string. The Ron format will serialize bytes using base64.

You can, of course, force a specific string encoding like hex or base64, but then it will be stored as a string in the serialized form. There are already many crates offering this which you can use very similar to serde_bytes.

[fjarri]

Well, yes, it will pass the data as a string to the specific format implementation. There will be some inefficiency if the format supports bytestrings natively, but "no encoding" can be one of the options.

There are already many crates offering this which you can use very similar to serde_bytes.

Could you name some? And the problem is that I cannot use both serde_bytes and a hypothetical crate dealing with human-readable formats; the choice has to be done inside serialize() based on serializer.is_human_readable().

[jonasbb]

Could you name some?

Sure, quick search for the keywords serde, base64, hex reveals at least these crates base64-serde, hex-buffer-serde, hex-serde, serde_with, serde-bytes-repr, serde-hex, serdect, stremio-serde-hex.

If you need to rely on is_human_readable you might need to adapt them a bit for your use case.

[fjarri]

base64-serde, serde-hex, stremio-serde-hex

Do not support efficient serialization in binary formats.

hex-buffer-serde, hex-serde

No base64 support, and never will be given the name.

serde_with

If you are referring to "De/Serializing a type using the Display and FromStr traits", this also means no binary format support. (And, also, one would have to use wrapper types)

serde-bytes-repr

Weird composition required with a lot of boilerplate.

serdect

Specifically focused on constant-timeness.

To summarize, since serde does not allow easy composition of two crates for handling binary and human-readable formats, this functionality has to be made available from one crate. And this one seems like the best place for it. Of course I could fork it and add the required functionality, and thus dilute the ecosystem even more, but before doing that I'd like to at least hear from one of the maintainers.

[dzmitry-lahoda]

CosmWasm has special types for human redabale base64 and hex stings. So it is useful in some areas to have.

[fjarri]

For people encountering this issue, I've published https://crates.io/crates/serde-encoded-bytes. I think this is not going to be implemented in serde-bytes, so this issue can be closed.