Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,31 @@
# 0.5.0

## Performance

- perf: values are no longer run through `:erlang.term_to_binary/1` for adapters that store Erlang terms natively (`Cache.ETS`, `Cache.Agent`, `Cache.PersistentTerm`, `Cache.ConCache`, `Cache.Counter`). The encode/decode round trip was pure overhead on those adapters, and the decode dominated the lookup it was attached to. On a 500k-entry ETS table, `get/1` of a ~10KB body drops from **29,981 ns to 6,211 ns (4.8x)** and `put/2` from **23,542 ns to 10,804 ns (2.2x)**. The win holds across payload sizes — a small map goes from 2,105 ns to 302 ns (7.0x).
- `Cache.PersistentTerm` now hands out the stored term itself on every read, restoring the zero-copy property the adapter exists for.

## Features

- feat: added the optional `c:Cache.native_term_storage?/1` callback, letting an adapter declare that it stores Erlang terms natively. It is resolved at compile time, so there is no runtime branch on the read or write path. The callback is optional and defaults to encoding, so third-party adapters keep their current behaviour unchanged.

## Bug Fixes

- fix: `:compression_level` is reachable. It was unusable on every path — no adapter declares it, so `NimbleOptions` rejected it on compile-time adapter opts, and it resolved to `nil` before it could reach the encoder otherwise. It is now an option on the `use Cache` line (`compression_level: 6`), it is taken off the adapter opts before they are validated so the `opts: [compression_level: 6]` spelling works too, and it is never handed to the adapter. Setting it forces encoding on adapters that hold terms natively — asking for compression is asking for bytes. A cache using a strategy adapter raises at compile time rather than ignoring the option.
- fix: `Cache.ConCache.get_or_store/3` followed by `get/1` no longer raises. `get_or_store/3` writes through ConCache directly, bypassing the encode in `put/3`, so the matching `get/1` tried to `binary_to_term/1` a raw term.
- fix: a binary value wrapped in braces but not valid JSON (eg `"{oops}"`) no longer raises `Jason.DecodeError` on read. `Cache.TermEncoder.decode/1` used `Jason.decode!/1`, and now falls back to returning the binary unchanged.
- fix: raw `Cache.ETS` operations (`match_object/1`, `select/1`, `tab2list/0`, `foldl/2`) now see the terms that were `put`, rather than the opaque encoded binaries they used to return.
- fix: caching a JSON string hands back the string. `encode/2` stored a brace-wrapped binary unencoded, so `decode/1` had to guess what it was looking at — `put(:k, ~s({"a": 1}))` followed by `get(:k)` returned `%{"a" => 1}`, a `String` in and a `Map` out. Binaries are now always run through `:erlang.term_to_binary/1`, and `decode/1` keys off the external term format version byte rather than the shape of the payload, so nothing is guessed.
- fix: `decode/1` no longer raises on a binary that is not an encoded term. It used to reach `:erlang.binary_to_term/1` for anything that was not digits or brace-wrapped, which raised `ArgumentError` on a value written into the store by something other than this library.

## Breaking Changes

- Values held by native-term adapters are now stored as terms rather than encoded binaries. This is not observable through `get/1`, `put/3` and `delete/1`, which round-trip exactly as before. It is observable if you read the underlying store directly (`:ets.lookup/2`, `:persistent_term.get/1`, `ConCache.get/2`) or through the raw ETS API — those now return terms, which is what they were always meant to return.
- `Cache.DETS` is unchanged and still encodes, so existing `.dets` files stay readable. `Cache.ETS` with `:rehydration_path` also still encodes, so existing table dumps stay loadable.
- `Cache.HashRing`, and `Cache.MultiLayer` under `broadcast_mode: :replicate`, also still encode. Those strategies hand the stored value to another node, so a rolling deploy has 0.4.x and 0.5.x reading each other's writes for the same key and they have to agree on the representation. Their wire format is unchanged from 0.4.x and a mixed-version cluster is safe.
- An in-memory cache populated by an older version and read by this one would return raw binaries, but ETS, Agent, PersistentTerm and ConCache do not survive a restart, and every representation that outlives a node — disk, Redis, another node — is still encoded, so there is no upgrade path on which that can happen.
- A brace-wrapped or all-digit binary is now stored encoded rather than raw. Keys written by an earlier version are not in external term format, so they still decode the way they always did: a raw JSON string in Redis reads back as a map, a raw digit string as an integer. Only values written from this version on are type-stable. Code that was reading those keys out of Redis with another tool and expecting readable JSON gets an encoded term instead — write JSON through `json_set/3` (RedisJSON), which is a separate path and unchanged.

# 0.4.9

## Performance
Expand Down
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,27 @@ MyModule.put("key", "value") #> :ok
MyModule.get("key") #> {:ok, "value"}
```

### Compression

Values are serialised with `:erlang.term_to_binary/1` for adapters that cannot hold an
Erlang term. Pass a `compression_level` (`1..9`) to compress them on the way in:

```elixir
defmodule MyModule do
use Cache,
adapter: Cache.Redis,
name: :my_name,
compression_level: 6,
opts: [...opts]
end
```

The level belongs to the encoder rather than to the store, so it is never handed to the
adapter. Setting it also forces encoding on adapters that would otherwise hold the term
as it is (`Cache.ETS`, `Cache.Agent`, `Cache.PersistentTerm`, `Cache.ConCache`) — asking
for compression is asking for bytes. It is not supported on a cache using a strategy
adapter, which encodes through what it wraps; set it there instead.

## Adapters
- `Cache.Agent` - Simple agent based caching
- `Cache.DETS` - Disk persisted caching with [dets](https://www.erlang.org/doc/man/dets.html)
Expand Down
35 changes: 30 additions & 5 deletions guides/explanation/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,36 @@ When you `use Cache` in your module, the macro:

## Term Encoding and Compression

ElixirCache includes internal term encoding functionality that handles serialization and deserialization of Elixir terms. This allows you to store complex Elixir data structures in any cache backend. The encoding system:

1. Uses Erlang's term_to_binary for efficient serialization
2. Applies configurable compression to reduce memory usage
3. Automatically handles decoding when retrieving values
ElixirCache lets you store any Elixir term in any backend. How a term gets there depends on
what the backend can actually hold, and that is decided per adapter rather than globally.

Adapters that store Erlang terms natively — `Cache.ETS`, `Cache.Agent`, `Cache.PersistentTerm`,
`Cache.ConCache`, `Cache.Counter` — receive the term as-is. Serialising it first would cost a
`term_to_binary/1` on every write and a `binary_to_term/1` on every read while buying nothing,
and on a hot read path that decode dominates the lookup it is attached to.

Adapters that cannot hold a term serialise it with `:erlang.term_to_binary/1`:

- `Cache.Redis` stores bytes on the wire, so terms must be serialised before they leave the node.
- `Cache.DETS` owns a durable on-disk format, and so does `Cache.ETS` when configured with
`:rehydration_path`. Both keep encoding so that files written by earlier versions stay readable.
- `Cache.HashRing` rpcs the stored value to the node that owns the key, and `Cache.MultiLayer`
under `broadcast_mode: :replicate` pushes it to the other nodes' layers. Both keep encoding:
during a rolling deploy two versions of this library read each other's writes for the same
key, so the representation has to stay stable across versions.

Setting `:compression_level` forces encoding on any adapter — an explicit request to compress
is honoured even when the backend could have held the term directly.

Adapters declare this through the optional `c:Cache.native_term_storage?/1` callback. It is
resolved once at compile time, so there is no runtime branch on the read or write path.
The callback is optional and defaults to encoding, so third-party adapters are unaffected.

An encoded value is read back by its format rather than by its shape. External term format
always begins with the version byte `131` and every value this library encodes is in it, so
decoding never guesses at a payload. A stored value that is *not* in that format was written
either by an older version or by something other than this library, and keeps the reading it
has always had — a raw JSON string decodes to a map, a raw digit string to an integer.

## Sandboxing for Tests

Expand Down
73 changes: 68 additions & 5 deletions lib/cache.ex
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,22 @@ defmodule Cache do
ErrorMessage.t_ok_res()
@callback delete(cache_name :: atom, key :: atom | String.t()) :: ErrorMessage.t_ok_res()

@doc """
Returns true when the adapter stores Erlang terms natively, given its resolved options.

Adapters that hold terms in memory (`Cache.ETS`, `Cache.Agent`, `Cache.PersistentTerm`,
`Cache.ConCache`, `Cache.Counter`) do not need values to be run through
`:erlang.term_to_binary/1` first — doing so costs a full encode on every write and a
full decode on every read while buying nothing. Adapters that store bytes on the wire
(`Cache.Redis`) or that own a durable on-disk format (`Cache.DETS`) must return `false`.

This callback is optional. Adapters that do not implement it are assumed to need
encoding, which is what every adapter did before this callback existed.
"""
@callback native_term_storage?(adapter_opts :: Keyword.t()) :: boolean()

@optional_callbacks native_term_storage?: 1

defmacro __using__(opts) do
sandbox_opt = Keyword.get(opts, :sandbox?, false)

Expand Down Expand Up @@ -96,7 +112,21 @@ defmodule Cache do
adapter_opts = check_adapter_opts.(unquote(opts[:opts]))

@adapter_opts adapter_opts
@compression_level if is_list(@adapter_opts), do: @adapter_opts[:compression_level]

# A strategy encodes inside the strategy module, through the adapter or the cache
# modules it wraps, so a `:compression_level` set here would never reach an
# encoder. Say that at compile time rather than quietly handing back uncompressed
# values.
if not is_nil(unquote(opts[:compression_level])) or
(is_list(@adapter_opts) and not is_nil(@adapter_opts[:compression_level])) do
raise ArgumentError, """
`:compression_level` is not supported on a cache using a strategy adapter.

Got: #{inspect(@cache_strategy_module)} in #{inspect(__MODULE__)}

Set it on the cache module or layer that does the storing instead.
"""
end

def cache_name, do: @cache_name
def cache_adapter, do: @cache_adapter
Expand Down Expand Up @@ -283,10 +313,33 @@ defmodule Cache do
pre_check_runtime_options.(adapter_opts)
end

adapter_opts = if opts[:sandbox?], do: [], else: check_adapter_opts.(opts[:opts])
# `:compression_level` belongs to the encoder, not to the store, so it comes off
# the adapter opts before they are validated. No adapter declares it, and an
# adapter that forwards its opts to the store would choke on an option that was
# never meant for it.
{configured_compression_level, configured_adapter_opts} =
case opts[:opts] do
configured_adapter_opts when is_list(configured_adapter_opts) ->
Keyword.pop(configured_adapter_opts, :compression_level)

configured_adapter_opts ->
{nil, configured_adapter_opts}
end

adapter_opts = if opts[:sandbox?], do: [], else: check_adapter_opts.(configured_adapter_opts)

@adapter_opts adapter_opts
@compression_level if is_list(@adapter_opts), do: @adapter_opts[:compression_level]
@compression_level opts[:compression_level] || configured_compression_level

# Resolved against the *configured* adapter and its *configured* opts rather than
# `@cache_adapter`/`@adapter_opts`, which are swapped out under `sandbox?: true`.
# A sandboxed cache must encode exactly like the adapter it stands in for, or a
# value would round-trip differently in test than in production.
#
# A compression level is an explicit request to compress, so it encodes even on an
# adapter that could have held the term as it is.
@cache_encode_terms? not is_nil(@compression_level) or
Cache.TermEncoder.encoding_required?(opts[:adapter], opts[:opts])

unquote(adapter_use_ast)

Expand Down Expand Up @@ -336,8 +389,18 @@ defmodule Cache do
@cache_adapter.child_spec({@cache_name, adapter_options()})
end

@compile {:inline, encode_value: 1, decode_value: 1}

if @cache_encode_terms? do
defp encode_value(value), do: Cache.TermEncoder.encode(value, @compression_level)
defp decode_value(value), do: Cache.TermEncoder.decode(value)
else
defp encode_value(value), do: value
defp decode_value(value), do: value
end

def put(key, ttl \\ nil, value) do
value = Cache.TermEncoder.encode(value, @compression_level)
value = encode_value(value)
key = maybe_sandbox_key(key)

:telemetry.span(
Expand Down Expand Up @@ -372,7 +435,7 @@ defmodule Cache do
res

{:ok, value} ->
{:ok, Cache.TermEncoder.decode(value)}
{:ok, decode_value(value)}

other ->
other
Expand Down
6 changes: 6 additions & 0 deletions lib/cache/agent.ex
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ defmodule Cache.Agent do
@impl Cache
def opts_definition, do: []

@doc """
Values are held as terms in the Agent's map, so encoding them is pure overhead.
"""
@impl Cache
def native_term_storage?(_opts), do: true

@impl Cache
def start_link(opts \\ []) do
with {:error, {:already_started, pid}} <- Agent.start_link(fn -> %{} end, opts) do
Expand Down
9 changes: 9 additions & 0 deletions lib/cache/con_cache.ex
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,15 @@ defmodule Cache.ConCache do
@impl Cache
def opts_definition, do: @opts_definition

@doc """
ConCache is ETS-backed and stores terms. Encoding is pure overhead, and it made
`get_or_store/3` unusable — that function writes through ConCache directly, bypassing
the encode in `Cache.put/3`, so a later `get/1` tried to `binary_to_term/1` a raw term
and raised.
"""
@impl Cache
def native_term_storage?(_opts), do: true

@impl Cache
def start_link(opts) do
cache_opts =
Expand Down
7 changes: 7 additions & 0 deletions lib/cache/counter.ex
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,13 @@ defmodule Cache.Counter do
@impl Cache
def opts_definition, do: @opts_definition

@doc """
Values are integers in an `:counters` array. `Cache.TermEncoder.encode/2` already
passed integers through untouched, so this only removes a no-op function call.
"""
@impl Cache
def native_term_storage?(_opts), do: true

@impl Cache
def start_link(opts) do
parent = self()
Expand Down
17 changes: 17 additions & 0 deletions lib/cache/dets.ex
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,23 @@ defmodule Cache.DETS do
@impl Cache
def opts_definition, do: @opts_definition

@doc """
DETS keeps encoding, even though `:dets.insert/2` takes terms and would serialise
them itself.

The reason is the file on disk. Every `.dets` file written by an earlier version of
this library holds `:erlang.term_to_binary/1` blobs as its values. Those files outlive
the upgrade — that is the entire point of a disk cache. If this adapter switched to
native terms, the first read after an upgrade would hand the caller a raw binary where
it expects a term, silently, with no error to trace it back to.

The cost of keeping it is one redundant serialisation of data that is already headed
for a disk write, which is dominated by the I/O. The cost of dropping it is corrupting
every existing on-disk cache. That trade is not close.
"""
@impl Cache
def native_term_storage?(_opts), do: false

@impl Cache
def start_link(opts) do
parent = self()
Expand Down
14 changes: 14 additions & 0 deletions lib/cache/ets.ex
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,20 @@ defmodule Cache.ETS do
@impl Cache
def opts_definition, do: @opts_definition

@doc """
`:ets.insert/2` stores terms, so encoding is pure overhead — and it actively breaks
the raw ETS API this adapter exposes (`match_object/1`, `select/1`, `tab2list/0`,
`foldl/2`), which sees opaque binaries instead of the terms that were put.

The exception is `:rehydration_path`. That option dumps the table to disk with
`:ets.tab2file/2` and reads it back on the next boot, so the table is a durable
format. Encoding is kept there for the same reason it is kept for `Cache.DETS`:
a table dumped by an older version holds `term_to_binary/1` blobs, and switching to
native terms would hand callers raw binaries where they expect terms.
"""
@impl Cache
def native_term_storage?(opts), do: is_nil(opts[:rehydration_path])

def opts_definition(opts) when is_list(opts) do
if Keyword.keyword?(opts) do
validate_and_normalize_ets_options(opts)
Expand Down
8 changes: 8 additions & 0 deletions lib/cache/persistent_term.ex
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,14 @@ defmodule Cache.PersistentTerm do
@impl Cache
def opts_definition, do: []

@doc """
`:persistent_term` exists to hand out zero-copy references to already-built terms.
Encoding a value into a binary and decoding it on every read defeats the entire
point of the adapter, so it never encodes.
"""
@impl Cache
def native_term_storage?(_opts), do: true

@impl Cache
def start_link(opts) do
Task.start_link(fn ->
Expand Down
7 changes: 7 additions & 0 deletions lib/cache/redis.ex
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,13 @@ defmodule Cache.Redis do
@impl Cache
def opts_definition, do: @opts_definition

@doc """
Redis stores bytes on the wire. Terms must be serialised before they leave the node,
so this adapter always encodes.
"""
@impl Cache
def native_term_storage?(_opts), do: false

@impl Cache
def start_link(opts) do
if !opts[:uri] do
Expand Down
Loading
Loading