Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/concepts/extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,26 @@ Extensions are opinionated, higher-level abstractions built on RedisVL's core pr

You don't need to understand schemas, indexes, or queries to use extensions—they handle that complexity for you.

## Choosing a Vectorizer

`SemanticCache`, `SemanticMessageHistory` and `SemanticRouter` embed text, so each needs a {doc}`vectorizer </concepts/utilities>`. Pass one explicitly:

```python
from redisvl.extensions.cache.llm import SemanticCache
from redisvl.utils.vectorize import HFTextVectorizer

cache = SemanticCache(
name="llmcache",
vectorizer=HFTextVectorizer(model="redis/langcache-embed-v2"),
)
```

`redis/langcache-embed-v2` is trained for matching cache prompts. Any vectorizer works, including hosted providers that need no local model.

Leaving the argument out is deprecated. The extension still falls back to a local Hugging Face model, which needs the `sentence-transformers` extra and downloads the model on first use: `redis/langcache-embed-v1` for `SemanticCache`, and `sentence-transformers/all-mpnet-base-v2` for the other two. It also emits a `DeprecationWarning` naming the model. A future major release will require the argument.

Stored data only matches queries embedded with the model that wrote it. If you relied on the fallback, pass that same model explicitly to keep using what you have stored. Two models with the same output size, such as `langcache-embed-v1` and `-v2`, fit the same index without an error, so a mismatch shows up only as wrong matches. To move a cache to a different model of the same output size, call `clear()` and let it refill; if the size differs, call `delete()` and construct the cache again with the new vectorizer. `overwrite=True` rebuilds the index but keeps the stored entries and their old vectors. Distances also differ between models, so check your distance threshold after switching.

## Semantic Cache

LLM API calls are expensive and slow. If users ask similar questions, you're paying to generate similar answers repeatedly. Semantic caching solves this by storing responses and returning cached answers when similar prompts are seen again.
Expand Down
7 changes: 6 additions & 1 deletion docs/user_guide/07_message_history.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,12 @@
"outputs": [],
"source": [
"from redisvl.extensions.message_history import SemanticMessageHistory\n",
"semantic_history = SemanticMessageHistory(name='tutor')\n",
"from redisvl.utils.vectorize import HFTextVectorizer\n",
"\n",
"semantic_history = SemanticMessageHistory(\n",
" name='tutor',\n",
" vectorizer=HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2'),\n",
")\n",
"\n",
"semantic_history.add_messages(chat_history.get_recent(top_k=8))"
]
Expand Down
7 changes: 6 additions & 1 deletion docs/user_guide/how_to_guides/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

from redisvl.extensions.cache.llm import SemanticCache
from redisvl.utils.vectorize import HFTextVectorizer

tracer_provider = TracerProvider()
tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
Expand All @@ -48,7 +49,11 @@ metrics.set_meter_provider(
MeterProvider(metric_readers=[PeriodicExportingMetricReader(OTLPMetricExporter())])
)

cache = SemanticCache(name="llmcache", redis_url="redis://localhost:6379")
cache = SemanticCache(
name="llmcache",
vectorizer=HFTextVectorizer(model="redis/langcache-embed-v2"),
redis_url="redis://localhost:6379",
)
cache.store(prompt="What is the capital of France?", response="Paris")
cache.check(prompt="What's the capital of France?")
```
Expand Down
3 changes: 3 additions & 0 deletions docs/user_guide/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,9 @@ RedisVL does not guess its way around this. A credential that cannot ask whether
```python
cache = SemanticCache(
name="llmcache",
# Use the model that wrote the cache: the schema check is skipped, so a
# different model of the same size matches the wrong entries silently.
vectorizer=HFTextVectorizer(model="redis/langcache-embed-v2"),
redis_url="redis://localhost:6379",
create_index=False,
)
Expand Down
53 changes: 29 additions & 24 deletions redisvl/extensions/cache/llm/semantic.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from typing import Any
from warnings import warn

from pydantic import ValidationError
from redis import Redis
Expand All @@ -21,6 +22,7 @@
REDIS_KEY_FIELD_NAME,
RESPONSE_FIELD_NAME,
UPDATED_AT_FIELD_NAME,
implicit_vectorizer_deprecation,
)
from redisvl.index import AsyncSearchIndex, SearchIndex
from redisvl.query import VectorQuery
Expand All @@ -40,17 +42,9 @@

logger = get_logger("[RedisVL]")

# Emitted when the caller did not choose a vectorizer and the index predates this
# call, so its vectors may have been written with the old default model. Remove
# this warning in future releases.
DEFAULT_VECTORIZER_CHANGED_WARNING = (
"The default vectorizer has changed from `sentence-transformers/all-mpnet-base-v2` "
"to `redis/langcache-embed-v1` in version 0.6.0 of RedisVL. "
"For more information about this model, please refer to https://arxiv.org/abs/2504.02268 "
"or visit https://huggingface.co/redis/langcache-embed-v1. "
"To continue using the old vectorizer, please specify it explicitly in the constructor as: "
"vectorizer=HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2')"
)
# Used only when the caller passes no vectorizer, which is deprecated. It stays
# at the model existing caches were written with until the fallback is removed.
DEFAULT_VECTORIZER_MODEL = "redis/langcache-embed-v1"


class SemanticCache(BaseLLMCache):
Expand Down Expand Up @@ -86,7 +80,11 @@ def __init__(
ttl (Optional[int], optional): The time-to-live for records cached
in Redis. Defaults to None.
vectorizer (Optional[BaseVectorizer], optional): The vectorizer for the cache.
Defaults to HFTextVectorizer.
Pass one explicitly. Omitting it is deprecated: it falls back to
``HFTextVectorizer(model="redis/langcache-embed-v1")``
with a ``DeprecationWarning``, and a future major release will make it
required. Data already stored under the fallback only matches queries
embedded with that same model.
filterable_fields (Optional[List[Dict[str, Any]]]): An optional list of RedisVL fields
that can be used to customize cache retrieval with filters.
redis_client(Optional[Redis], optional): A redis client connection instance.
Expand Down Expand Up @@ -122,14 +120,23 @@ def __init__(
.. code-block:: python

from redisvl.extensions.cache.llm import SemanticCache
from redisvl.utils.vectorize import HFTextVectorizer

vectorizer = HFTextVectorizer(model="redis/langcache-embed-v2")

# RedisVL creates the index if it is missing
cache = SemanticCache(name="llmcache", redis_url="redis://localhost:6379")
cache = SemanticCache(
name="llmcache",
vectorizer=vectorizer,
redis_url="redis://localhost:6379",
)

# the index is managed externally, or this credential cannot run
# FT.INFO -- assume the index exists and issue no index command
# FT.INFO -- assume the index exists and issue no index command.
# The vectorizer must be the model that populated the index.
cache = SemanticCache(
name="llmcache",
vectorizer=vectorizer,
redis_url="redis://localhost:6379",
create_index=False,
)
Expand Down Expand Up @@ -164,14 +171,20 @@ def __init__(
)
self._vectorizer = vectorizer
else:
# Create the default vectorizer
warn(
implicit_vectorizer_deprecation(
"SemanticCache", DEFAULT_VECTORIZER_MODEL, dtype
),
DeprecationWarning,
stacklevel=3,
)
vectorizer_kwargs = kwargs

if dtype:
vectorizer_kwargs.update(dtype=dtype)

self._vectorizer = HFTextVectorizer(
model="redis/langcache-embed-v1",
model=DEFAULT_VECTORIZER_MODEL,
**vectorizer_kwargs,
)

Expand Down Expand Up @@ -209,9 +222,6 @@ def __init__(

if create_index:
if not self.overwrite and self._index.exists():
if not vectorizer:
logger.warning(DEFAULT_VECTORIZER_CHANGED_WARNING)

existing_index = SearchIndex.from_existing(
name, redis_client=self._index._redis_client
)
Expand All @@ -224,11 +234,6 @@ def __init__(
# Create the search index in Redis
self._index.create(overwrite=self.overwrite, drop=False)
else:
# The flag asserts the index already exists, which is exactly this
# warning's precondition -- so it still applies here.
if not vectorizer:
logger.warning(DEFAULT_VECTORIZER_CHANGED_WARNING)

logger.debug(
f"create_index=False: assuming index {name!r} exists over prefix "
f"{schema.index.prefix!r} with {self._vectorizer.dims} vector "
Expand Down
22 changes: 22 additions & 0 deletions redisvl/extensions/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,25 @@
"path to drop it. To remove every entry while leaving the index in place, "
"use clear()."
)

# Warned by SemanticCache, SemanticMessageHistory and SemanticRouter when the
# caller passes no vectorizer. The fallback stays unchanged until it is removed,
# so data stored under it keeps matching.
IMPLICIT_VECTORIZER_DEPRECATED: str = (
"{owner} was created without a vectorizer, so it fell back to {call}. This "
"fallback is deprecated, and a future major release will require the "
"vectorizer argument. Pass a vectorizer explicitly; to keep matching data "
"already stored under the fallback, pass the same one: vectorizer={call}."
)


def implicit_vectorizer_deprecation(owner: str, model: str, dtype: str | None) -> str:
"""Format IMPLICIT_VECTORIZER_DEPRECATED with the exact fallback call.

``dtype`` is included when the caller set it, because a replacement that
drops it embeds at a different width than the data already stored.
"""
args = f"model='{model}'" + (f", dtype='{dtype}'" if dtype else "")
return IMPLICIT_VECTORIZER_DEPRECATED.format(
owner=owner, call=f"HFTextVectorizer({args})"
)
20 changes: 19 additions & 1 deletion redisvl/extensions/message_history/semantic_history.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from typing import Any
from warnings import warn

from redis import Redis

Expand All @@ -13,6 +14,7 @@
SESSION_FIELD_NAME,
TIMESTAMP_FIELD_NAME,
TOOL_FIELD_NAME,
implicit_vectorizer_deprecation,
)
from redisvl.extensions.message_history import BaseMessageHistory
from redisvl.extensions.message_history.schema import (
Expand All @@ -34,6 +36,10 @@

logger = get_logger(__name__)

# Used only when the caller passes no vectorizer, which is deprecated. It stays
# at the model existing data was written with until the fallback is removed.
DEFAULT_VECTORIZER_MODEL = "sentence-transformers/all-mpnet-base-v2"


class SemanticMessageHistory(BaseMessageHistory):

Expand Down Expand Up @@ -66,6 +72,11 @@ def __init__(
prefix (Optional[str]): Prefix for the keys for this message data.
Defaults to None and will be replaced with the index name.
vectorizer (Optional[BaseVectorizer]): The vectorizer used to create embeddings.
Pass one explicitly. Omitting it is deprecated: it falls back to
``HFTextVectorizer(model="sentence-transformers/all-mpnet-base-v2")``
with a ``DeprecationWarning``, and a future major release will make it
required. Data already stored under the fallback only matches queries
embedded with that same model.
distance_threshold (float): The maximum semantic distance to be
included in the context, in Redis COSINE units [0-2]. Defaults
to 0.3.
Expand Down Expand Up @@ -115,13 +126,20 @@ def __init__(
f"Provided dtype {dtype} does not match vectorizer dtype {vectorizer.dtype}"
)
else:
warn(
implicit_vectorizer_deprecation(
"SemanticMessageHistory", DEFAULT_VECTORIZER_MODEL, dtype
),
DeprecationWarning,
stacklevel=3,
)
vectorizer_kwargs = kwargs

if dtype:
vectorizer_kwargs.update(**{"dtype": dtype})

vectorizer = HFTextVectorizer(
model="sentence-transformers/all-mpnet-base-v2",
model=DEFAULT_VECTORIZER_MODEL,
**vectorizer_kwargs,
)

Expand Down
40 changes: 36 additions & 4 deletions redisvl/extensions/router/semantic.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from pathlib import Path
from typing import Any, NoReturn, Self
from warnings import warn

import redis.commands.search.reducers as reducers
import yaml
Expand All @@ -12,6 +13,7 @@
CREATE_INDEX_OVERWRITE_CONFLICT,
EXTERNAL_INDEX_DROP_CONFLICT,
ROUTE_VECTOR_FIELD_NAME,
implicit_vectorizer_deprecation,
)
from redisvl.extensions.router.schema import (
DistanceAggregationMethod,
Expand Down Expand Up @@ -40,6 +42,10 @@

logger = get_logger(__name__)

# Used only when the caller passes no vectorizer, which is deprecated. It stays
# at the model existing data was written with until the fallback is removed.
DEFAULT_VECTORIZER_MODEL = "sentence-transformers/all-mpnet-base-v2"

_DISTANCE_THRESHOLD_ADAPTER = TypeAdapter(DistanceThreshold)


Expand Down Expand Up @@ -82,7 +88,12 @@ def __init__(
Args:
name (str): The name of the semantic router.
routes (List[Route]): List of Route objects.
vectorizer (BaseVectorizer, optional): The vectorizer used to embed route references. Defaults to default HFTextVectorizer.
vectorizer (BaseVectorizer, optional): The vectorizer used to embed route references.
Pass one explicitly. Omitting it is deprecated: it falls back to
``HFTextVectorizer(model="sentence-transformers/all-mpnet-base-v2")``
with a ``DeprecationWarning``, and a future major release will make it
required. Data already stored under the fallback only matches queries
embedded with that same model.
routing_config (RoutingConfig, optional): Configuration for routing behavior. Defaults to the default RoutingConfig.
redis_client (Optional[SyncRedisClient], optional): Redis client for connection. Defaults to None.
redis_url (str, optional): The redis url. Defaults to redis://localhost:6379.
Expand Down Expand Up @@ -121,13 +132,20 @@ def __init__(
f"Provided dtype {dtype} does not match vectorizer dtype {vectorizer.dtype}"
)
else:
warn(
implicit_vectorizer_deprecation(
"SemanticRouter", DEFAULT_VECTORIZER_MODEL, dtype
),
DeprecationWarning,
stacklevel=3,
)
vectorizer_kwargs = kwargs

if dtype:
vectorizer_kwargs.update(**{"dtype": dtype})

vectorizer = HFTextVectorizer(
model="sentence-transformers/all-mpnet-base-v2",
model=DEFAULT_VECTORIZER_MODEL,
**vectorizer_kwargs,
)

Expand Down Expand Up @@ -994,7 +1012,16 @@ def to_dict(self) -> dict[str, Any]:
.. code-block:: python

from redisvl.extensions.router import SemanticRouter
router = SemanticRouter(name="example_router", routes=[], redis_url="redis://localhost:6379")
from redisvl.utils.vectorize import HFTextVectorizer

router = SemanticRouter(
name="example_router",
routes=[],
vectorizer=HFTextVectorizer(
model="sentence-transformers/all-mpnet-base-v2"
),
redis_url="redis://localhost:6379",
)
router_dict = router.to_dict()
"""
return {
Expand Down Expand Up @@ -1058,10 +1085,15 @@ def to_yaml(self, file_path: str, overwrite: bool = True) -> None:
.. code-block:: python

from redisvl.extensions.router import SemanticRouter
from redisvl.utils.vectorize import HFTextVectorizer

router = SemanticRouter(
name="example_router",
routes=[],
redis_url="redis://localhost:6379"
vectorizer=HFTextVectorizer(
model="sentence-transformers/all-mpnet-base-v2"
),
redis_url="redis://localhost:6379",
)
router.to_yaml("router.yaml")
"""
Expand Down
Loading
Loading