Redis Vector Storage Adapter

Note

(December, 2025): The RedisVectorStorageAdapter is new and experimental functionality introduced as a “beta” feature. Its functionality might not yet be fully stable and is subject to change in future releases.

The RedisVectorStorageAdapter enables advanced semantic similarity search for ChatterBot using Redis® as a vector database. Unlike traditional keyword-based storage adapters, this adapter uses vector embeddings and cosine similarity to understand conversational context and find semantically related responses.

Key Features:

• Semantic Understanding: Matches responses based on meaning, not just keywords

• Vector Embeddings: Uses HuggingFace sentence-transformers/all-mpnet-base-v2 model for state-of-the-art text encoding

• Confidence Scoring: Returns similarity scores (0.0-1.0) based on vector distance for intelligent response ranking

• Performance Optimized: Automatic NoOpTagger eliminates unnecessary spaCy processing overhead

• Context-Aware Responses: Finds conversationally appropriate responses even when exact words differ

Vectors are mathematical representations of text (multi-dimensional embeddings) that capture semantic meaning. The adapter calculates similarity between text by measuring the cosine distance between their vector representations, allowing ChatterBot to understand that “How are you?” is similar to “How’s it going?” even with different words.

For example, consider the following words:

        (Speaking)
            ●
           / \
          /   \
(Poetry) ●-----● (Rhyming)
          \   /
           \ /
            ●
        (Writing)

The acts of “speaking” and “writing” are both forms of communication, so they are included in the same cluster, but they are somewhat opposite to each other. Both “poetry” and “rhyming” closely related, and in some cases might possibly be used as synonyms within the context of either types of speech or types of writing.

Redis Setup

Before you use the RedisVectorStorageAdapter you will need to install the dependencies required for Redis and generating vectors. This can be done using the chatterbot[redis] extra when installing ChatterBot. For example:

pip install chatterbot[redis]

You will also need to have a Redis server running, with the additional modules installed that enable searching using vectors. And easy way to run one locally is to use Docker:

docker-compose.yml

services:
  redis:
    # Use the latest version of the redis-stack image
    image: redis/redis-stack-server:latest
    # Expose the default Redis port
    ports:
      - "6379:6379"
    # Persist the Redis data
    volumes:
      - ./.database/redis/:/data

To start the Redis container, run:

docker compose up -d

Likewise, you can run docker compose ps to review the status of your container, and docker compose down to stop it. For more information on Docker and docker compose, see the Docker Compose documentation.

Redis Configuration

To use the RedisVectorStorageAdapter you will need to provide the following argument when configuring your ChatterBot instance:

from chatterbot import ChatBot

chatbot = ChatBot(
      'Redis Bot',
      storage_adapter='chatterbot.storage.RedisVectorStorageAdapter',
      # Optional: Override the default Redis URI
      # database_uri='redis://localhost:6379/0'
)

Storage-Aware Architecture

The Redis adapter automatically configures ChatterBot for optimal performance with vector-based search:

• Automatic Tagger Selection: Uses NoOpTagger instead of PosLemmaTagger to eliminate spaCy model loading overhead

• Semantic Vector Search: Automatically selects SemanticVectorSearch algorithm instead of text-based comparison

• No Manual Configuration: These optimizations are applied automatically when using the Redis adapter

This “storage-aware” design means ChatterBot adapts its processing pipeline based on the storage adapter’s capabilities, ensuring maximum performance and accuracy for vector-based semantic search.

# No need to specify tagger or search algorithm - Redis adapter handles it!
chatbot = ChatBot(
    'Semantic Bot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter'
)
# Automatically uses:
# - NoOpTagger (no spaCy overhead)
# - SemanticVectorSearch (vector similarity)
# - Confidence scores from cosine similarity

Semantic Search vs. Traditional Text Search

The Redis adapter uses semantic vector search instead of traditional pattern matching:

Comparison: Semantic Vector Search vs. Text-Based Search

Feature	Traditional Text Search (SQL)	Semantic Vector Search (Redis)
Search Method	POS-lemma bigram matching	768-dimensional vector similarity
Context Understanding	Structural patterns only	Deep semantic meaning
“How are you?” matches “How’s it going?”	❌ No (different lemmas)	✅ Yes (similar meaning)
Confidence Scoring	Levenshtein distance	Cosine similarity (1 - distance/2)
Processing Overhead	Requires spaCy models	No spaCy needed (NoOpTagger)
Best For	Exact pattern matching	Conversational AI, context understanding

Example: Semantic Similarity in Action

# These inputs find similar responses despite different words:
response1 = chatbot.get_response("What's the weather like?")
response2 = chatbot.get_response("How's the climate today?")
# Both queries find weather-related responses due to semantic similarity

# Confidence scores help rank responses:
# - Vector distance 0.1 → confidence ~0.95 (very similar)
# - Vector distance 0.5 → confidence ~0.75 (somewhat similar)
# - Vector distance 1.0 → confidence ~0.50 (loosely related)

Class Attributes

class chatterbot.storage.RedisVectorStorageAdapter(**kwargs)

Warning

BETA feature (Released March, 2025): this storage adapter is new and experimental. Its functionality and default parameters might change in the future and its behavior has not yet been finalized.

The RedisVectorStorageAdapter allows ChatterBot to store conversation data in a redis instance using vector embeddings for semantic similarity search.

All parameters are optional, by default a redis instance on localhost is assumed.

Parameters:

• database_uri (str) – eg: redis://localhost:6379/0’, The database_uri can be specified to choose a redis instance.

• embedding_model (str) – The name of the embedding model to use. Default: ‘sentence-transformers/all-mpnet-base-v2’ (768-dim, balanced speed/quality). Alternatives: ‘all-MiniLM-L6-v2’ (384-dim, faster), ‘multi-qa-mpnet-base-dot-v1’ (768-dim, Q&A optimized), ‘paraphrase-multilingual-mpnet-base-v2’ (768-dim, multilingual).

• embedding_provider (str) – The embedding provider to use. Options: ‘huggingface’ (default), ‘openai’, ‘cohere’. Requires corresponding packages (langchain-openai, langchain-cohere).

• embedding_kwargs (dict) – Additional keyword arguments to pass to the embedding provider. For HuggingFace: model_kwargs (device, torch_dtype), encode_kwargs (normalize_embeddings, batch_size). For OpenAI: model name (e.g., ‘text-embedding-3-small’), dimensions. For Cohere: model name (e.g., ‘embed-english-v3.0’).

Architecture:

Unlike SQL storage adapters that use indexed text fields (search_text, search_in_response_to) for string-based similarity matching, Redis uses vector embeddings for semantic similarity. The ‘in_response_to’ field is embedded as a vector, enabling the system to find statements that respond to semantically similar inputs.

When used with SemanticVectorSearch, this adapter returns the best matching response directly from Phase 1 search. The semantic vector similarity already captures contextual closeness, making the traditional Phase 2 variation search (used in indexed text search) redundant.

For SQL with indexed text search:

• Phase 1 finds a match based on string similarity (Levenshtein distance)

• Phase 2 finds variations of that match to get diverse responses

• This makes sense because you might have multiple instances of similar statements learned from different conversations that provide different response options

For Redis with semantic vectors:

• Phase 1 finds semantically similar responses using vector embeddings

• The semantic similarity already captures the “closeness” we want

• Phase 2 would be redundant - we already have the best semantic match

• The vector search inherently considers the entire semantic space, not just exact string matches, so additional variation searching is unnecessary

NOTES:

• Unlike other database based storage adapters, the RedisVectorStorageAdapter does not leverage search_text and search_in_response_to fields for indexing. Instead, it uses vector embeddings to find similar statements based on semantic similarity. This allows for more flexible and context-aware matching.

class RedisMetaDataType

Subclass for redis config metadata type enumerator.

close()

Close the Redis client connection.

count() → int

Return the number of statement entries in the database.

create(text, in_response_to=None, tags=None, **kwargs)

Creates a new statement matching the keyword arguments specified. Returns the created statement.

create_many(statements)

Creates multiple statement entries.

drop()

Remove all existing documents from the database.

filter(page_size=4, **kwargs)

Returns a list of objects from the database. The kwargs parameter can contain any number of attributes. Only objects which contain all listed attributes and in which all values match for all listed attributes will be returned.

kwargs:

• conversation

• persona

• tags

• in_response_to

• text

• exclude_text

• exclude_text_words

• persona_not_startswith

• search_text_contains

• search_in_response_to_contains

• order_by

get_preferred_search_algorithm()

Redis uses semantic vector search instead of text-based matching. Returns the name of the SemanticVectorSearch algorithm.

get_preferred_tagger()

Redis uses vector embeddings and doesn’t need POS-lemma indexing. Returns NoOpTagger to avoid unnecessary spaCy processing.

get_random()

Returns a random statement from the database.

get_statement_model()

Return the statement model.

remove(statement)

Removes the statement that matches the input text. Removes any responses from statements where the response text matches the input text.

update(statement)

Modifies an entry in the database. Creates an entry if one does not exist.

Performance Considerations

Vector Embedding Model: By default, the Redis adapter uses sentence-transformers/all-mpnet-base-v2 from HuggingFace:

• Dimensions: 768-dimensional embeddings

• Model Size: ~420MB (downloaded once, cached locally)

• Performance: ~2000 sentences/second on CPU

• Quality: State-of-the-art semantic similarity (as of 2025)

First-Time Setup: The embedding model downloads automatically on first use:

# First initialization downloads model (~420MB)
chatbot = ChatBot('Bot', storage_adapter='chatterbot.storage.RedisVectorStorageAdapter')
# Subsequent uses load from cache (fast startup)

Memory Usage: Redis vector storage requires more memory than SQL due to embedding storage:

• Each statement: ~3KB (768 floats × 4 bytes)

• 10,000 statements: ~30MB vector data

• Trade-off: Higher memory for better semantic understanding

Embedding Model Configuration

Added in version 1.2.8: Support for configurable embedding models and providers.

The Redis adapter now supports custom embedding models and providers, allowing you to optimize for different use cases:

Default Configuration

By default, the adapter uses HuggingFace’s sentence-transformers/all-mpnet-base-v2:

chatbot = ChatBot(
    'Bot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter'
)
# Uses: sentence-transformers/all-mpnet-base-v2 (768-dim, balanced)

Alternative HuggingFace Models

Choose different models based on your requirements:

Fast/Lightweight Model (for high-throughput applications):

chatbot = ChatBot(
    'FastBot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter',
    embedding_model='all-MiniLM-L6-v2'
)
# Model: all-MiniLM-L6-v2
# Dimensions: 384 (vs 768 default)
# Size: 80MB (vs 420MB default)
# Speed: 5x faster
# Quality: ~10% lower accuracy

Q&A Optimized Model (for question-answering chatbots):

chatbot = ChatBot(
    'QABot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter',
    embedding_model='multi-qa-mpnet-base-dot-v1'
)
# Model: multi-qa-mpnet-base-dot-v1
# Trained specifically on Q&A datasets
# Better performance on question-answer pairs

Multilingual Model (for multi-language support):

chatbot = ChatBot(
    'MultilingualBot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter',
    embedding_model='paraphrase-multilingual-mpnet-base-v2'
)
# Model: paraphrase-multilingual-mpnet-base-v2
# Supports 50+ languages
# Same 768 dimensions as default

Advanced Embedding Configuration

Pass additional parameters to the embedding model:

chatbot = ChatBot(
    'CustomBot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter',
    embedding_model='all-MiniLM-L6-v2',
    embedding_kwargs={
        'model_kwargs': {
            'device': 'cpu',  # or 'cuda' for GPU
            'torch_dtype': 'float16'  # Reduce memory usage
        },
        'encode_kwargs': {
            'normalize_embeddings': True,  # L2 normalization
            'batch_size': 32  # Process 32 texts at once
        }
    }
)

Alternative Embedding Providers

OpenAI Embeddings (cloud-based, requires API key):

# Requires: pip install langchain-openai
# Set: export OPENAI_API_KEY="your-api-key"

chatbot = ChatBot(
    'OpenAIBot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter',
    embedding_provider='openai',
    embedding_model='text-embedding-3-small',
    embedding_kwargs={'dimensions': 1536}
)
# Pros: High quality, fast API calls
# Cons: Costs money per token, requires internet

Cohere Embeddings (cloud-based, requires API key):

# Requires: pip install langchain-cohere
# Set: export COHERE_API_KEY="your-api-key"

chatbot = ChatBot(
    'CohereBot',
    storage_adapter='chatterbot.storage.RedisVectorStorageAdapter',
    embedding_provider='cohere',
    embedding_model='embed-english-v3.0'
)
# Pros: Optimized for semantic search
# Cons: Subscription-based, requires internet

Model Selection Guide

Embedding Model Comparison

Use Case	Recommended Model	Dimensions	Trade-offs
General chatbot (default)	all-mpnet-base-v2	768	Best balance of speed and quality
High-throughput / limited resources	all-MiniLM-L6-v2	384	5x faster, smaller size, slight quality loss
Question-answering bot	multi-qa-mpnet-base-dot-v1	768	Better Q&A performance, same size as default
Multilingual bot	paraphrase-multilingual-mpnet-base-v2	768	Supports 50+ languages, same size
Cloud/Production (API)	OpenAI or Cohere	1024-1536	Higher quality, costs money, requires API key

Example: Complete Configuration

See examples/redis_embedding_examples.py for complete working examples of all embedding configurations.

More on Vector Databases & Semantic Search

For those looking to learn more about vector databases, vector embeddings, and semantic search in AI applications:

Vector Database & AI Learning Resources

Topic	Resource Link
What is a vector database?	https://www.mongodb.com/resources/basics/databases/vector-databases
Why use a vector database?	https://stackoverflow.blog/2023/09/20/do-you-need-a-specialized-vector-database-to-implement-vector-search-well/
How to choose a vector database?	https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/vector-search-ai
Redis as a vector database	https://redis.io/docs/latest/develop/interact/search-and-query/advanced-concepts/vectors/
Sentence Transformers (Embeddings)	https://www.sbert.net/
Understanding Cosine Similarity	https://en.wikipedia.org/wiki/Cosine_similarity
Vector Search for AI/LLMs	https://www.pinecone.io/learn/vector-search-basics/

_{* Redis is a registered trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd.}

Ask a question under the chatterbot tag	Report an issue on GitHub
Subscribe to updates via @chatterbot.us on Bluesky