Redis®*: Vector search and RAG

Semantic search, recommendations, and retrieval-augmented generation with Redis

👋 Welcome to the Stackhero documentation!

Stackhero provides a ready-to-use Redis cloud solution offering numerous advantages, including:

• Redis Commander web interface included.
• Unlimited message size and transfers.
• Updates made easy with just one click.
• Optimal performance and enhanced security thanks to a private, dedicated infrastructure.

Save time and make your life easier: it only takes 5 minutes to try Stackhero's Redis cloud hosting solution!

If you are developing semantic search, a recommendation engine, or a retrieval-augmented generation (RAG) pipeline, you need a reliable solution to store embeddings and quickly retrieve the closest matches to a query. Your Redis instance on Stackhero provides this functionality natively: the query engine indexes vectors and responds to similarity searches in just a few milliseconds. There is no need to add a separate vector database alongside Redis: everything runs in one place.

This approach is both productive and efficient thanks to two key features:

• Hybrid queries. You can combine vector similarity search with standard filters such as numeric ranges, tags, or full text, all within a single request. For example: "the 5 chunks closest to this question, but only from documents this user is allowed to read, published after January" can be retrieved in a single query, without chaining multiple calls.
• Simplified architecture. Embeddings, cache, and sessions all coexist, sharing credentials, backups, and monitoring. This reduces complexity and overhead.

Before you start

Enable the Search module (and JSON if you wish to store your documents as JSON) in the Modules section of your service configuration on the Stackhero dashboard. For a detailed step-by-step guide, see the Search and JSON guide.

Creating a vector index

To declare a vector field, specify its dimension and distance metric. The dimension should match your embedding model: for example, 1536 for OpenAI text-embedding-3-small, 768 for many open-source models, and so on.

FT.CREATE chunksIndex
  ON HASH
  PREFIX 1 chunk:
  SCHEMA
    content TEXT
    documentId TAG
    publishedAt NUMERIC
    embedding VECTOR HNSW 6
      TYPE FLOAT32
      DIM 1536
      DISTANCE_METRIC COSINE

HNSW creates a graph index that keeps queries fast even as your collection grows, making it a good choice above a few thousand vectors. For small collections where exact results are a priority, you can use FLAT for exhaustive search.

COSINE is the recommended metric for most text embedding models. You can also choose L2 or IP depending on your use case.

Querying

A K nearest neighbours (KNN) query is written as follows. The vector is supplied as raw little-endian float32 bytes:

FT.SEARCH chunksIndex "*=>[KNN 5 @embedding $queryVector AS score]"
  PARAMS 2 queryVector "<rawBytes>"
  SORTBY score
  RETURN 2 content score
  DIALECT 2

To combine similarity search with filters, simply replace * with a filter expression:

FT.SEARCH chunksIndex "(@documentId:{doc42} @publishedAt:[1735689600 +inf])=>[KNN 5 @embedding $queryVector AS score]"
  PARAMS 2 queryVector "<rawBytes>"
  SORTBY score
  DIALECT 2

Vector queries require DIALECT 2. If you omit this, the query uses the older syntax and returns an error.

A RAG pipeline in Node.js

import { createClient, SCHEMA_FIELD_TYPE, SCHEMA_VECTOR_FIELD_ALGORITHM } from 'redis';

const client = createClient({ url: process.env.STACKHERO_REDIS_URL_TLS });
await client.connect();

const DIMENSIONS = 1536;
const toBytes = embedding => Buffer.from(new Float32Array(embedding).buffer);

// 1. Create the index once, at startup
try {
  await client.ft.create(
    'chunksIndex',
    {
      content: SCHEMA_FIELD_TYPE.TEXT,
      documentId: SCHEMA_FIELD_TYPE.TAG,
      embedding: {
        type: SCHEMA_FIELD_TYPE.VECTOR,
        ALGORITHM: SCHEMA_VECTOR_FIELD_ALGORITHM.HNSW,
        TYPE: 'FLOAT32',
        DIM: DIMENSIONS,
        DISTANCE_METRIC: 'COSINE'
      }
    },
    { ON: 'HASH', PREFIX: 'chunk:' }
  );
}
catch (error) {
  if (!error.message.includes('Index already exists')) {
    throw error;
  }
}

// 2. Index your chunks (embedding comes from your model provider)
async function indexChunk({ id, documentId, content, embedding }) {
  await client.hSet(`chunk:${id}`, {
    content,
    documentId,
    embedding: toBytes(embedding)
  });
}

// 3. Retrieve the chunks closest to a question, restricted to one document
async function retrieve({ questionEmbedding, documentId, count = 5 }) {
  const results = await client.ft.search(
    'chunksIndex',
    `(@documentId:{${documentId}})=>[KNN ${count} @embedding $queryVector AS score]`,
    {
      PARAMS: { queryVector: toBytes(questionEmbedding) },
      SORTBY: 'score',
      RETURN: [ 'content', 'score' ],
      DIALECT: 2
    }
  );

  return results.documents.map(({ value }) => value.content);
}

// 4. Feed them to your model as context
const context = await retrieve({ questionEmbedding, documentId: 'doc42' });
const prompt = `Answer using only this context:\n\n${context.join('\n\n')}\n\nQuestion: ${question}`;

The same in Python

import os
import numpy as np
import redis
from redis.commands.search.field import TextField, TagField, VectorField
from redis.commands.search.index_definition import IndexDefinition, IndexType
from redis.commands.search.query import Query

DIMENSIONS = 1536

r = redis.from_url(os.environ['STACKHERO_REDIS_URL_TLS'])

# Create the index once, at startup
try:
    r.ft('chunksIndex').create_index(
        (
            TextField('content'),
            TagField('documentId'),
            VectorField(
                'embedding',
                'HNSW',
                {'TYPE': 'FLOAT32', 'DIM': DIMENSIONS, 'DISTANCE_METRIC': 'COSINE'},
            ),
        ),
        definition=IndexDefinition(prefix=['chunk:'], index_type=IndexType.HASH),
    )
except redis.ResponseError as error:
    if 'Index already exists' not in str(error):
        raise


def index_chunk(chunk_id, document_id, content, embedding):
    r.hset(
        f'chunk:{chunk_id}',
        mapping={
            'content': content,
            'documentId': document_id,
            'embedding': np.array(embedding, dtype=np.float32).tobytes(),
        },
    )


def retrieve(question_embedding, document_id, count=5):
    query = (
        Query(f'(@documentId:{{{document_id}}})=>[KNN {count} @embedding $queryVector AS score]')
        .sort_by('score')
        .return_fields('content', 'score')
        .dialect(2)
    )
    results = r.ft('chunksIndex').search(
        query,
        query_params={'queryVector': np.array(question_embedding, dtype=np.float32).tobytes()},
    )
    return [document.content for document in results.docs]

Sizing your service

Vectors are stored in memory. You can estimate memory requirements with the following formula:

numberOfVectors x dimensions x 4 bytes, plus around 30–50% extra for the HNSW graph.

For example, one million vectors of 1536 dimensions use about 6 GB for the raw vectors. A 20 GB plan is a comfortable starting point for this scale. You can reduce memory usage by:

• Using a smaller model. For example, a 768-dimension model halves memory usage compared to a 1536-dimension model.
• Storing as FLOAT16 instead of FLOAT32 if your model supports it, which halves the space again.

You can monitor actual usage with FT.INFO chunksIndex and the used_memory metric in your Prometheus monitoring. If your requirements increase, upgrading your plan is straightforward and can be done directly from your dashboard.

Good to know

• Vector indexes only work on database 0, as with all search indexes in Redis.
• Vector sets offer a simpler alternative with no setup required. The VADD and VSIM commands are built into Redis, allowing you to use similarity search immediately, without enabling any modules. This is convenient for simple recommendation scenarios on a single set. The query engine remains preferable if you need filters, pagination, or multiple fields.
• Building an index over existing keys runs in the background. You can monitor progress with FT.INFO. Searches return partial results until indexing is complete.

To learn more about vector search in Redis, see the official Redis vector search documentation.