Running vector search in production

conceptThe concept

• halfvec stores 16-bit floats: half the size of vector, with negligible recall loss for most text models. You can index embedding::halfvec(n) as an expression while keeping full precision in the table.
• Binary quantization (binary_quantize(v)) keeps one bit per dimension: 32 times smaller. Search the bit index by Hamming distance for a few hundred candidates, then re-rank them by exact distance on the full vectors.
• Partial indexes and partitioning: one HNSW index per large tenant or per language keeps each graph small and filters exact.
• Fewer dimensions: several current models support shortened embeddings (for example 3072 down to 1024) with modest quality loss.

Rough sizing: rows times (8 + 4 times dims) bytes for the vectors, and an HNSW index of similar size plus the graph links. Ten million 1536-dimension vectors are about 60 GB before indexing.

whyWhy it matters

The HNSW index should fit in memory for fast queries. Halving or quartering the bytes per vector often turns "needs a bigger server" into "fits comfortably".

Use caseReal-world scenario

A legal search product has 40 million chunk embeddings at 1024 dimensions. It stores full vectors, indexes binary_quantize(embedding) with HNSW, fetches 200 candidates by Hamming distance and re-ranks them by cosine distance on the full vectors. The index is 32 times smaller and recall at 10 stays above 0.97.

Try itExamples

Run and edit these in the interactive lesson.

SELECT pg_column_size(v) AS vector_bytes,
       pg_column_size(v::halfvec) AS halfvec_bytes,
       pg_column_size(binary_quantize(v)) AS bit_bytes
FROM (SELECT ARRAY(SELECT random() FROM generate_series(1, 1536))::real[]::vector AS v) t;

SELECT rows, dims,
       pg_size_pretty(rows::bigint * (8 + 4 * dims)) AS full_vectors,
       pg_size_pretty(rows::bigint * (8 + 2 * dims)) AS as_halfvec,
       pg_size_pretty(rows::bigint * (8 + dims / 8)) AS as_bits
FROM (VALUES (1000000, 1536), (10000000, 1536), (10000000, 384)) v(rows, dims);

BEGIN;
CREATE TABLE emb AS
SELECT g AS id, ARRAY(SELECT random() - 0.5 FROM generate_series(1, 64) WHERE g > 0)::real[]::vector(64) AS v
FROM generate_series(1, 3000) g;
CREATE INDEX ON emb USING hnsw ((binary_quantize(v)::bit(64)) bit_hamming_ops);
WITH q AS (SELECT v FROM emb WHERE id = 1),
candidates AS (
  SELECT id, v FROM emb
  ORDER BY binary_quantize(v)::bit(64) <~> (SELECT binary_quantize(v) FROM q)
  LIMIT 50
)
SELECT id, round((v <=> (SELECT v FROM q))::numeric, 4) AS cosine_distance
FROM candidates
ORDER BY v <=> (SELECT v FROM q)
LIMIT 5;
ROLLBACK;

CREATE INDEX ON kb_chunks USING hnsw ((embedding::halfvec(1536)) halfvec_cosine_ops);

SELECT id FROM kb_chunks
ORDER BY embedding::halfvec(1536) <=> $1::halfvec(1536)
LIMIT 10;

internalsUnder the hood

HNSW inserts update the graph and are slower than B-tree inserts; bulk loads are faster if you load first and build the index afterwards. Deleted rows leave graph nodes that VACUUM must repair, so tables with heavy churn need regular vacuuming or periodic reindexing.

Avoid / preferDo this, not that

SELECT id FROM docs
ORDER BY binary_quantize(embedding)::bit(1024) <~> binary_quantize($1)
LIMIT 10;

SELECT id FROM (
  SELECT id, embedding FROM docs
  ORDER BY binary_quantize(embedding)::bit(1024) <~> binary_quantize($1)
  LIMIT 200
) c
ORDER BY embedding <=> $1
LIMIT 10;

-- one HNSW index over 500 tenants, then WHERE tenant_id = $1

-- partition by tenant group or add partial indexes for the largest tenants,
-- and enable hnsw.iterative_scan for the rest

PitfallsCommon mistakes

• Indexing full-precision vectors when halfvec would have halved memory with no measurable loss.
• Loading millions of rows into a table that already has an HNSW index instead of building it afterwards.
• Ignoring VACUUM on high-churn embedding tables.

LimitsWhen not to use it

Consider a dedicated vector engine when you need billions of vectors, GPU-accelerated search, or multi-node vector search without sharding PostgreSQL. Keep the source of truth and metadata in PostgreSQL even then.

SecuritySecurity notes

Multi-tenant vector search must filter by tenant inside the database, not after the results come back; post-filtering in the application leaks the existence of other tenants' documents through result counts and timing.

ChallengeTry it yourself

BEGIN;
CREATE TABLE emb AS
SELECT g AS id, ARRAY(SELECT random() - 0.5 FROM generate_series(1, 64) WHERE g > 0)::real[]::vector(64) AS v
FROM generate_series(1, 2000) g;
CREATE INDEX emb_full ON emb USING hnsw (v vector_cosine_ops);
CREATE INDEX emb_half ON emb USING hnsw ((v::halfvec(64)) halfvec_cosine_ops);
CREATE INDEX emb_bit ON emb USING hnsw ((binary_quantize(v)::bit(64)) bit_hamming_ops);
SELECT relname, pg_size_pretty(pg_relation_size(oid)) AS size
FROM pg_class
WHERE relname IN ('emb_full', 'emb_half', 'emb_bit')
ORDER BY pg_relation_size(oid) DESC;
ROLLBACK;

InterviewQuestions you should be able to answer

QuizCheck your understanding

4 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.

Go deeperOfficial documentation

• pgvector README
• Index types