Exploring ColBERT with RAGatouille
Condensed from Simon Willison’s TIL “Exploring ColBERT with RAGatouille” (Jan 2024), which this note originally reproduced in full; see the original for the long walkthrough.
What ColBERT is
A regular embedding model stores one vector per document. ColBERT (Khattab and Zaharia, SIGIR 2020) keeps a vector per token (“late interaction”): query and document are encoded separately, so document vectors can be precomputed, then scored by MaxSim. For each query token take the largest similarity over the document’s tokens, then sum over query tokens. The authors report being two orders of magnitude faster than BERT cross-encoders at comparable effectiveness. ColBERTv2 (NAACL 2022) adds residual compression, cutting storage 6-10x.
RAGatouille
A Python library wrapping ColBERT (Apache 2.0). Its README lists Python 3.9-3.11 and no official Windows support (use WSL2); the repo had roughly 4k stars and 90 open issues, so treat it as lightly maintained. PyLate (MIT, built on Sentence Transformers) is a more recent library for training and using ColBERT-style models, including reranking without an index.
Code that still follows the README pattern
from ragatouille import RAGPretrainedModel
rag = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0")
rag.index(collection=texts, document_ids=ids, document_metadatas=metas,
index_name="blog", max_document_length=180, split_documents=True)
results = rag.search("what is shot scraper?") # list of dicts: content, score, rank Simon’s 3,000-post blog produced about 1.19M token vectors (128-dim centroids, 64-byte residual codes) in a 91MB index, taking minutes and ~2GB RAM on CPU.
Reranking without an index works on any candidate list, for example BM25 or vector results:
docs = rag.rerank(query="What is Datasette Lite?", documents=candidates, k=5) He measured 0.47s for 10 documents on a laptop.
Related
rerankers-and-cross-encoders · hybrid-search-and-rank-fusion · embedding-models · _rag-relevance-moc
Sources
- Simon Willison TIL: https://til.simonwillison.net/llms/colbert-ragatouille (original article; accessed via the note’s stored copy, link not re-fetched 2026-09-30)
- ColBERT: https://arxiv.org/abs/2004.12832 ; ColBERTv2: https://arxiv.org/abs/2112.01488 (accessed 2026-09-30)
- RAGatouille: https://github.com/AnswerDotAI/RAGatouille (accessed 2026-09-30)
- PyLate: https://github.com/lightonai/pylate (accessed 2026-09-30)