Vector databases

A vector database with the entire retrieval pipeline built in. You add documents; the platform reads each one to text, splits it along its own structure, embeds every chunk, and stores the vectors. A query then runs hybrid semantic-and-keyword retrieval, reranks the candidates against the question with a cross-encoder, and returns each passage with a citation to the exact region it came from. Every stage runs on named infrastructure you choose the model for, and none of it is yours to host, scale, or operate.

The verb set is small on purpose: the pipeline does the work. Define a corpus once at module scope, add documents to it, and search it in plain language. What comes back is typed: a hit carries its score, its text, and a citation, and the document and chunk views show how a source was split when a result surprises you. Hybrid retrieval and reranking are both on by default, and both are the right call in almost every case: keyword matching still finds the part numbers, error codes, and proper nouns an embedding model never learned, and reranking is the single biggest lever on quality. Each can be toggled off per query when latency matters more.

This is the right tool only for unstructured text you query by meaning. Anything you can express as a filter belongs in the SQL database, where the answer is exact and cheaper. One corpus is shared across dev and prod, ingestion is content-addressed so re-adding the same bytes is free, and how a corpus is chunked and embedded is configured out of band rather than in code, so what runs and what you wrote cannot drift.

Operating a corpus at scale happens from code too. A corpus can be connected to an external origin and pulled in with sync. From there, its ingestion jobs can be listed and inspected, its documents paged with documentsPage and counted with count, a single document handed out as a signed shareUrl, and a matched set removed with removeWhere. The corpus you search is the corpus you fill.

Passages can also say what they are about. A document asserts its entities, your own ids, when it is added, and a dictionary pushed with pushDictionary tags every passage whose text names one. Filter any search, count, or listing by entities, see what an entity appears beside with entities(), and list every passage about it with passages().