Retrieval

Retrieval is the process of searching for and returning relevant information from a data source — a database, document collection, knowledge base, or the web. In the context of AI, retrieval is what allows models to access up-to-date or organisation-specific information that was not included in their training data.

Why retrieval matters for AI

Large language models have a knowledge cutoff — they only know what was in their training data up to a certain date. They also have no access to your company's internal documents, policies, or databases. Retrieval bridges this gap by fetching relevant information and including it in the model's context window alongside your prompt.

How retrieval works in practice

The most common retrieval pattern involves these steps:

1. Query formulation: Your prompt or question is converted into a search query
2. Search execution: The query is run against a data source (vector database, search index, API)
3. Ranking: Results are scored by relevance and the most relevant are selected
4. Context injection: The retrieved information is added to the model's prompt as context
5. Generation: The model generates a response informed by the retrieved context

This pattern is the foundation of retrieval-augmented generation (RAG).

Types of retrieval

• Keyword search: Traditional text matching — find documents containing specific words
• Semantic search: Uses embeddings to find documents with similar meaning, even if they use different words
• Hybrid search: Combines keyword and semantic approaches for better results
• Structured retrieval: Queries databases using SQL or similar languages to fetch specific data points

Quality of retrieval determines quality of output

The single biggest factor in RAG system performance is retrieval quality. If the retrieval step returns irrelevant documents, the model either ignores them or — worse — incorporates incorrect information into its response. Improving retrieval is often more impactful than changing the language model.

Retrieval in enterprise AI

Most enterprise AI applications are fundamentally retrieval problems. A customer service chatbot retrieves relevant help articles. A legal research tool retrieves relevant case law. An internal knowledge assistant retrieves relevant company policies. The AI model is the interface, but retrieval is the engine.