Vector databases – what are they and what are the right use cases for them

Introduction

Before we discuss vector databases and when to consider using them, let’s start with a simple question.

What is a vector?

In terms of data representation, a vector is a sequence of numbers (or scalars) that can represent data points in a multidimensional space.

It’s not a new concept – its use in mathematics was first introduced by polymath Hermann Grassmann in the mid-19th century. In this article, we will discuss how the vector representation of data works with machine learning, natural language processing (NLP), and image processing. Using vectors to represent data has been around since the 1980s (through Spatial databases). You’ve probably used a spatial database more than once – Google Maps? Two billion users a month rely on that database, which uses vectors to deliver accurate results efficiently to get where they need to go.

So, let’s discuss the difference between a traditional database and a vector database. A traditional database is organized on structured data (think rows and columns) and can be paired with a relational database. Traditional databases can be queried with exact match (or range) queries and filtered based on specific criteria, relying on structured query languages (like SQL).

Vector databases represent and store data as vectors, numerical representations of those data points in high-dimensional spaces. High-dimensional spaces are datasets that contain many variables (columns) compared to the number of data points (rows). Vectors capture the underlying meaning of data, enabling similarity searches.

Understanding Vectors

To understand vectors, it starts with embedding.

Embedding is the numerical representation (vector) that captures the meaning and relationships of text, image, and audio files. Each embedding or vector has a high-dimensional aspect. That means that each vector can have hundreds or thousands of dimensions—each dimension captures a different feature or aspect.

When creating those vectors, the goal is to keep data with similar meanings close to each other in the vector space to allow for an optimized similarity search.

Word embedding

Here’s an example of word embedding (provided by Google Gemini):
We have two sentences:

Sentence 1: “The cat is on the mat.”
Sentence 2: “A feline rested on the rug.”

An embedding model (like those used in NLP) would convert these sentences into vectors. These vectors might look something like:

Sentence 1: [0.23, -0.87, 0.56, …, 0.12]
Sentence 2: [0.25, -0.85, 0.58, …, 0.10]

Because the sentences have similar meanings, their vectors would be relatively close to each other in the high-dimensional space.

Image embedding

It is common to use a convolutional neural network (CNN) to extract features of an image and generate image embeddings.¹

Here’s an example of image embedding (provided by Google Gemini):

An image of a dog might be represented by a vector like:
[0.78, 0.12, -0.45, …, 0.91]

A similar vector would be used for another image of a dog. Conversely, a significantly different vector would be used for a picture of a cat.

Digging into convolutional neural networks a little more, they excel at learning hierarchical features from images. Their layers can detect patterns like edges, textures, and more complex object parts.

Image embeddings from convolutional neural networks can be used for:

• Similarity search: Finding images that may be visually similar (think two images with two different dog breeds, both are sitting)
• Image retrieval: Searching for images based on their content
• Image classification: Using the embeddings as features for classification tasks
• Facial recognition: Representing faces as embeddings for comparison

An important aspect of vector databases is their capacity to store, index, and manage high-dimensional data. This concept is not easy to understand, so I asked Gemini to give me an easy way to describe it.

Imagine you want to describe a simple object, like a color. You might use just one number: its position on a rainbow scale (e.g., 0 for red, 100 for violet). That’s a one-dimensional space.

Now, let’s describe a point on a map.

You need two numbers: its latitude and its longitude. That’s a two-dimensional space.

If you want to describe a cube, use its length, width, and height. That’s a three-dimensional space, and we can easily visualize all of these.

Core concepts of vector databases

A vector database is designed for efficiently storing, indexing, and querying high-dimensional vector embeddings. These embeddings are numerical representations of data, capturing the semantic meaning (or features) of various forms of unstructured data. We covered one of the core concepts above – embeddings, and we’ll dig deeper into a few more core concepts.

High-Dimensional Vectors

• Vector embeddings typically have a large number of dimensions.
• Each dimension captures a nuanced aspect of the data, allowing for a rich and detailed representation.
• An embedding with many dimensions requires specialized indexing (and querying) techniques.

Similarity Search:

• Unlike a traditional database (which focuses on exact matches), vector databases excel at finding data points similar to a query vector.
• Similarity is determined by measuring the distance between vectors in the high-dimensional space.²

Standard distance metrics include:

• Cosine Similarity: measures the angle between two vectors, which are useful for text and high-dimensional data, as it’s insensitive to magnitude. Magnitude in this reference is a calculation that allows for the normalization of vectors, enabling the metric to measure the similarity based on the orientation or direction of the vectors rather than their size.
• Euclidean Distance: The straight-line distance between two points in the vector space. This is the fundamental distance metric in various fields, including mathematics, physics, computer science, and particularly in machine learning algorithms.
• Dot Product: Another measure of vector similarity. It’s used in geometry, physics, and computer science.

Indexing:

• Vector databases employ specialized indexing techniques to perform similarity searches across large datasets of high-dimensional vectors efficiently.
• Traditional indexing methods used in relational databases are not efficient for high-dimensional data.

Standard vector indexing algorithms include:

• Hierarchical Navigable Small Worlds (HNSW): builds a multi-layer graph structure for an efficient approximate nearest neighbor (ANN) search.
• Inverted File Index (IVF): Divides the vector space into partitions and searches within the most relevant partitions.
• Product Quantization (PQ): Compresses vectors to reduce memory usage and speed up distance calculations.

Approximate Nearest Neighbor (ANN) Search:

• Due to the computational cost of finding the exact nearest neighbors in high-dimensional space, vector databases often use ANN algorithms.
• ANN algorithms sacrifice some accuracy to achieve significantly faster search speeds, which is acceptable for many similarity search applications.

These core concepts revolve around representing complex data as high-dimensional vectors, indexing these vectors for efficient similarity search using specialized algorithms and distance metrics, and performing approximate nearest-neighbor searches to retrieve semantically related information quickly.

Use cases for a vector database

By now, you’ve got a good understanding of the concepts around vector databases and their differences.

1. Semantic Search and Information Retrieval: Imagine searching for “pictures of cute fluffy animals” and getting back images that look like cute fluffy animals, even if the captions don’t explicitly use those words. Vector databases excel at understanding the meaning behind text and images. By embedding queries and documents (text, images, audio, video) into a shared vector space, the database can find the most semantically similar items, going beyond simple keyword matching. It powers more intelligent search engines and recommendation systems.

2. Personalized Recommendations: Imagine your favorite streaming service suggesting movies or shows you might like. Vector databases can store user preferences (represented as vectors based on their viewing history, ratings, etc.) and content characteristics (also as vectors). By finding the closest content vectors to a user’s preference vector, the system can deliver highly personalized and relevant recommendations for movies, music, products, articles, and more.

3. Fraud Detection: In the financial world, vector databases can help teams identify unusual patterns that indicate fraudulent activity. By representing user transactions, account details, and other relevant information as vectors, the system can detect anomalies – data points far away from typical behavior in the vector space. This process can identify potentially fraudulent activities that a rule-based system might miss.

4. Chatbots and Conversational AI: When you interact with a chatbot, it needs to understand the nuances of your questions. Vector databases store embeddings of previous conversations, user queries, and knowledge base articles. The chatbot can retrieve relevant information and generate more contextually appropriate and helpful responses by embedding a new user query and finding the most similar existing vectors.

5. Image and Video Similarity Search: Beyond just text, vector databases are powerful for finding similar images or videos. By extracting features from visual content and embedding them as vectors, you can perform searches like “find me other dresses that look like this one” or “show me all video clips with a similar scene.” It has applications in e-commerce, media management, and content moderation.

6. Drug Discovery: In pharmaceutical research, vector databases can help accelerate discovery. On average, it takes 10 years to bring a drug from initial discovery to market approval. Utilizing a vector database, in drug discovery, molecules can be represented as vectors based on their properties. Researchers can then search the database for molecules similar to known drugs or target compounds, potentially identifying new drug candidates faster – potentially improving the time to clinical trials (and eventually to market).

While vector databases offer exciting possibilities, there are some common challenges and important considerations to keep in mind when working with them:

Challenges:

1. The Curse of Dimensionality: Vector embeddings can have hundreds (or thousands) of dimensions to capture the richness of the data completely. This high dimensionality can lead to the “curse of dimensionality,” where distance metrics become less meaningful, and overall search performance degrades. Finding the right balance between information richness and dimensionality is critical.³

2. Scalability and Performance: Maintaining efficient search performance can become challenging as the data volume and vectors’ dimensionality increase. Indexing high-dimensional data for fast similarity search is a complex problem. Indexing techniques (like HNSW, Annoy, etc.) have trade-offs regarding build time, query speed, and memory usage. Scaling the database horizontally to handle large datasets while preserving query latency requires careful architectural design.

3. Choosing the Right Embedding Model: The quality of the vector embeddings is paramount. The choice of embedding model (e.g., different transformer models for text, CNN architectures for images) significantly impacts the effectiveness of the vector database. Understanding the specific data type and the desired semantic understanding is critical; selecting the right model that is appropriate can have an impact on your success.

4. Indexing and Query Optimization: Efficiently indexing and querying high-dimensional vector data requires specialized techniques. Understanding the characteristics of different indexing algorithms and how they perform under various workloads is essential for optimizing query latency and throughput. Choosing the right distance metric (e.g., cosine similarity, Euclidean distance) for the specific use case also impacts performance and relevance.

5. Data Updates and Maintenance: Real-world data is dynamic. Updating vectors in an extensive database while maintaining index integrity can be challenging. You need to develop strategies for handling inserts, deletes, and updates to avoid performance degradation.

Monitoring and Observability: Monitoring the performance and health of a vector database is crucial. Effective monitoring and alerting systems are important for identifying and addressing potential issues.

Considerations:

1. Defining the Use Case and Data: The first step is to clearly understand the specific problem you are trying to solve with a vector database and the nature of your data. This will guide the choice of embedding model, indexing strategy, and distance metric.

2. Embedding Strategy: How will you generate the vector embeddings? Will you use pre-trained models or train your own? What are the trade-offs regarding accuracy, computational cost, and customization?

3. Indexing Technique Selection: Which indexing algorithm best suits your data volume, dimensionality, query patterns, and performance requirements? Understanding the trade-offs between approximate nearest neighbor (ANN) search algorithms (speed vs. accuracy) is essential.

4. Distance Metric: Which distance metric aligns best with the notion of similarity for your data? Cosine similarity is often used for text embeddings, while Euclidean distance might be more appropriate for other data types.

5. Accuracy vs. Speed Trade-off: Many ANN search algorithms offer a trade-off between search speed and accuracy (recall). You need to determine the acceptable level of approximation for your application.

6. Integration with Existing Systems: How will a vector database integrate with your existing data pipeline and infrastructure? Considerations should include ingestion, query execution, and synchronization.

7. Cost and Infrastructure: Vector databases can have specific infrastructure requirements – especially for large-scale deployments. You need to fully understand the costs associated with storage, compute resources, and managed service offerings.

8. Security and Privacy: If your vector database contains sensitive information, your team needs to ensure that the appropriate security measures are in place to protect the data. Standard security tactics like encryption and access control are necessary to ensure that you’re meeting compliance requirements.

Whether your focus is on developing a Retrieval-Augmented Generation (RAG) for an LLM or a semantic search feature, keeping these challenges and considerations in mind – you’ll be able to tap into the full power of vector databases.

Successfully integrating a vector database into your product or feature hinges on a clear understanding of its capabilities and a planned approach to implementation. You will need to clearly define your use case and have a solid understanding of your data. Those two steps will inform your decisions regarding the following:

1. Embedding Strategy: Which embedding model (pre-trained or custom-trained) will best represent your data’s semantic meaning for your specific problem?

2. Indexing Technique: Given your data volume, dimensionality, and query speed requirements, select the most appropriate approximate nearest neighbor (ANN) indexing algorithm (e.g., HNSW for balanced performance, PQ for memory efficiency).

3. Distance Metric: Choose the distance metric (e.g., Cosine Similarity for semantic text search, Euclidean for more general feature comparisons) that accurately reflects ‘similarity’ for your application.

Outside of the initial setup, you will have ongoing operational considerations that need to be addressed.

From planning for data updates to establishing monitoring and observability to track performance (and resource utilization) and ensure your chosen solution aligns with your scalability, cost, security, and privacy requirements, you can harness the full power of vector databases for intelligent data retrieval and advanced applications by addressing these factors proactively.

Some analysts predict that the Global LLM and Agentic AI markets could reach $36 billion and $47 billion in revenue by 2030, respectively. Understanding the infrastructure and underlying technology that will support that growth is important. Making the right decisions now will ensure scalability and future growth.

This article was a collaborative effort between Qdrant and Position².

Qdrant is a leading open-source, high-performance vector database and vector search engine. Built to power semantic search, recommendation engines, Agentic AI memory and AI-driven retrieval. Qdrant offers both a managed service (Qdrant Cloud) or the option to run it on-premise.

Position² is an AI-first Growth Marketing firm that drives growth for innovative brands. We understand your needs, and that understanding drives us to develop the right marketing mix for your company. Our unique process of mapping content to each customer journey stage and deep knowledge of AdTech & Martech platforms deliver exceptional results for your ABM programs. We power our integrated marketing campaigns with the most advanced tactics in Paid Acquisition, Marketing Automation, CRM, Analytics, and cutting-edge Content Creation, Design, and Web Development.