← Back to all stories

Scaling Beyond RAM: The Architecture of DiskANN and SSD-Resident Vector Search

Hierarchical Navigable Small World (HNSW) graphs are the default algorithm for vector search: they construct multi-layer geometric graphs that allow logarithmic nearest-neighbor routing. But HNSW has a severe hardware constraint: the entire graph must reside in physical RAM. At 50 million to 1 billion vectors, the RAM cost becomes economically untenable.

The Billion-Vector RAM Wall

A billion 1536-dimensional float32 vectors require 6 terabytes of storage for raw vectors alone, plus additional terabytes for graph adjacency pointers. Renting multi-terabyte RAM server clusters costs tens of thousands of dollars per month, whereas high-speed NVMe SSD storage is over 20x cheaper per gigabyte.

[HNSW: 100% In-Memory Graph (Extremely Expensive at Scale)]
All 1 Billion Vectors + Graph Links ──► Loaded in Server RAM (Cost: $$$$$)

[DiskANN Architecture: Compressed RAM Routing + NVMe SSD Graph Storage]
Query
  │
  ▼
[Compressed In-Memory 1-bit / 2-bit Quantized Index] (Navigates near target in RAM)
  │
  ▼ (Executes 2-3 Fast NVMe SSD Sector Reads via io_uring)
[Full Precision Vamana Graph Nodes Stored on NVMe SSD]
  │
  ▼
High-Precision Nearest Neighbors at 1/10th the Hardware Cost!

The DiskANN Breakthrough: Vamana Graphs on NVMe

DiskANN redesigned vector graph search around the physical block access characteristics of modern NVMe SSDs:

  1. Vamana Graph Topology: A graph topology with a bounded maximum out-degree and long-range shortcuts that minimizes the number of sequential disk hops required to find nearest neighbors.
  2. Two-Tier Memory Split: An aggressively compressed (e.g. 1-bit Product Quantized) vector index lives in a tiny RAM footprint, guiding the initial search to the general neighborhood.
  3. Batched Asynchronous SSD Reads: The final precision graph traversal executes directly against raw NVMe SSD blocks using asynchronous Linux io_uring syscalls, achieving sub-5ms latencies over billions of vectors.

DiskANN proved that vector databases can scale to billions of records on a single workstation without enterprise RAM clusters.

Reference Paper / Context: DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node (Subramanya et al., Microsoft Research) — Read source ↗
About the Author

Vikram Samal is an AI systems architect focusing on test-time reasoning, high-throughput inference runtimes, and distributed agent infrastructure. Writing weekly architectural stories on Sundays.

Previous
← The Frustration of Web Scraping: Why Accessibility Trees Beat Raw HTML for Browser Agents
Next
The Surgical Knife: How Knowledge Editing Replaced Foundation Model Retraining →