Knowledge Vault

RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval

2 min read

Overview

Sarthi et al. (ICLR 2024, arXiv:2401.18059) — Introduces RAPTOR, a retrieval-augmented generation approach that recursively embeds, clusters, and summarizes text chunks into a tree structure. At inference time, retrieval traverses multiple levels of abstraction, enabling answers that require integrating information across entire documents rather than just short contiguous chunks.

The Problem

Standard RAG retrieves only short contiguous text chunks, limiting holistic understanding. For thematic questions requiring multi-part reasoning (e.g., “How did Cinderella reach her happy ending?”), top-k chunk retrieval cannot capture enough context.

The RAPTOR Tree

Construction (bottom-up):

  1. Leaf layer — original text chunks with embeddings
  2. Clustering — chunks grouped by embedding similarity (using soft clustering via Gaussian Mixture Models to allow multi-cluster membership)
  3. Summarization — LLM generates text summary of each cluster
  4. Recursion — summaries become new nodes, repeat clustering and summarizing until a root layer is reached

Two retrieval strategies:

  • Tree traversal — top-down layer-by-layer selection
  • Collapsed tree — flatten all nodes into a single retrieval pool, retrieve by embedding similarity regardless of tree level

Key Results

Tested with UnifiedQA, GPT-3, and GPT-4:

  • RAPTOR + GPT-4 improves QuALITY benchmark by 20% absolute accuracy over previous best
  • State-of-the-art on NarrativeQA (books/movies), QASPER (NLP papers), and QuALITY (medium-length passages)
  • Collapsed tree retrieval generally outperforms tree traversal
  • Significant improvements on questions requiring multi-step reasoning across document sections

Why It Matters

RAPTOR demonstrates that hierarchical summarization during indexing — not just during generation — fundamentally changes what RAG systems can answer. The tree structure captures discourse-level relationships that flat chunk retrieval misses entirely.

Graph View

Backlinks