Unified Data Navigation System

Complete control over data at any scale, from 10 tokens to 10M tokens.

Automatic Injection

All navigation tools are automatically injected into every MCP service. No configuration needed - these 19 tools are always available for data manipulation.

Three-Tier Architecture

Tier 1: Memory

In-memory JSON manipulation

7 tools

✅ Fully implemented

Tier 2: Storage

Reference management & chunking

5 tools

✅ Fully implemented

Tier 3: Index

Elastic search & aggregation

7 tools

🚧 Coming soon

Tier 1: Memory Navigation Tools

Work with in-memory JSON data or Redis-cached references. Perfect for immediate data access and manipulation.

json_get_by_path

Extract data using JSONPath expressions

// Extract data using JSONPath
{
  "name": "json_get_by_path",
  "arguments": {
    "json": {"users": [{"name": "Alice"}, {"name": "Bob"}]},
    "path": "$.users[*].name"
  }
}
// Returns: ["Alice", "Bob"]

json_list_fields

Explore JSON structure without loading all data

json_slice_array

Get array slices with Python-like syntax

json_filter_array

Filter arrays using JSONPath filter expressions

json_count

Count elements efficiently without loading all data

json_search

Search for values in complex JSON structures

json_get_field

Simple field access for top-level properties

Tier 2: Storage Navigation Tools

Handle reference management and chunking for large datasets. Automatically manages storage backend selection.

store_with_reference

Explicitly store data with LLM-controlled options

// Store large data with chunking
{
  "name": "store_with_reference",
  "arguments": {
    "data": [/* 10,000 items */],
    "storage": "auto",
    "chunking": {
      "strategy": "by_tokens",
      "size": 1000
    },
    "ttl": 300
  }
}
// Returns: {
//   "_ref": "ref_abc123_1234567890",
//   "_storage": "redis",
//   "_size_tokens": 45000,
//   "_chunking": {
//     "total_chunks": 45,
//     "chunk_size": 1000
//   }
// }

retrieve_chunk

Get data in token-aware chunks

// Get data in manageable chunks
{
  "name": "retrieve_chunk",
  "arguments": {
    "ref": "ref_abc123_1234567890",
    "mode": "tokens",
    "start": 0,
    "size": 1000,
    "include_overlap": true
  }
}
// Returns chunk with continuation info

peek_reference

Get metadata about stored data without loading it

list_references

See all available references in the session

release_reference

Explicitly free storage when done

Token Management

Automatic Optimization

  • • Small data (<10KB): Direct return
  • • Medium data (<5MB): Redis reference
  • • Large data (>5MB): Elastic storage (when available)

Token Estimates

Every reference includes token estimates:

_size_bytes: 41280
_size_tokens: 10320
_chunking: {
strategy: "by_tokens"
chunk_size: 1000
total_chunks: 11
}

LLM Control

LLMs explicitly control:

  • • Storage backend selection (memory, redis, elastic)
  • • Chunking strategy and size
  • • TTL for stored data
  • • When to index vs. when to scan

Usage Patterns

Pattern 1: Progressive Data Exploration

  1. 1. API returns large response → Automatic reference creation
  2. 2. json_list_fields → See structure
  3. 3. json_get_field → Get specific fields
  4. 4. json_slice_array → Work with subsets

Pattern 2: Large Dataset Processing

  1. 1. store_with_reference → Explicit storage with chunking
  2. 2. peek_reference → Understand size and structure
  3. 3. retrieve_chunk → Process in manageable pieces
  4. 4. release_reference → Clean up when done

JSONPath Support

Full JSONPath expression support for complex data navigation:

Basic Paths

  • $.field - Root field
  • $.users[0] - First user
  • $.users[*].name - All names
  • $..email - All emails (recursive)

Filters

  • $[?(@.price > 100)] - Price filter
  • $[?(@.active == true)] - Active items
  • $[?(@.name =~ /^A/)] - Regex match
  • $[?(@.tags contains 'new')] - Array contains

Coming Soon: Tier 3 Index Navigation

Elastic-powered search and aggregation for massive datasets:

  • • index_for_search - Create searchable index
  • • search_indexed - Hybrid search
  • • scan_indexed - Deterministic enumeration
  • • aggregate_indexed - Server-side aggregations
  • • explain_indexed - Search scoring
  • • project_indexed - Field projection
  • • drop_index - Clean up indices