Batch Operations

Process hundreds of operations in a single request. Reduce latency, save costs, and simplify error handling.

Batch Request Format

{
  "tool": "batch_execute",
  "arguments": {
    "operations": [
      {
        "id": "op_1",
        "method": "create_user",
        "params": {"name": "Alice", "email": "[email protected]"}
      },
      {
        "id": "op_2",
        "method": "create_user",
        "params": {"name": "Bob", "email": "[email protected]"}
      },
      {
        "id": "op_3",
        "method": "send_welcome_email",
        "params": {"user_ref": "$op_1.id"},
        "depends_on": ["op_1"]
      }
    ],
    "options": {
      "parallel": true,
      "stop_on_error": false,
      "timeout_ms": 30000
    }
  }
}

Execution Strategies

Parallel Execution

Run independent operations simultaneously

• Maximum concurrency control
• Automatic dependency resolution
• Optimal for I/O-bound operations

Sequential Execution

Process operations in order

• Guaranteed ordering
• Simpler error handling
• Better for dependent operations

Batch Response Handling

Response Structure

{
  "batch_id": "batch_abc123",
  "status": "partial_success",
  "results": [
    {
      "id": "op_1",
      "status": "success",
      "result": {"id": 123, "name": "Alice"},
      "duration_ms": 45
    },
    {
      "id": "op_2",
      "status": "success",
      "result": {"id": 124, "name": "Bob"},
      "duration_ms": 42
    },
    {
      "id": "op_3",
      "status": "failed",
      "error": {
        "code": "RATE_LIMIT",
        "message": "Rate limit exceeded",
        "retry_after": 60
      },
      "duration_ms": 15
    }
  ],
  "summary": {
    "total": 3,
    "succeeded": 2,
    "failed": 1,
    "duration_ms": 102
  }
}

Performance Benefits

Round trip, not N

N→1

Network hops

Tool call for the model

API call

Without Batching

100 operations × 200ms latency = 20 seconds
100 operations × $0.001 = $0.10
100 rate limit slots consumed

With Batching

1 batch request × 300ms = 0.3 seconds
1 batch operation × $0.04 = $0.04
1 rate limit slot consumed

Error Handling Strategies

Fail Fast

Stop batch on first error. Good for dependent operations.

"stop_on_error": true

Partial Success

Continue despite errors. Return both successes and failures.

"stop_on_error": false

Automatic Retry

Retry failed operations with exponential backoff.

"retry_policy": {"max_attempts": 3}

Advanced Features

Operation Dependencies

Reference results from earlier operations

{
  "id": "op_2",
  "method": "update",
  "params": {
    "id": "$op_1.result.id",
    "status": "active"
  },
  "depends_on": ["op_1"]
}

Conditional Execution

Skip operations based on conditions

{
  "id": "op_3",
  "method": "notify",
  "params": {...},
  "condition": "$op_1.status == 'success'"
}

Progress Streaming

Real-time updates via SSE

data: {"completed": 25, "total": 100}
data: {"completed": 50, "total": 100}
data: {"completed": 100, "total": 100}

Transaction Support

All-or-nothing execution

{
  "options": {
    "transactional": true,
    "isolation": "serializable"
  }
}

Best Practices

✓

Batch similar operations

Group operations of the same type for optimal processing

✓

Set reasonable batch sizes

100-500 operations per batch is typically optimal

✓

Include idempotency keys

Ensure safe retries with unique operation IDs

✓

Monitor partial failures

Always check individual operation results

Process hundreds at once

Turn hours of sequential API calls into seconds with intelligent batching.

Get Started Try Batch Demo