Advanced
Hermes at Scale: Batch Processing, Provider Routing & Prompt Caching
Master Hermes's power features for running at scale: parallel batch processing, multi-provider routing, credential pools, and automatic prompt caching to cut costs and maximize throughput.
01
What Is Batch Processing and When to Use It
›
02
Designing Your Input Dataset
›
03
Running Your First Batch
›
04
Checkpointing, Resuming, and Output Structure
›
05
Trajectory Data and Quality Filtering
›
06
Provider Configuration Architecture
›
07
Credential Pools and Key Rotation
›
08
Provider Routing, Fallback Chains, and the Pareto Router
›
09
Auxiliary Model Routing for Cost Optimization
›
10
Prompt Caching: Cross-Session Memory That Saves Money
›
11
Custom Endpoints and Self-Hosted Models
›
12
Timeouts, Stale Detection, and Production Hardening
›
13
The Gateway: Programmatic Access to Hermes
›