Provider Routing, Fallback Chains, and the Pareto Router

When Your Primary Provider Fails

Every API provider has downtime, rate limits, and outages. If your Hermes configuration points at a single provider and that provider goes down, Hermes stops working until it comes back. For interactive sessions, you notice quickly and can switch manually. For a batch job running overnight, a two-hour outage can halt your work without you knowing until the morning.

Fallback provider chains solve this. You configure an ordered list of backup providers, and when the primary fails for any reason -- rate limit, connectivity error, unexpected downtime -- Hermes automatically tries the next provider in the chain. The agent's session continues without interruption.

Configuring Fallback Providers

The fallback_providers key in config.yaml takes an ordered list of provider/model pairs:

fallback_providers:
  - provider: nous
    model: deepseek/deepseek-chat
  - provider: openrouter
    model: google/gemini-2.5-flash

When the primary provider fails, Hermes tries the first entry in this list. If that also fails, it tries the second. If all fallbacks are exhausted, it falls back to your default agent model as a final safety net.

Each entry specifies both a provider and a model. The provider tells Hermes which API endpoint to use; the model tells it which specific model to request. You can mix providers freely in a fallback chain -- Nous Portal, OpenRouter, direct Anthropic, or any other supported provider.

A practical fallback chain might look like: primary is the cheapest model that handles most tasks well, first fallback is a more capable model on a different provider, second fallback is a premium model as a last resort. This way, most requests go to the cheap option, occasional failures route to something more reliable, and the most capable model is reserved for situations where everything else failed.

What Triggers a Fallback

Fallback triggers on provider-level failures: connection errors, rate limit responses (429), service unavailable errors (503), and authentication failures (401 on the primary). It does not trigger on model-level failures -- if the model returns an error response because your prompt was malformed, that is not a provider failure and the fallback does not activate.

Understanding what triggers fallback helps you design the chain correctly. If you are hitting rate limits frequently on your primary provider, adding a fallback is the right solution. If you are getting model errors because your prompts are too long, you need to fix the prompts -- a fallback will not help because the backup provider will return the same error.

OpenRouter Provider Routing

OpenRouter is a meta-provider: it routes your requests to multiple underlying AI providers (Anthropic, Google, Meta, Mistral, and many others) and presents them all through a single API endpoint. This makes OpenRouter both a provider and a routing layer.

When using OpenRouter as your provider, you can influence which underlying providers it uses via the provider_routing block in your config:

provider_routing:
  order: [anthropic, google]
  sort: throughput

The order field lists provider preferences in priority sequence. OpenRouter will try Anthropic first, then Google, when both are available for the model you requested. This is useful when you have a preference for a specific upstream provider but want a fallback.

The sort field determines how OpenRouter selects among equally-preferred providers. Three options are available: price picks the cheapest, throughput picks the fastest, and latency picks the one with the lowest time-to-first-token. For batch processing where cost matters more than speed, price is usually the right choice. For interactive sessions where responsiveness matters, latency is often better.

The Pareto Code Router

The Pareto Code Router is an OpenRouter feature that automatically selects the best upstream model for coding tasks based on capability scores rather than manual selection. You access it via a special model identifier:

model: openrouter/pareto-code
extra_body:
  plugins:
    - id: pareto-router
      min_coding_score: 0.5

When you use openrouter/pareto-code, OpenRouter analyzes the available models by their coding benchmark scores and routes your request to the best available option that meets your minimum coding score threshold. The min_coding_score parameter (0.0 to 1.0) sets the quality floor -- models below this score are not considered.

The advantage of the Pareto router is that it adapts automatically as model capabilities change. You do not need to update your config when a new, better coding model becomes available on OpenRouter -- the router picks it up automatically. You just set the minimum score you need and let the router handle model selection.

This is particularly useful for batch processing coding-focused datasets where you want to maximize code quality without manually tracking which model is currently the best option on OpenRouter.

Combining Routing Features

These routing features compose. You can use the Pareto router as your primary model, a specific OpenRouter model as your first fallback, and a direct Anthropic model as your second fallback:

model: openrouter/pareto-code
extra_body:
  plugins:
    - id: pareto-router
      min_coding_score: 0.6
provider_routing:
  order: [anthropic, google]
  sort: price
fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4-5
  - provider: anthropic
    model: claude-opus-4-8

This configuration: routes coding requests through OpenRouter's Pareto router (preferring Anthropic's backend, optimized for price), falls back to a specific Claude model on OpenRouter if the Pareto router fails, and falls back to direct Anthropic as a last resort.

Testing Your Fallback Chain

Fallback chains are only useful if they actually work when the primary fails. Test yours before you rely on it in production. The easiest way is to temporarily set your primary provider to an invalid endpoint or model name, then run a short batch and confirm that the fallback activates correctly. If it does, you know the chain is wired up correctly. If it does not, you need to investigate the configuration before trusting it to protect a real batch job.