Configure a Resilient Provider Fallback Chain

Provider fallback is a continuity feature, not a substitute for choosing a dependable primary model. Hermes uses three resilience layers: credential pools rotate keys for the same provider first, the primary fallback chain can switch to another provider-and-model pair, and auxiliary tasks can resolve providers independently. The cross-provider layers are optional, but they preserve conversation history, tool calls, and context when a supported failure forces a switch.[1]

Map the failure domains

Start by separating account exhaustion from provider failure. Multiple keys for one provider belong in a credential pool; a backup on another provider belongs in fallback_providers. Side work such as vision, web extraction, compression, skill discovery, approvals, title generation, and MCP helpers belongs under auxiliary.<task>. This separation prevents a single broad chain from becoming an accidental routing policy for every workload.[1]

Configure the primary chain

The supported interactive entry point is:

hermes fallback

The manager uses the same provider picker and credential validation as hermes model. It supports add, list, remove, and clear operations and persists the ordered chain in the top-level fallback_providers: list in config.yaml. A minimal direct configuration is:[1]

fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4

Every entry needs both provider and model; incomplete entries are ignored. The older singular fallback_model key remains supported for compatibility, but the plural list is current and takes priority when both exist. Custom OpenAI-compatible endpoints may also supply base_url and key_env.[1]

Understand the runtime contract

Primary fallback can follow exhausted retries for rate limits and server errors, while authentication failures and not-found responses switch immediately. Invalid or repeatedly empty responses can also trigger it. The change is turn-scoped: Hermes restores the primary for each new user message, activates fallback at most once within a turn, and lets normal error handling take over if the fallback also fails. When a provider reports a future rate-limit reset, Hermes can stay on the fallback until that reset passes instead of bouncing needlessly.[1]

A switch also invalidates the prompt cache for the new provider and model. The next request may reread the full conversation at normal input-token pricing; returning to the primary can cause another full read. Reliability may therefore cost more during long, frequently switching sessions.[1]

Give auxiliary work an explicit policy

With provider: auto, an auxiliary task tries the main route, then its task-specific fallback_chain, then the top-level fallback chain, and finally built-in discovery. An explicit auxiliary provider follows a narrower capacity-error ladder: its own provider, any task-specific chain, the main agent route, then a warning and the original error. Use a per-task chain only when its desired order or timeout differs from the main policy.[1]

Deployment checklist

• Distinguish same-provider key rotation from cross-provider failover.
• Put fallback entries in the intended order.
• Confirm every entry has both a provider and a model.
• Decide which auxiliary tasks need their own chain.
• Budget for prompt-cache misses during provider switches.
• Document credentials and reset windows without storing secret values.

Common pitfalls

• Treating fallback_providers as a session-wide permanent model switch; it is normally turn-scoped.
• Expecting a chain to cascade indefinitely inside one turn; primary fallback activates at most once.
• Mixing the legacy singular key with the current list and assuming the legacy value wins.
• Pointing a custom endpoint at an API key value instead of naming its environment variable with key_env.
• Ignoring cache-reset costs when a long conversation alternates providers.

Verification

Open hermes fallback and inspect the saved order. Confirm the primary and each backup have valid credentials, then exercise a controlled non-production failure and verify that the same conversation continues on the backup. On the next turn, confirm Hermes either retries the primary or remains on fallback only when a reported reset time has not elapsed. For auxiliary tasks, verify the task-specific chain is used before the top-level chain when both are configured.

Official references

• [1] Hermes Agent: Fallback Providers