HeyRocky
Deploy Hermes with AWS Bedrock

Deploy Hermes with AWS Bedrock

Hermes connects to Amazon Bedrock as a native provider through the Bedrock Converse API, not through an OpenAI-compatible endpoint. This path is a good fit when model access, billing, identity, and optional guardrails should stay inside an AWS account.

Model availability, quotas, and supported regions can vary by account and model. Confirm the target model in AWS before treating a deployment as ready.

What this path requires

  • AWS credentials available through a supported boto3 credential source, such as an instance role, named profile, AWS CLI configuration, or environment variables.
  • boto3 installed with Hermes’ Bedrock extra.
  • Permission to invoke models and, if using discovery, list foundation models and inference profiles.
  • A target region and access to at least one suitable model or inference profile.

Setup checklist

  • Choose an AWS identity for the runtime; prefer a role on AWS-hosted compute over long-lived keys.
  • Grant only the Bedrock permissions the workflow needs.
  • Install Bedrock support with uv pip install -e ".[bedrock]" from the Hermes Agent checkout.
  • Run hermes model and choose More providers → AWS Bedrock.
  • Select the intended region and a model shown by discovery.
  • Prefer an inference profile ID when the underlying model does not support direct on-demand invocation.
  • Run hermes doctor before enabling gateways or unattended jobs.
  • Complete one harmless chat request and record the provider, region, and model ID that actually worked.
cd ~/.hermes/hermes-agent
uv pip install -e ".[bedrock]"
hermes model
hermes doctor
hermes chat

Region, credentials, and model IDs

Hermes resolves the Bedrock region from bedrock.region, then AWS_REGION, then AWS_DEFAULT_REGION, with us-east-1 as the documented fallback. Set the region explicitly for production so a shell change cannot silently move traffic.

Bedrock often expects an inference profile ID such as one prefixed with us. or global. rather than a bare foundation-model ID. The model picker discovers available choices, so use its current result instead of copying a model name from an old example.

Optional Bedrock Guardrails can be configured under bedrock.guardrail. Treat that as a separate policy rollout: confirm the identifier, version, trace setting, and streaming behavior in a non-production test before applying it to every invocation.

Gateway rollout

A Hermes gateway reads the same provider configuration as the CLI. First prove Bedrock in an interactive session; only then run hermes gateway setup and start the gateway. This separates provider failures from messaging-platform failures.

Common pitfalls

  • Pointing Hermes at a Bedrock OpenAI-compatible route instead of using the native bedrock provider.
  • Installing Hermes without the Bedrock dependency and diagnosing the result as an IAM failure.
  • Using a bare model ID when Bedrock requires an inference profile.
  • Selecting a model that is unavailable in the configured region or account.
  • Granting broad AWS access when only invocation and discovery permissions are needed.
  • Treating automatic retry as a substitute for checking Bedrock quotas after a ThrottlingException.

Verification steps

  1. Run hermes doctor and confirm credentials, boto3, Bedrock reachability, and model discovery are reported successfully.
  2. Start a fresh Hermes session and submit a low-risk prompt.
  3. Confirm the configured provider is bedrock, the region is intentional, and the selected model ID matches the working request.
  4. If guardrails are enabled, test an allowed prompt and an intentionally blocked test prompt in a controlled environment.
  5. If a gateway will use Bedrock, send one test message through that gateway and inspect its logs before wider rollout.

Official documentation

https://hermes-agent.nousresearch.com/docs/guides/aws-bedrock

Related pages