Configure Hermes Speech Recognition Language and Models

Hermes uses speech-to-text, or STT, for CLI voice mode and inbound voice messages on supported gateways. The two choices that most affect results are the language hint and the transcription model. A known language prevents short or accented clips from being misdetected, while the model choice controls local storage, latency, cost, and accuracy.

Hermes stores these settings under stt in config.yaml. Find or edit the active file with the supported CLI instead of assuming where a profile keeps it:

hermes config path
hermes config edit

Start with an explicit baseline

This local configuration is a practical starting point for English speech:

stt:
  enabled: true
  echo_transcripts: true
  provider: local
  language: "en"
  local:
    model: "base"
    language: ""
    initial_prompt: ""
    vad: true
    vad_min_silence_ms: 500
    no_speech_prob_threshold: 0.6
    logprob_threshold: -1.0

stt.enabled controls automatic transcription. In gateway use, disabling it does not discard the audio: Hermes still caches the file and passes its path to the agent for a custom pipeline. stt.echo_transcripts controls whether the gateway posts the raw transcript back to the chat as a microphone-prefixed message. Set it to false when the agent needs the transcript but the chat should not display it.

Apply the language-resolution order

Hermes resolves language consistently for built-ins, command providers, and plugins:

stt.<provider>.language
stt.language
HERMES_LOCAL_STT_LANGUAGE
Provider auto-detection

The default global value is stt.language: "en". Therefore, leaving stt.local.language blank does not by itself restore auto-detection; the global English hint still wins. For Spanish, set stt.language: "es". For Japanese, use "ja". A provider-specific value is useful when one backend serves a fixed-language workflow while other providers use the global setting.

For genuinely multilingual audio, set the global and provider-specific language fields to empty strings and make sure HERMES_LOCAL_STT_LANGUAGE is not set:

stt:
  provider: local
  language: ""
  local:
    language: ""

Auto-detection is flexible, but the official configuration guide warns that Whisper can misidentify short or accented clips. Prefer a fixed ISO 639-1 code when the language is known.

Choose a local Whisper model

The local provider runs faster-whisper on the machine. Install it separately if it is unavailable:

pip install faster-whisper

Set the model under stt.local.model:

tiny is about 75 MB and fastest, with basic quality.
base is about 150 MB and is the default balance of speed and quality.
small is about 500 MB and improves accuracy at moderate cost.
medium is about 1.5 GB and slower, with stronger quality.
large-v3 is about 3 GB, slowest, and the highest-quality listed local option.

The model downloads on first use. Choose based on the actual host rather than assuming the largest model is always operationally best. For short commands on a CPU-only machine, base or small may be a better latency tradeoff than large-v3.

stt.local.initial_prompt can bias Whisper toward expected vocabulary or script. Keep it focused on domain terms rather than using it as a general agent instruction.

Select a cloud provider and model

Hermes also documents these built-in routes:

groq uses GROQ_API_KEY. Its model override is STT_GROQ_MODEL, commonly whisper-large-v3-turbo.
openai accepts VOICE_TOOLS_OPENAI_KEY first and falls back to OPENAI_API_KEY. Configure stt.openai.model as whisper-1, gpt-4o-mini-transcribe, gpt-4o-transcribe, or gpt-transcribe.
mistral uses MISTRAL_API_KEY and supports stt.mistral.model, including voxtral-mini-latest and voxtral-mini-2602. The documented optional install, run from the Hermes Agent checkout, is:

cd ~/.hermes/hermes-agent && uv pip install -e ".[mistral]"

xai uses XAI_API_KEY; set stt.xai.model: "grok-stt" to use that route explicitly.

A cloud example using OpenAI is:

stt:
  enabled: true
  provider: openai
  language: "en"
  openai:
    model: "gpt-4o-mini-transcribe"
    language: ""

Provider credentials belong in the Hermes environment file, not in config.yaml. If the requested provider is unavailable, Hermes can fall back through local, Groq, and OpenAI paths, so a transcript appearing is not proof that the selected backend was healthy.

Control silence and background noise

Local transcription enables Silero voice-activity detection by default. With stt.local.vad: true, silence and noise are filtered before Whisper. Hermes also drops a segment only when both its no-speech probability and low-confidence threshold are met. This reduces phantom phrases without discarding quiet real speech as aggressively.

Set stt.local.vad: false when the goal is to transcribe music or ambient, non-speech audio. For ordinary voice notes, leave VAD enabled. The separate CLI recording controls such as voice.silence_threshold and voice.silence_duration decide when recording stops; they are not STT language or model settings.

Apply and test the change

Restart a gateway after editing its configuration:

hermes gateway restart

For CLI voice mode, exit and relaunch hermes, then use:

/voice status
/voice on

Press Ctrl+B, speak a short phrase containing names or domain vocabulary, and compare the visible transcript with what was said. Test both a clean clip and a realistic noisy clip before increasing model size.

Pitfalls

Do not use display.language to configure recognition. It translates a limited set of static UI messages, not STT or agent replies.
Do not assume a blank provider language means auto-detect while stt.language or HERMES_LOCAL_STT_LANGUAGE still has a value.
Do not select a large local model without accounting for its download size and slower inference.
Do not store API keys in the YAML configuration.
Do not disable VAD for ordinary speech unless you have a specific reason; raw silence and ambient audio can produce hallucinated text.
Do not mistake fallback success for proof that the requested provider and model were used.

Verification checklist

Verify stt.enabled is true and the intended stt.provider is selected.
Verify the effective language follows the documented provider, global, environment, then auto-detect order.
Verify the configured local or cloud model name is supported by that provider.
Verify the required package or provider credential is available without exposing it in output.
Verify a short, accented, and noisy sample transcribes acceptably.
Verify stt.echo_transcripts matches the privacy and chat-UX requirement.
Verify the gateway was restarted or the CLI was relaunched after the edit.