Skip to main content
Most users only need the setup from the Self-Hosting Guide. This page is the full reference for customizing providers, tuning features, and hardening your deployment.
Honcho loads configuration in this priority order (highest wins):
  1. Environment variables (always take precedence)
  2. .env file
  3. config.toml file
  4. Built-in defaults
Use .env for secrets and overrides, config.toml for base settings. Or use environment variables exclusively — whatever fits your deployment. Copy the examples to get started:

​
Environment Variable Naming

All config values map to environment variables:
  • {SECTION}_{KEY} for top-level section settings (e.g., DB_CONNECTION_URI → [db].CONNECTION_URI)
  • {KEY} for app-level settings (e.g., LOG_LEVEL → [app].LOG_LEVEL)
  • Use __ inside {KEY} for nested settings (e.g., DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT, DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL)

​
LLM Configuration

The Self-Hosting Guide covers the basic setup: either the built-in OpenAI defaults or one OpenAI-compatible endpoint/model for all features. This section covers recommended model tiers, using multiple providers, and per-feature tuning.
All Honcho agents (deriver, dialectic, dream) require tool calling. Your models must support the OpenAI tool calling format.

​
Choosing Models

Model choice matters more for tool-use reliability than raw intelligence: You can mix providers freely — for example, use Gemini for the deriver and Claude for dreaming.

​
Provider Types

For OpenAI-compatible proxies (OpenRouter, vLLM, Ollama, etc.), use transport = "openai" and set MODEL_CONFIG__OVERRIDES__BASE_URL on each feature to point at your endpoint.
Some OpenAI-compatible providers don’t support OpenAI Structured Outputs (json_schema). Set DERIVER_MODEL_CONFIG__STRUCTURED_OUTPUT_MODE=json_object to request loose JSON mode and inject the schema into the prompt instead.This setting only applies to the deriver on the openai transport — it is the only feature that uses structured output. The dialectic, summarizer, and dreamer don’t request structured output, so the setting has no effect there, and the anthropic/gemini transports reject it.

​
Tiered Model Setup

Once you’re past initial setup, you can assign different models per feature for better cost/quality tradeoffs. This example uses OpenRouter with light/medium/heavy tiers:

​
Direct Vendor Keys

Instead of an OpenAI-compatible proxy, you can use vendor APIs directly. Each transport picks up its own LLM_{TRANSPORT}_API_KEY. If you keep the built-in defaults, only LLM_OPENAI_API_KEY is required:
To use Gemini or Anthropic directly, override the features you want to move:

​
Self-Hosted (vLLM / Ollama)

Use transport = "openai" and set MODEL_CONFIG__OVERRIDES__BASE_URL on each feature:
Set MODEL_CONFIG__TRANSPORT, MODEL_CONFIG__MODEL, and MODEL_CONFIG__OVERRIDES__BASE_URL for each feature the same way. The same overrides are available in config.toml:

​
Thinking Budget

Built-in defaults do not set MODEL_CONFIG__THINKING_BUDGET_TOKENS or MODEL_CONFIG__THINKING_EFFORT. Add one only when your chosen model supports it. Use MODEL_CONFIG__THINKING_EFFORT for OpenAI reasoning models:
Use MODEL_CONFIG__THINKING_BUDGET_TOKENS for Anthropic and Gemini models. Set it to 0 or omit it for providers that don’t support extended thinking:

​
Provider-Specific Parameters

Each model config supports an overrides.provider_params dict for passing arbitrary parameters to the underlying provider SDK. Use this for vendor-specific features that aren’t part of the standard config:
Because provider params live on each model config, background workers such as the Deriver and Dreamer can use longer request timeouts while synchronous chat paths keep tighter defaults. timeout gotchas:
  • The value is validated at config load: it must coerce to a positive, finite number of seconds (numbers or numeric strings like "3600"), or the process refuses to start with an error naming the offending config path. This applies to both the primary model config and its fallback.overrides.
  • The unit is always seconds, regardless of transport. OpenAI and Anthropic receive it as the SDK’s timeout kwarg; Gemini has no such kwarg, so Honcho converts it to milliseconds on http_options.timeout.
  • When unset, nothing is forwarded and each SDK’s default applies — adding this key is opt-in and changes no existing behavior.
  • A too-tight timeout doesn’t fail once: the aborted request goes through the normal retry/fallback chain before the caller sees an error, so the observed latency is several multiples of the timeout.

​
Transport passthrough keys

Three keys inside provider_params are recognized as request-level escape hatches and forwarded to the underlying transport. Where a transport actually validates and merges one of these keys, its value must be a mapping — a non-mapping value raises a configuration error (see the per-transport behavior below; a key a transport ignores is not validated):
  • extra_body — merged into the request body
  • extra_headers — extra HTTP headers
  • extra_query — extra URL query parameters
How each transport forwards them differs:
  • OpenAI and Anthropic forward all three as identically-named SDK kwargs (extra_body, extra_headers, extra_query).
  • Gemini has no SDK kwargs for these. It merges extra_body into the GenerateContentConfig dict and folds extra_headers into http_options.headers; extra_query is unsupported and silently ignored.
The merge is shallow and operator-wins: if Honcho and your config both set the same top-level key inside extra_body, your value replaces Honcho’s. You are responsible for choosing a coherent combination — e.g. unset thinking_budget_tokens when supplying an extra_body.thinking for Anthropic-via-proxy, since Honcho will not translate between the two shapes. Because Gemini merges extra_body directly into GenerateContentConfig (rather than a nested request body), an extra_body written for OpenAI/Anthropic generally will not transfer to Gemini unchanged — and a key collision there can overwrite a field Honcho manages (thinking_config, response_schema, tools, …).

​
Changing Transport

When changing a feature’s transport, always specify model explicitly. Partial overrides that change transport without model will keep the previous model name, which may not be valid for the new provider.

​
General LLM Settings

​
Embedding Configuration

Embeddings use their own nested model config, separate from the main text-generation LLM settings.
EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE defaults to 2048 for OpenAI. For Gemini the client applies a conservative default of 100 — Gemini does not document a per-request limit. Set it when an OpenAI-compatible embedding provider accepts fewer inputs per request, such as DashScope text-embedding-v4 with a limit of 10. Forwarding dimensions= to OpenAI-compatible providers is controlled by EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE:
  • auto (default): forwards dimensions= when the operator has explicitly set EMBEDDING_VECTOR_DIMENSIONS — provenance, not value — and the configured model is not on the known-rejecting list (currently text-embedding-ada-002). Explicit EMBEDDING_VECTOR_DIMENSIONS=1536 does trigger the forward; this is how text-embedding-3-large truncation to 1536 is expressed. Deployments that leave the setting unset get their existing behavior (dimensions= is not forwarded).
  • always: always forward, regardless of whether EMBEDDING_VECTOR_DIMENSIONS was set. Use for OpenAI-compatible self-hosted providers that require it. Do not pick always just for same-as-default truncation — auto handles that case correctly as long as you set EMBEDDING_VECTOR_DIMENSIONS=1536 explicitly in your environment. always is the right answer when your config layer might strip explicit “default-valued” envs, or when you want defense-in-depth.
  • never: never forward. Explicit opt-out for providers that reject the parameter (e.g. text-embedding-ada-002 if it slips past the known-rejecting allowlist).

​
Bootstrapping non-default dimensions

EMBEDDING_VECTOR_DIMENSIONS is treated as immutable for the life of a deployment. The pgvector schema is dim-pinned by Alembic at 1536 by default; if you want a different dim, you must ALTER the empty columns once at bootstrap time. Install order for a non-default dim:
Existing deployments at 1536 with text-embedding-3-small need no action — step 3 detects matching dims and skips. The script refuses to ALTER tables that already contain non-null embeddings. To switch dim or model on a populated deployment, stand up a new deployment at the new configuration and migrate data out of band; there is no in-place re-embedding affordance. See Changing Embeddings for the destroy + rebuild recipe and the same-dim model-swap caveat. External vector stores (Turbopuffer, LanceDB) do not need bootstrap setup. Namespaces are per-workspace and lazy-created on first write at whatever dim the embedding client returns. Use --report to inventory the existing namespaces against the configured dim:
The startup validator at src/startup/embedding_validator.py enforces the dim invariant at boot for both the API (src/main.py lifespan) and the deriver (src/deriver/__main__.py). A mismatch crashes the process with an actionable error before any HTTP route is served or any queue task is processed. VECTOR_STORE_DIMENSIONS is deprecated. EMBEDDING_VECTOR_DIMENSIONS is the single source of truth; setting VECTOR_STORE_DIMENSIONS explicitly emits a startup warning and is otherwise ignored. The field will be removed in a future release; drop it from your .env to silence the warning. The VECTOR_STORE_MIGRATED flag still exists and still controls dual-write / cutover semantics for legacy tenants moving between storage backends (pgvector ↔ turbopuffer ↔ lancedb). It is unrelated to dimension configuration after this release.

​
Feature-Specific Model Configuration

Each feature can use a different provider and model. Below are all the tuning knobs. Dialectic API: The Dialectic API provides theory-of-mind informed responses. It uses a tiered reasoning system with five levels:
Per-Level Configuration: Each reasoning level has its own provider, model, and settings:
Environment variables for nested levels use double underscores:
Deriver (Theory of Mind): The Deriver extracts facts from messages and builds theory-of-mind representations of peers.
Peer Card:
Summary Generation: Session summaries provide compressed context for long conversations — short summaries (frequent) and long summaries (comprehensive).
Dream Processing: Dream processing consolidates and refines peer representations during idle periods.
Surprisal-Based Sampling (Advanced): Optional subsystem for identifying unusual observations during dreaming:

​
Core Configuration

​
Application Settings

Optional Integrations:

​
Database

​
Authentication

Generate a secret: python scripts/generate_jwt_secret.py

​
Cache (Redis)

Redis caching is optional. Honcho works without it but benefits from caching in high-traffic scenarios.

​
Webhooks

​
Vector Store

​
Monitoring

​
Prometheus Metrics

Honcho exposes /metrics endpoints for scraping:
  • API process: Port 8000
  • Deriver process: Port 9090

​
CloudEvents Telemetry

​
Sentry

​
Reference config.toml

A complete config.toml with all defaults. Copy and modify what you need:

​
Database Migrations

​
Troubleshooting

  1. Database connection errors — Ensure DB_CONNECTION_URI uses postgresql+psycopg:// prefix. Verify database is running and pgvector extension is installed.
  2. Authentication issues — Generate and set AUTH_JWT_SECRET when AUTH_USE_AUTH=true. Use python scripts/generate_jwt_secret.py.
  3. LLM provider errors — Verify API keys are set. Check model names match your provider’s format. Ensure models support tool calling.
  4. Deriver not processing — Check logs. Increase DERIVER_WORKERS for throughput. Verify database and LLM connectivity.
  5. Dialectic level issues — Unset level fields inherit from the built-in defaults. For Anthropic, THINKING_BUDGET_TOKENS must be >= 1024 when enabled. For providers without budgeted thinking, omit it or set it to 0. MAX_OUTPUT_TOKENS must exceed THINKING_BUDGET_TOKENS.
  6. Vector store issues — For Turbopuffer, set the API key. Check that EMBEDDING_VECTOR_DIMENSIONS matches your embedding model — the startup validator will refuse to boot on a mismatch.