> ## Documentation Index
> Fetch the complete documentation index at: https://honcho.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Configuration Guide

> Complete reference for configuring Honcho providers, features, and infrastructure

> [!NOTE]
>
> Most users only need the setup from the [Self-Hosting Guide](https://honcho.dev/docs/v3/contributing/self-hosting.md#llm-setup). This page is the full reference for customizing providers, tuning features, and hardening your deployment.

Honcho loads configuration in this priority order (highest wins):

1. **Environment variables** (always take precedence)
2. **`.env` file**
3. **`config.toml` file**
4. **Built-in defaults**

Use `.env` for secrets and overrides, `config.toml` for base settings. Or use environment variables exclusively — whatever fits your deployment. Copy the examples to get started:

```bash theme={null}
cp .env.template .env
cp config.toml.example config.toml
```

### Environment Variable Naming

All config values map to environment variables:

* `{SECTION}_{KEY}` for top-level section settings (e.g., `DB_CONNECTION_URI` → `[db].CONNECTION_URI`)
* `{KEY}` for app-level settings (e.g., `LOG_LEVEL` → `[app].LOG_LEVEL`)
* Use `__` inside `{KEY}` for nested settings (e.g., `DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT`, `DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL`)

## LLM Configuration

The [Self-Hosting Guide](https://honcho.dev/docs/v3/contributing/self-hosting.md#llm-setup) covers the basic setup: either the built-in OpenAI defaults or one OpenAI-compatible endpoint/model for all features. This section covers recommended model tiers, using multiple providers, and per-feature tuning.

> [!NOTE]
>
> All Honcho agents (deriver, dialectic, dream) require tool calling. Your models must support the OpenAI tool calling format.

### Choosing Models

Model choice matters more for tool-use reliability than raw intelligence:

| Tier | Example models | Use case | Notes |
| - | - | - | - |
| **Light** | Gemini 2.5 Flash, GLM-4.7-Flash | Deriver, summary, dialectic minimal/low | High throughput, cheap, reliable tool use |
| **Medium** | Claude Haiku 4.5, Grok 4.1 Fast | Dialectic medium/high | Good reasoning + tool use balance |
| **Heavy** | Claude Sonnet 4, GLM-5 | Dream, dialectic max | Best quality for rare/complex tasks |

You can mix providers freely — for example, use Gemini for the deriver and Claude for dreaming.

### Provider Types

| Transport value | What it connects to | API key env var |
| - | - | - |
| `openai` | OpenAI or any OpenAI-compatible endpoint (OpenRouter, Together, Fireworks, LiteLLM, vLLM, Ollama) | `LLM_OPENAI_API_KEY` |
| `anthropic` | Anthropic Claude (direct) | `LLM_ANTHROPIC_API_KEY` |
| `gemini` | Google Gemini (direct) | `LLM_GEMINI_API_KEY` |

For OpenAI-compatible proxies (OpenRouter, vLLM, Ollama, etc.), use `transport = "openai"` and set `MODEL_CONFIG__OVERRIDES__BASE_URL` on each feature to point at your endpoint.

> [!NOTE]
>
> Some OpenAI-compatible providers don't support OpenAI Structured Outputs (`json_schema`). Set `DERIVER_MODEL_CONFIG__STRUCTURED_OUTPUT_MODE=json_object` to request loose JSON mode and inject the schema into the prompt instead.
>
> This setting only applies to the **deriver** on the **`openai`** transport — it is the only feature that uses structured output. The dialectic, summarizer, and dreamer don't request structured output, so the setting has no effect there, and the anthropic/gemini transports reject it.

### Tiered Model Setup

Once you're past initial setup, you can assign different models per feature for better cost/quality tradeoffs. This example uses OpenRouter with light/medium/heavy tiers:

```bash theme={null}
LLM_OPENAI_API_KEY=sk-or-v1-...

# All features route through OpenRouter via overrides.base_url
# (You can set this on each feature's MODEL_CONFIG)

# Light tier — high throughput, cheap
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=google/gemini-2.5-flash-lite
DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL=https://openrouter.ai/api/v1
SUMMARY_MODEL_CONFIG__TRANSPORT=openai
SUMMARY_MODEL_CONFIG__MODEL=google/gemini-2.5-flash
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__MODEL=google/gemini-2.5-flash-lite
DIALECTIC_LEVELS__low__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__low__MODEL_CONFIG__MODEL=google/gemini-2.5-flash-lite

# Medium tier — better reasoning
DIALECTIC_LEVELS__medium__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__medium__MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
DIALECTIC_LEVELS__high__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__high__MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
DIALECTIC_LEVELS__max__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__max__MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5

# Heavy tier — best quality for complex tasks
DREAM_DEDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_DEDUCTION_MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
DREAM_INDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_INDUCTION_MODEL_CONFIG__MODEL=anthropic/claude-haiku-4-5
```

### Direct Vendor Keys

Instead of an OpenAI-compatible proxy, you can use vendor APIs directly. Each transport picks up its own `LLM_{TRANSPORT}_API_KEY`.

At configuration load, Honcho warns if a non-empty `LLM_{TRANSPORT}_API_KEY` is set but no configured workload uses that transport. The check includes all dialectic levels, the deriver, summary, both dream specialists, fallback models, and embeddings. It logs the setting name, never the credential value, and does not prevent startup. Remove unused keys from your configuration when you migrate providers.

If you keep the built-in defaults, only `LLM_OPENAI_API_KEY` is required:

```bash theme={null}
LLM_OPENAI_API_KEY=...

# Built-in model defaults
# - deriver: openai / gpt-5.4-mini
# - dialectic (all levels): openai / gpt-5.4-mini
# - summary: openai / gpt-5.4-mini
# - dream specialists: openai / gpt-5.4-mini
# - embeddings: openai / text-embedding-3-small
```

To use Gemini or Anthropic directly, override the features you want to move:

```bash theme={null}
LLM_GEMINI_API_KEY=...
DERIVER_MODEL_CONFIG__TRANSPORT=gemini
DERIVER_MODEL_CONFIG__MODEL=gemini-2.5-flash

LLM_ANTHROPIC_API_KEY=...
DREAM_DEDUCTION_MODEL_CONFIG__TRANSPORT=anthropic
DREAM_DEDUCTION_MODEL_CONFIG__MODEL=claude-haiku-4-5
```

### Self-Hosted (vLLM / Ollama)

Use `transport = "openai"` and set `MODEL_CONFIG__OVERRIDES__BASE_URL` on each feature:

```bash theme={null}
# vLLM
LLM_OPENAI_API_KEY=not-needed
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=your-model-name
DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL=http://localhost:8000/v1

# Ollama
LLM_OPENAI_API_KEY=ollama
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=llama3.3:70b
DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL=http://localhost:11434/v1
```

Set `MODEL_CONFIG__TRANSPORT`, `MODEL_CONFIG__MODEL`, and `MODEL_CONFIG__OVERRIDES__BASE_URL` for each feature the same way.

The same overrides are available in `config.toml`:

```toml theme={null}
[deriver.model_config]
transport = "openai"
model = "my-local-model"

[deriver.model_config.overrides]
base_url = "http://localhost:8000/v1"
api_key_env = "DERIVER_LOCAL_API_KEY"
```

### Thinking Budget

Built-in defaults do not set `MODEL_CONFIG__THINKING_BUDGET_TOKENS` or `MODEL_CONFIG__THINKING_EFFORT`. Add one only when your chosen model supports it.

Use `MODEL_CONFIG__THINKING_EFFORT` for OpenAI reasoning models:

```bash theme={null}
DERIVER_MODEL_CONFIG__THINKING_EFFORT=minimal
DIALECTIC_LEVELS__max__MODEL_CONFIG__THINKING_EFFORT=medium
```

Use `MODEL_CONFIG__THINKING_BUDGET_TOKENS` for Anthropic and Gemini models. Set it to `0` or omit it for providers that don't support extended thinking:

```bash theme={null}
SUMMARY_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
DREAM_DEDUCTION_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
```

### Provider-Specific Parameters

Each model config supports an `overrides.provider_params` dict for passing arbitrary parameters to the underlying provider SDK. Use this for vendor-specific features that aren't part of the standard config:

```toml theme={null}
[deriver.model_config.overrides.provider_params]
# These are passed directly to the provider SDK
verbosity = "low"
# Per-request timeout in seconds; useful for queued workers that can wait longer
timeout = 3600.0
```

Because provider params live on each model config, background workers such as
the Deriver and Dreamer can use longer request timeouts while synchronous
chat paths keep tighter defaults.

`timeout` gotchas:

* The value is validated **at config load**: it must coerce to a positive,
  finite number of seconds (numbers or numeric strings like `"3600"`), or the
  process refuses to start with an error naming the offending config path.
  This applies to both the primary model config and its `fallback.overrides`.
* The unit is always **seconds**, regardless of transport. OpenAI and
  Anthropic receive it as the SDK's `timeout` kwarg; Gemini has no such
  kwarg, so Honcho converts it to milliseconds on `http_options.timeout`.
* When unset, nothing is forwarded and each SDK's default applies — adding
  this key is opt-in and changes no existing behavior.
* A too-tight timeout doesn't fail once: the aborted request goes through the
  normal retry/fallback chain before the caller sees an error, so the
  observed latency is several multiples of the timeout.

#### Extended thinking with forced tools and schemas

Anthropic models reject extended thinking alongside a forced tool choice. `thinking_tool_choice_conflict` decides what Honcho does when a request would combine them:

* **`throw`**: raise a validation error before the request is sent
* **`override_thinking`**: drop thinking for that call
* **`override_tool`**: keep thinking and relax the tool use instead

```bash theme={null}
DIALECTIC_LEVELS__medium__MODEL_CONFIG__OVERRIDES__PROVIDER_PARAMS__THINKING_TOOL_CHOICE_CONFLICT=override_thinking
```

Any other value fails config load at startup, so a typo never reaches a request.

How each transport applies it:

* **Anthropic** checks forced tool choices (`any` or a named tool) when `thinking_budget_tokens` is set. Unset means `throw`. `override_tool` relaxes the tool choice to `auto`.
* **OpenAI** is for Anthropic models behind an OpenAI-compatible proxy, which may turn a `json_schema` response format into a tool the model must call. So the check covers structured-output requests as well as forced tool choices, whenever the request asks for thinking (`thinking_effort`, `thinking_budget_tokens`, or an `extra_body` `reasoning_effort`, `reasoning` or `thinking` that isn't disabled). Unset means no check, because OpenAI's own reasoning models accept these combinations, so set it only on configs that reach Anthropic models. `override_thinking` removes `reasoning_effort` and those `extra_body` keys. `override_tool` relaxes a forced tool choice to `auto` and requests a schema as `json_object` with the schema in the prompt.

Without a policy, a proxy may resolve the schema case itself by leaving the schema tool optional, so the model can answer in prose and the structured-output parse fails.

#### Transport passthrough keys

Three keys inside `provider_params` are recognized as request-level escape hatches and forwarded to the underlying transport. Where a transport actually validates and merges one of these keys, its value must be a mapping — a non-mapping value raises a configuration error (see the per-transport behavior below; a key a transport ignores is not validated):

* **`extra_body`** — merged into the request body
* **`extra_headers`** — extra HTTP headers
* **`extra_query`** — extra URL query parameters

How each transport forwards them differs:

* **OpenAI and Anthropic** forward all three as identically-named SDK kwargs (`extra_body`, `extra_headers`, `extra_query`).
* **Gemini** has no SDK kwargs for these. It merges `extra_body` into the `GenerateContentConfig` dict and folds `extra_headers` into `http_options.headers`; `extra_query` is **unsupported and silently ignored**.

The merge is shallow and **operator-wins**: if Honcho and your config both set the same top-level key inside `extra_body`, your value replaces Honcho's. You are responsible for choosing a coherent combination — e.g. unset `thinking_budget_tokens` when supplying an `extra_body.thinking` for Anthropic-via-proxy, since Honcho will not translate between the two shapes.

Because Gemini merges `extra_body` directly into `GenerateContentConfig` (rather than a nested request body), an `extra_body` written for OpenAI/Anthropic generally will not transfer to Gemini unchanged — and a key collision there can overwrite a field Honcho manages (`thinking_config`, `response_schema`, `tools`, …).

```toml theme={null}
# Example: route an OpenAI-compatible proxy and tag requests for tracing
[deriver.model_config.overrides.provider_params.extra_headers]
X-Proxy-Route = "vertex"

[deriver.model_config.overrides.provider_params.extra_body]
# Provider-native body fields the standard config doesn't expose
anthropic_beta = ["context-1m-2025-01-15"]
```

### Changing Transport

When changing a feature's `transport`, always specify `model` explicitly. Partial overrides that change transport without model will keep the previous model name, which may not be valid for the new provider.

### General LLM Settings

```bash theme={null}
LLM_DEFAULT_MAX_TOKENS=2500

# Tool output limits (to prevent token explosion)
LLM_MAX_TOOL_OUTPUT_CHARS=10000  # ~2500 tokens at 4 chars/token
LLM_MAX_MESSAGE_CONTENT_CHARS=2000  # Max chars per message in tool results
```

### Embedding Configuration

Embeddings use their own nested model config, separate from the main text-generation LLM settings.

```bash theme={null}
# Embedding vector settings
EMBEDDING_VECTOR_DIMENSIONS=1536
EMBEDDING_MAX_INPUT_TOKENS=8192
EMBEDDING_MAX_TOKENS_PER_REQUEST=300000

# Embedding transport/model selection
EMBEDDING_MODEL_CONFIG__TRANSPORT=openai  # openai, gemini
EMBEDDING_MODEL_CONFIG__MODEL=text-embedding-3-small
EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE=10  # optional per-request input cap
EMBEDDING_MODEL_CONFIG__TIMEOUT=90.0       # optional client HTTP timeout (seconds)

# Optional endpoint overrides
EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL=http://localhost:8000/v1
EMBEDDING_MODEL_CONFIG__OVERRIDES__API_KEY_ENV=EMBEDDING_CUSTOM_API_KEY
```

`EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZE` defaults to 2048 for OpenAI. For
Gemini the client applies a conservative default of 100 — Gemini does not
document a per-request limit. Set it when an OpenAI-compatible embedding
provider accepts fewer inputs per request, such as DashScope
`text-embedding-v4` with a limit of 10.

`EMBEDDING_MODEL_CONFIG__TIMEOUT` is an optional client HTTP timeout in
seconds. OpenAI-compatible transports receive it as the SDK `timeout` kwarg
(omitted when unset, so the SDK default applies). Gemini converts it to
milliseconds on `http_options.timeout`, and keeps its existing 10-minute
default when unset. The value is validated at config load the same way as
LLM `provider_params.timeout` (positive, finite number of seconds).

Forwarding `dimensions=` to OpenAI-compatible providers is controlled by `EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE`:

* `auto` (default): forwards `dimensions=` when **the operator has explicitly set `EMBEDDING_VECTOR_DIMENSIONS`** — provenance, not value — and the configured model is not on the known-rejecting list (currently `text-embedding-ada-002`). Explicit `EMBEDDING_VECTOR_DIMENSIONS=1536` *does* trigger the forward; this is how `text-embedding-3-large` truncation to 1536 is expressed. Deployments that leave the setting unset get their existing behavior (`dimensions=` is not forwarded).
* `always`: always forward, regardless of whether `EMBEDDING_VECTOR_DIMENSIONS` was set. Use for OpenAI-compatible self-hosted providers that require it. Do not pick `always` *just* for same-as-default truncation — `auto` handles that case correctly as long as you set `EMBEDDING_VECTOR_DIMENSIONS=1536` explicitly in your environment. `always` is the right answer when your config layer might strip explicit "default-valued" envs, or when you want defense-in-depth.
* `never`: never forward. Explicit opt-out for providers that reject the parameter (e.g. `text-embedding-ada-002` if it slips past the known-rejecting allowlist).

The embedding wire format is controlled by `EMBEDDING_MODEL_CONFIG__ENCODING_FORMAT_MODE`. The `openai` SDK sends `encoding_format=base64` when the caller passes nothing, and some OpenAI-compatible providers answer that with an error or with empty data, so Honcho always sends the format explicitly:

* `auto` (default): `base64` when no `EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL` is set or it points at `api.openai.com`, `float` otherwise. base64 is roughly 3.6x smaller on the wire than JSON floats, so this keeps the compact format for real OpenAI and only pays the larger payload where compatibility requires it.
* `float`: always request floats. Use for a provider that rejects base64 but sits behind a host `auto` reads as OpenAI-compatible-but-capable.
* `base64`: always request base64. Use for a proxy that fronts real OpenAI (Azure OpenAI, LiteLLM) where `auto` cannot tell from the host that base64 is safe, and you want the smaller payload.

Both formats decode to identical vectors, so switching modes does not require re-embedding.

#### Bootstrapping non-default dimensions

`EMBEDDING_VECTOR_DIMENSIONS` is treated as immutable for the life of a deployment. The pgvector schema is dim-pinned by Alembic at `1536` by default; if you want a different dim, you must ALTER the empty columns once at bootstrap time.

Install order for a non-default dim:

```bash theme={null}
# 1. Apply migrations (creates default vector(1536) schema)
alembic upgrade head

# 2. Set the dim you want
export EMBEDDING_VECTOR_DIMENSIONS=768

# 3. ALTER the empty columns to the target dim
uv run python scripts/configure_embeddings.py --dry-run   # preview
uv run python scripts/configure_embeddings.py --yes       # apply

# 4. Start API and deriver — both run the startup validator and refuse
# to serve traffic if the schema and EMBEDDING_VECTOR_DIMENSIONS disagree.
```

Existing deployments at 1536 with `text-embedding-3-small` need no action — step 3 detects matching dims and skips.

The script refuses to ALTER tables that already contain non-null embeddings. To switch dim or model on a populated deployment, stand up a new deployment at the new configuration and migrate data out of band; there is no in-place re-embedding affordance. See [Changing Embeddings](https://honcho.dev/docs/v3/contributing/changing-embeddings.md) for the destroy + rebuild recipe and the same-dim model-swap caveat.

External vector stores (Turbopuffer, LanceDB) do not need bootstrap setup. Namespaces are per-workspace and lazy-created on first write at whatever dim the embedding client returns. Use `--report` to inventory the existing namespaces against the configured dim:

```bash theme={null}
uv run python scripts/configure_embeddings.py --report
```

The startup validator at `src/startup/embedding_validator.py` enforces the dim invariant at boot for both the API (`src/main.py` lifespan) and the deriver (`src/deriver/__main__.py`). A mismatch crashes the process with an actionable error before any HTTP route is served or any queue task is processed.

`VECTOR_STORE_DIMENSIONS` is **deprecated**. `EMBEDDING_VECTOR_DIMENSIONS` is the single source of truth; setting `VECTOR_STORE_DIMENSIONS` explicitly emits a startup warning and is otherwise ignored. The field will be removed in a future release; drop it from your `.env` to silence the warning.

The `VECTOR_STORE_MIGRATED` flag still exists and still controls dual-write / cutover semantics for legacy tenants moving between storage backends (pgvector ↔ turbopuffer ↔ lancedb). It is unrelated to dimension configuration after this release.

### Feature-Specific Model Configuration

Each feature can use a different provider and model. Below are all the tuning knobs.

**Dialectic API:**

The Dialectic API provides theory-of-mind informed responses. It uses a tiered reasoning system with five levels:

```bash theme={null}
# Global dialectic settings
DIALECTIC_MAX_OUTPUT_TOKENS=8192
DIALECTIC_MAX_INPUT_TOKENS=100000
DIALECTIC_MAX_QUERY_LENGTH=10000  # Peer chat query limit, in characters
DIALECTIC_SESSION_HISTORY_MAX_TOKENS=4096
DIALECTIC_BUDGET_ENABLED=false  # budget line at levels with 3+ tool iterations
DIALECTIC_BRIEF_ENABLED=false  # memory layout, counts, and dreamer map notes
```

**Per-Level Configuration:**

Each reasoning level has its own provider, model, and settings:

```toml theme={null}
# config.toml example
[dialectic.levels.minimal]
MAX_TOOL_ITERATIONS = 1
MAX_OUTPUT_TOKENS = 250
TOOL_CHOICE = "any"

[dialectic.levels.minimal.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.low]
MAX_TOOL_ITERATIONS = 5
TOOL_CHOICE = "any"

[dialectic.levels.low.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.medium]
MAX_TOOL_ITERATIONS = 2

[dialectic.levels.medium.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.high]
MAX_TOOL_ITERATIONS = 4

[dialectic.levels.high.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.max]
MAX_TOOL_ITERATIONS = 10

[dialectic.levels.max.model_config]
transport = "openai"
model = "gpt-5.4-mini"
```

Environment variables for nested levels use double underscores:

```bash theme={null}
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__TRANSPORT=openai
DIALECTIC_LEVELS__minimal__MODEL_CONFIG__MODEL=gpt-5.4-mini
DIALECTIC_LEVELS__minimal__MAX_TOOL_ITERATIONS=1
DIALECTIC_LEVELS__minimal__MAX_OUTPUT_TOKENS=250
DIALECTIC_LEVELS__minimal__TOOL_CHOICE=any
```

**Deriver (Theory of Mind):**

The Deriver extracts facts from messages and builds theory-of-mind representations of peers.

```bash theme={null}
DERIVER_ENABLED=true

# LLM settings
DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=gpt-5.4-mini
DERIVER_MAX_INPUT_TOKENS=25000
DERIVER_MAX_CUSTOM_INSTRUCTIONS_TOKENS=2000
# DERIVER_MODEL_CONFIG__THINKING_EFFORT=minimal
# DERIVER_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
# DERIVER_MODEL_CONFIG__TEMPERATURE=0.7  # Optional temperature override
# DERIVER_MODEL_CONFIG__STRUCTURED_OUTPUT_MODE=json_object  # for providers without json_schema support

# Backup model (optional)
# DERIVER_MODEL_CONFIG__FALLBACK__MODEL=claude-haiku-4-5
# DERIVER_MODEL_CONFIG__FALLBACK__TRANSPORT=anthropic

# Worker settings
DERIVER_WORKERS=1  # Increase for higher throughput
DERIVER_SCHEDULER=deriver  # Which process runs the scheduling loops: "deriver" or "api"
DERIVER_POLLING_SLEEP_INTERVAL_SECONDS=1.0
# Adaptive polling: when idle/erroring, the sleep interval grows from the base
# toward DERIVER_POLLING_SLEEP_MAX_INTERVAL_SECONDS by the multiplier each cycle,
# then snaps back to base when work is found. Cuts steady-state query load.
DERIVER_POLLING_BACKOFF_ENABLED=true
DERIVER_POLLING_SLEEP_MAX_INTERVAL_SECONDS=30.0
DERIVER_POLLING_BACKOFF_MULTIPLIER=2.0
# Jitter so instances that start together don't poll in lockstep. Startup: sleep
# a random delay in [0, value] before the first poll (0.0 disables; only applies
# when DERIVER_SCHEDULER=deriver). Per-cycle:
# multiply every poll sleep by a random factor in [1-ratio, 1+ratio] (0.0 disables).
DERIVER_POLLING_STARTUP_JITTER_SECONDS=30.0
DERIVER_POLLING_JITTER_RATIO=0.5
DERIVER_STALE_SESSION_TIMEOUT_MINUTES=5

# Queue management
DERIVER_QUEUE_ERROR_RETENTION_SECONDS=2592000  # 30 days

# Observation settings
DERIVER_DEDUPLICATE=true
DERIVER_LOG_OBSERVATIONS=false
DERIVER_WORKING_REPRESENTATION_MAX_OBSERVATIONS=100
DERIVER_REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS=512
DERIVER_REPRESENTATION_BATCH_TARGET_INPUT_TOKENS=1024
DERIVER_REPRESENTATION_BATCH_MAX_AGE_SECONDS=1800
```

**Peer Card:**

```bash theme={null}
PEER_CARD_ENABLED=true
```

**Summary Generation:**

Session summaries provide compressed context for long conversations — short summaries (frequent) and long summaries (comprehensive).

```bash theme={null}
SUMMARY_ENABLED=true
SUMMARY_MODEL_CONFIG__TRANSPORT=openai
SUMMARY_MODEL_CONFIG__MODEL=gpt-5.4-mini
SUMMARY_MAX_TOKENS_SHORT=1500
SUMMARY_MAX_TOKENS_LONG=6000
SUMMARY_MAX_INPUT_TOKENS=100000
# SUMMARY_MODEL_CONFIG__THINKING_EFFORT=minimal
# SUMMARY_MODEL_CONFIG__THINKING_BUDGET_TOKENS=1024
SUMMARY_MESSAGES_PER_SHORT_SUMMARY=20
SUMMARY_MESSAGES_PER_LONG_SUMMARY=60
```

`SUMMARY_MAX_TOKENS_SHORT` / `SUMMARY_MAX_TOKENS_LONG` are the output caps for each summary call. The prompt asks for about 60% of the cap as a soft target, so a summary has room to finish instead of being cut off. If you set `SUMMARY_MODEL_CONFIG__MAX_OUTPUT_TOKENS`, it acts as a ceiling: startup fails if either summary cap is larger than it.

`SUMMARY_MAX_INPUT_TOKENS` caps the whole summary prompt: instructions, previous summary and messages. The messages in a summary are chosen by count (`SUMMARY_MESSAGES_PER_*`), so long messages can add up to more than the model accepts. When they do, the oldest messages are left out of that summary, with a warning in the log, and a single message larger than the budget on its own is clipped. Keep it below the context window of the summary model and of its fallback. It must be at least twice the larger of `SUMMARY_MAX_TOKENS_SHORT` and `SUMMARY_MAX_TOKENS_LONG`, because the previous summary is always sent in full and messages need room beside it.

**Dream Processing:**

Dream processing consolidates and refines peer representations during idle periods.

```bash theme={null}
DREAM_ENABLED=true
DREAM_DOCUMENT_THRESHOLD=50
DREAM_IDLE_TIMEOUT_MINUTES=60
DREAM_MIN_HOURS_BETWEEN_DREAMS=8
DREAM_ENABLED_TYPES=["omni"]
DREAM_MAX_TOOL_ITERATIONS=20  # card-refresh dreams only
DREAM_DEDUCTION_MAX_TOOL_ITERATIONS=12
DREAM_INDUCTION_MAX_TOOL_ITERATIONS=10
DREAM_BRIEF_ENABLED=false  # memory layout, counts and the since-last-dream list
DREAM_BRIEF_MAX_NEW_CONCLUSIONS=50
DREAM_BRIEF_MAX_NEW_CONCLUSION_TOKENS=8000
DREAM_BUDGET_ENABLED=false  # budget line + write window for deduction/induction
DREAM_NOTES_MODE=off  # working notes between dreams: off | shadow | on
DREAM_NOTES_MAX_TOKENS=3000
# DREAM_WRITE_WINDOW_ITERATIONS=2  # default: 15% of each specialist's limit
DREAM_HISTORY_ENABLED=true  # Representation History: record each dream's changes
DREAM_HISTORY_RETENTION_DAYS=30

# Specialist model configs (each is independent)
DREAM_DEDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_DEDUCTION_MODEL_CONFIG__MODEL=gpt-5.4-mini
DREAM_INDUCTION_MODEL_CONFIG__TRANSPORT=openai
DREAM_INDUCTION_MODEL_CONFIG__MODEL=gpt-5.4-mini
```

**Surprisal-Based Sampling (Advanced):**

Optional subsystem for identifying unusual observations during dreaming:

> [!WARNING]
>
> Four of the seven tree types — `kdtree` (the default), `balltree`, `graph` and
> `prototype` — require scikit-learn, which ships behind the optional `surprisal`
> extra rather than as a base dependency (it pulls in scipy, \~200MB installed on
> Linux, for a subsystem that is disabled by default). Install it with
> `uv sync --extra surprisal`. The default Docker image does **not** include it,
> so build from source with `docker build --build-arg INSTALL_SURPRISAL=true .`
> or `INSTALL_SURPRISAL=true docker compose up -d --build` when using the example
> Compose file. This installs the dependencies; also set
> `DREAM_SURPRISAL__ENABLED=true` to enable sampling.
>
> `rptree`, `covertree` and `lsh` need only numpy and work out of the box. With
> `DREAM_SURPRISAL__ENABLED=false` (the default) none of this applies.

```bash theme={null}
DREAM_SURPRISAL__ENABLED=false
DREAM_SURPRISAL__TREE_TYPE=kdtree  # needs the 'surprisal' extra; see warning above
DREAM_SURPRISAL__TREE_K=5
DREAM_SURPRISAL__SAMPLING_STRATEGY=recent
DREAM_SURPRISAL__SAMPLE_SIZE=200
DREAM_SURPRISAL__TOP_PERCENT_SURPRISAL=0.10
DREAM_SURPRISAL__MIN_HIGH_SURPRISAL_FOR_REPLACE=10
DREAM_SURPRISAL__INCLUDE_LEVELS=["explicit", "deductive"]
```

## Core Configuration

### Application Settings

```bash theme={null}
LOG_LEVEL=INFO  # DEBUG, INFO, WARNING, ERROR, CRITICAL
SESSION_OBSERVERS_LIMIT=10
GET_CONTEXT_MAX_TOKENS=100000
MAX_MESSAGE_SIZE=25000
MAX_FILE_SIZE=5242880  # 5MB
PAGINATION_COUNT_LIMIT=100000
EMBED_MESSAGES=true
EMBEDDING_MAX_INPUT_TOKENS=8192
EMBEDDING_MAX_TOKENS_PER_REQUEST=300000
NAMESPACE=honcho
```

`PAGINATION_COUNT_LIMIT` caps the rows a list endpoint counts for `total`. Up to the limit `total` is exact; beyond it, `total` is the limit plus one (`100001` by default), raised just past the current page while a next page exists, so `pages` still carries a client to the end of the list. Cursor pagination (`?cursor=`) never counts.

**Optional Integrations:**

```bash theme={null}
TELEMETRY_ENABLED=true  # required for Langfuse
LANGFUSE_HOST=https://cloud.langfuse.com
LANGFUSE_PUBLIC_KEY=your-langfuse-public-key
LANGFUSE_SECRET_KEY=your-langfuse-secret-key
COLLECT_METRICS_LOCAL=false
LOCAL_METRICS_FILE=metrics.jsonl
REASONING_TRACES_FILE=traces.jsonl
```

Langfuse traces are exported from the same LLM call stream as
[CloudEvents telemetry](#cloudevents-telemetry), so the exporter only starts
when `TELEMETRY_ENABLED=true`. With telemetry off, Langfuse keys are ignored.
`TELEMETRY_ENDPOINT` is not required: without it, Langfuse still gets traces
and no CloudEvents are sent.

Honcho uses Langfuse Python SDK v5, which requires Langfuse Cloud or a
self-hosted Langfuse server at v4.59.0 or later. A self-hosted Langfuse v3
server does not accept its traces.

### Database

```bash theme={null}
# Connection (required)
DB_CONNECTION_URI=postgresql+psycopg://postgres:postgres@localhost:5432/postgres
# Role that owns the schema and runs migrations; defaults to DB_CONNECTION_URI
# DB_OWNER_CONNECTION_URI=postgresql+psycopg://owner:owner@localhost:5432/postgres

# Pool settings
DB_SCHEMA=public
DB_POOL_PRE_PING=true
DB_POOL_SIZE=10
DB_MAX_OVERFLOW=20
DB_POOL_TIMEOUT=5
DB_POOL_RECYCLE=300
DB_POOL_USE_LIFO=true
DB_SQL_DEBUG=false
# Per-connection establish timeout (seconds) so a single connection attempt
# fails fast instead of hanging when the server/pooler is unreachable.
DB_CONNECT_TIMEOUT_SECONDS=2
# Seconds a migrator waits for another migrator's lock before failing
DB_MIGRATION_LOCK_WAIT_SECONDS=300
```

### Authentication

```bash theme={null}
AUTH_USE_AUTH=false  # Set to true to require JWT tokens
AUTH_JWT_SECRET=your-super-secret-jwt-key  # Required when auth is enabled
```

Generate a secret: `python scripts/generate_jwt_secret.py`

### Cache (Redis)

Redis caching is optional. Honcho works without it but benefits from caching in high-traffic scenarios.

```bash theme={null}
CACHE_ENABLED=false
CACHE_URL=redis://localhost:6379/0?suppress=true
CACHE_NAMESPACE=honcho
CACHE_DEFAULT_TTL_SECONDS=300
CACHE_DEFAULT_LOCK_TTL_SECONDS=5  # Cache stampede prevention
```

With the cache enabled, offset-paginated list requests (`?page=N`) are also served faster: after each page, Honcho remembers where it ended, so the next page seeks straight to it instead of scanning past every earlier row. Responses are unchanged, except that `total` on pages after the first may be up to `CACHE_PAGINATION_COUNT_TTL_SECONDS` old.

```bash theme={null}
CACHE_PAGINATION_OFFSET_SHIM=true
CACHE_PAGINATION_POSITION_TTL_SECONDS=600
CACHE_PAGINATION_COUNT_TTL_SECONDS=60
```

List endpoints also accept a `cursor` query parameter (empty for the first page, then each response's `next_page`), which pages by cursor with no count at all, with or without the cache.

### Webhooks

```bash theme={null}
WEBHOOK_SECRET=your-webhook-signing-secret
WEBHOOK_MAX_WORKSPACE_LIMIT=10
```

### Vector Store

```bash theme={null}
VECTOR_STORE_TYPE=pgvector  # Options: pgvector, turbopuffer, lancedb, qdrant, chromadb
VECTOR_STORE_MIGRATED=false
VECTOR_STORE_NAMESPACE=honcho
# Embedding dim is configured via EMBEDDING_VECTOR_DIMENSIONS — see the
# Embedding Configuration section. VECTOR_STORE_DIMENSIONS is deprecated.

# Turbopuffer-specific
VECTOR_STORE_TURBOPUFFER_API_KEY=your-turbopuffer-api-key
VECTOR_STORE_TURBOPUFFER_REGION=us-east-1

# LanceDB-specific
VECTOR_STORE_LANCEDB_PATH=./lancedb_data

# Qdrant-specific
VECTOR_STORE_QDRANT_URL=http://localhost:6333
VECTOR_STORE_QDRANT_API_KEY=your-qdrant-api-key  # optional
VECTOR_STORE_QDRANT_PREFER_GRPC=false
VECTOR_STORE_QDRANT_GRPC_PORT=6334
VECTOR_STORE_QDRANT_HTTPS=false  # optional, inferred from URL scheme
VECTOR_STORE_QDRANT_PREFIX=  # optional, for reverse-proxy path prefix
VECTOR_STORE_QDRANT_TIMEOUT=  # optional, request timeout in seconds

# ChromaDB-specific
VECTOR_STORE_CHROMA_CLIENT_MODE=http        # Options: http, cloud
VECTOR_STORE_CHROMA_HOST=localhost          # http mode (self-hosted server)
VECTOR_STORE_CHROMA_PORT=8000               # http mode
VECTOR_STORE_CHROMA_SSL=false               # http mode
VECTOR_STORE_CHROMA_API_KEY=your-chroma-api-key  # cloud mode (required)
VECTOR_STORE_CHROMA_TENANT=your-tenant-id        # cloud mode (optional)
VECTOR_STORE_CHROMA_DATABASE=your-database       # cloud mode (optional)
```

LanceDB is an optional extra and is not included in the default Docker image. Build with `docker build --build-arg INSTALL_LANCEDB=true .` (or `INSTALL_LANCEDB=true docker compose up -d --build`), or run `uv sync --extra lancedb` for manual setups. Note the extra is unavailable on Intel macOS.

The ChromaDB backend requires the `chromadb` extra, which is not installed by default. Install it for manual deployments as shown below. The standard Docker image does not include ChromaDB, and this backend does not currently have an `INSTALL_CHROMADB` build argument.

```bash theme={null}
uv sync --extra chromadb
```

Use `http` with a self-hosted Chroma server or `cloud` with Chroma Cloud. Honcho rejects embedded `persistent` mode because its API and workers run in separate processes, and [Chroma does not support concurrent writers sharing a local persistence path](https://cookbook.chromadb.dev/core/system_constraints/). For local development, run `chroma run --path ./chroma_data --port 8001` and set `VECTOR_STORE_CHROMA_PORT=8001` so Chroma uses a different port from Honcho.

ChromaDB queries retry connection failures, rate limiting, and temporary server errors up to three attempts, waiting 0.1 seconds and then 0.2 seconds between attempts. These short retries can recover brief interruptions; they may not outlast rate-limit windows. ChromaDB 1.5.9 disables HTTP timeouts, so retries do not bound request duration: a stalled server can leave a request waiting indefinitely. If all attempts fail with transient errors, queries return no results. Failed writes remain eligible for reconciliation. Authentication and invalid-request errors remain visible. Startup dimension sampling fails if a sampled namespace cannot be inspected or has a dimension mismatch; missing namespaces are allowed. Selecting ChromaDB without its optional dependency raises an installation error when the backend is initialized.

## Monitoring

### Prometheus Metrics

Honcho exposes `/metrics` endpoints for scraping:

* **API process**: Port 8000
* **Deriver process**: Port 9090

```bash theme={null}
METRICS_ENABLED=false
METRICS_NAMESPACE=honcho
```

### CloudEvents Telemetry

```bash theme={null}
TELEMETRY_ENABLED=false
TELEMETRY_ENDPOINT=https://telemetry.honcho.dev/v1/events
TELEMETRY_HEADERS='{"Authorization": "Bearer your-token"}'
TELEMETRY_BATCH_SIZE=100
TELEMETRY_FLUSH_INTERVAL_SECONDS=1.0
TELEMETRY_MAX_RETRIES=3
TELEMETRY_MAX_BUFFER_SIZE=10000
```

#### Trace payloads

Enable content-addressed LLM traces alongside telemetry:

```bash theme={null}
TELEMETRY_ENABLED=true
TELEMETRY_TRACE_PAYLOADS_ENABLED=true
# Optional: an empty list captures all call purposes.
TELEMETRY_TRACE_PURPOSES='["dialectic.answer", "deriver.representation", "summary.short", "summary.long"]'
```

`llm.call.traced` and `embedding.call.traced` use schema version 2. LLM traces
record each provider attempt, including retries, fallback calls, stream setup
failures, and interrupted streams. Stream traces preserve the output received
before interruption. `trace.content` remains at schema version 1.

| Fields | Meaning |
| - | - |
| `workspace_name`, `session_id` | Workspace and canonical internal session ID, when the operation has a session. Session names are not used as trace identity. |
| `run_id`, `trace_id`, `span_id`, `parent_span_id`, `iteration`, `step_seq` | Run and call correlation. Single-call operations can have a null iteration and step zero. |
| `observers`, `observed`, `peer_name`, `agent_type`, `track_name` | LLM producer identity. `observers` lists every observer the call writes to, including single-observer agents, so batched derivation is not a special case. |
| `source_message_ids`, `queue_item_ids` | Public message IDs and internal queue IDs associated with the LLM operation. Summary message IDs cover its current input window; these fields do not enumerate retrieved tool context or history compressed into an earlier summary. |
| `duration_ms`, `outcome`, `error_class`, `attempt`, `retry_attempts`, `is_final_attempt` | Execution diagnostics for both LLM and embedding calls. `is_final_attempt` means the configured retry budget is exhausted, so an earlier successful attempt can have `false`, and so can a failed attempt that ends the call early because the provider rejected the request and no fallback is configured. |
| `was_stream`, `was_fallback`, `effective_max_output_tokens` | LLM execution mode, selected fallback, and effective output limit. |
| `system_prompt_ref`, `system_prompt_refs`, `output_reasoning_ref` | References into `trace.content` for the system prompt and normalized provider reasoning details. `system_prompt_refs` lists each system message as the provider saw it; `system_prompt_ref` is a single ref for the whole system prompt — the lone message when there is one, otherwise their concatenation shipped under its own hash. |
| `custom_instructions_ref` | Reference into `trace.content` for the operator-written custom instructions added to this call's prompt (role `__custom_instructions__`, `honcho_authored=false`). Unset when no instructions applied. Unlike the system prompt that embeds them, this record is tenant-visible. |

Fields remain null or empty when unavailable or inapplicable. Root spans have no
parent; `parent_event_id` is populated only when a caller supplies an actual
causal event ID. `raw_response_ref` remains reserved because capture stores
normalized responses rather than provider wire payloads. Stream token counts
include only usage exposed by the normalized backend; embedding input tokens
remain estimates. Older trace payloads can omit the new fields.

The CloudEvent ID formula is unchanged. Use the identity and correlation fields
in `data` to group calls; content references continue to deduplicate by hash.

### Sentry

```bash theme={null}
SENTRY_ENABLED=false
SENTRY_DSN=https://your-sentry-dsn@sentry.io/project-id
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
SENTRY_PROFILES_SAMPLE_RATE=0.1
```

## Reference config.toml

A complete config.toml with all defaults. Copy and modify what you need:

```toml theme={null}
[app]
LOG_LEVEL = "INFO"
SESSION_OBSERVERS_LIMIT = 10
EMBED_MESSAGES = true
NAMESPACE = "honcho"

[db]
CONNECTION_URI = "postgresql+psycopg://postgres:postgres@localhost:5432/postgres"
POOL_SIZE = 10
MAX_OVERFLOW = 20

[auth]
USE_AUTH = false

[cache]
ENABLED = false
URL = "redis://localhost:6379/0?suppress=true"
DEFAULT_TTL_SECONDS = 300

[deriver]
ENABLED = true
WORKERS = 1

[deriver.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[peer_card]
ENABLED = true

[dialectic]
MAX_OUTPUT_TOKENS = 8192

[dialectic.levels.minimal]
MAX_TOOL_ITERATIONS = 1
MAX_OUTPUT_TOKENS = 250
TOOL_CHOICE = "any"

[dialectic.levels.minimal.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.low]
MAX_TOOL_ITERATIONS = 5
TOOL_CHOICE = "any"

[dialectic.levels.low.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.medium]
MAX_TOOL_ITERATIONS = 2

[dialectic.levels.medium.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.high]
MAX_TOOL_ITERATIONS = 4

[dialectic.levels.high.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dialectic.levels.max]
MAX_TOOL_ITERATIONS = 10

[dialectic.levels.max.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[summary]
ENABLED = true
MAX_TOKENS_SHORT = 1500
MAX_TOKENS_LONG = 6000
MAX_INPUT_TOKENS = 100000

[summary.model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dream]
ENABLED = true

[dream.deduction_model_config]
transport = "openai"
model = "gpt-5.4-mini"

[dream.induction_model_config]
transport = "openai"
model = "gpt-5.4-mini"

[webhook]
MAX_WORKSPACE_LIMIT = 10

[metrics]
ENABLED = false

[telemetry]
ENABLED = false
TRACE_PAYLOADS_ENABLED = false
TRACE_PURPOSES = []
TRACE_MAX_BYTES = 262144

[vector_store]
TYPE = "pgvector"

[sentry]
ENABLED = false
```

## Database Migrations

```bash theme={null}
uv run alembic current          # Check status
uv run alembic upgrade head     # Upgrade to latest
uv run alembic downgrade <rev>  # Downgrade to specific revision
uv run alembic revision --autogenerate -m "Description"  # Create new migration
```

Migrations hold a session-level advisory lock, so they need a session-mode connection (direct Postgres or a session pooler), not a transaction-mode pooler. A Supabase-style `:6543` port is rewritten to `:5432` automatically.

## Troubleshooting

1. **Database connection errors** — Ensure `DB_CONNECTION_URI` uses `postgresql+psycopg://` prefix. Verify database is running and pgvector extension is installed.

2. **Authentication issues** — Generate and set `AUTH_JWT_SECRET` when `AUTH_USE_AUTH=true`. Use `python scripts/generate_jwt_secret.py`.

3. **LLM provider errors** — Verify API keys are set. Check model names match your provider's format. Ensure models support tool calling.

4. **Deriver not processing** — Check logs. Increase `DERIVER_WORKERS` for throughput. Verify database and LLM connectivity.

5. **Dialectic level issues** — Unset level fields inherit from the built-in defaults. For Anthropic, `THINKING_BUDGET_TOKENS` must be >= 1024 when enabled. For providers without budgeted thinking, omit it or set it to `0`. `MAX_OUTPUT_TOKENS` must exceed `THINKING_BUDGET_TOKENS`.

6. **Vector store issues** — For Turbopuffer, set the API key. Check that `EMBEDDING_VECTOR_DIMENSIONS` matches your embedding model — the startup validator will refuse to boot on a mismatch.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.