How to Read This Changelog
How to Read This Changelog
Each release is documented with:
- Added: New features and capabilities
- Changed: Modifications to existing functionality
- Deprecated: Features that will be removed in future versions
- Removed: Features that have been removed
- Fixed: Bug fixes and corrections
- Security: Security-related improvements
Version Format
Honcho follows Semantic Versioning:- MAJOR version for incompatible API changes
- MINOR version for backwards-compatible functionality additions
- PATCH version for backwards-compatible bug fixes
Honcho API and SDK Changelogs
- Honcho API
- Python SDK
- TypeScript SDK
- Honcho CLI
Added
- Multi-tenancy: one deployment can serve many tenants, behind
MULTI_TENANT(default off). Every tenant-scoped table now carries atenant_idthat leads its primary key, unique constraints and foreign keys. With the flag on, Postgres row-level security scopes every read and write to the tenant named by the caller’s JWT, which must carry a tenant claim (tn); an admin key is admin only within its own tenant. Each tenant’s vectors live under their own namespace, keyed by the tenant’svector_correlation_id, and a tenant without one is refused rather than given an empty namespace. With the flag off nothing changes: Honcho runs single-tenant on plain Postgres, and the optionalTENANT_IDnames the instance’s tenant (defaultdefault). It is fixed for the life of the database, and boot is refused if the rows carry another tenant (#1264, #1546) - Tenant registry at
/v3/tenants, for a control plane that provisions tenants: create (idempotent; the same id with different fields is a 409), get, list, delete, andPATCHforderivation_pausedand a set-oncevector_correlation_id. It authenticates withTENANT_API_SECRET, not a JWT, and is disabled unless both that secret andMULTI_TENANTare set (#1264) - The deriver claims work round-robin across tenants, so one tenant’s backlog cannot starve another’s, and skips the billable work of a tenant with
derivation_pausedset. Deletion, webhooks and other maintenance keep running, and paused work stays queued and is reported on its own per-tenant gauges.DERIVER_TENANT_FAIRNESS_METRICS(default off) adds per-tenant claim counts, claim wait and in-flight units. UnderMULTI_TENANT, CloudEvents carry atenantidattribute, the attribution counters atenant_idlabel, and Sentry atenant_idtag (#1264) - Startup refuses to boot under
MULTI_TENANTwhen isolation cannot hold: a transaction-mode pooler (DB_POOLER_MODE), RLS not enforced, no RLS-bypassing service role (DB_SERVICE_CONNECTION_URI), auth disabled on an API instance, or a tenant role that can read a partition directly. Every deployment, single-tenant included, also refuses to boot when the deriver’squeue_item_batchestable or its triggers are missing (#1264, #1552) - Representation History: each dream is recorded as a run, readable through
POST /dreams/list(newest first, filtered byobserved_idand optionallyobserver_id) andGET /dreams/{run_id}. A run lists the conclusions the dream created, each with the conclusions it was derived from; the conclusions it removed, as they read when removed, including ones a near-duplicate from the same dream replaced (replaced_by); and the peer card before and after. Each change is recorded in the transaction that makes it, so failed and interrupted runs show what they committed. Created conclusions and their sources are resolved when the run is read:current,removed(with the text it had andremoved_in_run), orunavailableonce deleted. Only removed conclusions’ text is copied; deleting their session deletes the copies, and deleting a conclusion through the API deletes its copy, including one a dream already removed; deleting a workspace deletes its history. Changes are kept forDREAM_HISTORY_RETENTION_DAYS(default 30); after that a run keeps only the created conclusions that still exist and getspruned_at, and a run with nothing left is deleted. Recording can be turned off withDREAM_HISTORY_ENABLED=false. Workspace and admin keys only. DDL-only migrationa7e3c1d9f2b4addsdream_runsanddream_run_changes(#1542) DB_OWNER_CONNECTION_URI(optional) is the connection migrations run on; unset, they useDB_CONNECTION_URIas before. When it is set, the migrations grant the app role (andDB_SERVICE_CONNECTION_URI’s role, if set) access to the tables and sequences the owner created, so the app can run as a role that owns nothing. UnderMULTI_TENANTthe owner also owns the message-search functions, so it must be able to bypass RLS; the startup validator refuses to boot otherwise (#1264)INSTALL_SURPRISALDocker build argument (defaultfalse) installs the scikit-learn dependencies for surprisal-based dream sampling, independently ofINSTALL_LANCEDB. Both example Compose services pass it through. Any value other thantrueorfalsefails the build (#1522)
Changed
- Upgrading runs 14 migrations.
e5fe7f8bcf62addstenant_idto every tenant-scoped table and rebuilds their primary keys, unique constraints and foreign keys withtenant_idfirst. Adding the column is catalog-only, but the key rebuilds block writes to each table while they run, and take longer the bigger the table is. Each statement runs under the migrations’ 5-minutestatement_timeout, so plan a window for a large database. An upgraded database keeps its tables unpartitioned. The new and replaced indexes (conclusion lists, peer context, session context, session peers, the reconciler claim, message text search) build without a long write lock; to build them ahead of the upgrade, run the steps inmigrations.utils.create_index_onlineby hand under the same names, and the revisions find them done. Releases from 3.2.2 on run against the new schema, so they stay safe rollback targets (#1264) - The deriver caps its concurrent work units at
DERIVER_WORKERS_PER_POOL_CONNECTION(default 4.0) times its DB pool (DB_POOL_SIZE+DB_MAX_OVERFLOW), soDERIVER_WORKERScan no longer exhaust the pool under load. With the default pool that is 120 units, so it only lowers a deriver that runs a small pool with many workers; the deriver logs the cap at boot when it binds and exports it asderiver_effective_worker_cap. Deployments onDB_POOL_CLASS=nullare not capped.DERIVER_WORKERSnow accepts up to 512, from 100 (#1264) - The deriver claims from
queue_item_batches, a one-row-per-work-unit aggregate that triggers onqueuekeep current, instead of aggregating the queue on every poll. The migration backfills it from unprocessed items, so an instance upgraded mid-backlog claims exactly what was pending (#1264) - Message search indexes lead with
tenant_idandworkspace_name, so a search narrows to the workspace before reading the heap. They needbtree_gin, which the migrations create; where the migration role can’t create it, the old indexes stay and search works as before (#1264) - The dream-due scan reads per-collection counters of explicit conclusions, which the document writers keep current, instead of grouping every explicit conclusion on each pass (#1264)
- Offset-paginated list totals are counted up to
PAGINATION_COUNT_LIMIT(default 100,000) and reported as that limit plus one when there are more, so a list call on a large workspace no longer counts every row on every page. Pages still reach the end of the list, and cursor pagination is unchanged (#1500) - Under
DERIVER_SCHEDULER=api, one API replica measures the deriver backlog, enqueues due dreams and cleans up stale claims for the fleet, under a lease in Redis, and the others serve its snapshot. Before, every replica ran the same cross-tenant counts. Without Redis, or on any Redis error, each replica measures for itself as before (#1557) - The reconciler’s pending-document gauge counts up to 100,000 instead of running an uncapped count on every pass (#1552)
- Message and conclusion reads (lists, lookups, search, session context, representation, clone) also skip rows under a tombstoned workspace, peer or session, matching parents within the tenant. Representation History does the same: a run about a tombstoned peer or in a tombstoned workspace is not listed and its
GETis a 404, and conclusions and removed copies from a tombstoned session resolve asunavailable. Migration4f9c2a7e8d31adds two partial indexes on tombstoned peers and sessions, built online. Nothing setsdeleted_atyet, so behavior is unchanged (#1474, #1548, #1559)
Fixed
- A Redis error no longer fails a request that reads a cached workspace, peer, session or collection: the read falls back to the database and logs a warning.
CACHE_MAX_CONNECTIONS(default 50) sets the Redis pool size per process, or per node withCACHE_CLUSTER; before, the cache library’s default of 10 applied. When a Redis Cluster client cannot initialize (for example, no startup node answers), the error is now raised as aConnectionError: startup retries and then falls back to the in-memory cache, and a request reads the database. Before, the client was left with no nodes, and its next command raisedIndexError(#1550) GET /workspaces/{id}/queue/statuscounts with one row per session instead of one row per queue item, so a large backlog no longer makes it slow. The response is unchanged (#1550)- Each process runs at most a quarter of its DB pool (
DB_POOL_SIZE+DB_MAX_OVERFLOW) of message enqueue writes at once, so a burst of enqueues cannot take every connection from request handlers (#1550) - A tool loop on the Anthropic transport no longer fails with a 400 (
thinking.signature: Input should be a valid string) after a proxy serves one of its rounds from a non-Anthropic fallback. That round’s thinking block has no signature, and Anthropic rejects any request that replays one. When a request holds an unsigned thinking block, the Anthropic backend removes every thinking block from it and keeps the text and tool calls; budget-thinking models (Claude 4.5 and earlier) run that call with thinking off, and adaptive models keep their thinking settings. Requests whose thinking blocks are all signed are unchanged (#1529) - A tool loop whose fallback model on another provider answered a round no longer fails when the next round goes back to the primary (Anthropic:
Unexpected role "tool"; OpenAI: rejected Anthropic content blocks). History is kept as typed messages and rendered for each attempt’s provider (#1501) - Message and conclusion creation no longer returns 422 when content quotes a tokenizer special token such as
<|endoftext|>. The string is tokenized as ordinary text everywhere content is counted or embedded, including the token estimators, which used to fall back silently to a length guess (#1517) - A queue item with a malformed work-unit key no longer stops the deriver. Before, it leaked the worker’s slot, so a one-worker deriver stopped claiming for good. Its pending rows are now marked processed with a sanitized error, and the worker’s slot is released even if cleanup fails (#1507)
- A structured-output attempt that fails is recorded in LLM telemetry and Langfuse with the model’s raw output, and an attempt whose response the deriver rejects (an empty, truncated, filtered or refused representation) is recorded as an error instead of a success. Langfuse generations carry
outcomeanderror_classmetadata so failed ones can be filtered (#1498)
Added
- The workspace session list can report and filter by scope membership.
?include_scopes=truegives each session ascopeslist of the scopes it is in ([]for none), fetched in one query per page; without it the response is unchanged. Bodyscopelists only that scope’s sessions (404 if the scope doesn’t exist) andin_no_scope: truelists only sessions in no scope; both combine withfiltersand pagination, andtotalcounts the filtered result. Clients no longer need to list every scope’s members to show which scopes a session is in (#1477) - The scope list can filter by name and count each scope’s sessions. Body
filterstakes{"id": {"icontains": "sup"}}and the other list operators, matching the unprefixed scope name (never the internalscope.prefix), pluscreated_atandmetadata.?include_session_count=truegives each scope asession_countof its active member sessions, counted the same wayscopes/{id}/sessions/listlists them, in one query per page; without it the response is unchanged (#1478) POST /scopes/{scope_id}/sessions/removeremoves up to 100 sessions from a scope in one request, mirroring bulk add. The batch is all-or-nothing: if any named session doesn’t exist it returns 404 and no membership changes. Sessions that aren’t members are ignored, and only sessions whose membership actually ended get ascope_removalreconciliation task, so a stray ID can’t clear the scope’s dream notes. Memberships are locked in session order, so it can’t deadlock against a concurrent bulk add of the same sessions (#1480)DIALECTIC_MAX_QUERY_LENGTH([dialectic] MAX_QUERY_LENGTH) sets the longest peer chatqueryaccepted, in characters, up to 200,000. It defaults to 10,000, the previous fixed limit, so nothing changes unless you raise it; workspace chat keeps its fixed 10,000-character limit. The dialectic’s prefetch search now truncates a query over the embedding token limit instead of erroring and silently losing every prefetched conclusion (#1443)- Explicit conclusions cite the messages they came from. The deriver emits
source_indicesper conclusion, resolved against the batch and stored as edges in a newconclusion_source_messagestable (DDL-only migrationc4d8e2f6a1b3, no backfill: citations are per-derivation, so rows written earlier surface null). Conclusions responses carrysource_message_ids(public message ids, citation order; null for derived levels), andsource_message_idsjoins the documents filter allowlist with the samecontains/in/ bare-list grammar assource_ids, soPOST /conclusions/listanswers “which conclusions cite this message?”. Cited messages may belong to any peer in the batch, not only the observed one. When a conclusion is derived again, exact or semantic deduplication merges the new citations into the kept conclusion, and copies made when a session joins a scope keep theirs. The batch-levelmessage_idsmetadata is unchanged (#935)
Changed
- Offset-paginated list requests count at most
PAGINATION_COUNT_LIMITrows (default 100,000) fortotal, instead of every row the list selects, so a list call on a large workspace no longer pays for the whole workspace on every page. Up to the limittotalis exact; beyond it,totalreports the limit plus one (100001by default) to signal there are more, raised just past the current page while a next page exists, sopagesstill carries a client to the end of the list, wheretotalbecomes exact again. Thetotalfield’s description in the OpenAPI spec documents this. With the cache disabled, offset requests now also run through the keyset machinery the shim uses. Cursor pagination is unchanged and never counts - Updating a workspace’s
configurationreplaces the stored object instead of merging into it, so{}clears explicit settings and the workspace falls back to system defaults. Before, keys left out of the update were silently kept. Omittingconfiguration, sending null, or updating only metadata leaves it as it is; peer and session configuration updates still merge. To change one setting, send the full configuration (#1458) - Peer chat (the dialectic) answers as a single turn. Answers no longer end with an offer or a question the caller can’t reply to (“If you want, I can…”, “Which statement is correct?”); contradictions in memory are still reported. A query where a harness wraps the end user’s message is no longer answered as if Honcho were the assistant in that conversation. Workspace chat is unchanged (#1451)
- Streamed peer and workspace chat return the answer the dialectic’s tool loop already has, instead of discarding it and making a second, streamed LLM call over the same context. A streamed answer now usually arrives as a single chunk when the loop finishes rather than token by token; only an answer synthesized at the tool-call limit still streams as it is generated. Streaming saves that extra call, returns the same answer as non-streaming, and no longer comes back empty when the extra call returned only thinking (#1443)
- Groundwork for deleting workspaces, peers, sessions and messages. Migration
d8c4f6a2b901adds a nullabledeleted_atcolumn to each (no default, backfill or index), and workspace, peer, session and scope reads skip rows that are tombstoned or whose parent is. Nothing setsdeleted_atyet, so behavior is unchanged (#1429, #1450) - Langfuse Python SDK upgraded from 4.6.1 to 5.0.2, which sends traces on the Langfuse v4 ingestion path ahead of Langfuse Cloud’s v4 cutover on 2026-11-16. Self-hosted Langfuse must be v4.59.0 or later; a v3 server no longer receives Honcho’s traces. Observation metadata values are now JSON-encoded on the span (
"honcho"rather thanhoncho), so saved Langfuse metadata filters written against the old shape may need updating. Trace names, user ids and the trace tree are unchanged (#1452) - Summaries the provider withholds for their content (finish reason
content_filter,safety, …) are no longer reported to Sentry as errors, since nothing on Honcho’s side can fix them. They are counted in thesummary_rejectionsmetric withreason="content_filtered"instead (#1457)
Fixed
- Removing a session from a scope and adding it back no longer loses the scope’s memory of that session. The removal’s cleanup task could run after the re-add (another deriver replica, a retry, a backlog) and soft-delete the session’s scope conclusions, including ones derived after the re-add, plus the scope’s dream notes and the session’s backfill status. The cleanup now locks the membership and skips if the session is a member again. Session and workspace deletion now delete memberships before conclusions, in the order the cleanup takes them, so the two can’t deadlock (#1479)
- Chat, the deriver, summaries and dreams no longer fail when an OpenAI-compatible model returns a usage total but omits the prompt and completion counts. The answer is kept and the missing counts are recorded as zero, instead of the request erroring and being sent again (#1460)
- Claude 5.x models work as configured models instead of failing over to the fallback. The Anthropic backend sends adaptive thinking with an
effortinstead of a thinking budget on Claude 4.6+ and 5.x, relaxes a forcedtool_choicetoautoon models that reject it (Opus and Sonnet 5.5), skips the assistant prefill where it’s rejected, and stops replaying earlier thinking blocks once the tool loop has changed the tools or history (#1442) - Requests the API can’t store or parse return 422 instead of 500: a path ID that fails the name pattern on a get-or-create route, a number outside Postgres’s range, NUL or a lone surrogate in a JSON field, a
pageabove 2^31−1, and a malformed or empty PDF upload. NUL is stripped from workspace, peer and messageconfiguration, as it already was from metadata. A numeric metadata filter ({"gte": 0}) no longer fails when the key holds mixed value types; strings that aren’t safely numeric drop out of ordered comparisons. Two requests racing to create the same workspace no longer turn the loser into a 409 (#1445) - Peer cards written by the agents’
update_peer_cardtool and session summaries no longer fail to save when the model output contains a NUL character, which Postgres rejected along with the whole write. NUL is stripped first, and a summary that is only NUL is retried like an empty one (#1455) - When a message is both a short and a long summary boundary (every long boundary under the default 20/60 cadence), both summaries are now kept. The two saves run concurrently, and each wrote back the whole summaries map it had read, possibly from cache, so the later one could drop the other or revert it to its previous version. Each save now merges only its own entry, on the server (#1493)
- A summary that fails to fetch or save at such a boundary is no longer dropped without a trace. Only a refused (empty) summary reached the queue; any other failure let the queue item complete with no log line, and that window of messages never reached a later summary. Every failure is now logged with its summary type and raised to the queue, preferring a transient error so the item is retried. On the retry, a summary that was already saved is skipped (#1493)
- A deriver queue item whose error message contains a NUL or a lone surrogate is now marked errored. Postgres rejected that error text, so the item was never marked and the worker ran it again straight away, LLM call included, over and over. Those characters are now stored escaped (
\x00,\ud83d). If marking an item errored fails for any other reason, the worker now releases the work unit and backs off instead of rerunning the item at once (#1492)
Security
- pypdf moves from 6.18.1 to 6.21.2 for three high-severity advisories where a crafted PDF makes parsing run for a long time (GHSA-gmh7-3g9r-mjh5, GHSA-jr96-gvgh-j3r4, GHSA-6c7f-9265-g8fw).
POST .../messages/uploadpasses any authenticated caller’s PDF to the parser. Parsing also moves off the event loop, so a slow PDF holds up only its own upload instead of every request on that API worker (#1476)
Security
- The Docker image ships certifi 2026.10.19, up from 2026.4.22, so outbound TLS from the API and the deriver (LLM providers, embeddings, webhooks) verifies against the current CA bundle. The build’s uv moves from 0.9.24 to 0.13.5. Self-hosted deployments that install from
uv.lockneed uv 0.13.5 or newer: older uv ignores the lockfile’s per-packageexclude-neweroverride for certifi and resolves an older bundle (#1439)
Fixed
- In filters, a
"*"wildcard or an empty{}now counts as TRUE insideAND,ORandNOTinstead of being dropped.{"OR": [{"id": "*"}, {"id": "bob"}]}used to narrow to bob and now matches every row, and{"NOT": [{"id": "*"}]}or{"NOT": [{}]}used to match every row and now matches none. A wildcard or{}at the top level still selects every row.source_ids: ["a", "*"]now matches conclusions linked toainstead of every conclusion. Queries never widen past their workspace (#1433) - LLM calls no longer retry a request the provider rejected outright when no fallback model is configured. A 4xx other than 408, 409, 425 or 429 (for example a 400 for a prompt over the context window, or a 401) now fails on the first attempt instead of being sent three times with about 8s of backoff. Transient failures (408, 409, 425, 429, 5xx, connection errors) and the response checks that consume attempts (refused or empty summaries and representations, structured answers that miss their schema) are retried as before. With a fallback configured nothing changes: the rejected request is still retried and the final attempt goes to the fallback. Applies to the deriver, summarizer, dialectic and dreamer, which all call through
honcho_llm_call(#1435)
Added
- Workspace, peer and session responses carry
uid, the immutable identifier Honcho assigns (the row’s internal id).idis unchanged and stays the client-chosen name, so clients can start storinguidas a reference that won’t move if names become renameable. Every route that returns these objects includes it: get-or-create, list, update, clone, session peers, peer sessions and scope sessions. Additive, no migration (#1394) - Named-scope API keys: a key bound to one named scope, for recall and for message ingestion (including creating new member sessions). Recall automatically uses the key’s projection, session access requires active scope membership, and a restricted key cannot enroll unrelated sessions or use global resource routes. New keys carry a dedicated JWT audience, so an older server rejects them instead of ignoring the scope claim and granting workspace access. On self-hosted instances with authentication enabled, mint one with
scripts/generate_jwt.py; the scopes documentation lists the supported routes and limits. Managed (Groudon) keys need separate support, and the high-level Python SDK and CLI helpers still attempt workspace provisioning, so restricted callers use REST or the SDKs’ HTTP transports (#1341) - Dreamer working memory, all opt-in and off by default so today’s behaviour is unchanged: the dreamer and the dialectic show the agent its remaining tool budget (
DREAM_BUDGET_ENABLED/DIALECTIC_BUDGET_ENABLED) and a brief describing how memory is laid out, with conclusion counts by level and, for the dreamer, the conclusions created since the last dream (DREAM_BRIEF_ENABLED/DIALECTIC_BRIEF_ENABLED). The dreamer also reserves a write window at the end of each pass so planned writes are not lost when the iteration limit hits, and keeps capped working notes per representation from one dream to the next (DREAM_NOTES_MODE:off,shadoworon). Deduction and induction get their own iteration limits (DREAM_DEDUCTION_MAX_TOOL_ITERATIONS/DREAM_INDUCTION_MAX_TOOL_ITERATIONS);DREAM_MAX_TOOL_ITERATIONSnow applies to card refresh only. Telemetry gainshit_iteration_limit,write_window_usedand thenotes_*/new_conclusions_*fields (#1378) - Telemetry emits
workspace.created,peer.created,session.createdandscope.createdCloudEvents, once per row actually inserted, including resources created implicitly (posting messages into a new workspace, a workspace created throughPOST /peers). Repeat requests emit nothing. Still gated byTELEMETRY_ENABLED(defaultfalse), so self-hosted deployments without telemetry are unaffected (#1360)
Changed
- This release includes a migration. It adds a
deletion_generationcounter (INTEGER NOT NULL DEFAULT 0, a catalog-only change) to workspaces, sessions and collections, and five partial indexes over unprocessed queue rows so pending work can be found by the peer, session or scope peer it touches. Nothing reads the counter or the indexes yet; they prepare the deletion lifecycle work, and the previous release’s code runs fine against the new schema. The indexes are builtCONCURRENTLY, so writes continue, but a build waits for open transactions that touchedqueueand for any older snapshot in the database, and after 60 minutes the migration fails and leaves an invalid index, which the next run drops and rebuilds. One index (ix_queue_pending_observers) needs thebtree_ginextension; where it cannot be created, the migration skips that index and logs the statement to create it later. On large or busy databases, build the indexes ahead of the upgrade with the statements in the troubleshooting guide, and the migration then has nothing to do (#1409) - Startup warns when an LLM API key is set for a transport no workload uses — across primary models, fallbacks, every dialectic level and embeddings — as provider migrations can leave populated credentials behind silently. The warning names only the setting, never the value, and does not block startup (#1344)
Fixed
provider_params.thinking_tool_choice_conflictnow applies on the OpenAI transport, where Anthropic models behind an OpenAI-compatible proxy used to ignore it. With thinking requested, it covers structured-output requests (the proxy turns the schema into a forced tool call) and forced tool choices.override_thinkingdrops thinking for the call;override_toolrelaxes the tool choice toautoand requests the schema asjson_object;throwraises. Unset leaves requests unchanged, but a config that already sets the key on the OpenAI transport now has it applied, including against OpenAI’s own reasoning models, so set it only where the model is Anthropic. An unknown value now fails config load instead of failing requests with a 422 at runtime. Previously a proxy could leave the schema tool optional when thinking was on, so chats with aresponse_formatcould come back as prose and fail with a 500 (#1402)- The deriver accepts
PromptRepresentation.explicitas a bare string or a list of strings from OpenAI-compatible structured-output backends, coercing each non-blank string into an observation instead of failing validation and dropping the whole batch to the empty fallback. Blank strings are dropped; genuinely malformed shapes still raise (#538) - The deriver fails loudly on malformed or degraded structured output instead of silently discarding the batch as an honest empty: bad payloads raise inside the retried LLM call, so they are retried, reach the fallback model, and error the queue item if every attempt fails. A clean-stop
{"explicit": []}is still accepted. The summarizer gets the same treatment for blank responses cut off or filtered. Failure telemetry carries only the failure class, model and a payload digest, never the payload (#1373) - A chat request whose
response_formatanswer still fails its schema after retries returns 422 (The model's answer did not match the requested response_format) instead of an opaque 500 that the SDKs retry and Sentry drops (#1379) - Tool-loop answers that miss their
response_model— a plain-string refusal, orjson_objectmode’s empty answer — are checked inside the retried call and raise, so they use the configured retries and fallback model before the request fails, instead of returning as a success and only then failing validation (#1393) - Dialectic chat no longer overflows the prompt on oversized tool output: message tools scope sessions in SQL instead of binding every session name as a parameter (which past 65,535 sessions hit the PostgreSQL parameter cap), tool error text is capped at
LLM.MAX_TOOL_OUTPUT_CHARSinstead of returning the expanded SQL statement to the model, and prompt truncation caps a single tool result at half the budget and drops middle units before the opening query (#1390) - Request bodies containing a lone surrogate escape (the half of a split emoji, or text decoded with the wrong codec) return the usual 422 validation body instead of a 500, so clients lose the single bad field’s error rather than the whole batch (#1387)
Added
- ChromaDB vector store backend: set
VECTOR_STORE_TYPE=chromadband pick a client withVECTOR_STORE_CHROMA_CLIENT_MODE:http(self-hosted Chroma server, the default) orcloud(Chroma Cloud, needsVECTOR_STORE_CHROMA_API_KEY). Embeddedpersistentmode is rejected at startup, since it isn’t safe across Honcho’s API and deriver processes. Each Honcho namespace gets its own Chroma collection and index. Ships as an optional extra (uv sync --extra chromadb), so default installs and the Docker image don’t include it (#868) - Custom instructions reach every agent, not just the deriver. A new top-level
custom_instructionsin workspace and session configuration is the shared fallback, andsummary,dialectic(new section),dream, andpeer_cardeach take their owncustom_instructionsoverride. Message, session, and workspace levels are checked in that order and the first that sets a value wins; within a level the module’s value beats the shared one, and""means no instructions.reasoning.custom_instructionsstays deriver-only, andpeer_cardinstructions add to the dream instructions without falling back to the shared value. No migration: configuration is JSONB (#1303) - Telemetry records custom instructions.
representation.completed, the summary event,dialectic.completed, anddream.specialistcarrycustom_instructions_tokensandcustom_instructions_source(e.g.session.shared,workspace.summary);llm.call.tracedlinks the injected text throughcustom_instructions_ref; Langfuse generations carry it under thecustom_instructionsmetadata key. Additive fields, so_schema_versionis unchanged (#1309) python -m src.migrate [revision]runs migrations under a Postgres advisory lock, so replicas that start together no longer race onalembic upgrade(tuple concurrently updated). A waiter polls for up toDB_MIGRATION_LOCK_WAIT_SECONDS(default 300), then fails naming the holder’s pid.scripts/migrate_db.pyandscripts/provision_db.pywrap it (#1268)- List endpoints take an opt-in
cursorquery parameter: send it empty for the first page, then pass each response’snext_page. Cursor mode returns a keyset-paginated page with noCOUNTand one index seek per page, so deep walks stay cheap. Withoutcursorthe response is the unchangedPage, and a server that predates this ignores the parameter, so clients can detect support from the response shape. A malformed cursor returns a 400 (#1326)
Changed
- The deriver prompt states the target peer’s id as the subject of every conclusion about that peer, instead of “the target peer” or “the user” (#1273)
search_messages(and thesearch_memoryfallback andsearch_messages_temporal) drops whole trailing snippets when its output exceeds the tool budget and appends[N of M snippets shown - K omitted to fit output budget], instead of cutting the text mid-snippet with no notice. Tool-call telemetry reportswas_truncatedand the pre-truncation size (#1265)- Reconciler backfills are registered through a generic
Backfilldefinition, each scheduled asreconciler:backfill.<name>only while it has pending rows. The existing document-sources backfill keeps its queue shape, so pre-3.3 workers sharing the queue still run it during a rolling deploy (#1275) /openapi.jsonno longer hardcodeshttps://api.honcho.devas its first server. Self-hosted/docs“Try it out” and generated clients target the origin they were served from; behind a prefix-stripping proxy, set--root-path(#1103)- Gemini
gemini-embedding-2models embed up to 8192 input tokens instead of being capped at 2048. Other Gemini models keep the 2048 cap (#925) uv.lockbumpsstarlette(1.0.0 → 1.6.0, CVE-2026-48710),python-multipart,anyio, andidnato patched versions, which the Docker image picks up throughuv sync --frozen(#1266)- Session summaries ask for about 60% of their token cap instead of a hard word limit, leaving headroom so dense content finishes instead of being cut off mid-summary. The default caps rise from 1000/4000 to 1500/6000 (
SUMMARY_MAX_TOKENS_SHORT/SUMMARY_MAX_TOKENS_LONG), so default summaries stay about the same length. WhenSUMMARY_MODEL_CONFIG__MAX_OUTPUT_TOKENSis set it is a ceiling: settings fail to load if either cap exceeds it, so a deployment that sets it below 6000 must lower the caps to match or raise it (#1318) - The deriver’s semantic deduplication looks up candidates for the whole batch in one query instead of one query per conclusion (#1202)
- With the cache enabled, offset-paginated list requests (
?page=N) seek from where the previous page ended instead of scanning past every earlier row, so installed SDKs walking long lists get faster without an upgrade. Responses are unchanged, except thattotalon pages after the first may be up toCACHE_PAGINATION_COUNT_TTL_SECONDS(default 60) old.CACHE_PAGINATION_OFFSET_SHIM(defaulttrue) turns it off,CACHE_PAGINATION_POSITION_TTL_SECONDS(default 600) sets how long a page position lives, andpagination_offset_shim{outcome}counts first, hit, and miss pages. Adds thesqlakeysetdependency (#1326) - Message search’s
content ILIKEfallback is backed by a newpg_trgmGIN index,ix_messages_content_trgm, instead of scanning every message in the workspace (scans held connections for 10s+ on large deployments). This release includes a migration. The index is roughly 1.5× the size ofmessages, so plan disk headroom before upgrading. It is builtCONCURRENTLY, so writes continue, but the build waits for open transactions that touchedmessagesand for any older snapshot in the database, such as a long-running query on any table. After 60 minutes the migration fails and leaves an invalid index, which the next run drops and rebuilds. On large or busy databases, build the index ahead of the upgrade with the statement in the troubleshooting guide, and the migration will no-op. Ifpg_trgmcan’t be created (no privilege, or not available on the server), the migration skips the index and logs the statement to create it later (#1335)
Removed
DIALECTIC_HISTORY_TOKEN_LIMITandDREAM_HISTORY_TOKEN_LIMIT, along with the unusedget_recent_historyagent tool they fed. A leftover value is ignored (#1308)
Fixed
- Cloning a session matched the source’s messages, peers, and cutoff message by session name alone, so it could copy a same-named session’s data from another workspace or fail with a 500. Lookups are scoped to the workspace. Cloned messages also keep
token_count, and cloning a session with no messages copies its peers and commits instead of returning a 201 for a session that was rolled back (#1284) - JWTs with an expiry (
--expiresonscripts/generate_jwt.py,expires_atonPOST /v3/keys) failed every request with a 401, becauseexpwas written as an ISO string instead of a NumericDate. Expiring keys now authenticate until they expire, and an expired key returnsJWT expired(#1025) - OpenAI
gpt-6models failed with a 400unsupported_parameterbecause Honcho sentmax_tokens; they now getmax_completion_tokenslikegpt-5and the o-series (#1233) - A summary that hits its cap while repeating itself (fewer than 35% distinct 4-word sequences) is discarded and the previous summary kept, instead of overwriting it. Rejections are counted in
summary_rejections{summary_type,reason}(#1318) - Session peer and webhook list endpoints had no
ORDER BY, so their pages could skip or repeat rows. They order by(joined_at, id)and(created_at, id)(#1326) - A connection dropped by the pooler during the representation’s semantic conclusion lookup no longer fails the request with
PendingRollbackError. On read-only sessions the connection is rolled back and the lookup degrades to no semantic conclusions (#1335)
Added
CACHE_CONNECT_TIMEOUT_SECONDS(default 5) andCACHE_CONNECT_RETRIES(default 3). The Redis client retries a failed connect with exponential backoff (0.1s base, 0.5s cap), so an instance that boots alongside several others no longer fails on a singleTimeout connecting to server(#1234)
Changed
- Listing a peer’s sessions uses a new
session_peers (workspace_name, peer_name, session_name)index and starts from that peer’s rows instead of walking every session in the workspace (28.9ms to 0.15ms on a 326k-row test table). This release includes a migration. The index is not builtCONCURRENTLY, so writes tosession_peerswait while it builds (#1237) - LLM provider 5xx and connection errors are no longer sent to Sentry on every attempt by the Anthropic, OpenAI, and Google GenAI SDK integrations. Honcho reports the final error once its retries are exhausted, so an outage that recovers on retry no longer alerts. Client errors (400, 401, 404, 429) are still reported (#1242)
- Dialectic and dreamer runs are typed
agentin Langfuse, so they appear in the Agent Graph. Generation output with tool calls is an OpenAI-style assistant message carrying the tool arguments and any thinking, instead of just the tool names. Thelangfusefloor rises to>=4.6.1(#1258) - The deriver prompt and output schema no longer carry example facts. New rules resolve relative dates against each message’s
time, treat details in a question or proposal the target peer accepts as stated by them, skip conversational mechanics (greetings, thanks, acknowledgements), and keep one fact per conclusion with its qualifiers attached and no reworded duplicates (#1263)
Removed
- Inline Langfuse instrumentation and the
LANGFUSE_EXPORTER_MODEsetting. Langfuse traces now come only from the exporter over the captured LLM call stream, which has been the default since 3.0.12. A leftoverLANGFUSE_EXPORTER_MODE=inlineis ignored. The exporter only registers whenTELEMETRY_ENABLEDis true, so deployments that set Langfuse keys with telemetry off no longer get traces (#1253)
Fixed
- Context routes (
GET /peers/{id}/context,POST /peers/{id}/representation,GET /sessions/{id}/context) truncate asearch_querylonger thanEMBEDDING.MAX_INPUT_TOKENS(8192) before embedding it. Before, the embed failed and the route returned a 200 with semantic results silently dropped. Session context also releases its DB connection before the embedding call (#1255) - Langfuse traces reflect what happened: generations show the call’s real latency, every trace has one root observation,
user_idand the trace name sit on every observation, run and step spans stay open until the run ends, and tool spans carry the result and real duration (#1235) - A Redis connection dropped while idle is reconnected and the command retried, instead of failing the request. uvloop raised a
RuntimeErroron the closed transport, which redis-py did not treat as a connection error (#1245)
Added
- Chat evidence conclusions carry
observer_idandobserved_id, matching the Conclusions response. Workspace chat reads across every peer into one evidence list, so each row now says which pair it belongs to without re-fetching it (#1193) thinking_tool_choice_conflictprovider param for Anthropic model configs, which decides what happens when extended thinking meets a forced tool choice (Anthropic rejects the combination):throw(default, raises a validation error before the request is sent),override_thinking(drops thinking for that call), oroverride_tool(relaxes the tool choice toauto). Set it per model config, e.g.DIALECTIC_LEVELS__medium__MODEL_CONFIG__OVERRIDES__PROVIDER_PARAMS__THINKING_TOOL_CHOICE_CONFLICT=override_thinking(#1211)
Changed
source_idsis[]rather thannullfor explicit conclusions on the Conclusions response, matching chat evidence. Clients that checked fornullshould check for an empty list (#1193)- Workspace chat keeps its forced tool choice until a recall tool runs cleanly, capped at 3 forced rounds, so an answer always rests on at least one search instead of the prefetched overview. A tool that returns an error does not count.
get_workspace_statsis removed from the workspace toolset, since the prefetch already supplies the same overview (#1210) - The deriver wraps each batch message in a tag carrying its peer and whether it is a target, and derives only from target messages, instead of relying on the model to match a name prefix. Message content that imitates the tag is escaped so it cannot claim target attribution (#1192)
- Smaller runtime footprint: the Docker image drops from 627MB to 426MB. PDF extraction uses pypdf instead of pdfplumber, provider SDKs import only for the configured backend, and scikit-learn moves behind a new optional
surprisalextra. Self-hosted deployments that enableDREAM.SURPRISAL.ENABLEDwith an sklearn-backed tree type need the extra;rptree,covertree, andlshstill work without it (#1201)
Fixed
- Chat with
include_evidenceno longer fails with “Instance is not bound to a Session” when evidence is read after its DB session closes (#1212) - On Redis Cluster, each process releases the idle connection its startup PING leaves on the default node. Every client picks the same slot-0 primary, so these sockets piled up there (92% of that node’s connections in one observed cluster) and could hit
maxclients, which stops every new client from initialising (#1198) - Dreamer tools return an actionable error for malformed arguments (a string instead of a list, non-object items) instead of raising a
TypeErrormid-loop. Valid sibling observations in the same call still land (#1217) - Deleting a session no longer loads the content of every embedding chunk to rebuild vector IDs; chunk counts are aggregated in SQL (#1206)
Added
- Conclusion attribution on the public API:
source_idsandtimes_derivedon the Conclusions response (already stored on documents, previously stripped).GET /v3/workspaces/{workspace_id}/conclusions/{conclusion_id}returns a single conclusion with those fields.source_idsis filterable via JSONB containment ({"source_ids": {"contains": "<id>"}});id,level,source_ids, andtimes_derivedare explicit entries in the documents filter allowlist. The samecontainsfix applies to other JSONB columns, which previously generated invalid ILIKE-on-JSONB SQL. Message attribution (conclusion → source messages) and chain traversal are follow-ups (#952) - Opt-in
include_evidenceon pair and workspace chat (POST /peers/{id}/chatandPOST /workspaces/{id}/chat). When true, the response carries the conclusions and messages the agent read plus the tools it called, so a caller can inspect what an answer was built from. Evidence is collated from what the agent accessed, never reported by the model — it over-reports (a conclusion appears because it was seen, not because the answer used it) but is deterministic and costs no model tokens. Off by default so existing callers are unchanged. Streaming puts evidence on the terminal SSE event only (#1129)
Changed
- CloudEvents LLM and embedding traces (schema version 2) now carry session, workspace, observers, observed, agent type, track name, source message ids, queue item ids, duration, outcome, retry, and a full
system_prompt_ref. Streamed calls record once at the provider boundary, including interrupted streams. New fields are nullable so v1 archives still load (#1166) - Request middleware falls back to
User-Agentas the CloudEventsclient.hostwhenX-Honcho-Hostis absent, so raw REST clients are distinguishable from SDKs without waiting for SDK upgrades. An explicitX-Honcho-Hoststill wins (#1182)
Fixed
- Dreamer-written conclusions no longer persist fabricated
source_ids. Unresolvable ids are stripped at the write path; an observation left below its level’s minimum real sources is rejected (#945) - LLM provider outages (connection failures and upstream 5xx) now surface as 503 instead of an opaque 500. Exhausted retries re-raise the provider error instead of a tenacity
RetryError. 4xx including 429 is unchanged. The streaming path is not covered (#1165) - Peer-scoped keys on
POST /sessionsandPOST /peersmay only name their own peer, must already be a member to get an existing session, and may not send metadata or configuration for one. Creating a fresh session for itself is unchanged. Session-scoped keys are denied onPOST /peers(#1172) get_working_representationreclaims unused query budget when semantic or most-derived search returns fewer unique documents than requested, filling the remainder from recent observations while preserving search priority (#1008)- Redis Cluster async client no longer leaks connections on topology re-init. The pin is now
redis>=8.0.1,<9.0.0(was<8.0.0, which excluded the upstream fix in redis-py 8.0.0) (#1157)
Added
- Qdrant vector store backend (
VECTOR_STORE_TYPE=qdrant) as an optionalqdrantextra. Point at a server withVECTOR_STORE_QDRANT_URL(optional API key, gRPC, HTTPS, prefix, timeout). SameVECTOR_STORE_MIGRATEDcutover path as the other backends (#683) - MCP stdio host (
bun --cwd mcp src/stdio.ts) for local clients, plus a Streamable HTTP host (bun src/http.ts/mcp/Dockerfile) with anmcpcompose service. HTTP requires Bearer on every request including established sessions; idle sessions expire afterMCP_SESSION_IDLE_MS(default 30m) and are capped atMCP_SESSION_MAX(default 128). Sessions are in-process — run one replica (#1102) - MCP tools take
scope(name or list) andsessions(session-id allowlist) onchat,page/size/reverseonlist_sessionsandget_session_messages, pluslist_scopes,get_scope_sessions, andworkspace_chat(#1139) - Deterministic OpenAI-compatible mock provider (
src/mock_provider) so the stack can run with no model key and no spend. Same image, different entrypoint; chat answers from the request’s JSON Schema and embeddings are hash-derived (lexical search only — no semantic recall) (#1094) - Ephemeral sandbox (
sandbox/sandbox.sh): seed snapshots a Postgres template database, reset drops and recreates it (plus Redis flush) without restarting services. Default provider is mock;--provider realreads gitignored credentials. Reset refuses a stale snapshot (Alembic revision / fixture hash / provider mode) (#1111) DERIVER.SCHEDULER=api(envDERIVER__SCHEDULER) moves deriver/dream timers onto the API process so deriver replicas only consume work. Default remainsderiver. When the API owns scheduling, deriver startup poll jitter is skipped (#1136, #1149)- Deriver backlog as Prometheus gauges on the API (
deriver_outstanding_work_seconds,deriver_queue_work_units_eligible/_claimed,deriver_queue_items_pending,deriver_queue_oldest_pending_age_seconds,dreams_due,deriver_metrics_last_success_timestamp_seconds) and as JSON atGET /deriver/metrics. Service-wide DB values — aggregate withmax()/avg(), neversum()(#1115) - Docker API worker count via
API_WORKERS(default 1) on the image entrypoint (#1088) - Client identity on CloudEvents: request middleware reads
X-Honcho-Host,X-Honcho-Plugin, andX-Honcho-Agent-Modelinto a nestedclientobject next tohoncho_version. Null outside a request (deriver worker). Emitter-injected fields are exempt from per-event schema versioning (#1125)
Fixed
- Workspace chat requires a tool call on the first turn instead of answering from the prefetch overview alone.
lowwas the only level that left tool choice asauto, and those calls were skipping search. Pair chat is unchanged. The workspace prompt is also marked non-interactive so it stops offering the caller a menu (#1120)
Changed
- Server
requires-pythonis>=3.13, matching the production image. Self-hosters on 3.10–3.12 need to upgrade; SDK and CLI floors are unchanged (#1090)
Fixed
- Concurrent
create_documentswriters to the same collection deadlocked ontimes_derivedreinforcement UPDATEs issued in batch order; the error was swallowed per-document, the batch was lost, and the queue item was marked processed. Writers now lock target rows withSELECT ... ORDER BY id FOR UPDATEbefore applying, abort the batch onSQLAlchemyErrorinstead of continuing through a dead session, and retry transient errors (deadlock, serialization failure, lock/statement timeout, lost connection) up toMAX_RETRYABLE_ATTEMPTSinstead of burning the item (#1033) - Scope backfill no longer embeds, writes, and syncs every planned copy at once. A 14k-document session is ~580MB of vectors; several concurrent backfills OOM-killed the deriver at its 1000Mi limit and crash-looped because the work units never completed. Phases 2–4 now run per chunk of 500 specs, reload source embeddings per chunk, and drop them once synced. Membership is locked across chunk writes so a concurrent leave cannot commit between the check and the inserts (#1104)
- Model-generated observations with NUL bytes (
\u0000) no longer fail the exact-content dedup pre-fetch with a PostgresDataErrorthat dropped the whole observer batch. Ingress already stripped NUL from user content; the deriver now strips it so stored text matches embedded text. All-NUL content is dropped rather than stored empty (#1095) search_messagesno longer forwardstop_k=0to Turbopuffer (which requires 1..10000). Zero/negative limits short-circuit to empty results; tool limits are floored at 1. The documents path was already guarded (#970); this closes the message path (#1084)- OpenAI-compatible tool-call turns with
content=nullkeep null through history replay instead of being coerced to"". Providers that bind reasoning state to the exact assistant message shape were breaking on the empty string. Tool-less null still becomes""(#1064) - The production image now ships
pyproject.tomlin the runtime stage, so the service reports its real version instead ofunknownin OpenAPI and telemetry (#1074)
Added
- Scopes: a named grouping of sessions that acts as a visibility boundary on recall, implemented as a facade over an observer peer (
scope.{name}with{"kind": "scope"}). Developers manage them exclusively through/v3/workspaces/{workspace_id}/scopes(create-or-get, list, get, add/list/remove session membership) and an optionalscopesfield on session create — never through the observer/observed mechanics. Scope peers cannot author messages, cannot be a chat or representationtarget, are excluded frompeers.listby default (PeerGet.kind="scope"/"all"switches the view), and are rejected on the generic session-peer routes. Workspace-level key required; peer- and session-scoped keys get 401. Legacy peers occupying a reservedscope.name without the kind flag are refused with 409, never adopted (#884) scoperead option on chat, representation, session context, and workspace search. A single scope swaps the observer to the backing scope peer so conclusion recall, peer cards, and message tools stay inside that scope’s membership. A list of scopes takes the union of member sessions (capped atMAX_SESSION_ALLOWLIST_ENTRIES) and executes via the session-allowlist path. Empty scopes fail closed.scopeis mutually exclusive withfiltersandsession_id. Workspace- or admin-level key required (403 otherwise). Scope peers are also rejected aspeer_target/peer_perspectiveon session context and as the path peer ortargetonGET /peers/{id}/context(#897)- Scope backfill-by-copy and removal reconciliation. Adding a session that already has messages copies its explicit-level documents into the scope’s collections (no LLM re-derivation; idempotent via
copied_from). Removing a session soft-deletes those copies and fail-closed cascades to derived documents whosesource_idsintersect anything removed, then enqueues acard_refreshdream withrebuild=Trueplus an omni dream.GET /v3/workspaces/{workspace_id}/scopes/{scope_id}/statusreports per-session backfill state (pending/completed/failed, plusdocs_copied) (#904) - Workspace-level chat at
POST /v3/workspaces/{workspace_id}/chat: agentic dialectic over the whole workspace instead of a single (observer, observed) pair. Prefetches workspace stats and the top active peers’ self cards, then searches pair-scoped memory with[observer->observed]attribution. Supportssession_id,scope,reasoning_level,response_format, and SSE streaming (#931) - MCP workspace discovery: tools accept
workspace_id, the worker honors an optionalX-Honcho-Workspace-IDconnection header, andlist_workspace/create_workspacetools let clients pick or create a workspace instead of relying on the SDK default (#1020) - MCP
searchalso queries conclusions in parallel with messages whenpeer_idis given, returning{messages, conclusions}. The conclusions leg degrades to[]on error so search never gets worse than before (#974) - Prometheus metrics for physical DB connections, visible even under
DB_POOL_CLASS=null:db_connections_open(gauge) anddb_connections_established(counter), hooked to SQLAlchemy connection-lifecycle events and registered on both the API and the deriver (#1055) - Bounded-label Prometheus series are zero-initialized at process start so an absent series means a broken scrape rather than “nothing happened” (#927)
Changed
- Workspace and pair chat system prompts now describe Honcho, peers, and the harness on their own terms, and render only the tools the request actually offers. The pair prompt no longer advertises a write tool that is not in the loadout (#1066)
- Deriver idle polling backoff is longer and no longer reset by periodic reconciler work, so downstream connection pools can cull idle DB connections (#1015)
- LLM provider SDKs are lazy-loaded so idle API and deriver processes no longer pay for every provider at import time (#1011)
- Production image is a multi-stage build: LanceDB/PyArrow move behind an optional
lancedbextra (INSTALL_LANCEDB=trueto restore them), FastAPI’s unused cloud CLI is dropped, and the venv is copied into the runtime image with final ownership so Docker does not double the layer. Default unpacked image is about 663 MB (was 1.7 GB) (#1014) - Redis Cluster cache keys hash-tag the namespace so one deployment’s keys land on a single shard instead of opening a connection to every node. No behaviour change on a non-cluster backend; existing keys age out by TTL (#1058)
- Deriver extraction prompt no longer leaks its own few-shot examples into extracted conclusions (#1028)
Fixed
- Observer-scoped
get_observation_contextno longer materializes every session the observer has ever joined into asession_name IN (...)list (twice in one statement). Past ~32k sessions that hit psycopg’s bind-parameter ceiling and 500’d. The observer half is now a correlatedEXISTSoversession_peers, two bind parameters regardless of membership size (#1065) - Re-adding an already-active session peer no longer advances
joined_at, sopeer_perspectivesearch keeps messages from the original join. Genuine leave-and-rejoin still starts a new window (#1059) - Transient embedding-provider errors (for example an OpenAI-compatible 200 with empty
data: []) were relabeled as token-limit errors. Only genuine oversize input raisesEmbeddingTokenLimitError; other provider errors propagate unchanged (#791) - The filter DSL now fails closed with a 422 instead of a 500 on bad shapes, coerces operands by column type (so
{"session_id": {"ne": "abc"}}is a string inequality rather than “invalid numeric”), and treatsNOT/neas null-safe (IS NOT TRUE/IS DISTINCT FROM) so negation no longer drops rows whose field is unset. Closed-set columns likelevelreject unknown values. Session-allowlist entries must be well-formed ids (*is 422, not a silent widen) (#947) neon JSONB metadata keys is null-safe: a missing key is not equal to the compared value, so{"metadata": {"foo": {"ne": "bar"}}}includes rows wherefoois unset (#1036)- Oversized texts in
simple_batch_embedare truncated to the embedding token cap instead of failing the whole batch. Representation processing reports failed observer saves inRepresentationCompletedEventand raises when every observer save fails (#1019) - Assistant
reasoning_content(DeepSeek / some OpenRouter models) is preserved across tool-loop turns. Previously the tool loop dropped thinking content before building the next assistant history message, so continuation requests failed.reasoning_detailsstill takes precedence when both are present (#1034) create_observationsnow honorsDERIVER_DEDUPLICATEinstead of hardcodingdeduplicate=True, matching the representation write path (#1018)provider_params.timeoutis forwarded to the OpenAI-compatible embedding client, not just the LLM client (#1024)- Conclusions semantic-search validation errors name the field and the constraint instead of returning a generic 422 (#960)
- OpenAI-compatible embedding calls request
encoding_format=floatso providers that default to base64 do not break pgvector inserts (#938) - Gemini batch embedding works for
gemini-embedding-2*models, which rejected the previous request shape (#745) - MCP OAuth with no advertised scopes no longer defaults to read-only (which 403’d chat and search POSTs). Protected-resource metadata advertises read and write (#1004)
Added
- Session allowlist on the Dialectic and representation via a constrained
filtersbody onPOST /peers/{peer_id}/chatand/representation, supporting only thesession_idkey (a session id, a bare list, or{"in": [...]}). Unsupported keys and shapes are rejected with 422 rather than silently ignored, it composes withsession_id(which must be included in the allowlist when both are given), and it is capped at 1,000 sessions per request. Enforcement is uniform and fail-closed at every recall chokepoint: scoped conclusion recall is restricted tolevel == "explicit"(dream-derived conclusions carry a singlesession_namebut are synthesized across all sessions, so that stamp can’t be scoped on),get_reasoning_chainis unavailable under an allowlist, and an empty allowlist short-circuits to empty results everywhere. Workspace keys pass the allowlist as-given; peer-scoped JWTs must be an active member of every allowlisted session (401 otherwise) (#882) - Bare-list membership sugar in the filter DSL:
{"session_id": ["s1", "s2"]}is now shorthand for{"session_id": {"in": [...]}}on regular columns generically. JSONB metadata columns are excluded and keep containment semantics. Strictly additive, since a bare list on a regular column previously compiled to a type-mismatched equality that matched nothing (#881) - Optional structured outputs on the Dialectic:
response_format(a JSON Schema with root typeobject) on peer chat makescontenta JSON string conforming to that schema. Only a conservative subset of JSON Schema is supported, with DoS guards and non-recursive$refsupport (#896) - Combined tool calling and structured output in the LLM transport layer, with per-backend request shaping: OpenAI routes tool-carrying structured requests through
create()with an explicitjson_schemaresponse format (parse()500s on non-strict function tools), Anthropic skips the{JSON prefill when tools are present sotool_useblocks stay reachable, and Gemini injects a schema instruction into the final turn instead of using nativeresponse_schema(rejected alongside function calling before Gemini 3). All backends skip structured-output parsing on tool-call turns, which carry no consumable content (#907) card_refreshdream type: a lightweight dream that runs only the peer-card update, for event-driven refreshes such as membership changes and cold starts. Handled by a newCardRefreshSpecialistrestricted toget_recent_observations,search_memory, andupdate_peer_card(no observation-mutating tools) with a tool-iteration cap ofmin(6, DREAM.MAX_TOOL_ITERATIONS).POST /v3/workspaces/{workspace_id}/schedule_dreamacceptsdream_type=card_refreshplus arebuildflag, which omits the existing card from the prompt so the specialist rebuilds it solely from observations present in the collection. Card refreshes never advance the omni dream guard pair (last_dream_at/last_dream_document_count) (#883)- Full-fidelity LLM trace stream, with Langfuse as one projection over it: each call is captured once (
CapturedLLMCall) and fanned out to a CloudEvents trace stream (llm.call.traced/trace.content) and a Langfuse exporter, both reconstructing trace → run → step → generation from the same source of truth. AddsTELEMETRY_TRACE_PAYLOADS_ENABLED(defaultfalse),TELEMETRY_TRACE_MAX_BYTES(default 262144, per-message cap with oversized content clipped),TELEMETRY_TRACE_PURPOSES(JSON list ofCallPurposevalues; empty means all), andLANGFUSE_EXPORTER_MODE(exporterby default;inlineis kept for one release for side-by-side validation). Embedding calls are traced, dreamer branches nest under one dream trace, tool calls become spans under their step, and high-volume events are sampled deterministically.TRACE_ENDPOINTis dropped (#845) - Redis Cluster support via
CACHE_CLUSTER(for example GCP Memorystore for Redis Cluster), alongside a newCACHE_LOCK_WAIT_CHECK_INTERVAL_SECONDS(#905) EMBEDDING_MODEL_CONFIG__MAX_BATCH_SIZEcaps texts per embedding request for OpenAI-compatible providers with smaller limits than OpenAI’s, such as DashScopetext-embedding-v4(10) and Alibaba Bailianqwen3.7-text-embedding(20). When unset, native provider defaults are preserved (OpenAI 2048, Gemini 100) (#983)- Per-request provider timeouts via
provider_params.timeouton any model config, validated at config load so a bad value fails at startup with the exact config path instead of surfacing per-request as a retried 500. Good values normalize to float seconds; Gemini’s is converted to milliseconds (#832) RepresentationCompletedEventnow reports deduplication counts:exact_dup_in_batch_count,exact_dup_existing_count,semantic_dup_rejected_count, andsemantic_dup_replaced_count(#910)- OAuth discovery for MCP clients: the MCP worker serves
/.well-known/oauth-protected-resource(RFC 9728) without auth so clients can discover the authorization server, and a 401 now carriesWWW-Authenticate: Bearer resource_metadata="..."(exposed cross-origin) to start the flow (#923) - Prometheus metrics for the immediate-embed fast path: tasks shed because
EMBEDDING_MAX_PENDING_EMBED_TASKSwas reached, and the current in-flight task count (#892) - Docs: a detailed system architecture diagram, a Codex integration guide (#879), a structured-outputs page (#896), a section on filtering conclusions by reasoning level (#851), a health-check endpoint reference, and SDK updates (#867)
Changed
- Breaking config change:
DERIVER_REPRESENTATION_BATCH_MAX_TOKENSis split into two settings that were previously conflated —DERIVER_REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS(default 512), the producer-side minimum a work unit accumulates before the deriver claims it, where0disables the gate; andDERIVER_REPRESENTATION_BATCH_TARGET_INPUT_TOKENS(default 1024), the consumer-side maximum context-window tokens per deriver LLM call. Deployments setting the old name must migrate (#889) - The immediate-embed fast path now applies backpressure:
EMBEDDING_MAX_PENDING_EMBED_TASKS(default 50) caps in-flight embed tasks, and once saturated, message creation skips the fast path entirely and the reconciler embeds on its next cycle.0disables the fast path (#892) - Explicit-level documents are now kept session-pure, so memory can be built by copying explicit documents between collections. Enforcement refuses rather than rewrites:
create_documentsrejects explicit documents with a nullsession_name, exact dedup keys on (content, level, session-for-explicit), semantic dedup scopes candidate search to the same level and — for explicit documents — the same session, and the genericcreate_observationstool rejectslevel='explicit'outside message-ingestion (deriver) context. Derived levels keep cross-session consolidation (#883) - Sentry’s
before_sendfilter is centralized asdefault_before_sendinsrc/telemetry/sentry.pyinstead of living only in the API’smain.py, so the deriver gets the same non-actionable-exception filtering. All Sentry events also carry anamespacetag for correlation (#934, #870) - The minimal deriver’s extraction examples no longer teach inferences its own output schema forbids. The
EXAMPLESblock demonstrated deriving a specific birthday from “I just had my 25th birthday last Saturday”, deriving residence from a single visit (“I took my dog for a walk in NYC” → “alice lives in NYC”), and a ”+ general knowledge” deductive output the deriver has no channel for. The replacements stay inside the schema’s contract and teach the boundary: the dog/NYC message is kept and shown extracting correctly, and a separate example shows “lives in NYC” is valid when actually stated (#985) - Dreamer specialists are instructed not to output summaries (#894)
session_nameis deprecated for scoping in favor of the session allowlist. It is not removed and not aliased: it also pins the query to one session, bypasses observer scoping, and drives session-history injection into the dialectic prompt, so it has no drop-in replacement (#882)- The MCP worker no longer requires the
X-Honcho-User-NameorX-Honcho-Assistant-Nameheaders (#923)
Fixed
- Session scoping was applied to only one of the three working-representation query paths:
session_namereached the recent-documents query, but the semantic and most-derived paths ignored it, solimit_to_sessionleaked cross-session conclusions into perspectives. The allowlist is now threaded uniformly through all three paths and pushed down to pgvector and external vector stores (#881) - Empty membership lists failed open in the vector-store filter builders, silently widening scope: LanceDB dropped empty
INclauses and Turbopuffer emitted a bareIn []with undocumented semantics. Both now emit an explicit always-false predicate, and_build_filter_conditionschecksis not Nonerather than truthiness so an empty list is no longer treated likeNone(#881, #882) - Session-scoped CRUD helpers ignored the session allowlist entirely, so a caller could read a session the allowlist forbids. The API routes guarded this with a 422, but the dialectic tools call these CRUD functions directly and bypassed it.
_semantic_search_messages(coveringsearch_messagesandsearch_messages_temporal),grep_messages,get_messages_by_date_range,get_recent_history, andget_observation_contextnow return[]whensession_nameis set and outside the allowlist (#882) - The cache client logged the full Redis URL — including the password — at INFO and WARNING on every connection attempt and failure, exposing the live credential in container logs and downstream aggregation. Credentials are now redacted across userinfo, the
?password=(redis-py) and?secret=(cashews) query params, scheme-less URLs whose password is invisible to.port/.passwordparsing, and malformed URLs, whose fallback previously echoed the raw input verbatim (#869) - A
top_kof0reached the vector store, where Turbopuffer rejects it with a 400 (top_k must be between 1 and 10000). A non-positivetop_know returns[]before the embedding call, and the semantic budget floors at 1 so an explicitly requested search isn’t silently allocated zero (#970) - Gemini clients had no HTTP timeout, so a stalled socket wedged the deriver worker’s uvloop event loop, which the in-process reconciler shares. A 10-minute timeout is now set on both the Gemini LLM client and the Gemini embedding client (#903)
- Dreamer conclusions were dated to ingestion time rather than their latest source observation, and their timestamps are now normalized (#890)
- Langfuse I/O annotation was gated on
LANGFUSE_PUBLIC_KEYinstead oflangfuse_inline_enabled, so in the defaultexportermode it calledupdate_current_generation()with no active span — logging “No active span in current context” roughly 14 times per dialectic run and building throwawaymodel_dumppayloads on every LLM call. Separately,AgentToolSummaryCreatedEventhardcodedrun_id="deriver"/iteration=0, pollutingrun_idgrouping in the CloudEvents stream with a phantom run; both fields are now optional and the resource id is keyed onmessage_id:summary_type(schema_version 2 → 3) (#845) - Assistant tool calls were dropped from the captured trace stream for OpenAI and Gemini:
build_captured_messagesread only{role, content, tool_call_id}, but those providers keep tool calls outsidecontent, so replayed tool-call turns landed as empty content and Gemini lost its text and tool results entirely. Tool calls are now normalized per provider into a unifiedtool_callsfield and folded into the content hash. Gemini’sthought_signatureis bytes, somodel_dump(mode="json")raisedUnicodeDecodeErrorinsideemit_trace, silently dropping whole tool-calling iterations from the trace stream (billing and Langfuse were unaffected); it is now base64-encoded on the telemetry path while replay keeps the raw bytes (#845) EmbeddingClient.encodingforced full client construction, raising “OpenAI API key is required” even though tiktoken needs no credentials. The document dedup tie-break only needs.encodingfor token counting, so any test hitting that path failed in environments without embedding keys — notably CI for pull requests from forks. The encoding is now resolved from the configured model directly, falling back tocl100k_base, and the underlying client’s encoding is reused only when it has already been constructed (#955)- The Docker build failed under Podman because the uv build inputs weren’t copied (#878)
- LanceDB was installed on macOS Intel, where it doesn’t work. A PEP 508 marker excludes
darwin/x86_64and the LanceDB vector-store import is wrapped so a misconfiguration surfaces as a clear config error (#496) - Prompt checks requiring the literal token “json” for
json_objectmode are now satisfied in lowercase (#887) - Reverted an unintended
RepresentationCompletedEventschema-version increment - Documented preinstalling pgvector as a privileged role for deployments where the
DB_CONNECTION_URIrole deliberately cannot create extensions (managed Postgres, Kubernetes operators, NixOS).CREATE EXTENSION IF NOT EXISTS vectordoes not help there, because Postgres checks the privilege before checking whether the extension exists. Docker Compose is unaffected, since the bundled stack connects as thepostgressuperuser (#984)
Added
api_request_duration_secondsPrometheus histogram tracking per-route request latency, labeled by method and endpoint (#837)- LLM
provider_paramspassthroughs (extra_body/extra_headers/extra_query) are now forwarded to the underlying provider transport across all backends, with shape validation that rejects non-mapping values (#821) structured_output_modemodel-config option to usejson_objectmode for OpenAI-compatible providers that lack native Structured Outputs support (used by the deriver) (#820)- OpenRouter app-attribution headers (
HTTP-Referer/X-Openrouter-Title) are now sent on OpenAI-compatible clients when the configured base URL is OpenRouter, so requests are attributed to “Honcho” in OpenRouter’s dashboard (#805) - Langfuse traces are now tagged with user and session IDs for easier trace filtering (#814)
DERIVER_REPRESENTATION_BATCH_MAX_AGE_SECONDS(default 1800s) lets sub-threshold representation work units flush once their oldest unprocessed queue item ages out. Set it to0to keep the legacy behavior where sub-threshold tails wait indefinitely unlessDERIVER_FLUSH_ENABLED=true(#826)- Conclusion responses now include a
levelfield (explicit,deductive,inductive,contradiction); list/query endpoints support filtering bylevelviafilters, with reserved filter keys protected from being overridden by user-supplied filters (#851)
Changed
- Peer-scoped JWTs now get read-only access to the sessions their peer is an active member of (session context, summaries, peers, their own per-session config, search, and message reads). Session-scoped JWTs remain confined to their session and cannot reach peer routes (#679)
- Compacted Honcho’s log output, with guarded ms/s metric formatting that falls back to a plain string for non-numeric values (#836)
- Sentry now drops noisy infra/scrape transactions: the reconciler opens a transaction only once a batch has rows (idle cycles emit none), and a
traces_samplerreturns0.0for/metrics,/health,/openapi.json,/docs,/redoc, and the deriver metrics server.SENTRY.TRACES_SAMPLE_RATEstill governs real traffic (#834)
Fixed
- Peer- and session-scoped JWTs were effectively workspace-scoped: authorization walked the route’s declared scope and fell through to a workspace match, so a
{w, p: alice}token could act on any peer in the workspace. JWTs are now authorized by their narrowest claim and never widen to workspace access (#679) - The keys API now rejects creating a peer- or session-scoped key without a workspace. Such keys were minted successfully but failed verification on every request (#679)
- Agent-supplied observation IDs carrying the display-format
id:prefix are now normalized (prefix and trailing whitespace stripped) beforesource_idsare stored and onget_reasoning_chainlookups, fixing corrupted provenance links and broken reasoning-chain traversal (#795) - Fixed a
create_treekeyword-argument mismatch in the Dreamer’s surprisal tree construction (#749) - Providers that omit output-token counts (observed with Gemini on tool-loop completions) returned
output_tokens=None, which raised a Pydantic validation error that aborted the call and crashed the Dreamer’s induction phase before inductive conclusions were persisted.Noneis now coerced to0so token accounting degrades gracefully (#809) - Document creation now performs exact (case-insensitive, whitespace-trimmed) content deduplication before the existing semantic dedup step: exact duplicates within a batch collapse to a single insert, and an exact match against a live document reinforces it (atomic
times_derivedincrement) instead of creating a new row (#861) - The OpenAI backend passed
tool_choicethrough raw while the Anthropic and Gemini backends translate Honcho’s canonical vocabulary to their native form, so on a mixed-provider fallback chain (for example Gemini primary → OpenAI backup) a canonical"any"reached OpenAI unchanged and was rejected as an invalid param. The OpenAI backend now converts it, mirroring the others:any/required→required,auto/nonepass through, and a tool-name string or{"name": ...}dict becomes a function selection (#850) - Langfuse
@observeauto-capture serialized every argument ofhoncho_llm_call_innerinto the generation span input, includingclient_override(a liveAsyncOpenAI/genaiclient) andselected_config(which carriesapi_key). Auto-capture deep-copied the client into a half-constructed object whose teardown raised (AsyncHttpxClientWrapper ... no attribute '_state'on OpenAI, flooding stderr;BaseApiClient ... no attribute '_http_options'on Gemini), and it leakedModelConfig.api_keyinto traces. Capture is now an explicit allowlist:capture_input/capture_outputare disabled and curated, serializable input and output are stamped instead, with tuning knobs surfaced asmodel_parametersvia a secret-bearing denylist and per-call token usage mirrored asusage_details(#849)
Added
- Messages are now embedded via a background task rather than blocking API request
- Read-only DB session mode (
get_read_db/tracked_db(..., read_only=True)) so reads don’t hold a transaction open across the work CORS_ORIGINSenv var to configure CORS allowed origins without editing source; defaults match the prior hardcoded list, so self-hosted deployments behind custom domains can whitelist their frontend (#697)scripts/generate_jwt.py— utility for minting scoped or admin Honcho JWTs (--admin,--workspace/--peer/--session,--expireswith human-friendly durations,--print-only) without calling the keys API (#757)STALE_WORK_UNIT_CLEANUP_INTERVAL_SECONDS(default 60s) — minimum jittered spacing between deriver stale-work-unit cleanup runs, so cleanup no longer runs on every seconds-scale poll (0.0keeps the legacy every-poll behavior) (#773)
Changed
- Optimized the deriver and dreamer prompt cache prefixes to improve prompt-cache hit rates (#806)
Fixed
times_derivedis now properly reinforced when a duplicate conclusion is detected. It had been pinned at 1 for nearly every conclusion (the reject-new branch dropped the increment and the new-wins branch reset the count to 1), soORDER BY times_derived DESCfell back to arbitrary heap order and froze stale conclusions to the front of injected context. Reinforcement is now an atomic increment and both most-derived queries gained acreated_at DESCrecency tiebreaker (#768)- Webhook creation now correctly rejects private/internal IP addresses (#793)
Changed
- Connection acquisition is now a single attempt with no server-side retry, on a vanilla
AsyncSession. A newDB_CONNECT_TIMEOUT_SECONDS(default 2s) bounds the attempt so a saturated or unreachable pooler fails fast instead of holding a client connection open to re-knock. A saturated DB now surfaces to the caller — the API returns an error and the deriver backs off and retries on a later poll — which lets the pooler drain rather than amplifying saturation.
Added
- Deriver poll jitter so instances that start together don’t poll in lockstep:
DERIVER_POLLING_STARTUP_JITTER_SECONDS(random delay before the first poll, default 30s) andDERIVER_POLLING_JITTER_RATIO(±fraction applied to every poll sleep, default 0.5). Both disable at0.0; the underlying backoff schedule is unchanged.
Removed
- Reverted the connection-checkout retry and
HonchoAsyncSessioncustom session introduced in 3.0.8. Removed theDB_CONNECTION_RETRY_ENABLED/DB_CONNECTION_RETRY_MAX_DELAY_SECONDS/DB_CONNECTION_RETRY_BACKOFF_INITIAL_SECONDS/DB_CONNECTION_RETRY_BACKOFF_MAX_SECONDSsettings, thedb_connection_acquisitions{outcome=...}Prometheus counter, and thedb.pool.acquireSentry span. Alerting built ondb_connection_acquisitionsshould migrate todb_pool_connections/db_queries_in_flight.
Added
- Connection-checkout retry with bounded exponential backoff (tenacity) on
get_db/tracked_db: transient transaction-pooler (Supavisor) rejections — SQLAlchemyTimeoutErrorandOperationalError— now retry with backoff instead of surfacing as 500s under client-connection saturation. Gated byDB_CONNECTION_RETRY_ENABLEDwith configurable delay/backoff knobs; ~10s default budget (#758) HonchoAsyncSession— a lazyAsyncSessionthat checks out its pooled connection (with retry) on the first DB-touching call rather than at construction. Request handlers doing non-DB work (embedding, file, LLM) before their first query no longer pin a pooler connection across it. Only the checkout is retried; the statement still runs exactly once, so writes are never duplicated (#758)- Adaptive deriver queue polling: the poll interval backs off when the queue is idle or erroring (base → max, doubling each cycle) and snaps back to base the moment work is claimed, cutting steady-state query load against the DB. Gated by
DERIVER_POLLING_BACKOFF_ENABLEDwith configurable max/multiplier (#758) - New Prometheus
db_pool_connectionsgauge (checked_out / checked_in / size / overflow), labeledapi|deriver, registered in both the API lifespan and the deriver metrics server (#758) - New Prometheus
db_connection_acquisitions{outcome=ok|retried|exhausted}counter — the alertable early-warning signal that connection checkouts are retrying through pooler rejection, before requests start failing (#758) - New Prometheus
db_queries_in_flightgauge — statements actually executing on the wire (via SQLAlchemy cursor-execute events). Paired withchecked_out, the gap reveals connections held but parked (the “idle in transaction during an external call” antipattern). Gated onMETRICS.ENABLEDfor zero overhead when off (#758) - Explicit
SqlalchemyIntegrationin both the API and deriver Sentry inits; connection acquisition wrapped in adb.pool.acquirespan with live pool stats captured on retry exhaustion (#758)
Changed
- Default
POOL_TIMEOUTlowered to 5s, with validation that it stays under the connection-retry budget when a pooled (non-null)POOL_CLASSis configured;config.toml.exampleand the v2/v3 configuration docs updated to match (#758) HonchoAsyncSessionwraps every DB-touching session method (execute / scalar / scalars / flush / merge / refresh / commit / get / get_one / stream / stream_scalars / delete) so the lazy-checkout-with-retry guarantee has no holes; the acquired flag resets onclose()/reset()so a reused session re-acquires on next use (#758)
Fixed
- Roll the session back on a retryable checkout failure before retrying — a failed autobegin could otherwise leave it pending-rollback, making the next connection attempt raise instead of cleanly re-checking-out (#758)
- Guard
DBPoolCollector.collect()so a pool-read/import hiccup can’t raise and abort the entire/metricsscrape (Prometheus drops all metrics if any collector raises) (#758) - Clamp the pool overflow gauge to ≥ 0 (it could report negative before the pool fills) (#758)
- Removed a double-sleep in the deriver idle poll so the backoff cap is a true cap rather than 2× (#758)
Added
- New
src/llm/package as the single owner of provider runtime: clients, backends, history adapters, tool loop, request builder, credentials, and caching policy (#459) - New cloudevent
LLMCallCompletedEvent(llm.call.completed) fires once per provider hit with full cost-attribution context: transport/provider_label, model, token counts with cache breakdown, finish_reason, outcome, retry/fallback state, duration, tool-call shape, streaming flag, and agent correlation (run_id+ iteration) (#637) RepresentationCompletedEventnow carriestotal_input_tokensfor full-trace cost attribution; per-emitterhoncho_versioninjection; deterministic per-run_idhigh-volume sampler viaTelemetrySettings.HIGH_VOLUME_SAMPLE_RATE(#637)- Deriver custom instructions: per-workspace/peer guidance threaded into the deriver prompt with a
MAX_CUSTOM_INSTRUCTIONS_TOKENSbudget (default 2000); deriverMAX_INPUT_TOKENSraised 23000 → 25000 (#609) - Configurable embedding dimensions:
EMBEDDING_MODEL_CONFIG__DIMENSIONS_MODE(auto/always/never) controls whether OpenAIdimensions=is forwarded (#678) - New
honcho-clipackage — Python CLI for inspecting and managing peers, sessions, and configuration against a Honcho deployment (#424) HONCHO_API_URLenv var support in the MCP Worker for self-hosted deployments (#575)- API ID
max_lengthincreased from 100 to 512 acrossWorkspaceCreate,PeerCreate, andSessionCreateto align with the DB schema (#684) AttemptPlandataclass pins per-retry provider selection across stream-final retries so streaming doesn’t bounce back to primary after the tool loop has settled on fallback (#459)- Gemini JSON-schema sanitizer for
function_declarations— strips keywords Gemini’s validator rejects while preserving semantics for other backends (#459)
Changed
- All LLM orchestration moved out of
src/utils/clients.pyintosrc/llm/with modules split by responsibility (#459) - Default
ModelConfigfactories (deriver, summary, dreamer specialists, dialectic levels) normalized with no extra parameters set by default; operators add transport/thinking overrides explicitly (#459) - OpenAI reasoning-model routing widened to cover
gpt-5.xando1/o3/o4— these models receivemax_completion_tokensinstead ofmax_tokens(#459) - Peer card prompts reframed as stable identity markers; induction specialist now opts out of peer card writes so only deduction touches the card (#686)
- Vector store queries no longer fetch embedding vectors — only document metadata is returned, reducing payload size and DB load (pgvector, lancedb, turbopuffer) (#682)
- Langfuse trace metadata now includes
namespace,model, andproviderso traces can be filtered by deployment slice (#565) - Deriver: model-aware tokenizer (replaces the previously hardcoded encoding) and explicit guard on empty message content (#647)
- Dialectic level defaults now merge correctly with per-level overrides (#656)
- Default dialectic tool choice switched to
auto(#630) - Vector sync given a substantial retry budget to tolerate transient embedding provider outages (#604)
AgentToolConclusionsDeletedEventpayload now carrieslevels(#612)- Turbopuffer:
InternalServerErrorcaught and surfaced as a warning rather than a hard failure; vector store sync errors downgraded to warnings (#561)
Fixed
reversequery parameter is now honored on the v3 workspace list, peer list, workspace-scoped session list, and peer-scoped session list. Honcho SDKs at 2.1.0+ were already sendingreverse=truefor these routes but the server silently ignored it. Ties oncreated_atnow fall back to the internal nanoididfor stable ordering across pages (#685)- LLM client factories now receive
base_urlfromLLMSettingsfor default providers — operators pointing at OpenAI-compatible proxies viaLLM__OPENAI_BASE_URLwere previously ignored on the default path (#643, fixes #641) - Internal N+1 query in dialectic agent tool execution — collapsed per-iteration DB lookups into a single fetch (#652)
- Dreamer threshold and time-guard semantics: count filter now includes only
documents.level == 'explicit'(was inflating threshold via dreamer-created levels and creating a feedback loop);last_dream_atwrite relocated from enqueue to process so duplicate enqueues or failed runs no longer reset the 8-hour time guard (#573) - Deriver: blank observations are filtered out before embedding (previously triggered noisy embedding calls and persisted empty rows) (#615)
- Surprisal module: filter format corrected from
{"level": levels}to{"level": {"in": levels}}— the prior call silently returned 0 results and made the entire Surprisal phase of the Dream cycle a no-op (#581, fixes #559) - Removed hardcoded
stop_sequencesoverride from DeriverModelConfig(was clobbering operator-configured stop sequences) (#587) - Embedding client:
embed()now wraps single-string input in an array, restoring compatibility with OpenAI-compatible third-party providers that reject scalar input (#586) - Docker Compose: deriver service startup gated on the API service healthcheck — prevents races where the deriver starts before the API has run migrations (#689)
- Docker image:
HEALTHCHECKdirective removed from the shared base image; service-level health checks now belong in each service’s own configuration (#530) - Removed strict parameter validation for thinking params on Anthropic and OpenAI transports — was rejecting valid per-transport configs (#686)
- Stream-final retries pin to the
AttemptPlanthat succeeded rather than re-running provider selection through the outercurrent_attemptContextVar (#459) - Gemini
cached_contentreuse keys now includesystem_instructionandtool_configso cache hits don’t cross configurations (#459) - CrewAI example updated for the latest CrewAI protocol (#631)
Removed
src/utils/clients.pydeleted; its responsibilities are split acrosssrc/llm/registry.py,src/llm/credentials.py, and the backend-specific modules (#459)HEALTHCHECKdirective from the shared Docker image (#530)
Changed
- Tightened transaction scopes across search, agent tools, queue manager, and webhook delivery to minimize DB connection hold time during external operations (#525)
- Search operations refactored to two-phase pattern — external work (embeddings, LLM calls) completes before opening a transaction (#525)
- Agent tool executor performs external operations before acquiring DB sessions (#525)
- Queue manager transaction scope reduced to only the critical section (#525)
- Webhook delivery no longer holds a DB session parameter (#525)
Fixed
- Session leakage in non-session-scoped dialectic chat calls (#526)
Added
- Health check endpoint (
/health) for container orchestration and load balancer probes (#510)
Added
- JSONB metadata validation enforces 100 key limit and max depth of 5 (#419)
Changed
- Schemas refactored from single
schemas.pyintoschemas/api.py,schemas/configuration.py, andschemas/internal.pywith backwards-compatible re-exports (#419)
Fixed
- Missing
deleted_atfilter onRepresentationManager._query_documents_recent()and._query_documents_most_derived()allowed soft-deleted documents to leak into the deriver’s working representation (#456) CleanupStaleItemsCompletedEventemitted spuriously when no queue item was actually deleted (#454)- Empty JSON file uploads caused unhandled errors; now returns normalized error responses (#434)
- Memory leak:
_observation_locksswitched toWeakValueDictionaryto prevent unbounded growth (#419) - SQL injection in
dependencies.py: parameterizedset_configcalls to prevent injection via request context (#419) - NUL byte crashes: string inputs (message content, queries, peer cards) now stripped at schema level (#419)
- Filter recursion depth capped at 5 to prevent stack overflow (#419)
- Dedup-skipped observations now correctly reflected in created counts (#477)
- External vector store support for message search — routes queries through configured external vector store with oversampling and deduplication to handle chunked embeddings (#479)
- Dialectic agent no longer holds a DB connection during LLM calls — embeddings are pre-computed before tool execution, DB sessions isolated in
extract_preferences,query_documentsno longer accepts a DB session parameter (#477)
Added
- Consolidated session context into a single DB session with 40/60 token budget allocation between summary and messages
- Observation validation via
ObservationInputPydantic schema with partial-success support and batch embedding with per-observation fallback - Peer card hard cap of 40 facts with case-insensitive deduplication and whitespace normalization
- Safe integer coercion (
_safe_int) for all LLM tool inputs to handle non-integer values like"Infinity" - Embedding pre-computation and reuse across multiple search calls in dialectic and representation flows
- Peer existence validation in dialectic chat endpoints — raises ResourceNotFoundException instead of silently failing
- Logging filter to suppress noisy
GET /metricsaccess logs - Oolong long-context aggregation benchmark (synth and real variants, 1K–4M token context windows)
- MolecularBench fact quality evaluation (ambiguity, decontextuality, minimality scoring)
- CoverageBench information recall evaluation (gold fact extraction, coverage matching, QA verification)
- LoCoMo summary-as-context baseline evaluation
- Webhook delivery tests, dependency lifecycle tests, queue cleanup tests, summarizer fallback tests
- Parallel test execution via pytest-xdist with worker-specific databases
test_reasoning_levels.pyscript for LOCOM dataset testing across reasoning levels
Changed
- Workspace deletion is now async — returns 202 Accepted, validates no active sessions (409 Conflict), cascade-deletes in background
- Redis caching layer now stores plain-dict instead of ORM objects, with v2-prefixed keys, storage, resilient
safe_cache_set/safe_cache_deletehelpers, and deferred post-commit cache invalidation - All
get_or_create_*CRUD operations now use savepoints (db.begin_nested()) instead of commit/rollback for race condition prevention - Reconciler vector sync uses direct ORM mutation instead of batch parameterized UPDATE statements
- Summarizer enforces hard word limit in prompt and creates fallback text for empty summaries with
summary_tokens = 0 - Blocked Gemini responses (SAFETY, RECITATION, PROHIBITED_CONTENT, BLOCKLIST) now raise
LLMErrorto trigger retry/backup-provider logic - Gemini client explicitly sets
max_output_tokensfrommax_tokensparameter - All deriver and metrics collector logging replaced with structured
logging.getLogger(__name__)calls - Dreamer specialist prompts updated to enforce durable-facts-only peer cards with max 40 entries and deduplication
GetOrCreateResultchanged fromNamedTupletodataclasswithasync post_commit()method- FastAPI upgraded from 0.111.0 to 0.131.0; added pyarrow dependency
- Queue status filtering to only show user-facing tasks (representation, summary, dream); excludes internal infrastructure tasks
Fixed
- JWT timestamp bug —
JWTParams.twas evaluated once at class definition time instead of per-instance - Session cache invalidation on deletion was missing
get_peer_card()now properly propagatesResourceNotFoundExceptioninstead of swallowing itset_peer_card()ensures peer exists viaget_or_create_peers()before updating- Backup provider failover with proper tool input type safety
- Removed
setup_admin_jwt()from server startup - Sentry coroutine detection switched from
asyncio.iscoroutinefunctiontoinspect.iscoroutinefunction
Removed
explicit.pyandobex.pybenchmarks replaced by coverage.py and molecular.py- Claude Code review automation workflow (
.github/workflows/claude.yml) - Coverage reporting from default pytest configuration
Added
- Documentation for reasoning_level and Claude Code plugin
Changed
- Gave dreaming sub-agents better prompting around peer card creation, tweaked overall prompts
Fixed
- Added message-search fallback for memory search tool, necessary in fresh sessions
- Made FLUSH_ENABLED a config value
- Removed N+1 query in search_messages
Added
- Agentic Dreamer for intelligent memory consolidation using LLM agents
- Agentic Dialectic for query answering using LLM agents with tool use
- Reasoning levels configuration for dialectic (
minimal,low,medium,high,max) - Prometheus token tracking for deriver and dialectic operations
- n8n integration
- Cloud Events for auditable telemetry
- External Vector Store support for turbopuffer and lancedb with reconciliation flow
Changed
- API route renaming for consistency
- Dreamer and dialectic now respect peer card configuration settings
- Observations renamed to Conclusions across API and SDKs
- Deriver to buffer representation tasks to normalize workloads
- Local Representation tasks to create singular QueueItems
- getContext endpoint to use
search_queryrather than forcelast_user_message
Fixed
- Dream scheduling bugs
- Summary creation when start_message_id > end_message_id
- Cashews upgrade to prevent NoScriptError
- Memory leak in
accumulate_metriccall
Removed
- Peer card configuration from message configuration; peer cards no longer created/updated in deriver process
Added
- Message level configurations
- CRUD operations for observations
- Comprehensive test cases for harness
- Peer level get_context
- Set Peer Card Method
- Manual dreaming trigger endpoint
Changed
- Configurations to support more flags for fine-grained control of the deriver, peer cards, summaries, etc.
- Working Representations to support more fine-grained parameters
Fixed
- File uploads to match
MessageCreatestructure - Cache invalidation strategy
Added
- Redis caching to improve DB IO
- Backup LLM provider to avoid failures when a provider is down
Changed
- QueueItems to use standardized columns
- Improved Deduplication logic for Representation Tasks
- More finegrained metrics for representation, summary, and peer card tasks
- DB constraint to follow standard naming conventions
Added
- Unified
Representationclass - vllm client support
- Periodic queue cleanup logic
- WIP Dreaming Feature
- LongMemEval to Test Bench
- Prometheus Client for better Metrics
- Performance metrics instrumentation
- Error reporting to deriver
- Workspace Delete Method
- Multi-db option in test harness
Changed
- Working Representations are Queried on the fly rather than cached in metadata
- EmbeddingStore to RepresentationFactory
- Summary Response Model to use public_id of message for cutoff
- Semantic across codebase to reference resources based on
observerandobserved - Prompts for Deriver & Dialectic to reference peer_id and add examples
Get Contextroute returns peer card and representation in addition to messages and summaries- Refactoring logger.info calls to logger.debug where applicable
Fixed
- Gemini client to use async methods
Added
- Get peer cards endpoint (
GET /v2/peers/{peer_id}/card) for retrieving targeted peer context information
Changed
- Replaced Mirascope dependency with small client implementation for better control
- Optimized deriver performance by using joins on messages table instead of storing token count in queue payload
- Database scope optimization for various operations
- Batch representation task processing for ~10x speed improvement in practice
Fixed
- Separated clean and claim work units in queue manager to prevent race conditions
- Skip locked ActiveQueueSession rows on delete operations
- Langfuse SDK integration updates for compatibility
- Added configurable maximum message size to prevent token overflow in deriver
- Various minor bugfixes
Added
getSummariesendpoint to get all available summaries for a session directly- Peer Card feature to improve context for deriver and dialectic
Changed
- Session Peer limit to be based on observers instead, renamed config value to
SESSION_OBSERVERS_LIMIT Messagescan take a custom timestamp for thecreated_atfield, defaulting to the current timeget_contextendpoint returns detailedSummaryobject rather than just summary content- Working representations use a FIFO queue structure to maintain facts rather than a full rewrite
- Optimized deriver enqueue by prefetching message sequence numbers (eliminates N+1 queries)
Fixed
- Deriver uses
get_contextinternally to prevent context window limit errors - Embedding store will truncate context when querying documents to prevent embedding token limit errors
- Queue manager to schedule work based on available works rather than total number of workers
- Queue manager to use atomic db transactions rather than long lived transaction for the worker lifecycle
- Timestamp formats unified to ISO 8601 across the codebase
- Internal get_context method’s cutoff value is exclusive now
Added
- Arbitrary filters now available on all search endpoints
- Search combines full-text and semantic using reciprocal rank fusion
- Webhook support (currently only supports queue_empty and test events, more to come)
- Small test harness and custom test format for evaluating Honcho output quality
- Added MCP server and documentation for it
Changed
- Search has 10 results by default, max 100 results
- Queue structure generalized to handle more event types
- Summarizer now exhaustive by default and tuned for performance
Fixed
- Resolve race condition for peers that leave a session while sending messages
- Added explicit rollback to solve integrity error in queue
- Re-introduced Sentry tracing to deriver
- Better integrity logic in get_or_create API methods
Fixed
- Summarizer module to ignore empty summaries and pass appropriate one to get_context
- Structured Outputs calls with OpenAI provider to pass strict=True to Pydantic Schema
Added
- Test harness for custom Honcho evaluations
- Better support for session and peer aware dialectic queries
- Langfuse settings
- Added recent history to dialectic prompt, dynamic based on new context window size setting
Fixed
- Summary queue logic
- Formatting of logs
- Filtering by session
- Peer targeting in queries
Changed
- Made query expansion in dialectic off by default
- Overhauled logging
- Refactor summarization for performance and code clarity
- Refactor queue payloads for clarity
Added
- File uploads
- Brand new “ROTE” deriver system
- Updated dialectic system
- Local working representations
- Better logging for deriver/dialectic
- Deriver Queue Status no longer has redundant data
Fixed
- Document insertion
- Session-scoped and peer-targeted dialectic queries work now
- Minor bugs
Removed
- Peer-level messages
Changed
- Dialectic chat endpoint takes a single query
- Rearranged configuration values (LLM, Deriver, Dialectic, History->Summary)
Fixed
- Migration/provision scripts did not have correct database connection arguments, causing timeouts
Fixed
- Database initialization was misconfigured and led to provision_db script failing: switch to consistent working configuration with transaction pooler
Added
- Ergonomic SDKs for Python and TypeScript (uses Stainless underneath)
- Deriver Queue Status endpoint
- Complex arbitrary filters on workspace/session/peer/message
- Message embedding table for full semantic search
Changed
- Overhauled documentation
- BasedPyright typing for entire project
- Resource filtering expanded to include logical operators
Fixed
- Various bugs
- Use new config arrangement everywhere
- Remove hardcoded responses
Added
- Ability to get a peer’s working representation
- Metadata to all data primitives (Workspaces, Peers, Sessions, Messages)
- Internal metadata to store Honcho’s state no longer exposed in API
- Batch message operations and enhanced message querying with token and message count limits
- Search and summary functionalities scoped by workspace, peer, and session
- Session context retrieval with summaries and token allocatio
- HNSW Index for Documents Table
- Centralized Configuration via Environment Variables or config.toml file
Changed
- New architecture centered around the concept of a “peer” replaces the former “app”/“user”/“session” paradigm
- Workspaces replace “apps” as top-level namespace
- Peers replace “users”
- Sessions no longer nested beneath peers and no longer limited to a single user-assistant model. A session exists independently of any one peer and peers can be added to and removed from sessions.
- Dialectic API is now part of the Peer, not the Session
- Dialectic API now allows queries to be scoped to a session or “targeted” to a fellow peer
- Database schema migrated to adopt workspace/peer/session naming and structure
- Authentication and JWT scopes updated to workspace/peer/session hierarchy
- Queue processing now works on ‘work units’ instead of sessions
- Message token counting updated with tiktoken integration and fallback heuristic
- Queue and message processing updated to handle sender/target and task types for multi-peer scenarios
Fixed
- Improved error handling and validation for batch message operations and metadata
- Database Sessions to be more atomic to reduce idle in transaction time
Removed
- Metamessages removed in favor of metadata
- Collections and Documents no longer exposed in the API, solely internal
- Obsolete tests for apps, users, collections, documents, and metamessages
Added
- Normalize resources to remove joins and increase query performance
- Query tracing for debugging
Changed
/listendpoints to not require a request bodymetamessage_typetolabelwith backwards compatibility- Database Provisioning to rely on alembic
- Database Session Manager to explicitly rollback transactions before closing the connection
Fixed
- Alembic Migrations to include initial database migrations
- Sentry Middleware to not report Honcho Exceptions
Added
- JWT based API authentication
- Configurable logging
- Consolidated LLM Inference via
ModelClientclass - Dynamic logging configurable via environment variables
Changed
- Deriver & Dialectic API to use Hybrid Memory Architecture
- Metamessages are not strictly tied to a message
- Database provisioning is a separate script instead of happening on startup
- Consolidated
session/chatandsession/chat/streamendpoints
Previous Releases
For a complete history of all releases, see our GitHub Releases page.Getting Help
If you encounter issues using the Honcho API or its SDKs:- Open an issue on GitHub
- Join our Discord community for support