Streaming Responses

Using streaming responses with Honcho SDKs

Honcho’s SDKs can receive chat answers as a stream of text chunks over server-sent events, so your application can consume the answer through the same streaming interface it uses for other model output.

The dialectic is an agent loop. It will stream when it can but it’s designed to answer as fast as possible.

​

When to Use Streaming

Streaming is particularly useful for:

• Chat interfaces that already consume streamed model output
• Server-sent-event pipelines that forward answers to a client
• Long answers that hit the agent’s tool-call limit, which stream as they are generated

​

Streaming with the Chat Endpoint

One of the primary use cases for streaming in Honcho is with the chat endpoint. This lets you receive the answer about a user over the same streaming interface as other model output.

​

Prerequisites

Python

from honcho import Honcho

# Initialize client (using the default workspace)
honcho = Honcho()

# Create or get peers
user = honcho.peer("demo-user")
assistant = honcho.peer("assistant")

# Create a new session
session = honcho.session("demo-session")

# Add peers to the session
session.add_peers([user, assistant])

# Store some messages for context (optional)
session.add_messages([
    user.message("Hello, I'm testing the streaming functionality")
])

​

Streaming from the Chat Endpoint

Python

import time

# Basic streaming example
response_stream = user.chat("What can you tell me about this user?", stream=True)

for chunk in response_stream.iter_text():
    print(chunk, end="", flush=True)  # Print each chunk as it arrives
    time.sleep(0.01)  # Optional delay for demonstration

​

Streaming Workspace Chat

Workspace chat streams the same way through chat_stream / chatStream on the client. There is no peer to anchor the question; the agent reads across every peer in the workspace.

Python

for chunk in honcho.chat_stream("What themes come up across all users this week?"):
    print(chunk, end="", flush=True)

Over REST, both endpoints return text/event-stream when stream is true. Each event is {"delta": {"content": "..."}, "done": false}, followed by a final {"done": true}.

​

Working with Streaming Data

When working with streaming responses, consider these patterns:

1. Progressive Rendering - Update your UI as chunks arrive instead of waiting for the full response
2. Buffered Processing - Accumulate chunks until a logical break (like a sentence or paragraph)
3. Token Counting - Monitor token usage in real-time for applications with token limits
4. Error Handling - Implement appropriate error handling for interrupted streams

​

Example: Restaurant Recommendation Chat

Python

import asyncio
from honcho import Honcho

async def restaurant_recommendation_chat():
    # Initialize client
    honcho = Honcho()

    # Create peers
    user = honcho.peer("food-lover")
    assistant = honcho.peer("restaurant-assistant")

    # Create session
    session = honcho.session("food-preferences-session")

    # Add peers to session
    await session.add_peers([user, assistant])

    # Store multiple user messages about food preferences
    user_messages = [
        "I absolutely love spicy Thai food, especially curries with coconut milk.",
        "Italian cuisine is another favorite - fresh pasta and wood-fired pizza are my weakness!",
        "I try to eat vegetarian most of the time, but occasionally enjoy seafood.",
        "I can't handle overly sweet desserts, but love something with dark chocolate."
    ]

    # Add the user's messages to the session
    session_messages = [user.message(message) for message in user_messages]
    await session.add_messages(session_messages)

    # Print the user messages
    for message in user_messages:
        print(f"User: {message}")

    # Ask for restaurant recommendations based on preferences
    print("\nRequesting restaurant recommendations...")
    print("Assistant: ", end="", flush=True)
    full_response = ""

    # Stream the response using the user's peer to get recommendations
    response_stream = user.chat(
        "Based on this user's food preferences, recommend 3 restaurants they might enjoy in the Lower East Side.",
        stream=True,
        session_id=session.id
    )

    for chunk in response_stream.iter_text():
        print(chunk, end="", flush=True)
        full_response += chunk
        await asyncio.sleep(0.01)

    # Store the assistant's complete response
    await session.add_messages([
        assistant.message(full_response)
    ])

# Run the async function
if __name__ == "__main__":
    asyncio.run(restaurant_recommendation_chat())

​

Performance Considerations

When implementing streaming:

• Consider connection stability for mobile or unreliable networks
• Implement appropriate timeouts for stream operations
• Be mindful of memory usage when accumulating large responses
• Use appropriate error handling for network interruptions

Streaming responses provide a more interactive and engaging user experience. By implementing streaming in your Honcho applications, you can create more responsive AI-powered features that feel natural and immediate to your users.