Working with Session Context

Learn how to use get_context() to retrieve and format conversation context for LLM integration

The get_context() method is a powerful feature that retrieves formatted conversation context from sessions, making it easy to integrate with LLMs like OpenAI, Anthropic, and others. This guide covers everything you need to know about working with session context.

By default, the context includes a blend of summary and messages which covers the entire history of the session. Summaries are automatically generated at intervals and recent messages are included depending on how many tokens the context is intended to be. You can specify any token limit you want, and can disable summaries to fill that limit entirely with recent messages.

Basic Usage

The get_context() method is available on all Session objects and returns a SessionContext that contains the formatted conversation history.

Python

from honcho import Honcho

# Initialize client and create session
honcho = Honcho()
session = honcho.session("conversation-1")

# Get basic context (not very useful before adding any messages!)
context = session.get_context()

Context Parameters

The get_context() method accepts several optional parameters to customize the retrieved context:

Token Limits

Control the size of the context by setting a maximum token count:

Python

# Limit context to 1500 tokens
context = session.get_context(tokens=1500)

# Limit context to 3000 tokens for larger conversations
context = session.get_context(tokens=3000)

Summary Mode

Enable summary mode (on by default) to get a condensed version of the conversation:

Python

# Get context with summary enabled -- will contain both summary and messages
context = session.get_context(summary=True)

# Combine summary=False with token limits to get more messages
context = session.get_context(summary=False, tokens=2000)

Converting to LLM Formats

The SessionContext object provides methods to convert the context into formats compatible with popular LLM APIs. When converting to OpenAI format, you must specify the assistant peer to format the context in such a way that the LLM can understand it.

OpenAI Format

Convert context to OpenAI’s chat completion format:

Python

# Create peers
alice = honcho.peer("alice")
assistant = honcho.peer("assistant")

# Add some conversation
session.add_messages([
    alice.message("What's the weather like today?"),
    assistant.message("It's sunny and 75°F outside!")
])

# Get context and convert to OpenAI format
context = session.get_context()
openai_messages = context.to_openai(assistant=assistant)

# The messages are now ready for OpenAI API
print(openai_messages)
# [
#   {"role": "user", "content": "What's the weather like today?"},
#   {"role": "assistant", "content": "It's sunny and 75°F outside!"}
# ]

Anthropic Format

Convert context to Anthropic’s Claude format:

Python

# Get context and convert to Anthropic format
context = session.get_context()
anthropic_messages = context.to_anthropic(assistant=assistant)

# Ready for Anthropic API
print(anthropic_messages)

Complete LLM Integration Examples

Using with OpenAI

Python

import openai
from honcho import Honcho

# Initialize clients
honcho = Honcho()
openai_client = openai.OpenAI()

# Set up conversation
session = honcho.session("support-chat")
user = honcho.peer("user-123")
assistant = honcho.peer("support-bot")

# Add conversation history
session.add_messages([
    user.message("I'm having trouble with my account login"),
    assistant.message("I can help you with that. What error message are you seeing?"),
    user.message("It says 'Invalid credentials' but I'm sure my password is correct")
])

# Get context for LLM
messages = session.get_context(tokens=2000).to_openai(assistant=assistant)

# Add new user message and get AI response
messages.append({
    "role": "user",
    "content": "Can you reset my password?"
})

response = openai_client.chat.completions.create(
    model="gpt-4",
    messages=messages
)

# Add AI response back to session
session.add_messages([
    user.message("Can you reset my password?"),
    assistant.message(response.choices[0].message.content)
])

Multi-Turn Conversation Loop

Python

def chat_loop():
    """Example of a continuous chat loop using get_context()"""

    session = honcho.session("chat-session")
    user = honcho.peer("user")
    assistant = honcho.peer("ai-assistant")

    while True:
        # Get user input
        user_input = input("You: ")
        if user_input.lower() in ['quit', 'exit']:
            break

        # Add user message to session
        session.add_messages([user.message(user_input)])

        # Get conversation context
        context = session.get_context(tokens=2000)
        messages = context.to_openai(assistant=assistant)

        # Get AI response
        response = openai_client.chat.completions.create(
            model="gpt-4",
            messages=messages
        )

        ai_response = response.choices[0].message.content
        print(f"Assistant: {ai_response}")

        # Add AI response to session
        session.add_messages([assistant.message(ai_response)])

# Start the chat loop
chat_loop()

Advanced Context Usage

Context with Summaries for Long Conversations

For very long conversations, use summaries to maintain context while controlling token usage:

Python

# For long conversations, use summary mode
long_session = honcho.session("long-conversation")

# Get summarized context to fit within token limits
context = long_session.get_context(summary=True, tokens=1500)
messages = context.to_openai(assistant=assistant)

# This will include a summary of older messages and recent full messages
print(f"Context contains {len(messages)} formatted messages")

Context for Different Assistant Types

You can get context formatted for different types of assistants in the same session:

Python

# Create different assistant peers
chatbot = honcho.peer("chatbot")
analyzer = honcho.peer("data-analyzer")
moderator = honcho.peer("moderator")

# Get context formatted for each assistant type
chatbot_context = session.get_context().to_openai(assistant=chatbot)
analyzer_context = session.get_context().to_openai(assistant=analyzer)
moderator_context = session.get_context().to_openai(assistant=moderator)

# Each context will format the conversation from that assistant's perspective

Best Practices

1. Token Management

Always set appropriate token limits to control costs and ensure context fits within LLM limits:

Python

# Good: Set reasonable token limits based on your model
context = session.get_context(tokens=3000)  # For GPT-4
context = session.get_context(tokens=1500)  # For smaller models

# Good: Use summaries for very long conversations
context = session.get_context(summary=True, tokens=2000)

2. Context Caching

For applications with frequent context retrieval, consider caching context when appropriate:

Python

# Cache context for multiple LLM calls within the same request
context = session.get_context(tokens=2000)
openai_messages = context.to_openai(assistant=assistant)
anthropic_messages = context.to_anthropic(assistant=assistant)

# Use the same context object for multiple format conversions

3. Error Handling

Always handle potential errors when working with context:

Python

try:
    context = session.get_context(tokens=2000)
    messages = context.to_openai(assistant=assistant)

    # Use messages with LLM API
    response = openai_client.chat.completions.create(
        model="gpt-4",
        messages=messages
    )

except Exception as e:
    print(f"Error getting context: {e}")
    # Handle error appropriately

Conclusion

The get_context() method is essential for integrating Honcho sessions with LLMs. By understanding how to:

• Retrieve context with appropriate parameters
• Convert context to LLM-specific formats
• Manage token limits and summaries
• Handle multi-turn conversations

You can build sophisticated AI applications that maintain conversation history and context across interactions while integrating seamlessly with popular LLM providers.