Skip to main content

Streaming with Tool Calls

Combining streaming with tool calling requires special handling. Tool calls arrive as fragments that must be accumulated.

How Tool Calls Stream

Tool calls arrive in delta fragments, not as complete objects:

Key Learnings ⚠️

1. id May Be Null

The tool_call.id might come as null in some responses. Always check:
Fallback ID: If id is still empty after streaming completes, generate one:

2. Arguments Are Fragmented

Arguments stream character-by-character. Accumulate them:

3. Index Identifies Tool Calls

For parallel tool calls, index identifies which tool call to update:

Complete Streaming Tool Handler

Multi-Turn Streaming Loop

See Also