feat(streaming): add streaming support for LLM responses (fixes #211)
Implement Server-Sent Events (SSE) streaming for OpenAI-compatible providers:
- Add StreamChunk, StreamOptions, and StreamError types to traits module
- Add supports_streaming() and stream_chat_with_system() to Provider trait
- Implement SSE parser for OpenAI streaming responses (data: {...} format)
- Add streaming support to OpenAiCompatibleProvider
- Add streaming support to ReliableProvider with error propagation
- Add futures dependency for async stream support
Features:
- Token-by-token streaming for real-time feedback
- Token counting option (estimated ~4 chars per token)
- Graceful error handling and logging
- Channel-based stream bridging for async compatibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
ccc48824cf
commit
d94e78c621
3 changed files with 325 additions and 3 deletions
|
|
@ -3,7 +3,7 @@ name = "zeroclaw"
|
|||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
authors = ["theonlyhennygod"]
|
||||
license = "MIT"
|
||||
license = "Apache-2.0"
|
||||
description = "Zero overhead. Zero compromise. 100% Rust. The fastest, smallest AI assistant."
|
||||
repository = "https://github.com/zeroclaw-labs/zeroclaw"
|
||||
readme = "README.md"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue