Streaming API Responses
Real-time token delivery via Server-Sent Events
π Key Concepts
- SSE vs WebSocket β SSE is simpler (HTTP), one-directional (serverβclient), auto-reconnects. Perfect for LLM streaming.
- Implementation β StreamingResponse with media_type='text/event-stream'. Yield 'data: {token}\n\n' for each chunk.
- Client side β EventSource API in JavaScript. Onmessage callback fires for each token. Easy to implement.
- Error handling β Send 'data: [ERROR] message\n\n' on failure. Client reconnects automatically with SSE.
π‘ Practice: Try implementing each concept yourself before moving on. Reading about RAG and building RAG are very different things.