Case study · 2025
Streaming conversation runtime
Owner, AI Platform @ Leena AI
The real-time backbone every conversation between an employee and an AI agent flows through, for 500+ enterprises: SSE and WebSocket streaming, server-driven UI components, reconnect message replay, and agent token auth, on Node.js/TypeScript microservices across multi-region AWS.
The problem
Agent replies stream token by token, and networks are not polite about it: sockets drop, laptops sleep, pods restart mid-stream.
A lost half-reply in an enterprise conversation is a support ticket. The runtime has to deliver every message across all of that, for 500+ enterprises at once.
One agent turn through the runtime
- ws: message receivedthe user hits send
- auth: agent tokenJWT / ACL checked before the model sees anything
- runtime: build contextconversation and prompt assembled
- llm: stream tokensthe reply streams as it is generated
- tool: executethe agent takes an action and continues
- sse: delivertext and server-driven UI components reach the client+ reconnecta dropped client gets the missed messages replayed
Key decisions
Replay on reconnect
When a socket drops or a pod restarts mid-stream, the client reconnects and the runtime replays what it missed, so the conversation continues instead of losing a half-streamed reply.
Stream UI, not just text
The runtime streams server-driven UI components alongside text, so agents can return structured interfaces the client renders.
Auth at the edge of the turn
Every agent token is verified (JWT / ACL) before context is built or the model is called, so permissions are decided once, up front.