# Streaming AI responses over SSE — a practical guide

> Server-sent events end to end — parsing chunk frames correctly, relaying streams through your own backend, and the buffering bugs that only appear in production.

Published 2026-08-12 · eroq.ai — canonical: https://eroq.ai/blog/streaming-sse-practical-guide


Streaming is the difference between a character who is *present* and a spinner. The mechanics are simple; the bugs are all in the details this guide covers.

## The wire format

With `stream: true`, `/v1/chat/completions` answers `text/event-stream`. Each frame is a `data:` line and a blank line; deltas arrive in the standard chunk shape:

```
data: {"id":"cmpl_…","object":"chat.completion.chunk","choices":[{"delta":{"content":"That "}}]}

data: {"id":"cmpl_…","object":"chat.completion.chunk","choices":[{"delta":{"content":"noise"}}]}

data: {"object":"chat.completion.usage","usage":{"credits_spent":3,"credits_remaining":997}}

data: [DONE]
```

Two eroq-specific frames worth knowing: the **usage event** just before `[DONE]` carries the meter, and an **error event** (`{"error":{…}}`) replaces the crash you would otherwise have to infer from a dropped connection. If the stream errors before any content arrived, the call has already refunded itself.

## Parsing without the classic bug

The classic bug: treating every network chunk as a complete frame. TCP does not respect your line breaks — a frame can arrive split across reads. Buffer, split on `\n\n`, and keep the remainder:

```js
const reader = res.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
while (true) {
  const { done, value } = await reader.read()
  if (done) break
  buffer += decoder.decode(value, { stream: true })   // stream: true matters for UTF-8
  const frames = buffer.split('\n\n')
  buffer = frames.pop() ?? ''                          // last piece may be incomplete
  for (const frame of frames) {
    const data = frame.replace(/^data: /, '')
    if (data === '[DONE]') continue
    const parsed = JSON.parse(data)
    if (parsed.error) throw new Error(parsed.error.message)
    const delta = parsed.choices?.[0]?.delta?.content
    if (delta) render(delta)
  }
}
```

The `{ stream: true }` on `decoder.decode` is not decoration: without it, a multi-byte character split across chunks becomes mojibake. Emoji-heavy roleplay finds this bug within the hour.

## Relaying through your backend

Never ship your API key to a browser — relay the stream. The relay is thin, but two details make or break it:

```js
// Node/Express-style relay
app.post('/chat', async (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream')
  res.setHeader('Cache-Control', 'no-cache')
  res.flushHeaders()                                   // 1. headers out immediately

  const upstream = await fetch('https://eroq.ai/v1/chat/completions', { /* … */ })
  for await (const chunk of upstream.body) {
    res.write(chunk)                                   // 2. relay bytes, re-frame nothing
  }
  res.end()
})
```

1. **Flush headers immediately**, or your reverse proxy may buffer the whole response and deliver it at once — streaming that arrives as a block. On nginx, also set `X-Accel-Buffering: no`.
2. **Relay bytes verbatim.** Parsing and re-serializing frames in the relay doubles your bug surface for zero value. Parse on the client, where you render.

## UX details that separate good from great

- **Render on a small timer** (30–50ms), not per-delta — per-token DOM writes jank on mobile.
- **Show the first token fast, then let it flow.** Perceived latency lives almost entirely in time-to-first-token.
- **On mid-stream failure, keep the partial text** and offer a retry affordance. A half-reply that stays beats a reply that vanishes — and since interrupted streams past first output were still generated, keeping the text respects what was paid for.
- **Let users abort.** Closing your relay's response should close the upstream request; an abandoned stream you keep consuming is money spent rendering to nobody.

Flat pricing has one more consequence here: streaming costs exactly what buffering costs — the [same 1 or 3 credits](/pricing). There is no reason not to stream, which is why every example in [the docs](/docs/streaming) does.