Stream Chat Responses in Real Time with Meliai API

Streaming lets your application display model output token-by-token as it is generated rather than waiting for the complete response. This dramatically improves perceived latency and gives users immediate feedback — particularly valuable for conversational interfaces, long-form generation, and anything where time-to-first-token matters. Meliai’s streaming implementation follows the OpenAI server-sent events (SSE) protocol, so any client already using OpenAI streaming works without modification.

Enabling Streaming

Set stream: true in your request body to switch from a single JSON response to a stream of SSE events. The OpenAI SDK handles the underlying SSE protocol for you, exposing each chunk as an iterable object.

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["MELIAI_API_KEY"],
    base_url="https://api.meliai.ai/v1",
)

stream = client.chat.completions.create(
    model="<MODEL_ID>",
    messages=[{"role": "user", "content": "Write a short poem about the Baltic Sea."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.MELIAI_API_KEY,
  baseURL: "https://api.meliai.ai/v1",
});

const stream = await client.chat.completions.create({
  model: "<MODEL_ID>",
  messages: [{ role: "user", content: "Write a short poem about the Baltic Sea." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

curl https://api.meliai.ai/v1/chat/completions \
  -H "Authorization: Bearer $MELIAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<MODEL_ID>",
    "messages": [{"role": "user", "content": "Write a short poem about the Baltic Sea."}],
    "stream": true
  }'

SSE Format

When streaming is enabled, Meliai sends the response as a sequence of server-sent events over the HTTP connection. Each event is a line prefixed with data: followed by a JSON object. The stream is terminated by the special sentinel data: [DONE]. A single chunk looks like this:

data: {
  "id": "chatcmpl-abc123",
  "object": "chat.completion.chunk",
  "created": 1718000000,
  "model": "<MODEL_ID>",
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": "The Baltic"
      },
      "finish_reason": null
    }
  ]
}

The final chunk before data: [DONE] will have finish_reason set to "stop" (or another stop reason) and an empty delta.content. If you are parsing the raw SSE stream yourself, skip any line that does not start with data: and stop processing when you encounter data: [DONE].

Getting Usage in Streaming Mode

By default, token usage is not included in streaming responses. To receive a final chunk containing prompt and completion token counts, set stream_options.include_usage to true.

Python

stream = client.chat.completions.create(
    model="<MODEL_ID>",
    messages=[{"role": "user", "content": "Write a short poem about the Baltic Sea."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)
    # The last chunk has no choices but carries usage stats
    if chunk.usage:
        print(f"\n\nPrompt tokens: {chunk.usage.prompt_tokens}")
        print(f"Completion tokens: {chunk.usage.completion_tokens}")

The usage chunk arrives after the finish_reason chunk and before data: [DONE]. Its choices array is empty, and it carries a populated usage field identical in structure to a non-streaming response.

Add the :speed routing suffix to your model ID — for example mistral-small:speed — to bias routing toward the lowest-latency European provider. This reduces time-to-first-token for interactive streaming applications without changing your code.

Overview

Models & Routing

Features

Privacy

Stream Chat Responses in Real Time with Meliai API

Enabling Streaming

SSE Format

Getting Usage in Streaming Mode

​Enabling Streaming

​SSE Format

​Getting Usage in Streaming Mode

Enabling Streaming

SSE Format

Getting Usage in Streaming Mode