Streaming

With "stream": true, ARK sends the answer token by token as server-sent events, so your users see text the moment it is generated.

API reference: Chat completions

Request a stream

stream = client.chat.completions.create(
    model="arklabsai/Qwen3.6-35B-A3B",
    messages=[{"role": "user", "content": "Explain the EU AI Act in three bullet points."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        delta = chunk.choices[0].delta
        print(delta.content or "", end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

const stream = await client.chat.completions.create({
  model: "arklabsai/Qwen3.6-35B-A3B",
  messages: [{ role: "user", content: "Explain the EU AI Act in three bullet points." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  if (chunk.choices.length) process.stdout.write(chunk.choices[0].delta.content ?? "");
  if (chunk.usage) console.log("\n", chunk.usage);
}

curl -N https://api.ark-labs.cloud/api/v1/chat/completions \
  -H "Authorization: Bearer $ARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "arklabsai/Qwen3.6-35B-A3B",
    "messages": [{"role": "user", "content": "Explain the EU AI Act in three bullet points."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

Event format

Each event is a data: line holding one JSON chunk, followed by a blank line. The stream ends with data: [DONE]. The first chunk arrives with the first generated token.

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1792345678,"model":"arklabsai/Qwen3.6-35B-A3B","service_tier":"default","system_fingerprint":null,"choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk", … "choices":[{"index":0,"delta":{"role":"assistant","content":" EU"},"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk", … "choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}],"usage":null}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk", … "choices":[],"usage":{"prompt_tokens":24,"completion_tokens":118,"total_tokens":142, …}}

data: [DONE]

• Text arrives in delta.content; with reasoning on, the thinking arrives first in delta.reasoning.
• The last content chunk carries the finish_reason; earlier chunks have null.
• Tool calls arrive as one chunk per call, with the complete arguments string, not in fragments.

Usage in a stream

With stream_options.include_usage, every chunk carries "usage": null and one extra chunk with empty choices and the full usage arrives just before [DONE]. That chunk also holds credits_used and credit_balance on ARK Cloud.

To show a live token counter, add "continuous_usage_stats": true next to include_usage. Each content chunk then carries running prompt_tokens, completion_tokens and total_tokens. These running counts are informational; the request is billed once, by the final usage.

{ "stream": true, "stream_options": { "include_usage": true, "continuous_usage_stats": true } }

Errors and disconnects

• An error before the first token (an unknown model, no capacity, a bad parameter) comes back as a normal JSON error with its HTTP status.
• An error after streaming has started cannot change the status any more. ARK sends one data: {"error": {…}} event and ends the stream without [DONE]. Treat a stream that ends without [DONE] as failed.