Limits and capacity

ARK Cloud has no per-key rate limits. What bounds a request is the model's context and the capacity free at that moment.

Context length

Each model in GET /api/v1/models reports max_tokens: its context length, the most tokens one request (or one stateful session) can hold, prompt and output together.

Read the live values from /api/v1/models rather than hard-coding them.

How max_tokens is checked

• max_tokens above the context length is rejected with 400 "max_tokens must be between 1 and N".
• ARK reserves room for the prompt plus max_tokens before generating. A request whose prompt plus max_tokens cannot be placed is answered with 429, not cut short. Ask for the output you need, not the maximum.
• Without max_tokens, the model's default applies (2,048 tokens on ARK Cloud models).

Capacity and 429

ARK serves many requests per model at once. When a model is at capacity, a new request waits briefly for room and is then refused with 429, rather than hanging.

• Retry 429 with exponential backoff; see Retrying.
• If 429 persists for one request while others succeed, the request is probably too large: reduce max_tokens or the prompt.
• For reserved capacity, a dedicated deployment or high sustained volumes, talk to us about ARK Core.

Timeouts

• Generation stops after 10 minutes. You receive the text produced so far with finish_reason: "inference_timeout" (Anthropic route: end_turn).
• Stateful sessions close after a period without requests (currently 90 seconds).
• Long prompts take time before the first token. Set your client timeout well above your expected generation time, and prefer streaming for long answers.

Other limits