Research
Measurements and studies behind ARK, each with its method and data. The newest first.
What made ARK fast, measured.
Prompt caching
Repeated prompts are restored, not read again: a repeated 64,000-token document gets its first token 9× sooner on ARK Cloud.
Prefill batching
Long prompts read across every GPU of a split model at once: a 64,000-token prompt gets its first token in 4.7 s instead of 8.4.
Paged attention
Short and long requests now share every decoding step: 2.6× the speed per stream when request lengths are mixed.
Quantized attention
An 8-bit attention cache: twice the context in the same GPU memory, and up to 30% faster decoding steps on long contexts.
Decode batching
Every request in flight shares one decoding step: 9.3× the throughput of ARK 1.0 at full load, at a tenth of the energy per token.
Fused kernels
4-bit models decoded through kernels built for batches: 2.6× the speed with 16 concurrent requests.
Inverted processing loops
The same models, engine and GPUs as ARK 1.1, and one stream decodes 1.51× as fast.
Mixing GPU generations
One model across GPUs of different generations, sizes and counts: how ARK splits it, routes by speed and load, and what to watch.
ARK and vLLM on the same GPUs
Same model, precision, GPUs, prompts and benchmark, both at defaults: vLLM wins for one user; at 64 users ARK delivers 2.5× the throughput and a 7× faster first token.
Energy per token on ARK
21 Wh per million tokens on Qwen3.6-35B-A3B, 76 on Qwen3.5-122B-A10B under real-traffic load, measured on our own deployments.
ARK Core under real traffic
Throughput, time to first token and reliability for four production models from one to sixty concurrent requests, measured through the public API.
Long answers at scale
Qwen3.5-122B-A10B writing 2,000-token answers to up to 64 users at once, and what the API layer costs.
ARK 2.0 across 19 models
The release benchmark: single-stream speed, 64-stream throughput and energy per token on four RTX 5090.