Energy per million tokens
A GPU draws much of its power whether it serves one request or many. ARK shares every step across all the requests in flight, on any GPU, so the same energy produces more tokens as traffic grows. Every deployment we measured uses 1.8 to 4.7 times less energy per token with 16 or more requests at once than with one.
| Model | GPUs | 1 at a time | 16+ at once | Per request |
|---|---|---|---|---|
| Qwen3.6-35B-A3B | 4× RTX 4070 Ti SUPER | 100 Wh | 21 Wh | 0.025 Wh |
| Qwen3.5-4B | 2× RTX 4060 Ti | 39 Wh | 22 Wh | 0.028 Wh |
| Qwen3.5-122B-A10B | 4× RTX 5090 | 191 Wh | 76 Wh | 0.093 Wh |
| Bielik-11B-v3.0-Instruct | 4× RTX 3060 | 272 Wh | 90 Wh | 0.045 Wh |
Watt-hours per million tokens processed, and per request at 16 or more at once. A request here averages about 1,200 tokens (500 for Bielik), roughly ten prompt tokens for each answer token: the input-heavy mix of real traffic.
Generating answers
The figures above are dominated by reading prompts, as real traffic is. Generating answers costs more per token, because each token is its own step. In the ARK 2.0 release benchmark on four RTX 5090, 15 chat models generating for 64 users at once took between 70 and 340 Wh per million generated tokens, 100 for the median model: a tenth of what ARK 1.0 needed. See decode batching.
Measured on ARK 2.0
These runs date from 15 and 16 October 2026, on ARK 2.0. ARK 2.1 reads long prompts 1.6 to 1.8 times as fast on the same GPUs (see prefill batching), so we expect today's energy per token to be lower. We have not re-measured it yet; until we do, these are the numbers we stand behind.
How we measured
- Deployments. Our production deployments of four models, the ones that serve ARK Cloud: Qwen3.6-35B-A3B on four RTX 4070 Ti SUPER, Qwen3.5-4B on two RTX 4060 Ti, Qwen3.5-122B-A10B on four RTX 5090 in two machines, Bielik-11B-v3.0-Instruct on four RTX 3060.
- Traffic. Summarisation requests through the API, about ten prompt tokens per answer token, from 1 to 60 users who each send their next request as soon as the last one is answered. The prompt cache served 0 to 8% of the prompt tokens; those tokens are left out of the count, so every watt-hour is divided only by tokens the GPUs actually computed.
- Energy. The power of the GPUs and CPUs of the machines serving the model, read from their hardware sensors, multiplied by the time each load level ran. Idle draw is included, nothing is subtracted, and the time is rounded up to whole sensor sampling windows, which overstates energy slightly. Not included: memory, fans, power-supply losses, networking and cooling.
- Pooled. The 16-or-more figure adds up energy and tokens over every load level from 16 users upward, so a single sensor reading cannot move it much.
These figures were calculated from measurements on ARK Labs' own equipment. Results may vary depending on the hardware used.