Own the runtime. Own the cost curve.
Platform teams building an internal AI service want predictable cost and full control. ARK Core serves open-weight models on your NVIDIA GPUs under a per-GPU licence. More requests use more of your capacity, never a bigger per-token invoice.
Same API and models on ARK Cloud and on your own GPUs. Prototype today, move in-house without a rewrite.
Why internal AI platforms get expensive.
Per-token prices keep falling, but bills don't. Gartner predicts the inference cost of each agentic workflow will rise more than fivefold through 2028.
Gartner, 17 August 2026. Agentic models need 5–30× more tokens per task than a standard chatbot (Gartner, March 2026).
Cheaper tokens, bigger bills
Every new internal use case adds to a per-token invoice, and agents multiply the tokens behind each task. Uber reportedly used up its entire 2026 AI coding-tools budget in about four months.
Fortune, 26 May 2026, citing reports.
The price list changes under you
Usage-based billing, discounts that end at a contracted cap, prices that jump past a prompt length. Your budget follows someone else's rate card.
Your engine slows as teams arrive
A serving engine tuned for one user at a time levels off as concurrent users grow. Each new team makes everyone slower.
Big models, real hardware
The models you want need more memory than one GPU holds.
Built for many users sharing the GPUs.
We ran ARK 2.1 and vLLM 0.27.1 on the same two RTX 5090 on the same day, with the same model at the same 4-bit precision, the same prompts, the same third-party benchmark script and both engines at default settings. vLLM is faster for a single user. With 4,000-token prompts ARK serves more from 8 users upward.
| Users | ARK 2.1, total tok/s | vLLM, total tok/s | ARK first token | vLLM first token |
|---|---|---|---|---|
| 1 | 88 | 160 | 0.33 s | 1.58 s |
| 16 | 764 | 511 | 1.95 s | 12.88 s |
| 64 | 1,655 | 652 | 6.82 s | 47.92 s |
Qwen3.6-35B-A3B, 4,000-token prompts, two repeats; zero failed requests on either engine. Full method and caveats.
An engine built for shared, internal traffic.
One deployment serves many teams and many models, with throughput that grows as more people use it.
Flat cost per GPU
A per-GPU licence with no per-token fee. Your cost is set by the GPUs you run, not by a provider's next price change.
Throughput that grows with users
ARK 2.0 batches every request in flight, so total output rises as teams pile on instead of being shared out between them.
Long prompts, read fast
ARK 2.1 reads prompts across all the GPUs of a split model at once. A 64,000-token prompt gets its first token in 4.7 seconds instead of 8.4, and bursts of long prompts are read 1.6 to 1.8× faster.
Agents that don't re-read everything
Repeated prompts come back from cache automatically, 9× sooner for a repeated 64,000-token document. Stateful sessions keep a conversation on the server, so each turn sends only the new message.
Big models, elastic fleet
Split a model over GPUs of different generations and sizes across machines on standard Ethernet; Qwen3.5-122B-A10B runs in production on four RTX 5090 plus an RTX 4060 Ti. Servers join or leave without a restart, and with a second copy of each stretch a failed server's requests are replayed on another path.
Operable from day one
You get Prometheus metrics, structured logs with a correlation id per request, prompt fields redacted by default, and a test suite that writes JUnit results on your deployment. Per-request token usage, with cached and reasoning tokens included, lets FinOps see cost per team and per task.
“By 2028, 60% of Global 500 companies will embed AI FinOps control at inference.” — Gartner Top Strategic Predictions, 15 September 2026
What you need to run it.
Any NVIDIA GPU from Turing to Blackwell, whether GeForce, workstation or data centre (from the RTX 3060 to the H100, H200 and L40S), on Linux x86-64 with standard Ethernet between machines. Different generations and sizes can serve the same model. Arm-based systems such as GB200 are not supported. Full requirements are on the ARK Core and Hardware pages.
Support by fleet size: see the levels on the pricing page.
Start where you are.
ARK Core Recommended
The platform and its models, delivered as Python packages with documentation and release packages, licensed per GPU. Your team runs it under systemd with your own identity, logging and monitoring. Our engineers can help with sizing and rollout.
ARK CorePrototype todayARK Cloud
Same API and models on ARK Cloud. Build against it now and switch the base URL when your GPUs are ready.
ARK CloudBenchmark it on your own traffic.
Published benchmarks are a starting point. Your prompts, models and concurrency decide the real numbers. Send us your GPU inventory and the models your teams need, and we reply with a deployment plan and the throughput to expect. Or try the API on ARK Cloud first, with free starter credits.