The ARK platform, on your own GPUs.
ARK Core is the software that runs ARK Cloud, licensed for your infrastructure. Serve open-weight models on NVIDIA GPUs, from a single workstation to a fleet of machines, behind the same OpenAI- and Anthropic-compatible API.
Big models on the GPUs you can get.
ARK splits a model across several GPUs and machines and serves many requests on each one. A model that needs more memory than one card holds runs across several smaller cards, over standard Ethernet.
Many GPUs, one model
Qwen3.5-122B-A10B runs on four RTX 5090 across two machines in production. Add GPUs to add capacity.
Mixed generations
RTX 3060, 4060 Ti, 4070 Ti SUPER and 5090 serve side by side in one deployment.
Many users per GPU
ARK 2.0 serves 64 concurrent streams on four RTX 5090 for every model we tested. See the numbers.
Everything between your GPUs and your applications.
API gateway
The OpenAI-compatible /api/v1 and Anthropic-compatible /api/anthropic/v1 APIs, with authentication through Keycloak or your own system via HTTP hooks, and per-request usage you can bill on.
Compute servers
They hold the model weights. Large models are split across GPUs and machines; models appear in the API as soon as their servers come online, with no restart.
Models, ready to serve
Any model in a supported family, fine-tunes included, converted for ARK and delivered with its upstream licence.
Observability
Prometheus metrics, structured logs with a correlation id that follows each request across servers, and prompt fields redacted by default.
Verification
A test suite that exercises the API on your deployment and writes standard JUnit results, so you know it works before your users do.
What you need to run it.
ARK Core runs on NVIDIA GPUs under Linux. Not sure your hardware fits? Send us your inventory and the models you want.
| GPUs | NVIDIA, from the Turing generation to Blackwell: data-centre, workstation and GeForce cards. See supported hardware. Customers run ARK on L40S, A100 and RTX PRO 6000; our own fleet and benchmarks use RTX 3060 12 GB, RTX 4060 Ti 16 GB, RTX 4070 Ti SUPER 16 GB and RTX 5090 32 GB. |
|---|---|
| Driver | NVIDIA driver 580 or newer (CUDA 13). |
| Operating system | Linux on x86-64. |
| Runtime | Python 3.12 or newer. ARK is delivered as Python packages; services run under systemd. |
| Network | Standard Ethernet between machines. No NVLink or InfiniBand needed. |
| Identity | Keycloak, or your own authentication and billing service through HTTP hooks. |
Run it yourself, or with us.
ARK Core
The platform and the models, with documentation and release packages. You bring identity, logging and monitoring, and run it with your own team.
- Per-GPU licence
- Support by fleet size
- Upgrade to Tailored at any time
ARK Tailored
Our engineers size, deploy and tune ARK on your infrastructure, integrate it with your identity and observability, and hand it over to your team.
- Capacity planned for your traffic
- Proof of concept on your hardware
- Direct line to the ARK team
ARK Cloud
Not ready to run GPUs? Use the same API and models on ARK Cloud today and move in-house later without changing your code.
Tell us what you run.
Send your GPU inventory and the models you need. We reply with a deployment plan and a quote.