Blackwell
RTX 5090
Serves Qwen3.5-122B-A10B on ARK Cloud, four cards in two machines.
Hardware
Your GPUs on one side, open models on the other.
ARK sits in between. On the models side, every model family it supports and their fine-tunes; on this side, every NVIDIA GPU from Turing to Blackwell, from GeForce cards to the data centre. With ARK Core, any of these models runs on any mix of these GPUs.
- Supported GPUs
- 92NVIDIA cards by name: data centre, workstation and GeForce
- Generations
- 2018 to todayTuring, Ampere, Ada Lovelace, Hopper, Blackwell
- Benchmarked with every release
- 3 generationsAmpere, Ada Lovelace and Blackwell, on RTX 3060 to RTX 5090
- Mixing
- Any mixdifferent generations and sizes in one deployment
Benchmarked with every release
Benchmarks
Three GPU generations we benchmark with every release.
ARK Cloud serves its production traffic on these cards, and every ARK release is benchmarked on them. Every number on our benchmark pages was measured on them.
Ada Lovelace
RTX 4070 Ti SUPER and RTX 4060 Ti
Serve Qwen3.6-35B-A3B and Qwen3.5-4B on ARK Cloud, and the image encoder of Qwen3.5-122B-A10B.
Ampere
RTX 3060
Serves Bielik-11B-v3.0-Instruct on ARK Cloud, four cards in one machine.
Supported GPUs
Every NVIDIA GPU from Turing to Blackwell.
Every card below runs the same ARK release and the same converted models as the ones we benchmark. Which models fit, and for how many users, depends on the GPU memory you put behind them, not on the card class.
Since 2024
Blackwell
The newest generation: the B200 and B300 in HGX and DGX systems, the RTX PRO Blackwell cards and the GeForce RTX 50 series.
Workstation
Since 2022
Hopper
Data-centre GPUs with the highest memory bandwidth ARK runs on.
Since 2022
Ada Lovelace
From the low-power L4 to the L40S and RTX 6000 Ada, and the whole GeForce RTX 40 series.
Workstation
Since 2020
Ampere
The A100 and its data-centre siblings, the RTX A workstation cards and the GeForce RTX 30 series.
Data centre
Workstation
Since 2018
Turing
The oldest generation ARK runs on: the T4, the Quadro RTX and T-series cards, GeForce RTX 20 and GTX 16.
Data centre
Workstation
GeForce
What "supported" means
One build for every card.
A GPU is supported when ARK runs on it as shipped: the same release, the same converted models and the same configuration as on the cards we benchmark. ARK's GPU code is compiled for NVIDIA compute capabilities 7.5, 8.0, 8.6, 8.9, 9.0, 10.0, 10.3 and 12.0, which covers every card listed above.
- Benchmarked cards serve ARK Cloud, and every ARK release is benchmarked on them.
- Supported cards run the same software but are not in our benchmark rack. Customers run ARK on cards such as the L40S, A100 and RTX PRO 6000.
- Speed follows the card: reading prompts scales with its compute, generating answers with its memory bandwidth. Data-centre cards with more of both, such as the H100 and H200, go faster than the consumer cards we benchmark.
- Memory decides which models fit and how much attention a deployment has. Cards of any size can be combined: see patchwork clusters.
- Laptops with a listed GeForce or workstation GPU are supported the same way as the desktop card.
Not supported today
- Generations before Turing: Volta (V100, TITAN V), Pascal (P100, P40, GeForce GTX 10 series) and older. CUDA 13, which ARK is built on, no longer compiles for them.
- Arm-based systems: Grace Hopper (GH200), Grace Blackwell (GB200, GB300), Jetson and DGX Spark. ARK runs on Linux on x86-64.
- GPUs from other makers: AMD, Intel and Apple.
Need one of these? Talk to us.
Patchwork clusters
Every GPU you have, working as one.
Bring machines with whatever NVIDIA GPUs they carry. ARK joins them over ordinary Ethernet into one deployment and splits each model across them, so cards of different generations and sizes serve the same model together.
- One model, three machines, two GPU generations: Qwen3.5-122B-A10B runs in production on four RTX 5090 in two machines and an RTX 4060 Ti in a third.
- Mix generations and sizes, from an RTX 3060 to data-centre GPUs.
- Grow by adding a machine: its servers join the running deployment and their models appear in the API, with no restart.
How ARK uses GPUs
Memory adds up across cards and machines.
One model, many GPUs
A model that needs more memory than one card holds is split across several, in one machine or across machines. A 122-billion-parameter model runs on four 32 GB cards in two machines.
Mixed generations
Cards of different generations and sizes can serve side by side in one deployment. Add GPUs to add capacity, without replacing the ones you have.
Standard networking
Machines talk over ordinary Ethernet. No NVLink, no InfiniBand, no special switches.
Many users per GPU
Every request in flight shares each GPU step, so a card serves dozens of users at once. See decode batching.
Around the GPUs
What else a server needs.
| Driver | NVIDIA driver 580 or newer (CUDA 13). |
|---|---|
| Operating system | Linux on x86-64. |
| Runtime | Python 3.12 or newer. ARK is delivered as Python packages; services run under systemd. |
| Network | Standard Ethernet between machines. |
What it can run
Models for your GPUs
Every model family ARK supports, with its official models and fine-tunes, from 0.6B to 480B parameters. Which ones fit depends on the GPU memory you have.
Models What to expectPerformance on real GPUs
Measured throughput, time to first token and energy per token on RTX 3060 to RTX 5090 cards, with the method and data behind every number.
Benchmarks
Your hardware
Tell us what you run.
Send us your GPU inventory and the models you want to serve. We reply with a layout and what to expect from it.