DevelopersIntegrationsAnythingLLM

AnythingLLM with ARK

Use ARK for both halves of retrieval: a Qwen model for chat and bge-m3 for multilingual embeddings, through AnythingLLM's Generic OpenAI providers.

By Mintplex LabsMIT licenceSourceVendor docs

AnythingLLM: An all-in-one AI application: chat with your documents, build workspaces and agents, on your desktop or in Docker.

Before you start

You need an ARK API key: sign in to ARK Cloud and copy it from the dashboard, or use a key from your own ARK deployment and its base URL. The examples use arklabsai/Qwen3.6-35B-A3B; for the hardest tasks use arklabsai/Qwen3.5-122B-A10B. Any chat model from the catalogue works.

1. Install

Install
docker pull mintplexlabs/anythingllm

2. Connect it to ARK

AnythingLLM reads its settings from the .env file it mounts at /app/server/.env. Set the Generic OpenAI provider for chat and for embeddings:

.env
LLM_PROVIDER='generic-openai'
GENERIC_OPEN_AI_BASE_PATH='https://api.ark-labs.cloud/api/v1'
GENERIC_OPEN_AI_MODEL_PREF='arklabsai/Qwen3.6-35B-A3B'
GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT=262144
GENERIC_OPEN_AI_MAX_TOKENS=4096
GENERIC_OPEN_AI_API_KEY='your-ark-api-key'

EMBEDDING_ENGINE='generic-openai'
EMBEDDING_BASE_PATH='https://api.ark-labs.cloud/api/v1'
EMBEDDING_MODEL_PREF='arklabsai/bge-m3'
EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192
GENERIC_OPEN_AI_EMBEDDING_API_KEY='your-ark-api-key'

In the app: Settings → AI Providers → LLM, choose Generic OpenAI and fill in Base URL, API Key, the model id, a model context window of 262144 and Max Tokens. For embeddings: Settings → AI Providers → Embedder, Generic OpenAI, with arklabsai/bge-m3.

Good to know

  • Set the model context window explicitly; AnythingLLM otherwise assumes 4,096 tokens.
  • The embedding chunk length is counted in characters, so 8192 stays within bge-m3's 8,192-token input limit.
  • Embedders are system-wide: changing the embedder later means re-embedding your documents.
  • These tools send the whole conversation on every turn. ARK's prompt cache speeds up the repeated part of each prompt; every prompt token is still billed.

Next

Read about tool calling and limits and capacity on ARK, or browse other integrations.

Configuration keys follow AnythingLLM's official documentation as of October 2026. If the tool has changed since, its docs take precedence.