Gantrix / guides / Ollama
Run AI agents locally with Ollama
Ollama runs open models on your own machine. Gantrix puts a team of agents on top of it: it plans a goal as steps, runs the ready ones in parallel, and has a judge check each result. This guide gets you from nothing to a finished task, with no cloud account.
What you need
| You need | Check with | Why |
|---|---|---|
| Node.js 26.13 or newer | node -v | Runs the Gantrix API, worker, dashboard and CLI |
| Docker, running | docker info | Postgres holds the state; Valkey holds the job queue |
| Ollama, running | ollama list | Serves the models. On a Mac run it natively, not in Docker, so it can use the GPU |
| About 5 GB of disk | The default model, qwen3:8b |
Five steps to a first task
- 1
Install Gantrix and pull a model
./init.shinstalls nvm and Node, installs dependencies, starts Docker infra and the server. The model download is most of the wait. Or by hand:git clone git@github.com:PerDiemInc/gantrix.git && cd gantrix ./init.sh # all-in-one; skip the next lines nvm use && npm install npm run setup npm run infra:up npm run db:migrate ollama pull qwen3:8b
- 2
Start the API, the worker and the dashboard
npm startruns all three, or use three terminals, one command each.npm start npm run dev:api npm run dev:worker npm run dev:web
- 3
Give agents a skill
A skill is a folder with a
SKILL.md. This one lets agents read and write files in the task's own folder.npm run gantrix -- skills add examples/skills/workspace-files npm run gantrix -- agents add researcher -d "Collects facts and writes notes" -s workspace-files --max-concurrency 2 npm run gantrix -- agents add editor -d "Turns notes into a polished summary" -s workspace-files
- 4
Run a goal
The planner splits it into steps. The two research steps run at the same time; the comparison waits for both.
npm run gantrix -- run "Write tea.md and coffee.md with 3 facts each, then compare.md with a table"
- 5
Watch it on the Live page
Open
http://127.0.0.1:4701, paste the token fromnpm run token, and choose Live.
On an M5 Pro with qwen3:8b, this exact goal finished in 1 minute 6 seconds and 8,406 tokens, with no replans.
See which models are loaded
Ollama loads a model into memory the first time an agent calls it, and Gantrix asks it to keep the model for ten minutes after the last call. On a laptop that memory matters, so the Live page shows it:
- Agents. Who is working, on which task, on which model, and for how long.
- Loaded models. Size in memory, GPU or CPU, context length, which agents use each one, and when it unloads.
- Local or cloud. How many agents send prompts off your machine. With Ollama only, that number is zero.
From a terminal, ollama ps shows the same list of loaded models.
Choosing models
- One default.
GANTRIX_DEFAULT_MODELsets the model for the planner, the judges and every agent without its own. - A model per agent. Give a small model to simple agents and a bigger one to the agent that writes code. Any model from
ollama listworks. - How many run at once.
GANTRIX_LLM_CONCURRENCYcaps model calls in flight; the default is 2. - Other local servers. vLLM, llama.cpp, LM Studio and LiteLLM connect as OpenAI-compatible servers.
- Cloud when you choose it. Add an OpenAI or Anthropic connection, or hand a project to Claude Code. Those are labelled cloud.
What stays on your machine
With the defaults: the prompts, the files agents write, the plan, the logs and the database. The API listens on 127.0.0.1 and needs your token on every request. Task content leaves only if you add a cloud connection or give an agent the web_fetch tool.