Gantrix

Gantrix / guides / Ollama

Run AI agents locally with Ollama

Ollama runs open models on your own machine. Gantrix puts a team of agents on top of it: it plans a goal as steps, runs the ready ones in parallel, and has a judge check each result. This guide gets you from nothing to a finished task, with no cloud account.

What you need

You needCheck withWhy
Node.js 26.13 or newernode -vRuns the Gantrix API, worker, dashboard and CLI
Docker, runningdocker infoPostgres holds the state; Valkey holds the job queue
Ollama, runningollama listServes the models. On a Mac run it natively, not in Docker, so it can use the GPU
About 5 GB of diskThe default model, qwen3:8b

Five steps to a first task

  1. 1

    Install Gantrix and pull a model

    ./init.sh installs nvm and Node, installs dependencies, starts Docker infra and the server. The model download is most of the wait. Or by hand:

    git clone git@github.com:PerDiemInc/gantrix.git && cd gantrix
    ./init.sh   # all-in-one; skip the next lines
    nvm use && npm install
    npm run setup
    npm run infra:up
    npm run db:migrate
    ollama pull qwen3:8b
  2. 2

    Start the API, the worker and the dashboard

    npm start runs all three, or use three terminals, one command each.

    npm start
    npm run dev:api
    npm run dev:worker
    npm run dev:web
  3. 3

    Give agents a skill

    A skill is a folder with a SKILL.md. This one lets agents read and write files in the task's own folder.

    npm run gantrix -- skills add examples/skills/workspace-files
    npm run gantrix -- agents add researcher -d "Collects facts and writes notes" -s workspace-files --max-concurrency 2
    npm run gantrix -- agents add editor -d "Turns notes into a polished summary" -s workspace-files
  4. 4

    Run a goal

    The planner splits it into steps. The two research steps run at the same time; the comparison waits for both.

    npm run gantrix -- run "Write tea.md and coffee.md with 3 facts each, then compare.md with a table"
  5. 5

    Watch it on the Live page

    Open http://127.0.0.1:4701, paste the token from npm run token, and choose Live.

On an M5 Pro with qwen3:8b, this exact goal finished in 1 minute 6 seconds and 8,406 tokens, with no replans.

See which models are loaded

Ollama loads a model into memory the first time an agent calls it, and Gantrix asks it to keep the model for ten minutes after the last call. On a laptop that memory matters, so the Live page shows it:

  • Agents. Who is working, on which task, on which model, and for how long.
  • Loaded models. Size in memory, GPU or CPU, context length, which agents use each one, and when it unloads.
  • Local or cloud. How many agents send prompts off your machine. With Ollama only, that number is zero.

From a terminal, ollama ps shows the same list of loaded models.

Choosing models

  • One default. GANTRIX_DEFAULT_MODEL sets the model for the planner, the judges and every agent without its own.
  • A model per agent. Give a small model to simple agents and a bigger one to the agent that writes code. Any model from ollama list works.
  • How many run at once. GANTRIX_LLM_CONCURRENCY caps model calls in flight; the default is 2.
  • Other local servers. vLLM, llama.cpp, LM Studio and LiteLLM connect as OpenAI-compatible servers.
  • Cloud when you choose it. Add an OpenAI or Anthropic connection, or hand a project to Claude Code. Those are labelled cloud.

What stays on your machine

With the defaults: the prompts, the files agents write, the plan, the logs and the database. The API listens on 127.0.0.1 and needs your token on every request. Task content leaves only if you add a cloud connection or give an agent the web_fetch tool.