N8N in queue mode

1. What? — Definition and context

N8N’s queue mode is a distributed execution architecture in which workflows are delegated to separate processes (workers) through a Redis queue. It replaces the default mode where everything runs in a single process.

Components of queue mode

Component	Role	Instance
Main	Web UI, API, workflow triggering	1 (singleton)
Workers	Actual workflow execution	N (scalable)
Redis	Bull queue + Pub/Sub coordination	1 (shared)
PostgreSQL	Workflow storage, credentials, history	1 (shared)

Bull Queue

Bull is a Node.js library that uses Redis to manage job queues. Each triggered workflow becomes a “job” that workers pick up and execute asynchronously.

Architecture diagram

---

2. Why? — Stakes and motivations

Issue with the default mode

By default, N8N executes workflows in the same process as the web interface. Consequences:

Symptom	Cause	Impact
Slow UI	CPU monopolised by a workflow	Degraded UX
Webhook timeouts	A long workflow blocks the response	Broken integrations
No parallelism	Single execution thread	Bottleneck
Crash propagation	A memory error brings everything down	Service outage

Use cases that justify queue mode

This setup addresses four concrete needs:

1. Long API calls — Workflows using Claude or AI can take 5–30 seconds per call. Without a queue, the UI would freeze.

2. Parallelism — Several webhooks can land at the same time (GitHub commits, Telegram messages). Each worker processes a job independently.

3. Batch tasks — Data imports or large synchronisations (Odoo ↔ N8N) run in the background without impacting the UI.

4. Reliability — Failed jobs are retried automatically. A crashing worker does not affect the others.

When to enable queue mode?

Queue mode is recommended as soon as you have workflows lasting more than 10 seconds, several active webhooks, or a high-availability requirement.

---

3. How? — Technical implementation

Docker Compose configuration

services:
  n8n:
    image: n8n-custom:latest  # Image bundling custom nodes
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=n8n-redis
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
    depends_on:
      - n8n-postgres
      - n8n-redis

  n8n-worker:
    image: n8n-custom:latest
    command: worker
    environment:
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=n8n-redis
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
    deploy:
      replicas: 5

  n8n-redis:
    image: redis:7-alpine
    volumes:
      - n8n-redis-data:/data

  n8n-postgres:
    image: postgres:15
    environment:
      - POSTGRES_DB=n8n

N8N_ENCRYPTION_KEY is critical

The N8N_ENCRYPTION_KEY variable must be identical on the main process and on every worker. This key encrypts the credentials stored in the database. If it differs, workers cannot decrypt credentials and workflows fail silently.

Worker sizing

The number of workers (5 in this configuration) is driven by two constraints:

Constraint	Calculation	Result
Rule of thumb	1 worker per vCPU + 1 buffer	4 + 1 = 5 workers
Available RAM	~400 MB per worker	5 × 400 = 2 GB (n8n-stack budget)

Built-in Data Tables

N8N provides Data Tables to store structured data directly inside the instance. The current setup uses around twenty, organised by domain:

Domain	Main tables
Telegram	Telegram Authorized users, Telegram Banned Users, Telegram Pending Approvals
Docker	Docker_image_policies, Docker_pending_updates, Docker_pending_upd_approvals
Notifications	notification_routing, notification_history, pending_notifications, pending_deferred, escalation_tracking
Errors	error_dead_letter_queue, error_handling_config
Telemetry	claude_code_active_sessions, claude_code_sessions_v2
External sync	Odoo_github_project_mapping, timetrackr_user_mapping, content_pipeline
Workflows	n8n_trigger_whitelist, n8n_pending_actions, n8n_datatable_metadata

Data Tables vs database

Data Tables are handy for simple configuration or temporary data. For larger volumes or complex queries, prefer PostgreSQL or Qdrant.

AI Stack integration

N8N talks to the AI Stack over internal HTTP:

// HTTP Request to CLI Ollama (Codex by default)
{
  "url": "http://cli-ollama:11434/api/generate",
  "method": "POST",
  "body": {
    "model": "codex-yolo",
    "prompt": "Analyse this message...",
    "stream": false
  }
}

`-yolo` model required from N8N

N8N workflows that have no human in the loop must use a -yolo model (e.g. codex-yolo, gemini-flash-yolo). A non-yolo model switches to interactive plan mode and the webhook times out after 120 s.

AI use cases:

• Intent detection — The Telegram Orchestrator uses CLI Ollama to route messages
• MCP via gateway — Admin workflows through the whitelisted MCP tools (n8n_get_workflow, n8n_executions, etc.)
• RAG — Semantic search in Qdrant to enrich answers
• Error analysis — error-analyst profile consumed by the GEH AI Auto-Fix workflow

Operational commands

# Check active workers
docker compose -f n8n-stack/docker-compose.yaml ps

# Logs of a specific worker
docker logs n8n-stack-n8n-worker-3 --tail 100

# Scale workers dynamically
docker compose -f n8n-stack/docker-compose.yaml up -d --scale n8n-worker=8

# Inspect the Redis queue
docker exec n8n-redis redis-cli LLEN bull:jobs:waiting

---

4. What if? — Perspectives and limits

Current limits

Limit	Impact	Mitigation
Single Redis instance	SPOF for the queue	Regular snapshots, fast rebuild
No priorities	Jobs are FIFO	Critical workflows are not prioritised
Manual scaling	No autoscaling	Monitoring + periodic adjustments

Evolution scenarios

If load grows significantly:

• Add workers up to the RAM ceiling
• Move to a beefier VPS (vertical scaling)
• Split Redis/PostgreSQL onto their own instances

If a single workflow hogs resources:

• Create a dedicated queue for long workflows
• Configure per-workflow concurrency limits
• Use sub-workflows to break processing into stages

If reliability becomes critical:

• Deploy Redis in Sentinel mode (HA)
• Add a second N8N main instance in standby
• Externalise to Redis Cloud or AWS ElastiCache

Metrics to watch

To anticipate problems, monitor through the Monitoring Stack:

Metric	Alert threshold	Action
bull:jobs:waiting	> 50 jobs	Add workers
Worker CPU	> 80% sustained	Optimise workflows or scale
Failed executions	> 5%	Investigate the errors

---

Related pages

Infrastructure

• VPS Architecture — Overview
• AI Stack — Claude Ollama + Qdrant
• Monitoring Stack — N8N metrics

Workflows

• Telegram Orchestrator — Central control bot
• Docker auto-updates — Automated upgrades
• Notification Hub — Centralised alerting

Reference

• Glossary — Queue, worker, Bull