Files
classroomio/docker/docs/SELF_HOST.md
T
Abdulmuizz HamzatandRotimi Best 6d594b4296 feat: harden self-hosting + add pull-based deploy, Coolify/Railway (#666)
* feat: enhance self-hosting support and CI/CD workflows

- Updated .env.example to clarify auth secret generation and self-hosting mode.
- Added CI workflow for build and lint checks on PRs and main branch pushes.
- Improved docker-publish workflow to trigger on relevant file changes and added multi-arch support.
- Enhanced upload-openapi-spec workflow with caching for dependencies.
- Removed obsolete Dockerfile for API service and added jobs worker to Docker setup.
- Updated self-host documentation to reflect new jobs worker requirement.
- Modified Dockerfiles for API and Dashboard to optimize dependency fetching and installation.
- Enhanced docker-compose.yaml with health checks and jobs worker configuration.
- Updated docker-push.sh to support multi-architecture builds and improved logging.
- Revised documentation for publishing images and self-hosting to include new features and requirements.

* feat(self-host): safe versioned releases + pull-based image pipeline

Builds on the pull-based images compose (docker-compose.images.yaml) to make
pulling pre-built images safe for self-hosters:

- docker-publish: stable-releases + edge tag policy. Git tag vX.Y.Z publishes
  X.Y.Z/X.Y/X/latest; a push to main publishes only :edge (latest never moves
  on a routine merge).
- Pre-publish smoke-test gate: each image is booted against ephemeral
  postgres+redis and healthchecked before any push (docker/smoke-test.sh).
- CIO_VERSION pinning in .env.example; run-docker-full-stack.sh --images flag
  to pull instead of build.
- Docs: SELF_HOST pull-from-images + versioning/upgrade/rollback sections,
  PUBLISHING_IMAGES tag policy + pnpm release flow, self-hosting.mdx quickstart.

* feat: enhance self-hosting support and CI/CD workflows

- Updated .env.example to clarify auth secret generation and self-hosting mode.
- Added CI workflow for build and lint checks on PRs and main branch pushes.
- Improved docker-publish workflow to trigger on relevant file changes and added multi-arch support.
- Enhanced upload-openapi-spec workflow with caching for dependencies.
- Removed obsolete Dockerfile for API service and added jobs worker to Docker setup.
- Updated self-host documentation to reflect new jobs worker requirement.
- Modified Dockerfiles for API and Dashboard to optimize dependency fetching and installation.
- Enhanced docker-compose.yaml with health checks and jobs worker configuration.
- Updated docker-push.sh to support multi-architecture builds and improved logging.
- Revised documentation for publishing images and self-hosting to include new features and requirements.

* ci: run build-only on PRs, defer lint until repo-wide format

* docs: point Railway template link at the new template URL

* fix: address Greptile review + dashboard CI build env

- docker-push.sh: only also-tag :latest for real semver releases, so
  VERSION=edge/main/dev can't clobber the stable latest pointer (P1).
- run-docker-full-stack.sh: stop deriving OBJECT_STORAGE_PUBLIC_ENDPOINT from
  the dashboard origin (broke presigned URLs for videos/documents buckets);
  derive only the public media base URL and document the S3-API route.
- ci.yml: provide PUBLIC_IS_SELFHOSTED at build time (dashboard reads it from
  SvelteKit's strict $env/static/public); bump cache actions v3 -> v4.
- upload-openapi-spec.yml: bump cache actions v3 -> v4.
- smoke-test.sh: dashboard gate now fails on 5xx instead of accepting any response.

* ci: scope PR build gate to the api + dashboard closures

Building every package surfaced pre-existing rot in apps that aren't part of
this PR (storybook's missing @lucide/svelte import, course-app prettier debt).
The gate's purpose is to verify the two Docker-image apps; the rest are built on
PRs by their own Cloudflare Workers builds. Filter to @cio/api... and
@cio/dashboard... so the gate covers exactly what it should.

* fix(docker-push): make latest_arg always return 0

Use an `if` instead of `is_release_version && printf` so the helper returns 0
even when it emits no tag. A non-zero return inside the $(latest_arg ...) build
argument is fragile under set -e (and would abort with inherit_errexit), which
would break the edge/main builds the policy explicitly supports.

* fix(docker): copy patches/ into jobs image so pnpm install resolves patchedDependencies

The jobs Dockerfile ran `pnpm install` without copying patches/, so pnpm
failed hashing package.json#pnpm.patchedDependencies (layerchart patch),
breaking the self-host full-stack build. api/dashboard already copy patches/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(docker): build jobs worker deps in topological order

The jobs Dockerfile built workspace packages with a hand-written list that
omitted transitive deps (e.g. @cio/question-types, consumed by @cio/utils),
so `tsc` failed with TS2307 during the self-host build. Use the recursive
`pnpm -r --filter @cio/jobs-worker... build` like the api/dashboard images,
which builds the full dependency closure in the correct order.

* refactor(docker): move jobs Dockerfile to docker/Dockerfile.jobs and align dep install

Move apps/jobs/Dockerfile -> docker/Dockerfile.jobs so all three image recipes
live together in docker/, and update every reference (docker-compose.yaml,
docker-push.sh, docker-publish.yml, railway.jobs.json, self-hosting.mdx).

Also align its dependency install with the api/dashboard images: replace the
hand-maintained list of workspace package.json copies (which silently missed
transitive deps like @cio/core / @cio/ai-assistant, leaving their node_modules
uninstalled and breaking the build with TS2307) with the proven pnpm-fetch
pattern: fetch the full lockfile into a cached store, COPY . ., then an offline
filtered install of @cio/jobs-worker... This installs the complete dependency
closure regardless of how the workspace graph grows.

* refactor(docker): move docker-compose.yaml to repo root for Coolify compatibility

Coolify's Docker Compose build pack forces Base Directory to the repo root and
resolves build contexts from there, so the compose's `context: ..` (needed when
the file lived in docker/) climbed above the repo and failed with
`lstat /artifacts/docker: no such file or directory`.

Move docker/docker-compose.yaml -> ./docker-compose.yaml and change the build
contexts from `..` to `.` so the context is the repo root with no upward path.
This works identically for run-docker-full-stack.sh, manual `docker compose up`,
and Coolify. Dockerfiles stay referenced as docker/Dockerfile.*. Update the
references in run-docker-full-stack.sh and the docs accordingly. The images
compose (docker-compose.images.yaml) is unaffected (no build context) and stays
in docker/.

* Add self-hosting documentation for Coolify and Railway

- Created new documentation for deploying ClassroomIO on Coolify using a pull-based Docker Compose stack.
- Added comprehensive guide for self-hosting ClassroomIO with Docker, covering both pre-built images and building from source.
- Introduced a new meta.json file to organize self-hosting documentation pages.
- Developed a Railway deployment guide with environment variable configurations for API, dashboard, and jobs worker.
- Updated self-hosting blog post to link directly to the new Railway guide.
- Removed obsolete Railway configuration files (railway.api.json, railway.dashboard.json, railway.jobs.json).
- Enhanced run-docker-full-stack.sh script to improve MinIO credential generation and service readiness checks.
- Updated docker-compose files to enforce strong secret requirements for authentication.
- Adjusted self-hosting references in the Docker documentation to include new guides.

* fix(docker): point references at root-level docker-compose.images.yaml

The pull-based compose file moved to the repo root (alongside docker-compose.yaml).
Update IMAGES_COMPOSE_FILE in run-docker-full-stack.sh, the file's own header
example, and the docs (docker.mdx curl URL, coolify.mdx path + blob link,
SELF_HOST.md relative link + curl URL).

* fix(self-host): address review round 2 — drop unused AUTH_BEARER_TOKEN, pull-by-default, doc fixes

- Remove AUTH_BEARER_TOKEN everywhere (unused in app code; PRIVATE_SERVER_KEY is the real
  dashboard<->api key). Simplifies run-docker-full-stack.sh secret generation.
- run-docker-full-stack.sh now defaults to PULLING pre-built images; add --build for source.
- docs: recommend MinIO, 'a way' (not 'recommended way') to self-host, remove Direct API Access
  section, trim Common Commands, SMTP vars per-line, fix PRIVATE_SERVER_KEY wording.

* feat(db): versioned migrations for self-host with auto-baseline

- Regenerate a single clean baseline migration; db:setup now runs drizzle-kit migrate (not push).
- Add db:generate / db:migrate / db:baseline scripts.
- baselineMigrationsIfNeeded(): adopt an existing (push/supabase) schema into drizzle's journal so
  migrate won't try to recreate existing objects ("type/table already exists"). No-ops on fresh
  and already-tracked databases; runs automatically in db:setup before migrate.

* ci(openapi): drop stale ci.yml reference in turbo-cache comment

ci.yml was removed earlier in this review; the openapi workflow's turbo cache still
warms from its own prior runs via restore-keys, so this is a comment-only fix.

* fix(db): make baseline squash-safe for migrate-tracked databases

baselineMigrationsIfNeeded returned early whenever drizzle.__drizzle_migrations
had any rows. After a migration squash, an instance tracked under the old
journal (pre-squash hashes) would then let `drizzle-kit migrate` re-run the
consolidated baseline — whose journal `when` is newer than every old created_at
— and crash on `CREATE TYPE/TABLE "..." already exists` at startup.

Now, when history exists but the consolidated baseline hash is absent, insert
that hash (marking it already applied) so migrate skips it. Only the baseline
entry (idx 0) is reconciled, so genuinely pending future migrations still run.

Verified end-to-end against Postgres: reproduces the original crash, confirms
the fix no-ops migrate on a pre-squash DB, and confirms a later 0001 migration
still applies. Fresh-DB and already-adopted paths unchanged.

* fix(docker): add explicit id to pnpm-store cache mounts

Railway's Metal builder rejects BuildKit cache mounts that omit an
explicit id (it does not default id to target like standard Docker/
BuildKit do), failing the build at parse time with:

  dockerfile invalid: flag '--mount=type=cache,target=/pnpm/store'
  is missing an id argument

Add id=pnpm-store to all six cache mounts across the api, dashboard,
and jobs images. Harmless on Docker Desktop / CI / GitHub Actions, and
required for the Railway self-host build path.

* fix(docker): drop BuildKit cache mounts for Railway compatibility

Railway's builder rejects cache-mount ids not prefixed with its
per-service cacheKey, which can't be hardcoded in a Dockerfile shared
across CI, local Docker, and three Railway services. Remove the
--mount=type=cache flag (keep --store-dir /pnpm/store); the store is
baked into the fetch layer and reused by the offline install, so
behaviour is unchanged. Supersedes 9c4633f2c.

* fix(docker): force LF for shell scripts and Dockerfiles via .gitattributes

cio-api crash-looped on Railway with 'exec /app/entrypoint.sh: No such
file or directory' because the entrypoint shipped with CRLF endings
(Windows checkout, core.autocrlf=true) — the '#!/bin/sh\r' shebang is
unresolvable in the container. Add .gitattributes pinning *.sh and
Dockerfile* to LF so container-executed files are never CRLF regardless
of contributor OS.

* fix(db): serialize db-setup with an advisory lock to prevent baseline double-insert

Concurrent db:setup runs (e.g. multiple API replicas booting together) could both
pass the baseline 'already present?' check and insert duplicate rows into
drizzle.__drizzle_migrations, which has no unique constraint on hash. Wrap baseline
+ migrate in a pg_advisory_lock held on a dedicated connection so concurrent
startups serialize: the loser blocks, then sees the baseline applied and no-ops.

* docs(railway): update template deploy URL to classroomio-1

---------
Co-authored-by: Rotimi Best <best@classroomio.com>
2026-06-23 10:17:09 +01:00

14 KiB

ClassroomIO Docker Self-Hosting

Prerequisites

  • Docker Desktop installed and running
  • git installed

Quick Start

git clone https://github.com/classroomio/classroomio.git
cd classroomio
cp .env.example .env
# Edit .env — set your dashboard domain and secrets
./run-docker-full-stack.sh

The script builds and starts: postgres, redis, minio (object storage), api (localhost:3081), dashboard (localhost:3082), and the jobs worker (background processing). Database schema setup runs automatically inside the api container on startup (see docker/entrypoint-api.sh; skip with SKIP_DB_SETUP=true).

It auto-generates secure values for PRIVATE_SERVER_KEY and BETTER_AUTH_SECRET when they are missing or left at placeholders, and randomizes MinIO credentials on first provision.

The jobs worker is required, not optional. Without it, video processing, captions/transcription, AI course generation, and most emails are enqueued in Redis but never run — uploads stay stuck "processing" with no error. See Background Jobs Worker below.

Run from Pre-built Images (no build)

The Quick Start builds images from source — that needs the full repo and ~8 GB RAM for the dashboard build. Most self-hosters should instead pull the published classroomio/{api,dashboard,jobs} images. You only need docker-compose.images.yaml and a .env:

# 1. Get just the compose file + env template (no full clone required)
curl -O https://raw.githubusercontent.com/classroomio/classroomio/main/docker-compose.images.yaml
curl -o .env https://raw.githubusercontent.com/classroomio/classroomio/main/.env.example
# 2. Edit .env — set CIO_VERSION (pin a release!), DASHBOARD_ORIGIN, secrets, SMTP
# 3. Pull and start
CIO_VERSION=1.4.2 docker compose -f docker-compose.images.yaml --env-file .env pull
CIO_VERSION=1.4.2 docker compose -f docker-compose.images.yaml --env-file .env up -d

From a full clone you can also use the helper: ./run-docker-full-stack.sh --images.

Secrets aren't auto-generated in this path (the helper script does that). Set BETTER_AUTH_SECRET and PRIVATE_SERVER_KEY to strong random values yourself.

Versioning & Upgrades

Published images use a stable-releases + edge tag policy:

Tag Meaning Use it for
1.4.2 (exact) An immutable, smoke-tested release Production — always pin this
1.4, 1 Latest patch/minor of that line Tracking a line
latest The newest released version Trying it out
edge The last main build (may be unstable) Testing upcoming changes

latest only moves when a real release is tagged — a routine merge to main publishes edge, never latest. Pin CIO_VERSION to an exact version in production so a new release can't break you unattended.

# Upgrade: bump the version, pull, recreate
CIO_VERSION=1.5.0 docker compose -f docker-compose.images.yaml --env-file .env pull
CIO_VERSION=1.5.0 docker compose -f docker-compose.images.yaml --env-file .env up -d

# Rollback: set CIO_VERSION back to the previous release and repeat

Schema migrations run automatically on api startup (db:setup), so upgrades need no manual DB step. If a release consolidates ("squashes") older migrations, the startup baseline detects an existing instance and adopts the new baseline as already-applied — your database is upgraded in place without re-running schema that already exists.

(Maintainers: every published image is boot-smoke-tested in CI before it ships — see PUBLISHING_IMAGES.md.)

Environment Variables

All Docker services read from a single root .env file. See .env.example for the full annotated template with required/optional grouping.

Key points:

  • Required: PUBLIC_IS_SELFHOSTED=true, DASHBOARD_ORIGIN (set this to your real domain in production — it also drives public media URLs, see Production Object Storage).
  • API routing: Browser dashboard calls go to the dashboard origin and are proxied to PRIVATE_SERVER_URL (default: http://api:3081). You do not need API/dashboard cookie-domain matching for normal dashboard auth.
  • Direct API access (optional): Set PUBLIC_SERVER_URL and TRUSTED_ORIGINS only if browsers or third-party clients need to call the API origin directly.
  • CSP (runtime): ALLOWED_EXTERNAL_DOMAINS (overrides all) or per-directive: CSP_SCRIPT_SRC_DOMAINS, CSP_STYLE_SRC_DOMAINS, CSP_CONNECT_SRC_DOMAINS, CSP_FRAME_SRC_DOMAINS, CSP_FONT_SRC_DOMAINS, CSP_MEDIA_SRC_DOMAINS. These are read at container startup — no image rebuild needed. The API does not need to be added for normal dashboard calls.
  • Auth cookies: No cookie-domain env is needed. The dashboard proxy makes auth first-party and Better Auth sets host-only dashboard cookies.
  • Auto-generated: PRIVATE_SERVER_KEY, BETTER_AUTH_SECRET (by ./run-docker-full-stack.sh). A strong value you set yourself is never overwritten.
  • Auto-configured: All MINIO_* / OBJECT_STORAGE_* vars (by the startup script, with randomized MinIO credentials).
  • Email (effectively required): SMTP (or Zoho). See Email — without it, signup verification, password reset, and invites do not send.
  • Optional: Google OAuth, Unsplash, LICENSE_KEY (enterprise).

PRIVATE_SERVER_KEY must be the same value in both API and dashboard — the script ensures this.

Verify

docker compose --env-file .env -p classroomio -f docker-compose.yaml ps
curl -sS http://localhost:3081/    # API — returns JSON
curl -I  http://localhost:3082/    # Dashboard — returns 200

Seed Demo Data

Schema setup runs automatically on api startup. To additionally seed demo data, run the seed script inside the running api container:

docker exec cio-api pnpm --filter @cio/db db:setup:seed

Command Reference

All commands use this prefix — abbreviated as dc below:

dc = docker compose --env-file .env -p classroomio -f docker-compose.yaml
Task Command
Full stack (pulls pre-built images, with MinIO) ./run-docker-full-stack.sh
Build from source instead of pulling ./run-docker-full-stack.sh --build
Exclude MinIO ./run-docker-full-stack.sh --no-minio
API-only smoke test dc up --build -d postgres redis api
Service status dc ps
Stream logs dc logs -f api dashboard
Rebuild one service dc up -d --build api
Restart (env change, no rebuild) dc restart api dashboard
Stop dc down
Stop + delete volumes dc down -v

Build Images Locally

docker build -f docker/Dockerfile.api -t classroomio/api:latest .
docker build -f docker/Dockerfile.dashboard -t classroomio/dashboard:latest .

Inspect Container Env Vars

docker exec cio-api env | sort
docker exec cio-dashboard env | sort

Container names: cio-postgres, cio-redis, cio-api, cio-dashboard, cio-jobs, cio-minio.

MinIO (Object Storage)

MinIO is included by default. The startup script auto-configures env vars, starts MinIO, and creates videos, documents, and media buckets.

  • API endpoint: http://localhost:9000
  • Web UI: http://localhost:9001 (default: minioadmin / minioadmin)

To start MinIO manually (e.g. after --no-minio):

docker compose -f docker-compose.yaml --profile minio up -d
  • Web UI: http://localhost:9001 (credentials are the randomized MINIO_ROOT_USER / MINIO_ROOT_PASSWORD written to your .env).

Security: The startup script randomizes MINIO_ROOT_USER / MINIO_ROOT_PASSWORD on first provision and mirrors them into OBJECT_STORAGE_ACCESS_KEY_ID / OBJECT_STORAGE_SECRET_ACCESS_KEY. If you change them, keep both pairs in sync.

MinIO Web UI Not Loading

If port 9001 does not respond, recreate the container:

docker rm -f cio-minio
docker compose -f docker-compose.yaml --profile minio up -d

Data is preserved in the minio-data volume.

Background Jobs Worker

The jobs service (cio-jobs) is a BullMQ worker that processes everything ClassroomIO does asynchronously:

  • Video processing — probe, thumbnails, audio extraction, transcoding
  • Transcription / captions — via OpenAI Whisper (needs OPENAI_API_KEY)
  • AI course generation — the "generate a course with AI" agent
  • Emails — invites, notifications, and other non-auth emails
  • Maintenance — periodic cleanup

It shares Redis and the database with the API. It is started automatically by the full-stack script. If it is not running, jobs accumulate in Redis and silently never complete.

docker compose --env-file .env -p classroomio -f docker-compose.yaml logs -f jobs

Note: the in-course AI tutor chat runs synchronously inside the API and does not need the worker. Course generation, transcription, and media processing do.

Email

Email is effectively required: signup verification and password reset are sent by the API, and invites/notifications are sent by the worker. Configure one of:

  • SMTP: SMTP_HOST, SMTP_PORT, SMTP_USER, SMTP_PASSWORD, SMTP_SENDER
  • Zoho / ZeptoMail: ZOHO_TOKEN

Both the api and jobs services read these. Without them, auth and invite flows appear to succeed in the UI but no email is delivered (ECONNREFUSED in logs).

Production Object Storage

The bundled MinIO is fine for a single host. Two things matter for a real deployment:

  1. Public media URLs must point at your domain, not localhost. Media URLs are embedded in pages served to browsers, so http://localhost:9000/... breaks for every remote visitor. The startup script derives OBJECT_STORAGE_PUBLIC_ENDPOINT and OBJECT_STORAGE_MEDIA_PUBLIC_BASE_URL from DASHBOARD_ORIGIN when it is a real domain — make sure your reverse proxy routes <your-domain>/media to MinIO on port 9000 (or set those two variables explicitly).
  2. External S3 / Cloudflare R2 instead of MinIO. Run with --no-minio and set the OBJECT_STORAGE_* variables (endpoint, public endpoint, access key, secret, buckets) to your provider. The startup script fails fast if --no-minio is used without an OBJECT_STORAGE_ENDPOINT, so you cannot accidentally launch with no storage.

Backups & Persistence

All state lives in named Docker volumes: postgres-data, redis-data, minio-data (see docker-compose.yaml). docker compose ... down -v deletes them — and all your data. Back up regularly:

# Database (most important)
docker exec cio-postgres pg_dump -U postgres classroomio > backup-$(date +%F).sql

# Object storage — copy the minio-data volume, or use `mc mirror` to an off-box bucket

Restore the database into a fresh stack with psql/pg_restore against cio-postgres.

Scaling & Repeated Startups

db:setup runs on every api container start (idempotent schema sync). If you run multiple API replicas, set SKIP_DB_SETUP=true on all but one to avoid concurrent migration races, and run setup once out-of-band. The worker queues can also be scaled independently by overriding the jobs container command with the per-queue start:* scripts (e.g. start:media, start:media-transcribe) — see apps/jobs/package.json.

HLS Video Playback

Signed HLS playback (HLS_SIGNING_SECRET) is validated by the tenant-router Cloudflare Worker, which is not part of this compose stack. Self-hosted deployments serve media via the object storage public URL above; signed-HLS delivery is out of scope for the compose setup.

Rebuild After Code Changes

This setup builds images from source (no live hot reload). Code changes require a rebuild:

# One service
docker compose --env-file .env -p classroomio -f docker-compose.yaml up -d --build api

# Both (e.g. shared packages/* changed)
docker compose --env-file .env -p classroomio -f docker-compose.yaml up -d --build api dashboard

# Full stack
./run-docker-full-stack.sh

Troubleshooting

"No space left on device"

df -h                          # check host disk
docker system df               # check Docker usage
docker builder prune -f        # free build cache
docker image prune -a -f       # free unused images
docker container prune -f      # free stopped containers

Warning: docker volume prune removes volumes not used by any container. If run after docker compose ... down, it deletes postgres-data, redis-data, and minio-data. Use only the prunes above to free space safely.

Then restart: ./run-docker-full-stack.sh

"The database system is in recovery mode" (57P03)

Postgres has not finished starting. Wait 30-60 seconds and retry. The compose healthcheck requires a successful SELECT 1 before dependent services start.

MinIO "Storage reached its minimum free drive threshold"

Same root cause as disk full. Free space using the steps above and restart MinIO:

docker compose --env-file .env -p classroomio -f docker-compose.yaml restart minio

SMTP Errors in Logs

ECONNREFUSED 127.0.0.1:465 means SMTP is not configured. This does not block startup — configure SMTP or Zoho when you need email.

Port Conflicts

lsof -nP -iTCP:3081 -sTCP:LISTEN
lsof -nP -iTCP:3082 -sTCP:LISTEN