Running in Docker
DipDup provides prebuilt Docker images hosted Docker Hub and GHCR. You can use them as is or as a base to build custom images.
link | latest tag |
---|---|
Docker Hub | dipdup/dipdup:8 |
GitHub Container Registry | ghcr.io/dipdup-io/dipdup:8 |
GitHub Container Registry (Nightly) | ghcr.io/dipdup-io/dipdup:next |
All base images are based on python:3.12-slim-bookworm, support amd64 and arm64 architectures. Default user is dipdup
with UID 1000 and home directory /home/dipdup
. Entrypoint is set to dipdup
command.
Nightly builds are published on every push to the next
branch for developers' convenience. Do not use nightlies in production!
Running the base image
To run DipDup in container, you need to copy or mount your project directory and config file to the container. The simplest way to do this is to use bind mounts.
Given your project source code is in dipdup_indexer
directory, you can run DipDup container with the following command:
docker run \
-v dipdup_indexer:/home/dipdup/dipdup_indexer \
dipdup/dipdup:8
-c dipdup_indexer run
If you're using SQLite database, you can also mount it as a volume:
docker run \
-v dipdup_indexer:/home/dipdup/dipdup_indexer \
-v dipdup_indexer.sqlite:/home/dipdup/dipdup_indexer.sqlite \
dipdup/dipdup:8
-c dipdup_indexer -c dipdup_indexer/configs/dipdup.sqlite.yaml run
Extending the base image
In deploy
project directory, you can find default Dockerfile
with the following contents:
{{ header }}
FROM dipdup/dipdup:8
# FROM ghcr.io/dipdup-io/dipdup:8
# FROM ghcr.io/dipdup-io/dipdup:next
# COPY --chown=dipdup pyproject.toml README.md .
# RUN pip install .
COPY --chown=dipdup . dipdup_indexer
WORKDIR /home/dipdup/dipdup_indexer
To change the base image or install additional Python dependencies from pyproject.toml
, uncomment the corresponding lines in the Dockerfile
.
Version pinning
It's recommended to always use the latest version running make update
command from time to time. This way you will always get the latest bug fixes and improvements. However, you can pin the project to specific framework version using X.Y and X.Y.Z image tags.
If your project manifest looks like this:
[project]
dependencies = [
"dipdup==1.2.3"
]
Your Dockerfile
should look like this:
FROM dipdup/dipdup:1.2.3
Docker Compose
Here's an example compose.yaml
file:
{{ header }}
name: dipdup_indexer
services:
dipdup:
build:
context: ..
dockerfile: deploy/Dockerfile
restart: always
env_file: .env
ports:
- 46339
- 9000
command: ["-C", "compose", "run"]
depends_on:
- db
- hasura
db:
image: postgres:16
ports:
- "${POSTGRES_HOST_PORT:-5432}:5432"
volumes:
- db:/var/lib/postgresql/data
restart: always
env_file: .env
environment:
- POSTGRES_USER=dipdup
- POSTGRES_DB=dipdup
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
healthcheck:
test: ["CMD-SHELL", "pg_isready -U dipdup"]
interval: 10s
timeout: 5s
retries: 5
hasura:
image: hasura/graphql-engine:latest
ports:
- "${HASURA_HOST_PORT:-8080}:8080"
depends_on:
- db
restart: always
environment:
- HASURA_GRAPHQL_DATABASE_URL=postgres://dipdup:${POSTGRES_PASSWORD}@db:5432/dipdup
- HASURA_GRAPHQL_ADMIN_SECRET=${HASURA_SECRET}
- HASURA_GRAPHQL_ENABLE_CONSOLE=true
- HASURA_GRAPHQL_DEV_MODE=true
- HASURA_GRAPHQL_LOG_LEVEL=info
- HASURA_GRAPHQL_ENABLE_TELEMETRY=false
- HASURA_GRAPHQL_UNAUTHORIZED_ROLE=user
- HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES=true
volumes:
db:
Environment variables are expanded in the DipDup config file; PostgreSQL password and Hasura secret are forwarded from host environment in this example.
configs/dipdup.compose.yaml
file contains environment-specific config overrides.
{{ header }}
database:
kind: postgres
host: ${POSTGRES_HOST:-db}
port: 5432
user: ${POSTGRES_USER:-dipdup}
password: ${POSTGRES_PASSWORD:-changeme}
database: ${POSTGRES_DB:-dipdup}
hasura:
url: http://${HASURA_HOST:-hasura}:8080
admin_secret: ${HASURA_SECRET:-changeme}
allow_aggregations: ${HASURA_ALLOW_AGGREGATIONS:-true}
select_limit: ${HASURA_SELECT_LIMIT:-10000}
camel_case: ${HASURA_CAMEL_CASE:-true}
sentry:
dsn: ${SENTRY_DSN:-''}
environment: ${SENTRY_ENVIRONMENT:-''}
prometheus:
host: 0.0.0.0
port: 8000
api:
host: 0.0.0.0
port: 46339
mcp:
host: 0.0.0.0
port: 9999
api_url: http://dipdup:46339
Note the command string in compose.yaml
:
services:
dipdup:
command: ["dipdup", "-C", "compose", "run"]
Build and run the containers:
docker-compose up -d --build
Docker Swarm
Scaffolded projects contain a compose file for Docker Swarm. Before spawning this stack create external networks traefik-public
and prometheus-private
. Optionally, deploy Traefik and Prometheus and attach them to these networks to get a fully functional stack.
{{ header }}
services:
dipdup:
image: ${IMAGE:-ghcr.io/dipdup-io/dipdup}:${TAG:-8}
depends_on:
- db
- hasura
command: ["-C", "swarm", "run"]
env_file: .env
networks:
- internal
- prometheus-private
deploy:
mode: replicated
replicas: ${INDEXER_ENABLED:-1}
labels:
- prometheus-job=${SERVICE}
- prometheus-port=8000
placement: &placement
constraints:
- node.labels.${SERVICE} == true
logging: &logging
driver: "json-file"
options:
max-size: "10m"
max-file: "10"
tag: "\{\{.Name\}\}.\{\{.ImageID\}\}"
db:
image: postgres:16
volumes:
- db:/var/lib/postgresql/data
env_file: .env
environment:
- POSTGRES_USER=dipdup
- POSTGRES_DB=dipdup
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
healthcheck:
test: ["CMD-SHELL", "pg_isready -U dipdup"]
interval: 10s
timeout: 5s
retries: 5
networks:
- internal
deploy:
mode: replicated
replicas: 1
placement: *placement
logging: *logging
hasura:
image: hasura/graphql-engine:latest
depends_on:
- db
environment:
- HASURA_GRAPHQL_DATABASE_URL=postgres://dipdup:${POSTGRES_PASSWORD}@dipdup_indexer_db:5432/dipdup
- HASURA_GRAPHQL_ADMIN_SECRET=${HASURA_SECRET}
- HASURA_GRAPHQL_ENABLE_CONSOLE=true
- HASURA_GRAPHQL_DEV_MODE=false
- HASURA_GRAPHQL_LOG_LEVEL=warn
- HASURA_GRAPHQL_ENABLE_TELEMETRY=false
- HASURA_GRAPHQL_UNAUTHORIZED_ROLE=user
- HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES=true
networks:
- internal
- traefik-public
deploy:
mode: replicated
replicas: 1
labels:
- traefik.enable=true
- traefik.http.services.${SERVICE}.loadbalancer.server.port=8080
- "traefik.http.routers.${SERVICE}.rule=Host(`${HOST}`) && (PathPrefix(`/v1/graphql`) || PathPrefix(`/api/rest`))"
- traefik.http.routers.${SERVICE}.entrypoints=http,${INGRESS:-ingress}
- "traefik.http.routers.${SERVICE}-console.rule=Host(`${SERVICE}.${SWARM_ROOT_DOMAIN}`)"
- traefik.http.routers.${SERVICE}-console.entrypoints=https
- traefik.http.middlewares.${SERVICE}-console.headers.customrequestheaders.X-Hasura-Admin-Secret=${HASURA_SECRET}
- traefik.http.routers.${SERVICE}-console.middlewares=authelia@docker,${SERVICE}-console
placement: *placement
logging: *logging
volumes:
db:
networks:
internal:
traefik-public:
external: true
prometheus-private:
external: true
Mounting data directory
When debugging or developing in Docker, you may want to mount /home/dipdup/.local/share/dipdup
from container to the local directory to preserve cache, codegen artifacts and crash dumps.
You will probably need to adjust filesystem permissions to allow the container to write to the mounted directory using chown
or chmod
commands.