Config

Developing a DipDup indexer begins with creating a YAML config file of a specific format. DipDup expects it to be located in the project root and named dipdup.yaml. However, you can provide any path with a -c CLI option.

Structure

Config consists of multiple top-level mappings. In the table below they are grouped by sections for convenience, but in the config file, they can be in any order. The only required section is Header.

See Config reference guide for the full list of available options.

Headerspec_version*DipDup project specification; currently "3.0"
package*Python package name
InventorydatabaseDatabase configuration
contractsContracts that need to be indexed
datasourcesDatasources to gather information from
IndexesindexesIndex definitions
templatesAlso index definitions, but with placeholders that make them reusable
HookshooksCallbacks to run manually or by schedule
jobsSchedules for hooks
IntegrationsapiInternal API configuration
hasuraHasura GraphQL Engine configuration
sentrySentry configuration
prometheusPrometheus configuration
MiscellaneousadvancedTunables that affect framework behavior
customMapping of user-defined values; neither typed nor validated
loggingConfigure logging verbosity

Config merging

DipDup allows you to customize the configuration for specific environments or workflows. It works similarly to Docker Compose anchors but only for top-level sections. If you want to override a nested property, you need to recreate the entire top-level section.

Environment-specific configurations

You can organize your configuration files in a configs/ directory within your project.

Base template contains three environment-specific configurations:

my-dipdup-project/
├── dipdup.yaml              # Root config `database` and `hasura` sections
├── configs/
   ├── dipdup.sqlite.yaml   # SQLite configuration override
   ├── dipdup.compose.yaml  # Docker Compose configuration override
   └── dipdup.swarm.yaml    # Docker Swarm configuration override

To merge multiple DipDup config files, provide the -c command-line option multiple times. Root config must be specified first.

Terminal
# Merge root config with SQLite-specific settings
dipdup -c . -c configs/dipdup.sqlite.yaml run

# Using a shorthand
dipdup -C sqlite run

# Combining multiple configuration overrides
dipdup -C sqlite -C mainnet run
Note
Tip: Avoid adding sections related to database and integrations to the root config unless local configuration is required to run the indexer.

Creating custom configurations

Config merging can be easily extended. Some ideas include:

  • dipdup.testnet.yaml to override datasources and contracts sections
  • dipdup.debug.yaml to override logging and sentry sections

Exporting the merged configuration

You can use the config export command to see the merged configuration that will be used by DipDup. This is useful for debugging and understanding how different configurations are combined.

Terminal
# Export the merged configuration
dipdup -C sqlite config export

# Show all available options
dipdup config export -h

Environment variables

DipDup supports compose-style variable expansion in dipdup.yaml and other configuration files. You can use placeholders in the format ${VARIABLE_NAME} or ${VARIABLE_NAME:-default_value} to substitute environment variables at runtime. If a variable is not set and no default value is provided, DipDup will raise an error.

This feature is useful for:

  • Keeping sensitive data like API keys or passwords out of your version-controlled configuration files.
  • Customizing deployments for different environments (development, staging, production) without altering the base configuration.

You can use these placeholders anywhere throughout the configuration file. For example:

dipdup.yaml
database:
  kind: postgres
  host: ${POSTGRES_HOST:-localhost}
  password: ${POSTGRES_PASSWORD}

There are multiple ways to pass environment variables to DipDup:

  • Export them in the shell before running DipDup
  • Create the env file and pass it to DipDup with the -e CLI option

See Environment Variables for advanced usage, troubleshooting, and built-in variables.

Contract typenames

typename is an alias for the particular contract script, meaning that two contracts sharing the same code can have the same type name. If this field is not set, a contract alias will be used instead.

dipdup.yaml
contracts:
  kusd_dex_mainnet:
    kind: tezos
    address: KT1CiSKXR68qYSxnbzjwvfeMCRburaSDonT2
    typename: quipu_fa12
  tzbtc_dex_mainnet:
    kind: tezos
    address: KT1N1wwNPqT5jGhM91GQ2ae5uY8UzFaXHMJS
    typename: quipu_fa12

If multiple contracts you index have the same interface but different code, see F.A.Q. to learn how to avoid conflicts.

Reindexing

In some cases, DipDup can't proceed with indexing without a full wipe. Several reasons trigger reindexing:

reasondescription
manualReindexing triggered manually from callback with ctx.reindex.
migrationApplied migration requires reindexing. Check release notes before switching between major DipDup versions to be prepared.
rollbackReorg message received from datasource and can not be processed.
config_modifiedOne of the index configs has been modified.
schema_modifiedDatabase schema has been modified. Try to avoid manual schema modifications in favor of SQL scripts.

It is possible to configure desirable action on reindexing triggered by a specific reason.

actiondescription
exception (default)Raise ReindexingRequiredError and quit with error code. The safest option since you can trigger reindexing accidentally, e.g., by a typo in config. Don't forget to set up the correct restart policy when using it with containers.
wipeDrop the whole database and start indexing from scratch. Be careful with this option!
ignoreIgnore the event and continue indexing as usual. It can lead to unexpected side-effects up to data corruption; make sure you know what you are doing.

To configure actions for each reason, add the following section to the DipDup config:

advanced:
  reindex:
    manual: wipe
    migration: exception
    rollback: ignore
    config_modified: exception
    schema_modified: exception

Advanced options

Flags related to the project are set in the advanced section of the config (most likely in dipdup.yaml).

flagdescription
early_realtimeEstablish realtime connection and start collecting messages while sync is in progress (faster, but consumes more RAM).
decimal_precisionOverwrite precision if it's not guessed correctly based on project models.
postpone_jobsDo not start job scheduler until all indexes reach the realtime state.
rollback_depthA number of levels to keep for rollback.
unsafe_sqliteDisable journaling and data integrity checks. Use only for testing.
Help and tips -> Join our Discord
Ideas or suggestions -> Issue Tracker
GraphQL IDE -> Open Playground