Release Notes: 8.4

DipDup 8.4 introduces improvements to environment variable management, a new watchdog service for enhanced reliability, and numerous stability fixes.

Watchdog service

A new watchdog service has been added to help detect and handle situations where the indexer may become unresponsive. Common causes for the indexer hanging include:

• Long-running callback functions. Possible causes include infinite loops, heavy computations, or blocking I/O operations.
• Stalled SQL transactions, which can occur due to deadlocks or long-running queries.
• Unresponsive Websocket connections (ping-pong successful but no data received).

The watchdog monitors these triggers and take action if they exceed a specified timeout. Actions can be set to warning, exception, or ignore for each type.

Example configuration with default values:

advanced:
  watchdog:
    callback:
        action: warning
        timeout: 10
    transaction:
        action: warning
        timeout: 10
    websocket:
        action: warning
        timeout: 60

Managing env variables

Envvfile generation

Before this release, DipDup used .env.default files to help users manage environment variables. These files were automatically generated on every init call and included in the project package. When setting up new environments, users were expected to copy the .env.default file to .env and modify it as needed. This approach had several drawbacks:

• .env.default stubs could easily become outdated, leading to potential misconfiguration.
• Including any envfile-alike files in the project package increased the risk of accidentally committing secrets to git.

In this release, .env.default files have been removed. Instead, use the config env command to manage environment variables in a more robust and flexible way. It ensures that your environment is always up to date, allows you to resolve the current environment, chain configuration files as needed, and add internal DIPDUP_ variables.

# Show variables used in the root config
dipdup config env

# Create a new envfile for `compose` environment
dipdup -C compose config env -o .env

# Add another config to the chain and include internal variables
dipdup -C compose -C dev config env -o .env --internal

dipdup.env file

Now you can create a dipdup.env file in the current working directory. It will be loaded automatically when you run DipDup and take precedence over other sources. This is mostly useful for internal variables.

## Enable debug mode for every `dipdup` command from cwd
echo "DIPDUP_DEBUG=1" >> dipdup.env

Changes since 8.3

Added

• cli: Added --no-types option to init command to skip generating type classes.
• cli: Added -h shorthand for --help option in all commands.
• cli: Added loading env-file dipdup.env if presented in the current directory.
• config: Added realtime flag to datasource config to enable websockets/polling implicitly.
• context: Added configurable watchdog service to notify about long-running callbacks and transactions.

Fixed

• cli: Fixed config env -i command output.
• cli: Fixed discovering package path.
• cli: Fixed printing help message when running commands without arguments.
• cli: Skip logging indexer status if it has not changed.
• codegen: Fixed loading ABIs from the project with no ABI datasources configured.
• config: Do not trigger config_modified reindexing when datasource URLs are updated.
• config: Fixed config export command crash when advanced.reindex section is present in config.
• datasources: Do not run datasources not linked to any index and without realtime flag set.
• env: Skip detection of some variables if set explicitly.
• http: Fixed merging request URL with base path.
• models: Fixed serializing UUIDs in JSONField.
• project: Fixed make image command and default workdir.
• project: Fixed missing codegen headers in project base.
• substrate.events: Fixed crash caused by choosing incorrect runtime metadata for decoding.
• substrate.node: Fixed event index field.
• substrate: Fixed loading type registries when specified in config.
• substrate: Fixed processing several vector types.

Removed

• project: Do not generate and track .env.default files on init. Use config env command for new environments.

See you soon! 👋

DipDup is a free open-source software created by the community and for the community. Join our socials to discuss this release, ask any questions, or participate in development.

Twitter | Discord | GitHub