Contributing

DipDup is a free and open-source software licensed under the MIT License. The source code is available on GitHub.

Contributor Guide

Thanks for considering contributing to DipDup! We're happy to have you here. This document is meant to help you get started with the project. If you have any questions, feel free to open an issue on GitHub or join our Discord server. Have fun!

Workflow

• Fork the repository, clone it, and git checkout next.
• Create a new branch with git checkout -b <your-branch-name>.
• Make your changes. Run make format lint to perform basic code checks.
• When finished, push your branch with git push origin --set-upstream <your-branch-name>.
• Create a pull request to merge <your-branch-name> into next. Maintainers will review your pull request and may make comments, ask questions, or request changes. When all feedback has been addressed the pull request will be approved, and after all checks have passed it will be merged by a maintainer.

Development environment

You'll need Python 3.12 and PDM package manager to run DipDup locally. To set up the development environment, run pdm venv create python3.12 && pdm sync && $(pdm venv activate). To see the list of development commands, run make help.

Maintainer Guide

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

General

• All code in this repository MUST be licensed under the MIT License.
• Python code in this repository MUST run on Python 3.12. Using modern language features is encouraged.
• Python code in this repository MUST run in Linux, macOS, and Docker environments, and on amd64, arm64 architectures. CI will help you with that.

Git workflow

• Default branches are next for the latest release and current for the previous one. next is the default branch for pull requests.
• Branch names SHOULD follow the prefix/short-description format. Prefixes currently in use: feat for features, fix for bugfixes, docs for documentation, exp for experiments, ref for refactoring, ci for GitHub Actions, scripts and Docker stuff, aux for everything else.
• Commits in pull requests MUST be squashed when merging to next.
• Issues and pull requests MUST have a descriptive title; they SHOULD be linked to each other, appropriately labeled, and assigned to maintainers while in progress.

Codestyle

We use several tools to enforce the code style and decent code quality: black for autoformatting, ruff for linting, and mypy for type checking. All checks MUST pass before the code is merged to the default branch. Anything not enforced by these tools is up to the developer. But here are some recommendations:

• Consistency is the key. If you see a pattern in the codebase, follow it. Use meaningful names for variables, and avoid abbreviations.
• Use NOTE/TODO/FIXME prefixes for meaningful comments, Avoid inline comments. It helps a lot in navigating the codebase.
• Lazy imports are important to keep startup time low for tiny commands. We also do this for project imports.
• Some methods and attributes are made private to avoid cluttering the public API. Feel free to access them from the outside if you know what you are doing.
• Finally, about exact language features. f-string formatting is preferred over other syntax. Be careful with the walrus operator. Don't forget else in conditional expressions. Listen to your mom. We do not yet have a consensus on match-case.

Changelog

• All changes that affect the user (developer) experience MUST be documented in the CHANGELOG.md file.
• The changelog MUST follow the "Keep a Changelog" specification (CI will break otherwise). Group order we use: Added, Fixed, Changed, Deprecated, Removed, Performance, Security, Other.
• Lines describing changes MUST be sorted and start with the component name (usually the Python module name).

Documentation

• We have a pretty complex process of building and deploying docs. It starts with Markdown files in the docs directory. Then scripts/docs.py script generates several dynamic pages, API references, processes custom Cookiecutter-style macros, and so on. The resulting Markdown is pushed to the private frontend repository via the docs.yml GitHub Action. docs.py script code should answer most of your questions.
• All public APIs MUST be documented using docstrings. We use the reStructuredText (reST) syntax.
• A page in the "Release notes" section MUST accompany all major and minor releases. Avoid using #include macro in Release notes as they should not change after the publication.

Dependencies

• All dependencies MUST be declared in pyproject.toml file and pinned to non-breaking versions. We are more of an application than a library, so no asterisks, please.

Security

• GitHub Dependabot alerts about vulnerable dependencies MUST be investigated and resolved as soon as possible.
• Security-related bugfixes MUST be mentioned in the changelog under the "Security" section.

Privacy

• DipDup MUST NOT collect any data from users.
• DipDup SHOULD NOT perform network requests to APIs not defined in config as datasources. Current exceptions: version check with GitHub.

Docker images

• DipDup dockerfiles use autogenerated requirements.txt files. Maintainers MUST run pdm run update script on every change in dependencies.
• Docker images for stable releases MUST be published on Docker Hub and GitHub Container Registry.
• Maintainers MAY publish arbitrary images on GHCR and remove them when not needed.

Scaffolding

• Project templates SHOULD cover all index types available in DipDup.
• They also MAY contain additional features and integrations.

Demo projects

• Demos are stored in the src directory. They MUST be generated automatically from project templates using replay files.
• Maintainers SHOULD run pdm demos command regularly to ensure that demo projects are up to date.

Releases

• Release versions MUST conform to Semantic Versioning. Releases that introduce breaking changes MUST be major ones.
• The latest major version is the only one supported in general. Important fixes SHOULD be backported to the previous major release.

Release process

Releasing a new version currently requires some manual actions:

• Ensure that all GH issues and PRs are closed and linked to the milestone.
• Checkout to aux/X.Y.Z branch from next (or current for backports). Update DipDup version in pyproject.toml.
• Run make before_release to lock dependencies, dump requirements.txt files, generate demo projects etc.
• Commit and push all changes with msg like Bump version X.Y.Z. Open a PR, and link it to the milestone.
• Now you may want to switch Docker images of demos we host to aux-X.Y.Z tag as a smoke test.
• Merge the PR, then git tag X.Y.Z && git push origin X.Y.Z. Wait for release.yml and docs.yml pipelines to finish.
• Don't forget an announcement on Twitter and Discord.

MIT License

MIT License

Copyright (c) 2021 Baking Bad

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.