Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
192 lines (129 loc) · 8.14 KB

CONTRIBUTING.md

File metadata and controls

192 lines (129 loc) · 8.14 KB
Raw

Contributing to LuxonisTrain

This guide is intended for our internal development team. It outlines our workflow and standards for contributing to this project.

Pre-requisites

Clone the repository and navigate to the root directory:

git clone git@github.com:luxonis/luxonis-train.git
cd luxonis-train

Install uv and sync the development environment:

uv sync

Note

This creates a .venv with the package installed in editable mode, together with the dev dependencies.

To also install the AIMET support, run:

uv sync --extra aimet

The requirements*.txt files are exports of uv.lock for users installing with pip. Do not edit them manually. After changing the dependencies, run:

scripts/export_requirements.sh

Note

A pre-commit hook runs the script for you - you only need to stage the refreshed files.

Pre-commit Hooks

We use pre-commit hooks to ensure code quality and consistency. The hooks are run by prek, a faster drop-in replacement for pre-commit.

  1. Run uv run prek install in the root directory.
  2. The hooks will now run automatically on git commit.
    • If a hook fails, it will print an error message and abort the commit.
    • Some hooks will also modify the files in-place to fix found issues.

To run all the hooks manually, use uv run prek run --all-files.

Do not commit directly to main - the no-commit-to-branch hook blocks it.

Cognitive Complexity

The complexipy pre-commit hook gates the cognitive complexity of every function. A function must stay at complexity 15 or below.

When the hook fails, it prints ranked refactor plans for failing functions (the --suggest-refactors flag). Follow one of the plans, or simplify the function in another way. The usual levers: guard clauses, merged conditions, and extraction of helpers.

To check by hand, run:

uv run complexipy luxonis_train tests

Documentation

We write docstrings in the Google style. pydoctor builds the API docs from them. The [tool.pydoctor] section of pyproject.toml holds its options.

Do not write types in docstrings. The annotations of the signature hold the types, and the docs show them from there.

Single backticks link to a name in luxonis_train. Put a name from another library, such as ``torch.nn.Conv2d``, in double backticks. The docs build does not download the indexes of other projects, so it cannot link to them.

def resize(image: Tensor, size: int) -> Tensor:
    """Resize an image to a square.

    Args:
        image: The image with shape ``[C, H, W]``.
        size: The side of the result, in pixels.

    Returns:
        The resized image with shape ``[C, size, size]``.

    """

To verify that your documentation is formatted correctly, run the following command:

uv run --group docs pydoctor

Type Checking

The codebase is type-checked using pyright, pinned in the dev dependency group to match CI. To run type checking, use the following command in the root project directory:

uv run pyright --warnings --project pyproject.toml

Editor Support

Tests

We use pytest for testing. The tests are located in the tests directory. To run the tests with coverage, use the following command:

uv run pytest --cov=luxonis_train --cov-report=html

This command will run all tests and generate HTML coverage report.

Tip

The coverage report will be saved to htmlcov directory. If you want to inspect the coverage in more detail, open htmlcov/index.html in a browser.

Tip

You can choose to run only the unit-tests or only the integration tests by adding -m unit or -m integration to the pytest command.

Important

Mark an integration test with @pytest.mark.gpu when it trains, tunes or quantizes a model. CI runs these tests on the GPU runner and all the other tests on a CPU runner. A test without the marker still passes on the CPU, but it no longer covers the GPU code of the training, such as mixed precision.

Important

If a new feature is added, a new test should be added to cover it. The minimum overall test coverage for a PR to be merged is 90%. The minimum coverage for new files is 80%.

GitHub Actions

Our GitHub Actions workflow is run when a new PR is opened.

  1. First, the pre-commit hooks must pass and the documentation must be built successfully.
  2. Next, the type checking is run.
  3. If all previous checks pass, the tests are run.

The tests run in five suites, one job each. The GPU suites start only after the cpu suite passes, so that a failure there costs no GPU time:

Suite Tests Runner
cpu The unit tests, and the integration tests without the gpu marker ubuntu-latest
misc The integration tests with the gpu marker ubuntu-t4-4core
predefined_light, predefined_heavy test_predefined_models.py, the light and the heavy variants ubuntu-t4-4core
combinations test_combinations.py ubuntu-t4-4core

CI does not run test_overfit_convergence.py, which has its own overfit_convergence marker.

Two rules keep the GPU suites from running when they cannot find anything new:

  • A pull request that changes only docstrings, comments, Markdown files or the settings of the linters and of the API docs runs the cpu suite only. scripts/only_docstrings_changed.py decides this.
  • A draft pull request skips predefined_heavy. The suite runs when you mark the pull request as ready for review.

Tip

Review the GitHub Actions output if your PR fails.

Important

Successful completion of all the workflow checks is required for merging a PR.

Releases

A release needs two manual steps: start the workflow, then merge the pull request it opens.

  1. Run the Release PR workflow from the Actions tab. Give it major, minor, patch, or an explicit version such as 1.2.3.
  2. The workflow changes __version__ in luxonis_train/__init__.py and opens a release/vX.Y.Z pull request that lists the changes after the last tag.
  3. Review the pull request, wait for CI, and merge it.
  4. The Release Tag workflow then tags vX.Y.Z-beta on the merge commit and creates the GitHub release with generated notes.
  5. The release publication starts the PyPI upload.

Both workflows are thin callers. The shared logic lives in luxonis-ml, in .github/workflows/reusable-release-pr.yaml and .github/workflows/reusable-release-tag.yaml, so every Luxonis repository releases the same way.

.github/release.yaml groups the generated notes into categories from the pull request labels.

Important

The release runs on the WORKFLOW_SECRET secret. The default token does not start CI on a bot pull request, and it cannot start the publish workflow from a release event.

Making and Submitting Changes

  1. Make changes in a new branch with a descriptive prefix such as feat/ or fix/.
  2. Test your changes locally.
  3. Commit your changes (pre-commit hooks will run).
  4. Push your branch and create a pull request.
  5. The team will review and merge your PR.