-
Notifications
You must be signed in to change notification settings - Fork 106
Releasing to download.pytorch.org and pypi
If its the first time you are releasing you library using this workflow. Please contact MSL Infra PyTorch team for assistance.
This document provides detailed information about the release workflows in the pytorch/test-infra repository that manage the distribution of PyTorch Ecosystem packages to download.pytorch.org and pypi
This workflow manages the release process for distributing PyTorch Ecosystem libraries binaries to download.pytorch.org, the primary distribution channel for PyTorch packages including wheels and libtorch distributions. Please note: The workflow requires PyTorch Dev Infra team member approval. Its advised to start the release with this workflow.
Workflow Purpose
Primary Goal: Publish PyTorch Ecosystem libraries binaries to the official download server Distribution Channel: download.pytorch.org Package Types: Wheels and libtorch archives
To Perform the release of Ecosystem library to the download.pytorch.org follow these steps:
- Create and Land a PR increasing version of a package you would like to promote: https://github.com/pytorch/test-infra/pull/7361
- Important! Validation Step: Run the https://github.com/pytorch/test-infra/blob/main/.github/workflows/release-download-pytorch-org.yml with dryrun enabled, make sure to select correct Ecosystem library. Inspect the log of the workflow and make sure you see correct packages being listed for promotion during
Promote library to download.pytorch.orgstep. For example:
+ aws s3 cp --dryrun --acl public-read --recursive --metadata-directive COPY --exclude '*' --include '*torch_tensorrt-2.9.0*' s3://pytorch/whl/test s3://pytorch/whl/
(dryrun) copy: s3://pytorch/whl/test/cu126/torch_tensorrt-2.9.0+cu126-cp310-cp310-manylinux_2_28_aarch64.whl to s3://pytorch/whl/cu126/torch_tensorrt-2.9.0+cu126-cp310-cp310-manylinux_2_28_aarch64.whl
(dryrun) copy: s3://pytorch/whl/test/cu126/torch_tensorrt-2.9.0+cu126-cp310-cp310-manylinux_2_28_aarch64.whl.metadata to s3://pytorch/whl/cu126/torch_tensorrt-2.9.0+cu126-cp310-cp310-manylinux_2_28_aarch64.whl.metadata
- Run the promotion with dry-run disabled: https://github.com/pytorch/test-infra/blob/main/.github/workflows/release-download-pytorch-org.yml
- Wait for next run of update s3 (run on hourly basis) workflow to complete. Or manually start this workflow: https://github.com/pytorch/test-infra/blob/main/.github/workflows/update-s3-html.yml
- Validate the new package is visible via download.pytorch.org by navigating to package url and inspecting for example:
https://download.pytorch.org/whl/cu126/torch-tensorrt/ - Test that new package is installable via download.pytorch.org
--index-url https://download.pytorch.org/whl/cu126
There are 2 workflows responsible for pypi updates. Staging workflow - This workflow manages the staging process for PyPI releases by publishing packages to pytorch-backup s3 and prepare these packages, stripping package suffixes, for example +cpu,+cu130 etc...
Release workflow: This workflow handles the publication of PyTorch packages to the Python Package Index (PyPI), making them installable via pip install torch.
Requirement enable pypi promotion configure Trusted publisher for your project, with following settings: https://docs.pypi.org/trusted-publishers/ Repository: pytorch/test-infra Workflow: release-pypi.yml Environment name: promote-env
To Perform the release of Ecosystem library to pypi follow these steps:
- Create and Land PR increasing version number of package as well as specifying builds that need to be promoted, for example: https://github.com/pytorch/test-infra/pull/7360
- Run Staging worklfow with dry-run enabled https://github.com/pytorch/test-infra/blob/main/.github/workflows/release-stage-pypi.yml
- Make sure your required packages in the workflow log for example:
+ aws s3 cp --dryrun executorch-1.0.0-cp310-cp310-macosx_12_0_arm64.whl s3://pytorch-backup/executorch-1.0.0-pypi-staging/
(dryrun) upload: ./executorch-1.0.0-cp310-cp310-macosx_12_0_arm64.whl to s3://pytorch-backup/executorch-1.0.0-pypi-staging/executorch-1.0.0-cp310-cp310-macosx_12_0_arm64.whl
- Run Staging worklfow with dry-run disabled https://github.com/pytorch/test-infra/blob/main/.github/workflows/release-stage-pypi.yml
- Before running promotion to pypi its advised to reach out to member of
MSL Infra PyTorch team - Run Promotion with dry-run enabled https://github.com/pytorch/test-infra/blob/main/.github/workflows/release-pypi.yml
- Make sure you see only the packages that should be uploaded to pypi in the Copy staged binaries step:
download: s3://pytorch-backup/executorch-1.0.0-pypi-staging/executorch-1.0.0-cp312-cp312-manylinux_2_28_x86_64.whl to dist/executorch-1.0.0-cp312-cp312-manylinux_2_28_x86_64.whl
- Run Promotion with dry-run disabled https://github.com/pytorch/test-infra/blob/main/.github/workflows/release-pypi.yml Make sure Publish Package to pypi step completes successfully
- Validate that the package has been pubished to pypi: https://pypi.org/
- Validate that install via pypi works for all published packages