Skip to content

Commit b4c81d1

Browse files
Hanbin HuBichengYing
Hanbin Hu
and
BichengYing
authored
Release flow (#93)
* Scripts for wrapping examples * Create release.yml * Update wrap_examples.sh * New flow test * Change back * Change trigger * Change tag_name * Try draft and release * Change to publish tag * PyPI flow added. * Add token for test_pypi * Combine pypi flow with release * Add graft for flatbuffers * Checkout code for pypi release * Disable heartbeat by default (#89) * Update the docker release flow * Updated Docker release flow * Update docker release flow * Update version for docker test * Release Flow clean up for PyPI * Change to commit to official PYPI site Co-authored-by: ybc <bichengying@gmail.com>
1 parent 31b775a commit b4c81d1

File tree

6 files changed

+111
-6
lines changed

6 files changed

+111
-6
lines changed

‎.github/workflows/release.yml

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
on:
2+
push:
3+
tags:
4+
- 'v*'
5+
6+
name: Release
7+
8+
jobs:
9+
build:
10+
name: Github Release
11+
runs-on: ubuntu-latest
12+
steps:
13+
- name: Checkout code
14+
uses: actions/checkout@v2
15+
- name: Wrap the examples
16+
run: |
17+
bash scripts/wrap_examples.sh
18+
- name: Create Release
19+
id: create_release
20+
uses: actions/create-release@v1
21+
env:
22+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
23+
with:
24+
tag_name: ${{ github.ref }}
25+
release_name: ${{ github.ref }}
26+
draft: false
27+
prerelease: false
28+
- name: Upload Release Asset
29+
id: upload-release-asset
30+
uses: actions/upload-release-asset@v1
31+
env:
32+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
33+
with:
34+
upload_url: ${{ steps.create_release.outputs.upload_url }}
35+
asset_path: ./examples.tar.gz
36+
asset_name: examples.tar.gz
37+
asset_content_type: application/tar+gzip
38+
deploy:
39+
name: PyPI Release
40+
runs-on: ubuntu-latest
41+
needs: build
42+
steps:
43+
- name: Checkout code
44+
uses: actions/checkout@v2
45+
with:
46+
submodules: recursive
47+
- name: Set up Python
48+
uses: actions/setup-python@v2
49+
with:
50+
python-version: 3.7
51+
- name: Install dependencies
52+
run: |
53+
python -m pip install --upgrade pip
54+
pip install setuptools wheel twine
55+
- name: Build and publish
56+
env:
57+
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
58+
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
59+
run: |
60+
python setup.py sdist
61+
twine upload dist/*

‎MANIFEST.in

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@ include README.rst LICENSE requirements.txt
44
prune .eggs
55

66
graft third_party/boost
7+
graft third_party/flatbuffers

‎dockerfile.cpu

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,8 +68,9 @@ RUN pip install bluefog
6868

6969
RUN mkdir /bluefog
7070
RUN cd /bluefog && \
71-
wget https://github.com/Bluefog-Lib/bluefog/releases/download/v0.2.2/examples_v0.2.2.tar.gz && \
72-
tar -zxv -f examples_v0.2.2.tar.gz && \
73-
rm examples_v0.2.2.tar.gz
71+
VERSION=`python -c "import bluefog; print(bluefog.__version__)"` && \
72+
wget https://github.com/Bluefog-Lib/bluefog/releases/download/v$VERSION/examples.tar.gz && \
73+
tar -zxv -f examples.tar.gz && \
74+
rm examples.tar.gz
7475

7576
WORKDIR "bluefog"

‎dockerfile.gpu

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,9 @@ RUN pip install bluefog
7878

7979
RUN mkdir /bluefog
8080
RUN cd /bluefog && \
81-
wget https://github.com/Bluefog-Lib/bluefog/releases/download/v0.2.2/examples_v0.2.2.tar.gz && \
82-
tar -zxv -f examples_v0.2.2.tar.gz && \
83-
rm examples_v0.2.2.tar.gz
81+
VERSION=`python -c "import bluefog; print(bluefog.__version__)"` && \
82+
wget https://github.com/Bluefog-Lib/bluefog/releases/download/v$VERSION/examples.tar.gz && \
83+
tar -zxv -f examples.tar.gz && \
84+
rm examples.tar.gz
8485

8586
WORKDIR "bluefog"

‎scripts/run_unittest.sh

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/bin/bash
2+
# Assume running under the root directory of Bluefog with `test` folder
3+
NUM_PROC=4
4+
MPIRUN="mpirun -np $NUM_PROC --allow-run-as-root"
5+
PYTEST="pytest -s -vv"
6+
7+
$PYTEST ./test/torch_basics_test.py
8+
$MPIRUN $PYTEST ./test/torch_basics_test.py
9+
$MPIRUN $PYTEST ./test/torch_ops_test.py
10+
$MPIRUN $PYTEST ./test/torch_win_ops_test.py

‎scripts/wrap_examples.sh

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
#!/bin/bash
2+
# Assume running under the root directory of Bluefog
3+
set -e
4+
set -x
5+
6+
RAND_ID=$(git log -1 --format='%H' | fold -w 16 | head -n 1)
7+
TMP_DIR=/tmp/bluefog_${RAND_ID}
8+
TMP_EXAMPLE_DIR=${TMP_DIR}/examples
9+
TMP_TEST_DIR=${TMP_DIR}/test
10+
11+
mkdir ${TMP_DIR}
12+
mkdir ${TMP_EXAMPLE_DIR}
13+
mkdir ${TMP_TEST_DIR}
14+
15+
cp scripts/run_unittest.sh ${TMP_DIR}
16+
17+
cp examples/*.py ${TMP_EXAMPLE_DIR}
18+
cp examples/*.ipynb ${TMP_EXAMPLE_DIR}
19+
20+
cp test/pytest.ini ${TMP_TEST_DIR}
21+
cp test/common.py ${TMP_TEST_DIR}
22+
cp test/torch_basics_test.py ${TMP_TEST_DIR}
23+
cp test/torch_ops_test.py ${TMP_TEST_DIR}
24+
cp test/torch_win_ops_test.py ${TMP_TEST_DIR}
25+
26+
cd ${TMP_DIR}
27+
tar -czf examples.tar.gz *
28+
cd -
29+
cp ${TMP_DIR}/examples.tar.gz .
30+
31+
rm -r ${TMP_DIR}

0 commit comments

Comments
 (0)