Browse Source

feat(ci): update ci and readme

tags/v1.9.0
huangxinda 4 years ago
parent
commit
ce9ad07a27
5 changed files with 30 additions and 15 deletions
  1. +8
    -4
      .github/workflows/ci.yml
  2. +1
    -1
      README.md
  3. +3
    -3
      README_CN.md
  4. +2
    -1
      ci/cmake.sh
  5. +16
    -6
      ci/docker_env/Dockerfile

+ 8
- 4
.github/workflows/ci.yml View File

@@ -23,12 +23,13 @@ jobs:
runs-on: self-hosted
needs: [check-commit]
container:
image: localhost:5000/megengine-ci:latest
image: localhost:5000/megengine-ci:v1
steps:
- name: Checkout MegEngine
uses: actions/checkout@v2
- name: Checkout submodules
run: |
apt update&&apt install ninja-build
./third_party/prepare.sh
./third_party/install-mkl.sh
- name: Build MegEngine
@@ -46,9 +47,9 @@ jobs:
runs-on: self-hosted
needs: [check-commit]
container:
image: localhost:5000/megengine-ci:latest
image: localhost:5000/megengine-ci:v1
volumes:
- /usr/local/cuda-10.1-libs:/usr/local/cuda-10.1-libs
- /usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs
options: --gpus all --shm-size 1g
env:
NCCL_LAUNCH_MODE: PARALLEL
@@ -57,6 +58,7 @@ jobs:
uses: actions/checkout@v2
- name: Checkout submodules
run: |
apt update&&apt install ninja-build
./third_party/prepare.sh
./third_party/install-mkl.sh
- name: Build MegEngine
@@ -72,8 +74,10 @@ jobs:
run: ./ci/run_cpp_test.sh cuda
auto-merge:
if: ${{ github.ref == 'refs/heads/try-import' }}
runs-on: ubuntu-latest
runs-on: self-hosted
needs: [cpu-test, gpu-test]
container:
image: localhost:5000/megengine-ci:v1
steps:
- name: Checkout MegEngine
uses: actions/checkout@v2


+ 1
- 1
README.md View File

@@ -12,7 +12,7 @@ MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto

## Installation

**NOTE:** MegEngine now supports Python installation on Linux-64bit/Windows-64bit/MacOS(CPU-Only)-10.14+ platforms with Python from 3.5 to 3.8. On Windows 10 you can either install the Linux distribution through [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) or install the Windows distribution directly. Many other platforms are supported for inference.
**NOTE:** MegEngine now supports Python installation on Linux-64bit/Windows-64bit/MacOS(CPU-Only)-10.14+/Android 7+(CPU-Only) platforms with Python from 3.5 to 3.8. On Windows 10 you can either install the Linux distribution through [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) or install the Windows distribution directly. Many other platforms are supported for inference.

### Binaries



+ 3
- 3
README_CN.md View File

@@ -13,7 +13,7 @@ MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深

## 安装说明

**注意:** MegEngine 现在支持在 Linux-64bit/Windows-64bit/macos-10.14及其以上 (MacOS只支持cpu) 等平台上安装 Python 包,支持Python3.5 到 Python3.8。对于 Windows 10 用户,可以通过安装 [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) 进行体验,同时我们也原生支持Windows。MegEngine 也支持在很多其它平台上进行推理运算。
**注意:** MegEngine 现在支持在 Linux-64bit/Windows-64bit/macos-10.14/Android 7+ 及其以上 (MacOS/Android只支持cpu) 等平台上安装 Python 包,支持Python3.5 到 Python3.8。对于 Windows 10 用户,可以通过安装 [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl) 进行体验,同时我们也原生支持Windows。MegEngine 也支持在很多其它平台上进行推理运算。

### 通过包管理器安装

@@ -26,8 +26,8 @@ python3 -m pip install megengine -f https://megengine.org.cn/whl/mge.html

## 通过源码编译安装

* CMake编译细节请参考 [BUILD_README.md](scripts/cmake-build/BUILD_README.md)
* Python绑定编译细节请参考 [BUILD_PYTHON_WHL_README.md](scripts/whl/BUILD_PYTHON_WHL_README.md)
* CMake 编译细节请参考 [BUILD_README.md](scripts/cmake-build/BUILD_README.md)
* Python 绑定编译细节请参考 [BUILD_PYTHON_WHL_README.md](scripts/whl/BUILD_PYTHON_WHL_README.md)

## 如何参与贡献



+ 2
- 1
ci/cmake.sh View File

@@ -27,7 +27,8 @@ function build() {
-DMGE_WITH_DISTRIBUTED=${DMGE_WITH_DISTRIBUTED} \
-DMGE_WITH_CUDA=${DMGE_WITH_CUDA} \
-DMGE_WITH_TEST=ON \
-DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DMGE_WITH_CUSTOM_OP=ON
make -j$(($(nproc) * 2)) -I ${build_dir}
make develop
popd >/dev/null


+ 16
- 6
ci/docker_env/Dockerfile View File

@@ -24,17 +24,27 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
swig \
vim \
wget \
libgl1-mesa-glx \
libsm6 \
libxext6 \
zlib1g-dev \
# GitLab Runner need Git 2.18 or higher to create a local Git repository
&& add-apt-repository ppa:git-core/ppa -y && apt-get install --no-install-recommends -y git \
&& rm -rf /var/lib/apt/lists/*

RUN cd /tmp ; wget https://cmake.org/files/v3.14/cmake-3.14.4.tar.gz;tar -xzvf cmake-3.14.4.tar.gz;cd cmake-3.14.4;./configure; make -j32; make install
RUN cd /tmp ; wget https://cmake.org/files/v3.15/cmake-3.15.2.tar.gz;tar -xzvf cmake-3.15.2.tar.gz;cd cmake-3.15.2;./configure; make -j32; make install

RUN git lfs install

ENV PATH=${PATH}:/usr/local/cuda/bin \
LIBRARY_PATH=${LIBRARY_PATH}:/usr/local/cuda/lib:/usr/local/cuda/lib64:/usr/local/cuda/lib/stubs:/usr/local/cuda/lib64/stubs:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib64:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib64 \
LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/lib64:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/lib64:/tmp/build/cuda/dnn/cuda-stub/libcuda.so \
CPATH=${CPATH}:/usr/local/cuda/include:/usr/local/cuda-10.1-libs/cudnn-v7.6.0/include:/usr/local/cuda-10.1-libs/TensorRT-5.1.5.0/include \
CUDA_BIN_PATH=/usr/local/cuda
RUN pip3 install --upgrade pip

# TODO: set following envs in github environment.
ENV CUDA_ROOT_DIR=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1 \
TRT_ROOT_DIR=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5 \
TENSORRT_ROOT_DIR=${TRT_ROOT_DIR} \
CUDNN_ROOT_DIR=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3 \
PATH=/usr/bin:${CUDA_ROOT_DIR}/bin:${CUDA_ROOT_DIR}/nsight-compute-2019.4.0:$PATH \
LIBRARY_PATH=${LIBRARY_PATH}:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib/stubs:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/lib64/stubs:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib64 \
LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/lib64:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/lib64 \
CPATH=${CPATH}:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1/include:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cudnn-v7.6.3/include:/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/TensorRT-6.0.1.5/include \
CUDA_BIN_PATH=/usr/local/cuda-10.1-cudnn-7.6.3-trt-6.0.1.5-libs/cuda-10.1

Loading…
Cancel
Save