Skip to content

fix install doc and yaml path #843

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 48 additions & 22 deletions docs/en/tutorials/install_en.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,48 @@
# Installation

---
This tutorial introduces how to install PaddleClas and its requirements.

## Introduction
## 1. Install PaddlePaddle

This document introduces how to install PaddleClas and its requirements.
`PaddlePaddle 2.0` or later is required for PaddleClas. You can use the following steps to install PaddlePaddle.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.0我记得是不是有点问题,这里是不是要写的更细一些,比如2.0.2


## Install PaddlePaddle
### 1.1 Environment requirements

Python 3.x, CUDA 10.0, CUDNN7.6.4 nccl2.1.2 and later version are required at first, For now, PaddleClas only support training on the GPU device. Please follow the instructions in the [Installation](http://www.paddlepaddle.org.cn/install/quick) if the PaddlePaddle on the device is lower than 2.0.0.
- python 3.x
- cuda >= 10.1 (necessary if you want to use paddlepaddle-gpu)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们现在对显卡驱动版本是不是也有限制?是要大于440还是多少来着?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已补充相关信息

- cudnn >= 7.6.4 (necessary if you want to use paddlepaddle-gpu)
- nccl >= 2.1.2 (necessary if you want the use distributed training/eval)
- gcc 8.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是正好是8.2还是要高于,gcc版本是个不太好调的东西,如果高版本可用,最好标>=吧

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的,我改成大于等于,目前paddle提供的安装包和docker都是要求gcc82或者以上了


Docker is recomended to run Paddleclas, for more detailed information about docker and nvidia-docker, you can refer to the [tutorial](https://www.runoob.com/docker/docker-tutorial.html).

### Install PaddlePaddle using pip
If you do not want to use docker, you can skip section 1.2 and go into section 1.3 directly.


### 1.2 (Recommended) Prepare a docker environment. The first time you use this docker image, it will be downloaded automatically. Please be patient.


```
# Switch to the working directory
cd /home/Projects
# You need to create a docker container for the first run, and do not need to run the current command when you run it again
# Create a docker container named ppcls and map the current directory to the /paddle directory of the container
# It is recommended to set a shared memory greater than or equal to 8G through the --shm-size parameter
sudo docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it docker pull paddlepaddle/paddle:2.1.0 /bin/bash

# Use the following command to create a container if you want to use GPU in the container
sudo nvidia-docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it docker pull paddlepaddle/paddle:2.1.0-gpu-cuda10.2-cudnn7 /bin/bash
```

You can also visit [DockerHub](https://hub.docker.com/r/paddlepaddle/paddle/tags/) to get more docker images.

```
# use ctrl+P+Q to exit docker, to re-enter docker using the following command:
sudo docker container exec -it ppcls /bin/bash
```

### 1.3 Install PaddlePaddle using pip

If you want to use PaddlePaddle on GPU, you can use the following command to install PaddlePaddle.

Expand All @@ -25,11 +56,11 @@ If you want to use PaddlePaddle on CPU, you can use the following command to ins
pip3 install paddlepaddle --upgrade -i https://mirror.baidu.com/pypi/simple
```

### Install PaddlePaddle from source code

You can also compile PaddlePaddle from source code, please refer to [Installation](http://www.paddlepaddle.org.cn/install/quick).
**Note:**
* If you have already installed CPU version of PaddlePaddle and want to use GPU version now, you should uninstall CPU version of PaddlePaddle and then install GPU version to avoid package confusion.
* You can also compile PaddlePaddle from source code, please refer to [PaddlePaddle Installation tutorial](http://www.paddlepaddle.org.cn/install/quick) to more compilation options.

Verify Installation
### 1.4 Verify Installation process

```python
import paddle
Expand All @@ -43,14 +74,15 @@ python3 -c "import paddle; print(paddle.__version__)"
```

Note:
- Make sure the compiled version is later than PaddlePaddle2.0.
- Make sure the compiled source code is later than PaddlePaddle2.0.
- Indicate **WITH_DISTRIBUTE=ON** when compiling, Please refer to [Instruction](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#id3) for more details.
- When running in docker, in order to ensure that the container has enough shared memory for data read acceleration of Paddle, please set the parameter `--shm_size=8g` at creating a docker container, if conditions permit, you can set it to a larger value.
- When running in docker, in order to ensure that the container has enough shared memory for dataloader acceleration of Paddle, please set the parameter `--shm_size=8g` at creating a docker container, if conditions permit, you can set it to a larger value.



## Install PaddleClas
## 2. Install PaddleClas

**Clone PaddleClas: **
### 2.1 Clone PaddleClas source code

```
git clone https://github.com/PaddlePaddle/PaddleClas.git -b develop
Expand All @@ -62,16 +94,10 @@ If it is too slow for you to download from github, you can download PaddleClas f
git clone https://gitee.com/paddlepaddle/PaddleClas.git -b develop
```

**Install requirements**
### 2.2 Install requirements

```
pip3 install --upgrade -r requirements.txt -i https://mirror.baidu.com/pypi/simple
```

If the install process of visualdl failed, you can try the following commands.
PaddleClas dependencies are listed in file `requirements.txt`, you can use the following command to install the dependencies.

```
pip3 install --upgrade visualdl -i https://mirror.baidu.com/pypi/simple
pip3 install --upgrade -r requirements.txt -i https://mirror.baidu.com/pypi/simple
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是区分国内和国外用户?国内用户才建议使用baidu的源

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

国外用户应该也是可以用百度源的,显式地写出来,可以更换


What's more, visualdl is just supported in python3, so python3 is needed if you want to use visualdl.
75 changes: 47 additions & 28 deletions docs/zh_CN/tutorials/install.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,52 @@
# 安装说明

---
本章将介绍如何安装PaddleClas及其依赖项。

## 一、简介

本章将介绍如何安装PaddleClas及其依赖项。
## 1. 安装PaddlePaddle

运行PaddleClas需要`PaddlePaddle 2.0`或更高版本。可以参考下面的步骤安装PaddlePaddle。

### 1.1 环境要求

- python 3.x
- cuda >= 10.1 (如果使用paddlepaddle-gpu)
- cudnn >= 7.6.4 (如果使用paddlepaddle-gpu)
- nccl >= 2.1.2 (如果使用分布式训练/评估)
- gcc 8.2

## 二、安装PaddlePaddle
建议使用我们提供的docker运行PaddleClas,有关docker、nvidia-docker使用请参考[链接](https://www.runoob.com/docker/docker-tutorial.html)。

运行PaddleClas需要`PaddlePaddle 2.0`或更高版本。请参照[安装文档](http://www.paddlepaddle.org.cn/install/quick)中的说明进行操作
如果不使用docker,可以直接跳过1.2部分内容,从1.3部分开始执行

### 通过pip安装PaddlePaddle

如果已经安装好了cuda、cudnn、nccl或者安装好了docker、nvidia-docker运行环境,可以pip安装最新GPU版本PaddlePaddle
### 1.2 (建议)准备docker环境。第一次使用这个镜像,会自动下载该镜像,请耐心等待。

```
# 切换到工作目录下
cd /home/Projects
# 首次运行需创建一个docker容器,再次运行时不需要运行当前命令
# 创建一个名字为ppcls的docker容器,并将当前目录映射到容器的/paddle目录下

如果您希望在CPU环境下使用docker,使用docker而不是nvidia-docker创建docker,设置docker容器共享内存shm-size为8G,建议设置8G以上
sudo docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it docker pull paddlepaddle/paddle:2.1.0 /bin/bash

如果希望使用GPU版本的容器,请运行以下命令创建容器。
sudo nvidia-docker run --name ppcls -v $PWD:/paddle --shm-size=8G --network=host -it docker pull paddlepaddle/paddle:2.1.0-gpu-cuda10.2-cudnn7 /bin/bash
```


您也可以访问[DockerHub](https://hub.docker.com/r/paddlepaddle/paddle/tags/)获取与您机器适配的镜像。

```
# ctrl+P+Q可退出docker 容器,重新进入docker 容器使用如下命令
sudo docker container exec -it ppcls /bin/bash
```

### 1.3 通过pip安装PaddlePaddle

运行下面的命令,通过pip安装最新GPU版本PaddlePaddle

```bash
pip3 install paddlepaddle-gpu --upgrade -i https://mirror.baidu.com/pypi/simple
Expand All @@ -25,10 +58,12 @@ pip3 install paddlepaddle-gpu --upgrade -i https://mirror.baidu.com/pypi/simple
pip3 install paddlepaddle --upgrade -i https://mirror.baidu.com/pypi/simple
```

### 源码编译PaddlePaddle
**注意:**
* 如果先安装了CPU版本的paddlepaddle,之后想切换到GPU版本,那么需要首先卸载CPU版本的paddle,再安装GPU版本的paddle,否则容易导致使用的paddle版本混乱。
* 您也可以从源码编译安装PaddlePaddle,请参照[PaddlePaddle 安装文档](http://www.paddlepaddle.org.cn/install/quick)中的说明进行操作。

您也可以从源码编译安装PaddlePaddle,请参照[安装文档](http://www.paddlepaddle.org.cn/install/quick)中的说明进行操作。

### 1.4 验证是否安装成功

使用以下命令可以验证PaddlePaddle是否安装成功。

Expand All @@ -47,18 +82,12 @@ python3 -c "import paddle; print(paddle.__version__)"
- 从源码编译的PaddlePaddle版本号为0.0.0,请确保使用了PaddlePaddle 2.0及之后的源码编译。
- PaddleClas基于PaddlePaddle高性能的分布式训练能力,若您从源码编译,请确保打开编译选项,**WITH_DISTRIBUTE=ON**。具体编译选项参考[编译选项表](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#id3)。
- 在docker中运行时,为保证docker容器有足够的共享内存用于Paddle的数据读取加速,在创建docker容器时,请设置参数`--shm_size=8g`,条件允许的话可以设置为更大的值。
-

**运行环境需求:**

- Python3
- CUDA >= 9.0
- cuDNN >= 7.6.4
- nccl >= 2.1.2
## 2. 安装PaddleClas


## 三、安装PaddleClas

**克隆PaddleClas模型库:**
### 2.1 克隆PaddleClas模型库

```bash
git clone https://github.com/PaddlePaddle/PaddleClas.git -b develop
Expand All @@ -70,20 +99,10 @@ git clone https://github.com/PaddlePaddle/PaddleClas.git -b develop
git clone https://gitee.com/paddlepaddle/PaddleClas.git -b develop
```



**安装Python依赖库:**
### 2.2 安装Python依赖库

Python依赖库在`requirements.txt`中给出,可通过如下命令安装:

```bash
pip3 install --upgrade -r requirements.txt -i https://mirror.baidu.com/pypi/simple
```

visualdl可能出现安装失败,请尝试

```bash
pip3 install --upgrade visualdl -i https://mirror.baidu.com/pypi/simple
```

此外,visualdl目前只支持在python3下运行,因此如果希望使用visualdl,需要使用python3。
8 changes: 4 additions & 4 deletions ppcls/configs/Vehicle/ResNet50.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,10 @@ DataLoader:
Train:
dataset:
name: "CompCars"
image_root: "/work/dataset/CompCars/image/"
label_root: "/work/dataset/CompCars/label/"
image_root: "./dataset/CompCars/image/"
label_root: "./dataset/CompCars/label/"
bbox_crop: True
cls_label_path: "/work/dataset/CompCars/train_test_split/classification/train_label.txt"
cls_label_path: "./dataset/CompCars/train_test_split/classification/train_label.txt"
transform_ops:
- DecodeImage:
to_rgb: True
Expand Down Expand Up @@ -106,7 +106,7 @@ DataLoader:
# TOTO: modify to the latest trainer
dataset:
name: "CompCars"
image_root: ".dataset/CompCars/image/"
image_root: "./dataset/CompCars/image/"
label_root: "./dataset/CompCars/label/"
cls_label_path: "./dataset/CompCars/train_test_split/classification/test_label.txt"
bbox_crop: True
Expand Down
12 changes: 6 additions & 6 deletions ppcls/configs/Vehicle/ResNet50_ReID.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ DataLoader:
Train:
dataset:
name: "VeriWild"
image_root: "/work/dataset/VeRI-Wild/images/"
cls_label_path: "/work/dataset/VeRI-Wild/train_test_split/train_list_start0.txt"
image_root: "./dataset/VeRI-Wild/images/"
cls_label_path: "./dataset/VeRI-Wild/train_test_split/train_list_start0.txt"
transform_ops:
- DecodeImage:
to_rgb: True
Expand Down Expand Up @@ -104,8 +104,8 @@ DataLoader:
# TOTO: modify to the latest trainer
dataset:
name: "VeriWild"
image_root: "/work/dataset/VeRI-Wild/images"
cls_label_path: "/work/dataset/VeRI-Wild/train_test_split/test_3000_id_query.txt"
image_root: "./dataset/VeRI-Wild/images"
cls_label_path: "./dataset/VeRI-Wild/train_test_split/test_3000_id_query.txt"
transform_ops:
- DecodeImage:
to_rgb: True
Expand All @@ -130,8 +130,8 @@ DataLoader:
# TOTO: modify to the latest trainer
dataset:
name: "VeriWild"
image_root: "/work/dataset/VeRI-Wild/images"
cls_label_path: "/work/dataset/VeRI-Wild/train_test_split/test_3000_id.txt"
image_root: "./dataset/VeRI-Wild/images"
cls_label_path: "./dataset/VeRI-Wild/train_test_split/test_3000_id.txt"
transform_ops:
- DecodeImage:
to_rgb: True
Expand Down
2 changes: 1 addition & 1 deletion ppcls/utils/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ def str2num(v):
if len(ks) == 1:
# assert ks[0] in dl, ('{} is not exist in {}'.format(ks[0], dl))
if not ks[0] in dl:
logger.warning('A new filed ({}) detected!'.format(ks[0], dl))
print('A new filed ({}) detected!'.format(ks[0], dl))
dl[ks[0]] = str2num(v)
else:
override(dl[ks[0]], ks[1:], v)
Expand Down