Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleClas into slim

Author: RainFrost1

.github/ISSUE_TEMPLATE/---clas-issue-.md

@@ -0,0 +1,18 @@
+---
+name: 问题反馈
+about: PaddleClas问题反馈
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+欢迎您使用PaddleClas并反馈相关问题，非常感谢您对PaddleClas的贡献！
+提出issue时，辛苦您提供以下信息，方便我们快速定位问题并及时有效地解决您的问题：
+ 1. PaddleClas版本以及PaddlePaddle版本：请您提供您使用的版本号或分支信息，如PaddleClas release/2.2和PaddlePaddle 2.1.0
+ 2. 涉及的其他产品使用的版本号：如您在使用PaddleClas的同时还在使用其他产品，如PaddleServing、PaddleInference等，请您提供其版本号
+ 3. 训练环境信息：
+  a. 具体操作系统，如Linux/Windows/MacOS
+  b. Python版本号，如Python3.6/7/8
+  c. CUDA/cuDNN版本， 如CUDA10.2/cuDNN 7.6.5等
+ 4. 完整的代码(相比于repo中代码，有改动的地方)、详细的错误信息及相关log

README_ch.md

@@ -8,6 +8,7 @@
 
 **近期更新**
 
+- 2021.08.11 更新7个[FAQ](docs/zh_CN/faq_series/faq_2021_s2.md)。
 - 2021.06.29 添加Swin-transformer系列模型，ImageNet1k数据集上Top1 acc最高精度可达87.2%；支持训练预测评估与whl包部署，预训练模型可以从[这里](docs/zh_CN/models/models_intro.md)下载。
 - 2021.06.22,23,24 PaddleClas官方研发团队带来技术深入解读三日直播课。课程回放：[https://aistudio.baidu.com/aistudio/course/introduce/24519](https://aistudio.baidu.com/aistudio/course/introduce/24519)
 - 2021.06.16 PaddleClas v2.2版本升级，集成Metric learning，向量检索等组件。新增商品识别、动漫人物识别、车辆识别和logo识别等4个图像识别应用。新增LeViT、Twins、TNT、DLA、HarDNet、RedNet系列30个预训练模型。
@@ -50,6 +51,10 @@ Res2Net200_vd预训练模型Top-1精度高达85.1%。
 - [图像识别快速体验](./docs/zh_CN/tutorials/quick_start_recognition.md)
 - [图像识别系统介绍](#图像识别系统介绍)
 - [识别效果展示](#识别效果展示)
+- 图像分类快速体验
+    - [尝鲜版](./docs/zh_CN/tutorials/quick_start_new_user.md)
+    - [进阶版](./docs/zh_CN/tutorials/quick_start_professional.md)
+    - [社区版](./docs/zh_CN/tutorials/quick_start_community.md)
 - 算法介绍
     - [骨干网络和预训练模型库](./docs/zh_CN/ImageNet_models_cn.md)
     - [主体检测](./docs/zh_CN/application/mainbody_detection.md)
@@ -74,11 +79,14 @@ Res2Net200_vd预训练模型Top-1精度高达85.1%。
     - [知识蒸馏](./docs/zh_CN/advanced_tutorials/distillation/distillation.md)
     - [模型量化](./docs/zh_CN/extension/paddle_quantization.md)
     - [数据增广](./docs/zh_CN/advanced_tutorials/image_augmentation/ImageAugment.md)
-- FAQ(暂停更新)
+- FAQ
+    - [图像识别任务FAQ](docs/zh_CN/faq_series/faq_2021_s2.md)
     - [图像分类任务FAQ](docs/zh_CN/faq.md)
 - [许可证书](#许可证书)
 - [贡献代码](#贡献代码)
 
+<a name="图像识别系统介绍"></a>
+## 图像识别系统介绍
 
 <div align="center">
 <img src="./docs/images/structure.png"  width = "400" />

deploy/configs/build_cartoon.yaml

@@ -1,6 +1,6 @@
 Global:
   rec_inference_model_dir: "./models/cartoon_rec_ResNet50_iCartoon_v1.0_infer/"
-  batch_size: 1
+  batch_size: 32
   use_gpu: True
   enable_mkldnn: True
   cpu_num_threads: 10

deploy/configs/build_logo.yaml

@@ -1,6 +1,6 @@
 Global:
   rec_inference_model_dir: "./models/logo_rec_ResNet50_Logo3K_v1.0_infer/"
-  batch_size: 1
+  batch_size: 32
   use_gpu: True
   enable_mkldnn: True
   cpu_num_threads: 10

deploy/configs/build_product.yaml

@@ -1,6 +1,6 @@
 Global:
   rec_inference_model_dir: "./models/product_ResNet50_vd_aliproduct_v1.0_infer"
-  batch_size: 1
+  batch_size: 32
   use_gpu: True
   enable_mkldnn: True
   cpu_num_threads: 10

deploy/configs/build_vehicle.yaml

@@ -1,6 +1,6 @@
 Global:
   rec_inference_model_dir: "./models/vehicle_cls_ResNet50_CompCars_v1.0_infer/"
-  batch_size: 1
+  batch_size: 32
   use_gpu: True
   enable_mkldnn: True
   cpu_num_threads: 10

deploy/python/predict_cls.py

@@ -41,6 +41,29 @@ def __init__(self, config):
         if "PostProcess" in config:
             self.postprocess = build_postprocess(config["PostProcess"])
 
+        # for whole_chain project to test each repo of paddle
+        self.benchmark = config["Global"].get("benchmark", False)
+        if self.benchmark:
+            import auto_log
+            import os
+            pid = os.getpid()
+            self.auto_logger = auto_log.AutoLogger(
+                model_name=config["Global"].get("model_name", "cls"),
+                model_precision='fp16'
+                if config["Global"]["use_fp16"] else 'fp32',
+                batch_size=config["Global"].get("batch_size", 1),
+                data_shape=[3, 224, 224],
+                save_path=config["Global"].get("save_log_path",
+                                               "./auto_log.log"),
+                inference_config=self.config,
+                pids=pid,
+                process_name=None,
+                gpu_ids=None,
+                time_keys=[
+                    'preprocess_time', 'inference_time', 'postprocess_time'
+                ],
+                warmup=2)
+
     def predict(self, images):
         input_names = self.paddle_predictor.get_input_names()
         input_tensor = self.paddle_predictor.get_input_handle(input_names[0])
@@ -49,29 +72,67 @@ def predict(self, images):
         output_tensor = self.paddle_predictor.get_output_handle(output_names[
             0])
 
+        if self.benchmark:
+            self.auto_logger.times.start()
         if not isinstance(images, (list, )):
             images = [images]
         for idx in range(len(images)):
             for ops in self.preprocess_ops:
                 images[idx] = ops(images[idx])
         image = np.array(images)
+        if self.benchmark:
+            self.auto_logger.times.stamp()
 
         input_tensor.copy_from_cpu(image)
         self.paddle_predictor.run()
         batch_output = output_tensor.copy_to_cpu()
+        if self.benchmark:
+            self.auto_logger.times.stamp()
+        if self.postprocess is not None:
+            batch_output = self.postprocess(batch_output)
+        if self.benchmark:
+            self.auto_logger.times.end(stamp=True)
         return batch_output
 
 
 def main(config):
     cls_predictor = ClsPredictor(config)
     image_list = get_image_list(config["Global"]["infer_imgs"])
 
-    assert config["Global"]["batch_size"] == 1
-    for idx, image_file in enumerate(image_list):
-        img = cv2.imread(image_file)[:, :, ::-1]
-        output = cls_predictor.predict(img)
-        output = cls_predictor.postprocess(output, [image_file])
-        print(output)
+    batch_imgs = []
+    batch_names = []
+    cnt = 0
+    for idx, img_path in enumerate(image_list):
+        img = cv2.imread(img_path)
+        if img is None:
+            logger.warning(
+                "Image file failed to read and has been skipped. The path: {}".
+                format(img_path))
+        else:
+            img = img[:, :, ::-1]
+            batch_imgs.append(img)
+            img_name = os.path.basename(img_path)
+            batch_names.append(img_name)
+            cnt += 1
+
+        if cnt % config["Global"]["batch_size"] == 0 or (idx + 1
+                                                         ) == len(image_list):
+            if len(batch_imgs) == 0:
+                continue
+
+            batch_results = cls_predictor.predict(batch_imgs)
+            for number, result_dict in enumerate(batch_results):
+                filename = batch_names[number]
+                clas_ids = result_dict["class_ids"]
+                scores_str = "[{}]".format(", ".join("{:.2f}".format(
+                    r) for r in result_dict["scores"]))
+                label_names = result_dict["label_names"]
+                print("{}:\tclass id(s): {}, score(s): {}, label_name(s): {}".
+                      format(filename, clas_ids, scores_str, label_names))
+            batch_imgs = []
+            batch_names = []
+    if cls_predictor.benchmark:
+        cls_predictor.auto_logger.report()
     return
 
 

deploy/python/predict_rec.py

@@ -54,27 +54,48 @@ def predict(self, images, feature_normalize=True):
         input_tensor.copy_from_cpu(image)
         self.paddle_predictor.run()
         batch_output = output_tensor.copy_to_cpu()
-        
+
         if feature_normalize:
             feas_norm = np.sqrt(
                 np.sum(np.square(batch_output), axis=1, keepdims=True))
             batch_output = np.divide(batch_output, feas_norm)
-            
+
+        if self.postprocess is not None:
+            batch_output = self.postprocess(batch_output)
         return batch_output
 
 
 def main(config):
     rec_predictor = RecPredictor(config)
     image_list = get_image_list(config["Global"]["infer_imgs"])
 
-    assert config["Global"]["batch_size"] == 1
-    for idx, image_file in enumerate(image_list):
-        batch_input = []
-        img = cv2.imread(image_file)[:, :, ::-1]
-        output = rec_predictor.predict(img)
-        if rec_predictor.postprocess is not None:
-            output = rec_predictor.postprocess(output)
-        print(output)
+    batch_imgs = []
+    batch_names = []
+    cnt = 0
+    for idx, img_path in enumerate(image_list):
+        img = cv2.imread(img_path)
+        if img is None:
+            logger.warning(
+                "Image file failed to read and has been skipped. The path: {}".
+                format(img_path))
+        else:
+            img = img[:, :, ::-1]
+            batch_imgs.append(img)
+            img_name = os.path.basename(img_path)
+            batch_names.append(img_name)
+            cnt += 1
+
+        if cnt % config["Global"]["batch_size"] == 0 or (idx + 1) == len(image_list):
+            if len(batch_imgs) == 0: 
+                continue
+                
+            batch_results = rec_predictor.predict(batch_imgs)
+            for number, result_dict in enumerate(batch_results):
+                filename = batch_names[number]
+                print("{}:\t{}".format(filename, result_dict))
+            batch_imgs = []
+            batch_names = []
+
     return
 
 

deploy/utils/predictor.py

@@ -28,7 +28,7 @@ def __init__(self, args, inference_model_dir=None):
         if args.use_fp16 is True:
             assert args.use_tensorrt is True
         self.args = args
-        self.paddle_predictor = self.create_paddle_predictor(
+        self.paddle_predictor, self.config = self.create_paddle_predictor(
             args, inference_model_dir)
 
     def predict(self, image):
@@ -59,11 +59,12 @@ def create_paddle_predictor(self, args, inference_model_dir=None):
             config.enable_tensorrt_engine(
                 precision_mode=Config.Precision.Half
                 if args.use_fp16 else Config.Precision.Float32,
-                max_batch_size=args.batch_size)
+                max_batch_size=args.batch_size,
+                min_subgraph_size=30)
 
         config.enable_memory_optim()
         # use zero copy
         config.switch_use_feed_fetch_ops(False)
         predictor = create_predictor(config)
 
-        return predictor
+        return predictor, config

deploy/vector_search/README_en.md

@@ -0,0 +1,95 @@
+# Vector search
+
+## 1. Introduction
+
+Some vertical domain recognition tasks (e.g., vehicles, commodities, etc.) require a large number of recognized categories, and often use a retrieval-based approach to obtain matching predicted categories by performing a fast nearest neighbor search with query vectors and underlying library vectors. The vector search module provides the basic approximate nearest neighbor search algorithm based on Baidu's self-developed Möbius algorithm, a graph-based approximate nearest neighbor search algorithm for maximum inner product search (MIPS). This module provides python interface, supports numpy and tensor type vectors, and supports L2 and Inner Product distance calculation.
+
+Details of the Mobius algorithm can be found in the paper.（[Möbius Transformation for Fast Inner Product Search on Graph](http://research.baidu.com/Public/uploads/5e189d36b5cf6.PDF), [Code](https://github.com/sunbelbd/mobius)）
+
+## 2. Installation
+
+### 2.1 Use the provided library files directly
+
+This folder contains the compiled `index.so` (compiled under gcc8.2.0 for Linux) and `index.dll` (compiled under gcc10.3.0 for Windows), which can be used directly, skipping sections 2.2 and 2.3.
+
+If the library files are not available due to a low gcc version or an incompatible environment, you need to manually compile the library files under a different platform.
+
+**Note：** Make sure that C++ compiler supports the C++11 standard.
+
+### 2.2 Compile and generate library files on Linux
+
+Run the following command to install gcc and g++.
+
+```
+sudo apt-get update
+sudo apt-get upgrade -y
+sudo apt-get install build-essential gcc g++
+```
+
+Check the gcc version by the command `gcc -v`.
+
+`make` can be operated directly. If you wish to regenerate the `index.so`, you can first use `make clean` to clear the cache, and then use `make` to generate the updated library file.
+
+### 2.3 Compile and generate library files on Windows
+
+You need to install gcc compiler tool first, we recommend using [TDM-GCC](https://jmeubank.github.io/tdm-gcc/articles/2020-03/9.2.0-release), you can choose the right version on the official website. We recommend downloading [tdm64-gcc-10.3.0-2.exe](https://github.com/jmeubank/tdm-gcc/releases/download/v10.3.0-tdm64-2/tdm64-gcc-10.3.0-2.exe).
+
+After the downloading, follow the default installation steps to install. There are 3 points to note here:
+
+1.  The vector search module depends on openmp, so you need to check the `openmp` installation option when going on to `choose components` step, otherwise it will report an error `libgomp.spec: No such file or directory`, [reference link](https://github.com/dmlc/xgboost/issues/1027)
+2.  When being asked whether to add to the system environment variables, it is recommended to check here, otherwise you need to add the system environment variables manually later.
+3. The compile command is `make` on Linux and `mingw32-make` on Windows, so you need to distinguish here.
+
+After installation, you can open a command line terminal and check the gcc version with the command `gcc -v`.
+
+Run the command `mingw32-make` to generate the `index.dll` library file under the folder (deploy/vector_search). If you want to regenerate the `index.dll` file, you can first use `mingw32-make clean` to clear the cache, and then use `mingw32-make` to generate the updated library file.
+
+### 2.4 Compile and generate library files on MacOS
+
+Run the following command to install gcc and g++:
+
+```
+brew install gcc
+```
+
+#### Caution：
+
+1. If prompted with `Error: Running Homebrew as root is extremely dangerous and no longer supported... `, refer to this [link](https://jingyan.baidu.com/article/e52e3615057a2840c60c519c.html)
+2.  If prompted with `Error: Failure while executing; tar --extract --no-same-owner --file... `, refer to this [link](https://blog.csdn.net/Dawn510/article/details/117787358).
+
+After installation the compiled executable is copied under /usr/local/bin, look at the gcc in this folder: 
+
+```
+ls /usr/local/bin/gcc*
+```
+
+The local gcc version is gcc-11, and the compile command is as follows: (If the local gcc version is gcc-9, the corresponding command should be `CXX=g++-9 make`)
+
+```
+CXX=g++-11 make
+```
+
+## 3. Quick use
+
+```
+import numpy as np
+from interface import Graph_Index
+
+# Random sample generation
+index_vectors = np.random.rand(100000,128).astype(np.float32)
+query_vector = np.random.rand(128).astype(np.float32)
+index_docs = ["ID_"+str(i) for i in range(100000)]
+
+# Initialize index structure
+indexer = Graph_Index(dist_type="IP") #support "IP" and "L2"
+indexer.build(gallery_vectors=index_vectors, gallery_docs=index_docs, pq_size=100, index_path='test')
+
+# Query
+scores, docs = indexer.search(query=query_vector, return_k=10, search_budget=100)
+print(scores)
+print(docs)
+
+# Save and load
+indexer.dump(index_path="test")
+indexer.load(index_path="test")
+```