Skip to content

Timesformer reader align #171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

HydrogenSulfate
Copy link
Collaborator

@HydrogenSulfate HydrogenSulfate commented Jul 5, 2021

  1. add TimeSformer's performance in README.md and README_cn.md
  2. add TimeSformer's config file in configs/recognition/timesformer/timesformer_k400_videos.yaml
  3. add TimeSformer's en doc and Zh-CN doc
  4. add TimeSformer's two image about main arch and attention arch
  5. add UniformCrop in paddlevideo/loader/pipelines/augmentations.py, and corresponding UniformCropMetric in paddlevideo/metrics/uniform_crop_metric.py
  6. add pyav's video decode method in paddlevideo/loader/pipelines/decode.py
  7. add linspace_sample strategy in paddlevideo/loader/pipelines/sample.py
  8. add VisionTransformer in paddlevideo/modeling/backbones/vit.py
  9. update the different forward method in paddlevideo/modeling/framework/recognizers/recognizer2d.py while using TimeSformer and ViT based model
  10. delete coloring and AverageMeter two unused import statement in paddlevideo/tasks/train.py
  11. add an new training strategy gradient accumulation for TimeSformer and follow-up models, if used, in paddlevideo/tasks/train.py
  12. fixed a bug when record the learning_rate in training process, when batch_size=1.
  13. update the load_ckpt function for TimeSformer and follow-up ViT based models
  14. update run.sh for TimeSformer
  15. add a new InputSpec in tools/export_model.py for TimeSformer, even though there still remain some todo for inference model
  16. add TimeSformer_Inference_helper class in tools/utils.py
  17. add inference for timesformer and corresponding content in docs
  18. fixed some hyper-link bugs in start.md
  19. Update the description of TenCrop.
  20. Change the order of data augmentation to align reader.
  21. Add a new style of augmentation(almost in tensor mode), when backend is pyav
  22. update utils.summary to fix some bugs where FLOPs could be negtive
  23. Add trunc_normal_ function
  24. Fix head/base.py line 108's bug

def load_ckpt(model, weight_path):
def load_ckpt(model,
weight_path,
num_patches=None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

后面这三个参数用 **kargs,load接口里出现这三个参数有点奇怪

print(params_info)

if args.FLOPs:
flops_info = paddle.flops(model, [1, num_seg, 3, img_size, img_size],
print_detail=True)
flops_info = paddleslim.analysis.flops(model, [1, 1, num_seg, 3, img_size, img_size])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

paddle.flops接口跑不通吗?

@huangjun12 huangjun12 merged commit e0aae6e into PaddlePaddle:develop Jul 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants