-
Notifications
You must be signed in to change notification settings - Fork 798
Closed as not planned
Labels
Description
Describe the bug
When training anomalib AiVad ( with a video dataset ) it stops on the validation/testing.
With the next script:
from anomalib.data import Avenue, UCSDped, ShanghaiTech
from anomalib.models import AiVad
from anomalib.engine import Engine
# Load the avenue dataset, model and engine.
datamodule = Avenue(train_batch_size=8, eval_batch_size=8, num_workers=2)
datamodule.prepare_data()
model = AiVad()
engine = Engine()
# Train the model
engine.train(model=model, datamodule=datamodule)
Dataset
Avenue
Model
Other (please specify in the field below)
Steps to reproduce the behavior
yes | conda create -n anomalib_env python=3.10
conda activate anomalib_env
git clone https://github.com/openvinotoolkit/anomalib.git
cd anomalib
pip install -e .
anomalib install -v
OS information
OS information:
- AWS: g4dn.xlarge
- OS: "centos rhel fedora" / "Amazon Linux 2"
- Python version: 3.10.14
- Anomalib version: from source 1.1.0
- pytorch-lightning: 2.2.1
Expected behavior
Continue the validation and testing.
Screenshots
No response
Pip/GitHub
GitHub
What version/branch did you use?
latest/main
Configuration YAML
None
Logs
File "/home/ec2-user/SageMaker/datasets/test.py", line 30, in <module>
engine.train(model=model, datamodule=datamodule)
File "/home/ec2-user/SageMaker/anomalib/src/anomalib/engine/engine.py", line 836, in train
self.trainer.test(model, test_dataloaders, ckpt_path=ckpt_path, datamodule=datamodule)
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 754, in test
return call._call_and_handle_interrupt(
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/trainer/call.py", line 44, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 794, in _test_impl
results = self._run(model, ckpt_path=ckpt_path)
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 989, in _run
results = self._run_stage()
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/trainer/trainer.py", line 1028, in _run_stage
return self._evaluation_loop.run()
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/loops/utilities.py", line 182, in _decorator
return loop_run(self, *args, **kwargs)
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 134, in run
self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 405, in _evaluation_step
call._call_callback_hooks(trainer, hook_name, output, *hook_kwargs.values())
File "/home/ec2-user/anaconda3/envs/anomalib_env/lib/python3.10/site-packages/lightning/pytorch/trainer/call.py", line 208, in _call_callback_hooks
fn(trainer, trainer.lightning_module, *args, **kwargs)
File "/home/ec2-user/SageMaker/anomalib/src/anomalib/callbacks/visualizer.py", line 88, in on_test_batch_end
for result in generator(
File "/home/ec2-user/SageMaker/anomalib/src/anomalib/utils/visualization/image.py", line 149, in _visualize_batch
image = cv2.resize(image, dsize=(width, height), interpolation=cv2.INTER_AREA)
cv2.error: OpenCV(4.9.0) /io/opencv/modules/imgproc/src/resize.cpp:3789: error: (-215:Assertion failed) !dsize.empty() in function 'resize'
Code of Conduct
- I agree to follow this project's Code of Conduct