Results do not match the description in the paper.

[wmshen02]

Issue description

Thanks for your efforts on building general benchmark on imputation tasks. When I observe the imputation log files, I find the final results is also contrary to the results of a single round of testing. For example, the following is a partial screenshot of the CSDI_PhysioNet2012.log. We can find that the MAE reported in each round is around 0.33. Why does the last row report an average MAE of 0.25? And my MAE is also maintained at around 0.3. This situation appears in lots of logs. I‘m looking forward to your reply.

Your contribution

I will star this repository.

[github-actions]

Hi there 👋,

Thank you for your attention to PyPOTS! You can follow me on GitHub to receive the latest news of PyPOTS.
If you find PyPOTS helpful to your work, please star⭐️ this repository to get this issue prioritized.
Your star is your recognition, which can help more people notice PyPOTS and grow PyPOTS community.
It matters and is definitely a kind of contribution to the community.

Best,
Wenjie

[WenjieDu]

Wangmeng and our team had a discussion on this issue. We checked everything and ensured that there's no code bug regarding this. We also found such issue happened to other models trained on the dataset but not only CSDI. The only possible explanation is, regenerating PhysioNet2012 dataset (point rate01) during experiments leads to the final results overwriting.

We appreciate the feedback from Wangmeng and our community. PyPOTS is upgrading fast in recent months and more new algorithms are included. We plan to update our code for TSI-Bench in this repo to cover more algorithms in the near future.