The Scaler in test.zip predict.py requires train data, but when upload with train data, the file will exceed 200MB

[GuibinChen92057]

In the predict.py, the forecast_one func, we need to use the train_data.get_scaler() to transform and inverse_trainsform test_x, but when we put the 'data/wtb_245days.csv' in our submit file, it must exceed 200MB. How can we handle with it?

[shaido987]

What I'm doing with the scaler is saving it in a pickle file during training and including it in the submitted files. Then the scaler can be loaded and used at prediction time. This has the additional advantage of being faster as well.

[GuibinChen92057]

Thank you for your reply, but I still have some question. What if I adopt other "scaler" function? For example, the released version using the scaler defined in the Wind Turbine Data.py file, so we can invoke it faster in our upload predict.py file. But if I define other form of "scaler" (i.e., si-ki learn StandardScaler), then will error happens when uploaded? many thx if you can reply!

…

---- Replied Message ---- | From | ***@***.***> | | Date | 07/11/2022 09:23 | | To | ***@***.***> | | Cc | ***@***.******@***.***> | | Subject | Re: [PaddlePaddle/PaddleSpatial] The Scaler in test.zip predict.py requires train data, but when upload with train data, the file will exceed 200MB (Discussion #248) | What I'm doing with the scaler is saving it in a pickle file during training and including it in the submitted files. Then the scaler can be loaded and used at prediction time. This has the additional advantage of being faster as well. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[ramber1836]

@GuibinChen92057
You can customize the wind_turbine_data.py with your favorite Scaler implementation.
The backend evaluation does not presume any specific scaler function.