Skip to content

Commit 4497b1a

Browse files
ramber1836ramber1836
committed
Initial commit of Conformer
1 parent 13bd2d9 commit 4497b1a

21 files changed

+2042
-1
lines changed

‎research/Conformer/README.md

+141-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,13 @@
11

22
## Introduction
3-
This code corresponds to the paper "Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution" (will be available soon).
3+
4+
This code corresponds to the [PaddlePaddle](https://www.paddlepaddle.org.cn/en) implementation of the paper "Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution".
5+
The PyTorch version can be found [here](https://github.com/yl4467/Conformer).
6+
7+
8+
We propose a long-term time-series forecasting model, named Conformer.
9+
The framework overview of the proposed Conformer is as follows:
10+
![model](figure/frame.jpg)
411

512

613
If you find this code or any of the ideas in the paper useful, please cite:
@@ -18,3 +25,136 @@ series = {ICDE '23}
1825
```
1926

2027

28+
### NOTE
29+
30+
* The experimental result reported in the paper is implemented on top of the PyTorch version.
31+
In the implementation of PaddlePaddle version, we employ the same setup as reported in the paper,
32+
the forecast performance of this version is different from the result reported in the paper.
33+
34+
* Further parameter tuning is required to obtain comparable performance as the PyTorch version.
35+
36+
* Both the sliding-window attention and full attention are supported in this implementation.
37+
38+
* The installation of the PaddlePaddle framework can refer to
39+
[this link](https://www.paddlepaddle.org.cn/documentation/docs/zh/install/index_cn.html).
40+
41+
42+
## Requirement
43+
44+
* Python 3.6
45+
* matplotlib == 3.1.1
46+
* numpy == 1.19.4
47+
* pandas == 0.25.1
48+
* scikit_learn == 0.21.3
49+
* paddlepaddle == 2.2.2
50+
51+
Dependencies should be installed using the following command before training:
52+
`
53+
pip install -r requirements.txt
54+
`
55+
56+
## Data Preparation
57+
58+
Change the **$root_path** and **$data_path** if necessary.
59+
This code can only support **.csv** format right now.
60+
61+
The public datasets used in this paper are listed as follows.
62+
63+
***Electricity:*** https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014
64+
65+
***Traffic:*** http://pems.dot.ca.gov
66+
67+
***Weather:*** https://www.bgc-jena.mpg.de/wetter/
68+
69+
***Exchange_rate*** https://github.com/laiguokun/multivariate-time-series-data
70+
71+
***ETTs:*** https://github.com/zhouhaoyi/ETDataset
72+
73+
Besides, two collected datasets can be found here: i.e.,
74+
[WindPower](../../paddlespatial/datasets/WindPower) and [AirDelay](../../paddlespatial/datasets/AirDelay).
75+
76+
77+
## Usage
78+
79+
You can train the model with the following commands
80+
(some options, like `--root_path` and `--pred_len`, need to be configured in advance).
81+
82+
```
83+
# weather
84+
python -u train.py --data WTH --root_path $1 --pred_len $2
85+
86+
# electricity
87+
python -u train.py --data ECL --root_path $1 --pred_len $2
88+
89+
# exchange_rate
90+
python -u train.py --data EXCH --root_path $1 --pred_len $2
91+
92+
# ETTm1
93+
python -u train.py --data ETTm1 --root_path $1 --pred_len $2
94+
95+
# ETTh1
96+
python -u train.py --data ETTh1 --root_path $1 --pred_len $2
97+
98+
# Wind power
99+
python -u train.py --data elec --root_path $1 --pred_len $2
100+
```
101+
102+
103+
## Parameter Descriptions
104+
105+
### The parameters related to the model architecture
106+
107+
The proposed 'Conformer' contains three main components,
108+
the parameters of model architecture are as follows.
109+
110+
| Params | Description |
111+
|:------------------|:-------------------------------------------------------------------|
112+
| --e_layers | number of encoder layer |
113+
| --d_layers | number of decoder layer |
114+
| --enc_lstm | number of lstm used in encoder |
115+
| --dec_lstm | number of lstm used in decoder |
116+
| --normal_layer | the number of normalizing flow layers |
117+
| --d_model | the embedding dimension |
118+
| --n_head | the number of attention head |
119+
| --window | the window size of sliding window attention |
120+
| --weight | the weight between the decoder output and normalizing flow results |
121+
122+
123+
### Experiment settings
124+
125+
The parameters that used for experiments.
126+
127+
| Params | Description |
128+
|:--------------|:----------------------------------------------------|
129+
| --feature | forecasting tasks,including [M, S, MS] |
130+
| --freq | frenquency of time series |
131+
| --target | target feature when the forecasting task is MS or S |
132+
| --seq_len | the length of input sequence |
133+
| --label_len | token length of decoder input |
134+
| --pred_len | the prediction length |
135+
| --enc_in | the dimension of encoder input |
136+
| --dec_in | the dimension of decoder input |
137+
| --c_out | the dimension of output |
138+
| --checkpoints | the path to save the checkpoints |
139+
| --root_path | the root path of files |
140+
| --data_path |
141+
142+
143+
### Training settings
144+
145+
The parameters that used for training the model.
146+
147+
| Params | Description |
148+
|:----------------|:------------------------------|
149+
| --learning_rate | the learning of optimizer |
150+
| --batch_size | the batch size of input data |
151+
| --train_epochs | the number of training epoch |
152+
| --itr | the repeated experiment times |
153+
| --loss | the loss function type |
154+
155+
156+
157+
158+
159+
160+

‎research/Conformer/data/__init__.py

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+

0 commit comments

Comments
 (0)