Question: How to set seq_len ?

[pfeatherstone]

What is a good number for seq_len ?
What are the trade-offs for shorter or longer seq_len?
Like, why can't seq_len==1 ?
Infinite recurrence is infinite recurrence no matter what the value is right?

[lucidrains]

@pfeatherstone you still want seq_len to be as long as possible. i would keep it at 2048 - 4096

all the issues that plague RNNs will come back (vanishing gradients, retaining past information across steps, etc)