improve the initializer interface for layers

[dzhwinter]

The parameters in layers need a better flexible and user-friendly initializer interface. Currently, most of our layers have the fixed init method, such as conv2d use the gaussian initializer conv2d.
We need to rewrite that part.

[jacquesqiao]

this initializer for conv2d is designed in purpose, and it's a good initialization method. But it needs to allow users to define their own initializer~

[qingqing01]

For the convolution operators, maybe we need to define a MSRAInit and set it by default. The MSRAInit in caffe2 : https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.h#L462

[abhinavarora]

I agree with @qingqing01. I have opened an issue for this #5752.

[dzhwinter]

I think we need to make the initialized method configurable. Even though the MSRAInit is perfect, when a user wants to do some init trick, our design will block their use cases.

[dzhwinter]

For example, this paper and its variants. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwjyxfznscbXAhVQ1mMKHdgDBdwQFggnMAA&url=https%3A%2F%2Farxiv.org%2Fabs%2F1706.02677&usg=AOvVaw2FEBNWmB_5ah6W-73eD2yA

In a word, we need to give the right to users.

[abhinavarora]

@dzhwinter, I am working on a PR to add that flexibility. Will send you all review soon.

[abhinavarora]

Tasks:

• Design pattern for letting users pass initializers Improve the initializer Interface for fc, sequence_conv and conv2d layers #5760
• Ensure that all parameters created through layer_helper have initializers, if not then create a default initializer based on dtype Improve the initializer Interface for fc, sequence_conv and conv2d layers #5760
• Fix FC layer Improve the initializer Interface for fc, sequence_conv and conv2d layers #5760
• Fix Sequence_conv layer Improve the initializer Interface for fc, sequence_conv and conv2d layers #5760
• Fix conv2d layer Improve the initializer Interface for fc, sequence_conv and conv2d layers #5760
• Add MSRA initializer for rectifiers Implementing the MSRA initializer for rectifier units #5805