What is the purpose served by 1x1 convolutions in the papers?

- Network in Network

- GoogleNet

One reason which I came across for using them was to reduce dimensions. Are there any specific advantages for using them?

PS. I haven't come across any work which actually reduces dimension. They always have the same number of dimension as output.

- Network in Network

- GoogleNet

One reason which I came across for using them was to reduce dimensions. Are there any specific advantages for using them?

PS. I haven't come across any work which actually reduces dimension. They always have the same number of dimension as output.

- You can think of them as a coordinate-dependent transformation in the filter space.Oct 13, 2015
- Additional to dim reduction, I find a similarity between sum product nets and 1x1 using concolutional networks and it implies more non-linearity which is believed to be useful.Oct 13, 2015
- I don't really get how it helps. As +Michael Rotman said, We can imagine it as some form of coordinate transformation in the filter space - Rotation + Scaling of the Coordinate space. But Why do we have to optimize it? We can simply run PCA or something to get the results right?Oct 13, 2015
- It is much more general than rotations/scaling as the transformation belongs to the group GL(n,R). You should check whether the mixing of the filters yields better results than just multiplying each pixel by a different learnable weight (filter independent, identity matrix in the filter space)Oct 13, 2015
- Can you please elaborate what you mean by "mixing" and "just multiplying"?Oct 13, 2015
- I think it depends on the way you think about them. I like this one:

"In Convolutional Nets, there is no such thing as "fully-connected layers". There are only convolution layers with 1x1 convolution kernels and a full connection table." (c) Yann LeCunOct 13, 2015 - As Eren said, they are used for dimensionality reduction. Specifically, the number of feature maps are getting reduced.Oct 19, 2015