Consider the following statement(s) fromDeep Learning book (p. 333, chapter 9: Convolutional Networks) byIan Goodfellow et al.
Convolution is thus dramatically more efficient than dense matrixmultiplication in terms of the memory requirements andstatisticalefficiency.
Book is saying that statistical efficiency is due to the decrease in the number of parameters due to convolution (using kernel) compared to fully connected feed forward neural networks.
What is meant by statistical efficiency in this context? And how does decrease in the number of parameters increase statistical efficiency?
1 Answer1
Statistical efficiency in this context essentially means that a CNN would require fewer training examples than a fully connected network to learn. Intuitively this seems reasonable: more parameters to learn should mean more samples needed. Of course it is always desirable to minimise the number of training samples needed, so that's a definite advantage of CNNs.
There isa paper on the efficiency of CNNs which attempts to make that statement more precise. They examine the case of a convolutional network using a linear activation function.
You mustlog in to answer this question.
Explore related questions
See similar questions with these tags.