ML | NN | DA Practice

Question

Suppose that you are training a neural network for classification, but you notice that the training loss is much lower than the validation loss. Which of the following can be used to address the issue (select all that apply)?

• A.  Use a network with fewer layers
• B. Decrease dropout probability √
• C. Increase $L2$ regularization weight
• D. Increase the size of each hidden layer

Answer 1

As model is overfitting, we have to reduce the model complexity. A, C does the same. Adding drop-out is adding regularization. So reducing dropout is equivalent to less regularization, so it will not help.