This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | Next revision Both sides next revision | ||
cs501r_f2016:lab6 [2016/09/26 23:33] wingated |
cs501r_f2016:lab6 [2016/09/26 23:35] wingated |
||
---|---|---|---|
Line 85: | Line 85: | ||
For this part, you should implement L1 regularization on the weights. This will change your computation graph a bit, and specifically will change your cost function -- instead of optimizing ''cross_entropy'', you must optimize ''cross_entropy + lam*regularizer'', where ''lam'' is the \lambda parameter from the class slides. | For this part, you should implement L1 regularization on the weights. This will change your computation graph a bit, and specifically will change your cost function -- instead of optimizing ''cross_entropy'', you must optimize ''cross_entropy + lam*regularizer'', where ''lam'' is the \lambda parameter from the class slides. | ||
- | You should place an L1 regularizer on each of the weight and bias variables (a total of 6). A different way of saying this is that the regularization term should be sum of the absolute value of all of the individual variables from all of the weights and biases; that entire sum is then multiplied by \lambda | + | You should place an L1 regularizer on each of the weight and bias variables (a total of 8). A different way of saying this is that the regularization term should be sum of the absolute value of all of the individual variables from all of the weights and biases; that entire sum is then multiplied by \lambda |
You should experiment with a few different values of lambda, and generate a similar plot to those in Part 1 and Part 2. You should test at least the values ''[0.1, 0.01, 0.001]''. | You should experiment with a few different values of lambda, and generate a similar plot to those in Part 1 and Part 2. You should test at least the values ''[0.1, 0.01, 0.001]''. |