Debugging ML The (ideal) plots Iterations during
Description: Debugging ML The (ideal) plots Iterations during training Iterations during training Accuracy Loss Train Val Train Val Overfitting (mild) Validation accuracy does not improve as training loss goes down Iterations during training Iterations
Related Topics
Download Presentation
"Debugging ML The (ideal) plots Iterations during" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Debugging ML<br>
slide2. The (ideal) plots Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide3. Overfitting (mild) Validation accuracy does not improve as training loss goes down Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide4. Overfitting (bad) Validation accuracy decreases as training loss goes down Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide5. Underfitting Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide6. Underfitting vs overfitting Model capacity Loss Underfitting Overfitting<br>
slide7. Optimization issues<br>
slide8. Optimization issues: Large step size Iterations during training Loss Train Small step sizes cause slow learning,
Large step sizes cause divergence<br>
slide9. SGD with momentum Stochastic gradient is stochastic
Can reduce variance using momentum
Standard update:
With momentum:<br>
slide10. Optimization issues: convergence Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide2. The (ideal) plots Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide3. Overfitting (mild) Validation accuracy does not improve as training loss goes down Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide4. Overfitting (bad) Validation accuracy decreases as training loss goes down Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide5. Underfitting Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>
slide6. Underfitting vs overfitting Model capacity Loss Underfitting Overfitting<br>
slide7. Optimization issues<br>
slide8. Optimization issues: Large step size Iterations during training Loss Train Small step sizes cause slow learning,
Large step sizes cause divergence<br>
slide9. SGD with momentum Stochastic gradient is stochastic
Can reduce variance using momentum
Standard update:
With momentum:<br>
slide10. Optimization issues: convergence Iterations during training Iterations during training Accuracy Loss Train Val Train Val<br>