Even though Stochastic Gradient Descent sounds fancy, it is just a simple addition to "regular" Gradient Descent. This video sets up the problem that Stochastic Gradient Descent solves and then shows how it does it. Along the way, we discuss situations where Stochastic Gradient Descent is most useful, and some cool features that aren't that obvious.

NOTE: There is a small typo at 9:03. The values for the intercept and slope should be the most recent estimates, 0.86 and 0.68, instead of the original random values, 0 and 1.

NOTE: This StatQuest assumes you already understand "regular" Gradient Descent. If not, check out the 'Quest:    • Gradient Descent, StepbyStep  

When I was researching Stochastic Gradient Descent, I found a ton of cool websites that provided lots of details. Here are some of my favorites:

Sebastian Ruder has a nice writeup: http://ruder.io/optimizinggradientd...

...as the Usupervised Feature Learning and Deep Learning Tutorial: http://deeplearning.stanford.edu/tuto...

For a complete index of all the StatQuest videos, check out:
https://statquest.org/videoindex/

If you'd like to support StatQuest, please consider...

Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF https://statquest.gumroad.com/l/wvtmc
Paperback https://www.amazon.com/dp/B09ZCKR4H6
Kindle eBook https://www.amazon.com/dp/B09ZG79HXC

Patreon:   / statquest  
...or...
YouTube Membership:    / @statquest  

...a cool StatQuest tshirt or sweatshirt:
https://shop.spreadshirt.com/statques...

...buying one or two of my songs (or go large and get a whole album!)
https://joshuastarmer.bandcamp.com/

...or just donating to StatQuest!
https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
  / joshuastarmer  

Corrections:
9:03. The values for the intercept and slope should be the most recent estimates, 0.86 and 0.68, instead of the original random values, 0 and 1.
9:33 the slope should be 0.7.

#statquest #sgd