Einstein’s desk a few hours after his death. Source: LIFE Magazine

While machine learning has a rich history dating back to 1959, the field is evolving at an unprecedented rate. In a recent article, I discussed why the broader artificial intelligence field is booming and likely will for some time to come. Those interested in learning ML may find it daunting to get started.

As I prepare to start my Ph.D. program in the Fall, I’ve been scouring the web for good resources on all aspects of machine learning and NLP. Typically, I’ll find an interesting tutorial or video, and that leads to three or four more tutorials or videos, and before I know it, I have 20 tabs of new material I need to go through. (On a side note, Tab Bundler has been helpful to stay organized.)

After finding over 25 ML-related “cheat sheets”, I created a post that links to all the good ones.

To help others that are going through a similar discovery process, I’ve put together a list of the best tutorial content that I’ve found so far. It’s by no means an exhaustive list of every ML-related tutorial on the web — that would be overwhelming and duplicative. Plus, there is a bunch of mediocre content out there. My goal was to link to the best tutorials I found on the important subtopics within machine learning and NLP.

By tutorial, I’m referring to introductory content that is intending to teach a concept succinctly. I’ve avoided including chapters of books, which have a greater breadth of coverage, and research papers, which generally don’t do a good job in teaching concepts. Why not just buy a book? Tutorials are helpful when you’re trying to learn a specific niche topic or want to get different perspectives.

I’ve split this post into four sections: Machine Learning, NLP, Python, and Math. I’ve included a sampling of topics within each section, but given the vastness of the material, I can’t possibly include every possible topic.

For future posts, I may create a similar list of books, online videos, and code repos as I’m compiling a growing collection of those resources too.

If there are good tutorials you are aware of that I’m missing, please let me know! I’m trying to limit each topic to five or six tutorials since much beyond that would be repetitive. Each link should have different material from the other links or present information in a different way (e.g. code versus slides versus long-form) or from a different perspective.

Machine Learning

Machine Learning is Fun! (medium.com/@ageitgey)

Machine Learning Crash Course: Part I, Part II, Part III (Machine Learning at Berkeley)

An Introduction to Machine Learning Theory and Its Applications: A Visual Tutorial with Examples (toptal.com)

A Gentle Guide to Machine Learning (monkeylearn.com)

Which machine learning algorithm should I use? (sas.com)

Activation and Loss Functions

Sigmoid neurons (neuralnetworksanddeeplearning.com)

What is the role of the activation function in a neural network? (quora.com)

Comprehensive list of activation functions in neural networks with pros/cons (stats.stackexchange.com)

Activation functions and it’s types-Which is better? (medium.com)

Making Sense of Logarithmic Loss (exegetic.biz)

Loss Functions (Stanford CS231n)

L1 vs. L2 Loss function (rishy.github.io)

The cross-entropy cost function (neuralnetworksanddeeplearning.com)

Bias

Role of Bias in Neural Networks (stackoverflow.com)

Bias Nodes in Neural Networks (makeyourownneuralnetwork.blogspot.com)

What is bias in artificial neural network? (quora.com)

Perceptron

Perceptrons (neuralnetworksanddeeplearning.com)

The Perception (natureofcode.com)

Single-layer Neural Networks (Perceptrons) (dcu.ie)

From Perceptrons to Deep Networks (toptal.com)

Regression

Introduction to linear regression analysis (duke.edu)

Linear Regression (ufldl.stanford.edu)

Linear Regression (readthedocs.io)

Logistic Regression (readthedocs.io)

Simple Linear Regression Tutorial for Machine Learning (machinelearningmastery.com)

Logistic Regression Tutorial for Machine Learning (machinelearningmastery.com)

Softmax Regression (ufldl.stanford.edu)

Gradient Descent

Learning with gradient descent (neuralnetworksanddeeplearning.com)

Gradient Descent (iamtrask.github.io)

How to understand Gradient Descent algorithm (kdnuggets.com)

An overview of gradient descent optimization algorithms (sebastianruder.com)

Optimization: Stochastic Gradient Descent (Stanford CS231n)

Generative Learning

Generative Learning Algorithms (Stanford CS229)

A practical explanation of a Naive Bayes classifier (monkeylearn.com)

Support Vector Machines

An introduction to Support Vector Machines (SVM) (monkeylearn.com)

Support Vector Machines (Stanford CS229)

Linear classification: Support Vector Machine, Softmax (Stanford 231n)

Backpropagation

Yes you should understand backprop (medium.com/@karpathy)

Can you give a visual explanation for the back propagation algorithm for neural networks? (github.com/rasbt)

How the backpropagation algorithm works (neuralnetworksanddeeplearning.com)

Backpropagation Through Time and Vanishing Gradients (wildml.com)

A Gentle Introduction to Backpropagation Through Time (machinelearningmastery.com)

Backpropagation, Intuitions (Stanford CS231n)

Deep Learning

Deep Learning in a Nutshell (nikhilbuduma.com)

A Tutorial on Deep Learning (Quoc V. Le)

What is Deep Learning? (machinelearningmastery.com)

What’s the Difference Between Artificial Intelligence, Machine Learning, and Deep Learning? (nvidia.com)

Optimization and Dimensionality Reduction

Seven Techniques for Data Dimensionality Reduction (knime.org)

Principal components analysis (Stanford CS229)

Dropout: A simple way to improve neural networks (Hinton @ NIPS 2012)

How to train your Deep Neural Network (rishy.github.io)

Long Short Term Memory (LSTM)

A Gentle Introduction to Long Short-Term Memory Networks by the Experts (machinelearningmastery.com)

Understanding LSTM Networks (colah.github.io)

Exploring LSTMs (echen.me)

Anyone Can Learn To Code an LSTM-RNN in Python (iamtrask.github.io)

Convolutional Neural Networks (CNNs)

Introducing convolutional networks (neuralnetworksanddeeplearning.com)

Deep Learning and Convolutional Neural Networks (medium.com/@ageitgey)

Conv Nets: A Modular Perspective (colah.github.io)

Understanding Convolutions (colah.github.io)

Recurrent Neural Nets (RNNs)

Recurrent Neural Networks Tutorial (wildml.com)

Attention and Augmented Recurrent Neural Networks (distill.pub)

The Unreasonable Effectiveness of Recurrent Neural Networks (karpathy.github.io)

A Deep Dive into Recurrent Neural Nets (nikhilbuduma.com)

Reinforcement Learning

Simple Beginner’s guide to Reinforcement Learning & its implementation (analyticsvidhya.com)

A Tutorial for Reinforcement Learning (mst.edu)

Learning Reinforcement Learning (wildml.com)

Deep Reinforcement Learning: Pong from Pixels (karpathy.github.io)

Generative Adversarial Networks (GANs)

What’s a Generative Adversarial Network? (nvidia.com)

Abusing Generative Adversarial Networks to Make 8-bit Pixel Art (medium.com/@ageitgey)

An introduction to Generative Adversarial Networks (with code in TensorFlow) (aylien.com)

Generative Adversarial Networks for Beginners (oreilly.com)

Multi-task Learning

An Overview of Multi-Task Learning in Deep Neural Networks (sebastianruder.com)

NLP

A Primer on Neural Network Models for Natural Language Processing (Yoav Goldberg)

The Definitive Guide to Natural Language Processing (monkeylearn.com)

Introduction to Natural Language Processing (algorithmia.com)

Natural Language Processing Tutorial (vikparuchuri.com)

Natural Language Processing (almost) from Scratch (arxiv.org)

Deep Learning and NLP

Deep Learning applied to NLP (arxiv.org)

Deep Learning for NLP (without Magic) (Richard Socher)

Understanding Convolutional Neural Networks for NLP (wildml.com)

Deep Learning, NLP, and Representations (colah.github.io)

Embed, encode, attend, predict: The new deep learning formula for state-of-the-art NLP models (explosion.ai)

Understanding Natural Language with Deep Neural Networks Using Torch (nvidia.com)

Deep Learning for NLP with Pytorch (pytorich.org)

Word Vectors

Bag of Words Meets Bags of Popcorn (kaggle.com)

On word embeddings Part I, Part II, Part III (sebastianruder.com)

The amazing power of word vectors (acolyer.org)

word2vec Parameter Learning Explained (arxiv.org)

Word2Vec Tutorial — The Skip-Gram Model, Negative Sampling (mccormickml.com)

Encoder-Decoder

Attention and Memory in Deep Learning and NLP (wildml.com)

Sequence to Sequence Models (tensorflow.org)

Sequence to Sequence Learning with Neural Networks (NIPS 2014)

Machine Learning is Fun Part 5: Language Translation with Deep Learning and the Magic of Sequences (medium.com/@ageitgey)

How to use an Encoder-Decoder LSTM to Echo Sequences of Random Integers (machinelearningmastery.com)

tf-seq2seq (google.github.io)

Python

7 Steps to Mastering Machine Learning With Python (kdnuggets.com)

An example machine learning notebook (nbviewer.jupyter.org)

Examples

How To Implement The Perceptron Algorithm From Scratch In Python (machinelearningmastery.com)

Implementing a Neural Network from Scratch in Python (wildml.com)

A Neural Network in 11 lines of Python (iamtrask.github.io)

Implementing Your Own k-Nearest Neighbour Algorithm Using Python (kdnuggets.com)

Demonstration of Memory with a Long Short-Term Memory Network in Python (machinelearningmastery.com)

How to Learn to Echo Random Integers with Long Short-Term Memory Recurrent Neural Networks (machinelearningmastery.com)

How to Learn to Add Numbers with seq2seq Recurrent Neural Networks (machinelearningmastery.com)

Scipy and numpy

Scipy Lecture Notes (scipy-lectures.org)

Python Numpy Tutorial (Stanford CS231n)

An introduction to Numpy and Scipy (UCSB CHE210D)

A Crash Course in Python for Scientists (nbviewer.jupyter.org)

scikit-learn

PyCon scikit-learn Tutorial Index (nbviewer.jupyter.org)

scikit-learn Classification Algorithms (github.com/mmmayo13)

scikit-learn Tutorials (scikit-learn.org)

Abridged scikit-learn Tutorials (github.com/mmmayo13)

Tensorflow

Tensorflow Tutorials (tensorflow.org)

Introduction to TensorFlow — CPU vs GPU (medium.com/@erikhallstrm)

TensorFlow: A primer (metaflow.fr)

RNNs in Tensorflow (wildml.com)

Implementing a CNN for Text Classification in TensorFlow (wildml.com)

How to Run Text Summarization with TensorFlow (surmenok.com)

PyTorch

PyTorch Tutorials (pytorch.org)

A Gentle Intro to PyTorch (gaurav.im)

Tutorial: Deep Learning in PyTorch (iamtrask.github.io)

PyTorch Examples (github.com/jcjohnson)

PyTorch Tutorial (github.com/MorvanZhou)

PyTorch Tutorial for Deep Learning Researchers (github.com/yunjey)

Math

Math for Machine Learning (ucsc.edu)

Math for Machine Learning (UMIACS CMSC422)

Linear algebra

An Intuitive Guide to Linear Algebra (betterexplained.com)

A Programmer’s Intuition for Matrix Multiplication (betterexplained.com)

Understanding the Cross Product (betterexplained.com)

Understanding the Dot Product (betterexplained.com)

Linear Algebra for Machine Learning (U. of Buffalo CSE574)

Linear algebra cheat sheet for deep learning (medium.com)

Linear Algebra Review and Reference (Stanford CS229)

Probability

Understanding Bayes Theorem With Ratios (betterexplained.com)

Review of Probability Theory (Stanford CS229)

Probability Theory Review for Machine Learning (Stanford CS229)

Probability Theory (U. of Buffalo CSE574)

Probability Theory for Machine Learning (U. of Toronto CSC411)

Calculus

How To Understand Derivatives: The Quotient Rule, Exponents, and Logarithms (betterexplained.com)

How To Understand Derivatives: The Product, Power & Chain Rules (betterexplained.com)

Vector Calculus: Understanding the Gradient (betterexplained.com)

Differential Calculus (Stanford CS224n)

Calculus Overview (readthedocs.io)