This is a work in progress.
See the listing below for planned modules and publication status.
No spam or information selling. Ever.
We'll look at what makes NLP exciting, what makes it challenging, and what we'll learn in this course.
The first step in NLP is usually to chop our documents into smaller pieces in a process called Tokenization. We'll look at the challenges involved and how to get it done.
Depending on our goal, we may preprocess text further. We'll cover case-folding, stop word removal, stemming, and lemmatization. We'll go over their use cases and their tradeoffs.
We'll look at tagging our tokens with useful information including part-of-speech tags and named entity tags. We'll also explore different types of sentence parsing to help extract the meaning of a sentence.
To perform calculations or use machine learning algorithms, we need to first turn our text into numbers. We'll take our first step here by looking at the simplest representation possible, then look at how to perform document similarity.
We'll consider the shortcomings of the basic bag-of-words approach, then improve our vectors with TF-IDF and use it for document search.
Building Models: Finding Patterns for Fun and Profit
Through a high-level overview of modelling, we'll look at the different types of machine learning, how to evaluate model performance, and what to do when things go wrong.
Naive Bayes: Fast and Simple Text Classification
We'll learn how the Naive Bayes classifier works under the hood, see how accuracy can go wrong and how to use precision and recall instead, and then build a text classifier while working through problems along the way.
Topic Modelling: Automatically Discovering Topics in Documents
What do you do when you need to make sense of a pile of documents and have no other information? We'll learn one approach to this problem using Latent Dirichlet Allocation. We'll cover how it works, then build a model to discover topics present in a document and to search for similar documents.
Neural Networks I: Core Mechanisms and Coding One From Scratch
Neural networks have led to incredible breakthroughs in all things AI, and while training and deploying large neural networks can be daunting, they're still pretty simple at their core. In this module, we'll cover how neural networks work and how they "learn" to get better over time. And to further ground our knowledge, we'll then build a neural network from scratch. By the end of this module, you'll have a clear understanding of their core mechanisms.
Neural Networks II: Effective Training Techniques
Beyond core mechanisms, there are a variety of details when it comes to training effective neural networks. In this module, we'll learn how to set learning rates, how to deal with complex loss surfaces, regularization techniques to fight overfitting, and more. We'll also return to NLP by building a basic deep learning model for text classification. This will conclude our two-part deep dive into neural networks and set us up for the rest of the course.
We'll learn a method to vectorize words such that words with similar meanings have closer vectors (aka "embeddings"). This was a breakthrough in NLP and boosted performance on a variety of NLP problems while addressing the shortcomings of previous approaches. We'll look at how to create these word embeddings and how to use them in our models.
Recurrent Neural Networks and Language Models
How do you get a computer to generate coherent text? In this module, we'll learn how to do this using a technique called recurrence. We'll go beyond the bag-of-words approaches we've seen so far, and start capturing the information in word order. We'll then learn how to build two new types of models: a part-of-speech tagger, and a language model to generate text.
Sequence-to-Sequence and Attention
Whether it's translation, summarization, or even answering questions, a lot of NLP tasks come down to transforming one type of sequence into another. In this module, we'll learn to do that using encoders and decoders. We'll then look at the weaknesses of the standard approach, and enhance our model with Attention. In the demo, we'll build a model to translate languages for us.