Hi everybody, welcome to lesson two. Thanks for coming back! Slight change of environment here: we had a bit of an "administrative issue" at our university — somebody booked our room — so I'm doing this from the study at home. So sorry about the lack of decorations behind me.

I'm actually really, really pumped about this lesson. It feels like going back to what things were like in the very early days, because we're doing some really new, really cool stuff which hasn't really been in courses like this before. So, I'm super, super excited. Thanks a lot for coming back after lesson one, and I hope it's worth you coming back. I think you're going to love it! I'm really excited about this.

Now, remember that the course goes with the book, so be sure that you're following along. I mean, not following along in the book simultaneously, because we're covering similar things in different directions, but read the book as well. Remember, the book is entirely available for free as well. You can go to the `fastai/fastbook` repo to see the notebooks, or through course.fast.ai you can read it there — for example, through Colab.

And also remember that the book has a lot of stuff that we didn't cover in the course, like stuff I find pretty interesting about the history of neural networks, some of which has some really interesting personal stories, actually. At the end of each chapter, there is a quiz. Remember, it's not a bad idea before you watch the video to read the quiz. If you want to read the Chapter 2 quiz and then come back, that's not a bad idea. Then make sure that you can do the quiz after you've watched the video and read Chapter 2 of the book.

Something I didn't mention last week is there's also a very cool thing that Radek — who I mentioned last week — has written, called aiquizzes.com, which is a site specifically for quizzes about the book. It actually uses spaced-repetition learning techniques to make sure that you never forget. So, do check out aiquizzes.com. It's all brand new questions, different to the ones in the book, and they're really nicely curated and put together.

Remember, as well as course.fast.ai, there's also forums.fast.ai. Course.fast.ai is where you want to go to get links to all the notebooks, Kaggle stuff, and all that. You'll also find on forums.fast.ai that every lesson has an official topic with all the information you'll need. Generally, there'll be a bit more info on the forums; we try to keep the course lean and mean, and the forums are a bit more detailed.

So if you look for the Lesson 2 official topic, or here's the Lesson 1 official topic so far, after just a few days since I recorded it — and we haven't even launched the course, it's just the people doing it live — there's already a lot of replies, and that can get pretty overwhelming. So be aware that there's a button at the bottom of my post that says "summarize this topic."

If you hit that, then you'll just see the most upvoted replies, and that's a really good way to just make sure that you hit on the main stuff. You'll just get the upvoted stuff from fast.ai legends like Sanyam and Tanishq. So hopefully you'll find that a useful way to use the forum.

One of the cool things about this week is I put up the "show us what you've made" post, as promised, and already a lot of people have posted. I took this screenshot a few days ago and it's way above 39 replies already. I had a lot of trouble deciding which ones to share because they're all so good, so I decided to go the easy route and just pick the first ones posted.

The very, very first one to be posted is a damaged car classifier, which worked out pretty well. I really liked what Matt, the creator, said about this: "Wow, it's a bit uncomfortable to run this code because I don't really understand yet, but I'm just doing it." Good on you, Matt, for just doing it! That's the way to get started. It's all going to make sense, don't worry.

Very nice to see that the next one posted was actually a blog post in fastpages describing some experiments they ran over the week and what they found. Next one was the amazing beard detector, which was mainly created because it's very easy to get from bird to beard by changing one letter to two, and this is doing a very good job of finding gentlemen with beards.

Then this next one is another level again: it's a whole in-production web app to classify food, which is kind of like extra credit. Apparently we're up to 80 replies now in that thread; thank you, Sanyam!

This was actually created by Suvash, who's been doing the courses for a few years now. One day, you too might be able to create your very own web app and put it in production. And when I say one day, more specifically... today! I'm actually going to show you how to do this right now. It's quite a lucky coincidence that Suvash put this up there, because it's exactly the topic that we're going to pick today.

So, how do we go about putting a model in production? Step one, you've kind of done: figure out what problem you want to solve, figure out how to find data for it, gather some data, and so forth. What's the first step after you've got your data? The next step is data cleaning. If you go to Chapter 2 of the book...

Here is the book. You can open it in Colab directly from the course, or if you've cloned it to your computer, you can do it there. Remember, course.fast.ai will run you through exactly how to run these notebooks. Chapter 2 is all about putting stuff in production. Remember, we hit Shift + Enter to run cells and execute them.

We're going to go to the part of the book where we start cleaning data. A lot of people ask me what little tricks I use for getting around a Jupyter notebook so quickly and easily. One really nice one is this navigate menu, which doesn't appear by default. If you install `jupyter_contrib_nbextensions`, you just `pip install` them, follow the instructions, and restart Jupyter. Colab already has a table of contents, by the way, so this is if you're using something local. You'll see "Nbextensions" appear, and if you click "Table of Contents (2)", that gives you this handy navigation bar.

The other thing I really like is collapsible headings, which gives me these nice little toggles to close and open sections. The best part for me is if I hit the right arrow, it goes to the end of a section, and if I hit the left arrow, it goes to the start of a section. Up, left, down, right — very handy! So that's collapsible headings.

In the book, we use the Bing API for searching images. I've gone ahead and replaced Bing with DuckDuckGo (DDG) because the Bing API requires getting an SDK key, which, honestly, is the hardest thing in deep learning: figuring out the Bing Azure website and getting that sorted out. DuckDuckGo doesn't require a key!

So, just like last week, we can search for things. In the book, we did a bear detector because when I wrote it, my toddler was very interested in me helping identify teddy bears, and I certainly didn't want her accidentally cuddling a grizzly bear. So we search for grizzly bears, black bears, and teddy bears, and get rid of the ones that failed.

A few people have asked on the forum how to find out more information about any Python, fastai, or PyTorch object. There are a few tips in the book: if you put a double question mark (`??`) next to any function name, you'll get the whole source code for it. If you put a single question mark (`?`), you'll get brief information. If you have `nbdev` installed, you can type `doc()` and that will give you a link straight to the documentation with examples, as well as a link to the source code on GitHub where you can click around definitions.

What I promised you is that we're now going to clean the data. I'm going to tell you something you might find really surprising: before you clean the data, you train a model! I know that sounds backwards to what you've probably heard a thousand times — that first you clean your data, and then you train your model. But first we're going to train a model, and you'll see why in a moment.

To train a model, just like before, we use a `DataBlock` to grab our DataLoaders. There's a little sidebar here in the book about different ways we can resize images: we can squish, crop, or pad. With squishing, you end up with thin bears, but you keep all content. Cropping takes the center of the image to get a square aspect ratio, but you lose the edges. Padding adds black borders around the image to keep the aspect ratio without losing content.

Another very interesting approach is `RandomResizedCrop`. Instead of static resizing, `RandomResizedCrop` grabs a different crop of the image every time. During the week, someone on the forum wanted to recognize pictures of French and German texts from large scanned documents. They asked what to do with images that were too big, and I suggested `RandomResizedCrop`. That way, you grab different bits of the image across multiple epochs.

This idea of getting different pictures each time from the same image is called data augmentation. If you use `aug_transforms`, you can apply rotation, squishing, warping, recoloring, and saturation adjustments to make every picture different. Generally speaking, if you're training for more than five or ten epochs, you'll probably want to use `RandomResizedCrop` and `aug_transforms`.

So I've got an interesting question here from Alex in our audience, which is: "Is this copying the image multiple times or something like that?" The answer is no, we're not copying the image. Every epoch, the image gets read and transformed in RAM on the fly — cropped, recolored, warped, and so forth. So there are no copies stored on your computer, but effectively the model sees infinitely different variations. That's a great question!

We're going to use `RandomResizedCrop` and `aug_transforms` to get our DataLoaders, and then train our model. In this case, we only did four epochs of fine-tuning, achieving under 3% error rate.

Now, let me show you the confusion matrix. The confusion matrix is meaningful when your labels are categories; it shows what category errors you're making. Here, it shows that there were two times a grizzly bear was predicted as a black bear, and two times a black bear was predicted as a grizzly bear, with zero errors on teddy bears.

The really interesting thing we can do with this model is create a `ClassificationInterpretation` object and call `plot_top_losses()`. It tells us the places where the loss is highest. Loss measures how good our prediction is: loss will be high if we predict wrongly with high confidence, or if we predict correctly with low confidence.

Here's an example where we predicted grizzly when it was actually a black bear, and the model was 96% sure. An expert would go back and check those images. We also see cases where it predicted correctly (teddy/teddy) but had a high loss because it was only 66% or 72% confident.

That brings us to `ImageClassifierCleaner`. We pass our learner (`learn`) to `ImageClassifierCleaner`. It displays the images ordered by loss and confidence. We can scroll through the highest-loss images, check if they are incorrectly labeled, and either change their category or click delete.

After selecting which images to reclassify or delete, running the cleanup cell iterates through those indices, deleting the bad files or moving them to the correct folders.

This is a powerful technique that almost nobody knows about or uses: before you start data cleaning, always build a model first to find out what things are difficult to recognize and to uncover data errors!

Since I only have a 4GB GPU on my laptop, it's very important for me to close and halt notebooks when finished to free up GPU memory. Normal system RAM can swap to disk when full, but GPUs cannot swap — when GPU RAM runs out, it crashes. So make sure to close unused notebooks using the GPU.

As you watch this video, remember that I'm not asking you to run all this code simultaneously. Watch it first, then go back, pause, and follow along. A lot of students find it helpful to watch the entire video through without touching the keyboard to get the big picture, and then watch it a second time while coding along.

Now that we've cleaned our data, how do we put our model into production? In the book, we use Voilà, but nowadays most people will find Hugging Face Spaces much more useful, combined with Gradio.

Tanishq Abraham — one of the TAs for this course — has written a fantastic blog post covering everything we're covering today. I want to give a big shout-out to Tanishq: he's incredibly helpful, tenacious, thoughtful, and patient. Here's a picture of him a few years ago with Conan O'Brien when he was a child prodigy! Now he's a deep learning expert and fast.ai community leader.

To get started with Gradio and Hugging Face Spaces, go to the Hugging Face Spaces page. You sign up for free, click "Create new Space", give it a name, select the Apache 2.0 license, and choose Gradio as the Space SDK. Set it to public so you can share your work in your portfolio.

Hugging Face Spaces works through Git. Git is a standard tool for software development. You can use GitHub Desktop, which makes managing commits and reverts visually very simple. Most of the time, however, we use Git from the terminal.

If you're on Linux or Mac, you already have a terminal. If you're on Windows, get Windows Terminal from the Microsoft Store, and install WSL (Windows Subsystem for Linux) by running `wsl --install` in PowerShell as administrator. After a reboot, you have a full Ubuntu Linux environment running alongside Windows with complete GPU support.

In your terminal, clone your new space repository. Open the folder in Visual Studio Code by typing `code .`. Create an `app.py` file with the initial minimal code provided by Hugging Face (a simple `greet(name)` function returning `"Hello " + name`).

Commit and push `app.py` to Hugging Face Spaces. In about a minute, Hugging Face builds the application, providing a working Gradio interface running live in production!

Now we need to replace the dummy function with our deep learning model. First, we train a model on Kaggle or Colab (e.g., a cat vs. dog classifier). At the end of training, we call `learn.export('model.pkl')`.

Creating this pickle file is the only step that requires a GPU. Once `model.pkl` is exported, download it from Kaggle or Colab to your local machine, and move it into your Hugging Face Space repository folder.

Let's see how we load a saved model and run predictions locally in Python. We import `fastai` and `gradio`, and define any custom functions used during training (like `is_cat`). That's important: any external function used for labeling must be present in the script because the exported learner references it.

To load the model, call `load_learner('model.pkl')`. To run inference on an image, call `learn.predict(img)`. On a modern CPU/laptop, `predict()` takes only about 50 milliseconds! It returns the string prediction, the category index, and the class probabilities.

Gradio expects a prediction function that returns a dictionary mapping class labels to their float probabilities. Note that Gradio requires standard Python floats rather than PyTorch Tensors or NumPy floats, so we cast the probabilities using `float()`.

We instantiate `gr.Interface(fn=classify_image, inputs=gr.Image(), outputs=gr.Label(), examples=...)` and call `demo.launch()`. This creates a local web interface where you can upload images or click example images to get real-time predictions.

To convert our notebook experimentation into an `app.py` script automatically, we put `#export` at the top of every code cell containing code needed for production. At the bottom of the notebook, we run `notebook2script()` from `nbdev`, specifying `#| default_exp app`.

Then we push `app.py` and `model.pkl` to Git (`git commit`, `git push`). Hugging Face Spaces automatically updates and hosts our deep learning web app online for anyone to use!

Let's address a couple of questions from the forum. Someone asked: "What's the difference between a PyTorch model and a fastai learner?" We'll dive into that detail in upcoming lessons.

Another question: "How many epochs should we train for?" As you train, your error rate improves. If you train for too long, as we'll learn later, the model starts overfitting and error rate gets worse. So train until the results are good enough, or until error stops improving.

To download your exported model from Colab, open the file browser sidebar, right-click `model.pkl`, and click Download.

How do you set up Python and Jupyter on your local computer cleanly? To avoid environment problems, check out `fastai/fastsetup` on GitHub.

Run `setup-conda.sh`, which installs Mambaforge. Important note: do NOT use your system Python (the Python pre-installed on Linux/Mac). Modifying system Python often leads to disaster! Always use a standalone Conda/Mamba environment.

Once installed, run `mamba install -c fastchan fastai nbdev`. Mamba is a much faster drop-in replacement for Conda. Afterwards, simply run `jupyter notebook` in your terminal to launch Jupyter locally.

So far we've set up Hugging Face Spaces, created a Gradio interface, installed Git and Mamba, trained and exported a model, and converted notebooks with `nbdev`.

Besides Gradio, Hugging Face Spaces also supports Streamlit. Streamlit is more flexible than Gradio, though slightly harder to start with. Both are great tools for rapid prototyping.

What if you want to build a custom application beyond a prototype? At the bottom of every Gradio Space, there's a "View API" button. Gradio automatically exposes an HTTP API endpoint for your deployed model!

Because Hugging Face exposes an API endpoint, front-end developers (or data scientists learning basic JavaScript) can write a custom Web UI in HTML/JS that calls the Gradio backend via `fetch()`.

I built a simple sample website called `tinypets`. The HTML and JavaScript fit on a single screen! It reads an image input from the user, sends the data to the Hugging Face API, and parses the returned JSON containing prediction probabilities.

We can expand this to support multi-file uploads, or combine multiple models into a single app (like a pet classification model combined with an NLP QA model that answers questions about the identified breed).

How do you host a custom static web app for free? You don't need complex server software: HTML and JavaScript run directly in the user's browser.

You can host static web pages for free using GitHub Pages. The easiest way to set up a site is using `fastpages`, which sets up Jekyll themes automatically.

You can also fork existing GitHub Pages repositories (like `fastai/tinypets`), enable GitHub Pages in repository settings, and select Jekyll themes (like `pages-themes/hacker`) in `_config.yml`.

That was a quick tour of writing web apps and deploying deep learning models using Hugging Face Spaces, Gradio, JavaScript, and GitHub Pages — completely free!

Signing out: Hacker Jeremy Howard! In the next lesson, we'll dive into natural language processing (NLP), working with text instead of images. We'll look under the hood at stochastic gradient descent (SGD) and basic calculus. Thanks for watching, and see you next time!