Shortcuts

Welcome to PyTorch Tutorials

Share your thoughts. Take our short PyTorch Tutorials reader survey.

Learn the Basics

Familiarize yourself with PyTorch concepts and modules. Learn how to load data, build deep neural networks, train and save your models in this quickstart guide.

Get started with PyTorch

PyTorch Recipes

Bite-size, ready-to-deploy PyTorch code examples.

Explore Recipes


Learn the Basics

A step-by-step guide to building a complete ML workflow with PyTorch.

Getting-Started

Learning PyTorch with Examples

This tutorial introduces the fundamental concepts of PyTorch through self-contained examples.

Getting-Started

What is torch.nn really?

Use torch.nn to create and train a neural network.

Getting-Started

Visualizing Models, Data, and Training with Tensorboard

Learn to use TensorBoard to visualize data and model training.

Interpretability,Getting-Started,Tensorboard

TorchVision Object Detection Finetuning Tutorial

Finetune a pre-trained Mask R-CNN model.

Image/Video

Transfer Learning for Computer Vision Tutorial

Train a convolutional neural network for image classification using transfer learning.

Image/Video

Optimizing Vision Transformer Model

Apply cutting-edge, attention-based transformer models to computer vision tasks.

Image/Video

Adversarial Example Generation

Train a convolutional neural network for image classification using transfer learning.

Image/Video

DCGAN Tutorial

Train a generative adversarial network (GAN) to generate new celebrities.

Image/Video

torchaudio Tutorial

Learn to load and preprocess data from a simple dataset with PyTorch's torchaudio library.

Audio

Speech Command Recognition

Learn how to correctly format an audio dataset and then train/test an audio classifier network on the dataset.

Audio

Sequence-to-Sequence Modeling with nn.Transformer and torchtext

Learn how to train a sequence-to-sequence model that uses the nn.Transformer module.

Text

NLP from Scratch: Classifying Names with a Character-level RNN

Build and train a basic character-level RNN to classify word from scratch without the use of torchtext. First in a series of three tutorials.

Text

NLP from Scratch: Generating Names with a Character-level RNN

After using character-level RNN to classify names, leanr how to generate names from languages. Second in a series of three tutorials.

Text

NLP from Scratch: Translation with a Sequence-to-sequence Network and Attention

This is the third and final tutorial on doing “NLP From Scratch”, where we write our own classes and functions to preprocess the data to do our NLP modeling tasks.

Text

Text Classification with Torchtext

This is the third and final tutorial on doing “NLP From Scratch”, where we write our own classes and functions to preprocess the data to do our NLP modeling tasks.

Text

Language Translation with Transformer

Train a language translation model from scratch using Transformer.

Text

Reinforcement Learning (DQN)

Learn how to use PyTorch to train a Deep Q Learning (DQN) agent on the CartPole-v0 task from the OpenAI Gym.

Reinforcement-Learning

Train a Mario-playing RL Agent

Use PyTorch to train a Double Q-learning agent to play Mario.

Reinforcement-Learning

Deploying PyTorch in Python via a REST API with Flask

Deploy a PyTorch model using Flask and expose a REST API for model inference using the example of a pretrained DenseNet 121 model which detects the image.

Production

Introduction to TorchScript

Introduction to TorchScript, an intermediate representation of a PyTorch model (subclass of nn.Module) that can then be run in a high-performance environment such as C++.

Production,TorchScript

Loading a TorchScript Model in C++

Learn how PyTorch provides to go from an existing Python model to a serialized representation that can be loaded and executed purely from C++, with no dependency on Python.

Production,TorchScript

(optional) Exporting a Model from PyTorch to ONNX and Running it using ONNX Runtime

Convert a model defined in PyTorch into the ONNX format and then run it with ONNX Runtime.

Production

Building a Convolution/Batch Norm fuser in FX

Build a simple FX pass that fuses batch norm into convolution to improve performance during inference.

FX

Building a Simple Performance Profiler with FX

Build a simple FX interpreter to record the runtime of op, module, and function calls and report statistics

FX

(beta) Channels Last Memory Format in PyTorch

Get an overview of Channels Last memory format and understand how it is used to order NCHW tensors in memory preserving dimensions.

Memory-Format,Best-Practice,Frontend-APIs

Using the PyTorch C++ Frontend

Walk through an end-to-end example of training a model with the C++ frontend by training a DCGAN – a kind of generative model – to generate images of MNIST digits.

Frontend-APIs,C++

Custom C++ and CUDA Extensions

Create a neural network layer with no parameters using numpy. Then use scipy to create a neural network layer that has learnable weights.

Extending-PyTorch,Frontend-APIs,C++,CUDA

Extending TorchScript with Custom C++ Operators

Implement a custom TorchScript operator in C++, how to build it into a shared library, how to use it in Python to define TorchScript models and lastly how to load it into a C++ application for inference workloads.

Extending-PyTorch,Frontend-APIs,TorchScript,C++

Extending TorchScript with Custom C++ Classes

This is a continuation of the custom operator tutorial, and introduces the API we’ve built for binding C++ classes into TorchScript and Python simultaneously.

Extending-PyTorch,Frontend-APIs,TorchScript,C++

Dynamic Parallelism in TorchScript

This tutorial introduces the syntax for doing *dynamic inter-op parallelism* in TorchScript.

Frontend-APIs,TorchScript,C++

Autograd in C++ Frontend

The autograd package helps build flexible and dynamic nerural netorks. In this tutorial, exploreseveral examples of doing autograd in PyTorch C++ frontend

Frontend-APIs,C++

Registering a Dispatched Operator in C++

The dispatcher is an internal component of PyTorch which is responsible for figuring out what code should actually get run when you call a function like torch::add.

Extending-PyTorch,Frontend-APIs,C++

Extending Dispatcher For a New Backend in C++

Learn how to extend the dispatcher to add a new device living outside of the pytorch/pytorch repo and maintain it to keep in sync with native PyTorch devices.

Extending-PyTorch,Frontend-APIs,C++

Performance Profiling in PyTorch

Learn how to use the PyTorch Profiler to benchmark your module's performance.

Model-Optimization,Best-Practice,Profiling

Performance Profiling in Tensorboard

Learn how to use tensorboard plugin to profile and analyze your model's performance.

Model-Optimization,Best-Practice,Profiling

Hyperparameter Tuning Tutorial

Learn how to use Ray Tune to find the best performing set of hyperparameters for your model.

Model-Optimization,Best-Practice

Parametrizations Tutorial

Learn how to use torch.nn.utils.parametrize to put constriants on your parameters (e.g. make them orthogonal, symmetric positive definite, low-rank...)

Model-Optimization,Best-Practice

Pruning Tutorial

Learn how to use torch.nn.utils.prune to sparsify your neural networks, and how to extend it to implement your own custom pruning technique.

Model-Optimization,Best-Practice

(beta) Dynamic Quantization on an LSTM Word Language Model

Apply dynamic quantization, the easiest form of quantization, to a LSTM-based next word prediction model.

Text,Quantization,Model-Optimization

(beta) Dynamic Quantization on BERT

Apply the dynamic quantization on a BERT (Bidirectional Embedding Representations from Transformers) model.

Text,Quantization,Model-Optimization

(beta) Quantized Transfer Learning for Computer Vision Tutorial

Extends the Transfer Learning for Computer Vision Tutorial using a quantized model.

Image/Video,Quantization,Model-Optimization

(beta) Static Quantization with Eager Mode in PyTorch

This tutorial shows how to do post-training static quantization.

Quantization

PyTorch Distributed Overview

Briefly go over all concepts and features in the distributed package. Use this document to find the distributed training technology that can best serve your application.

Parallel-and-Distributed-Training

Single-Machine Model Parallel Best Practices

Learn how to implement model parallel, a distributed training technique which splits a single model onto different GPUs, rather than replicating the entire model on each GPU

Parallel-and-Distributed-Training

Getting Started with Distributed Data Parallel

Learn the basics of when to use distributed data paralle versus data parallel and work through an example to set it up.

Parallel-and-Distributed-Training

Writing Distributed Applications with PyTorch

Set up the distributed package of PyTorch, use the different communication strategies, and go over some the internals of the package.

Parallel-and-Distributed-Training

Getting Started with Distributed RPC Framework

Learn how to build distributed training using the torch.distributed.rpc package.

Parallel-and-Distributed-Training

Implementing a Parameter Server Using Distributed RPC Framework

Walk through a through a simple example of implementing a parameter server using PyTorch’s Distributed RPC framework.

Parallel-and-Distributed-Training

Distributed Pipeline Parallelism Using RPC

Demonstrate how to implement distributed pipeline parallelism using RPC

Parallel-and-Distributed-Training

Implementing Batch RPC Processing Using Asynchronous Executions

Learn how to use rpc.functions.async_execution to implement batch RPC

Parallel-and-Distributed-Training

Combining Distributed DataParallel with Distributed RPC Framework

Walk through a through a simple example of how to combine distributed data parallelism with distributed model parallelism.

Parallel-and-Distributed-Training

Training Transformer models using Pipeline Parallelism

Walk through a through a simple example of how to train a transformer model using pipeline parallelism.

Parallel-and-Distributed-Training

Training Transformer models using Distributed Data Parallel and Pipeline Parallelism

Walk through a through a simple example of how to train a transformer model using Distributed Data Parallel and Pipeline Parallelism

Parallel-and-Distributed-Training

Image Segmentation DeepLabV3 on iOS

A comprehensive step-by-step tutorial on how to prepare and run the PyTorch DeepLabV3 image segmentation model on iOS.

Mobile

Image Segmentation DeepLabV3 on Android

A comprehensive step-by-step tutorial on how to prepare and run the PyTorch DeepLabV3 image segmentation model on Android.

Mobile



Additional Resources

Examples of PyTorch

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

Checkout Examples

PyTorch Cheat Sheet

Quick overview to essential PyTorch elements.

Open

Tutorials on GitHub

Access PyTorch Tutorials from GitHub.

Go To GitHub

Run Tutorials on Google Colab

Learn how to copy tutorial data into Google Drive so that you can run tutorials on Google Colab.

Open

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources