Welcome to PyTorch Tutorials¶
Share your thoughts. Take our short PyTorch Tutorials reader survey.
Learn the Basics
Familiarize yourself with PyTorch concepts and modules. Learn how to load data, build deep neural networks, train and save your models in this quickstart guide.
Get started with PyTorchLearn the Basics
A step-by-step guide to building a complete ML workflow with PyTorch.
Learning PyTorch with Examples
This tutorial introduces the fundamental concepts of PyTorch through self-contained examples.
What is torch.nn really?
Use torch.nn to create and train a neural network.
Visualizing Models, Data, and Training with Tensorboard
Learn to use TensorBoard to visualize data and model training.
TorchVision Object Detection Finetuning Tutorial
Finetune a pre-trained Mask R-CNN model.
Transfer Learning for Computer Vision Tutorial
Train a convolutional neural network for image classification using transfer learning.
Optimizing Vision Transformer Model
Apply cutting-edge, attention-based transformer models to computer vision tasks.
Adversarial Example Generation
Train a convolutional neural network for image classification using transfer learning.
DCGAN Tutorial
Train a generative adversarial network (GAN) to generate new celebrities.
torchaudio Tutorial
Learn to load and preprocess data from a simple dataset with PyTorch's torchaudio library.
Speech Command Recognition
Learn how to correctly format an audio dataset and then train/test an audio classifier network on the dataset.
Sequence-to-Sequence Modeling with nn.Transformer and torchtext
Learn how to train a sequence-to-sequence model that uses the nn.Transformer module.
NLP from Scratch: Classifying Names with a Character-level RNN
Build and train a basic character-level RNN to classify word from scratch without the use of torchtext. First in a series of three tutorials.
NLP from Scratch: Generating Names with a Character-level RNN
After using character-level RNN to classify names, leanr how to generate names from languages. Second in a series of three tutorials.
NLP from Scratch: Translation with a Sequence-to-sequence Network and Attention
This is the third and final tutorial on doing “NLP From Scratch”, where we write our own classes and functions to preprocess the data to do our NLP modeling tasks.
Text Classification with Torchtext
This is the third and final tutorial on doing “NLP From Scratch”, where we write our own classes and functions to preprocess the data to do our NLP modeling tasks.
Language Translation with Transformer
Train a language translation model from scratch using Transformer.
Reinforcement Learning (DQN)
Learn how to use PyTorch to train a Deep Q Learning (DQN) agent on the CartPole-v0 task from the OpenAI Gym.
Train a Mario-playing RL Agent
Use PyTorch to train a Double Q-learning agent to play Mario.
Deploying PyTorch in Python via a REST API with Flask
Deploy a PyTorch model using Flask and expose a REST API for model inference using the example of a pretrained DenseNet 121 model which detects the image.
Introduction to TorchScript
Introduction to TorchScript, an intermediate representation of a PyTorch model (subclass of nn.Module) that can then be run in a high-performance environment such as C++.
Loading a TorchScript Model in C++
Learn how PyTorch provides to go from an existing Python model to a serialized representation that can be loaded and executed purely from C++, with no dependency on Python.
(optional) Exporting a Model from PyTorch to ONNX and Running it using ONNX Runtime
Convert a model defined in PyTorch into the ONNX format and then run it with ONNX Runtime.
Building a Convolution/Batch Norm fuser in FX
Build a simple FX pass that fuses batch norm into convolution to improve performance during inference.
Building a Simple Performance Profiler with FX
Build a simple FX interpreter to record the runtime of op, module, and function calls and report statistics
(beta) Channels Last Memory Format in PyTorch
Get an overview of Channels Last memory format and understand how it is used to order NCHW tensors in memory preserving dimensions.
Using the PyTorch C++ Frontend
Walk through an end-to-end example of training a model with the C++ frontend by training a DCGAN – a kind of generative model – to generate images of MNIST digits.
Custom C++ and CUDA Extensions
Create a neural network layer with no parameters using numpy. Then use scipy to create a neural network layer that has learnable weights.
Extending TorchScript with Custom C++ Operators
Implement a custom TorchScript operator in C++, how to build it into a shared library, how to use it in Python to define TorchScript models and lastly how to load it into a C++ application for inference workloads.
Extending TorchScript with Custom C++ Classes
This is a continuation of the custom operator tutorial, and introduces the API we’ve built for binding C++ classes into TorchScript and Python simultaneously.
Dynamic Parallelism in TorchScript
This tutorial introduces the syntax for doing *dynamic inter-op parallelism* in TorchScript.
Autograd in C++ Frontend
The autograd package helps build flexible and dynamic nerural netorks. In this tutorial, exploreseveral examples of doing autograd in PyTorch C++ frontend
Registering a Dispatched Operator in C++
The dispatcher is an internal component of PyTorch which is responsible for figuring out what code should actually get run when you call a function like torch::add.
Extending Dispatcher For a New Backend in C++
Learn how to extend the dispatcher to add a new device living outside of the pytorch/pytorch repo and maintain it to keep in sync with native PyTorch devices.
Performance Profiling in PyTorch
Learn how to use the PyTorch Profiler to benchmark your module's performance.
Performance Profiling in Tensorboard
Learn how to use tensorboard plugin to profile and analyze your model's performance.
Hyperparameter Tuning Tutorial
Learn how to use Ray Tune to find the best performing set of hyperparameters for your model.
Parametrizations Tutorial
Learn how to use torch.nn.utils.parametrize to put constriants on your parameters (e.g. make them orthogonal, symmetric positive definite, low-rank...)
Pruning Tutorial
Learn how to use torch.nn.utils.prune to sparsify your neural networks, and how to extend it to implement your own custom pruning technique.
(beta) Dynamic Quantization on an LSTM Word Language Model
Apply dynamic quantization, the easiest form of quantization, to a LSTM-based next word prediction model.
(beta) Dynamic Quantization on BERT
Apply the dynamic quantization on a BERT (Bidirectional Embedding Representations from Transformers) model.
(beta) Quantized Transfer Learning for Computer Vision Tutorial
Extends the Transfer Learning for Computer Vision Tutorial using a quantized model.
(beta) Static Quantization with Eager Mode in PyTorch
This tutorial shows how to do post-training static quantization.
PyTorch Distributed Overview
Briefly go over all concepts and features in the distributed package. Use this document to find the distributed training technology that can best serve your application.
Single-Machine Model Parallel Best Practices
Learn how to implement model parallel, a distributed training technique which splits a single model onto different GPUs, rather than replicating the entire model on each GPU
Getting Started with Distributed Data Parallel
Learn the basics of when to use distributed data paralle versus data parallel and work through an example to set it up.
Writing Distributed Applications with PyTorch
Set up the distributed package of PyTorch, use the different communication strategies, and go over some the internals of the package.
Getting Started with Distributed RPC Framework
Learn how to build distributed training using the torch.distributed.rpc package.
Implementing a Parameter Server Using Distributed RPC Framework
Walk through a through a simple example of implementing a parameter server using PyTorch’s Distributed RPC framework.
Distributed Pipeline Parallelism Using RPC
Demonstrate how to implement distributed pipeline parallelism using RPC
Implementing Batch RPC Processing Using Asynchronous Executions
Learn how to use rpc.functions.async_execution to implement batch RPC
Combining Distributed DataParallel with Distributed RPC Framework
Walk through a through a simple example of how to combine distributed data parallelism with distributed model parallelism.
Training Transformer models using Pipeline Parallelism
Walk through a through a simple example of how to train a transformer model using pipeline parallelism.
Training Transformer models using Distributed Data Parallel and Pipeline Parallelism
Walk through a through a simple example of how to train a transformer model using Distributed Data Parallel and Pipeline Parallelism
Image Segmentation DeepLabV3 on iOS
A comprehensive step-by-step tutorial on how to prepare and run the PyTorch DeepLabV3 image segmentation model on iOS.
Image Segmentation DeepLabV3 on Android
A comprehensive step-by-step tutorial on how to prepare and run the PyTorch DeepLabV3 image segmentation model on Android.
Additional Resources¶
Examples of PyTorch
A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
Checkout ExamplesRun Tutorials on Google Colab
Learn how to copy tutorial data into Google Drive so that you can run tutorials on Google Colab.
Open