Introduction to PyTorch
PyTorch is an open-source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Meta AI.
It offers a powerful Python-based API for developing and training neural networks, providing flexibility and ease of use.
Key Features:
- Dynamic Computational Graph: Allows for more intuitive debugging and model definition.
- Tensor Computing: Similar to NumPy, but with GPU acceleration.
- Autograd Engine: Automatically computes gradients for backpropagation.
- Rich Ecosystem: Extensive libraries and tools for data loading, model building, and deployment.
Getting Started with Tensors
Tensors are the fundamental data structure in PyTorch, analogous to NumPy arrays.
Creating Tensors:
You can create tensors from Python lists, NumPy arrays, or directly specify their shape and data type.
import torch
# From a list
data = [[1, 2], [3, 4]]
x_data = torch.tensor(data)
print(f"Tensor from list: {x_data}")
# From a NumPy array
import numpy as np
np_array = np.array([[5, 6], [7, 8]])
x_np = torch.from_numpy(np_array)
print(f"Tensor from NumPy array: {x_np}")
# With specific shape and type
x_zeros = torch.zeros((2, 3), dtype=torch.float32)
print(f"Zero tensor: {x_zeros}")
x_ones = torch.ones((3, 2), dtype=torch.int32)
print(f"One tensor: {x_ones}")
x_rand = torch.rand((2, 2))
print(f"Random tensor: {x_rand}")
Tensor Attributes:
Tensors have attributes like shape, data type, and the device they reside on.
print(f"Shape of x_data: {x_data.shape}")
print(f"Datatype of x_data: {x_data.dtype}")
print(f"Device of x_data: {x_data.device}")
Neural Network Modules (torch.nn)
The torch.nn module provides a high-level API for building and training neural networks.
Building a Simple Neural Network:
Define your network by inheriting from nn.Module and implementing the forward method.
import torch.nn as nn
import torch.nn.functional as F
class SimpleNet(nn.Module):
def __init__(self):
super(SimpleNet, self).__init__()
self.conv1 = nn.Conv2d(1, 6, 5) # 1 input channel, 6 output channels, 5x5 kernel
self.pool = nn.MaxPool2d(2, 2) # 2x2 window, stride 2
self.fc1 = nn.Linear(16 * 5 * 5, 120) # Example input size, adjust as needed
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10) # Assuming 10 output classes
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
# Flatten the tensor for the fully connected layers
x = x.view(-1, 16 * 5 * 5) # Adjust the flattened size based on previous layers
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
net = SimpleNet()
print(net)
Loss Functions and Optimizers:
PyTorch offers a variety of loss functions and optimizers for training.
# Example: Cross-entropy loss for classification
criterion = nn.CrossEntropyLoss()
# Example: Stochastic Gradient Descent optimizer
optimizer = torch.optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
Training and Inference
The training loop involves forward pass, loss calculation, backward pass, and optimizer step.
Training Loop Skeleton:
# Assuming you have:
# trainloader: DataLoader for training data
# net: Your neural network model
# criterion: Loss function
# optimizer: Optimizer
# for epoch in range(num_epochs):
# running_loss = 0.0
# for i, data in enumerate(trainloader, 0):
# inputs, labels = data
#
# # Zero the parameter gradients
# optimizer.zero_grad()
#
# # Forward pass
# outputs = net(inputs)
# loss = criterion(outputs, labels)
#
# # Backward pass and optimize
# loss.backward()
# optimizer.step()
#
# running_loss += loss.item()
# if i % 2000 == 1999: # print every 2000 mini-batches
# print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')
# running_loss = 0.0
#
# print('Finished Training')
Inference:
For inference, wrap your model and data in torch.no_grad() to disable gradient computation.
# with torch.no_grad():
# for data in testloader:
# images, labels = data
# outputs = net(images)
# # ... process outputs
Further Resources
Explore the official PyTorch documentation and community for more in-depth learning.