Titanic With Pytorch

Lesson 9: Solving the Titanic Problem with PyTorch 🔥🤖

Objective:

  • Understand the fundamental components of PyTorch for building neural networks.
  • Learn how to define, train, and evaluate a neural network for the Titanic survival prediction task using PyTorch.
  • Appreciate the advantages of using a deep learning framework.

Recap of Previous Lessons:

  • We’ve preprocessed the Titanic dataset (handling missing values, encoding categoricals, scaling numerical features).
  • We’ve split our data into training (X_train, y_train) and validation (X_val, y_val) sets.
  • We’ve built a logistic regression model and a simple neural network from scratch to understand the underlying mechanics.

1. Why PyTorch? (Conceptual - 10 min)

Building neural networks from scratch is invaluable for learning, but for larger, more complex models, or for leveraging hardware like GPUs, frameworks are essential.

Advantages of PyTorch (and similar frameworks like TensorFlow/Keras):

  • Automatic Differentiation (autograd): PyTorch automatically calculates gradients for backpropagation. No need to manually derive and implement gradient formulas!
  • Pre-built Layers & Modules: Provides optimized implementations of common layers (linear, convolutional, recurrent), activation functions, loss functions, etc.
  • Optimizers: Includes various optimization algorithms (SGD, Adam, RMSprop, etc.).
  • GPU Support: Easily run computations on NVIDIA GPUs for significant speedups in training deep models.
  • Dynamic Computation Graphs: PyTorch uses dynamic graphs (define-by-run), which can be more intuitive for some and flexible for models with varying structures.
  • Large Community & Ecosystem: Extensive documentation, tutorials, pre-trained models, and supporting libraries.

2. Core PyTorch Concepts (Conceptual - 15 min)

  • Tensors: The fundamental data structure in PyTorch, similar to NumPy arrays. Tensors can be moved to a GPU for accelerated computation.
import torch
x = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
print(x)
  • torch.nn.Module: The base class for all neural network modules. Your custom models will inherit from this.
  • Layers (e.g., torch.nn.Linear): Pre-defined layers. nn.Linear(in_features, out_features) creates a fully connected layer.
  • Activation Functions (e.g., torch.nn.ReLU, torch.nn.Sigmoid): Found in torch.nn or torch.nn.functional.
  • Loss Functions (e.g., torch.nn.BCELoss): Quantify the difference between predictions and true labels.
  • Optimizers (e.g., torch.optim.SGD, torch.optim.Adam): Implement algorithms to update model weights based on gradients.
  • autograd: PyTorch’s automatic differentiation engine.

3. Building a Neural Network for Titanic with PyTorch (Practical)

a) Data Preparation: Pandas to PyTorch Tensors (Practical - 15 min)

import torch
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline

# Assume 'train_df' is your loaded Titanic training data
# Dummy features and target
data = {
    'Pclass': [1, 2, 3, 1, 2, 3, 1, 2],
    'Sex': ['male', 'female', 'male', 'female', 'male', 'female', 'male', 'female'],
    'Age': [22, 38, 26, 35, 35, None, 54, 2],
    'SibSp': [1, 1, 0, 1, 0, 0, 0, 3],
    'Parch': [0, 0, 0, 0, 0, 0, 0, 1],
    'Fare': [7.25, 71.2833, 7.925, 53.1, 8.05, 8.4583, 51.8625, 21.075],
    'Embarked': ['S', 'C', 'S', 'S', 'S', 'Q', 'S', 'S'],
    'Survived': [0, 1, 1, 1, 0, 0, 0, 1]
}
train_df = pd.DataFrame(data)

# Separate target variable
X = train_df.drop('Survived', axis=1)
y = train_df['Survived']

# Define numerical and categorical features
numerical_features = ['Age', 'Fare', 'SibSp', 'Parch']
categorical_features = ['Pclass', 'Sex', 'Embarked']

# Create preprocessing pipelines for numerical and categorical features
numerical_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='mean')),
    ('scaler', StandardScaler())
])
categorical_pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('onehot', OneHotEncoder(handle_unknown='ignore'))
])

# Create a column transformer to apply different transformations to different columns
preprocessor = ColumnTransformer([
    ('numerical', numerical_pipeline, numerical_features),
    ('categorical', categorical_pipeline, categorical_features)
])

# Preprocess the data
X_processed = preprocessor.fit_transform(X)

# Split data
X_train_processed, X_val_processed, y_train_series, y_val_series = train_test_split(
    X_processed, y, test_size=0.2, random_state=42
)

# Convert to PyTorch Tensors
X_train_tensor = torch.tensor(X_train_processed, dtype=torch.float32)
y_train_tensor = torch.tensor(y_train_series.values, dtype=torch.float32).unsqueeze(1)
X_val_tensor = torch.tensor(X_val_processed, dtype=torch.float32)
y_val_tensor = torch.tensor(y_val_series.values, dtype=torch.float32).unsqueeze(1)

b) Defining the Neural Network (Practical - 20 min)

import torch.nn as nn

class TitanicNet(nn.Module):
    def __init__(self, input_size, hidden_size1, output_size):
        super(TitanicNet, self).__init__() 
        self.fc1 = nn.Linear(input_size, hidden_size1) 
        self.relu1 = nn.ReLU()                         
        self.fc2 = nn.Linear(hidden_size1, output_size) 
        self.sigmoid = nn.Sigmoid()

def forward(self, x):
        out = self.fc1(x)
        out = self.relu1(out)
        out = self.fc2(out)
        out = self.sigmoid(out) 
        return out

hidden_size1 = 32  
output_size = 1    
model = TitanicNet(input_size, hidden_size1, output_size)

c) Defining Loss Function and Optimizer (Practical - 10 min)

criterion = nn.BCELoss() 
learning_rate = 0.001 
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

d) Training Loop (Practical - 25 min)

num_epochs = 200  
batch_size = 16

train_losses = []
val_losses = []
val_accuracies = []

for epoch in range(num_epochs):
    model.train() 
    outputs = model(X_train_tensor)
    loss = criterion(outputs, y_train_tensor)
    train_losses.append(loss.item()) 
    optimizer.zero_grad()  
    loss.backward()        
    optimizer.step()

model.eval() 
    with torch.no_grad(): 
        val_outputs = model(X_val_tensor)
        val_loss = criterion(val_outputs, y_val_tensor)
        val_losses.append(val_loss.item())
        predicted_classes = (val_outputs > 0.5).float() 
        correct_predictions = (predicted_classes == y_val_tensor).sum().item()
        total_predictions = y_val_tensor.size(0)
        accuracy = correct_predictions / total_predictions
        val_accuracies.append(accuracy)

if (epoch + 1) % 20 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Train Loss: {loss.item():.4f}, Val Loss: {val_loss.item():.4f}, Val Accuracy: {accuracy:.4f}')

e) Evaluating the Model (Practical - 10 min)

import matplotlib.pyplot as plt

plt.figure(figsize=(12, 5))
plt.subplot(1, 2, 1)
plt.plot(train_losses, label='Training Loss')
plt.plot(val_losses, label='Validation Loss')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.title('Training and Validation Loss')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(val_accuracies, label='Validation Accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.title('Validation Accuracy')
plt.legend()
plt.tight_layout()
plt.show()

f) Making Predictions on New Data (Conceptual - 5 min)

# Assume 'test_df' is loaded and preprocessed similarly to X_train
# model.eval() 
# with torch.no_grad(): 
#     test_predictions_probs = model(X_test_tensor)
#     test_predictions_classes = (test_predictions_probs > 0.5).int()

Advantages Revisited & Next Steps (Conceptual - 10 min)

  1. Ease of Building: nn.Module and pre-built layers.
  2. Automatic Gradients: Powerful and convenient.
  3. Flexibility: Easy modifications.
  4. GPU Acceleration: Significant speedups.

Summary

In this lesson, you’ve successfully built, trained, and evaluated a neural network for the Titanic survival prediction task using PyTorch. You’ve learned about core PyTorch components like Tensors, nn.Module, layers, loss functions, optimizers, and the autograd system. This provides a solid foundation for tackling more complex deep learning problems.