# Titanic With Pytorch ## Lesson 9: Solving the Titanic Problem with PyTorch 🔥🤖 **Objective:** - Understand the fundamental components of PyTorch for building neural networks. - Learn how to define, train, and evaluate a neural network for the Titanic survival prediction task using PyTorch. - Appreciate the advantages of using a deep learning framework. ### Recap of Previous Lessons: - We’ve preprocessed the Titanic dataset (handling missing values, encoding categoricals, scaling numerical features). - We’ve split our data into training (`X_train`, `y_train`) and validation (`X_val`, `y_val`) sets. - We’ve built a logistic regression model and a simple neural network _from scratch_ to understand the underlying mechanics. ### 1. Why PyTorch? (Conceptual - 10 min) Building neural networks from scratch is invaluable for learning, but for larger, more complex models, or for leveraging hardware like GPUs, frameworks are essential. **Advantages of PyTorch (and similar frameworks like TensorFlow/Keras):** - **Automatic Differentiation (`autograd`):** PyTorch automatically calculates gradients for backpropagation. No need to manually derive and implement gradient formulas! - **Pre-built Layers & Modules:** Provides optimized implementations of common layers (linear, convolutional, recurrent), activation functions, loss functions, etc. - **Optimizers:** Includes various optimization algorithms (SGD, Adam, RMSprop, etc.). - **GPU Support:** Easily run computations on NVIDIA GPUs for significant speedups in training deep models. - **Dynamic Computation Graphs:** PyTorch uses dynamic graphs (define-by-run), which can be more intuitive for some and flexible for models with varying structures. - **Large Community & Ecosystem:** Extensive documentation, tutorials, pre-trained models, and supporting libraries. ### 2. Core PyTorch Concepts (Conceptual - 15 min) - **Tensors:** The fundamental data structure in PyTorch, similar to NumPy arrays. Tensors can be moved to a GPU for accelerated computation. ```lua import torch x = torch.tensor([[1.0, 2.0], [3.0, 4.0]]) print(x) ``` - **`torch.nn.Module`:** The base class for all neural network modules. Your custom models will inherit from this. - **Layers (e.g., `torch.nn.Linear`):** Pre-defined layers. `nn.Linear(in_features, out_features)` creates a fully connected layer. - **Activation Functions (e.g., `torch.nn.ReLU`, `torch.nn.Sigmoid`):** Found in `torch.nn` or `torch.nn.functional`. - **Loss Functions (e.g., `torch.nn.BCELoss`):** Quantify the difference between predictions and true labels. - **Optimizers (e.g., `torch.optim.SGD`, `torch.optim.Adam`):** Implement algorithms to update model weights based on gradients. - **`autograd`:** PyTorch’s automatic differentiation engine. ### 3. Building a Neural Network for Titanic with PyTorch (Practical) #### a) Data Preparation: Pandas to PyTorch Tensors (Practical - 15 min) ```python import torch import pandas as pd from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.compose import ColumnTransformer from sklearn.impute import SimpleImputer from sklearn.pipeline import Pipeline # Assume 'train_df' is your loaded Titanic training data # Dummy features and target data = { 'Pclass': [1, 2, 3, 1, 2, 3, 1, 2], 'Sex': ['male', 'female', 'male', 'female', 'male', 'female', 'male', 'female'], 'Age': [22, 38, 26, 35, 35, None, 54, 2], 'SibSp': [1, 1, 0, 1, 0, 0, 0, 3], 'Parch': [0, 0, 0, 0, 0, 0, 0, 1], 'Fare': [7.25, 71.2833, 7.925, 53.1, 8.05, 8.4583, 51.8625, 21.075], 'Embarked': ['S', 'C', 'S', 'S', 'S', 'Q', 'S', 'S'], 'Survived': [0, 1, 1, 1, 0, 0, 0, 1] } train_df = pd.DataFrame(data) # Separate target variable X = train_df.drop('Survived', axis=1) y = train_df['Survived'] # Define numerical and categorical features numerical_features = ['Age', 'Fare', 'SibSp', 'Parch'] categorical_features = ['Pclass', 'Sex', 'Embarked'] # Create preprocessing pipelines for numerical and categorical features numerical_pipeline = Pipeline([ ('imputer', SimpleImputer(strategy='mean')), ('scaler', StandardScaler()) ]) categorical_pipeline = Pipeline([ ('imputer', SimpleImputer(strategy='most_frequent')), ('onehot', OneHotEncoder(handle_unknown='ignore')) ]) # Create a column transformer to apply different transformations to different columns preprocessor = ColumnTransformer([ ('numerical', numerical_pipeline, numerical_features), ('categorical', categorical_pipeline, categorical_features) ]) # Preprocess the data X_processed = preprocessor.fit_transform(X) # Split data X_train_processed, X_val_processed, y_train_series, y_val_series = train_test_split( X_processed, y, test_size=0.2, random_state=42 ) # Convert to PyTorch Tensors X_train_tensor = torch.tensor(X_train_processed, dtype=torch.float32) y_train_tensor = torch.tensor(y_train_series.values, dtype=torch.float32).unsqueeze(1) X_val_tensor = torch.tensor(X_val_processed, dtype=torch.float32) y_val_tensor = torch.tensor(y_val_series.values, dtype=torch.float32).unsqueeze(1) ``` #### b) Defining the Neural Network (Practical - 20 min) ```python import torch.nn as nn class TitanicNet(nn.Module): def __init__(self, input_size, hidden_size1, output_size): super(TitanicNet, self).__init__() self.fc1 = nn.Linear(input_size, hidden_size1) self.relu1 = nn.ReLU() self.fc2 = nn.Linear(hidden_size1, output_size) self.sigmoid = nn.Sigmoid() def forward(self, x): out = self.fc1(x) out = self.relu1(out) out = self.fc2(out) out = self.sigmoid(out) return out hidden_size1 = 32 output_size = 1 model = TitanicNet(input_size, hidden_size1, output_size) ``` #### c) Defining Loss Function and Optimizer (Practical - 10 min) ```python criterion = nn.BCELoss() learning_rate = 0.001 optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate) ``` #### d) Training Loop (Practical - 25 min) ```python num_epochs = 200 batch_size = 16 train_losses = [] val_losses = [] val_accuracies = [] for epoch in range(num_epochs): model.train() outputs = model(X_train_tensor) loss = criterion(outputs, y_train_tensor) train_losses.append(loss.item()) optimizer.zero_grad() loss.backward() optimizer.step() model.eval() with torch.no_grad(): val_outputs = model(X_val_tensor) val_loss = criterion(val_outputs, y_val_tensor) val_losses.append(val_loss.item()) predicted_classes = (val_outputs > 0.5).float() correct_predictions = (predicted_classes == y_val_tensor).sum().item() total_predictions = y_val_tensor.size(0) accuracy = correct_predictions / total_predictions val_accuracies.append(accuracy) if (epoch + 1) % 20 == 0: print(f'Epoch [{epoch+1}/{num_epochs}], Train Loss: {loss.item():.4f}, Val Loss: {val_loss.item():.4f}, Val Accuracy: {accuracy:.4f}') ``` #### e) Evaluating the Model (Practical - 10 min) ```python import matplotlib.pyplot as plt plt.figure(figsize=(12, 5)) plt.subplot(1, 2, 1) plt.plot(train_losses, label='Training Loss') plt.plot(val_losses, label='Validation Loss') plt.xlabel('Epoch') plt.ylabel('Loss') plt.title('Training and Validation Loss') plt.legend() plt.subplot(1, 2, 2) plt.plot(val_accuracies, label='Validation Accuracy') plt.xlabel('Epoch') plt.ylabel('Accuracy') plt.title('Validation Accuracy') plt.legend() plt.tight_layout() plt.show() ``` #### f) Making Predictions on New Data (Conceptual - 5 min) ```python # Assume 'test_df' is loaded and preprocessed similarly to X_train # model.eval() # with torch.no_grad(): # test_predictions_probs = model(X_test_tensor) # test_predictions_classes = (test_predictions_probs > 0.5).int() ``` ### Advantages Revisited & Next Steps (Conceptual - 10 min) 1. Ease of Building: nn.Module and pre-built layers. 2. Automatic Gradients: Powerful and convenient. 3. Flexibility: Easy modifications. 4. GPU Acceleration: Significant speedups. ### Summary In this lesson, you’ve successfully built, trained, and evaluated a neural network for the Titanic survival prediction task using PyTorch. You’ve learned about core PyTorch components like Tensors, `nn.Module`, layers, loss functions, optimizers, and the `autograd` system. This provides a solid foundation for tackling more complex deep learning problems.