Deep Learning Interview Questions for Freshers

Backpropagation computes gradients of the loss with respect to all parameters using the chain rule. Forward pass: compute predictions and loss. Backward pass: starting from loss, compute ∂L/∂output, propagate backwards through each layer using chain rule: ∂L/∂W = ∂L/∂output × ∂output/∂W. Gradients accumulate in .grad attributes. Then optimizer updates: W = W - lr × ∂L/∂W. Key insight: gradients flow backward through the same computational graph used in forward pass. PyTorch autograd builds this graph dynamically — every tensor operation records itself for backward computation.

Dropout randomly zeros out neuron activations with probability p during training, forcing the network to learn redundant representations. During inference, no dropout — activations scaled by (1-p) to match expected values. Why it works: prevents co-adaptation (neurons relying on specific other neurons), approximates ensemble of 2^n different networks. Use dropout: (1) After dense/linear layers (p=0.1–0.5), (2) Not after batch norm (conflicting noise), (3) Less effective in CNNs (use instead: DropBlock), (4) Transformers use attention dropout + residual dropout (p=0.1). Too high dropout (>0.5) causes underfitting — tune on validation set.

Underfitting (high bias): model too simple, high training loss, high validation loss — both similar. Fix: increase model capacity, train longer, reduce regularization, better features. Overfitting (high variance): model memorized training data, low training loss, high validation loss — large gap. Fix: more data, dropout, L2 regularization, data augmentation, early stopping, reduce model size. Diagnosis: plot training vs validation loss curves over epochs. Ideal: both decrease, then validation plateaus. Use validation set (not test set!) for all tuning decisions — the test set is touched exactly once for final evaluation.

CNN processes images through: (1) Convolutional layers — learnable filters slide over input, detect local patterns (edges → textures → shapes → objects), (2) ReLU activation — non-linearity after each conv, (3) Pooling — MaxPool/AvgPool reduces spatial dimensions, builds spatial invariance, (4) Global Average Pooling — collapses spatial dimensions before dense layers, (5) Dense layers + softmax — final classification. Key properties: parameter sharing (same filter across all positions), local connectivity, translation equivariance. Modern CNNs: ResNet (skip connections for very deep networks), EfficientNet (compound scaling of depth/width/resolution).

Early stopping monitors a validation metric and stops training when it stops improving, saving the best checkpoint. Implementation: track best_val_loss, counter for patience. Each epoch: if val_loss < best_val_loss: save model, reset counter; else: counter++; if counter >= patience: break. Use patience=5–10 epochs. Save with torch.save(model.state_dict(), "best_model.pt") at each improvement. Load best model after training ends. Important: checkpoint the model state, not just the metric — you'll restore to the best point, not the last point. Use with ReduceLROnPlateau for automatic LR reduction before stopping.