Deep Learning from Scratch
FC · RNN/LSTM · U-Net · GPT, from scratch
Four architectures built from first principles in PyTorch, with the trained checkpoints to prove it: the from-scratch GPT cuts WikiText-2 test perplexity from 269.4 to 178.5.
Shows what both trained checkpoints really wrote for this prompt.
Real output from the trained checkpoints, precomputed: both models loaded from disk and sampled with the coursework's own sampler (temperature 0.8, top-k 50, seed 7150), then copied verbatim. These are small language models trained on WikiText-2, so imperfect prose is the honest result. The coursework repo is private.
Real output from the trained checkpoints, precomputed with the coursework's own sampler. No live model call; provenance in gpt-samples.json.
Graduate deep-learning coursework presented as one piece: four architectures written from scratch in PyTorch rather than imported, with the trained checkpoints on disk to back every number. HW1 implements a fully-connected layer as a custom torch.autograd.Function with hand-derived forward and backward passes, verified against test fixtures and used to solve XOR and Iris. HW4 hand-codes the recurrent and LSTM cells (manual input/forget/output/cell gates) for character-level Shakespeare generation: a 66K-param RNN (hidden 128) and an 873K-param LSTM (hidden 256, 2 layers). HW3 builds a 36-class semantic segmenter twice: a 17-layer fully-convolutional baseline, then an 89-layer ResNet-18 encoder + U-Net decoder with skip connections, class-weighted label-smoothed loss, 8x D8 augmentation, test-time augmentation and a 600-epoch cosine schedule. HW5 is a decoder-only GPT (multi-head self-attention, sinusoidal positional encodings) trained on WikiText-2 with the GPT-2 BPE tokenizer (50,257-token vocab): scaling the 13.7M-param base to 30.5M (d_model 256, 8 heads, 6 layers, doubled context) cuts test perplexity from 269.4 to 178.5.
- Python
- PyTorch
- Transformers
- GPT-2 BPE
- WikiText-2
- ResNet
- U-Net
- LSTM
- Hugging Face Datasets
- GPT test perplexity
- 269.4 → 178.5
- Improved GPT params
- 30.5M
- From-scratch LSTM
- 873K params
- Segmentation
- 36 classes · 89-layer U-Net
What I'd improve
The improved GPT's five changes (width/FFN, depth, context length, dropout, and longer training with a smaller batch) shipped as one bundle, so the 269.4 to 178.5 gain is unattributed; an ablation per change is the first experiment I'd add. The segmentation work has the mirror-image gap: the 89-layer model trains with class weights, D8 augmentation and TTA, but no held-out IoU figure was saved to disk, so this card makes no results claim for it. Re-running eval to capture that number, plus lightweight experiment tracking across all four homeworks, is the next step.