Handwritten notes on the Transformer architecture: self-attention, multi-head attention, positional encoding, and sampling strategies for text generation.
Pre-training, scaling laws, architectures, fine-tuning, and alignment · 4 pages total
Notes on LLM pre-training pipelines, multi-GPU distributed training with DDP vs. FSDP, Chinchilla scaling laws, and pre-training for domain adaptation.
Classical ML, optimization algorithms, probabilistic models, and loss functions · 147 pages total
Handwritten notes covering probability theory for ML: random variables, Bayes' theorem, distributions, expectation, variance, and probabilistic reasoning.
Handwritten notes on statistics for ML: descriptive statistics, hypothesis testing, confidence intervals, regression analysis, and maximum likelihood estimation.
Inference serving, memory bandwidth, hardware accelerators, and GPU execution.