Andrej Karpathy's 200-line Python script trains a GPT model from scratch, learning statistical patterns in human names and generating plausible new ones. The script's core loop involves tokenizing, embedding, attending, computing, and predicting the next token, with the model learning to minimize cross-entropy loss through backpropagation and RMSNorm normalization.