GPT = Generative Pretrained Transformer, a model that predicts the next token to generate text.

The core architecture is the Transformer, using attention and feed-forward layers to process input.

Tokens (text chunks) are turned into vectors via an embedding matrix, capturing meaning in high-dimensional space.

Attention blocks let tokens interact based on context; softmax turns final scores into probabilities.

Text generation = repeated prediction + sampling from probability distributions.

Most computations are just matrix multiplications; training adjusts the weights (parameters).

Temperature controls randomness in output (higher = more creative, lower = safer).

Source: https://www.youtube.com/watch?v=wjZofJX0v4M