Large Language Model Training and Inference
LLM training and inference covers the infrastructure, techniques, and engineering required to train large language models on massive text corpora and serve their predictions efficiently. It addresses distributed training, hardware selection, optimization, quantization, and serving architectures.
itArtificial intelligence and machine learning | OpenSkills.info
Intro
Large Language Model Training and Inference
This course is the deeper companion to LLM Foundations in this catalog. That course gives you the broad orientation: what an LLM is, the training stages at a high level, and how inference works in outline. This course goes further into two specific mechanics every AI engineer eventually runs into: how the size of a training run gets decided, how a raw base model becomes an aligned assistant, and what's actually happening — and what can be tuned — every time a model generates a response.
Pretraining at scale: what the scaling laws say
Training a large language model from scratch means choosing, ahead of time, how many parameters the model will have and how many tokens of text it will train on — and those two choices interact. For years, the field's default was to make models as large as the compute budget allowed, training them on whatever data was available without treating dataset size as an equally important lever.
DeepMind's 2022 "Chinchilla" scaling law study challenged that. Training over 400 models across a range of sizes, the researchers found that for a fixed compute budget, model size and training data should be scaled together, roughly in step — and that the field's existing large models were, by that measure, undertrained: too large for the amount of data they'd seen. They demonstrated it directly: Chinchilla, a 70-billion-parameter model trained on 1.4 trillion tokens, outperformed Gopher, a 280-billion-parameter model trained on roughly 300 billion tokens, despite using the same training compute. A model a quarter of Gopher's size, trained on four times more data, won.
The practical takeaway for anyone reasoning about model capability: parameter count alone doesn't tell you what a model can do. A smaller model trained compute-optimally on enough data can outperform a much larger one that was undertrained relative to its size — which is part of why raw parameter counts became a less reliable signal of model quality over time, and why current frontier models are trained on token counts far larger than early scaling assumptions suggested was needed.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://deepmind.google/blog/an-empirical-analysis-of-compute-optimal-large-language-model-training/
Supports
- Model size and training data should be scaled equally for optimal performance within a fixed compute budget
- Prior large models, including Gopher, were far too large for their compute budget and not trained on enough data
- A 4x smaller model trained on 4x more data would have been preferable for the compute used to train Gopher
- Chinchilla (70B parameters, 1.3-1.4T tokens) outperforms Gopher (280B parameters, ~300B tokens) on nearly every measured task at matched training compute
- https://arxiv.org/abs/2203.02155
Supports
- Three-stage RLHF process -- supervised fine-tuning on labeler demonstrations, reward model training on ranked outputs, reinforcement learning using the reward model
- The reinforcement learning stage uses proximal policy optimization (PPO) on the trained reward model
- Outputs from a 1.3B parameter InstructGPT model were preferred over outputs from the 175B parameter base GPT-3 model
- InstructGPT showed improvements in truthfulness and reductions in toxic output generation with minimal performance regressions on public NLP datasets
- https://huggingface.co/docs/transformers/main/en/generation_strategies
Supports
- Greedy search selects the next most likely token at each step and is the default decoding strategy
- Greedy search works well for short outputs but breaks down for longer sequences due to repetition
- Sampling (multinomial sampling) randomly selects a token based on the full probability distribution over the vocabulary
- Sampling reduces repetition and produces more creative and diverse outputs than greedy search
- Beam search tracks several candidate sequences and selects the one with the highest overall probability, suited to input-grounded tasks like translation or speech recognition
- https://huggingface.co/blog/how-to-generate
Supports
- Temperature adjusts the sharpness of the probability distribution; lowering it makes high-probability tokens more likely and approaches greedy decoding as it nears zero
- Top-k sampling filters to the k most probable next tokens and redistributes probability among them; a fixed k can exclude viable words in sharp distributions or include poor choices in flat distributions
- Top-p (nucleus) sampling chooses the smallest set of tokens whose cumulative probability exceeds a threshold p, adapting pool size to model confidence
- High-quality human language does not follow a distribution of purely high-probability next words, which is why greedy selection produces repetitive output
- https://huggingface.co/docs/transformers/main/en/llm_optims
Supports
- Loading a 70B parameter model requires roughly 256GB of memory for full precision weights, more than current GPUs (A100/H100, 80GB) provide
- A key-value (kv) cache stores past keys and values instead of recomputing them at each generation step
- Speculative decoding uses a smaller assistant model to generate candidate tokens verified by the larger model in a single forward pass, with no degradation in accuracy
- The assistant model should share the same tokenizer as the main model and be much smaller for the largest speedup
- Quantization reduces model weight precision (e.g., 8-bit or 4-bit) to lower memory usage, at the cost of a small amount of extra compute to quantize/dequantize
- Continuous batching groups incoming inference requests to maximize GPU utilization and throughput
- Self-attention compute and memory cost grows quadratically with input token count; FlashAttention-2 and SDPA reduce memory read/write operations to speed up inference