Large Language Model Fine-Tuning

Fine-tuning is the process of further training a pretrained language model on your own labeled examples so its weights shift toward your specific task or style. It exists for the cases where changing what you put into the model — the prompt, the retrieved context — isn't enough, and you need to change what's baked into the model itself.

That distinction is the whole subject. Prompting and retrieval-augmented generation shape a model's behavior at inference time, one request at a time. Fine-tuning shapes the model's weights once, so every future request already carries that behavior without needing to re-explain it.

Where fine-tuning fits in the adaptation ladder

Every major vendor converges on the same ordering, and it's worth taking seriously: try prompt engineering first, then retrieval, and reach for fine-tuning only once that cycle plateaus. OpenAI's guidance frames it directly — prompt engineering "may be all you need," and only after building evaluations to measure where prompting falls short does fine-tuning become the next step, typically when a task needs the model to consistently format responses a certain way or handle inputs no amount of prompt instruction reliably covers.

Microsoft Foundry's guidance gives concrete reasons to reach for fine-tuning specifically over the alternatives:

• Over prompt engineering — when you need higher quality than prompting alone delivers, need to train on more examples than fit in a context window, or want to cut per-request token costs and latency by baking instructions into the weights instead of repeating them in every prompt.
• Over RAG — when the problem is the model's behavior (tone, format, task execution) rather than its knowledge. Retrieval augments what the model knows; fine-tuning changes how it behaves. A support bot that needs to know your current return policy needs retrieval. A support bot that keeps answering in the wrong format despite clear instructions needs fine-tuning.

Reaching for fine-tuning before exhausting prompting and retrieval is a common and expensive mistake: it requires curating a labeled dataset, running and monitoring a training job, and then evaluating and redeploying — real engineering cost that prompt iteration doesn't carry.

Full fine-tuning versus parameter-efficient fine-tuning

Full fine-tuning updates every weight in the model. It offers the most flexibility but requires enough compute to hold gradients and optimizer state for the entire model, which is prohibitively costly for models with billions of parameters.

Parameter-efficient fine-tuning (PEFT) freezes the pretrained weights and trains a small number of additional parameters instead, reaching performance comparable to full fine-tuning at a fraction of the compute and storage cost — the approach behind most production fine-tuning today, including what Vertex AI and Microsoft Foundry use under the hood.

The dominant PEFT technique is LoRA (Low-Rank Adaptation). Instead of updating a full weight matrix, LoRA freezes it and injects a pair of small, trainable low-rank matrices alongside it; the update to the frozen weight is the product of those two smaller matrices. A low rank means fewer trainable parameters and a smaller adapter; a higher rank gives the adapter more capacity to represent complex adaptations, at the cost of more parameters to train. Because the original weights never change, you can swap different LoRA adapters onto the same base model for different tasks without retraining from scratch.

QLoRA extends LoRA by quantizing the frozen pretrained weights — typically to 4-bit precision — before training the low-rank adapters on top. The quantization sharply cuts the GPU memory needed to hold the base model, which is what makes fine-tuning large models feasible on a single consumer or single-node GPU rather than a multi-GPU cluster.

Three training methods

OpenAI and Microsoft Foundry both organize fine-tuning around the same three methods, because they solve three different problems:

• Supervised fine-tuning (SFT) — train on labeled input/output pairs so the model learns to reproduce a demonstrated behavior. This is the default choice and the best fit for most tasks: consistent formatting, domain-specific tone, structured extraction.
• Direct preference optimization (DPO) — train on pairs of responses where one is marked preferred over the other, so the model learns to favor the kind of response people rate higher. This targets response quality and preference rather than reproducing an exact example.
• Reinforcement fine-tuning (RFT) — train against a reward signal from a grading model rather than fixed labeled examples, letting the model optimize for a defined objective on tasks too complex to hand-label exhaustively. It runs longer and costs more than SFT because it involves generating and grading many rollouts during training.

SFT is where nearly every fine-tuning project starts. DPO and RFT are for cases where "reproduce this example" isn't precise enough to capture what "better" means.

Preparing training data

Training data for supervised fine-tuning is a set of input/output examples, typically formatted as JSON Lines (JSONL) with one example per line, each containing a conversation in the same message format used at inference time.

Size guidance is consistent across vendors: 10 examples is a technical minimum, but real improvement usually starts around 50–100 well-curated examples, and most tasks find a sweet spot in the hundreds. Beyond a few thousand examples, returns diminish unless the task itself is genuinely broad.

Quality matters more than quantity. OpenAI's best-practice guidance is explicit that model performance is bounded by the consistency of the people or process that produced the training data — inconsistent labeling, mismatched formatting between training examples and real inference calls, or a training set whose distribution doesn't match production traffic (for example, far more refusals in training data than the task actually needs in practice) all degrade the tuned model. The recommended order of operations is to fix data quality first, then scale up quantity, and only then tune hyperparameters like learning rate and epoch count — in that order, not the reverse.

Evaluating a fine-tuned model

A fine-tuned model needs the same rigor as any other model change: compare it against the base model on a held-out evaluation set it never trained on, not just the training data it was optimized against. Track checkpoints across training epochs to catch overfitting, where the model memorizes training examples rather than generalizing the underlying pattern. See the LLM evaluation course in this catalog for the grading methods and evaluation design that apply here as much as anywhere else.

The vendor landscape is shifting

This is worth knowing before you plan around it: OpenAI is winding down its self-serve fine-tuning platform. New organizations can no longer start fine-tuning jobs, and existing customers' ability to create new jobs is being phased out on a published timeline, though inference on already fine-tuned models continues until the underlying base model is deprecated. OpenAI's own stated reasoning is that newer base models follow instructions and formats well enough that prompting closes most of the gap fine-tuning used to fill.

That shift doesn't mean fine-tuning is disappearing — it means the center of gravity is moving. Google Cloud's Vertex AI and Microsoft Foundry both continue to offer managed supervised fine-tuning (including LoRA-based tuning) for their respective model families. Anthropic does not offer a self-serve fine-tuning API for Claude directly; Amazon Bedrock is the fully managed service AWS documents for fine-tuning select Claude models — not something you self-serve from the Claude API console.

Check current vendor documentation before committing to a fine-tuning plan built on a specific platform; availability here changes faster than most areas of this stack.

Where this course stops

This is an orientation to why fine-tuning exists, where it fits relative to prompting and retrieval, and the vocabulary you need to evaluate a specific vendor's fine-tuning offering: full versus parameter-efficient tuning, LoRA and QLoRA, and SFT, DPO, and RFT as training methods. It does not walk through a specific vendor's fine-tuning console end to end, or the underlying training mathematics. Those depend on your platform and change quickly enough that the links in this course point to where to get current, vendor-specific detail.