Large Language Model Fine-Tuning
LLM fine-tuning adapts a pre-trained large language model to a specific domain or task by continuing training on curated examples. It covers full fine-tuning, parameter-efficient methods like LoRA, dataset preparation, and evaluation of the specialized model.
itArtificial intelligence and machine learning | OpenSkills.info
Intro
Large Language Model Fine-Tuning
Fine-tuning is the process of further training a pretrained language model on your own labeled examples so its weights shift toward your specific task or style. It exists for the cases where changing what you put into the model — the prompt, the retrieved context — isn't enough, and you need to change what's baked into the model itself.
That distinction is the whole subject. Prompting and retrieval-augmented generation shape a model's behavior at inference time, one request at a time. Fine-tuning shapes the model's weights once, so every future request already carries that behavior without needing to re-explain it.
Where fine-tuning fits in the adaptation ladder
Every major vendor converges on the same ordering, and it's worth taking seriously: try prompt engineering first, then retrieval, and reach for fine-tuning only once that cycle plateaus. OpenAI's guidance frames it directly — prompt engineering "may be all you need," and only after building evaluations to measure where prompting falls short does fine-tuning become the next step, typically when a task needs the model to consistently format responses a certain way or handle inputs no amount of prompt instruction reliably covers.
Microsoft Foundry's guidance gives concrete reasons to reach for fine-tuning specifically over the alternatives:
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://developers.openai.com/api/docs/guides/model-optimization
Supports
- Prompt engineering is the recommended starting point before other optimization methods
- Fine-tuning is positioned as a later step once prompting and evaluation reveal a persistent gap
- Fine-tuning is useful when the model needs to consistently format responses or handle inputs prompting cannot reliably cover
- https://developers.openai.com/api/docs/guides/supervised-fine-tuning
Supports
- Supervised fine-tuning trains on example input/output pairs to customize model behavior
- Training data uses JSONL format with a messages array matching the inference message format
- Minimum 10 training examples; recommended starting point around 50 well-crafted examples
- Workflow includes upload, job creation, evaluation against holdout data, and deployment via a fine-tuned model ID
- OpenAI is winding down its self-serve fine-tuning platform; new organizations cannot start jobs, existing customers' job creation is being phased out, but inference on existing fine-tuned models continues until base model deprecation
- https://developers.openai.com/api/docs/guides/fine-tuning-best-practices
Supports
- Model performance is bounded by the consistency of the people/process producing training data
- Mismatched training data distribution (e.g., too many refusals) degrades performance
- Training examples should match inference-time format
- Recommended optimization order is data quality first, then quantity, then hyperparameters (epochs, learning rate, batch size)
- Doubling dataset size is a recommended quantity-scaling step after quality is addressed
- https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/fine-tuning
Supports
- Microsoft Foundry offers three fine-tuning methods -- supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement fine-tuning (RFT)
- SFT trains on labeled input/output pairs and is best for most scenarios
- DPO aligns models with human-preferred responses
- RFT uses reward signals from model graders for complex behaviors
- Data format is JSONL with Chat Completions API message format; minimum 10 examples, 50-100+ recommended for noticeable improvement
- Fine-tuning is recommended over prompt engineering for higher quality, more examples than context allows, and lower per-request tokens/latency
- Fine-tuning is recommended over RAG when the goal is changing model behavior rather than augmenting factual knowledge
- Fine-tuning implementation is LoRA-based to reduce complexity
- https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini-supervised-tuning
Supports
- Supervised fine-tuning customizes Gemini models using labeled training examples
- Fine-tuning is recommended when standard prompting proves insufficient and quality training data is available
- Vertex AI supports parameter-efficient fine-tuning approaches including LoRA and QLoRA
- https://huggingface.co/docs/peft/en/index
Supports
- PEFT fine-tunes a small number of additional model parameters instead of all parameters, reducing computational and storage cost
- PEFT methods yield performance comparable to a fully fine-tuned model while being more accessible on limited hardware
- https://huggingface.co/docs/peft/en/conceptual_guides/lora
Supports
- LoRA freezes pretrained weights and adds trainable low-rank decomposition matrices (matrix A and matrix B) instead of updating the full weight matrix
- The weight update is the product of the two smaller low-rank matrices
- Rank controls the capacity of the low-rank matrices -- lower rank means fewer parameters and less expressive adaptation, higher rank means more parameters and more expressive adaptation
- QLoRA extends LoRA by quantizing the pretrained weights (commonly to 4-bit precision) before training low-rank adapters, reducing GPU memory requirements with minimal performance loss
- https://aws.amazon.com/blogs/aws/fine-tuning-for-anthropics-claude-3-haiku-model-in-amazon-bedrock-is-now-generally-available/
Supports
- Amazon Bedrock is described as the only fully managed service that provides the ability to fine-tune Claude models
- Fine-tuning for Claude 3 Haiku in Amazon Bedrock reached general availability