Model Adaptation Strategies
Model adaptation strategies are techniques for customizing large language model behavior to suit specific tasks and domains. They include prompting, few-shot learning, retrieval-augmented generation, and fine-tuning, each suited to different requirements for knowledge, behavior change, and resource investment.
itArtificial intelligence and machine learning | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic — Model Adaptation Strategies
A pre-trained language model is a capable generalist until it meets an application with opinions about format, current documents, or both. Model adaptation closes that gap. The useful distinction is this: are you changing what the model can see, or how it behaves when it sees it? The rest is mostly consequences, wearing different hats.
Prompting changes input text at inference time. Instructions and output-format rules are the place to begin because they are quick to test and leave no permanent alteration. Add few-shot learning when a description is too vague: a few input-output examples demonstrate the pattern. Every example consumes context-window space, so an example collection can eventually become a small moving house inside every request.
Retrieval-Augmented Generation, or RAG, handles knowledge that is private, changing, or in need of attribution. It retrieves documents from an external store and places relevant material beside the prompt. Update the documents and the model can use new material without retraining. RAG is not a spell for changing habits, however. A retrieval system can hand over excellent facts and still receive an answer in an unwanted format.
Fine-tuning changes model weights through additional training, so a learned style, format, or task pattern persists. It is useful when prompting and examples do not produce consistent behavior. It is also an investment in curated examples, evaluation, compute, and future retraining. LoRA, a parameter-efficient fine-tuning method, narrows that investment by training small added matrices while the base model remains frozen. The model is not becoming magically omniscient. It is becoming reliably different in a chosen direction.
The practical combination is often fine-tuning for behavior and RAG for current knowledge. Read the Intro for the full spectrum and its trade-offs. Use the Slides when the architecture needs one mental picture. Keep the Cheatsheet nearby when choosing between prompts, examples, retrieval, and fine-tuning. The Quiz checks whether the distinction has survived contact with a multiple-choice question, a traditional and occasionally stern form of diplomacy.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.databricks.com/blog/rag-vs-fine-tuning
Supports
- RAG supplies retrieved knowledge at inference time, while fine-tuning changes a model through additional training.
- https://arxiv.org/html/2406.06399v1
Supports
- Model adaptation methods have different data, compute, evaluation, and maintenance requirements.
- https://arxiv.org/html/2403.01432v3
Supports
- Parameter-efficient fine-tuning methods adapt models by training a small subset of parameters or added modules.
- https://denser.ai/blog/rag-vs-fine-tuning/
Supports
- A practical comparison can use knowledge freshness, behavior change, and operating cost to choose between RAG and fine-tuning.
- https://machinelearningmastery.com/the-machine-learning-practitioners-guide-to-fine-tuning-language-models/
Supports
- Fine-tuning requires task-specific training data, model evaluation, and ongoing maintenance.
- https://arxiv.org/html/2408.01346v1
Supports
- LLM adaptation includes prompting, retrieval, full fine-tuning, and parameter-efficient alternatives.
- https://arxiv.org/abs/1706.03762
Supports
- The Transformer architecture was introduced in 2017.
- https://arxiv.org/abs/1801.06146
Supports
- ULMFiT established a practical transfer-learning approach using pretrained language models and task fine-tuning.
- https://arxiv.org/abs/1902.00751
Supports
- Adapter modules were proposed as a parameter-efficient transfer-learning method for NLP.
- https://arxiv.org/abs/2005.14165
Supports
- GPT-3 demonstrated few-shot task performance from text examples in the prompt.
- https://arxiv.org/abs/2005.11401
Supports
- RAG combines a parametric generator with external non-parametric memory.
- https://arxiv.org/abs/2101.00190
Supports
- Prefix-tuning keeps model parameters frozen while optimizing a task-specific prefix.
- https://arxiv.org/abs/2106.09685
Supports
- LoRA freezes pretrained weights and trains injected low-rank matrices.
- https://arxiv.org/abs/2305.14314
Supports
- QLoRA combines a frozen 4-bit quantized model with LoRA adapters.
- https://huggingface.co/docs/peft/index
Supports
- PEFT provides tooling for lightweight adapters such as LoRA.
- https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-fine-tuning.html
Supports
- Amazon SageMaker provides foundation-model fine-tuning workflows.
- https://cloud.google.com/vertex-ai/generative-ai/docs/models/tune-models
Supports
- Vertex AI provides tuning workflows for supported generative models.
- https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/fine-tuning
Supports
- Azure AI Foundry provides fine-tuning workflows for supported models.
- https://docs.together.ai/docs/fine-tuning-overview
Supports
- Together AI provides managed fine-tuning jobs.
- https://docs.fireworks.ai/fine-tuning/finetuning-intro
Supports
- Fireworks AI provides fine-tuning workflows.