Fine-tuning is further training of an existing, already trained model on a smaller set of examples for a specific task. The model keeps its general abilities and learns to behave in a particular way more reliably: follow a strict output format, classify documents into your categories, write in a house style, or handle a narrow technical vocabulary. It changes the model’s weights, which is what separates it from prompting and retrieval.
Fine-tuning starts from a foundation model that someone else has already trained, so it needs far less data and compute than training a model from scratch.
How Fine-Tuning Works
- Define the behaviour you want and how you will measure it.
- Prepare examples: usually hundreds to thousands of input and ideal-output pairs, drawn from real cases and reviewed by people who know the domain.
- Train: run a training job on a base model, either through a vendor’s hosted service or on your own hardware with an open-weight model.
- Evaluate: compare the fine-tuned model with the base model plus a good prompt on a held-out test set.
- Deploy and monitor: serve the new model version and watch for drift as real inputs change.
Main Methods
- Supervised fine-tuning (SFT): train on examples of correct outputs. Best for formats, classification and consistent instruction following.
- Preference tuning: train on pairs of a better and a worse answer, so the model learns which to prefer. Direct preference optimisation (DPO) is a common method, used for tone, style and summary focus.
- Reinforcement fine-tuning: the model’s outputs are scored by a grader and training rewards higher scores. Useful when correctness can be checked automatically, as in some reasoning and code tasks.
- Parameter-efficient fine-tuning (PEFT): instead of updating every weight, train a small number of added parameters. LoRA (Low-Rank Adaptation), the most widely used method, freezes the original weights and trains small low-rank matrices; its authors reported 10,000 times fewer trainable parameters and three times less GPU memory than full fine-tuning on a very large model (LoRA, arXiv 2106.09685). Because the result is a small add-on file (an adapter), one base model can serve several fine-tuned variants.
When to Fine-Tune and When Not To
Vendor guidance is consistent: start with evaluations and prompting, and treat fine-tuning as an optional later step. OpenAI’s optimisation guide says prompt engineering “may be all you need” for many tasks (OpenAI).
| Need | Better first tool |
|---|---|
| The model must know your current documents and facts | Retrieval-augmented generation (RAG) |
| Output must match a fixed format | Structured outputs, then fine-tuning if still unreliable |
| Consistent style, tone or classification at high volume | Fine-tuning |
| Lower cost or latency for a narrow task | Fine-tuning a smaller model, or distillation from a larger one |
| Data that must not leave your network | Fine-tuning an open-weight model you host |
Fine-tuning is a poor way to teach facts. Knowledge baked into weights cannot be cited, goes stale, and cannot be updated without retraining. Facts belong in retrieval; behaviour belongs in fine-tuning.
Risks and Failure Modes
- Catastrophic forgetting: the model gets better at the narrow task and worse at general instructions it used to follow.
- Bad examples: errors, bias or personal data in the training set are learned faithfully.
- Overfitting: strong scores on examples similar to training data, weak on real variety.
- Weakened safety behaviour: fine-tuning can reduce a model’s built-in refusals, so safety tests must be rerun.
- Platform dependence: hosted fine-tuning ties you to one vendor’s model and service. The same OpenAI guide now states that OpenAI is winding down its fine-tuning platform and that new users can no longer access it, a reminder to plan for portability.
- Maintenance: when the base model is retired or improved, the fine-tune must be redone and re-evaluated.
Related Terms
Fine-tuning is one of several ways to adapt a large language model. Model distillation trains a small model on a large model’s outputs. Synthetic data is often used to fill gaps in training examples. Every fine-tuning decision depends on LLM evaluation to show it beat the simpler options.
Fine-Tuning in Practice
Fine-tuning is the last option tried, after evaluations show that prompting and retrieval have stopped improving on a measured task. When it is used, the usual path is parameter-efficient tuning such as LoRA on an open-weight model, which keeps the result portable and lets it run in the client’s own environment when data residency requires it. Training examples come from cases the client has already checked, and the base model with its best prompt stays in the evaluation suite as the baseline each tuned version must beat. See machine learning development services.