What Is Fine-Tuning?

Fine-tuning is further training of an existing, already trained model on a smaller set of examples for a specific task. The model keeps its general abilities and learns to behave in a particular way more reliably: follow a strict output format, classify documents into your categories, write in a house style, or handle a narrow technical vocabulary. It changes the model’s weights, which is what separates it from prompting and retrieval.

Fine-tuning starts from a foundation model that someone else has already trained, so it needs far less data and compute than training a model from scratch.

How Fine-Tuning Works

  1. Define the behaviour you want and how you will measure it.
  2. Prepare examples: usually hundreds to thousands of input and ideal-output pairs, drawn from real cases and reviewed by people who know the domain.
  3. Train: run a training job on a base model, either through a vendor’s hosted service or on your own hardware with an open-weight model.
  4. Evaluate: compare the fine-tuned model with the base model plus a good prompt on a held-out test set.
  5. Deploy and monitor: serve the new model version and watch for drift as real inputs change.

Main Methods

  • Supervised fine-tuning (SFT): train on examples of correct outputs. Best for formats, classification and consistent instruction following.
  • Preference tuning: train on pairs of a better and a worse answer, so the model learns which to prefer. Direct preference optimisation (DPO) is a common method, used for tone, style and summary focus.
  • Reinforcement fine-tuning: the model’s outputs are scored by a grader and training rewards higher scores. Useful when correctness can be checked automatically, as in some reasoning and code tasks.
  • Parameter-efficient fine-tuning (PEFT): instead of updating every weight, train a small number of added parameters. LoRA (Low-Rank Adaptation), the most widely used method, freezes the original weights and trains small low-rank matrices; its authors reported 10,000 times fewer trainable parameters and three times less GPU memory than full fine-tuning on a very large model (LoRA, arXiv 2106.09685). Because the result is a small add-on file (an adapter), one base model can serve several fine-tuned variants.

When to Fine-Tune and When Not To

Vendor guidance is consistent: start with evaluations and prompting, and treat fine-tuning as an optional later step. OpenAI’s optimisation guide says prompt engineering “may be all you need” for many tasks (OpenAI).

NeedBetter first tool
The model must know your current documents and factsRetrieval-augmented generation (RAG)
Output must match a fixed formatStructured outputs, then fine-tuning if still unreliable
Consistent style, tone or classification at high volumeFine-tuning
Lower cost or latency for a narrow taskFine-tuning a smaller model, or distillation from a larger one
Data that must not leave your networkFine-tuning an open-weight model you host

Fine-tuning is a poor way to teach facts. Knowledge baked into weights cannot be cited, goes stale, and cannot be updated without retraining. Facts belong in retrieval; behaviour belongs in fine-tuning.

Risks and Failure Modes

  • Catastrophic forgetting: the model gets better at the narrow task and worse at general instructions it used to follow.
  • Bad examples: errors, bias or personal data in the training set are learned faithfully.
  • Overfitting: strong scores on examples similar to training data, weak on real variety.
  • Weakened safety behaviour: fine-tuning can reduce a model’s built-in refusals, so safety tests must be rerun.
  • Platform dependence: hosted fine-tuning ties you to one vendor’s model and service. The same OpenAI guide now states that OpenAI is winding down its fine-tuning platform and that new users can no longer access it, a reminder to plan for portability.
  • Maintenance: when the base model is retired or improved, the fine-tune must be redone and re-evaluated.

Fine-tuning is one of several ways to adapt a large language model. Model distillation trains a small model on a large model’s outputs. Synthetic data is often used to fill gaps in training examples. Every fine-tuning decision depends on LLM evaluation to show it beat the simpler options.

Fine-Tuning in Practice

Fine-tuning is the last option tried, after evaluations show that prompting and retrieval have stopped improving on a measured task. When it is used, the usual path is parameter-efficient tuning such as LoRA on an open-weight model, which keeps the result portable and lets it run in the client’s own environment when data residency requires it. Training examples come from cases the client has already checked, and the base model with its best prompt stays in the evaluation suite as the baseline each tuned version must beat. See machine learning development services.

Back to Glossary

Let's talk about your project

We'll help you harness AI and ML technologies to create innovative software solutions

File requirements: pdf, doc, docx, rtf, ppt, pptx — max 15 MB

No file selected