Instruction Tuning

A base model doesn't answer questions — it continues text. Ask it "What is photosynthesis?" and it might reply with more questions, because that's what its training data looked like. Instruction tuning is what fixes that: train the model on (instruction → response) pairs until it learns to follow instructions instead of merely predicting the next token. It's the single step that turns a text predictor into an assistant.

💡 In one line: Instruction tuning trains a base model on instruction-response pairs so it follows instructions instead of just continuing text.

The Problem It Solves

A base (pretrained) model only learned to predict the next token. So it:

  • Continues your prompt rather than answering it.
  • Ignores the intent behind a request.
  • Needs elaborate few-shot prompting to behave.

What is Instruction Tuning?

It's supervised fine-tuning (SFT) on a dataset of (instruction, response) pairs — often across many diverse tasks. The model learns the general skill of "follow what's asked," and — crucially — it generalises to instructions it never saw in training.

(You'll see it called SFT or instruction fine-tuning — same idea.)

The Data Format

Each example pairs an instruction with an ideal response:

json
{
  "instruction": "Summarize the following text in one sentence.",
  "input": "Photosynthesis is the process by which...",
  "output": "Plants convert sunlight into chemical energy."
}

Modern datasets use the chat format — system / user / assistant turns — and training usually masks the loss so the model learns to produce the response, not the prompt.

The Pipeline

Whiteboard
Whiteboard diagram


Base → instruction-tuned → aligned is the standard recipe behind today's chat models.

Where It Sits in the Stack

  • Pretraining → knows language.
  • Instruction tuning (SFT) → becomes helpful.
  • Alignment (RLHF / DPO) → becomes safe and preferred.

"Base" vs. "Instruct" in a model name (e.g. Llama-3-8B vs. Llama-3-8B-Instruct) is exactly this difference — always start from the Instruct variant unless you're doing the instruction tuning yourself.

Well-Known Datasets

FLAN (Google's multi-task collection), Alpaca (52K examples generated by an LLM — the self-instruct approach), Dolly (human-written), OpenAssistant, and ShareGPT-style conversations.

Task Diversity Is the Key

The magic ingredient isn't volume — it's diversity. Training across many different task types (summarise, classify, translate, extract, reason) teaches the meta-skill of instruction-following, which then transfers to unseen tasks. A narrow dataset teaches one task; a diverse one teaches following instructions.

Quality Beats Quantity

LIMA demonstrated the point: 1,000 carefully curated examples produced a strong instruction-follower — outperforming far larger but noisier datasets. A few thousand excellent examples beat a hundred thousand mediocre ones.

Instruction Tuning vs. Task Fine-Tuning

  • Instruction tuning — many tasks → general instruction-following.
  • Task fine-tuning — one task → a specialist (better at that task, worse at flexibility).

Doing It Yourself

You'd instruction-tune when you need a custom assistant persona, a domain-specific helper, or you're starting from a base model. In practice you'd use LoRA rather than full fine-tuning — same benefit, far cheaper.

Best Practices

  • Prioritise diverse, high-quality examples over raw volume.
  • Use the chat template the model expects.
  • Mask the loss on prompts; train on responses.
  • Hold out an eval set of unseen instruction types.
  • Start from an Instruct model unless you have a reason not to.

Summary

  • Instruction tuning trains a base model on (instruction, response) pairs.
  • It converts a text continuer into an instruction follower — the birth of the assistant.
  • Task diversity is what makes it generalise to unseen instructions.
  • Quality beats quantity (LIMA: ~1,000 great examples go a long way).
  • It's the SFT step in base → instruct → aligned (RLHF/DPO).Â