Introduction

Temperature is a parameter that controls the randomness of a language model's output by adjusting how the model selects its next token from among the possibilities it's considering. At its core, an LLM doesn't just predict a single "correct" next token — it calculates a probability distribution across its entire vocabulary, and temperature determines how boldly or conservatively the model samples from that distribution.

Temperature is one of the most commonly used output controls in generative AI, giving developers direct influence over whether a model behaves more predictably and focused, or more creative and varied, without requiring any changes to the underlying model itself.

Why Does Temperature Matter?

Temperature helps to:

• Control the balance between predictable and creative model outputs
• Adjust model behavior for different use cases without retraining
• Reduce or increase randomness in generated text as needed
• Influence consistency across repeated requests with the same prompt
• Provide a simple, direct lever for tuning generation style
• Work alongside other output controls like Top P and Top K to shape final results

Where Temperature Fits in the Generation Process

Whiteboard

Loading diagram...

How Temperature Works Mathematically (Simplified)

Before generating each token, the model produces raw scores (logits)
for every possible next token. Temperature divides these scores
before they're converted into probabilities:

adjusted_score = original_score / temperature

Temperature < 1.0 → sharpens differences → more predictable output
Temperature > 1.0 → flattens differences → more random/creative output
Temperature = 1.0 → uses the model's raw, unmodified probabilities

A Simple Illustration

Suppose the model is deciding the next word after "The sky is..."

Raw probabilities (illustrative):
"blue"   → 70%
"clear"  → 15%
"gray"   → 10%
"purple" →  5%

Low Temperature (e.g., 0.2):
"blue" becomes even more dominant (~95%+ chance) — highly predictable

High Temperature (e.g., 1.5):
Probabilities flatten out — "purple" becomes much more likely
to be chosen than it normally would be — more creative, less predictable

Temperature Value Ranges and Typical Behavior

Choosing Temperature Based on Use Case

Temperature = 0 vs Temperature = 1

Key Properties of Temperature

• Temperature adjusts the "sharpness" of the probability distribution before a token is sampled.
• Lower temperature values make output more focused, predictable, and repeatable.
• Higher temperature values make output more varied, creative, and less predictable.
• A temperature of 0 effectively makes generation deterministic, always choosing the top-probability token.
• Temperature is typically used alongside other controls like Top P and Top K for finer-grained control.

Where Is Temperature Used?

Advantages

• Provides a simple, intuitive lever for controlling output randomness
• Requires no model retraining — adjustable per request
• Allows the same underlying model to serve very different use cases
• Helps balance reliability against creativity based on specific needs
• Easy to combine with other sampling controls for fine-tuned behavior

Limitations

• Very low temperatures can produce overly repetitive or generic output
• Very high temperatures can lead to incoherent or nonsensical text
• The "right" temperature often requires experimentation for a given task
• Temperature alone doesn't guarantee factual accuracy, regardless of setting
• Effects can vary somewhat between different models and architectures

Real-World Examples

Best Practices

• Use low temperature for tasks requiring accuracy, consistency, or structured output.
• Use higher temperature for creative, exploratory, or brainstorming-style tasks.
• Test multiple temperature values for your specific use case rather than assuming a default is optimal.
• Combine temperature with Top P or Top K for more precise control over output variety.
• Avoid extremely high temperatures when coherence and correctness matter more than novelty.

Interview Tip

A common interview question is:

"What does temperature control in an LLM, and how would you choose a value for a factual Q&A application versus a creative writing tool?"

A strong answer is:

Temperature controls how randomly a model samples its next token by adjusting the sharpness of its probability distribution — lower temperature makes the model favor its highest-probability choices, producing more predictable output, while higher temperature flattens the distribution, allowing less likely tokens to be chosen more often, producing more varied and creative output. For a factual Q&A application, I'd use a low temperature, like 0.1 to 0.3, to prioritize accuracy and consistency, while for a creative writing tool, I'd use a higher temperature, like 0.8 to 1.2, to encourage more novel and varied output.

Giving concrete temperature values for both use cases makes your answer stronger.

Conclusion

Temperature provides a simple but powerful lever for shaping how predictable or creative a language model's output is, without requiring any changes to the model itself. Understanding how it works — and how to tune it for different use cases — sets up the next output control to explore: Top P, which offers a related but distinct way of shaping the token sampling process.