Introduction

The chain rule is a fundamental calculus technique used to differentiate composite functions — functions made up of one function nested inside another. It allows us to break down the derivative of a complex expression into the product of simpler derivatives, making it possible to differentiate functions that would otherwise be difficult to handle directly.

The chain rule is especially important in machine learning, where it forms the mathematical foundation of backpropagation, allowing gradients to be computed layer by layer through deep neural networks.

Why is the Chain Rule Important?

The chain rule helps to:

• Differentiate composite (nested) functions
• Break complex derivatives into simpler, manageable steps
• Power backpropagation for training neural networks
• Compute gradients through multiple layers of transformations
• Support optimization of functions built from combined operations
• Enable automatic differentiation used in deep learning frameworks

Applying the Chain Rule

Whiteboard

Loading diagram...

The Chain Rule Formula

For a composite function y = f(g(x)):

dy/dx = f'(g(x)) × g'(x)

In words: differentiate the outer function (keeping the inner function unchanged), then multiply by the derivative of the inner function.

Step-by-Step Example

Find the derivative of y = (3x + 1)²

Let outer function: f(u) = u²        →  f'(u) = 2u
Let inner function: u = g(x) = 3x+1  →  g'(x) = 3

Apply chain rule:
dy/dx = f'(g(x)) × g'(x)
      = 2(3x+1) × 3
      = 6(3x+1)
      = 18x + 6

Another Example (with Trigonometric Function)

Find the derivative of y = sin(x²)

Outer function: f(u) = sin(u)   →  f'(u) = cos(u)
Inner function: u = x²          →  g'(x) = 2x

dy/dx = cos(x²) × 2x = 2x·cos(x²)

Chain Rule with Multiple Nested Functions

The chain rule extends to functions nested more than twice by multiplying the derivative of each layer:

If y = f(g(h(x))), then:
dy/dx = f'(g(h(x))) × g'(h(x)) × h'(x)

This is exactly how backpropagation computes gradients across multiple layers of a neural network.

The Chain Rule in Backpropagation

In a neural network, the output is a composition of many functions (layers). The chain rule allows the gradient of the loss with respect to each weight to be computed by multiplying the local gradients at every layer, working backward from the output to the input.

∂Loss/∂w = ∂Loss/∂output × ∂output/∂hidden × ∂hidden/∂w

Key Properties of the Chain Rule

• The chain rule applies whenever a function is composed of two or more nested functions.
• Each layer's derivative is calculated independently, then multiplied together.
• The chain rule can be extended to any number of nested functions.
• In multivariable calculus, the chain rule generalizes using partial derivatives and the gradient.
• It is the mathematical basis for automatic differentiation in deep learning frameworks.

Chain Rule vs Other Differentiation Rules

Where is the Chain Rule Used?

Advantages

• Simplifies differentiation of complex, nested functions
• Forms the mathematical foundation of backpropagation in deep learning
• Enables efficient gradient computation through automatic differentiation
• Scales naturally to functions with many layers of composition
• Applicable to both single-variable and multivariable functions

Limitations

• Can become computationally intensive with many nested layers
• Requires correctly identifying the outer and inner functions
• Errors in identifying function composition can lead to incorrect derivatives
• Numerical instability can arise in very deep chains (e.g., vanishing/exploding gradients in deep networks)
• Requires each individual function in the chain to be differentiable

Real-World Examples

Best Practices

• Clearly identify the outer and inner functions before differentiating.
• Work through the derivative step-by-step, layer by layer, for complex compositions.
• Double-check by simplifying the original function first, if possible, to verify your answer.
• Understand the chain rule deeply before studying backpropagation, since it directly relies on it.
• Use automatic differentiation tools for very deep or complex nested functions in practice.

Interview Tip

A common interview question is:

"What is the chain rule, and how is it used in backpropagation?"

A strong answer is:

The chain rule is used to differentiate composite functions, where dy/dx = f'(g(x)) × g'(x) for y = f(g(x)). In backpropagation, a neural network's output is a composition of many layered functions, so the chain rule is applied repeatedly to compute the gradient of the loss with respect to each weight by multiplying the local gradients at every layer, working backward from the output to the input.

Mentioning the layer-by-layer multiplication and its direct role in backpropagation makes your answer stronger.

Conclusion

The chain rule is a foundational calculus technique for differentiating composite functions, breaking complex derivatives into simpler, multiplicable parts. Its role extends far beyond pure mathematics — it is the core mechanism that makes backpropagation possible, enabling neural networks to learn by efficiently computing gradients across every layer.