Introduction

A custom metric is a user-defined measurement used to monitor and report a model's performance during training and evaluation, distinct from the loss function that actually drives weight updates through backpropagation. While the previous topic covered customizing what a model optimizes for, custom metrics address a related but separate need: customizing what gets measured and reported to help you understand how well a model is actually performing, in terms that matter for your specific task.

Keras provides many built-in metrics (accuracy, precision, recall, and more), but real-world applications often need specialized, business-relevant, or research-specific measurements that aren't available out of the box — which is exactly what custom metrics are designed to provide.

Why Do Custom Metrics Matter?

Custom metrics help to:

• Measure exactly the aspects of performance that matter most for a specific task
• Provide clearer, more interpretable insight into model behavior than loss value alone
• Track specialized, domain-relevant measurements during training and evaluation
• Support metrics that need to accumulate state across an entire epoch, not just a single batch
• Complement custom loss functions with equally tailored performance reporting
• Bridge the gap between what a model is trained on and what stakeholders actually care about

Loss vs Metrics: An Important Distinction

Whiteboard

This is a critical distinction: the loss function is what the
optimizer actually uses to update weights via backpropagation,
while metrics are purely for monitoring and reporting — they
have no direct effect on how the model trains, even though
both are often computed from the same predictions and targets.

A model could, in principle, be trained to minimize one loss
while being monitored with several completely different metrics.

Two Ways to Define a Custom Metric

1. As a Simple Function

Simple function-based metrics work well for straightforward calculations that can be computed fresh from each batch, without needing to track any running state across an entire epoch.

2. As a Subclass of keras.metrics.Metric (Stateful Metrics)

The class-based approach is necessary when a metric needs to accumulate values correctly across an entire epoch (like a running average or a count-based ratio), rather than being recalculated independently for each individual batch.

Why Some Metrics Need State (and Others Don't)

Some metrics, like a simple per-batch accuracy calculation,
can be computed freshly each time without needing any memory
of previous batches.

Other metrics — like precision, recall, or the tolerance-based
accuracy above — genuinely need to accumulate correct counts
and total counts ACROSS an entire epoch to be calculated
correctly, since computing them independently per batch and
then averaging the results can produce a subtly different
(and often incorrect) final number.

This is why Keras provides the stateful Metric class: update_state()
accumulates values batch by batch, result() computes the final
metric from that accumulated state, and reset_state() clears
it at the start of each new epoch.

A Practical Example: Business-Relevant Metric

Scenario: For a churn prediction model, the business cares
specifically about "recall at the top 10% highest-risk
customers" — i.e., of the customers actually most likely to
churn, how many did the model correctly flag among its top
10% riskiest predictions?

No standard built-in metric measures exactly this, so a
custom metric is needed to track and report this specific,
business-relevant number during training and evaluation.

Using Multiple Metrics Simultaneously

Keras allows mixing built-in metric names (as strings), built-in metric objects, and custom metrics together in the same metrics list, all reported simultaneously during training and evaluation.

Custom Loss vs Custom Metrics

Function-Based vs Class-Based (Stateful) Metrics

Key Properties of Custom Metrics

• Metrics are used purely for monitoring and reporting — unlike loss, they don't affect backpropagation or weight updates.
• Simple metrics can be defined as plain functions; metrics needing accumulated state require subclassing keras.metrics.Metric.
• Stateful metrics implement update_state(), result(), and reset_state() to correctly track values across an epoch.
• Metrics don't need to be differentiable, unlike loss functions, since they aren't used in gradient computation.
• Multiple built-in and custom metrics can be tracked simultaneously during training and evaluation.

Where Are Custom Metrics Used?

Advantages

• Provides visibility into exactly the performance aspects that matter for a specific task
• Doesn't require differentiability, offering much more flexibility than loss functions
• Stateful metrics correctly handle epoch-level calculations like precision and recall
• Can be combined freely with built-in metrics for comprehensive performance monitoring
• Helps translate raw loss values into more interpretable, stakeholder-relevant numbers

Limitations

• Stateful metrics require more implementation effort than simple functions
• Incorrectly implemented state accumulation can produce subtly wrong reported values
• Custom metrics add complexity that requires careful testing to trust the reported numbers
• Too many tracked metrics can clutter training logs and make monitoring harder to interpret
• Metrics alone don't influence training — a poorly chosen loss function can't be fixed by better metrics

Real-World Examples

Best Practices

• Use built-in metrics whenever they adequately capture what you need to monitor.
• Use the stateful keras.metrics.Metric class for any metric involving ratios accumulated across an epoch (precision, recall, etc.).
• Keep custom metrics distinct in purpose from the loss function — metrics inform understanding, loss drives training.
• Validate custom metric calculations against known, manually computed values before trusting them in production.
• Track a focused, meaningful set of metrics rather than overwhelming logs with excessive, redundant measurements.

Interview Tip

A common interview question is:

"What is the difference between a loss function and a metric in Keras, and why might you need a stateful custom metric class instead of a simple function?"

A strong answer is:

A loss function is what the optimizer actually uses to compute gradients and update a model's weights during training, so it must be differentiable, while a metric is used purely for monitoring and reporting performance and has no direct effect on training — it doesn't even need to be differentiable. A stateful metric class becomes necessary for measurements like precision or recall, which need to accumulate correct counts and total counts across an entire epoch to be calculated accurately; computing these independently per batch and then averaging the results can produce a subtly incorrect final number, which is why Keras's keras.metrics.Metric class provides update_state(), result(), and reset_state() to handle this accumulation correctly.

Explaining specifically why per-batch averaging can be wrong for ratio-based metrics makes your answer stronger.

Conclusion

Custom metrics provide tailored, task-relevant insight into a model's performance, complementing custom loss functions by focusing purely on monitoring and reporting rather than driving the training process itself. With custom layers, loss functions, and metrics now all covered, the next topic explores the custom training loop — the most flexible level of customization, giving complete control over the entire training process rather than working within .fit()'s standard structure.