Introduction

Function calling is a capability that lets a language model recognize when a user's request requires executing a specific, predefined function — like checking the weather, querying a database, or performing a calculation — and respond by generating structured JSON specifying exactly which function to call and with what arguments, rather than attempting to answer the question directly from its own knowledge. Building directly on the JSON output concept covered in the previous topic, function calling applies structured output specifically to the problem of connecting a model's reasoning to real, executable code.

Critically, the model itself never actually executes the function — it only identifies the right function and generates the correctly structured arguments; the developer's application code is responsible for actually running that function and, typically, feeding the result back to the model to incorporate into its final response.

Why Does Function Calling Matter?

Function calling helps to:

• Bridge the gap between a model's language understanding and real, executable application logic
• Allow a model to reliably request specific actions with correctly structured, typed arguments
• Reduce reliance on fragile text parsing to figure out what a user actually wants done
• Provide the technical foundation for tool calling and agentic AI systems, covered next
• Enable models to access capabilities and information entirely outside their training data
• Support building genuinely interactive, action-taking AI applications rather than pure text generators

The Function Calling Workflow

Whiteboard

Defining a Function for the Model

This definition tells the model exactly what the function does, what arguments it accepts, which are required, and what format each argument should be in — essentially a schema, similar in spirit to the JSON schemas covered in the previous topic.

A Simple Illustrative Example

User: "What's the weather like in Chicago right now?"

Model's Response (structured, not a direct answer):
{
  "function": "get_current_weather",
  "arguments": {
    "location": "Chicago, IL",
    "unit": "fahrenheit"
  }
}

The application code then:
1. Receives this structured response
2. Actually calls a real weather API using these exact arguments
3. Gets back real data, e.g., {"temperature": 45, "condition": "cloudy"}
4. Sends this result back to the model

Model's Final Response (now with real data):
"It's currently 45°F and cloudy in Chicago."

Notice that the model never actually knows the weather itself — it correctly recognized that this request needed a function call, generated the right arguments, and then used the real returned data to craft its final, natural-language answer.

Why the Model Doesn't Execute Functions Itself

Language models generate text — they don't have direct access
to external systems, databases, or APIs on their own. Function
calling works by having the model generate a structured
REQUEST for what should be executed, while the actual execution
happens in the developer's own application code, which has the
real permissions, credentials, and system access needed to
safely perform that action.

This separation is an important safety and architecture
principle: the model proposes an action, but a human-controlled
system decides whether and how to actually carry it out.

How the Model Decides Whether to Call a Function

When given a set of available function definitions alongside
a user's request, the model reasons (similar to the Chain of
Thought process) about whether the request actually requires
one of those functions to be fulfilled properly, or whether
it can be answered directly from its own general knowledge.

"What's 2+2?" → No function call needed, answered directly
"What's the weather in Chicago?" → Function call needed,
since the model has no way to know current weather itself

Function Calling vs Plain JSON Output

Key Properties of Function Calling

• Function calling lets a model generate structured requests to execute predefined functions with specific arguments.
• The model itself never executes anything — it only proposes the function and arguments; application code does the actual execution.
• Function definitions specify a name, description, and a schema for expected parameters, similar to the JSON schemas covered earlier.
• The model reasons about whether a given request actually requires a function call or can be answered directly.
• Function results are typically fed back into the model so it can generate a final, natural-language response incorporating real data.

Where Is Function Calling Used?

Advantages

• Enables models to reliably trigger real, predefined actions rather than just generating text
• Reduces fragile, error-prone parsing of free-form text to determine user intent
• Keeps actual execution under the developer's control, supporting safer, more auditable systems
• Provides a clean, structured contract between the model's reasoning and application logic
• Forms the essential building block for tool calling and broader agentic AI capabilities

Limitations

• Requires upfront, careful definition of every available function and its parameter schema
• The model can occasionally choose the wrong function or generate incorrect arguments
• Adds architectural complexity compared to simple, direct text generation
• Requires application-level error handling for cases where function execution fails
• Still subject to the model's general reasoning limitations when deciding whether/which function to call

Real-World Examples

Best Practices

• Write clear, specific function descriptions and parameter definitions to help the model choose correctly.
• Mark required vs optional parameters explicitly in each function's schema.
• Validate all model-generated arguments in application code before actually executing a function.
• Handle cases gracefully where the model selects the wrong function or provides invalid arguments.
• Feed function results back into the model clearly, so it can generate an accurate, natural-language final response.

Interview Tip

A common interview question is:

"How does function calling work, and why doesn't the model execute the function itself?"

A strong answer is:

Function calling works by providing a model with definitions of available functions — their names, descriptions, and expected parameters — alongside a user's request. The model then reasons about whether the request requires one of these functions, and if so, generates structured JSON specifying which function to call and with what arguments. The model itself never executes the function, because language models only generate text and have no direct access to external systems, databases, or APIs; instead, the developer's application code receives this structured request, actually executes the function with real system access and permissions, and typically feeds the result back to the model so it can generate a final, natural-language response incorporating that real data.

Explaining the safety/architecture reasoning behind this separation makes your answer stronger.

Conclusion

Function calling bridges a model's reasoning capabilities with real, executable application logic, letting a model reliably request specific actions with correctly structured arguments while keeping actual execution safely under the developer's control. With this foundation in place, the final topic in this section — tool calling — extends this same concept further, covering how models can be equipped with broader, more general toolkits to complete complex, multi-step tasks.