Code Execution (Tool Usage)

LLMs are unreliable at exact math, data crunching, and precise logic — but code isn't. Code execution gives an agent a tool to write and run code, then use the exact output. It's how agents do reliable computation, data analysis, and file work — and because model-written code is untrusted, it must run in a sandbox.

💡 In one line: Code execution lets an agent write code and run it in a sandbox, using the exact result instead of guessing.

What is Code Execution?

It's a tool that lets the agent write code (usually Python) and run it in an isolated runtime, then use the output. The model generates the code; a sandbox executes it and returns the result — much like function calling, but the "function" is code the model just wrote.

Why Run Code?

LLMs are shaky at exact arithmetic, data processing, and precise logic. Code is exact and deterministic. So agents offload to code: math, data analysis, sorting/filtering, plotting, file/format conversion, and algorithms — anything a library does better than token prediction.

How It Works

The model writes code, the sandbox runs it, and errors loop back for a fix. 

Whiteboard
Whiteboard diagram


Sandboxing & Safety (Critical)

Running model-generated code is dangerous — it must be isolated:

  • No (or limited) network access.
  • Restricted filesystem.
  • Resource limits and timeouts.
  • Container / VM isolation.

Treat all generated code as untrusted. This is the number-one concern with code execution — get it wrong and you've handed an untrusted program the keys.

Code + Reflection

Code execution pairs beautifully with Reflection: run → error → the model reads the error → fixes → re-runs. Real execution provides an objective feedback signal (errors, test results), making this loop genuinely self-correcting.

Code Interpreter Tools

Hosted "code interpreter" / advanced-data-analysis features are exactly this — sandboxed code execution as a tool. The research idea behind it is PAL (Program-Aided Language models): reason by writing code rather than in prose.

Examples

  • Data analysis — load a CSV, compute stats, make a chart.
  • Exact calculation — do the math in code, not in the model's head.
  • File processing and coding agents.

Benefits

  • Accuracy and determinism.
  • Power — any library, any computation.
  • Verifiable — you can see and re-run the code.

Challenges & Safety

  • Security — the sandbox is non-negotiable; code is untrusted.
  • Resource abuse and infinite loops → enforce timeouts.
  • Error handling, plus cost and latency.

Best Practices

  • Always sandbox; limit network, filesystem, and resources; set timeouts.
  • Capture errors and return them to the model.
  • Validate output, and combine with reflection for self-correction.

Summary

  • Code execution lets an agent write and run code for exact results.
  • Use it for math, data, files, and algorithms — where reasoning is unreliable.
  • The loop: write → run in sandbox → output or fix on error.
  • Sandboxing is critical — treat generated code as untrusted.
  • It pairs naturally with Reflection and powers code-interpreter tools. EOF echo created