Knowledge Vault

Function Calling

3 min read

Overview

Function calling is the mechanism by which language models generate structured, executable invocations of external capabilities rather than free-form text responses. The model receives a schema describing available functions (name, description, parameter types), and when the user’s intent maps to a function, the model outputs a structured call with resolved arguments instead of — or in addition to — natural language. The calling application executes the function and returns the result, which the model incorporates into its subsequent response. Function calling transforms LLMs from knowledge retrieval systems into general-purpose reasoning engines that can read and write external state.

Key Concepts

  • Tool schema (JSON Schema) — The formal description of each callable function: name, description, parameter names, types, required/optional flags, and enumerated values. Schema quality directly determines selection accuracy; ambiguous descriptions cause the model to choose wrong functions or construct malformed arguments.
  • Parallel function calls — Models capable of parallel invocation can issue multiple independent calls in a single turn, then process all results together. This collapses multi-step sequential calls into a single round-trip for independent operations.
  • Forced function calls — The tool_choice parameter (or equivalent) forces the model to call a specific function rather than returning text. Used when the application needs structured output and free text is never acceptable.
  • Streaming function calls — Models stream function call arguments token-by-token, enabling the application to begin processing before the call is complete. Reduces perceived latency for large argument payloads.
  • Error recovery — When a function returns an error, the model receives the error message and must decide whether to retry with corrected arguments, call a different function, or surface the error to the user.
  • MCP (Model Context Protocol) — An open standard that decouples function definitions from specific model APIs, allowing the same tool server to be used with any MCP-compatible model.

Trade-offs

  • Function schemas consume context window tokens — large schemas reduce space for task context
  • The model can hallucinate function names or argument values that don’t exist in the schema
  • Parallel calls reduce latency but require the application to handle concurrent execution and partial failures
  • Forced function calls eliminate model flexibility, which is appropriate for strict structured-output scenarios but breaks graceful fallback
  • AI & Agents — function calling is the core mechanism enabling agents to take real-world actions
  • Tool Use — the broader pattern of which function calling is the primary technical implementation
  • Grounding — function calls to live APIs are a key grounding strategy for temporal freshness
  • Multi-Agent Systems — function calling enables inter-agent communication and external system integration within agent networks

Graph View

Backlinks