Function Calling / Tool Use (AI Agents)

What it is

Function calling, also called tool use, lets you hand the model a list of functions it is allowed to request. Instead of only generating text, the model can reply with 'call get_property_details with property_id 4471'. Your code runs the function, sends the result back, and the model writes the final answer using it.

The model never executes anything. It produces a structured request, your application decides whether to honour it, and the result goes back into the conversation as a message with the tool role. That split is what makes the pattern safe enough for production: the LLM proposes, your code disposes.

I use this in my voice agent. The model has a search_portfolio tool that runs the same embedding retrieval this site uses, and a show_on_screen tool that tells the browser which card to display. The model decides when to call them based on what the visitor asked.

Why it's needed

An LLM on its own cannot look anything up, cannot do reliable arithmetic and cannot take actions. Function calling gives it eyes and hands while keeping the actual execution in code you control. It is the foundation of every agent: an agent is just a loop of 'model picks a tool, code runs it, model sees the result' until the task is done.

It also replaces fragile prompt tricks. Before tool schemas, developers asked the model to 'reply in JSON with the function name' and parsed the text with regex. Native tool calling returns validated arguments that match the schema you declared.

How it works

• Declare each tool with a name, a one-sentence description and a JSON Schema for its parameters. The description is what the model reads to decide when to use it; write it like documentation for a junior developer.
• Send the messages plus the tools list. The response either contains normal text or one or more tool_calls with a name and JSON arguments.
• Parse the arguments, validate them, run the real function, and append a message with role 'tool' containing the result and the matching tool_call_id.
• Call the model again with the extended messages. It now answers in natural language, or requests another tool if the task needs more steps.
• Loop until the model returns plain text or you hit a step limit. The step limit is your safety net against runaway agents.
• Log every tool call with its arguments and result. That log is how you debug agent behaviour and measure what tools actually get used.

Code example

import json
from openai import OpenAI

client = OpenAI()

def search_portfolio(query: str) -> str:
    # In the real app this runs embedding retrieval over the knowledge base
    return "Abdur built a LangChain + ChromaDB chatbot at B1 Properties."

TOOLS = [{"type": "function", "function": {
    "name": "search_portfolio",
    "description": "Search Abdur's career knowledge base. Call this before answering any question about his experience, projects or skills.",
    "parameters": {"type": "object",
                   "properties": {"query": {"type": "string"}},
                   "required": ["query"]}}}]
FUNCS = {"search_portfolio": search_portfolio}

messages = [{"role": "user", "content": "Has he worked with vector databases?"}]
for _ in range(5):                                   # step limit
    r = client.chat.completions.create(model="gpt-4o-mini",
                                       messages=messages, tools=TOOLS)
    msg = r.choices[0].message
    if not msg.tool_calls:                           # final answer
        print(msg.content); break
    messages.append(msg)
    for call in msg.tool_calls:                      # run each request
        args = json.loads(call.function.arguments)
        result = FUNCS[call.function.name](**args)
        messages.append({"role": "tool", "tool_call_id": call.id,
                         "content": result})

Where I used this

Use cases

• Chatbots that look up live data: orders, listings, account status
• RAG as a tool, where the model decides when retrieval is needed instead of retrieving on every turn
• Agents that book, send, create or update records through your APIs
• Voice assistants that control what the screen shows while they talk
• Multi-step workflows: search, then calculate, then draft an email

Common mistakes

• Vague tool descriptions. 'Search' tells the model nothing; 'Search the career knowledge base, call before answering questions about experience' makes it reliable.
• Executing arguments without validation. The model can produce a malformed id or a path it should not touch. Validate like any other untrusted input.
• No step limit, so a confused model calls the same tool forever and your bill grows.
• Forgetting to append the assistant message that contained the tool_calls before appending the tool results. The API rejects the sequence.
• Too many tools. Past 10 to 15 tools the model picks wrong ones; group them or route to a sub-agent with a smaller set.

Interview questions