Function calling for open models

Function calling lets you define custom functions and provide LLMs with thecapability to call them to retrieve real-time information or interact withexternal systems like SQL databases or customer service tools.

For more conceptual information about function calling, seeIntroduction to function calling.

Use function calling

The following samples show how to use function calling.

Python

Before trying this sample, follow thePython setup instructions in theVertex AI quickstart using client libraries. For more information, see theVertex AIPython API reference documentation.

To authenticate to Vertex AI, set up Application Default Credentials. For more information, seeSet up authentication for a local development environment.

Before running this sample, make sure to set theOPENAI_BASE_URL environment variable.For more information, seeAuthentication and credentials.

fromopenaiimportOpenAIclient=OpenAI()response=client.chat.completions.create(model="MODEL",messages=[{"role":"user","content":"CONTENT"}],tools=[{"type":"function","function":{"name":"FUNCTION_NAME","description":"FUNCTION_DESCRIPTION","parameters":PARAMETERS_OBJECT,}}],tool_choice="auto",)
  • MODEL: The model name you want to use, for exampleqwen/qwen3-next-80b-a3b-instruct-maas.
  • CONTENT: The user prompt to send to the model.
  • FUNCTION_NAME: The name of the function to call.
  • FUNCTION_DESCRIPTION: A description of the function.
  • PARAMETERS_OBJECT: A dictionary that defines the function parameters, for example:
    {"type": "object", "properties": {"location": {"type": "string", "description": "The city and state"}}, "required": ["location"]}

REST

After youset up your environment, you can use REST to test a text prompt. The following sample sends a request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

  • PROJECT_ID: Your Google Cloud project ID.
  • LOCATION: A region that supports open models.
  • MODEL: The model name you want to use, for exampleqwen/qwen3-next-80b-a3b-instruct-maas.
  • CONTENT: The user prompt to send to the model.
  • FUNCTION_NAME: The name of the function to call.
  • FUNCTION_DESCRIPTION: A description of the function.
  • PARAMETERS_OBJECT: A JSON schema object that defines the function parameters, for example:
    {"type": "object", "properties": {"location": {"type": "string", "description": "The city and state"}}, "required": ["location"]}

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/endpoints/openapi/chat/completions

Request JSON body:

{  "model": "MODEL",  "messages": [    {      "role": "user",      "content": "CONTENT"    }  ],  "tools": [    {      "type": "function",      "function": {        "name": "FUNCTION_NAME",        "description": "FUNCTION_DESCRIPTION",        "parameters":PARAMETERS_OBJECT      }    }  ],  "tool_choice": "auto"}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to thegcloud CLI with your user account by runninggcloud init orgcloud auth login , or by usingCloud Shell, which automatically logs you into thegcloud CLI . You can check the currently active account by runninggcloud auth list.

Save the request body in a file namedrequest.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/endpoints/openapi/chat/completions"

PowerShell

Note: The following command assumes that you have logged in to thegcloud CLI with your user account by runninggcloud init orgcloud auth login . You can check the currently active account by runninggcloud auth list.

Save the request body in a file namedrequest.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/endpoints/openapi/chat/completions" | Select-Object -Expand Content

You should receive a successful status code (2xx) and an empty response.

Example

The following is the complete output you could expect after using theget_current_weather function to fetch meteorological information.

Python

fromopenaiimportOpenAIclient=OpenAI()response=client.chat.completions.create(model="qwen/qwen3-next-80b-a3b-instruct-maas",messages=[{"role":"user","content":"Which city has a higher temperature, Boston or new Delhi and by how much in F?"},{"role":"assistant","content":"I'll check the current temperatures for Boston and New Delhi in Fahrenheit and compare them. I'll call the weather function for both cities.","tool_calls":[{"function":{"arguments":"{\"location\":\"Boston, MA\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"},"id":"get_current_weather","type":"function"},{"function":{"arguments":"{\"location\":\"New Delhi, India\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"},"id":"get_current_weather","type":"function"}]},{"role":"tool","content":"The temperature in Boston is 75 degrees Fahrenheit.","tool_call_id":"get_current_weather"},{"role":"tool","content":"The temperature in New Delhi is 50 degrees Fahrenheit.","tool_call_id":"get_current_weather"}],tools=[{"type":"function","function":{"name":"get_current_weather","description":"Get the current weather in a given location","parameters":{"type":"object","properties":{"location":{"type":"string","description":"The city and state, e.g. San Francisco, CA"},"unit":{"type":"string","enum":["celsius","fahrenheit"]}},"required":["location"]}}}],tool_choice="auto")

curl

curl-XPOST\-H"Authorization: Bearer$(gcloudauthprint-access-token)"\-H"Content-Type: application/json"\https://us-central1-aiplatform.googleapis.com/v1/projects/sample-project/locations/us-central1/endpoints/openapi/chat/completions-d\'{  "model": "qwen/qwen3-next-80b-a3b-instruct-maas",  "messages": [    {      "role": "user",      "content": "Which city has a higher temperature, Boston or new Delhi and by how much in F?"    },    {      "role": "assistant",      "content": "I'llcheckthecurrenttemperaturesforBostonandNewDelhiinFahrenheitandcomparethem.I'll call the weather function for both cities.",      "tool_calls": [{"function":{"arguments":"{\"location\":\"Boston, MA\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"},"id":"get_current_weather","type":"function"},{"function":{"arguments":"{\"location\":\"New Delhi, India\",\"unit\":\"fahrenheit\"}","name":"get_current_weather"},"id":"get_current_weather","type":"function"}]    },    {      "role": "tool",      "content": "The temperature in Boston is 75 degrees Fahrenheit.",      "tool_call_id": "get_current_weather"    },    {      "role": "tool",      "content": "The temperature in New Delhi is 50 degrees Fahrenheit.",      "tool_call_id": "get_current_weather"    }  ],  "tools": [    {      "type": "function",      "function": {        "name": "get_current_weather",        "description": "Get the current weather in a given location",        "parameters": {          "type": "object",          "properties": {            "location": {              "type": "string",              "description": "The city and state, e.g. San Francisco, CA"            },            "unit": {              "type": "string",              "enum": ["celsius", "fahrenheit"]            }          },          "required": ["location"]        }      }    }  ],  "tool_choice": "auto"}'
After receiving the information retrieved through the calling the external`get_current_weather` function, the model can synthesize the information fromthe two `tool` responses and answer the user's question.The following is an example of what that model output could look like:
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Based on the current weather data:\n\n- **Boston, MA**: 75°F    \n- **New Delhi, India**: 50°F  \n\n**Comparison**:    \nBoston is **25°F warmer** than New Delhi.  \n\n**Answer**:    \nBoston has a higher temperature than New Delhi by 25 degrees Fahrenheit.","role":"assistant"}}],"created":1750450289,"id":"2025-06-20|13:11:29.240295-07|6.230.75.101|-987540014","model":"qwen/qwen3-next-80b-a3b-instruct-maas","object":"chat.completion","system_fingerprint":"","usage":{"completion_tokens":66,"prompt_tokens":217,"total_tokens":283}}

Model-specific guidance

The following sections provide model-specific guidance for functioncalling.

DeepSeek

DeepSeek models don't perform as well on function calling if you use a systemprompt. For optimal performance, omit the system prompt when using functioncalling with DeepSeek models.

Llama

meta/llama3-405b-instruct-maas doesn't supporttool_choice = 'required'.

OpenAI

When usingopenai/gpt-oss-120b-instruct-maas andopenai/gpt-oss-20b-instruct-maas, place your tool definitions in thesystem prompt for optimal performance. For example:

{"messages":[{"role":"system","content":"You are a helpful assistant with access to the following functions. Use them if required:\n..."},{"role":"user","content":"What's the weather like in Boston?"},...]}

These models don't supporttool_choice = 'required' or named tool calling.

Qwen

Qwen models perform best whentool_choice is explicitly set toauto ornone. Iftool_choice is unset, the model might not perform aswell.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-02-19 UTC.