New Function role added to GPT3.5 and GPT4 to give model to use API or functions as tools

The latest update (June 2023) from OpenAI added a new role called function into their tool box.
By describing functions in your prompts, the model can intelligently choose to output a JSON object containing arguments to call these functions based on user input — perfect for integrating with other tools or APIs

This is very similar to the LangChain package to allow mole to choose tools, instead, it’s more native and probably easier and have better performance.

The latest models (gpt-3.5-turbo-0613 and gpt-4-0613) have been fine-tuned to include this function role. And from June 27th, the stable gpt-4 or gpt-3.5-turbo will be automatically upgraded to this new version.
So for the following code example, very possible you don’t need to specify the specific model with date, and instead ust use “gpt-3.5-turbo” or “gpt-4”, it’s going to be same.

To demo how to pass in function as tools for the model to choose, we show the followng example.

First, we define a simple function. In reality, this could be a more complex function or API endpoint.

import openai
import json


OPENAI_API_KEY = 'your api key'
openai.api_key = OPENAI_API_KEY

# Example dummy function hard coded to return the same weather
# In production, this could be your backend API or an external API
def get_current_weather(location, unit="fahrenheit"):
    """Get the current weather in a given location"""
    weather_info = {
        "location": location,
        "temperature": "72",
        "unit": unit,
        "forecast": ["sunny", "windy"],
    }
    return json.dumps(weather_info)

Here is how we should define the messages sent to OpenAI:

# Step 1, send model the user query and what functions it has access to
response = openai.ChatCompletion.create(
	model="gpt-3.5-turbo-0613",
	messages=[{"role": "user", "content": "What's the weather like in Boston?"}],
	functions=[
		{
			"name": "get_current_weather",
			"description": "Get the current weather in a given location",
			"parameters": {
				"type": "object",
				"properties": {
					"location": {
						"type": "string",
						"description": "The city and state, e.g. San Francisco, CA",
					},
					"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
				},
				"required": ["location"],
			},
		}
	],
	function_call="auto",
)

As you can see, here are doing the usual thing asking a quesition through the “user” role, and the difference is we also passed a functions. If the model decides that function we pass in should be used answer the question,
it will return the corresponding output. And let’s print out the responsed message:

message = response["choices"][0]["message"]
print(message)

and we see the print results:

{
  "content": null,
  "function_call": {
    "arguments": "{\n\"location\": \"Boston\"\n}",
    "name": "get_current_weather"
  },
  "role": "assistant"
}

So instead of return the direct answer, it tells us it could to use a function call with the corresponding parameters according to what we have sent to it in the previous step.
So for us, we need to check the above output, if it has function_call key in the result, we should call our functions, and put the results back and send to OpenAI, and hopefully get the final user-facing messages.

Let’s code it up:

# Step 2, check if the model wants to call a function
if message.get("function_call"):
	function_name = message["function_call"]["name"]
	function_args = json.loads(message["function_call"]["arguments"])

	# Step 3, call the function
	# Note: the JSON response from the model may not be valid JSON
	function_response = get_current_weather(
		location=function_args.get("location"),
		unit=function_args.get("unit"),
	)

	# Step 4, send model the info on the function call and function response
	second_response = openai.ChatCompletion.create(
		model="gpt-3.5-turbo-0613",
		messages=[
			{"role": "user", "content": "What is the weather like in boston?"},
			message,
			{
				"role": "function",
				"name": function_name,
				"content": function_response,
			},
		],
	)
	return second_response

and if we check the output response from the above steps, we will see:

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "The current weather in Boston is sunny and windy with a temperature of 72 degrees Fahrenheit.",
        "role": "assistant"
      }
    }
  ],
  "created": 1686856448,
  "id": "chatcmpl-7Rmi0JDIiOfagxDFJq16dhiC6qagZ",
  "model": "gpt-3.5-turbo-0613",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 18,
    "prompt_tokens": 68,
    "total_tokens": 86
  }
}

Where it shows the answer that can be shown to the final user.