Search for a command to run...
I decided to take a career break and try out building apps using Generative AI. I am also learning NextJS and developing LaunchStack, starter code which I will use for my web projects
No comments yet. Be the first to comment.
In this series, I'll show examples on how to use Generative APIs from different providers. We'll use a variety of languages like Python, Go and Javascript. Articles in this series can you get started with Generative AI.
In this blog post, I'll walk you through the process of using Go to translate text using the Groq's Fast Inference API. What is Groq? Groq is a new company that's making AI work much faster than ever before. They've created special computer parts cal...
I’ve been exploring the Claude Agent SDK, and I had this idea — why not use it for non-coding workflows instead of relying on other agent frameworks like CrewAI or LangChain? To validate the idea, I built a simple example: a news researcher agent tha...
Introduction In the rapidly evolving world of AI, developers often need to integrate multiple AI providers into their applications. Whether you're using local models with Ollama, cloud services like OpenAI, or planning to add Anthropic or Google's Ge...
Introduction: Java’s Role in Enterprise AI Java has long been the backbone of enterprise systems, valued for its security, reliability, scalability, and platform independence. These very qualities that made Java the “engine of the enterprise” are now...
Email is essential for modern communication, but it often takes up more time than it should. Crafting thoughtful replies, keeping the right tone, and managing dozens of threads can quickly become overwhelming. This is where AI can step in and make a ...
OpenAI unveiled its latest innovation: OpenAI o3-mini. This new model is a major leap forward in efficient, high-quality reasoning—especially in STEM fields such as science, mathematics, and coding. Today, we explore what makes o3-mini so exciting, i...
On this page
In the world of conversational AI, the ability to integrate with external systems and retrieve information in real-time is a game-changer. Imagine being able to tap into the power of custom API calls, REST APIs, and other external systems to provide accurate and up-to-date responses to user questions.
This is precisely what Groq's function calling feature offers, allowing developers to harness the power of NLP processing to understand which function to call and retrieve the necessary information.
In this blog post, we'll explore how to use Groq's function calling feature to build conversational AI models that can interact with external systems, and demonstrate its capabilities using a simple example. We'll use an HTTP API written in Go to simulate an external API call which, ideally, will retrieve real data from a database. We'll break down the code into smaller parts, explaining each section in detail.
Here is the code, just copy paste it into a main.go file
package main
import (
"encoding/json"
"net/http"
"strings"
)
type CondoInfo struct {
Price string `json:"price"`
Location string `json:"location"`
}
var condos = map[string]CondoInfo{
"arton": {
Price: "Php 7,000,000",
Location: "Katipunan, Quezon City",
},
"gold residences": {
Price: "Php 8,000,000",
Location: "Near NAIA Terminal 1, Paranaque City",
},
"aruga mactan": {
Price: "Php 9,000,000",
Location: "Mactan, Cebu",
},
}
func getCondoLocation(w http.ResponseWriter, r *http.Request) {
condoName := strings.ToLower(r.URL.Query().Get("condo_name"))
condo, exists := condos[condoName]
if !exists {
http.Error(w, "Condo not found", http.StatusNotFound)
return
}
response := map[string]string{"location": condo.Location}
err := json.NewEncoder(w).Encode(response)
if err != nil {
return
}
}
func getCondoPrice(w http.ResponseWriter, r *http.Request) {
condoName := strings.ToLower(r.URL.Query().Get("condo_name"))
condo, exists := condos[condoName]
if !exists {
http.Error(w, "Condo not found", http.StatusNotFound)
return
}
response := map[string]string{"price": condo.Price}
err := json.NewEncoder(w).Encode(response)
if err != nil {
return
}
}
func main() {
http.HandleFunc("/get_condo_location", getCondoLocation)
http.HandleFunc("/get_condo_price", getCondoPrice)
err := http.ListenAndServe(":8080", nil)
if err != nil {
return
}
}
Run the API using the go command. This will run the API in http://localhost:8000
go run main.go
get_condo_location endpoint:
curl "http://localhost:8080/get_condo_location?condo_name=Arton"
{"location":"Katipunan, Quezon City"}
get_condo_price endpoint:
curl "http://localhost:8080/get_condo_price?condo_name=Arton"
{"price":"Php 7,000,000"}
This code is written in Python. This will respond to a query or prompt from a user and answer based on the data retrieved from an API. For this example, we will be using the Go API we created above to get the data to answer the ff. questions:
from groq import Groq
import os
import json
client = Groq(api_key=os.getenv('GROQ_API_KEY'))
MODEL = 'llama3-70b-8192'
Here, we import the necessary libraries, including Groq, os, and json. We then set up a Groq client using an API key stored in an environment variable. The MODEL variable specifies the language model we'll be using.
Install the groq dependency using pip install.
pip install groq
Ideally, responses from these functions should be from responses of your APIs. Right now, we will use our API we created earlier for demonstration purposes. Imagine that this function retrieves the price from a REST API. This is powerful because you can take advantage of NLP processing to understand which function to call. This is also available in OpenAI and this functionality works similar to it. But for this example, we will harness the power of Groq's fast inference!
def get_condo_price(condo_name):
"""Get the price of a condominium"""
return condo_price_api_call(condo_name)
def get_condo_location(condo_name):
"""Get the location of a condominium"""
return condo_location_api_call(condo_name)
Function 1.get_condo_price that takes a condominium name as input and returns the price of the condominium.
Function 2.get_condo_location that takes a condominium name as input and returns the location of the condominium.
Then, we'll define the functions to call the external API which in this case is the API we created initially with Go. Feel free to change this to call your own APIs.
def condo_price_api_call(condo_name):
url = f'http://localhost:8080/get_condo_price?condo_name={condo_name}'
response = requests.get(url)
if response.status_code == 200:
data = response.json()
return data.get('price')
else:
return None # Handle error cases here if needed
def condo_location_api_call(condo_name):
url = f'http://localhost:8080/get_condo_location?condo_name={condo_name}'
response = requests.get(url)
if response.status_code == 200:
data = response.json()
return data.get('location')
else:
return None
def run_conversation(user_prompt):
...
The run_conversation function takes a user prompt as input and sends it to the Groq model along with the available functions (get_condo_price and get_condo_location).
messages=[
{
"role": "system",
"content": "You are a function calling LLM that uses the data extracted from the function and responds to "
"the user with the result of the function."
},
{
"role": "user",
"content": user_prompt,
}
]
tools = [
{
"type": "function",
"function": {
"name": "get_condo_price",
"description": "Get the price of a condo or condominium name",
"parameters": {
"type": "object",
"properties": {
"condo_name": {
"type": "string",
"description": "The name of the condominium or condo",
}
},
"required": ["condo_name"],
},
},
},
{
"type": "function",
"function": {
"name": "get_condo_location",
"description": "Get the location of a condo or condominium name",
"parameters": {
"type": "object",
"properties": {
"condo_name": {
"type": "string",
"description": "The name of the condominium or condo",
}
},
"required": ["condo_name"],
},
},
}
]
We define the conversation and available functions to send to the model.
response = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=tools,
tool_choice="auto",
max_tokens=4096
)
We get the model's response to the conversation and available functions.
This is how the response object looks like.
ChatCompletionMessage(
content=None,
role='assistant',
function_call=None,
tool_calls=[
ChatCompletionMessageToolCall(
id='call_2nvf',
function=Function(
arguments='{"condo_name":"Arton"}',
name='get_condo_price'
),
type='function'
)
]
)
response_message = response.choices[0].message
tool_calls = response_message.tool_calls
if tool_calls: #checks if tool_calls is an object
...
We check if the model wanted to call a function.
available_functions = {
"get_condo_price": get_condo_price,
"get_condo_location": get_condo_location,
}
messages.append(response_message) # extend conversation with assistant's reply
for tool_call in tool_calls: #loop through tool_calls and get the function to call
function_name = tool_call.function.name
function_to_call = available_functions[function_name]
function_args = json.loads(tool_call.function.arguments)
function_response = function_to_call(
condo_name=function_args.get("condo_name")
)
messages.append(
{
"tool_call_id": tool_call.id,
"role": "tool",
"name": function_name,
"content": function_response,
}
) # extend conversation with function response
We call the function and append the function response to the conversation.
second_response = client.chat.completions.create(
model=MODEL,
messages=messages
) # get a new response from the model where it can see the function response
return second_response.choices[0].message.content
We get a new response from the model where it can see the function response.
prompt = "What is the price of Arton?"
print('Price:', run_conversation(prompt))
prompt = "Where is the Arton condo?"
print('Location:', run_conversation(prompt))
We test the conversation by providing two user prompts:
"What is the price of Arton?" and "Where is the Arton condo?"
python main.py
You should see this response in your terminal.
Full Source Code
Here's is the full source code of the example. Just save it in a main.py file.
from groq import Groq
import os
import json
import requests
client = Groq(api_key=os.getenv('GROQ_API_KEY'))
MODEL = 'llama3-70b-8192'
def get_condo_price(condo_name):
"""Get the price of a condominium"""
return condo_price_api_call(condo_name)
def condo_price_api_call(condo_name):
url = f'http://localhost:8080/get_condo_price?condo_name={condo_name}'
response = requests.get(url)
if response.status_code == 200:
data = response.json()
return data.get('price')
else:
return None # Handle error cases here if needed
def get_condo_location(condo_name):
"""Get the location of a condominium"""
return condo_location_api_call(condo_name)
def condo_location_api_call(condo_name):
url = f'http://localhost:8080/get_condo_location?condo_name={condo_name}'
response = requests.get(url)
if response.status_code == 200:
data = response.json()
return data.get('location')
else:
return None
def run_conversation(user_prompt):
# Step 1: send the conversation and available functions to the model
messages = [
{
"role": "system",
"content": "You are a function calling LLM that uses the data extracted from the function and responds to "
"the user with the result of the function. Do not mention about anything about the tool call."
"Just respond with the answer to the user prompt."
},
{
"role": "user",
"content": user_prompt,
}
]
tools = [
{
"type": "function",
"function": {
"name": "get_condo_price",
"description": "Get the price of a condo or condominium name",
"parameters": {
"type": "object",
"properties": {
"condo_name": {
"type": "string",
"description": "The name of the condominium or condo",
}
},
"required": ["condo_name"],
},
},
},
{
"type": "function",
"function": {
"name": "get_condo_location",
"description": "Get the location of a condo or condominium name",
"parameters": {
"type": "object",
"properties": {
"condo_name": {
"type": "string",
"description": "The name of the condominium or condo",
}
},
"required": ["condo_name"],
},
},
}
]
response = client.chat.completions.create(
model=MODEL,
messages=messages,
tools=tools,
tool_choice="auto",
max_tokens=4096
)
response_message = response.choices[0].message
tool_calls = response_message.tool_calls
# Step 2: check if the model wanted to call a function
if tool_calls:
# Step 3: call the function
# Note: the JSON response may not always be valid; be sure to handle errors
available_functions = {
"get_condo_price": get_condo_price,
"get_condo_location": get_condo_location,
} # only one function in this example, but you can have multiple
messages.append(response_message) # extend conversation with assistant's reply
# Step 4: send the info for each function call and function response to the model
for tool_call in tool_calls:
function_name = tool_call.function.name
function_to_call = available_functions[function_name]
function_args = json.loads(tool_call.function.arguments)
function_response = function_to_call(
condo_name=function_args.get("condo_name")
)
messages.append(
{
"tool_call_id": tool_call.id,
"role": "tool",
"name": function_name,
"content": function_response,
}
) # extend conversation with function response
second_response = client.chat.completions.create(
model=MODEL,
messages=messages
) # get a new response from the model where it can see the function response
return second_response.choices[0].message.content
prompt = "What is the price of Arton?"
print('Price:', run_conversation(prompt))
prompt = "Where is the Arton condo?"
print('Location:', run_conversation(prompt))
Summary
In this blog post, we created an HTTP API using Go to simulate retrieval of data which was used to answer user questions or queries.
We explored the function calling feature of the Groq APIs, which allows us to call external functions to retrieve information. We defined two functions, get_condo_price and get_condo_location, and used the run_conversation function to send the conversation and available functions to the model.
We then checked if the model wanted to call a function, called the function, and appended the function response to the conversation. Finally, we tested the conversation with two user prompts. The function calling feature of the Groq APIs provides a powerful way to build conversational AI models that can interact with external systems.
If you're interested in learning more about integrating Go with Generative AI, follow my blog for more tutorials and insights. This is just the start!
I do live coding in Twitch and Youtube. You can follow me if you'd like to ask me questions when I go live. I also post in LinkedIn, you can connect with me there as well.
Interested in implementing AI within your company, you can reach out to me.
Not yet sure how can I help? Book a FREE discovery call with me.