Multi-Turn Conversations and Tool Use

Setup

import os
BASE_URL = os.getenv("CS4973_BASE_URL")
API_KEY = os.getenv("CS4973_API_KEY")


from openai import OpenAI
import datetime

MODEL = "meta-llama/Meta-Llama-3.1-8B-Instruct"

CLIENT = OpenAI(base_url=BASE_URL, api_key=API_KEY)

def wrap(text: str) -> str:
    """
    Some basic text wrapping code that works better for me than the built-in
    textwrap module.
    """
    result_lines = [ ]
    current_line = []
    current_line_length = 0
    for line in text.split("\n"):
        for words in line.split(" "):
            if current_line_length + len(words) > 80:
                result_lines.append(" ".join(current_line))
                current_line = [ ]
                current_line_length = 0
            current_line.append(words)
            current_line_length += len(words) + 1
        result_lines.append(" ".join(current_line))
        current_line = [ ]
        current_line_length = 0
    return "\n".join(result_lines)

Multi-Turn Basics

Ignore tools for a bit. It’s important to realize that each query to an LLM is stateless.

def send_query(message: str):
    response = CLIENT.chat.completions.create(
        model=MODEL,
        messages=[
            {"role": "user", "content": message},
        ],
    )
    print(wrap(response.choices[0].message.content))

send_query("Where do the Red Sox play?")

The Boston Red Sox play their home games at Fenway Park. It is located in
Boston, Massachusetts, and is one of the most iconic baseball stadiums in the
United States.

send_query("What color is the stadium?")

You're referring to a stadium but didn't specify which one. There are many
stadiums around the world with different colors. Could you please provide more
context or information about the stadium you're referring to, such as its
location or name? That way, I can give you a more accurate answer.

So, we’ll make a little abstraction to manage multi-turn conversations.

resp = CLIENT.chat.completions.create(
        model=MODEL,
        messages=[
            { "role": "user", "content":"Where do the Red Sox play?" },
            { "role": "assistant", "content": "The Boston Red Sox play their home games at Fenway Park. It is located in Boston, Massachusetts, and is one of the most iconic baseball stadiums in the United States." },
            { "role": "user", "content": "What color is the stadium?" }
        ])
print(wrap(resp.choices[0].message.content))

Fenway Park, the home of the Boston Red Sox, is a predominantly green stadium.

class Conv:

    def __init__(self):
        self.prev_messages = [ ]
    
    def say(self, message):
        self.prev_messages.append({ "role": "user", "content": message })
        resp = CLIENT.chat.completions.create(
            model=MODEL,
            messages=self.prev_messages)
        resp_text = resp.choices[0].message.content
        self.prev_messages.append({ "role": "assistant", "content": resp_text })
        return resp_text

c = Conv()
print(c.say("Where do the Red Sox play?"))

The Boston Red Sox play at Fenway Park, which is located in Boston, Massachusetts. Fenway Park is considered one of the oldest and most historic baseball stadiums in the United States, with a rich history dating back to 1912.

print(c.say("What color is the stadium?"))

Fenway Park, the home stadium of the Boston Red Sox, is a historic stadium with a predominantly green color scheme, due to the lush grass field, and also the iconic Green Monster, the 37-foot-high left-field wall. The stadium's exterior features a reddish-brown brick with a green roof, but the most recognizable and iconic green feature is the tall left field wall, affectionately known as the "Green Monster."

Flight Database and Tool

This is what we had earlier. I’ve added a couple of more flights.

def parse_dates_in_item(item):
    others = {k: v for k, v in item.items() if k not in ["departure", "arrival"]}
    return {**others, "departure": datetime.datetime.fromisoformat(item["departure"]), "arrival": datetime.datetime.fromisoformat(item["arrival"])}

def find_flights(origin, destination, date: datetime.date):
    result = [ ]
    for flight in FLIGHT_DATABASE:
        if flight["origin"] != origin:
            continue
        if flight["destination"] != destination:
            continue
        if flight["departure"].date() != date:
            continue
        result.append({
            "airline": flight["airline"],
            "departure": flight["departure"],
            "arrival": flight["arrival"],
            "origin": flight["origin"],
            "destination": flight["destination"],
        })
    return result

FLIGHT_QUERIES = [
    {"query": "I need a flight from LAX to JFK on July 20th"},
    {"query": "Book me a flight from San Francisco to Chicago next Monday morning"},
    {"query": "I want to travel from Boston to Dallas on 5th August at 10 AM"},
    {"query": "Search for flights from SEA to DEN on Friday evening"},
    {"query": "Any flights from Orlando to Vegas on September 2nd or 3rd?"},
]

FLIGHT_DATABASE = [
    # For FLIGHT_QUERIES[0]
    {"airline": "Delta", "origin": "LAX", "destination": "JFK", "departure": "2023-07-20T08:00:00", "arrival": "2023-07-20T16:00:00"},
    # For FLIGHT_QUERIES[1]
    {"airline": "United", "origin": "SFO", "destination": "ORD", "departure": "2023-06-26T07:00:00", "arrival": "2023-06-26T13:00:00"},
    # For FLIGHT_QUERIES[2]
    {"airline": "Southwest", "origin": "BOS", "destination": "DFW", "departure": "2023-08-05T10:00:00", "arrival": "2023-08-05T14:00:00"},
    # For FLIGHT_QUERIES[3]
    {"airline": "JetBlue", "origin": "SEA", "destination": "DEN", "departure": "2023-01-06T18:00:00", "arrival": "2023-01-06T22:00:00"},
    # For FLIGHT_QUERIES[4]
    {"airline": "Delta", "origin": "MCO", "destination": "LAS", "departure": "2023-09-02T09:00:00", "arrival": "2023-09-02T12:00:00"},
    {"airline": "Delta", "origin": "MCO", "destination": "LAS", "departure": "2023-09-03T09:00:00", "arrival": "2023-09-03T12:00:00"},
    # Extras
    {"airline": "Delta", "origin": "BOS", "destination": "ATL", "departure": "2023-06-25T11:00:00", "arrival": "2023-06-25T13:00:00"},
    {"airline": "Delta", "origin": "BOS", "destination": "LAS", "departure": "2023-06-25T12:00:00", "arrival": "2023-06-26T13:00:00"},
    {"airline": "Delta", "origin": "BOS", "destination": "DFW", "departure": "2023-06-25T13:00:00", "arrival": "2023-06-26T13:00:00"},
    {"airline": "American Airlines", "origin": "IAH", "destination": "SEA", "departure": "2023-06-30T11:00:00", "arrival": "2023-06-30T14:00:00"},
    {"airline": "Delta", "origin": "LAX", "destination": "JFK", "departure": "2023-07-21T08:00:00", "arrival": "2023-07-21T16:00:00"},
    # From Boston to New York in Feb 1
    {"airline": "Delta", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T08:00:00", "arrival": "2023-02-01T09:00:00"},
    {"airline": "United", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T10:00:00", "arrival": "2023-02-01T11:00:00"},
    {"airline": "Jet Blue", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T12:00:00", "arrival": "2023-02-01T13:00:00"},
    {"airline": "Delta", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T14:00:00", "arrival": "2023-02-01T15:00:00"},
    {"airline": "Delta", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T16:00:00", "arrival": "2023-02-01T17:00:00"},
    {"airline": "United", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T18:00:00", "arrival": "2023-02-01T19:00:00"},
    {"airline": "Jet Blue", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T20:00:00", "arrival": "2023-02-01T21:00:00"},
    {"airline": "Delta", "origin": "BOS", "destination": "JFK", "departure": "2023-02-01T22:00:00", "arrival": "2023-02-01T23:00:00"},
]

FLIGHT_DATABASE = [ parse_dates_in_item(flight) for flight in FLIGHT_DATABASE ]

Multi-Turn Conversations and Tools

SYSTEM_PROMPT = """
You are a travel agent helping a customer book flights. To lookup flights, use
the following function:

import datetime

def find_flights(origin: str, destination: str, date: datetime.date) -> list: …

The function returns a list of records. Each record has the following shape:

{ “departure”: datetime.datetime, “arrival”: datetime.datetime, “airline”: str, “origin”: str, “destination”: str }

The origin and destination are given as airport codes. When you write code, save
the final answer in a variable called `result`.

- Do not run the code.
- Do not import anything.

Today is Monday, Jan 1 2023.
""".lstrip()

EXAMPLE0 = [ 
    { 
        "role": "user",
        "content": "i need to go to to boston from pvd tomorrow afternoon"
    },
    {
        "role": "assistant",
        "content": """

tmp = find_flights(“PVD”, “BOS”, datetime.date(2023, 1, 2)) result = [flight for flight in tmp if flight[“departure”].hour >= 12 and flight[“departure”].hour < 18]

""".lstrip()
    }
]

Query 1

User: get me a flight from BOS to LAX tomorrow Assistant: find_flights(“BOS”, “LAX”, tomorrrows-date)

Query 2

User: only morning flights

Actually becomes:

User: get me a flight from BOS to LAX tomorrow Assistant: find_flights(“BOS”, “LAX”, tomorrrows-date) User: only morning flights

Version 2:

User: get me a flight from BOS to LAX tomorrow Assistant: find_flights(“BOS”, “LAX”, tomorrrows-date) User: Running the code above gave these results:

Flight 1
Flight 2
....

only morning flights