A few years ago I worked at Rasa, a company that builds tools for chatbots. This was well before LLMs came on to the scene and it took me a while to realize how much simpler things are now that we have types that can interact with LLMs.

Old forms

One of the key features of the old Rasa stack was the ability to trigger "forms" (link to legacy docs) in a conversation. This is a branch in a conversation where we keep on querying the user for information until we have enough to trigger a follow-up action.

As a main example I always used the "pizza"-bot. A user might say "give me a pepperoni" pizza but this would not suffice because the bot would also need to know the size of the pizza. Rasa had a mechanism to deal with this, which worked kind of like a state machine where the user would keep on getting questions until everything was validated.

New forms

Instead of a state machine, we can now do something like this:

class Pizza(BaseModel):
    flavour: str
    """The type of pizza that the user wants to order"""
    size: Literal["small", "medium", "large"] 
    """The size of pizza that the user wants to order"""

agent = Agent(
    "openai:gpt-4.1",
    output_type=list[Pizza] | str,
    instructions="You are here to detect pizza purchases from the user. If the user does not ask for a pizza then you have to ask.",
)

resp = await agent.run(message_history=message_history)

The key element in this example is the list[Pizza] | str part.

• If the LLM cannot extract a Pizza type then it can proceed to send text back to the user asking for more information.
• The user is able to ask for more than one pizza in a conversation.
• The user can even tell the assistant to add or remove pizzas from "memory". The whole conversation can be passed to the system as a message_history and it will be able to update the belief on what list[Pizza] should represent.

From here we can also add tools to check for extra properties. But the main thing to appreciate here is just the list[Pizza] | str type. This is so incredibly flexible because it gives the LLM full control to clarify a property from the end-user.

Livestream

This realization came to me during a livestream, which you can watch here:

I was joined by Marcelo (of FastAPI, starlette, uvicorn and Pydantic fame) and he shares a lot of interesting notes as we explore building a pizza bot.