koaning.io

A few years ago I worked at Rasa, a company that builds tools for chatbots. This was well before LLMs came on to the scene and it took me a while to realize how much simpler things are now that we have types that can interact with LLMs.

Old forms

One of the key features of the old Rasa stack was the ability to trigger "forms" (link to legacy docs) in a conversation. This is a branch in a conversation where we keep on querying the user for information until we have enough to trigger a follow-up action.

As a main example I always used the "pizza"-bot. A user might say "give me a pepperoni" pizza but this would not suffice because the bot would also need to know the size of the pizza. Rasa had a mechanism to deal with this, which worked kind of like a state machine where the user would keep on getting questions until everything was validated.

New forms

Instead of a state machine, we can now do something like this:

class Pizza(BaseModel):
    flavour: str
    """The type of pizza that the user wants to order"""
    size: Literal["small", "medium", "large"] 
    """The size of pizza that the user wants to order"""

agent = Agent(
    "openai:gpt-4.1",
    output_type=list[Pizza] | str,
    instructions="You are here to detect pizza purchases from the user. If the user does not ask for a pizza then you have to ask.",
)

resp = await agent.run(message_history=message_history)

The key element in this example is the list[Pizza] | str part.

• If the LLM cannot extract a Pizza type then it can proceed to send text back to the user asking for more information.
• The user is able to ask for more than one pizza in a conversation.
• The user can even tell the assistant to add or remove pizzas from "memory". The whole conversation can be passed to the system as a message_history and it will be able to update the belief on what list[Pizza] should represent.

From here we can also add tools to check for extra properties. But the main thing to appreciate here is just the list[Pizza] | str type. This is so incredibly flexible because it gives the LLM full control to clarify a property from the end-user.

Livestream

This realization came to me during a livestream, which you can watch here:

I was joined by Marcelo (of FastAPI, starlette, uvicorn and Pydantic fame) and he shares a lot of interesting notes as we explore building a pizza bot.

The moving image below is only 49Kb and has an incredibly high resolution.

It's similar to a GIF but instead of showing moving images, it shows moving SVGs! The best part: Github supports these in their README.md files!

Getting these to work involves asciinema and svg-term-cli. After uploading the asciinema you can use the tool to download a file that you can immediately click and drag into a README, or you can use this snippet to keep things local:

> asciinema rec test.cast
  <do stuff in the terminal then ctrl-d>
> cat test.cast | svg-term --out=test.svg

It's something that I'm using extensively on bespoken.

How it works?

I was surpised to learn that moving SVGs were even a thing. But then I was reminded that animations are built into the svg spec.

• <animate> - animates individual attributes over time
• <animateTransform> - animates transformations like rotation, scaling, translation
• <animateMotion> - moves elements along a path

This is what the svg-term-cli leverages under the hood.

Just is an alternative to make. It allows you to make files that look like this:

# Show current directory tree
tree depth="3":
    find . -maxdepth {{depth}} -print | sed -e 's;[^/]*/;|____;g;s;____|;  |;g'

# Quick HTTP server on port 8000
serve port="8000":
    python3 -m http.server {{port}}

And from here you would be able to run just tree to run the commands under it just like make. However, notice how easy it is to add arguments here? That means you can do just serve 8080 too! That's already a killer feature but there's a bunch more to like.

global alias

The real killer feature, for me, is the -g flag. That allows me refer to a central just file, which contains all of my common utils. From there, I can also add this alias to my .zshrc file.

# Global Just utilities alias
alias kmd='just -g'

This is amazing because it allows you to have one global just-file with all the utilities that I like to re-use across projects. So if there are commands that you would love to reuse, you now have a convenient place to put it.

Especially with tools like uv for Python that let you inline all the dependencies, you can really move a lot of utils in. Not only that, you can also store this just-file with all your scrips and put it on Github.

working directories

There are more fancy features to help this pattern.

I have this command to start a web server that contains my blogging editor. Normally I would have to cd into the right folder and then call make write but now I can just add this to my global justfile via the working-directory feature.

# Deploy the blog
[working-directory: '/Users/vincentwarmerdam/Development/blog']
blog-deploy:
    make deploy

This lets me combine make commands that live on a project that I use a lot with a global file so that I can also run commands when I am not in the project folder.

From now on I can just type kmd write and kmd deploy to write from my blog and then to deploy it.

docs

It's also pretty fun to have an LLM add some fancy bash-fu.

# Extract various archive formats
extract file:
    #!/usr/bin/env bash
    if [ -f "{{file}}" ]; then
        case "{{file}}" in
            *.tar.bz2) tar xjf "{{file}}" ;;
            *.tar.gz)  tar xzf "{{file}}" ;;
            *.tar.xz)  tar xJf "{{file}}" ;;
            *.tar)     tar xf "{{file}}" ;;
            *.zip)     unzip "{{file}}" ;;
            *.gz)      gunzip "{{file}}" ;;
            *.bz2)     bunzip2 "{{file}}" ;;
            *.rar)     unrar x "{{file}}" ;;
            *.7z)      7z x "{{file}}" ;;
            *)         echo "Don't know how to extract {{file}}" ;;
        esac
    else
        echo "{{file}} is not a valid file"
    fi

# Kill process by port
killport port:
    lsof -ti:{{port}} | xargs kill -9

Notice how each command has some comments on top? Those are visible from the command line too! That means you'll also have inline documentation.

> just -g 
Available recipes:
    extract file            # Extract various archive formats
    killport port           # Kill process by port

chains

You can also chain commands together, just like in make, but there is a twist.

# Stuff
stuff: clean
    @echo "stuff"

# Clean
clean:
    @echo "clean"

If you were to run just stuff it would also run clean but you can also do this inline. So if you had this file:

# Stuff
stuff:
    @echo "stuff"

# Clean
clean:
    @echo "clean"

Then you could also do this:

just clean stuff 

Each command will only run once so you can't get multiple runs via: just clean clean clean but you can specify if some commands need to run before/after.

a:
  echo 'A!'

b: a && c d
  echo 'B!'

c:
  echo 'C!'

d:
  echo 'D!'

Running just b will show:

echo 'A!'
A!
echo 'B!'
B!
echo 'C!'
C!
echo 'D!'
D!

You can also choose to run just recusrively by calling just within a defition so expect a lot of freedom here.

just scratching the surface

just feels like one of those rare moments where you're exposed to a new tool and within half an hour you are already more productive.

I'm really only scratching the surface with what you can do with this tool, so make sure you check the documentation.