The Function Doesn't Exist

Claude shipped function calling this week, and here is the thing that took me a minute to fully clock: the function you're "calling" does not have to exist.

This is not a footnote. This is the whole trick.

You cram the function schema into the prompt, force the model to invoke it by completing that last line, and the model output — the thing that looks like a function call — that's your output. There is no runtime. There is no handler. The function format is the function. You just convinced a language model to emit structured JSON by dressing it up as an invocation of something that isn't running anywhere.

Simon Willison documented it, and there's a tool linked on that page for chunking streaming JSON from the extraction and handing it back to the user, which — if you've ever tried to parse a half-complete streamed JSON object — you know is the other half of the problem.

The part that actually matters for us: it works with Claude's vision endpoint. OpenAI has had function calling on GPT-4 Turbo for a while, but not on GPT-4 Vision Turbo — so you could look at an image or you could get structured output, not both. Claude gives you both. Look at an image, extract entities, get clean JSON back, no intermediate steps, no second model pass.

And then there's this: Haiku is apparently as capable as Opus for function-calling tasks. Which means the cheapest tier of the model handles structured extraction at the same reliability as the flagship. You don't need to run the expensive one.

What this does to the shape of a data analysis project is hard to overstate in April 2024 terms. A task that would have been a week of parsing, normalizing, munging — something like pulling structured data out of a corpus of documents — that's now a day, possibly less. Not because the hard part got easier. Because the hard part became a prompt and a schema.

The thing I keep coming back to is how mundane the mechanism is. You're not running anything. You're just agreeing with the model that a function was called. It's model theater, and it produces real output, and at some point the distinction between "the model pretended to call a function" and "the model called a function" stops mattering.

Which is, I think, the sentence that describes most of what's happening right now.