The M×N problem of tool calling and open-source models

rremilouf 5 days ago 19 commentsRead Article on thetypicalset.com

DE version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

31% Positive

Analyzed from 767 words in the discussion.

Discussion (19 Comments)Read Original on HackerNews

evelant•about 4 hours ago

I guess I fail to see why this is such a problem. Yes it would be nice if the wire format were standardized or had a standard schema description, but is writing a parser that handles several formats actually a difficult problem? Modern models could probably whip up a "libToolCallParser" with bindings for all popular languages in an afternoon. Could probably also have an automated workflow for adding any new ones with minimal fuss. An annoyance, yes, but it does not seem like a really "hard" problem. It seems more of a social problem that open source hasn't coalesced around a library that handles it easily yet or am I missing something?

HarHarVeryFunny•about 3 hours ago

There already exist products like LiteLLM that adapt tool calling to different providers. FWIW, incompatibility isn't just an opensource problem - OpenAI and Anthropic also use different syntax for tool registration and invocation.

I would guess that lack of standardization of what tools are provided by different agents is as much of a problem as the differences in syntax, since the ideal case would be for a model to be trained end-to-end for use with a specific agent and set of tools, as I believe Anthropic do. Any agent interacting with a model that wasn't specifically trained to work with that agent/toolset is going to be at a disadvantage.

jeremyjh•about 3 hours ago

Presumably the hosting services are resolving all of this in their OpenAI/Anthropic compatibility layer, which is what most tools are using. So this is really just a problem for local engines that have to do the same thing but are expected to work right away for every new model drop.

airstrike•about 5 hours ago

One of the most relevant posts about AI on HN this year. It's not hype-y, but it's imperative to discuss.

I find it strange that the industry hasn't converged in at least somewhat standardized format, but I guess despite all the progress we're still in the very early days...

kami23•about 4 hours ago

Sounds like we need another standard. /s

This is one of the first tech waves where I feel like I'm on the very very groundfloor for a lot of exploration and it only feels like people have been paying closer attention in the last year. I can't imagine too many 'standard' standards becoming a standard that quickly.

It's new enough that Google seems to be throwing pasta against the wall and seeing what products and protocols stick. Antigravity for example seems too early to me, I think they just came out with another type of orchestrator, but the whole field seems to be exploring at the same time.

Everyone and their uncle is making an orchestrator now! I take a very cautious approach lately where I haven't been loading up my tools like agents, ides, browsers, phones with too much extra stuff because as soon as I switch something or something new comes out that doesn't support something I built a workflow around the tool either becomes inaccessible to me, or now a bigger learning curve than I have the patience for.

I've been a big proponent of trying to get all these things working locally for myself (I need to bite the bullet on some beefy video cards finally), and even just getting tool calls to work with some qwen models to be so counterintuitive.

jrochkind1•about 1 hour ago

Depending on a vendors market position, they may not want to make it easy to switch, which is what standards do, no?

Witty0Gore•about 2 hours ago

Useful article, I was fighting with GLM's tool calling format just last night. Stripping and sanitization to make it compatible with my UI consistently has been... fun.

ontouchstart•about 3 hours ago

https://mariozechner.at/posts/2025-11-30-pi-coding-agent/#to...

goodmythical•about 2 hours ago

Clicking that directly yields: "hi orange site user, i'd prefer my stuff to stay off the radar of this particular community."

jonathanhefner•about 5 hours ago

Does anyone know why there hasn’t been more widespread adoption of OpenAI’s Harmony format? Or will it just take another model generation to see adoption?

kleton•about 3 hours ago

Don't inference servers like vllm or sglang just translate these things to openai-compat API shapes?

Nevermark•about 5 hours ago

Feedback: I don't usually comment on formatting, but that fat indent is too much. I applied "hide distracting items" to the graphic, and the indent is still there. Reader works.

ikidd•about 3 hours ago

This sounds like a problem that LLMs were built to solve.

jiehong•about 4 hours ago

Am I misunderstanding, or isn't this supposed to be the point of MCP?

akoumjian•about 4 hours ago

The models only output text. Tool calls are nothing more than specially formatted text which gets parsed and interpreted by the inference server (or some other driver) into something which can be picked up by your agent loop and executed. Models are trained in a wide variety of different delimiters and escape characters to indicate their tool calls (along with things like separate thinking blocks). MCP is mostly a standard way to share with your agent loop the list of tool names and what their arguments are, which then gets passed to the inference server which then renders it down to text to feed to the model.

perlgeek•about 4 hours ago

> Tool calls are nothing more than specially formatted text which gets parsed and interpreted by the inference server

I know this is getting off-topic, but is anybody working on more direct tool calling?

LLMs are based on neural networks, so one could create an interface where activating certain neurons triggers tool calls, with other neurons encoding the inputs; another set of neurons could be triggered by the tokenized result from the tool call.

Currently, the lack of separation between data and metadata is a security nightmare, which enables prompt injection. And yet all I've seen done about is are workarounds.

yorwba•about 3 hours ago

Each text token already represents the activation of certain neurons. There is nothing "more direct." And you cannot fully separate data and metadata if you want them to influence the output. At best you can clearly distinguish them and hope that this is enough for the model to learn to treat them differently.