Relay 1: a tool-calling model that reads instead of writes

Relay 1 does one job. Given what a person said and the tools an app offers, it decides which tools to call and with which arguments. It is 22 million parameters in 11.5 MB of model files (about 14 MB with its tokenizer, retrieval prefilter and runtime), and it runs on Gust, our runtime: natively on x86-64 and ARM64, and in a browser tab through WebAssembly. There is no GPU, no server and no Python in the path.

Most tool-calling models are language models, so they write the call one token at a time. Relay writes nothing. It reads the request and the tool schemas together, then points at the answer. That single change is why it is small, why it cannot invent a phone number, and why it is unusually good at filling slots. It is also where its limits come from, and this post spends as long on those as on the wins.

of model files; about 14 MB with the tokenizer, prefilter and runtime

11.5MB

parameters, of which 12.5M are the encoder

22M

mean over 12 evaluations. The best other on-device model scores 48.7

67.2

median request on 8 threads, in WebAssembly

39ms

What Relay does

An app hands Relay three things: the user’s message, the tools it offers (each a name, a description and its parameters), and optional context such as today’s date. Relay returns the calls to make, or no call at all when nothing fits.

User   Set an alarm for 6:45 tomorrow, then text Sam I'm running late
Tools  set_alarm(time, label?), send_message(recipient, body), get_weather(city), ...

Relay  set_alarm(time="2026-10-18T06:45:00")
       send_message(recipient="Sam", body="I'm running late")

The same machinery fills forms. In a dialogue about booking a restaurant, “a table for four at Bella Napoli at 8” becomes {restaurant: "Bella Napoli", party_size: 4, time: "20:00"}. Researchers call this dialogue state tracking, and it is the same problem as tool calling with a different name.

Everything happens locally. The user’s request is never sent anywhere, because there is nowhere for it to go.

Try it

This is Relay 1 itself, the shipped 11.5 MB model and its WebAssembly runtime, running in your browser, on as many threads as you pick under the examples (one, where the browser cannot share memory between threads). Every thread count gives the same calls; only the time changes. The demo’s tool lists are long and it reads them all, so its times are higher than Figure 11’s. Pick a flat, a phone or a car, ask it something, and watch the calls happen on the device. The words Relay read its values from are underlined, in your request and in the calls. Ask for something none of the tools covers and it makes no call, so nothing changes. Every example is one the model gets right, and “Play them all” runs them in turn.

It is the shipped model with no tricks, so it gets things wrong too, mostly in the ways the limits describe.

Use it

Relay is a download, not a service. Each way below fetches the same files from Hugging Face, checks them against SHA-256 sums and runs the same Gust code, so the calls are the same bits whichever you pick. All of them use one small interface, the Gust model API: open a model by id, give it typed values, get typed values back. Relay takes JSON and returns JSON, and the next model will need no new code in any of them.

A request is the message, the tools and the date. A tool is a name, a description and its parameters; a parameter has a name, a type (string, integer, number, boolean, array or object), a description, whether it is required, and optionally an enum.

const tools = [
  { name: 'show_map', description: 'Shows a location on the map.',
    params: [{ name: 'query', type: 'string', description: 'The place to show.', required: true }] },
  { name: 'turn_on_flashlight', description: 'Turns the flashlight on.', params: [] },
];

sealpup install relay-1
sealpup run relay-1 "Show me the Eiffel Tower on the map" --tools tools.json

# pip install sealresearch
import sealresearch as sr

relay = sr.Model.load("relay-1")
relay.run({"text": "Show me the Eiffel Tower on the map", "tools": tools})

// npm install @sealresearch/gust   (@sealresearch/relay is the same with a smaller surface)
import { Model } from '@sealresearch/gust';

const relay = await Model.load('relay-1');
relay.run({ text: 'Show me the Eiffel Tower on the map', tools });

Each returns [{tool: "show_map", args: {query: "Eiffel Tower"}}] for that tool list, and [] for “Tell me a joke”. Pass the date as context: {now: "2026-10-23T09:00:00"} in the request when the message says “tomorrow”. Tools written as JSON Schema, the way OpenAI, Anthropic and MCP describe them, are not this format and need converting first (sealpup run --tools does it for you). The request reference has every field and a short converter. The demo above is the JavaScript one, running in a worker and loading the same files from this site instead of Hugging Face.

Results

We compared Relay 1 with 12 other models on 12 evaluations, and we report all of them, including the ones it loses.

The on-device models here are the reason Relay exists. Cactus’s Needle in particular showed that a tool-calling model can be small enough to ship inside an app and still be useful, and that gave us the confidence to try. Relay takes a different route to the same goal, reading instead of writing, and the comparisons below are there to show what that route buys and what it costs.

The 12 evaluations

Eleven of the twelve are public. The twelfth is ours.

The sealed set is the one comparison where we know nobody had an advantage. We generated it with a different model family (microsoft/phi-4) and kept only the items that Qwen3.6 independently agreed with. We then verified it manually too. It was never published and never placed in any training folder, so it is clean for every model on the table, not only for ours.

Size against score

Start with the shape of the result. This is every model’s mean score over the 12 evaluations against the size of its weights file, on a log axis.

Among models small enough to ship inside an app, Relay is alone at the top. The nearest model below it is Qwen3.5-4B, a tenth of a point behind at 9.3 GB, about 800 times the size. The smallest model above it is Qwen3.5-9B, at 19.3 GB.

Evaluation by evaluation

An average hides where a model is strong. Here is Relay against the best of the other 12 models on each set, ordered from where it leads most to where it trails most.

Two patterns stand out. Relay leads, or ties, on Mobile Actions and the two SNIPS sets, where the task is to find the right values in a short request. And it trails wherever the task asks a model to reason about a long, unfamiliar API description or to compute a value. We come back to that below, because it is the honest center of this release.

Full results

The whole table at once, every cell lit by its score. Relay’s row is outlined. Read it as a map: bright columns are easy sets, and the places where Relay’s row stays bright while the large models’ rows dim are its territory.

What the table says

Among on-device models, Relay 1 is clearly ahead. Its mean is 67.2. The best other on-device mean is 48.7, from LFM2.5-1.2B, a model about 200 times larger on disk.

Against much larger models, it has a territory of its own. It is first of all 13 on Mobile Actions (tied with Qwen3.5-4B) and on both SNIPS sets. It beats DeepSeek V4 Flash on DSTC8 (59.6 against 55.9) and on the sealed set (83.6 against 79.8), and it beats DeepSeek V4.1 Flash on ATIS (80.8 against 75.7). Its overall mean matches Qwen3.5-4B, 67.2 against 67.1, a model with more than 200 times the parameters.

Where it loses is general tool calling over arbitrary schemas. On BFCL, ToolACE and NexusRaven, every model from 2B parameters up is ahead of Relay, by 16 to 33 points at 4B to 9B, and so is Qwen3.5-0.8B. These sets reward reasoning over long API descriptions you have never seen, and computing values that are not written in the request. A 22M-parameter encoder that copies values cannot do that yet.

What each model trained on

A benchmark score means little without knowing what the model has already seen.

None of the other models documents training on Glaive, ToolACE, xLAM, SGD/DSTC8, SNIPS, ATIS, NexusRaven, BitAgent or BFCL. That does not mean they did not, only that it is not disclosed. Two individual sets deserve a caveat. Glaive is a public training set, so the two smaller LFM2.5 models’ scores above 93 there may reflect that. And BitAgent’s 800 rows contain only 37 distinct requests, so its scores move in big steps; treat it as weak evidence either way.

How Relay works

Reading, not writing

A language model produces a call by generating text, and every generated token is a chance to invent something: a city that is not in the request, a digit dropped from a phone number. Relay is an encoder. It reads the request and every tool schema together in one pass, and then three small heads point at the answer.

1. The router decides, for each tool, whether to call it and how many times. A learned “none” score handles requests that no tool fits, which is how Relay abstains.
2. The anchors say, for each call, which part of the request it comes from. This is what separates “alarm at 7” from “timer for 10 minutes” in the same sentence.
3. The argument heads choose, for each parameter, a value program and its inputs.

Value programs are the core idea. Relay never generates a value. It picks a small, checkable recipe that builds the value from the request, and the recipe runs deterministically.

Because the programs are deterministic, Relay cannot hallucinate a phone number or a city that is not in the request. They are also why it is so strong at slot filling: 93.3 on SNIPS against 66.8 for the next best model.

Decoding rules

After the heads have scored everything, a few plain rules turn scores into calls.

• Required arguments. If a call would leave a required argument empty, Relay drops that call. In an earlier version, adding this rule lifted BFCL by 1.7 points and ToolACE by 2.5, at no cost elsewhere.
• Exclusive spans. Within a call, a stretch of text fills at most one argument, and the most confident one wins.
• Call order. A call that consumes an earlier call’s output goes after it. Otherwise calls follow the order the user stated them in.

Getting a good model into 11.5 MB

The shipped model has 21.8M parameters in 11.47 MB, and its exported graphs reproduce the PyTorch model exactly on our golden records. Most of them are the encoder:

The data

Everything Relay trained on is data we can license cleanly. That was not where we started. Early internal versions used public training splits such as DroidCall, SGD and Hermes. When we reviewed them, some turned out to be GPT-4 outputs with unclear terms of use (DroidCall), some CC BY-SA (SGD), and one SNIPS split had no licence at all. So we rebuilt Relay from scratch on data we can stand behind.

How we generate it

Every generated record goes through five steps, and about two thirds are thrown away along the way.

Learning from errors without overfitting

For the last round we split six of the benchmarks in two. Half A was for analysis: we studied Relay’s errors there, such as over-abstaining on oddly worded requests, values on question-only dialogue turns, role swaps and invented defaults. Half B stayed untouched until the final check. We then generated new, similar data for those skills, never benchmark data, and the gains held on half B:

So the improvement is a real skill, not a fit to the examples we looked at.

Speed

Almost all of a request’s time, about 96%, is the encoder, and it grows with the amount of tool text.

Where Relay falls short

• General tool calling still trails 2B+ models. BFCL, ToolACE and NexusRaven reward reasoning over long, unfamiliar API documentation, and computing or rephrasing values.
• Some values cannot be expressed. Relay cannot write text that is not in the request: a composed email body, a reworded search query, or “13-minute” turned into a different format. On DroidCall roughly 15 to 25% of rows need such a value, depending on how strictly you count.
• DroidCall conventions. Models fine-tuned on DroidCall’s own training data reach 74.5 to 85 in its paper. We do not train on it, because its data were generated with GPT-4-turbo, so we miss some of its labelling conventions.
• Many tools. Without the prefilter, accuracy degrades with long tool lists and the time grows with the tool text. The prefilter fixes both when threads are available; on one thread, a list of 20 to 100 tools still takes about 0.3 s once its vectors are cached and 1 to 2 s the first time, against 0.16 s for a typical seven-tool request.
• At most 32 parameters per tool are read.
• Benchmark noise. DroidCall has 200 rows, so each row is 0.5 points, and BitAgent has 37 distinct requests.
• Harness differences. Our scorer closely matches Cactus’s numbers for Needle 3, within 1.6 points (87.5, 45.5 and 51.8 for the three sets we share, against their published 86.0, 47.0 and 50.2). Our extraction protocol differs from Cactus’s chart, so Needle 3 scores lower on DSTC8 and SNIPS here than in its own published table. And one DeepSeek request per model failed at the API (the same ToolACE row for both, rejected for an empty tool name) and is scored as no call.

What we learned along the way

• Reading plus value programs is a strong prior. A 17M-parameter encoder was ahead of Needle 3 on five of its six benchmarks the first time we ran it, after one hour of training. The distilled 17M model was then ahead on all six. Both were early versions trained partly on the training splits of three of those benchmarks.
• Size helps extraction most. Moving from 17M to 32M lifted BFCL by 4 points and SNIPS 7-way by 6. A second training stage added about 1 and 2 more, and fixed most of the remaining slot-filling gap.
• Clean data cost us first, then paid back. The first build trained only on our own data lost SNIPS (94 down to 23) and DSTC8 (78 down to 44), because it had learned short-utterance slots and dialogue state from third-party sets. Targeted generation for dialogue, dense slots and Android brought SNIPS back to 93, lifted ATIS from 13 to 81 (the third-party-data version had 46), and beat the old model on four of five battle sets and the sealed set. DSTC8 and DroidCall are still below the third-party-data version, because those benchmarks reward their own labelling conventions.
• More “don’t call” data is not free. Teaching the model to hold back helped on Glaive but cost NexusRaven 8 points, because it also abstained on real requests. The simple required-argument rule did better than more data.
• Fair harnesses matter. In our harness LFM2.5-230M scores 62.7 on BFCL, against 46.3 in Cactus’s published chart (Liquid’s own figures are 43.3 on BFCL v3 and 21.0 on v4). That is why we ran every model, Needle 3 included, ourselves, in the same harness, rather than quoting anyone’s chart.

What’s next

Relay 2 is aimed at the gaps above: judgement over long, unfamiliar tools, multi-call requests, composed values and phone actions. It has to stay as fast as Relay 1 on a low-end device. Our first experiments toward it shaped the plan:

• Data before size. Every regression we have seen so far came from data, not from the model. We will clean the “no call” examples in our new Android data, which made a retrained teacher abstain less often. We will add contrast data for the errors our demo audit found: negation, corrections (“20, no, 22”), conditionals, opposite actions such as lock and unlock, and paraphrased requests. We will not train on DroidCall’s own data: in a test it lifted DroidCall by 6 to 9 points while costing 5 on abstaining.
• A teacher that passes our checks first. The student can only learn what the teacher knows. The teacher must hold NexusRaven, BitAgent and abstaining before we distil from it.
• About twice the student, trained for longer. On the same benchmarks our 150M teacher scores 4.6 points above Relay 1, most of it on DSTC8 and SNIPS 7-way. That gap is the student’s size, not its 4-bit weights. Students were still improving when our test runs stopped, so Relay 2 distils for much longer. It stays at 4-bit, at about 17 to 18 MB. 2-bit weights and a full 68M encoder are out: 3-bit already cost accuracy, and a 68M encoder would be about twice as slow on a low-end device.
• Architecture fixes for known limits. Instance-aware anchors, or set prediction, for requests that need several calls. A calibrated call versus no-call score. A tiny copy-or-generate decoder used only for composed text such as message bodies.
• The same speed on low-end hardware. A word-matching tool shortlist and integer kernels each make Relay faster, together enough to pay for the larger student. The shortlist keeps Relay 1’s accuracy and is 1.4 to 2 times faster on a Raspberry Pi with 24 tools.
• A much larger licence-clean Android world. 332 more hand-written Android functions, about 33k new verified training records, function-name masking, and error-driven data rounds.
• A vocabulary built only from our own data (see the caveat above).

We will report it the same way: one harness for every model, the sealed set, an untouched half of every benchmark we study, and, new for Relay 2, a larger abstention test, because the sealed set’s 102 no-call requests cannot separate a two-point difference from noise.