Run Rules a Model Writes
Your support team wants to change how tickets are routed. Today that's a ticket for engineering. Instead, they write the policy in plain English, a model turns it into code once, your app checks that code, and then it runs on every ticket, in your process, in under a millisecond, with no model call per ticket.
The hard part has always been the middle: running code a model wrote. Stof makes that the easy part, because the model's output is a document that can only see itself and the functions you give it.
The Agreement
Your app and the model agree on a small contract, and everything else is up to the model:
- Each rule is a function tagged
#[route]that takes aticketobject. - A ticket has
subject,body, andplan. Rules setticket.teamandticket.priority. - Rules can run in any order, so each one only sets what it's responsible for.
- The model includes
#[test]functions that show its rules working.
Your app owns one small function that runs every rule on a ticket. The model never writes this part:
/// Runs every #[route] rule on a ticket and returns the routed ticket.
fn route(input: map) -> map {
using ticket = Obj::from_map(input);
for (const rule in funcs(attributes = 'route')) rule(ticket);
ticket.to_map()
}
What the Model Writes
Given the policy "Refunds and billing questions go to billing. Outages go to infra and are urgent. Enterprise customers are always urgent. Everything else goes to support," a model writes something like this. Press Run to route a few tickets with it:
/// Billing questions go to billing; outages go to infra, urgently.
#[route]
fn by_topic(ticket: obj) {
const text = (ticket.subject + ' ' + ticket.body).lower();
if (text.contains('refund') || text.contains('charged')) ticket.team = 'billing';
else if (text.contains('outage') || text.contains('is down')) {
ticket.team = 'infra';
ticket.priority = 'urgent';
}
else ticket.team = 'support';
}
/// Enterprise customers are always urgent.
#[route]
fn by_customer(ticket: obj) {
if (ticket.plan == 'enterprise') ticket.priority = 'urgent';
else ticket.priority = ticket.priority ?? 'normal';
}
#[test]
fn refunds_go_to_billing() {
using ticket = new { subject: 'Refund please', body: 'I was charged twice', plan: 'pro' };
self.by_topic(ticket);
assert_eq(ticket.team, 'billing');
}
#[test]
fn outages_are_urgent() {
using ticket = new { subject: 'API is down', body: '', plan: 'free' };
self.by_topic(ticket);
self.by_customer(ticket);
assert_eq(ticket.priority, 'urgent');
}
// ---- the app's harness, and a demo ----
fn route(input: map) -> map {
using ticket = Obj::from_map(input);
for (const rule in funcs(attributes = 'route')) rule(ticket);
ticket.to_map()
}
#[main]
fn main() {
const tickets = [
{'subject': 'Charged twice', 'body': 'Please refund one', 'plan': 'pro'},
{'subject': 'Dashboard is down', 'body': '', 'plan': 'enterprise'},
{'subject': 'How do I export?', 'body': '', 'plan': 'free'},
];
for (const input in tickets) {
const routed = self.route(input);
pln(routed.get('subject') + ' -> ' + routed.get('team') + ', ' + routed.get('priority'));
}
}Anyone on the team can read that, including the people who wrote the policy. It's short because Stof reads like the JSON and JavaScript the model already knows.
Asking the Model
Give the model three things: the Stof skill, so it writes correct Stof; the contract above; and the policy. Ask for Stof only, so the response is the document:
// stofSkill is the text of SKILL.md; model.generate is your model API call
const prompt = (policy: string) => `
${stofSkill}
Write Stof rules for routing support tickets.
- Each rule is a function tagged #[route] that takes a ticket: obj.
- A ticket has subject, body, and plan. Set ticket.team and ticket.priority.
- Rules run in any order, so each rule only sets what it's responsible for.
- Include #[test] functions that check your rules on example tickets.
Reply with only the Stof document.
The policy:
${policy}`;
Checking It Before You Trust It
Three checks, in order. Each one fails with a plain message that says what's wrong and where, so the message goes straight back to the model for another try:
import { StofDoc } from '@formata/stof';
// Tickets your team has already routed by hand: the rules have to agree with them.
const EXAMPLES = [
{ ticket: { subject: 'Charged twice', body: 'Please refund one', plan: 'pro' }, team: 'billing', priority: 'normal' },
{ ticket: { subject: 'Dashboard is down', body: '', plan: 'enterprise' }, team: 'infra', priority: 'urgent' },
{ ticket: { subject: 'How do I export?', body: '', plan: 'free' }, team: 'support', priority: 'normal' },
];
async function check(rules: string): Promise<string | null> {
try {
// 1. It parses, and the model's own tests pass.
const tested = await StofDoc.new(rules, 'stof', 'test');
tested.setMaxExecutionTime(1000);
await tested.run('test');
tested.dispose();
// 2. It agrees with tickets your team already routed.
const doc = await StofDoc.new(HARNESS + rules);
doc.setMaxExecutionTime(100);
for (const { ticket, team, priority } of EXAMPLES) {
const routed = await doc.call('route', ticket) as Map<string, unknown>;
if (routed.get('team') !== team || routed.get('priority') !== priority) {
return `For ${JSON.stringify(ticket)} you routed to ${routed.get('team')}/${routed.get('priority')}, ` +
`but it should be ${team}/${priority}.`;
}
}
doc.dispose();
return null;
} catch (error) {
return String(error); // a parse error or a failed test, with the line it happened on
}
}
export async function compileRules(policy: string): Promise<string> {
let feedback = '';
for (let attempt = 0; attempt < 3; attempt++) {
const rules = await model.generate(prompt(policy) + feedback);
const problem = await check(rules);
if (!problem) return rules; // store this with the policy
feedback = `\n\nYour last version had a problem. Fix it:\n${problem}`;
}
throw new Error('the model could not write rules that pass the checks');
}
A mistake comes back looking like this, and it's exactly what the model needs to fix it:
parse error: expected ';'
--> line 3, column 16
|
3 | ticket.team = ;
| ^
The third check is a person. The rules are short and readable, so show them, with their tests, to whoever wrote the policy before switching them on.
Running It on Every Ticket
Load the stored rules once, and call route for each ticket:
const doc = await StofDoc.new(HARNESS + storedRules);
doc.setMaxExecutionTime(50); // milliseconds: a rule that loops forever fails, it doesn't hang
export async function routeTicket(ticket: Ticket) {
const routed = await doc.call('route', ticket) as Map<string, unknown>;
return Object.fromEntries(routed); // { ..., team: 'infra', priority: 'urgent' }
}
The rules can't read files, environment variables, or the network, and they can't call anything in your app. All they can do is look at the ticket and set fields on it. If you want them to know more, say a customer's open-ticket count, give them exactly that function: doc.lib('App', 'open_tickets', countOpenTickets), then add it to the contract.
Why This Works
- No model call per request. The model runs once, when the policy changes. Routing is fast, cheap, and gives the same answer every time.
- No infrastructure. No container or sandbox service to run. The rules run inside your app like any other data.
- Checked before it's trusted. Parse errors, the model's tests, your examples, and a person's review, all before a single real ticket.
- Readable. The rules are the documentation of the policy, so the people who own the policy can review them.
The same shape works for anything a model can write once and your app runs many times: lead scoring, fraud flags, content moderation rules, data cleanup, alert conditions. For logic that changes during a conversation, see Agent Tools at Runtime.