# Stof: run AI-generated logic safely, inside your app > Stof is JSON with functions: an open-source data runtime where one document holds data and the logic that works on it. Apps run it in a sandbox, in-process, in Rust, JavaScript/TypeScript, or Python, which makes it a safe way to run logic written by AI models, users, or customers. Stof is the policy engine behind Limitr. Stof is JSON with functions: one document holds data and the logic that works on it, and an app runs it in a sandbox, in-process, in Rust, JavaScript/TypeScript, or Python. Its main use is running logic that a model, user, or customer wrote, safely inside an app. Documents can only see themselves and the functions the host gives them; file system, environment, and network access are off by default. - Start with the overview: https://stof.dev/what-is-stof (what Stof is, use cases, comparisons with similar tools, FAQ) - Try it in the browser: https://stof.dev/playground - Install: npm i @formata/stof (JavaScript/TypeScript), cargo add stof (Rust), pip install stof (Python), cargo install stof-cli (command line) - Teach a model to write Stof: https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md - Source (Apache 2.0): https://github.com/dev-formata-io/stof - In production: Stof is the policy engine behind Limitr (https://limitr.dev), a pricing runtime for usage-based and AI software made by Formata, Inc. ## What is Stof? **Stof is JSON with functions.** One document holds data and the logic that works on it, and your app runs it in a sandbox, in-process, in Rust, JavaScript/TypeScript, or Python. Its main use is running logic that someone else wrote, like a model, a user, or a customer, safely inside your app. Stof is open source (Apache 2.0), made by [Formata, Inc.](https://limitr.dev), and is the policy engine behind [Limitr](https://limitr.dev). - Website and docs: [stof.dev](https://stof.dev) - Source: [github.com/dev-formata-io/stof](https://github.com/dev-formata-io/stof) - Packages: [`@formata/stof`](https://www.npmjs.com/package/@formata/stof) (npm), [`stof`](https://crates.io/crates/stof) (crates.io), [`stof`](https://pypi.org/project/stof/) (PyPI), [`stof-cli`](https://crates.io/crates/stof-cli) (command line) - Try it: [the playground](https://stof.dev/playground) runs Stof in your browser ## What it looks like The data looks like JSON. The function next to it works on that data: ```stof price: 20 discount: (p: float): float => p * 0.8 fn total() -> float { self.discount(self.price) } ``` Your app loads the document and works with it like data: ```typescript const doc = await StofDoc.new(source); await doc.call('total'); // 16 doc.set('price', 50); // change the data doc.parse('discount: (p: float): float => p * 0.5'); // change the logic, the same way await doc.call('total'); // 25 ``` The discount is a function stored in a field, so new logic goes in the same way new data does. ## The problem it solves Models are good at writing small pieces of logic: a routing rule, a validation check, a data transform, a scoring function, a tool. Products also keep needing logic their own team didn't write: customer rules, pricing policies, user-defined filters and workflows. Running that logic is the hard part: - Generated Python or JavaScript in a container or microVM is heavy, slow to start, and more infrastructure to run and secure. - `eval` in your app gives the code access to everything your app can reach. - Function calling (tool calls, MCP) only lets a model call functions you already wrote. It can't add new logic. - Rules written as JSON (rule DSLs) are limited, and awkward to read and write for people and models alike. - WebAssembly needs a compiler toolchain, binaries are large to send, and the logic lives apart from its data. With Stof, the model (or user, or customer) writes a document, and your app runs it like any other data it receives: in-process, in under a millisecond for a small document, with access to nothing but itself and the functions you give it. ## How it works: everything is data Stof has functions, but it isn't a language for writing apps. It's a document, and every part of it is data. A Stof document is a graph of objects, and fields, functions, types, and docs are all pieces of data attached to those objects, like components in an entity-component system (ECS): ```text root ├── price: 20 field ├── discount: (p) => p * 0.8 field holding a function └── total() function, data on the object like a field ``` That's what makes the rest possible: - **Anything you can do with data, you can do with logic.** Merge two documents, pull one object out with its functions, diff two versions, store it in a database, or send it over the wire as compact binary. - **A document can change while it runs.** It can parse new fields and functions into itself, or drop them, with no build step. - **It grows to fit your domain.** Your app adds its own functions, types and schemas live in the document, and in Rust you can add your own formats and kinds of data. Instead of designing a rules language, config format, or DSL for your product, describe your domain in Stof. - **There's very little to learn.** If you can read JSON and a bit of JavaScript, you can read Stof, and so can a model. ## Key facts | Fact | Detail | |---|---| | What it is | A data runtime: JSON with functions, types, units, and schemas, run in a sandbox | | Runs in | Rust (native), JavaScript/TypeScript (WebAssembly, in Node.js and browsers), Python | | Sandbox | Documents can only see themselves and the functions the host gives them. File system, environment, file imports, and network are off by default. Each call has a time limit. | | Speed | A small document parses and runs in under a millisecond; calls after that take microseconds | | Formats | Imports and exports JSON, YAML, TOML, and more; compact binary (`bstf`) for sending documents, functions included | | Language features | Types, prototypes, schemas (`#[schema]`), units (`5min + 30s == 330s`), destructuring, async, attributes, tests in the document (`#[test]`) | | Tooling | `stof` CLI (`stof run`, `stof test`, `stof lsp`), a [VS Code extension](https://marketplace.visualstudio.com/items?itemName=Formata.stof), and a [skill file](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md) to teach AI models Stof | | License | Apache 2.0 | | Made by | Formata, Inc. Created by CJ Cummings. | ## Use cases Reach for Stof when: - **A model, a user, or a customer gives you logic you need to run**: rules, filters, transforms, scoring, tool behavior, agent actions. - **You're about to invent a config format, rules engine, or DSL** for your product. - **The same logic has to run in more than one place** (browser, server, worker, edge) and give the same answer. - **You want to change behavior by sending a new document**, not by shipping a deploy. - **Logic should travel with the data it belongs to**, through APIs, queues, and databases. Common examples, each with a complete recipe in the [Cookbook](/cookbook/overview): - **[Logic a model writes](/cookbook/ai-rules)**: have a model turn a policy into rules once, check them, and run them on every request with no model call. - **[AI agents](/cookbook/agent-tools)**: let an agent write new tools and logic at runtime and run them safely, instead of only calling functions you wrote ahead of time. Keep agent state and context as a document that carries its own behavior. - **[Pricing and usage policies](/cookbook/usage-pricing)**: plans, limits, credits, and metering rules as documents pushed to running apps. This is how [Limitr](https://limitr.dev) uses Stof. - **[Smart configuration](/cookbook/config)**: config files that validate themselves and compute values, while still exporting to plain JSON, YAML, or TOML. - **[Rules and policies](/cookbook/customer-rules)**: routing, validation, eligibility, and approval rules that non-engineers or models can write. - **[Data unification](/cookbook/webhooks)**: load JSON, YAML, TOML, and API responses into one document, add logic, and hand back plain JSON. - **[Distributed systems](/cookbook/shared-logic)**: send a piece of logic with its data to wherever it needs to run, and get the same result there. Use something else for your own application code, compute-heavy work, or long-running services. Write those in your main language and give them to Stof documents as functions. ## Stof in production: Limitr Stof is the policy engine behind [Limitr](https://limitr.dev), a pricing runtime for usage-based and AI software. Limitr is an in-process usage control and pricing layer: apps use it to meter usage, enforce limits and credits, and charge customers, with margins in view. Every plan, credit limit, and validation rule is a Stof document, pushed from Limitr Cloud to running apps and evaluated in-process on every call. The Limitr engine is open source too. ## How Stof compares Stof sits between several kinds of tools. Here's where it overlaps and where it differs. **Code sandboxes for AI** (containers, microVMs like Firecracker, hosted sandboxes like E2B or Modal, Pyodide, V8 isolates): these run general Python or JavaScript in isolation, out of process. Choose them when the model needs a full language and its packages. Choose Stof when the logic is small, should run inside your app with no infrastructure, and should only reach what you allow. **Function calling and MCP**: a model calls tools you already wrote. Stof works alongside them: the model can write new logic as a document, and that logic calls the same functions you expose. **Rules engines and JSON rule formats** (JSON Logic, json-rules-engine, Drools, GoRules): rules as data or a separate rules language. Stof rules are readable code that is also data, with types, functions, and schemas, in one document that can carry its own data too. **Expression and policy languages** (Google's CEL, Open Policy Agent and Rego, AWS Cedar): designed to evaluate expressions or authorization policies against input. Stof is broader: whole documents with data, functions, types, and state that can change while they run. **Configuration languages** (Jsonnet, CUE, Dhall, Pkl, Nickel, KCL): these mostly generate static JSON or YAML. A Stof document can export static JSON, YAML, or TOML too, but it also stays live: its functions run when your app calls them. **Embedded scripting languages** (Lua, Starlark, Rhai, QuickJS): these embed a language for your code to call. In Stof the logic lives inside the data document, so it moves, merges, and serializes with the data. **WebAssembly** (Wasmtime, Extism): portable compiled code. Stof is readable source that models can write directly and that is small to send, with no compile step. (The Stof runtime itself is compiled to WebAssembly for JavaScript.) **Schema and validation tools** (JSON Schema, Zod, Pydantic): Stof documents can validate themselves with `#[schema]`, and the schema is part of the same document as the data. ## FAQ **Is Stof a programming language?** Stof has a language for its functions, but it's a data runtime: a document format where data and logic live together, plus the library that runs it. You don't write apps in Stof. You give your app documents that it loads, runs, and exports. **Is it safe to run Stof code that an AI model wrote?** Yes, that's what it's built for. A document can only read and change itself and call the functions your app gives it. File system, environment variables, file imports, and network access are off by default in every language, and a host turns them on explicitly (with an allowlist for network hosts). Each call has a time limit. **Can AI models write Stof?** Yes. It reads like JSON and JavaScript, which models already know. For better results, give your agent the [Stof skill](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md). Errors give the exact line and a plain explanation, so a model can fix its own mistakes. **Is JSON valid Stof?** Yes. Any JSON document is valid Stof, so you can add types and functions to data you already have and export it back to JSON, YAML, or TOML. **Which languages can run Stof?** Rust (the `stof` crate), JavaScript and TypeScript through WebAssembly (the `@formata/stof` npm package, for Node.js and browsers), and Python (the `stof` package on PyPI). The same document gives the same result in each. **How fast is it?** It runs inside your process, with no server or cold start. A small document parses and runs in under a millisecond, and later calls take microseconds. **Is Stof open source?** Yes, under the Apache 2.0 license, at [github.com/dev-formata-io/stof](https://github.com/dev-formata-io/stof). **Who makes Stof?** Formata, Inc., the company behind [Limitr](https://limitr.dev). Stof was created by CJ Cummings. **Where do I start?** Try the [playground](https://stof.dev/playground), then the [quickstart](/start/quickstart). Questions are welcome on [Discord](https://discord.gg/Up5kxdeXZt). --- ## About These Docs ## What's Where - **Start**: what Stof is, a quick start, installing it, using it from your app, and the tools. - **Learn**: the language, one topic per page. Read it in order the first time: each page builds on the ones before it. - **Libraries**: every built-in function, by library. Reference material to look things up in. - **Cookbook**: complete examples of real uses, with the app code and the Stof together. If you're new, read [What is Stof?](/what-is-stof) and the [Quick Start](./quickstart), then work through [Learn](/learn/fields) from the top. ## Runnable Examples Most examples have a **Run** button: str { \`Hello, \${name}!\` } #[main] fn main() { pln(self.greet()); } `} /> They run in your browser, using the same Stof runtime as the npm package. Nothing is sent to a server. Edit the code and run it again, or open it in the [playground](/playground) to keep working on it. Each one follows the same shape: the example's fields and functions, then a `#[main]` function that prints what's worth seeing. `#[main]` marks the functions that run when a document is run, here by the **Run** button. `pln(..)` prints a line to the output. A few examples have no **Run** button. Those use something this site's runtime doesn't have, like files or network access, or they're app code in TypeScript, Python, or Rust. ## Documents and Apps You'll see two kinds of code: Stof documents, and app code that uses them. In a real project, your app usually calls a document's functions directly (`doc.call('total', 3)`) or runs the ones with a certain attribute (`doc.run('triage')`), and uses what they return. `#[main]` and `pln` are just the simplest way to show a document working on its own. See [Using Stof in Your App](./using-stof). --- ## How Stof Works A few ideas explain almost everything Stof does. This page goes through them in order. ## A Document Is a Graph of Objects A Stof document is a set of **objects**. Each object holds named pieces of data, its **fields** and **functions**, and can have child objects: ```stof server: { port: 8080 address: 'localhost' fn url() -> str { `https://${self.address}:${self.port}` } } ``` Here `server` is an object with two fields and a function. Inside a function, three words find your way around: - `self` is the object the function is on, so `self.port` is the `port` field next to `url`. - `super` is that object's parent. - `root` is the top-level object above it. The top level of a document is an object too, the **main root**. A document can have other roots (`root Settings { .. }`), and imports can create them. Inside the runtime, a document is stored flat: a list of objects and a list of data, connected by IDs. Fields, functions, and other data (like an image or a PDF) are all the same kind of thing, a component attached to an object, a bit like an entity-component system in a game engine. That's why moving, copying, or sending part of a document is cheap, and why functions can be treated like any other data. ## Data and Logic in One Document Any JSON document is already a Stof document: ```json { "name": "Stof", "age": 30, "active": true } ``` The quotes, commas, and outer braces are optional. Add types where you want them, and a function next to the data: ```stof name: 'Stof' int age: 30 bool active: true fn greet() -> str { `Hello, ${self.name}!` } ``` Nothing moved to a separate codebase. The function sits on the same object as the fields, found the same way, and travels with them. ## Functions Are Data Because a function is a component on an object, everything you can do with data works for logic too: - **Store it in a field**: `discount: (p: float): float => p * 0.8`, then `self.discount(20)`. - **Find it by attribute**: `self.funcs('rule')` lists the functions tagged `#[rule]`, so new rules join in without anything else changing. - **Merge it**: parse two documents into one, and their functions are side by side. - **Send it**: export an object as binary Stof (`bstf`) and its functions go with it, ready to run on the other side. ## Sandboxed by Default A document can only see and change itself, and call the library functions it has. File system, environment variables, file imports, and the network are all off until the app running it turns them on, and it can turn on the network for only a few hosts. Each call also has a time limit. That's what makes it safe to run a document from somewhere you don't fully trust: a model, a customer, an API, another service. The app decides what the document can reach, usually by giving it a few specific functions of its own. See [Using Stof in Your App](./using-stof#what-a-document-can-reach). A document can check what it has before using it. `Lib?::func()` gives null when a library or function isn't there, so the same document works with or without an optional host library: ```stof const sent = App?::notify('order shipped') ?? false; ``` ## Documents Change Themselves A running document can parse new fields, types, and functions into itself, and remove them again. In this example the app supplies some rules (they could come from a database or a model), and the document loads them, runs them, and drops them: ```stof order: { total: 120, country: 'DE' } fn apply_rules() { using rules = new {}; // dropped when this function ends parse(await App::rules(), rules); // Stof source from the app for (const rule in rules.funcs('rule')) rule(self.order); } ``` ```typescript doc.lib('App', 'rules', async () => ` #[rule] fn vat(order: obj) { if (order.country == 'DE') order.vat = order.total * 0.19; } #[rule] fn free_shipping(order: obj) { order.shipping = order.total >= 100 ? 0 : 5; } `, true); await doc.call('apply_rules'); doc.record(); // { order: { total: 120, country: 'DE', vat: 22.8, shipping: 0 } } ``` There's no build step. Logic arrives as data, runs, and leaves. ## One Runtime, Everywhere Stof's runtime is written in Rust. The JavaScript package is that same runtime compiled to WebAssembly, and the Python package wraps it too, so a document gives the same result in a browser, in Node.js, on a server, and in Python. It runs in your app's process, with no server or container. Inside the runtime, each call runs as a lightweight process. Processes take turns, so `async` functions and `await` work the same way on every platform, including in a browser. See [Async](/learn/async). ## Back Out Again A document built from JSON and extended with types and functions still exports as JSON, YAML, TOML, or any other format the runtime has, at any time: ```typescript doc.stringify('json'); // functions are left out doc.stringify('yaml'); doc.blobify('bstf'); // everything, functions included ``` Add logic, run it, and hand the result to something that has never heard of Stof. --- ## Install Stof is a library you add to your app: an npm package, a Python package, or a Rust crate. There's also a command-line tool for running and testing `.stof` files. You don't need to install anything to learn the language: every example in these docs has a **Run** button, and the [playground](/playground) runs in your browser. Each tab below runs the same document: ```stof title="hello.stof" name: 'Stof' fn greet() -> str { `Hello, ${self.name}!` } #[main] fn main() { pln(self.greet()); } ``` ```bash npm i @formata/stof ``` [@formata/stof on npm](https://www.npmjs.com/package/@formata/stof) ```typescript title="index.ts" const doc = await StofDoc.new(` name: 'Stof' fn greet() -> str { \`Hello, \${self.name}!\` } #[main] fn main() { pln(self.greet()); } `); await doc.run(); // Hello, Stof! console.log(await doc.call('greet')); // Hello, Stof! ``` In JavaScript, `pln` writes to `console.log` and `err` to `console.error`. To send a document's output somewhere else, replace them with your own functions: `doc.lib('Std', 'pln', (...args) => ...)`. Usually you'll call functions and use what they return instead. ### Browsers and Bundlers `StofDoc.new` loads Stof's WebAssembly automatically in Node.js and in browsers without a bundler. A bundler like Vite or webpack changes how files are loaded, so give it the file yourself, once, before creating documents: ```typescript title="Vite" await initStof(stofWasm); ``` ```typescript title="Other bundlers (webpack, Rollup, ...)" await initStof(await stofWasm()); ``` Webpack doesn't load `.wasm` files on its own. If that import fails to parse, add a rule that treats them as files: ```js title="webpack.config.js" module: { rules: [{ test: /\.wasm$/, type: 'asset/resource' }] } ``` ```bash pip install stof ``` [stof on PyPI](https://pypi.org/project/stof/) ```python title="main.py" from pystof import Doc doc = Doc() doc.parse(""" name: 'Stof' fn greet() -> str { `Hello, ${self.name}!` } #[main] fn main() { pln(self.greet()); } """) doc.run() # Hello, Stof! print(doc.call('greet')) # Hello, Stof! ``` The package is named `stof` on PyPI and imported as `pystof`. ```bash cargo add stof ``` [stof on crates.io](https://crates.io/crates/stof) ```rust title="main.rs" use stof::model::Graph; use stof::runtime::Runtime; fn main() { let mut graph = Graph::default(); graph.parse_stof_src(r#" name: 'Stof' fn greet() -> str { `Hello, ${self.name}!` } #[main] fn main() { pln(self.greet()); } "#, None, Default::default()).unwrap(); graph.run(None, true).unwrap(); // Hello, Stof! let greeting = Runtime::call(&mut graph, "greet", vec![]).unwrap(); println!("{greeting:?}"); } ``` The default features cover the language, the standard libraries, and file access (which documents only get if you allow it). Add features for more: `http` (network requests), `image`, `pdf`, `docx`, and `age_encrypt`, or `full` for all of them. ```bash cargo install stof-cli ``` [stof-cli on crates.io](https://crates.io/crates/stof-cli). You'll need [Rust](https://www.rust-lang.org/tools/install), which comes with Cargo. ```bash stof run hello.stof # Hello, Stof! ``` The CLI is for running and testing your own `.stof` files, so documents it runs can use files, environment variables, and the network. See [Tools](./tools) for its other commands. ## Editor Support The [Stof extension for VS Code](https://marketplace.visualstudio.com/items?itemName=Formata.stof) highlights `.stof` files, and with the CLI installed it also shows errors as you type, goes to definitions, completes names, and renames across files. See [Tools](./tools). ## Teaching an AI Model Stof Models can read and write Stof already, since it looks like JSON and JavaScript. To make them reliable, give your agent the [Stof skill](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md): a short guide to the language and the mistakes to avoid, with the library references in [stof.zip](https://github.com/dev-formata-io/stof/blob/main/content/stof.zip). --- ## Stof Mission Every interchange format we have (JSON, YAML, TOML, XML) describes a snapshot. The moment it crosses a wire, it's inert again on the other side, and whatever system receives it has to already know, out-of-band, what to do with it. The logic that gives the data meaning lives somewhere else entirely: a separate codebase, a separate language, a separate deploy. That gap is where schema drift, versioning hell, and "the API changed and nobody told the client" all come from. **Stof exists to close that gap.** ## What Stof Is Stof is a data runtime. It isn't a data format, a programming language, or a database. The same way a JavaScript runtime executes JavaScript, a data runtime executes data. A Stof document doesn't just describe something; it validates itself, transforms itself, and acts, wherever the runtime lands. Concretely: valid JSON is already valid Stof. You aren't migrating to a new format. You're giving your existing data a place to keep the logic that used to live somewhere else. ## Principles These are the opinions the runtime is built around, and the constraints we design against. **Data and logic belong together.** Splitting them across a data file and a separate codebase is where the gap comes from. Stof documents carry both, as one unit. **Portable means portable.** The same document runs native, in WebAssembly, or embedded in another host, with the same behavior every time. If it doesn't run everywhere, it isn't portable. **Sandboxed by default.** A document can only see and manipulate itself unless you explicitly hand it more. Logic that arrives over the wire has to be safe to run without a review cycle first. **Documents should be able to grow.** A running document parsing new fields, new types, even new functions into itself is the model working as intended. Data that can only be replaced, never extended, is still just a snapshot. **Interop with all data formats.** A superset of JSON that works with every other data format, even ones that don't exist yet: binary, images, PDFs, DOCX, YAML, TOML, JSON, and so on. ## Where This Came From Stof comes out of a decade spent building parametric and geometry formats for CAD and graphics systems, the kind of work where a document's structure and its behavior have to travel together, and it gets complex quickly. That's where the core architectural insights come from: represent everything as a flat graph of nodes and data components, connected by pointers rather than nested copies, and moving or transforming part of the document stops being expensive. Stof started as a solo project while trying to send wasm over the wire, and it's been running in production since, including as the policy engine underneath [Limitr](https://limitr.dev), where every plan, credit limit, and validation rule is a live Stof document. ## Where This Is Going Everything above (the graph, the sandboxing, the portability, the self-transformation) is domain-agnostic. None of it was built for one use case. What changes across contexts is only what the document is asked to describe. ### Security A scoped auth context modeled as a document instead of a token you have to look up elsewhere: something that can grant, check, and revoke its own permissions, expand temporarily for a single operation, and expire on its own, without a separate service tracking its lifecycle. ### Graphics A scene graph is already Stof's native shape. Geometry, materials, and animation curves as typed fields and functions on the nodes they describe, instead of a mesh format with behavior bolted on by reference. Geometry is defined by the function that generates it, not just the triangles it resolves to at one point in time. ### AI Contexts Portable memory for an agent: what it knows and what it's allowed to do, traveling together, handed off between services instead of reconstructed at each one. A tool surface a document can extend on itself at runtime. The further-out version, documents that discover and wire themselves into each other and form a network without a central coordinator, is the reason the sandboxing model exists in the first place. ### Distributed Systems Configuration and coordination state that carries the logic that validates it. A service reading a config another service wrote doesn't have to trust it blindly: the document can check itself. ### Plugins A plugin as a sandboxed document instead of a compiled binary or an interpreted script with full host access: new behavior shipped into a running system as data, without a new deploy and without widening what that behavior is allowed to touch. ### Data Unification One graph that JSON, YAML, TOML, and binary sources all resolve into, so two services stop maintaining a translation layer just to agree on "the format we use" versus "the format they use." ### Datasets Evaluation logic, metadata, and transformation code living inside the dataset rather than in a notebook next to it: a dataset that can score its own outputs or validate its own rows. ### Smart Configs A config that enforces its own invariants: a value that can't be set outside its valid range, a field that recomputes a dependent value automatically, without a schema file in a different repo that someone forgets to update. --- This list isn't exhaustive, and it isn't a roadmap with dates attached. It's what falls out naturally from building one thing well: a document that carries its own data, its own logic, and its own boundaries, and runs the same way everywhere it lands. --- ## Quick Start A Stof document looks like JSON with functions. Press **Run**: str { \`Hello, \${self.name}!\` } #[main] fn main() { pln(self.greet()); } `} /> `name` is a field, like in JSON. `greet` is a function stored on the same object, and `self` is that object. `#[main]` marks the function that runs when the document is run. Change the name and run it again. That's the whole idea. The rest of this page runs a document from an app, which is how Stof is meant to be used. ## Run It From Your App You'll need [Node.js](https://nodejs.org) 18 or newer. Make a project and install Stof: ```bash mkdir stof-quickstart && cd stof-quickstart npm init -y npm i @formata/stof ``` Create `index.mjs`: ```javascript title="index.mjs" const doc = await StofDoc.new(` price: 20 discount: 0.2 fn total(quantity: int) -> float { self.price * quantity * (1 - self.discount) } `); console.log(await doc.call('total', 3)); // 48 ``` ```bash node index.mjs ``` Your app loaded a document, then called one of its functions by name and got a value back. The document runs inside your process, and it can only see itself. ## Change the Data, Then the Logic A document is data, so your app can change it while it runs. Add this to the end of `index.mjs`: ```javascript title="index.mjs" doc.set('discount', 0.5); console.log(await doc.call('total', 3)); // 30 // new logic, parsed in the same way as new data doc.parse(` fn total(quantity: int) -> float { const subtotal = self.price * quantity; if (quantity < 3) return subtotal; subtotal * (1 - self.discount) } `); console.log(await doc.call('total', 1)); // 20 console.log(await doc.call('total', 3)); // 30 ``` The second `total` replaced the first. It could just as well have come from a database, an API, a customer's settings, or an AI model. That's where Stof is most useful: running logic that your team didn't write, safely. ## Get It Back Out A document exports like any other data: ```javascript title="index.mjs" console.log(doc.stringify('json')); // {"price":20,"discount":0.5} console.log(doc.record()); // the same, as a JavaScript object ``` Functions aren't part of JSON, so they're left out. To keep them, export the binary format instead: `doc.blobify('bstf')`. ## Where to Go Next - **[Install](./install)**: Python, Rust, the CLI, browsers and bundlers, and the VS Code extension. - **[Using Stof in Your App](./using-stof)**: calling functions, host functions, permissions, time limits, and errors. - **[Learn](/learn/fields)**: the language, one topic at a time, starting with fields. - **[Cookbook](/cookbook/overview)**: complete examples of real uses. - **[Playground](/playground)**: try things in your browser. --- ## Tools ## The `stof` Command The CLI runs and tests `.stof` files on your machine. Install it with Cargo: ```bash cargo install stof-cli ``` Since it runs your own files, documents run by the CLI can use the file system, environment variables, imports from disk, and the network. That's different from an app embedding Stof, where all of those are off unless the app allows them. ### Running `stof run` parses a file (or a package directory) and calls its `#[main]` functions: ```bash stof run pricing.stof # every #[main] function stof run pricing.stof -a nightly # every #[nightly] function instead ``` When something fails, you get the message, the line, and the Stof call stack: ```text error: cannot multiply null: the right value is null or missing (use ?? to give a default) --> pricing.stof:4:5 | 4 | units * self.rat | ^ at root.total (pricing.stof:4:5) at root.main (pricing.stof:9:5) ``` ### Testing `stof test` runs every `#[test]` function and exits with an error code if any fail, so it works in CI: ```stof title="pricing.stof" rate: 2 fn total(units: int) -> float { units * self.rate } #[test] fn totals() { assert_eq(self.total(3), 6); } #[test(10)] // passes if the function returns 10 fn five_units() -> float { self.total(5) } ``` ```text $ stof test pricing.stof running 2 tests ... test root totals ... ok test root five_units ... ok test result: ok. 2 passed; 0 failed; finished in 0s ``` `stof test --leaks` also fails any test that leaves objects in the document it didn't clean up, and says where they are. See [Testing](/learn/testing) for writing tests and [Memory Management](/learn/memory-management) for cleaning up. ### Packages and Docs A directory with a `pkg.stof` file is a package: `stof run` and `stof test` take the directory, and the package's files are parsed together. `stof pkg ` bundles one into a single `.pkg` file, and `stof unpkg` unpacks it. `stof docs ` writes Markdown docs for a document from its doc comments (`///` and `/** */`), along with references for the libraries. ## VS Code The [Stof extension](https://marketplace.visualstudio.com/items?itemName=Formata.stof) highlights `.stof` files. With the CLI installed, it also starts the Stof language server (`stof lsp`) and gives you: - **Errors and warnings as you type**, including problems in imported files. - **Go to definition** for fields, functions, types, and imports. - **Hover** for signatures, field types and values, doc comments, and every library function. - **Completion** for fields, functions, variables, types, and libraries. - **Find references** and **rename** across a package. - **An outline** of objects, types, fields, and functions. If `stof` isn't on your PATH, set `stof.server.path` in the extension's settings to the CLI's full path. The language server reads a file in the context of its package when there's a `pkg.stof`, so types from other files resolve. It never runs your code beyond computing field values, and stops those after 2 seconds. Other editors can use the same server: run `stof lsp` as a language server over stdio. ## Teaching AI Models Stof Models already read and write Stof well, since it looks like JSON and JavaScript. To make them reliable, give your agent or coding assistant the [Stof skill](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md). It's a short guide to the language and the mistakes models (and people) tend to make, with each library's reference in [stof.zip](https://github.com/dev-formata-io/stof/blob/main/content/stof.zip). When a model's document fails, Stof's error messages say what went wrong and where, in plain language. Pass them back to the model and it can fix its own mistakes. --- ## Using Stof in Your App Stof runs inside your app, as a library. Your app creates documents, puts data and logic into them, calls their functions, and reads the results. This page covers each of those steps. The examples are TypeScript; [Python and Rust](#python-and-rust) work the same way, with the method names listed at the end. ## Loading a Document `StofDoc.new` creates a document from Stof source, a JavaScript object, or bytes: ```typescript const doc = await StofDoc.new(` pricing: { rate: 2 fn total(units: int) -> float { units * self.rate } } `); ``` A document can keep taking in more, in any format it knows. Each parse adds to what's already there: ```typescript doc.parse({ customer: { name: 'Ada', plan: 'pro' } }); // a JavaScript object doc.parse('limits:\n calls: 100\n', 'yaml'); // YAML (also 'json', 'toml', ...) doc.parse(rulesFromAModel); // more Stof: data, functions, or both ``` That's how data and logic from different places end up in one document. See [Formats](/learn/formats) for every format. ## Calling Functions `call` takes a path to a function and its arguments, and returns what the function returns: ```typescript await doc.call('pricing.total', 21); // 42 await doc.call('pricing.total', '4'); // 8: arguments are cast to the parameter's type ``` Paths start at the document's top level, so `'pricing.total'` is the `total` function on the `pricing` object. Calls are async because a document can do async work, like waiting on one of your functions. ## Running Code Without Keeping It Sometimes you have logic to run once, like a check or a script from a model, and don't want it in the document afterward. `callSource` parses a function, calls it as if it were defined at the top level of the document (or on the object you pass), and removes it after: ```typescript await doc.callSource(` fn quote(qty: int) -> float { self.pricing.total(qty) * self.discount() } fn discount() -> float { 0.9 } `, [10]); // the first function is called; the rest are helpers it can reach through self ``` Changes the function makes to the document stay; the function and its helpers don't. To take something out of the document yourself, `remove` takes a path, like `drop` in Stof: ```typescript doc.remove('pricing.old_total'); // a function or field doc.remove('customers.c_42'); // an object doc.remove('order.customer', true); // shallow: just the field, not the object it points to ``` ## Running by Attribute When you don't know what a document's functions are called, tag them. `run` calls every function with an attribute: ```typescript const doc = await StofDoc.new(` ticket: { text: 'Where is my refund?' } #[triage] fn route() { if (self.ticket.text.lower().contains('refund')) self.ticket.team = 'billing'; } `); await doc.run('triage'); // every #[triage] function doc.get('ticket.team'); // 'billing' ``` This is the usual setup when logic comes from somewhere else, like a model or a customer. The attribute is the agreement: your app runs `#[triage]` functions, and the document can have as many as it needs, with any names. `run()` with no attribute runs `#[main]` functions. ## Reading and Writing Values `get` and `set` read and write any value by path: ```typescript doc.get('pricing.rate'); // 2 doc.set('pricing.rate', 3); doc.set('customer.vip', true); // creates the field if it isn't there ``` Numbers, strings, booleans, lists, and maps come back as JavaScript values. An object comes back as its ID, a string you can pass to the functions that take a `node` (like `blobify` below). ## Giving Documents Your Functions A document can only call what it's given. `lib` adds one of your functions as a library function the document can call: ```typescript doc.lib('App', 'notify', (message: string) => notifications.send(message)); // async functions: pass true, and the document can await them doc.lib('App', 'customer_name', async (id: number) => (await db.customers.find(id)).name, true); ``` ```stof fn on_refund(id: int) { const name = await App::customer_name(id); App::notify(`Refund started for ${name}`); } ``` This is how you connect a document to the rest of your app on your terms: it gets these functions and nothing else. See [Adding Your Own](/libs/overview#adding-your-own) in Libraries. ## What a Document Can Reach A document can read and change itself and call library functions. It can't reach the file system, environment variables, other files, or the network unless you allow it: ```typescript doc.allowHttp(['api.example.com']); // network access to these hosts only doc.allowHttp(); // network access to any host ``` In Python and Rust, `allow_system()` also gives access to files and environment variables. The JavaScript package has no file system access at all. [Libraries](/libs/overview#what-a-document-can-reach) has the full table. Each call also has a time limit, 2 minutes by default, so a document that loops forever can't hang your app: ```typescript doc.setMaxExecutionTime(500); // milliseconds, or null for no limit ``` ```text execution timed out (max execution time exceeded) at root.spin (defined at 1:1) ``` ## Errors A function that throws makes `call` or `run` reject, with a plain message and the Stof call stack: ```typescript try { await doc.call('pricing.total'); } catch (error) { console.log(String(error)); // missing argument 'units' for total(units: int) } ``` A document that doesn't parse throws from `new` or `parse`, with the line and column: ```text parse error: unexpected end of input (expected '}') --> line 1, column 5 | 1 | a: { | ^ ``` For line numbers on runtime errors too, parse with the test profile: `StofDoc.new(src, 'stof', 'test')`. The default profile leaves them out to keep documents smaller. These messages are written to be read by people and by models. When a model wrote the document, pass the error back to it and ask for a fix. ## Exporting A document exports to any format it knows: ```typescript doc.stringify('json'); // a JSON string (functions are left out) doc.stringify('yaml'); doc.record(); // a plain JavaScript object ``` To keep the functions, export binary Stof (`bstf`). It holds the whole document, or one object, and loads anywhere Stof runs: ```typescript const bytes = doc.blobify('bstf', doc.get('pricing')); // just the pricing object, functions included const copy = await StofDoc.new(bytes, 'bstf'); await copy.call('total', 5); // 10 ``` Store those bytes, send them over HTTP or a WebSocket, and run them in another service, a browser, or Python. An `html` export is safe to put on a page (see [Formats](/learn/formats#markup-html-and-xml)), and your app can limit it to exactly the elements it renders, like its own components. A document can't change that list: ```typescript doc.allowHtmlTags(['div', 'section', 'ui-card', 'ui-metric']); el.innerHTML = doc.stringify('html', 'dashboard'); ``` A `stof` export is the document as readable Stof, with its functions as source (see [Formats](/learn/formats#stof-stof-and-bstf)). ### Editing Stof text in place To change a document people also edit by hand (Ex. a settings page saving into a hand-written file), edit its text instead of exporting it again: only what you change is rewritten, and comments, formatting, and everything else stay as they were. Nothing in the text runs while it's outlined or edited. ```typescript const defs = StofDoc.outline(text); // every field and function: path, kind, value, source const next = StofDoc.edit(text, [ { set: 'pricing.rate', value: 3 }, // a JSON value { set: 'pricing.limit', stof: '2GB' }, // Stof source { set_fn: 'pricing.total', stof: 'fn total(units: int) -> float { units * self.rate * 0.9 }' }, { rename: 'pricing.old', to: 'legacy' }, { remove: 'pricing.legacy' }, ]); ``` A set adds what isn't there, with the objects on its path, in the style of the object it goes into. If the text or the result doesn't parse, or an edit doesn't make sense, `edit` throws and nothing changes. Exports call a format's computed values (`#[attr]` and `#[text]` functions) first. A computed value that calls one of your async host functions can't finish in `stringify`: export from Stof instead, with `await doc.callSource('fn view() -> str { stringify("html", self) }', [], 'dashboard')`. ## Cleaning Up In JavaScript, a document lives in WebAssembly memory, which the garbage collector doesn't free. Call `doc.dispose()` when you're done with a document you created for a short task. ## Python and Rust The same steps, by language: | Step | JavaScript | Python | Rust | |---|---|---|---| | Create | `await StofDoc.new(src)` | `doc = Doc()` then `doc.parse(src)` | `Graph::default()` then `graph.parse_stof_src(src, None, Default::default())` | | Parse other formats | `doc.parse(src, 'yaml')` | `doc.string_import(src, 'yaml')` | `graph.string_import("yaml", src, None, &Default::default())` | | Call | `await doc.call('pricing.total', 21)` | `doc.call('pricing.total', [21])` | `Runtime::call(&mut graph, "pricing.total", vec![Val::from(21i64)])` | | Call source | `await doc.callSource(src, [21])` | `doc.call_source(src, [21])` | `Runtime::call_source(&mut graph, src, None, vec![Val::from(21i64)])` | | Remove | `doc.remove(path)` | `doc.remove(path)` | `graph.remove(path, false)` | | Run by attribute | `await doc.run('triage')` | `doc.run('triage')` | `graph.run(Some("triage".into()), true)` | | Read and write | `doc.get(path)`, `doc.set(path, value)` | `doc.get(path)`, `doc.set(path, value)` | [the crate docs](https://docs.rs/stof) | | Host functions | `doc.lib('App', 'notify', fn)` | `doc.lib('App', 'notify', fn)` | `graph.insert_libfunc(LibFunc { .. })` | | Network access | `doc.allowHttp(hosts)` | `doc.allow_http_hosts(hosts)` | `graph.allow_http_hosts(hosts)` | | Files and environment | not available | `doc.allow_system()` | `graph.allow_system()` | | Time limit | `doc.setMaxExecutionTime(ms)` | `doc.set_max_execution_time(seconds)` | `graph.max_execution_time = Some(duration)` | | HTML elements | `doc.allowHtmlTags(tags)` | `doc.allow_html_tags(tags)` | `graph.allow_html_tags(tags)` | | Keep function source | `doc.keepSource()` | `doc.keep_source()` | `graph.keep_source(true)` | | Edit text | `StofDoc.outline(text)`, `StofDoc.edit(text, edits)` | `Doc.outline(text)`, `Doc.edit(text, edits)` | `stof::parser::edit::{outline, edit}` | | Export | `doc.stringify('json')`, `doc.blobify('bstf')` | `doc.string_export('json')`, `doc.binary_export()` | `graph.stringify("json", None)`, `graph.blobify("bstf", None)` | The same document gives the same results in all three. --- ## Async Stof runs async work the same way on every platform, including in a browser, on a single thread. Each piece of async work is a **process**, and the runtime switches between processes so they all make progress. Every call your app makes, and every `#[main]` and `#[test]` function, runs as a process too. ## Async Functions and `await` `async fn` (or `#[async]` on a function, which is the same thing) makes a function async: calling it starts a new process and gives back a promise right away. `await` waits for the result: float { sleep(10ms); // stand-in for slow work 0.2 } #[main] fn main() { const rate = self.load_rate(); // a promise, returned immediately pln(await rate); } `} /> `await` works in any function, not just async ones. On a value that isn't a promise, it gives the value back as it is. ## Doing Things in Parallel Start several processes before awaiting any of them, and they run at the same time. `await` on a list waits for all of them: float { sleep(20ms); sku == 'pro' ? 20 : 5 } #[main] fn main() { // all three wait at the same time: about 20ms in total, not 60ms const prices = await [self.fetch_price('pro'), self.fetch_price('team'), self.fetch_price('free')]; pln(prices); } `} /> This is what makes `Http.fetch` fast in Stof: each request is its own process, so a document can start ten requests and await them together. ## Async Calls and Blocks `async` in front of any call runs it as a new process, even if the function isn't async itself. An `async { ... }` block runs its statements as a new process, and gives a promise for its last value: int { a + b } #[main] fn main() { const sum = async self.add(40, 2); const product = async { const v = 7; v * 6 }; pln(await sum); pln(await product); } `} /> One catch: a block with a single expression, like `async { 1 + 1 }`, is read as a set, not a block. Write it as a statement: `async { return 1 + 1; }`. An `async { ... }` block that isn't assigned to anything runs in the background, and the function carries on without waiting for it. ## Collecting Work to Await Later A common shape is starting work in a loop and awaiting it all at the end: str { sleep(5ms); \`notified \${user}\` } #[main] fn main() { const pending = []; for (const user in ['ada', 'grace', 'linus']) { pending.push_back(self.notify(user)); } for (const result in await pending) pln(result); } `} /> ## How Processes Take Turns Processes switch at loop iterations, function calls, `await`, and `sleep`. Code between those points runs without interruption, so a short calculation is never split in the middle. Two long loops in separate processes take turns, each making steady progress: str { let n = 0; for (const _ in 3) { n += 1; pln(\`\${label} \${n}\`); } label } #[main] fn main() { await [self.count('a'), self.count('b')]; } `} /> That's also why tests that share data can interfere with each other: they run as processes on the same document. See [Testing](./testing). ## Async Functions from Your App Your app can give a document async functions, like a database lookup or an API call, and the document awaits them like its own: ```typescript doc.lib('App', 'customer_plan', async (id: string) => (await db.customers.find(id)).plan, true); ``` ```stof fn can_export(customer_id: str) -> bool { const plan = await App::customer_plan(customer_id); plan == 'pro' } ``` ## Promise Types `Promise` is the type of a promise for a `T`. You rarely need to write it, since a promise is accepted wherever its result type is, but it's there for parameters and return types when you want to be explicit: int { 42 } fn report(value: Promise) -> str { \`total is \${await value}\` } #[main] fn main() { pln(self.report(self.total())); } `} /> Next, [Testing](./testing). --- ## Attributes An attribute is a tag on a field, function, or object: `#[name]`, or `#[name(value)]` with a value. You've already used some, like `#[main]` and `#[type]`. Attributes are data, so a document can read them while it runs, and so can your app. ## Built-In Attributes Stof gives these a meaning: | Attribute | On | Meaning | |---|---|---| | `#[main]` | function | Runs when the document is run (`stof run`, `doc.run()`) | | `#[test]`, `#[test(value)]`, `#[errors]` | function | A test, run by `stof test`. See [Testing](./testing) | | `#[run]`, `#[run(order)]` | function, field | A step run by `obj.run()`. See [Object Run](./object-run) | | `#[init]` | function | Runs once, right after the document is parsed | | `#[async]` | function | The function is async (same as `async fn`). See [Async](./async) | | `#[type]`, `#[extends('Name')]` | object | A type, and the type it builds on. See [Prototypes & Schemas](./prototypes-and-schemas) | | `#[constructor]`, `#[dropped]`, `#[static]` | function in a type | Runs on `new`, runs when an instance is dropped, called on the type | | `#[schema(...)]`, `#[schema_optional]` | field in a type | A validation rule | | `#[readonly]`, `#[private]` | field | Can't be assigned; visible only to its own object | | `#[no-export]` | field | Left out of exports like JSON and HTML | | `#[tag('name')]`, `#[attr]`, `#[text]` | field, function, object | How an object is written as HTML or XML: the element's name, an attribute, text inside it. See [Formats](./formats#markup-html-and-xml) | Any other name is yours to use. ## Your Own Attributes An attribute's value can be any value: a string, a number, a map, a function. That makes attributes a place to keep metadata next to what it describes: `self.attributes('name')` gives a field's or function's attributes as a map. Inside a function, `this.attributes()` gives its own. ## Finding Functions by Attribute `self.funcs('name')` lists the functions on an object that have an attribute, and `funcs(attributes = 'name')` lists them across the whole document, in the order they're written. That turns an attribute into an event or a plugin point: anything tagged joins in, with nothing else to register: 100; } #[main] fn main() { for (const handler in funcs(attributes = 'on_order')) { handler(self.order); } pln(stringify('json', self.order)); } `} /> Add another `#[on_order]` function anywhere in the document, from any file or at runtime, and it runs too. ## The Agreement with Your App The same idea is how an app runs logic it didn't write. The app runs functions by attribute, `doc.run('on_order')`, and the document can have any number of them, with any names: ```typescript // whatever the model wrote, like: // #[on_order] // fn add_tax() { self.order.tax = self.order.total * 0.2; } const doc = await StofDoc.new(rulesFromAModel); doc.parse({ order: { total: 120 } }); await doc.run('on_order'); // every #[on_order] function doc.get('order.tax'); // 24 ``` The attribute is the only thing the app and the document have to agree on. See [Using Stof in Your App](/start/using-stof#running-by-attribute). Next, [Object Run](./object-run). --- ## Collections Stof has four kinds of collections: lists, maps, sets, and tuples. Each has its own library of functions, called as methods: `list.push_back(4)`. This page covers the common ones. The full lists are in the [List](../libs/types/list-library), [Map](../libs/types/map-library), [Set](../libs/types/set-library), and [Tuple](../libs/types/tuple-library) libraries. ## Lists A list is an ordered sequence of any values: `sort()` sorts in place. `sort_by` takes a function that compares two items and returns a negative number, zero, or a positive number: b - a); // largest first pln(scores); } `} /> ## Maps A map holds values by key. Keys can be any value, but they're usually strings: ## Sets A set holds each value once: ## Tuples A tuple is a fixed group of values, read by position: ## Braces: Object, Map, or Set? Braces mean different things in different places, and it's worth knowing which: - In the document itself, and inside `new { ... }`, braces make an **object**: `server: { port: 8080 }`. - In an expression inside a function, braces make a **map** (`{'a': 1}`) or a **set** (`{1, 2}`). So inside a function, create objects with `new`: Collections are shared, not copied, when you assign them. See [Copied or Shared](./variables-and-references#copied-or-shared). Next, [Null & Initialization](./null-and-init). --- ## Control Flow Stof's control flow looks like JavaScript or C, with a few additions. ## `if` and `else` Braces are optional when a branch is a single statement: 10) pln('over the limit'); else if (usage > 5) pln('close to the limit'); else pln('fine'); } `} /> `if` is a statement. For a value that depends on a condition, use the ternary operator: `const label = usage > 10 ? 'over' : 'fine';`. ## `switch` A `switch` runs the first case that matches. Cases don't fall through, except that an empty case shares the next one's body. A `switch` can also be an expression that gives a value: ## `while` and `loop` `loop` repeats until a `break`: 0) remaining -= 1; pln(remaining); let tries = 0; loop { tries += 1; if (tries == 3) break; } pln(tries); } `} /> ## `for` `for...in` loops over a list, a map, a set, or a number (from 0 up to it). The C-style loop works too: Inside a `for...in` loop, `index` is the current position, and `first` and `last` say whether this is the first or last item: Loops can take items apart as they go: `for (const { name, plan } in users)` or `for (const (key, value) in map)`. See [Destructuring](./variables-and-references#destructuring). To loop over an object's fields, use `for (const (name, value) in obj.fields())`. ## `break`, `continue`, and Labels `continue` skips to the next pass, and `break` ends the loop. Label a loop with `^name` to break out of it from inside a nested loop: Without the label, `break` would only end the inner loop. ## `try` and `catch` `throw` raises an error, with any value: a string, a number, an object. `catch` handles it, and `catch (e: type)` only catches errors of that type: [Error Handling](./error-handling) goes further. Next, [Collections](./collections). --- ## The Document Graph So far, objects have been nested inside each other like JSON. Underneath, a document is a graph: a set of **objects**, each with **data** attached to it. Fields and functions are both data, and so are richer values like images and PDFs. Knowing this shape explains how paths work, why functions behave like fields, and why moving part of a document is cheap. ## Objects and Their Data An object is a node in the graph. It doesn't contain its fields and functions the way a JSON object contains values: it points to them. Internally, a document is one list of objects and one list of data, connected by IDs, a bit like an entity-component system. Nested objects are children in that graph: str { \`http://\${self.host}:\${self.port}\` } } #[main] fn main() { pln(self.server.url()); pln(self.server.path()); } `} /> `server` is an object with an identity and a path of its own (`root.server`), and inside `url()`, `self` is `server`. ## `self`, `super`, and `root` Paths inside a function start from one of three places: - `self`: the object the function is on. - `super`: that object's parent. - `root`: the top of the tree the object is in. str { \`\${self.lead.name} leads \${self.name} at \${super.name}\` } } #[main] fn main() { pln(self.team.describe()); } `} /> ## Every Path Has a Starting Point A path always starts at `self`, `super`, `root`, a variable, or a named root. A bare name like `team.name` inside a function means "the root named `team`", not "the field named `team`": This is the most common mistake when starting out, so Stof warns about it when the document is parsed: ```text warning: unknown name 'team' in fn root.main (main.stof:5:9): not a parameter, variable, or root, so it is always null (use self.team for a field) ``` If something is unexpectedly null, check for a missing `self.`. ## Named Roots A document can have more than one root. `root Name { ... }` declares one, and its name works as the start of a path from anywhere in the document: str { \`\${self.total} \${Settings.currency}\` } } #[main] fn main() { pln(self.invoice.label()); } `} /> The top level of a file is the **main root**. Imports can create roots too (`import './geo.stof' as Geo;`), which is how a package's files refer to each other. ## Creating Objects Inside a function, `new { ... }` creates an object. It's added to the document as a child of `self`, or of another object with `on`: Objects stay in the document until something removes them, even when no field points at them. For temporary objects, see [Memory Management](./memory-management). ## Inspecting the Graph A document can look at its own shape while it runs, using the [Object Library](../libs/types/object-library): That's how generic code works on any document: a validator that walks every object, an exporter, or a function that lists every tool an agent has. ## Moving Objects and Fields An object and the field that points to it are separate things in the graph, so each can move on its own. Moving an object changes its parent; moving a field changes where it's found: Nothing underneath is copied, however large it is. Next, [Prototypes & Schemas](./prototypes-and-schemas). --- ## Error Handling [Control Flow](./control-flow#try-and-catch) introduced `try` and `catch`. This page covers what you can throw, what Stof's own errors look like, and how errors reach your app. ## Throw Anything `throw` takes any value. A string is the simplest. An object carries structure, which is useful when the code that catches it needs to decide what to do: 500) throw(new { code: 402, reason: 'over the card limit' }); } #[main] fn main() { try { self.charge(900); } catch (error: obj) { pln(\`\${error.code}: \${error.reason}\`); } } `} /> ## Catch by Type `catch (e: type)` catches errors of that type. A union catches more than one, and a plain `catch` (or `catch (e)`) catches everything: ## Stof's Own Errors When Stof itself raises an error, like assigning to a `const` or calling a function that doesn't exist, the value you catch is a short error code: float { units * 2 } #[main] fn main() { try { self.plan = 'free'; } catch (e: str) { pln(e); } // AssignConst try { self.refund(); } catch (e: str) { pln(e); } // FuncDne: no such function try { self.total(); } catch (e: str) { pln(e); } // FuncArgs: wrong arguments } `} /> Match on these codes, not on the readable message: the codes stay the same, while messages may get better over time. The readable message is what your app and the CLI show (below). ## Propagation An error that isn't caught moves up through each function that called the one that threw it, until something catches it. If nothing does, the call fails: ## Rethrowing Catch an error, add to it, and throw again: ## Errors in Your App An error that isn't caught inside the document makes the app's `call` or `run` fail with a readable message and the Stof call stack: ```text cannot multiply null: the left value is null or missing (use ?? to give a default) at root.total (pricing.stof:4:5) at root.main (pricing.stof:9:5) ``` The CLI shows the failing line too. Messages are written to be understood by people and by models, so when a model wrote the document, give the error back to it and ask for a fix. See [Using Stof in Your App](/start/using-stof#errors). A function that throws doesn't undo what it already did, so changes made before the error stay in the document. Next, [The Document Graph](./document-graph). --- ## Documents and Fields A Stof document is made of objects, and objects hold **fields**: named values. If you know JSON, you already know most of it. ## JSON Is Stof Any JSON document is a valid Stof document: `#[main]` and the function after it aren't JSON. They're there so the **Run** button has something to run, and `self` is the document's top level. Every example in Learn ends this way. Stof also lets you leave out what JSON requires. The quotes around names, the commas, and the outer braces are all optional, so this is the same document: `address` is an **object**: a set of fields with a name. Objects can hold other objects, as deep as you need. ## Typed Fields A field can have a type. Its value is cast to that type, when it's declared and every time it's assigned: Fields without a type take whatever you give them. Add types where they help: they document what a field holds and catch bad values early. [Types & Units](./types-and-units) lists them all, including units like `5km` and `30s`. ## Fields That Can't Change `const` makes a field read-only. Assigning to it throws: ## Private Fields `#[private]` hides a field from code outside its object. Only functions on the same object can read it: str { '****' + self.api_key.substring(self.api_key.len() - 4) } } #[main] fn main() { pln(self.account.masked()); // the object's own function can read it pln(self.account.api_key); // from outside: null } `} /> `#[readonly]` is similar: anyone can read the field, but assigning to it throws. To keep a field out of exports like JSON, mark it `#[no-export]`. The `#[...]` before a field or function is an **attribute**, covered in [Attributes](./attributes). ## Null and Not Null Any field can be `null`, including a typed one. Add `!` to the type to make a field reject null: Reading a field that doesn't exist isn't an error. It's null: `self.missing` is `null`. [Null & Initialization](./null-and-init) covers working with null. ## More Than One Type A field can accept a union of types, or hold a tuple: A value that matches one of the union's types is kept as it is. Anything else is cast to the first type in the union that it can be. ## Creating Fields Assigning to a field that doesn't exist creates it, along with any objects on the way: This is how a document grows while it runs. Next, [Types & Units](./types-and-units). --- ## Formats A format is how Stof reads and writes one kind of data. Formats are named by a string, and the same names work everywhere: `parse(src, obj, 'yaml')`, `stringify('yaml', obj)`, `import yaml './file'`, and from your app, `doc.parse(src, 'yaml')`. `formats()` lists the ones a document has, and `format('name')` checks for one. ## Data: `json`, `yaml`, `toml` Fields in, fields out. Functions, types, and attributes aren't part of these formats, so they're left out of exports: `yml` is another name for `yaml`. ## Stof: `stof` and `bstf` These are the formats that keep everything: functions, types, attributes, and data. `stof` is the text you write. `bstf` is binary Stof: compact, fast to load, and the way to store or send part of a document with its logic: float { units * self.rate } } #[main] fn main() { const bytes = blobify('bstf', self.pricing); const copy = new {}; parse(bytes, copy, 'bstf'); pln(copy.total(21)); } `} /> A `stof` export is readable Stof: objects with their types and attributes, and values as literals (`1GB`, `set(..)`, `map((k, v))`). Parse it back with the `stof` format. Functions are written as their source when the host keeps it (`doc.keepSource()` before parsing; off by default, since it costs memory), and as `data@v1` blocks otherwise, so they still load and run. This playground keeps source. Comments, formatting, and object IDs aren't kept; `stof:graph` is the exact form, with IDs and compiled functions, for copying a document exactly: float { units * self.rate } } #[main] fn main() { pln(stringify('stof', self.pricing)); } `} /> ## Text: `text`, `md`, `bytes` These take content as a single value instead of reading structure from it: `text` (also `txt`) as a string in a field named `text`, `md` (also `markdown`) as a string in a field named `md`, and `bytes` as a `blob` in a field named `bytes`: ## Web: `urlencoded` Form data and query strings (also `www-form`). Nested objects use brackets, the way browsers send forms: ## Markup: `html` and `xml` These write objects as elements, which is how a document becomes a page, a component tree, or an XML file. An object is an element, and its fields, in order, are what's inside it. Three attributes say how: - `#[tag('name')]` names the element. Put it on the field, the object, or a type, and every instance of the type gets it. Without one, an element is named after its field. - `#[attr]` makes a field an attribute of the element (`#[attr('data-id')]` gives it a different name). - `#[text]` writes a value as text inside the element. Child objects become child elements, lists become one element per item, and any other value becomes `value`. `#[no-export]` fields, null values, and functions without `#[attr]` or `#[text]` are left out. str { self.label.len() > 10 ? 'wide' : 'narrow' } } #[tag('section')] dashboard: { #[tag('h2')] title: 'Revenue' Metric mrr: { label: 'MRR', value: (): float => 1000 + 250 } #[text] fn footer() -> str { 'Updated today' } #[no-export] cache: {} } #[main] fn main() { pln(stringify('html', self.dashboard)); } `} /> ### Computed Values A function marked `#[attr]` or `#[text]`, and a field that holds a function, is called when the object is exported, and its result is written. That includes functions from the object's type, and async functions, which are awaited. A `#[text]` function can return objects (or a list of them), which are written as elements, so part of the tree can be built by code: list { const out = []; for (const name in self.names) out.push_back(new Item { name } on self.made); out } #[no-export] made: {} } #[main] fn main() { pln(stringify('html', self.list)); } `} /> ### Safe HTML The `html` format is safe to put on a page, whatever the document holds, including a view a model wrote: - Text and attribute values are always escaped. - `script`, `style`, `iframe`, `object`, `embed`, `link`, `meta`, `template`, and the like are never written, along with what's inside them. - `on*` attributes and `javascript:` URLs are dropped. - Only the standard HTML elements and custom elements (with a `-` in the name) are written. Your app can allow exactly the elements it renders instead, and a document can't change that list (see [Using Stof](/start/using-stof#exporting)). An object without a tag is exported as a fragment: its content, with no element around it. ### XML `xml` works the same way, as data: it starts with an XML declaration, writes empty elements as ``, and fails on names that aren't valid XML. Parsing XML reverses the export, so a document comes back the way it was written. Attributes become `#[attr]` string fields, an element with only text becomes a string field, repeated elements become a list, and other text goes in a `#[text]` field named `text`: TeaCups18.50', order, 'xml'); pln(order.id); pln(order.item[1].text); pln(order.total); order.total = '20.00'; pln(stringify('xml', order)); } `} /> XML has no types, so values parse as strings (`'18.50'`): cast them where you need numbers (`order.total as float`). ## Files: `pdf`, images, `docx` These read richer files into a document: `pdf`, image formats (`png`, `jpeg`/`jpg`, `gif`, `webp`, `bmp`, `tiff`, `ico`), and `docx`. A PDF becomes a `Data` value in a field named `pdf`, and an image a `Data` in a field named `image`, which the [PDF](../libs/pdf-library) and [Image](../libs/image-library) libraries work with. They're available in Python, the CLI, and Rust with the matching features, but not in the JavaScript package. ## Packages: `pkg` Imports a directory with a `pkg.stof` file, or a `.pkg` file made by `stof pkg`. See [Packages](./import-and-export#packages). It reads files, so it needs file access, and it isn't in the JavaScript package. ## Which Formats Are Where | Formats | JavaScript | Python, CLI | Rust | |---|---|---|---| | `json`, `yaml`, `toml`, `stof`, `bstf`, `text`, `md`, `bytes`, `urlencoded`, `html` | Yes | Yes | Yes | | `xml` | Yes | Yes | Export; parsing with the `xml` feature | | `pkg` | No | Yes | Yes (default features) | | `pdf`, images, `docx` | No | Yes | With the `pdf`, `image`, and `docx` features | Rust apps can add formats of their own: anything that implements Stof's `Format` trait can be parsed into and exported from a document like the built-in ones. Next, [Async](./async). --- ## Functions A function lives on an object, next to that object's fields. Inside it, `self` is that object. ## Declaring and Calling float { units * self.rate } #[main] fn main() { pln(self.total(21)); } `} /> `fn` declares a function, with its parameters and their types in parentheses and its return type after `->`. You call a function through the object it's on, with a path: `self.total(21)`, or `self.billing.total(21)` for one on a child object. There are no global functions in a document, so a bare `total(21)` doesn't find it. ## Returning a Value The last expression in a function, with no `;` after it, is what the function returns. `return` returns early: str { if (score >= 90) return 'A'; if (score >= 80) return 'B'; 'C' } #[main] fn main() { pln(self.grade(95)); pln(self.grade(70)); } `} /> Two small rules: `return x;` needs its semicolon, and a semicolon after the last expression makes it a statement instead of the return value. ## Parameters Arguments are cast to the parameter's type, so passing `'5'` to an `int` parameter gives `5`. Parameters can have defaults, a default can use earlier parameters, and a `?` makes a parameter optional (null when it isn't passed): float { total } fn label(name: str, note?: str) -> str { note != null ? \`\${name} (\${note})\` : name } #[main] fn main() { pln(self.price(10)); pln(self.price(10, 3)); pln(self.price(quantity = 2, base = 5)); // named arguments, in any order pln(self.label('Pro')); pln(self.label('Pro', 'most popular')); } `} /> Calling with a missing or extra argument is an error that says exactly what's wrong, like `missing argument 'base' for price(base: float, ...)`. ## What a Function Can See A function sees its own parameters and variables, and the document through `self`, `super`, and `root`. It doesn't see the variables of whatever called it. To use a value from the caller, pass it in: str { \`tax is \${tax}\` } #[main] fn main() { const tax = 0.2; pln(self.describe(tax)); } `} /> That keeps every function self-contained: what it uses is either an argument or part of the document. ## Arrow Functions An arrow function is a function written as an expression. It can be stored in a variable, and unlike a named function, it sees the variables around it: amount * (1 + tax); for (const price in [10, 20, 30]) { pln(with_tax(price)); } } `} /> ## Functions Are Values A function is a value like any other, of type `fn`. Store it in a variable or field, pass it to another function, and call it later: price * 0.8 fn apply(price: float, rule: fn) -> float { rule(price) } fn double(price: float) -> float { price * 2 } #[main] fn main() { pln(self.discount(100)); // a field holding a function pln(self.apply(100, self.discount)); // passed as an argument pln(self.apply(100, self.double)); } `} /> This is why Stof logic can be changed like data: replacing `discount` with a different function changes what every caller gets. A function value also keeps its own `self`, so it works on its own object wherever it's called from. ## Calling Something That Might Not Be There Calling a function that doesn't exist is an error. Put `?` before the `.` to get null instead: `??` gives a default when the left side is null. [Null & Initialization](./null-and-init) has more. ## Library Functions Functions that aren't part of the document, like `pln`, string and list helpers, or functions your app adds, come from libraries: `Time::now()`, `'stof'.upper()`, `App::notify(..)`. See [Libraries](../libs/overview). ## Recursion `this` is the function that's running, so a function can call itself without using its own name: int { if (n <= 1) return n; this(n - 1) + this(n - 2) } #[main] fn main() { pln(self.fib(10)); } `} /> Functions can also have attributes (`#[test]`, `#[main]`, or your own, see [Attributes](./attributes)), run asynchronously ([Async](./async)), and belong to types ([Prototypes & Schemas](./prototypes-and-schemas)). Next, [Variables & References](./variables-and-references). --- ## Import & Export A document takes in data and logic in any format it knows, and exports to any of them. This page covers doing that from inside a document. For your app's side, see [Using Stof in Your App](/start/using-stof). ## Exporting `stringify(format, object)` exports an object as text, and `blobify(format, object)` as bytes: float { self.price } } #[main] fn main() { pln(stringify('json', self.plan)); pln(stringify('yaml', self.plan)); pln(blobify('bstf', self.plan).len()); } `} /> Data formats like JSON and YAML get the fields, without functions. A field marked `#[no-export]` is left out too. Binary Stof (`bstf`) keeps everything, functions included: it's how you store or send a piece of a document and run it somewhere else. ## Parsing at Runtime `parse(source, object, format)` reads data or Stof into the document while it runs: str { "parsed at runtime" }', self); pln(self.greet()); } `} /> `parse` only adds to the document. It can't reach files or the network, so parsing Stof from somewhere you don't trust is as safe as running the rest of the document. The format defaults to `stof`, and the object defaults to `self`. ## The `import` Statement `import` reads a file when the document is parsed: ```stof ``` - **Paths are relative to the file doing the importing**, so a package's files can import each other wherever it's run from. - **The file extension picks the format** (`.json`, `.yaml`, `.toml`, `.md`, `.stof`, ...). Name a format before the path to choose it yourself. An unknown extension imports as text, and an unknown format name is an error. - **Without `as`**, the file goes into the current object. `as self.x` puts it in a field. `as X`, without `self`, creates a root named `X`. Imports read files, so they need file access: they work in the CLI, and in an app only when it allows files with `allow_system()`. To bring in data from your app, use `parse` (or `doc.parse` from the app) instead. ## Where Imports Land Formats that hold structured data (JSON, YAML, TOML, Stof) add their fields. Formats that hold a single value put it in a field named after the format: | Format | Field | Example | |---|---|---| | `text` | `text` | `import text './notes.txt' as self.notes` gives `self.notes.text` | | `md` | `md` | `import './README.md'` gives `self.md` | | `bytes` | `bytes` | raw bytes, as a `blob` | | `pdf` | `pdf` | a `Data` value, see the [PDF library](../libs/pdf-library) | | image formats | `image` | a `Data` value, see the [Image library](../libs/image-library) | [Formats](./formats) lists every format. ## Packages A directory with a `pkg.stof` file is a package. Its `import` field lists the files it's made of: ```stof title="pkg.stof" name: 'pricing' version: '0.1.0' import: 'src/mod.stof' ``` `import pkg './pricing'` imports a whole package. A path starting with `@` refers to `stof/` under the working directory, so `import '@pricing/plans'` is `stof/pricing/plans.stof`. The CLI's `stof pkg` bundles a package into a single file. See [Tools](/start/tools#packages-and-docs). Next, [Formats](./formats). --- ## Memory Management Values like numbers, strings, lists, and maps are freed on their own when nothing uses them. Objects are different: an object is part of the document, and it stays there until something removes it, even if no field points to it anymore. For a document that runs once and is thrown away, that doesn't matter. For one that lives a long time, like a policy that runs on every request, temporary objects add up. ## Temporary Objects with `using` Declare a temporary object with `using` instead of `const`, and it's dropped automatically when its block or function ends, however it ends: normally, with `return`, with `break`, or with an error: bool { using order = new { total: total, tax: total * 0.2 } on self.orders; order.total + order.tax < 1000 } #[main] fn main() { pln(self.check(500)); pln(self.check(900)); pln(self.orders.children().len()); // 0: both were dropped } `} /> For a temporary object you pass straight into a call, put `using` in front of `new`. It's dropped when the enclosing block or function ends: ```stof self.handle(using new { id: 42, kind: 'refund' }); ``` Don't return a `using` object, since it's dropped as the function returns. And when you don't need an object at all, a map is simpler: `{'id': 42}`. ## `drop` `drop(obj)` removes an object, and everything inside it, right away. If its type has a `#[dropped]` function, that runs first: A `using` object runs its `#[dropped]` function too. ## Removing Fields `obj.remove('name')` removes a field. If the field holds an object, the object stays in the document unless you pass `shallow = false`: ## Many Temporaries at Once When a task creates lots of objects, possibly deep inside other functions, create them all on one scratch object and drop that. A plain block with `using` makes a tidy scope for it: obj { new { item: item, price: price } on scratch } #[main] fn main() { let total = 0; { using scratch = new {}; const lines = [ self.line('seat', 20, scratch), self.line('storage', 5, scratch), ]; for (const line in lines) total += line.price; pln(scratch.children().len()); } pln(total); } `} /> Everything created `on scratch` goes when `scratch` does. ## Finding Leaks `stof test --leaks` runs each test on its own and fails any test that leaves objects behind, saying where they are: ```text leaked: root @ build_order ... 1 object never dropped 1 on root (drop it, or declare it with `using`) ``` It's worth running on documents that stay in memory or get exported as binary Stof, since `bstf` includes every object still in the document. Next, [Units Reference](./units-reference). --- ## Null & Initialization `null` means "nothing here". Stof has no separate "undefined": a field set to `null` and a field that doesn't exist both read as `null`. ## Missing Is Null Reading a field that doesn't exist, even deep in a path, gives null instead of an error: ## Defaults with `??` `a ?? b` gives `a`, unless `a` is null, then `b`. It's the usual way to give a default: ## Math with Null Is an Error Adding, multiplying, or doing any other math with null throws an error, instead of quietly giving a wrong number. Usually it means a field is missing or misspelled: The same goes for running totals: start them at a value (`let total = 0;`), not empty. Comparisons are the exception: they don't throw, and `null < 10` is `true`, so check for null first when it matters. ## Null-Safe Calls Calling a function that doesn't exist throws, unlike reading a missing field. Add `?` before the `.` to get null instead. It works anywhere in a path: A `?` in front of a whole expression makes every call in it null-safe at once: `?self.load().parse().validate()` gives null if any of those functions is missing. For library functions, `Lib?::func()` does the same: see [Libraries](../libs/overview). ## Not Null Add `!` to a type to reject null. It works on fields, variables, parameters, and return types: str { \`sending to \${to}\` } #[main] fn main() { pln(self.send('ada@example.com')); try { self.send(null); } catch { pln('a null address is rejected'); } } `} /> ## Choosing a Value The ternary operator picks one of two values: self.limit ? 'over' : 'ok'; pln(status); } `} /> For more than two branches, a block can be an expression too: its last line without a `;` is its value. That turns a multi-step calculation into a single assignment: 1) 'over' else if (ratio > 0.8) 'close' else 'ok' }; pln(status); } `} /> Next, [Error Handling](./error-handling). --- ## Object Run `obj.run()` runs every function and field tagged `#[run]` on an object, in order. It turns an object into a workflow: a list of steps that live with the data they work on, and that you can reorder, extend, or send somewhere else like any other data. ## Steps Every `#[run]` function and field on the object runs when you call `run()`, in the order they're written: ## Ordering Steps A number moves a step. Unnumbered steps count as 0, so `#[run(-1)]` runs before them and `#[run(1)]` after, and steps with the same number keep the order they're written in: ## Steps That Are Objects A child object tagged `#[run]` runs its own steps, so each stage of a workflow can be an object with its own data: ## Steps From a Type A type can define steps for every instance. `run()` on an instance runs the type's steps on that instance, then any new steps the instance adds: Every step has a name: its field or function name. An instance step with the same name as a type step replaces it, in the same place, so each step runs once. A `null` field removes a type's step, and `#[run({'name': 'check'})]` lets a step with a different name replace `check`: Numbers work the same way across the type and the instance, so an instance can put a step anywhere: `#[run(-1)]` runs before the type's unnumbered steps. ## Choosing Steps `run()` takes two options. `prototype` picks which steps run: `'first'` (the default: the type's steps, then the instance's new ones), `'last'` (the instance's new steps first), `'none'` (only the steps on the instance itself), or `'only'` (only the type's steps). `steps` runs just the named steps, in that order: An object can set these for itself with its own attribute, like `#[run({'prototype': 'none'})]` or `#[run({'steps': ['load', 'send']})]`. ## Passing Arguments `#[run({'args': [...]})]` passes arguments to a step: ## A List of Steps `#[run]` on a list runs every item in it: functions, and objects with their own steps: { self.done = true; }, { #[run] fn inner() { super.sub_done = true; } } ] } #[main] fn main() { self.Workflow.run(); pln(self.Workflow.done); pln(self.Workflow.sub_done); } `} /> ## A Realistic Shape The pattern these build toward is a task that knows how to do its own work: fetch something, check what comes back, and record the result: ```stof #[type] Task: { ok: false result: null endpoint: 'https://api.example.com/rates' #[run] fn fetch() { const response = await Http::fetch(self.endpoint); self.result = Http::parse(response, new {}); } #[run] fn check() { self.ok = .schemafy(self.result); } } ``` This one doesn't run on this site, since it needs network access (see [Http](../libs/http-library)). Whatever a task does inside, it runs the same way: `task.run()`. ## Async Steps `run()` runs steps one after another, and an `async fn` step is awaited: it finishes before the next step starts. To run async work side by side instead, start it from a step, collect the promises, and await them in a later step. See [Async](./async) for how that works: Next, [Import & Export](./import-and-export). --- ## Prototypes & Schemas A **prototype** is an object that serves as a type: `#[type]` on an object makes it one. Instances get its fields as defaults and its functions as methods, and paired with `#[schema(...)]`, a type can validate data too. ## Declaring a Type str { \`\${self.id} (\${self.plan})\` } } #[main] fn main() { const customer = new Customer { id: 'cus_1' }; pln(customer.label()); // plan comes from the type's default pln(typename customer); } `} /> `new Customer { ... }` creates an instance, with your values on top of the type's defaults. An object in the document can have a type too: `Customer acme: { id: 'cus_2' }`. ## Constructors A function marked `#[constructor]` runs on every instance created with `new`: Constructors run only with `new`. An object declared with a type in the document gets the type's fields and functions, but not a constructor call. ## The Type Itself `` is a path to the type object. Calling a function on it runs with the type as `self`, which is how you write a static function, like a factory: Money { new Money { amount: amount, currency: 'EUR' } } fn label() -> str { \`\${self.amount} \${self.currency}\` } } #[main] fn main() { pln(.euros(20).label()); } `} /> ## Inheritance `#[extends('Name')]` makes a type build on another. It gets the parent's fields and functions, and can replace them: str { \`\${self.name}: \${self.monthly()}/mo\` } fn monthly() -> float { self.price } } #[type] #[extends('Plan')] AnnualPlan: { fn monthly() -> float { (self.price / 12).round(2) } } #[main] fn main() { const plan = new AnnualPlan { name: 'pro annual', price: 120 }; pln(plan.describe()); // uses AnnualPlan's monthly() pln(plan.monthly()); // call the parent's version pln(plan.instance_of('Plan')); } `} /> The parent's constructor runs before the child's. ## Types Inside Objects Types can live inside objects, to keep a large document organized. Refer to one by a qualified name: float { Num::sqrt(self.x.pow(2) + self.y.pow(2)) } } } #[main] fn main() { const p = new Geometry.Point { x: 3, y: 4 }; pln(p.length()); pln(.y); } `} /> A plain name like `Point` finds the nearest type with that name. A qualified name, `Geometry.Point`, is clearer when two types share a name, and it works from anywhere in the document. ## Typed Fields Inside Types A field whose type is another type casts what's assigned to it, filling in that type's defaults: ## Validating with Schemas A `#[schema(...)]` attribute on a type's field holds a function that checks a value. `schemafy` checks another object's fields against every rule, and returns whether they all pass: target_val != null && target_val.contains('@'))] str email: '' #[schema((target_val: int): bool => target_val != null && target_val >= 13)] int age: 0 } #[main] fn main() { const good = new { email: 'ada@example.com', age: 36 }; const bad = new { email: 'nope', age: 9 }; pln(.schemafy(good)); pln(.schemafy(bad)); } `} /> This is how to check data that comes from outside, like a request or a model's output, before using it. A rule gets what it needs by parameter name: `target_val` is the value being checked. A rule can also ask for `target` (the object being checked), `field` (the field's name), and `schema_val` (the type's default for the field). A field that's missing is checked as null, so rules for required fields should reject null, as these do. `#[schema_optional]` skips the check when a value is null or missing. ## Combining Rules A list of functions runs in order, and every one has to pass: bool { target_val != null && target_val.len() > 0 } #[type] Contact: { #[schema([self.not_empty, (target_val: str): bool => target_val.contains('@')])] str email: '' } #[main] fn main() { pln(.schemafy(new { email: '' })); pln(.schemafy(new { email: 'info@stof.dev' })); } `} /> ## Checking Nested Objects A bare `#[schema]` on a field that holds an object checks that object against the rules inside it: target_val != null && target_val >= 0)] cost: 0 } } #[main] fn main() { const order = new { shipping: new { cost: -5 } }; pln(.schemafy(order)); } `} /> ## Cleaning Up Instead of Rejecting `schemafy` can fix the object instead of only reporting. `remove_invalid = true` deletes fields that fail, and `remove_undefined = true` deletes fields the type doesn't declare: target_val != null && target_val.len() <= 20)] str name: '' } #[main] fn main() { const input = new { name: 'a name that is far too long', admin: true }; .schemafy(input, remove_invalid = true, remove_undefined = true); pln(stringify('json', input)); } `} /> Next, [Attributes](./attributes). --- ## Testing Tests live in the document next to what they test. A function marked `#[test]` is a test, and `stof test` runs them all. A test passes unless it throws. ## Writing Tests The assert functions throw when a check fails: float { units * self.rate } #[test] fn totals() { assert_eq(self.total(3), 6); assert_neq(self.total(1), 0); assert(self.total(0) == 0); assert_null(self.discount); pln('all passed'); } `} /> This example's **Run** button calls the test function directly. On your machine, `stof test` finds and runs every test. The asserts are `assert`, `assert_not`, `assert_eq`, `assert_neq`, `assert_null`, and `assert_not_null`, in the [Standard Library](../libs/standard-library#assertions). ## Expected Values and Errors `#[test(value)]` passes when the function returns that value. `#[errors]` passes when the function throws, for checking that bad input is rejected: ```stof #[test(10)] fn five_units() -> float { self.total(5) } #[test] #[errors] fn rejects_missing_units() { self.total(); } ``` ## Running Tests ```text $ stof test pricing.stof running 3 tests ... test root totals ... ok test root five_units ... ok test root rejects_missing_units ... ok test result: ok. 3 passed; 0 failed; finished in 0s ``` A failing test shows what failed and where: ```text failed: root @ totals ... error: assertion failed: '6' does not equal '7' --> pricing.stof:9:5 | 9 | assert_eq(self.total(3), 7); | ^ at root.totals (pricing.stof:9:5) ``` `stof test` exits with an error code when a test fails, so it fits in CI. Give it a directory to test a whole package. ## Checking for Leaks `stof test --leaks` runs tests one at a time and also fails any test that leaves objects behind in the document: ```text leaked: root @ build_order ... 1 object never dropped 1 on root (drop it, or declare it with `using`) ``` That matters for documents that live a long time, like a policy that runs on every request. See [Memory Management](./memory-management). ## Keep Tests Independent Tests run at the same time, as processes on one shared document. Two tests that change the same field can interfere with each other. Give each test its own data, with `new { ... }` (declared with `using` so it's cleaned up), and treat shared fields as read-only: ## Tests and Production Tests are part of the document, but they cost nothing in production. When an app loads a document normally, test functions are left out. To run them from an app, load the document with the test profile and run the `test` attribute: ```typescript const doc = await StofDoc.new(source, 'stof', 'test'); await doc.run('test'); // throws if a test fails ``` That's a good way to check logic a model wrote: ask it to include tests, and run them before trusting the rest. Next, [Memory Management](./memory-management). --- ## Types & Units Every value in Stof has a type. You don't have to write types down, since a value's type comes from the value itself, but you can, on fields, variables, and function parameters. A declared type casts what's assigned to it. ## The Basics 1.0.0); pln(self.raw.utf8()); } `} /> `null` is a value and a type of its own, for "nothing here". [Null & Initialization](./null-and-init) covers it. ## Numbers `int` and `float` work together the way you'd expect, with two things to know: Number methods like `round` return a new number and leave the original alone, so write `x = x.round(2)` to change `x`. The [Number Library](../libs/types/number-library) has them all. ## Strings Single and double quotes are the same. Backticks make a template string, and `${...}` puts a value inside it: A template string is always a string, even when every part is a number. A null inside one prints as `null`. `\$` writes a literal dollar sign. ## Casting with `as` `as` converts a value to another type. It works between numbers, strings, booleans, and units: A cast that can't work throws an error, which you can [catch](./error-handling). ## Checking Types `typeof` gives a value's basic type. `typename` gives the most specific name it has: the unit, or the type of an object: ## Collections Lists, maps, sets, and tuples hold other values: [Collections](./collections) covers working with them. ## Objects and Functions Are Values An object is a value of type `obj`, and a function is a value of type `fn`. Both can be stored in variables and fields, passed to functions, and returned from them. Objects can also have types of your own, like `User` or `Invoice`, which [Prototypes & Schemas](./prototypes-and-schemas) covers. ## Units Numbers can carry units, and Stof converts between them for you: The same works for time, data sizes, mass, temperature, and angles: A unit type like `m` or `ms` is a kind of `float`. A field typed `ms` stores milliseconds however you assign it, so `ms timeout: 2min` holds `120000`. Units that measure different things don't mix: adding `12km + 34s` drops the units and gives a plain number. The [Units Reference](./units-reference) lists every unit. ## Versions `ver` holds a [semantic version](https://semver.org), and versions compare the way they should: 1.0.0); pln(self.release.minor()); } `} /> ## Unions and `unknown` A union type accepts any of its types. `unknown` accepts anything, and leaves checking it up to you: str { typeof v == 'int' ? 'a number' : 'some text' } fn kind(v: unknown) -> str { typeof v } #[main] fn main() { pln(self.describe(5)); pln(self.describe('five')); pln(self.kind([1, 2])); } `} /> ## The `data` Type Underneath, every field and function is a piece of data attached to an object, and a `data` value is a handle to one. You won't need it often. It's what richer values like a loaded image or PDF are (`Data`, `Data`), and it lets you do things like attach the same field to more than one object: See the [Data Library](../libs/types/data-library) for more. Next, [Functions](./functions). --- ## Units Reference Every unit Stof supports, for looking things up. [Types & Units](./types-and-units#units) covers how units work. Each unit's abbreviation works after a number (`5ft`, `20kg`), and either the abbreviation or the full name works as a type: When two different units of the same kind combine, the result is in the larger of the two (`500m + 1km` is in `km`). Mixed angles always give radians. ## Length | Unit | Full Name | |---|---| | `km` | kilometers | | `hm` | hectometers | | `dcm` | decameters | | `m` | meters | | `dm` | decimeters | | `cm` | centimeters | | `mm` | millimeters | | `um` | micrometers | | `nm` | nanometers | | `mi` | miles | | `yd` | yards | | `ft` | feet | | `in` | inches | ## Mass | Unit | Full Name | |---|---| | `Gt` | gigatonnes | | `Mt` | megatonnes | | `t` | tonnes | | `kg` | kilograms | | `g` | grams | | `mg` | milligrams | | `ug` | micrograms | | `ng` | nanograms | | `pg` | picograms | | `Ton` | tons | | `lb` | lbs (imperial pounds) | | `oz` | ounces | ## Time | Unit | Full Name | |---|---| | `day` | days | | `hr` | hours | | `min` | minutes | | `s` | seconds | | `ms` | milliseconds | | `us` | microseconds | | `ns` | nanoseconds | `Time::now()` returns milliseconds, so adding or subtracting any other time unit works directly. ## Temperature | Unit | Full Name | |---|---| | `K` | kelvin | | `C` | celsius | | `F` | fahrenheit | ## Angles | Unit | Full Name | Range | |---|---|---| | `rad` | radians | clamped to ±360° equivalent | | `deg` | degrees | clamped to ±360° | | `prad` | pradians (positive radians) | clamped to [0°, 360°) | | `pdeg` | pdegrees (positive degrees) | clamped to [0°, 360°) | The positive versions keep angles between 0 and a full turn: casting `-90deg` to `pdeg` gives `270pdeg`. ## Memory (Decimal) | Unit | Full Name | |---|---| | `bit` / `bits` | | | `byte` / `bytes` | | | `KB` | kilobytes | | `MB` | megabytes | | `GB` | gigabytes | | `TB` | terabytes | | `PB` | petabytes | | `EB` | exabytes | | `ZB` | zettabytes | | `YB` | yottabytes | ## Memory (Binary) | Unit | Full Name | |---|---| | `KiB` | kibibytes | | `MiB` | mebibytes | | `GiB` | gibibytes | | `TiB` | tebibytes | | `PiB` | pebibytes | | `EiB` | exbibytes | | `ZiB` | zebibytes | | `YiB` | yobibytes | These are two different sets of units: `MB` is 1,000,000 bytes and `MiB` is 1,048,576, and the gap grows with size. When something says `GB` but means `GiB`, as is common in computing, cast to the unit it actually means. --- ## Variables & References Variables hold values while a function runs. They aren't part of the document: when the function ends, they're gone. ## `let` and `const` `let` declares a variable you can change. `const` declares one you can't reassign. Either can have a type, which casts what's assigned to it: `const` stops reassignment, not change: `const list = []; list.push_back(1);` is fine. ## Scopes A block (`{ ... }`) has its own scope. Code inside it can use the variables outside it, and the variables declared inside it end with the block: Functions are separate: a function doesn't see the variables of the function that called it. See [Functions](./functions#what-a-function-can-see). ## Destructuring Destructuring takes a value apart into variables in one step. From an object (or a map with string keys), name the fields you want: From a list or tuple, variables take the items in order, and `_` skips one: It works in loops too, which is handy for maps and lists of objects: A missing field or item gives null, or its default if it has one. Destructuring something of the wrong kind, like null, throws an error. Use `let` instead of `const` to change the variables afterward. ## Copied or Shared Simple values (numbers, strings, booleans, versions) are copied when you put them in a variable. Changing the variable doesn't change where it came from: Lists, maps, sets, and objects are shared instead. A variable holding a list is the same list, so changing it changes the original. Use `copy(..)` when you want a separate one: ## References with `&` `&` makes a simple value shared too: the variable refers to the original instead of a copy: The same works for arguments, so a function can change a variable it's given: And for list items, including in loops: Next, [Control Flow](./control-flow). --- ## Age Encryption (Age) `Age` encrypts a document, or part of one, so only the holder of a specific key or passphrase can read it back. It uses the [age](https://age-encryption.org) format, and it's available everywhere Stof runs, including JavaScript. ## `Age::generate(context: obj = self) -> Data` Generates a new identity — a public/private keypair — attached to the given object. 0); } `} /> ## `Age::public(age: Data) -> str` The public half of an identity — safe to share, since it can only encrypt, not decrypt. ## `Age::blobify(recipients: str | list | Data, format: str = 'stof', context?: obj) -> blob` Like `Std.blobify`, but encrypted to one or more recipients' public keys — only the matching private key can read the result back. 0); } `} /> ## `Age::parse(age: Data, bin: blob, context: obj = self, format: str = "stof") -> bool` The other half of `Age.blobify` — decrypts using the matching identity, then parses the result into `context` the same way `Std.parse` would: ## `Age::pass_blobify(passphrase: str, format: str = 'stof', context?: obj) -> blob` The simpler alternative to `Age.blobify` — a shared passphrase instead of a generated identity, when there's no real recipient to manage keys for. 0); } `} /> ## `Age::pass_parse(passphrase: str, bin: blob, context: obj = self, format: str = "stof") -> bool` Decrypts what `Age.pass_blobify` produced — the same passphrase in, the original document back out: --- ## File System Library (fs) `fs` reads and writes files. It's off by default, along with the environment functions and importing files from disk, so a document can only read and change itself. The host turns them all on together: ```python # Python doc.allow_system() ``` ```rust // Rust graph.allow_system(); ``` The `stof` CLI turns this on, since it runs your own files. The JavaScript package doesn't include `fs`, so the examples below don't run on this site. If a document should never touch the disk, don't call `allow_system()`, and a document that tries gets an error. Paths are relative to the process's working directory. ## `fs::read(path: str) -> blob` Reads a file into a binary blob. ```stof const bytes = fs::read("src/lib.rs"); ``` ## `fs::read_string(path: str) -> str` Same as `fs.read`, but as a string instead of raw bytes. ```stof const content = fs::read_string("src/lib.rs"); ``` ## `fs::write(path: str, content: str | blob) -> void` Writes to a file, overwriting it if it already exists. Throws if the containing directory doesn't exist. ```stof fs::write("src/text.txt", "testing"); ``` --- ## HTTP Network Library (Http) `Http` lets a document make network requests. Like file and environment access, it's off by default: a document can't reach the network unless the app running it says so. That matters when the document came from a model or a user, so the app decides exactly where it can connect. ## Turning It On The host turns on `Http`, either for any host or, better for documents you didn't write, for a list of hosts. A request to any other host throws, and so does a response that redirects to one. ```typescript // JavaScript / TypeScript (Http.fetch is backed by the platform's fetch) doc.allowHttp(['api.example.com']); // or doc.allowHttp() for any host ``` ```python # Python doc.allow_http_hosts(['api.example.com']) # or doc.allow_http() for any host ``` ```rust // Rust (the "http" feature) graph.allow_http_hosts(["api.example.com"]); // or graph.allow_http() for any host ``` An entry is a host name (any port) or a host and port, like `localhost:8080`. The `stof` CLI turns on `Http` for any host, since it runs your own files. The examples on this page aren't runnable on this site, because the docs don't give documents network access. ## How Requests Run `Http.fetch` is async: it starts the request in its own process, so the rest of the document keeps running, and you `await` the response when you need it. Start several requests before awaiting any of them and they run in parallel. See [Async](../learn/async) for how processes work. ## Making Requests ### `async Http::fetch(url: str, method: str = "get", body: str | blob = null, headers: map = null, timeout: seconds = null, query: map = null, bearer: str = null) -> Promise` Makes a request and returns the response as a map: `status`, `ok`, `headers`, `content_type`, and `bytes` (the body). Every other function on this page reads that map. `timeout` is in seconds, `query` adds URL query parameters, and `bearer` sets an `Authorization: Bearer` header. ```stof const resp = await Http::fetch("https://restcountries.com/v3.1/region/europe"); ``` ## Reading the Response ### `Http::success(response: map) -> bool` Status in `[200, 299]`. ```stof const resp = await Http::fetch("https://restcountries.com/v3.1/region/europe"); assert(Http::success(resp)); ``` ### `Http::client_error(response: map) -> bool` Status in `[400, 499]`. ```stof assert_not(Http::client_error(resp)); ``` ### `Http::server_error(response: map) -> bool` Status in `[500, 599]`. ```stof assert_not(Http::server_error(resp)); ``` ### `Http::text(response: map) -> str` The response body as UTF-8 text, the same as `Http::blob(response) as str`. ```stof const body = Http::text(resp); ``` ### `Http::blob(response: map) -> blob` The response body as raw bytes. ```stof const body = Http::blob(resp); ``` ### `Http::size(response: map) -> bytes` The response body's size, as a memory unit, so it converts to any other memory unit: ```stof const mib_body_size = Http::size(resp) as MiB; ``` ### `Http::parse(response: map, context: obj = self) -> obj` Parses the response body into an object (and returns it), using the response's `Content-Type` header to pick the format, so a JSON response is parsed as JSON. Without that header it's parsed as Stof. Throws if the format isn't one this document has, or if there's no body: ```stof const body = new {}; try { Http::parse(resp, body); } catch { /* didn't work out */ } ``` --- ## Image Library (Image) `Image` is in the Python package, the CLI, and Rust with the `image` feature. It isn't in the JavaScript package, so the examples below don't run on this site. An image loads into a document as a `Data` value, from bytes or by importing an image file, and every function on this page works on that value. ## Loading ### `Image::from_blob(bytes: blob) -> Data` Creates an image on the calling object from raw bytes, detecting the format. The bytes can come from anywhere: `fs.read`, an `Http` response, or the host. ```stof const bytes = fs::read("photo.png"); const img = Image::from_blob(bytes); ``` ## Reading Dimensions ### `Image::width(img: Data) -> int` ```stof const width = Image::width(img); ``` ### `Image::height(img: Data) -> int` ```stof const height = Image::height(img); ``` ## Transformations Every function in this section mutates the image in place and returns nothing — call it, then export or continue transforming the same `img` value. ### `Image::grayscale(img: Data) -> void` ```stof Image::grayscale(img); ``` ### `Image::invert(img: Data) -> void` ```stof Image::invert(img); ``` ### `Image::blur(img: Data, blur: float) -> void` Gaussian blur — `blur` is the sigma value. ```stof Image::blur(img, 2.5); ``` ### `Image::fast_blur(img: Data, blur: float) -> void` Same gaussian blur as `Image.blur`, trading some quality for speed — reach for this one over `blur` if you're processing images in bulk and the difference isn't visually significant for your use case. ```stof Image::fast_blur(img, 2.5); ``` ### `Image::brighten(img: Data, brighten: int) -> void` Positive brightens, negative darkens. ```stof Image::brighten(img, 20); ``` ### `Image::contrast(img: Data, contrast: float) -> void` Positive increases contrast, negative decreases it. ```stof Image::contrast(img, 15.0); ``` ### `Image::flip_horizontal(img: Data) -> void` ```stof Image::flip_horizontal(img); ``` ### `Image::flip_vertical(img: Data) -> void` ```stof Image::flip_vertical(img); ``` ### `Image::rotate_90(img: Data) -> void` Clockwise. ```stof Image::rotate_90(img); ``` ### `Image::rotate_180(img: Data) -> void` ```stof Image::rotate_180(img); ``` ### `Image::rotate_270(img: Data) -> void` Clockwise — equivalent to a 90° counter-clockwise rotation. ```stof Image::rotate_270(img); ``` ## Resizing Each of these returns a `bool` — whether the resize actually succeeded. ### `Image::resize(img: Data, width: int, height: int) -> bool` Preserves aspect ratio — the result fits within `width` × `height` but may not match it exactly. ```stof const resized = Image::resize(img, 800, 600); ``` ### `Image::resize_exact(img: Data, width: int, height: int) -> bool` Forces the exact dimensions given, ignoring the original aspect ratio. ```stof const resized = Image::resize_exact(img, 800, 600); ``` ### `Image::thumbnail(img: Data, width: int, height: int) -> bool` Same as `resize`, but optimized for shrinking rather than resizing in either direction. ```stof const made = Image::thumbnail(img, 128, 128); ``` ### `Image::thumbnail_exact(img: Data, width: int, height: int) -> bool` `thumbnail`'s exact-dimensions counterpart, the same way `resize_exact` relates to `resize`. ```stof const made = Image::thumbnail_exact(img, 128, 128); ``` ## Exporting Each function below turns the current state of `img` — after whatever transformations have run — into raw bytes of a specific format. ### `Image::blob(img: Data) -> blob` The default export — raw bytes in PNG format, equivalent to `Image.png`. ```stof const bytes = Image::blob(img); ``` ### `Image::png(img: Data) -> blob` ```stof const bytes = Image::png(img); ``` ### `Image::jpeg(img: Data) -> blob` ```stof const bytes = Image::jpeg(img); ``` ### `Image::webp(img: Data) -> blob` ```stof const bytes = Image::webp(img); ``` ### `Image::gif(img: Data) -> blob` ```stof const bytes = Image::gif(img); ``` ### `Image::bmp(img: Data) -> blob` ```stof const bytes = Image::bmp(img); ``` ### `Image::ico(img: Data) -> blob` ```stof const bytes = Image::ico(img); ``` ### `Image::tiff(img: Data) -> blob` ```stof const bytes = Image::tiff(img); ``` --- ## Markdown Library (Md) Two functions, both taking a Markdown string and converting it to something else. ## `Md::html(md: str) -> str` Markdown in, an HTML string out. ## `Md::json(md: str) -> str` Markdown in, a JSON string out — a full structural breakdown from the parser, not just the rendered result. More detail than `Md.html`, useful when you need to work with the document's structure rather than just display it. --- ## Libraries Libraries are the functions a document can call that aren't part of the document itself: printing, string and list helpers, time, encryption, and so on. They're grouped by name, and you call them with that name: `Time::now()`, `Num::abs(-2)`. ## Calling Library Functions There are three ways to call a library function, and they're all the same call: These docs use `Lib::func()` for calling a library directly, and the method form on values. `::` always means the library, so it can't be confused with a field or variable of the same name, and it reads as a library call at a glance. The method form works for every type library: a string's methods come from `Str`, a list's from `List`, an object's from `Obj`, and so on. `Std` functions don't need a name at all, so `pln(..)` is `Std::pln(..)`. `Lib?::func()` returns null instead of throwing when the library or function doesn't exist, which is handy for optional host libraries: ## What's Built In | Library | What it's for | Available in | |---|---|---| | [Std](./standard-library) | Printing, assertions, parsing and exporting, collections, introspection | Everywhere | | [Str](./types/string-library), [Num](./types/number-library), [List](./types/list-library), [Map](./types/map-library), [Set](./types/set-library), [Tup](./types/tuple-library) | Methods on strings, numbers, and collections | Everywhere | | [Obj](./types/object-library), [Fn](./types/function-library), [Data](./types/data-library) | Working with objects, functions, and data in the document graph | Everywhere | | [Blob](./types/blob-library), [Ver](./types/ver-library), [Prompt](./types/prompt-library) | Bytes, semantic versions, and prompt trees for AI contexts | Everywhere | | [Time](./time-library) | Timestamps, durations, and dates | Everywhere | | [Md](./markdown-library) | Markdown to HTML or JSON | Everywhere | | [Age](./age-library) | Encrypting parts of a document | Everywhere | | [Http](./http-library) | Network requests | When the host turns it on: JavaScript, Python, Rust (`http` feature), CLI | | [fs](./file-system-library) | Reading and writing files | When the host turns it on: Python, Rust, CLI | | [Image](./image-library) | Loading, editing, and exporting images | Python, Rust (`image` feature), CLI | | [Pdf](./pdf-library) | Text and images from PDFs | Python, Rust (`pdf` feature), CLI | "Everywhere" means the JavaScript package (Node.js and browsers), Python, Rust, and the `stof` CLI. The runnable examples on this site use the JavaScript package. ## What a Document Can Reach A document can always read and change itself and call the libraries above that are marked "Everywhere". Nothing on the outside, though: the file system, environment variables, importing files from disk, and the network are all off until the host turns them on: | Access | JavaScript | Python | Rust | |---|---|---|---| | Network (`Http`) | `doc.allowHttp(['api.example.com'])` | `doc.allow_http_hosts([...])` | `graph.allow_http_hosts([...])` | | Network, any host | `doc.allowHttp()` | `doc.allow_http()` | `graph.allow_http()` | | Files, environment, file imports | not available | `doc.allow_system()` | `graph.allow_system()` | The `stof` CLI turns everything on, since it runs your own files. Each call also has a time limit, 2 minutes by default, which the host can change: `doc.setMaxExecutionTime(ms)` (JavaScript), `doc.set_max_execution_time(seconds)` (Python), `graph.max_execution_time` (Rust). ## Adding Your Own The host can add its own library functions, and that's the usual way to give a document access to something specific: your database, your notification service, your app's state. A document can call those and nothing else it wasn't given. ```typescript const doc = await StofDoc.new(` #[main] fn main() { App::notify('order shipped'); } `); doc.lib('App', 'notify', (message: string) => console.log('notify:', message)); await doc.run(); ``` ```python from pystof import Doc doc = Doc() doc.lib('App', 'notify', lambda message: print('notify:', message)) ``` In Rust, a library function is a `LibFunc` inserted with `graph.insert_libfunc(..)`. --- ## PDF Library (Pdf) `Pdf` is in the Python package, the CLI, and Rust with the `pdf` feature. It isn't in the JavaScript package, so the examples below don't run on this site. A PDF parsed or imported into a document (with the `pdf` format, see [Import & Export](../learn/import-and-export)) becomes a `Data` value, which both functions on this page work on. Importing a file from disk needs the host to allow it with `allow_system()`; parsing PDF bytes the host hands over doesn't. ## `Pdf::extract_text(pdf: Data) -> str` Every page's text, concatenated into one string. ```stof const text = self.pdf.extract_text(); pln(text); ``` ## `Pdf::extract_images(pdf: Data) -> list` Every image on every page, as a list of maps carrying the image data alongside its dimensions. ```stof const images = self.pdf.extract_images(); pln(images.len()); pln(images[0].get('width'), ' ', images[0].get('height')); ``` --- ## Standard Library (Std) `Std` is the one library that never needs to be named — `Std::pln(...)` and `pln(...)` call the same function. Every other library needs its own prefix (`Time::now()`, `Http::fetch(...)`), but anything shown here works bare. One distinction worth keeping straight: this page is about *library* functions, which never need a receiver. A **document** function always does — `self.myFunc()`, `super.myFunc()`, through a variable, whatever fits — because it belongs to a specific node in the graph, not to the runtime itself. ## Output ### `pln(..) -> void` Prints all arguments to the standard output stream — this is what every other page's examples have been using the whole time. ### `err(..) -> void` Same as `pln`, but to the error stream instead of standard output. ### `str(..) -> str` Formats all arguments the same way `pln` would, but returns the result as a string instead of printing it. ### `dbg(..) -> void` Prints all arguments as debug output. Depending on how the host environment wires up logging, this may not surface in every output stream `pln` does — worth checking your browser console too if it looks quiet. ## Logging The five `log_*` functions map onto the standard levels from Rust's `log` crate. Same caveat as `dbg` — whether these are visible depends on what the host has configured to receive them. ### `log_info(..) -> void` ### `log_debug(..) -> void` ### `log_warn(..) -> void` ### `log_error(..) -> void` ### `log_trace(..) -> void` ## Debugging & Tracing Three deeper introspection tools — expect dense, VM-level output that looks different every run. That's normal; these aren't meant to be tidy. ### `peek(..) -> void` Prints your arguments, then a trace of the current process and the *next* instructions about to execute. Pass a trailing integer to control how many. ### `trace(..) -> void` Like `peek`, but shows the instructions that already *ran* leading up to this point, instead of what's coming next. ### `dbg_tracestack() -> void` Prints a snapshot of the current stack, with no arguments needed. ## Assertions Every one of these throws when the check fails — wrap the failing case in `try`/`catch` if you want execution to continue past it, same as anywhere else in Stof. ### `assert(value: unknown = false) -> void` ### `assert_eq(first: unknown, second: unknown) -> void` ### `assert_neq(first: unknown, second: unknown) -> void` ### `assert_not(value: unknown = true) -> void` ### `assert_null(value: unknown) -> void` Throws if the value isn't null. A function that returns nothing counts as null. ### `assert_not_null(value: unknown) -> void` Throws if the value is null. ## Errors & Process Control ### `throw(value: unknown = "Error") -> void` Throws any value, not just a string — the mechanism behind every `try`/`catch` example on the Error Handling page. ### `exit(..) -> void` Immediately halts the current process — nothing after it in the same process runs. Pass a promise to terminate *that* process's execution instead of the current one. ## Parsing & Exporting ### `parse(source: str | blob, context: str | obj = self, format: str = "stof", profile: str = "prod") -> bool` Covered in depth on Import & Export — the runtime counterpart to the `import` statement, and it works fine sandboxed since it's just operating on a string. str { "hello" }'); pln(self.hello()); } `} /> ### `stringify(format: str = "json", context: obj = null) -> str` Exports a string. Lossy for anything the target format can't represent — JSON has no concept of units, for instance, so they're silently dropped. Some formats call functions first: `#[attr]` and `#[text]` functions in `html` and `xml` (see [Formats](/learn/formats#computed-values)). They're called before the export is written, and async ones are awaited: str { 'Saved' } } #[main] fn main() { pln(stringify('html', self.note)); } `} /> ### `blobify(format: str = "json", context: obj = null) -> blob` Same idea as `stringify`, exporting raw bytes instead of a string. 0, ' ', bytes as str); } `} /> ## Constructing Collections Mostly useful for building an empty collection, or building one from values you already have in hand rather than writing out literal syntax. ### `list(..) -> list` ### `map(..) -> map` Takes tuples of key/value pairs. ### `set(..) -> set` ## Values & Memory ### `copy(val: unknown) -> unknown` A real deep copy — for an object, every field, function, and nested piece of data underneath it, recursively. ### `swap(first: unknown, second: unknown) -> void` Swaps two values in place — needs `&` on value types, the same reference operator from Variables & References, since a plain value type would just swap two disposable copies. ### `min(..) -> unknown` If an argument is a collection, only the values inside it are considered — not the collection as a whole. ### `max(..) -> unknown` ### `drop(..) -> bool | list` Covered in depth on Document Memory Management — drops fields, functions, objects, or data from the graph, running `#[dropped]` on anything that has it. Given several things to drop at once, returns a list of one bool per item. {}; const object = new {}; self.field = 42; const results = drop("self.field", func, object); pln(results); } `} /> ### `shallow_drop(..) -> bool | list` Same as `drop`, except if a dropped field pointed to an object, only the field is removed — the object itself stays in the graph. Useful when more than one field might point at the same object and you only want to disconnect this one. ## Infinity and NaN ### `inf() -> float` Positive infinity. Negate it for negative infinity. To check whether a number is infinite, use `val.inf()` from the [Number Library](./types/number-library). 1e308, ' ', -inf() < -1e308, ' ', limit.inf()); } `} /> ### `nan() -> float` Not a number (NaN). NaN never equals anything, including itself, so check for it with `val.nan()`. ## Introspection Mostly useful for writing code that has to reason about the document generically — a validator, a debugging tool, anything that can't assume a fixed shape ahead of time. ### `funcs(attributes: str | list | set = null) -> list` Every function in the graph, optionally filtered to only those carrying a given attribute. ### `callstack() -> list` A list of function pointers currently on the stack — last one is `this`, the function calling `callstack()` itself. ### `graph_id() -> str` A unique ID string for this graph/document. 10); } `} /> ### `format(format: str) -> bool` Is a given format loaded and available for `parse`/`stringify`/`blobify`? ### `format_content_type(format: str) -> str` The HTTP content-type header value for a loaded format — useful if you're serving Stof-produced data over the wire. ### `formats() -> set` Every format currently available to use. ### `lib(lib: str, func?: str) -> bool` Is a library — and optionally a specific function on it — loaded? ### `libs() -> set` Every library currently available. ### `prof(name: str) -> bool` Was this graph parsed under a given profile name? Profiles are how `parse`'s `profile` parameter tags a parse for later inspection. ## Environment These read and change the process's environment variables, so they're off by default: a document only has them when the host calls `allow_system()` (Rust, Python). The `stof` CLI turns them on, since it runs your own files. They aren't in the JavaScript package, so the examples below don't run on this site. ### `env(var: str) -> str` An environment variable's value, or null if it isn't set. ```stof const host = env("HOST"); ``` ### `env_vars() -> map` Every environment variable, as a map of names to values. ```stof const vars: map = env_vars(); ``` ### `set_env(var: str, value: str) -> void` Sets an environment variable for this process. ```stof set_env("HOST", "localhost"); ``` ### `remove_env(var: str) -> void` Removes an environment variable from this process. ```stof remove_env("HOST"); ``` ## Timing ### `sleep(time: ms) -> void` Pauses the current process for at least the given duration — possibly longer, never less. Other processes keep making progress while this one sleeps, same cooperative model as Async. = 50); } `} /> ## IDs & Text Helpers ### `nanoid(length: int = 21) -> str` A URL-safe random ID — collision probability is low and drops further as length increases. ### `xml(text: str, tag: str) -> str` Wraps a string in an XML tag — a small formatting helper, nothing more. ### `prompt(text: str = '', tag?: str) -> prompt` Builds a `prompt` value — a small tree of tagged text, meant for assembling structured prompts for AI contexts. `+=` appends a nested prompt as a child. --- ## Time Library (Time) Every timestamp in this library is a plain `ms` value (nanoseconds for the `_ns` variants) — which means normal unit arithmetic just works on them. `Time::diff(start) as seconds` reads naturally instead of needing a manual division. ## Current Time & Elapsed ### `now() -> ms` The current Unix timestamp, in milliseconds. ### `now_ns() -> ns` Same idea, in nanoseconds — for when millisecond resolution isn't fine enough. ### `diff(prev: float) -> ms` Shorthand for `Time::now() - prev` — how much time has passed since a timestamp you captured earlier. = 50); } `} /> ### `diff_ns(prev: float) -> ns` The nanosecond version of `diff`. = 49ms); } `} /> ### `sleep(time: float = 1000ms) -> void` An alias for `Std.sleep` — identical behavior, just callable through `Time` for anyone who'd rather keep every time-related call under one namespace. = 50); } `} /> ## Formatting RFC 3339 is the ISO-8601-flavored format most APIs use today; RFC 2822 is the older, email-header-style format — both convert cleanly to and from a plain `ms` timestamp. ### `now_rfc3339() -> str` ### `now_rfc2822() -> str` ### `to_rfc3339(time: float) -> str` ### `to_rfc2822(time: float) -> str` ### `from_rfc3339(time: str) -> ms` ### `from_rfc2822(time: str) -> ms` ## Reading Components All UTC — none of these read a local timezone. ### `year(ts: ms) -> int` ### `month(ts: ms) -> int` 1–12. ### `day_of_month(ts: ms) -> int` 1–31. ### `day_of_week(ts: ms) -> int` ISO convention — `0` is Monday, `6` is Sunday. ### `hour(ts: ms) -> int` 0–23. ### `minute(ts: ms) -> int` ### `second(ts: ms) -> int` ### `days_in_month(ts: ms) -> int` 28–31, leap years handled correctly for February. ## Calendar Arithmetic ### `add_days(ts: ms, n: int) -> ms` `n` can be negative. Time::now()); } `} /> ### `add_months(ts: ms, n: int) -> ms` If the day of month doesn't exist in the target month, it clamps to the last valid day — Jan 31 plus one month lands on Feb 28 (or 29). Time::now()); } `} /> ## Period Boundaries Every `start_of_*` function returns midnight UTC on the relevant boundary — useful for anything that resets on a schedule. ### `start_of_day(ts: ms) -> ms` ### `start_of_week(ts: ms, start_day: int = 0) -> ms` `start_day` follows the same ISO convention as `day_of_week` — `0` is Monday by default, but any day can be the week's start. ### `start_of_month(ts: ms) -> ms` ### `start_of_year(ts: ms) -> ms` ### `start_of_period(ts: ms, schedule: str) -> ms` The most flexible of the five — a schedule expression instead of a fixed unit. Returns `null` for an invalid schedule string: - `"monthly:N"` — the Nth day of every month (clamped to the month's length) - `"monthly:last"` — the last day of every month - `"weekly:mon"` — a given weekday, every week (`mon`/`tue`/`wed`/`thu`/`fri`/`sat`/`sun`) - `"nth_weekday:N:mon"` — the Nth occurrence of a weekday in the month (`N` is 1–4) - `"yearly:M-D"` — a fixed month and day every year - `"quarterly:D"` — a given day of the first month of each quarter (Jan/Apr/Jul/Oct) --- ## Agent Tools at Runtime With function calling, a model can only use the tools you wrote ahead of time. When a customer asks for something slightly different, like "how much have I spent this year?", the agent has to string together calls it wasn't designed for, or give up. Here the agent's tools live in a Stof document. Your app turns them into tool definitions for the model, runs the model's tool calls inside the document, and lets the agent write a new tool when it needs one. The new tool is sandboxed like everything else, and it's there for the rest of the conversation. ## The Agent Document Tools are functions tagged `#[tool('description')]`. They reach your systems only through functions your app provides (`App.orders`, `App.refund`). The document also has the app's two helpers: `tool_specs` turns the tools into JSON tool definitions, and `call_tool` runs one. Press **Run** to see the agent add a tool and use it. In this demo, a root named `App` stands in for your app's functions (so the tools call `App.orders` with a dot, which reaches a root or a library; `App::orders` would only reach the library): list { await App.orders(customer_id) } #[tool('Refund an order')] fn refund(order_id: str, amount: float) -> str { await App.refund(order_id, amount) } #[tool('Add a tool for yourself. source is Stof: one function tagged #[tool(description)] that uses App functions.')] fn add_tool(source: str) -> str { try { using staging = new {}; parse(source, staging); const added = staging.funcs('tool'); if (added.len() != 1) return 'add exactly one function tagged #[tool]'; parse(source, self); return 'added ' + added[0].name(); } catch (error) { return 'that tool did not parse: ' + str(error); } } } fn json_type(stof_type: str) -> str { switch (stof_type) { case 'str': 'string' case 'int': 'integer' case 'float': 'number' case 'bool': 'boolean' case 'list': 'array' default: 'object' } } /// The tools, as JSON tool definitions for the model's API. fn tool_specs() -> str { using specs = new { tools: [] }; for (const tool in self.tools.funcs('tool')) { const spec = new { name: tool.name(), description: tool.attributes().get('tool'), input_schema: new { type: 'object', properties: new {}, required: [] }, } on specs; for (const (name, type) in tool.params()) { spec.input_schema.properties.insert(name, new { type: self.json_type(type) }); spec.input_schema.required.push_back(name); } specs.tools.push_back(spec); } stringify('json', specs) } /// Runs a tool the model called, with the arguments it gave. fn call_tool(name: str, args: map) -> unknown { const tool = self.tools.get(name); if (tool == null || !tool.has_attribute('tool')) throw(\`no tool named \${name}\`); const values = []; for (const (param, _) in tool.params()) values.push_back(args.get(param)); tool.call_expanded(values) } // ---- for this demo only: stands in for the functions your app provides ---- root App { fn orders(customer_id: str) -> list { [{'id': 'o_1', 'total': 40}, {'id': 'o_2', 'total': 250}] } fn refund(order_id: str, amount: float) -> str { 'refunded ' + amount + ' on ' + order_id } } #[main] fn main() { // the model asks for a tool it doesn't have, then uses it pln(self.call_tool('add_tool', {'source': r#" #[tool('Total a customer has spent across their recent orders')] fn total_spent(customer_id: str) -> float { let total = 0; for (const order in await App.orders(customer_id)) total += order.get('total'); total } "#})); pln(self.call_tool('total_spent', {'customer_id': 'c_9'})); // a broken tool comes back as a message the model can act on pln(self.call_tool('add_tool', {'source': 'fn broken( {'})); } `} /> ## Giving the Agent Your Systems The functions you give the document are the only way its tools can reach anything. Put your limits in them, not in the tools, because the agent can write new tools that call them: ```typescript function connect(doc: StofDoc, customerId: string) { // scoped to this customer: a tool can't ask for anyone else's orders doc.lib('App', 'orders', async () => db.orders.forCustomer(customerId), true); doc.lib('App', 'refund', async (orderId: string, amount: number) => { if (amount > 100) throw new Error('refunds over $100 need a person to approve them'); return payments.refund(orderId, amount); }, true); doc.setMaxExecutionTime(2000); // no tool runs longer than 2 seconds } ``` The document has no network, file, or environment access, so `App.orders` and `App.refund` are all of its reach. A refund over $100 fails the same way whether it comes from the built-in `refund` tool or one the agent wrote. ## The Conversation Loop Each turn, give the model the current tools, and run whatever it calls: ```typescript const doc = await StofDoc.new(AGENT_SOURCE); connect(doc, customerId); // Maps from Stof become plain objects in JSON const toJSON = (value: unknown) => JSON.stringify(value, (_, v) => v instanceof Map ? Object.fromEntries(v) : v); while (true) { const tools = JSON.parse(await doc.call('tool_specs') as string).tools; const response = await model.chat({ messages, tools }); messages.push(response.message); if (!response.toolCalls.length) break; for (const call of response.toolCalls) { let result: string; try { result = toJSON(await doc.call('call_tool', call.name, call.input)); } catch (error) { result = `error: ${error}`; // a plain message the model can act on } messages.push({ role: 'tool', toolCallId: call.id, content: result }); } } ``` Because `tool_specs` reads the document each turn, a tool the agent adds with `add_tool` shows up in the very next request, with its description and parameter types. Tell the model in the system prompt that it can write tools, and give it the [Stof skill](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md) so it writes them correctly. ## Keeping the Agent Between Requests The agent is a document, so save it like data. Binary Stof keeps everything, including the tools it wrote: ```typescript // end of a request await sessions.save(sessionId, doc.blobify('bstf')); doc.dispose(); // next request const doc = await StofDoc.new(await sessions.load(sessionId), 'bstf'); connect(doc, customerId); // your functions aren't saved with it; connect them again ``` You can keep more than tools in the same document: notes the agent takes, the customer's preferences, a running summary of the conversation. It's all one value to store, and it moves between services or into a different language (a Python worker, say) without any translation. ## Why This Works - **Tools are data.** They're listed, described, saved, and sent like any other part of the document, so the tool definitions can't drift from the tools. - **New tools are safe to accept.** A tool the agent writes can only do what your functions let it, within a time limit. - **Mistakes are recoverable.** A tool that doesn't parse, or one that throws, comes back to the model as a plain message, and the model fixes it. To have a model write logic once and run it many times, rather than during a conversation, see [Run Rules a Model Writes](./ai-rules). --- ## Run Rules a Model Writes Your support team wants to change how tickets are routed. Today that's a ticket for engineering. Instead, they write the policy in plain English, a model turns it into code once, your app checks that code, and then it runs on every ticket, in your process, in under a millisecond, with no model call per ticket. The hard part has always been the middle: running code a model wrote. Stof makes that the easy part, because the model's output is a document that can only see itself and the functions you give it. ## The Agreement Your app and the model agree on a small contract, and everything else is up to the model: - Each rule is a function tagged `#[route]` that takes a `ticket` object. - A ticket has `subject`, `body`, and `plan`. Rules set `ticket.team` and `ticket.priority`. - Rules can run in any order, so each one only sets what it's responsible for. - The model includes `#[test]` functions that show its rules working. Your app owns one small function that runs every rule on a ticket. The model never writes this part: ```stof title="harness.stof" /// Runs every #[route] rule on a ticket and returns the routed ticket. fn route(input: map) -> map { using ticket = Obj::from_map(input); for (const rule in funcs(attributes = 'route')) rule(ticket); ticket.to_map() } ``` ## What the Model Writes Given the policy "Refunds and billing questions go to billing. Outages go to infra and are urgent. Enterprise customers are always urgent. Everything else goes to support," a model writes something like this. Press **Run** to route a few tickets with it: map { using ticket = Obj::from_map(input); for (const rule in funcs(attributes = 'route')) rule(ticket); ticket.to_map() } #[main] fn main() { const tickets = [ {'subject': 'Charged twice', 'body': 'Please refund one', 'plan': 'pro'}, {'subject': 'Dashboard is down', 'body': '', 'plan': 'enterprise'}, {'subject': 'How do I export?', 'body': '', 'plan': 'free'}, ]; for (const input in tickets) { const routed = self.route(input); pln(routed.get('subject') + ' -> ' + routed.get('team') + ', ' + routed.get('priority')); } } `} /> Anyone on the team can read that, including the people who wrote the policy. It's short because Stof reads like the JSON and JavaScript the model already knows. ## Asking the Model Give the model three things: the [Stof skill](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md), so it writes correct Stof; the contract above; and the policy. Ask for Stof only, so the response is the document: ```typescript // stofSkill is the text of SKILL.md; model.generate is your model API call const prompt = (policy: string) => ` ${stofSkill} Write Stof rules for routing support tickets. - Each rule is a function tagged #[route] that takes a ticket: obj. - A ticket has subject, body, and plan. Set ticket.team and ticket.priority. - Rules run in any order, so each rule only sets what it's responsible for. - Include #[test] functions that check your rules on example tickets. Reply with only the Stof document. The policy: ${policy}`; ``` ## Checking It Before You Trust It Three checks, in order. Each one fails with a plain message that says what's wrong and where, so the message goes straight back to the model for another try: ```typescript // Tickets your team has already routed by hand: the rules have to agree with them. const EXAMPLES = [ { ticket: { subject: 'Charged twice', body: 'Please refund one', plan: 'pro' }, team: 'billing', priority: 'normal' }, { ticket: { subject: 'Dashboard is down', body: '', plan: 'enterprise' }, team: 'infra', priority: 'urgent' }, { ticket: { subject: 'How do I export?', body: '', plan: 'free' }, team: 'support', priority: 'normal' }, ]; async function check(rules: string): Promise { try { // 1. It parses, and the model's own tests pass. const tested = await StofDoc.new(rules, 'stof', 'test'); tested.setMaxExecutionTime(1000); await tested.run('test'); tested.dispose(); // 2. It agrees with tickets your team already routed. const doc = await StofDoc.new(HARNESS + rules); doc.setMaxExecutionTime(100); for (const { ticket, team, priority } of EXAMPLES) { const routed = await doc.call('route', ticket) as Map; if (routed.get('team') !== team || routed.get('priority') !== priority) { return `For ${JSON.stringify(ticket)} you routed to ${routed.get('team')}/${routed.get('priority')}, ` + `but it should be ${team}/${priority}.`; } } doc.dispose(); return null; } catch (error) { return String(error); // a parse error or a failed test, with the line it happened on } } export async function compileRules(policy: string): Promise { let feedback = ''; for (let attempt = 0; attempt < 3; attempt++) { const rules = await model.generate(prompt(policy) + feedback); const problem = await check(rules); if (!problem) return rules; // store this with the policy feedback = `\n\nYour last version had a problem. Fix it:\n${problem}`; } throw new Error('the model could not write rules that pass the checks'); } ``` A mistake comes back looking like this, and it's exactly what the model needs to fix it: ```text parse error: expected ';' --> line 3, column 16 | 3 | ticket.team = ; | ^ ``` The third check is a person. The rules are short and readable, so show them, with their tests, to whoever wrote the policy before switching them on. ## Running It on Every Ticket Load the stored rules once, and call `route` for each ticket: ```typescript const doc = await StofDoc.new(HARNESS + storedRules); doc.setMaxExecutionTime(50); // milliseconds: a rule that loops forever fails, it doesn't hang export async function routeTicket(ticket: Ticket) { const routed = await doc.call('route', ticket) as Map; return Object.fromEntries(routed); // { ..., team: 'infra', priority: 'urgent' } } ``` The rules can't read files, environment variables, or the network, and they can't call anything in your app. All they can do is look at the ticket and set fields on it. If you want them to know more, say a customer's open-ticket count, give them exactly that function: `doc.lib('App', 'open_tickets', countOpenTickets)`, then add it to the contract. ## Why This Works - **No model call per request.** The model runs once, when the policy changes. Routing is fast, cheap, and gives the same answer every time. - **No infrastructure.** No container or sandbox service to run. The rules run inside your app like any other data. - **Checked before it's trusted.** Parse errors, the model's tests, your examples, and a person's review, all before a single real ticket. - **Readable.** The rules are the documentation of the policy, so the people who own the policy can review them. The same shape works for anything a model can write once and your app runs many times: lead scoring, fraud flags, content moderation rules, data cleanup, alert conditions. For logic that changes during a conversation, see [Agent Tools at Runtime](./agent-tools). --- ## A Self-Validating Config A config file says what the settings are, but not what they're allowed to be. The rules live somewhere else: a schema file, a validation function in the app, a comment that says "must be under 5 minutes". So a `request_timeout: 300` slips through because nobody knows whether it's seconds or milliseconds, or a production setting breaks at startup instead of in review. In Stof, the config carries its own rules. One type lists every setting with its default, its units, and what makes it valid. Each environment only lists what's different, and the config can check itself, in your app and in CI. ## Settings, Defaults, and Rules Press **Run** to build two environments, check them, and see the problems in a broken one: target_val != null && target_val > 0 && target_val < 65_536)] int port: 8080 #[schema((target_val: str): bool => target_val != null && target_val.starts_with('postgres://'))] str database_url: 'postgres://localhost/app' #[schema((target_val: s): bool => target_val != null && target_val >= 1s && target_val <= 5min)] s request_timeout: 30s #[schema((target_val: int): bool => target_val != null && target_val >= 1 && target_val <= 64)] int workers: 2 MB cache: 256MB /// Computed from the others, so it can't drift. fn max_connections() -> int { self.workers * 10 } } /// Each environment only says what's different. Config development: {} Config production: { port: 443 workers: 16 request_timeout: 10s cache: 2GB } Config broken: { port: 70_000 request_timeout: 10min } /// The names of the fields that fail their rules. fn problems(config: obj) -> list { using checked = copy(config); .schemafy(checked, remove_invalid = true); const failed = []; for (const (name, _) in config.fields()) { if (!checked.contains(name)) failed.push_back(name); } failed } #[main] fn main() { pln(self.production.max_connections()); pln(str(self.production.cache)); pln(stringify('json', self.production)); pln(self.problems(self.production)); pln(self.problems(self.broken)); } `} /> A few things are doing work here: - **Units.** `request_timeout` is in seconds, whatever you write: `10s`, `10000ms`, and `1min` all work, and `cache: 2GB` becomes `2000MB`. No more guessing what a bare number means. - **Defaults from the type.** `development` is empty and gets every default. `production` lists four settings. - **Computed settings.** `max_connections` is a function, so it always agrees with `workers`. - **Field-level errors.** `problems` checks a copy with `remove_invalid`, so whatever was removed is what failed. ## Loading It in Your App Pick the environment, fill in secrets from the environment, check it, and hand your app plain values: ```typescript export async function loadConfig(environment: string) { const doc = await StofDoc.new(await readFile('config.stof', 'utf8')); if (process.env.DATABASE_URL) doc.set(`${environment}.database_url`, process.env.DATABASE_URL); const problems = await doc.call('check', environment) as string[]; if (problems.length) throw new Error(`invalid ${environment} config: ${problems.join(', ')}`); const config = doc.record(doc.get(environment) as string); config.max_connections = await doc.call(`${environment}.max_connections`); doc.dispose(); return config; // { port: 443, workers: 16, request_timeout: 10, cache: 2000, ... } } ``` ```stof /// Problems with one environment's config, by name. fn check(environment: str) -> list { self.problems(self.get(environment)) } ``` Secrets stay out of the file: the app sets them, and the rules still check them. Units come out as numbers in the setting's unit (`request_timeout: 10` seconds, `cache: 2000` megabytes), so the app never converts. ## Checking Every Environment in CI Add a test, and `stof test config.stof` fails the build when any environment breaks a rule: ```stof #[test] fn every_environment_is_valid() { for (const environment in ['development', 'production']) { assert_eq(self.check(environment), []); } } ``` A bad change to production shows up in review, with the setting's name, instead of at startup. ## Exporting for Other Tools Something else needs JSON or YAML? Export any environment, with defaults filled in and units converted: ```typescript doc.stringify('json', doc.get('production') as string); // {"port":443,"workers":16,"request_timeout":10.0,"cache":2000.0,"database_url":"postgres://localhost/app"} ``` ## Why This Works - **The rules live with the settings.** Each setting's type, unit, default, and rule are on one line, so they can't drift apart. - **Environments stay short.** Each one lists only what's different, so a review shows the real changes. - **Mistakes are named.** A broken config says which setting failed, in your app and in CI. - **Still plain data.** Your app gets ordinary values, and other tools get JSON or YAML. See [Prototypes & Schemas](../learn/prototypes-and-schemas) for more on types and rules. --- ## Rules Your Customers Write Every B2B product reaches this point: each customer wants its own rules. Invoices over $10,000 need the CFO. Software goes through IT. New vendors need a review. So you design a rules format, usually JSON: ```json { "all": [ { "field": "amount", "op": ">", "value": 10000 }, { "field": "vendor.months_known", "op": "<", "value": 6 } ], "then": { "require": "cfo" } } ``` Then the requests start. Dates. "Any of these." A percentage of last month's spend. Each one means new operators, a new evaluator branch, and a new corner of the UI, and the format still can't say what customers mean. Instead, let the rules be Stof. Your platform defines the vocabulary, each customer's rules are a small document, and your app runs them in a sandbox. The rules can say anything a short function can, and you never extend a rules language again. ## The Vocabulary Your Platform Provides Your platform owns one part of the document: the types and functions rules can use, and the function that runs them. This is your API for rule writers, and it's ordinary Stof: ```stof title="platform.stof" #[type] Invoice: { float amount: 0 list approvals: [] /// Ask a role to approve this invoice before it's paid. fn require(role: str, reason: str) { self.approvals.push_back({'role': role, 'reason': reason}); } } /// Runs every #[approval] rule on an invoice, and returns the approvals it needs. fn evaluate(input: map) -> list { using invoice = Obj::from_map(input); invoice.set_prototype('Invoice'); invoice.approvals = []; for (const rule in funcs(attributes = 'approval')) rule(invoice); invoice.approvals } ``` To give rules more to work with, add to it: an `invoice.vendor_total(days)` method, a `Calendar.is_quarter_end()` function from your app, and so on. ## A Customer's Rules Each customer's rules are functions tagged `#[approval]`. They can be written in an editor in your app, by your support team, or by an AI assistant from a plain-English description (see [Run Rules a Model Writes](./ai-rules)). Press **Run** to evaluate a few invoices: 10_000 && invoice.vendor.months_known < 6) { invoice.require('cfo', 'large invoice from a new vendor'); } } /// Software purchases need IT, whatever the amount. #[approval] fn software(invoice: Invoice) { if (invoice.category == 'software') invoice.require('it', 'software purchases go through IT'); } // ---- the platform's part ---- #[type] Invoice: { float amount: 0 list approvals: [] fn require(role: str, reason: str) { self.approvals.push_back({'role': role, 'reason': reason}); } } fn evaluate(input: map) -> list { using invoice = Obj::from_map(input); invoice.set_prototype('Invoice'); invoice.approvals = []; for (const rule in funcs(attributes = 'approval')) rule(invoice); invoice.approvals } #[main] fn main() { const invoices = [ {'id': 'inv_1', 'amount': 25000, 'category': 'services', 'vendor': {'name': 'Acme', 'months_known': 2}}, {'id': 'inv_2', 'amount': 1200, 'category': 'software', 'vendor': {'name': 'Figma', 'months_known': 30}}, {'id': 'inv_3', 'amount': 300, 'category': 'office', 'vendor': {'name': 'Staples', 'months_known': 40}}, ]; for (const invoice in invoices) { const needed = self.evaluate(invoice); if (needed.len() == 0) pln(invoice.get('id') + ': approved'); for (const approval in needed) { pln(invoice.get('id') + ': needs ' + approval.get('role') + ' (' + approval.get('reason') + ')'); } } } `} /> Each rule is a few readable lines with a doc comment, and it's the customer's own record of how they work. ## Running Them Load the platform's part plus the customer's rules, and evaluate: ```typescript type Approval = { role: string; reason: string }; const toApprovals = (list: unknown) => (list as Map[]).map((approval) => Object.fromEntries(approval) as Approval); async function load(rules: string) { const doc = await StofDoc.new(PLATFORM + rules); doc.setMaxExecutionTime(100); return doc; } export async function approvalsFor(customer: Customer, invoice: Invoice): Promise { const doc = await load(customer.rules); // cache this per customer return toApprovals(await doc.call('evaluate', invoice)); } ``` One customer's rules can't see another's data, your database, or the network. They see the invoice they're given and the vocabulary you provide, and a rule that loops forever stops after 100ms. ## Previewing a Change Rules are code, so you can try a change before it's live. Run the current rules and the proposed ones over recent invoices, and show the customer exactly what would change: ```typescript export async function previewRules(current: string, proposed: string, recent: Invoice[]) { const before = await load(current); const after = await load(proposed); // throws with the line number if it doesn't parse const changes = []; for (const invoice of recent) { const was = toApprovals(await before.call('evaluate', invoice)).map((a) => a.role).sort().join(', ') || 'approved'; const now = toApprovals(await after.call('evaluate', invoice)).map((a) => a.role).sort().join(', ') || 'approved'; if (was !== now) changes.push({ invoice: invoice.id, was, now }); } return changes; } ``` Adding "anything over $5,000 needs a manager" might give: ```text [ { invoice: 'inv_1', was: 'cfo', now: 'cfo, manager' }, { invoice: 'inv_3', was: 'approved', now: 'manager' } ] ``` That's a far better review than reading a JSON tree, and it's something a JSON rule format rarely offers. ## Why This Works - **No rules language to build.** Your vocabulary is a Stof type with a few functions. Growing it means adding a function, not a new operator, parser branch, and UI widget. - **Rules say what customers mean.** Conditions, math, units, dates, and helper functions, in a few readable lines. - **Safe for many customers.** Each customer's rules run in their own sandbox, with a time limit, and only reach what you give them. - **Easy to keep.** Rules are text with comments and tests. Store them per customer, version them, diff them, and preview them before they go live. The same pattern fits webhooks and automations, pricing and discount rules, alert conditions, lead routing, data validation, and any "if this, then that" a product lets users configure. --- ## Feature Flags and Entitlements Feature flag services give you a dashboard and a fixed menu of targeting options: on, off, a percentage, a list of users, maybe an attribute match. The moment a rule doesn't fit the menu ("Team plans, plus anyone in the beta, but not in the EU yet"), it ends up in application code, split across the server and the front end. With Stof, each flag is a small function that decides whether it's on for a user. The flags are one document, evaluated in your process, on the server and in the browser alike, and changed by publishing a new version. ## The Flags Each flag is a function tagged `#[flag]` that takes a user. A helper gives every user a stable number from 0 to 99 for percentage rollouts. Press **Run** to evaluate the flags for two users and check the rollout across 1,000 users: bool { user.country == 'US' && self.bucket(user.id, 'new_checkout') < 25 } /// The AI assistant is a paid feature. #[flag] fn ai_assistant(user: obj) -> bool { user.plan == 'pro' || user.plan == 'team' } /// Bulk export: Team plans, and anyone in the beta. #[flag] fn bulk_export(user: obj) -> bool { user.plan == 'team' || self.beta.contains(user.id) } /// A number from 0 to 99 that's the same for a user every time, and different for each flag. fn bucket(user_id: str, flag: str) -> int { let hash = 0; for (const byte in (flag + ':' + user_id) as blob) hash = (hash * 31 + byte) % 1_000_003; hash % 100 } /// Every flag that's on for a user. fn flags_for(input: map) -> list { using user = Obj::from_map(input); const on = []; for (const flag in self.funcs('flag')) { if (flag(user)) on.push_back(flag.name()); } on.sort(); on } #[main] fn main() { pln(self.flags_for({'id': 'u_7', 'plan': 'free', 'country': 'US'})); pln(self.flags_for({'id': 'u_8', 'plan': 'team', 'country': 'DE'})); let rolled_out = 0; for (const i in 1000) { if (self.new_checkout(using new { id: 'u_' + i, country: 'US' })) rolled_out += 1; } pln(\`new_checkout is on for \${rolled_out} of 1000 US users\`); } `} /> A flag can be as simple as `true` or as specific as your business needs, and each one carries a comment that says why it exists. Plan entitlements, like `ai_assistant`, sit right next to experiments, so "what can this user do?" has one answer. ## On the Server Load the flags once, and evaluate them for each request. Every flag for a user takes well under a millisecond: ```typescript let flags = await StofDoc.new(await fetchPublishedFlags(), 'bstf'); flags.setMaxExecutionTime(20); app.use(async (req, res, next) => { req.flags = new Set(await flags.call('flags_for', { id: req.user.id, plan: req.account.plan, country: req.geo.country, }) as string[]); next(); }); app.post('/export', (req, res) => { if (!req.flags.has('bulk_export')) return res.status(403).end(); // ... }); ``` ## In the Browser The npm package runs in the browser too, so the front end can load the same document and evaluate flags locally, with no request per check and no flicker while flags load: ```typescript const flags = await StofDoc.new(new Uint8Array(await (await fetch('/flags.bstf')).arrayBuffer()), 'bstf'); const on = await flags.call('flags_for', currentUser) as string[]; ``` The browser gets the whole document, so it can see every rule and list in it. For flags with sensitive targeting, like a list of customer names, evaluate on the server and send the result instead. ## Changing Flags Flags change the same way [pricing plans](./usage-pricing#changing-plans-without-a-deploy) do: edit the document, publish it as binary Stof, and every service and browser swaps it in. Turning a feature off in an emergency is a one-line change: ```stof #[flag] fn new_checkout(user: obj) -> bool { false } ``` Keep the flags in version control with a few `#[test]` functions ("Team users get bulk export", "nobody outside the US gets the new checkout"), and publish from CI, so every flag change is reviewed and tested like code. ## Why This Works - **Any rule you can write.** Targeting isn't limited to a menu of options: it's a function, with access to everything you pass in. - **The same answer everywhere.** Server, browser, and workers evaluate one document with one runtime. - **Fast and local.** No network call per check, and nothing to wait for on page load. - **Reviewed like code.** Flags are text with comments and tests, in version control. Plan entitlements, kill switches, gradual rollouts, and A/B assignments all fit this shape. For limits and billing on top of entitlements, see [Usage Limits and Pricing Plans](./usage-pricing). --- ## Cookbook Each recipe here is a real problem, solved end to end: the Stof document, which you can run on the page, and the app code around it. Every one comes down to the same thing: logic that changes more often than your app, or comes from somewhere other than your codebase, as data your app can check and run safely. ## Recipes **Logic from AI models** - **[Run Rules a Model Writes](./ai-rules)**: turn a plain-English policy into code with a model, check it against tests and real examples, and run it on every request with no model call. - **[Agent Tools at Runtime](./agent-tools)**: tools as Stof functions, tool definitions generated from them, and an agent that writes new tools mid-conversation, all sandboxed. **Logic your customers and teams change** - **[Rules Your Customers Write](./customer-rules)**: per-customer business rules instead of a homegrown rules language, with a preview of what a change would do. - **[Usage Limits and Pricing Plans](./usage-pricing)**: plans, limits, and overage billing in one document every service asks, changed without a deploy. - **[Feature Flags and Entitlements](./feature-flags)**: targeting rules as functions, stable rollouts, and the same flags on the server and in the browser. **Data and logic that travel** - **[A Self-Validating Config](./config)**: settings with their units, defaults, and rules, environments that list only what's different, and field-level errors in CI. - **[One Shape for Data from Many Sources](./webhooks)**: a small mapping per webhook provider or feed, in any format, checked at the edge. - **[The Same Logic in the Browser and on the Server](./shared-logic)**: one checkout calculation for a TypeScript front end and a Python backend. ## Patterns You'll See The recipes share a handful of patterns. Once you know them, most new uses are a combination. ### The attribute is the contract Your app runs functions by attribute: `#[route]`, `#[approval]`, `#[flag]`, `#[tool]`. Whoever writes the logic (a model, a customer, another team) can add as many functions as they need, with any names, and your app finds them with `funcs(attributes = 'route')` or `doc.run('route')`. See [Attributes](../learn/attributes). ### Your app owns a small harness The part that runs the untrusted part is a few lines your app writes: take the input, call every tagged function, return the result. The recipes call it `route`, `evaluate`, `normalize`, and `call_tool`. It keeps the contract in one place, and the logic you accept never has to know how it's called. ### Maps in, maps out Your app passes plain objects (a JavaScript object or a Python dict), which arrive as maps. The harness turns one into an object with `Obj.from_map`, declares it with `using` so it's cleaned up afterward, and returns a map or JSON. Nothing piles up in a long-lived document. ### Your functions are the only door A document can't reach files, the network, or your app unless you give it a way. Give it specific functions with `doc.lib(..)`, scoped to the current customer or request, and put your limits inside them (like "refunds over $100 need a person"), because new logic can call them too. See [What a Document Can Reach](/start/using-stof#what-a-document-can-reach). ### Check before you trust Logic from outside goes through checks before it's live: it parses, its own `#[test]` functions pass, it agrees with examples you know the answer to, and its output fits a `#[schema]`. Every failure is a plain message with a line number, which a model can fix and a person can read. ### Logic ships as data Binary Stof (`bstf`) holds logic and data together. Publish a new document, and every service and browser swaps it in: new plans, flags, rules, or mappings, with no release. Keep the source in version control with tests, and publish from CI. ### Every call has a time limit Set one with `setMaxExecutionTime`, sized to the job: tens of milliseconds for a check on every request, seconds for an agent's tool. A runaway loop fails that one call, and nothing else. --- ## The Same Logic in the Browser and on the Server A checkout page shows the total as you type: shipping, discount codes, tax. The server computes the total again before charging the card, because it can't trust the browser. That's the same logic written twice, often in two languages, and every change to shipping or tax has to land in both. When they drift, a customer sees one total and pays another. Write it once, as a Stof document. The browser runs it through the npm package, the server runs it in Python (or Node.js, or Rust), and both get the same numbers, because it's the same document and the same runtime. ## The Logic Rates, discount codes, and the calculation, in one document. Press **Run** to price a cart: map { let subtotal = 0.0; for (const item in cart.get('items')) subtotal += item.get('price') * item.get('quantity'); const discount = (subtotal * (self.codes.get(cart.get('code') ?? '') ?? 0)).round(2); const discounted = subtotal - discount; const shipping = discounted >= self.rates.free_shipping_over ? 0 : self.rates.shipping; const tax = ((discounted + shipping) * (self.rates.tax.get(cart.get('country')) ?? 0)).round(2); { 'subtotal': subtotal, 'discount': discount, 'shipping': shipping, 'tax': tax, 'total': (discounted + shipping + tax).round(2), } } #[main] fn main() { const cart = { 'items': [{'sku': 'mug', 'price': 18, 'quantity': 2}, {'sku': 'tee', 'price': 25, 'quantity': 1}], 'code': 'SAVE10', 'country': 'DE', }; pln(self.quote(cart)); } `} /> This example uses decimal amounts to stay readable. When exact money matters, keep amounts in cents, as integers, the way payment APIs do. ## In the Browser Load the published document once, and price the cart whenever it changes. Each quote takes a fraction of a millisecond, with no request to the server: ```typescript const response = await fetch('/checkout.bstf'); const checkout = await StofDoc.new(new Uint8Array(await response.arrayBuffer()), 'bstf'); async function updateTotals(cart: Cart) { const quote = Object.fromEntries(await checkout.call('quote', cart) as Map); render(quote); // { subtotal: 61, discount: 6.1, shipping: 0, tax: 10.43, total: 65.33 } } ``` With a bundler, initialize Stof first. See [Browsers and Bundlers](/start/install#browsers-and-bundlers). ## On the Server The server loads the same document and computes the total it charges. Here it's Python, but Node.js and Rust work the same way: ```python from pystof import Doc checkout = Doc() checkout.binary_import(open('checkout.bstf', 'rb').read(), 'bstf') @app.post('/orders') def create_order(cart: dict): quote = checkout.call('quote', [cart]) # a dict: the same numbers the browser showed charge(cart['payment_method'], quote['total']) return quote ``` A JavaScript object and a Python dict both arrive in Stof as a map, and the result comes back as a `Map` or a `dict`. The server never trusts the total from the browser. It doesn't have to, because it gets the same answer itself. ## Publishing Changes New tax rates and promo codes are a change to the document. Publish it as binary Stof, and the browser and the servers pick it up together, with no app release: ```typescript const doc = await StofDoc.new(checkoutSource); await storage.put('checkout.bstf', doc.blobify('bstf')); ``` Add `#[test]` functions for the cases that matter ("free shipping at $50", "SAVE10 takes 10% off before tax"), and run `stof test` before publishing. ## Why This Works - **Written once.** One document is the logic for every place it runs, in any language Stof runs in. - **The same answer.** One runtime, so no rounding differences or drift between implementations. - **Instant in the browser.** Totals update locally, with the server as the source of truth. - **Changed without a release.** Rates and codes ship as a document to both sides at once. The same pattern fits form validation, quote and estimate calculators, eligibility checks, and anything else a user should see before the server decides. --- ## Usage Limits and Pricing Plans Pricing logic ends up everywhere. The API checks limits before each request, the billing job computes overages, the app shows "you've used 80%", and the pricing page lists what each plan includes. Each one has its own copy of the rules, and every pricing change is a coordinated deploy that someone gets slightly wrong. Put the plans in one Stof document: the numbers and the logic that uses them. Every service loads the same document and asks it questions, in-process, in a fraction of a millisecond. To change pricing, publish a new document. ## The Plans Each plan is an object of type `Plan`. The type holds the logic, and each plan holds its numbers. Press **Run** to check some usage against two plans: map { const requests = used.get('requests') + 1; const storage = used.get('storage_mb') as MB; if (storage > self.storage) { return {'allowed': false, 'reason': \`storage is full (\${self.storage} on \${self.name})\`}; } if (requests > self.requests && self.overage == 0) { return {'allowed': false, 'reason': \`\${self.name} includes \${self.requests} requests a month\`}; } {'allowed': true, 'percent_used': ((requests as float) / self.requests * 100).round(1)} } /// This month's bill. fn bill(used: map) -> float { const extra = Num::max(used.get('requests') - self.requests, 0); (self.price + extra * self.overage).round(2) } } Plan free: { name: 'Free' requests: 100 } Plan pro: { name: 'Pro' price: 20 requests: 5_000 overage: 0.002 storage: 10GB } #[main] fn main() { const used = {'requests': 6_200, 'storage_mb': 2_400}; pln(self.pro.check(used)); pln(self.free.check(used)); pln(self.pro.bill(used)); } `} /> Units do the conversions: usage arrives in megabytes, the plan says `10GB`, and the comparison is exact. A new plan is a few lines of numbers, and a new kind of limit is a few lines in `check`. ## Checking Every Request The API server loads the plans once and asks them about each request. It's a function call inside your process, so there's no pricing service to call and nothing to cache: ```typescript let pricing = await StofDoc.new(await fetchPublishedPlans(), 'bstf'); pricing.setMaxExecutionTime(20); export async function allow(account: Account, used: Usage) { const decision = await pricing.call(`${account.plan}.check`, used) as Map; return Object.fromEntries(decision); // { allowed: true, percent_used: 41.2 } } app.use(async (req, res, next) => { const decision = await allow(req.account, await usage.thisMonth(req.account)); if (!decision.allowed) return res.status(429).json({ error: decision.reason }); next(); }); ``` The billing job loads the same document and calls `bill`. The front end can load it too, through the npm package in the browser, to show usage bars and to power the pricing page, and every one of them gives the same answer. ## Changing Plans Without a Deploy The pricing service publishes the plans as binary Stof, which carries the logic along with the numbers. Every service swaps in the new document when it arrives: ```typescript // pricing service: publish the current plans const plans = await StofDoc.new(plansSource); await publish('pricing', plans.blobify('bstf')); // every service: swap in new plans as they arrive subscribe('pricing', async (bytes: Uint8Array) => { const next = await StofDoc.new(bytes, 'bstf'); next.setMaxExecutionTime(20); const old = pricing; pricing = next; old.dispose(); }); ``` Raising the Free plan to 500 requests, adding a plan for a single customer, or trying a new overage price is a new document, not a release. Keep the plans in version control with tests (`#[test]` functions that check a few bills), and publish from CI. ## Going Further: Limitr This is the core of [Limitr](https://limitr.dev), a pricing runtime for usage-based and AI software built on Stof. Limitr adds what a real pricing system needs around it: metering, credits, costs and margins for AI usage, an event ledger, invoicing, and a cloud service that streams usage from every running app and pushes policy changes back to it. Each plan, limit, and credit rule in Limitr is a Stof document evaluated in-process, exactly like this one. ## Why This Works - **One source of truth.** Every service asks the same document, so the API, billing, and the pricing page can't disagree. - **Fast.** A check runs in your process in a fraction of a millisecond (about 0.2ms in Node.js), with no network call. - **Changes are data.** New plans and prices ship as a document, in seconds, without a deploy. - **Readable.** The rules are short enough for the people who set pricing to review. The same approach works for [feature flags and entitlements](./feature-flags), rate limits, quotas, and any other rule that many services need to agree on. --- ## One Shape for Data from Many Sources Integrations pile up. Payments arrive from Stripe as JSON with amounts in cents, and from an older billing system as form posts with amounts in dollars. Orders come from a storefront, a marketplace, and a CSV upload. Each source needs an adapter, the adapters live in your app's code, and when a provider changes a field, fixing it means a deploy. With Stof, each source gets a small mapping document that turns its payload into your one shape. The shape validates itself, so a bad payload is caught at the edge. Adding or fixing a source means adding or changing a mapping, which is data. ## The Shape and the Mappings `Payment` is the shape the rest of your app uses, with a rule for each field. Each mapping is a function tagged `#[from('source')]`, and `normalize` picks the mapping, parses the raw body in whatever format it came in, and checks the result. Press **Run** to normalize a Stripe event and a form post, and to catch a bad one: target_val != null && target_val.len() > 0)] str id: '' #[schema((target_val: float): bool => target_val != null && target_val > 0)] float amount: 0 #[schema((target_val: str): bool => target_val != null && target_val.len() == 3)] str currency: '' str customer: '' str source: '' } /// Stripe sends JSON. Amounts are in cents and currencies are lowercase. #[from('stripe')] fn stripe(event: obj) -> Payment { const charge = event.data.object; new Payment { id: charge.id, amount: (charge.amount as float) / 100, currency: charge.currency.upper(), customer: charge.customer, source: 'stripe', } } /// The old billing system posts form data: ref=A-1009&amount=12.50&cur=usd&cust=42 #[from('legacy')] fn legacy(form: obj) -> Payment { new Payment { id: form.ref, amount: form.amount, currency: form.cur.upper(), customer: 'legacy-' + form.cust, source: 'legacy', } } /// Turns a raw webhook body from any source into a Payment, or throws. fn normalize(source: str, body: str, format: str) -> map { using raw = new {}; parse(body, raw, format); for (const mapper in funcs(attributes = 'from')) { if (mapper.attributes().get('from') == source) { using payment = mapper(raw); if (!.schemafy(payment)) throw(\`\${source} sent a payment that doesn't fit: \${body}\`); return payment.to_map(); } } throw(\`no mapping for \${source}\`); } #[main] fn main() { pln(self.normalize('stripe', '{"type": "charge.succeeded", "data": {"object": {"id": "ch_3Pq", "amount": 2000, "currency": "usd", "customer": "cus_9s"}}}', 'json')); pln(self.normalize('legacy', 'ref=A-1009&amount=12.50&cur=usd&cust=42', 'urlencoded')); try { self.normalize('legacy', 'ref=A-1010&amount=0&cur=usd&cust=42', 'urlencoded'); } catch (error) { pln(error); } } `} /> Each mapping reads like the provider's documentation: which field is which, and what to convert. `using` cleans up the raw payload and the payment object after each call, so a long-running worker doesn't accumulate them. ## In Your App Keep the shape in one file and each source's mapping in its own, load them into one document, and call `normalize` from your webhook handlers: ```typescript const mappings = await StofDoc.new(await readFile('payment.stof', 'utf8')); for (const file of await mappingFiles()) { // stripe.stof, legacy.stof, ... mappings.parse(await readFile(file, 'utf8')); } mappings.setMaxExecutionTime(50); async function normalize(source: string, body: string, format = 'json'): Promise { return Object.fromEntries(await mappings.call('normalize', source, body, format) as Map) as Payment; } app.post('/webhooks/stripe', express.text({ type: '*/*' }), async (req, res) => { verifyStripeSignature(req); // your app still checks who sent it await payments.record(await normalize('stripe', req.body)); res.sendStatus(200); }); app.post('/webhooks/legacy', express.text({ type: '*/*' }), async (req, res) => { await payments.record(await normalize('legacy', req.body, 'urlencoded')); res.sendStatus(200); }); ``` A payload that doesn't fit throws before it reaches your database, with the source and the body in the message. ## Adding a Source A new source is a new mapping file, and nothing else in your app changes. Mappings are also a good job for a model: give it the [Stof skill](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md), the `Payment` type, and a few sample payloads from the provider's docs, and check what it writes against those samples before you add it (see [Run Rules a Model Writes](./ai-rules)). Since mappings are data, you can store them in a database and load new ones without restarting. The formats are built in: `json`, `yaml`, `toml`, `urlencoded`, and `text`. See [Formats](../learn/formats). ## Why This Works - **One shape, checked at the edge.** Every source produces the same validated `Payment`, so the rest of your app trusts it. - **One small file per source.** Each mapping is a few readable lines, and a change to one provider touches one file. - **Any format in.** JSON, form data, YAML, or TOML, parsed by Stof, mapped the same way. - **Safe to change at runtime.** Mappings can only read their payload and build the result, within a time limit. The same pattern fits order and inventory feeds, CRM syncs, analytics events, log formats, and partner APIs: anywhere many shapes need to become one. --- ## Blob Library (Blob) Linked to the `blob` type — the raw-bytes counterpart to `str`, useful anywhere binary data crosses a boundary (`blobify`, `Http`, `fs`, images). ## Byte Access ### `Blob::at(bytes: blob, index: int) -> int` The byte value at an index. ### `Blob::len(bytes: blob) -> int` ### `Blob::size(bytes: blob) -> bytes` Same as `len`, but as a `bytes` unit value instead of a plain int. ## Encoding & Decoding ### `Blob::utf8(bytes: blob) -> str` The default conversion for a plain `as str` cast too. ### `Blob::from_utf8(val: str) -> blob` ### `Blob::base64(bytes: blob) -> str` ### `Blob::from_base64(val: str) -> blob` ### `Blob::url_base64(bytes: blob) -> str` URL-safe Base64 — swaps the characters that aren't URL-safe in standard Base64. ### `Blob::from_url_base64(val: str) -> blob` --- ## Data Library (Data) Every field and function is backed by a `Data` handle — this library is how you work with that handle directly, rather than through the field or function it's attached to. Advanced, low-level territory; most code never needs this. ## Locating Data ### `Data::field(path: str) -> data` A data pointer to a field, by path from the current object. ### `Data::id(ptr: data) -> str` str { 'hi' } #[main] fn main() { const func: fn = self.hi; const id = func.data().id(); pln(id.len() > 0); } `} /> ### `Data::from_id(id: str) -> data` The other direction — reconstruct a pointer from an ID string. str { 'hi' } #[main] fn main() { const func: fn = self.hi; const id = func.data().id(); pln(Data::from_id(id) == func.data()); } `} /> ### `Data::libname(ptr: data) -> str` Which library this data is linked to, if any. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.data().libname()); } `} /> ### `Data::objs(ptr: data) -> list` Every object this data is attached to. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.data().objs().front() == self); } `} /> ## Attaching, Moving & Dropping ### `Data::attach(ptr: data, obj: obj) -> bool` Makes this data reachable from an additional object, under the same name. str { 'hi' } #[main] fn main() { const func: fn = self.hi; const other = new {}; func.data().attach(other); pln(other.hi()); } `} /> ### `Data::move(ptr: data, from: obj, to: obj) -> bool` A drop and an attach in one step — removes the data from one object and places it on another. str { 'hi' } #[main] fn main() { const func: fn = self.hi; const other = new {}; func.data().move(self, other); pln(other.hi(), ' ', self.hi); } `} /> ### `Data::drop(ptr: data) -> bool` str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.data().drop()); } `} /> ### `Data::drop_from(ptr: data, obj: obj) -> bool` Drops from one specific object — if that was the only object referencing it, the data leaves the document entirely. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.data().drop_from(self)); } `} /> ## Serialization ### `Data::blob(ptr: data) -> blob` Serializes the data (name, attributes, value — everything) into bytes. str { 'hi' } #[main] fn main() { const func: fn = self.hi; const bin = func.data().blob(); pln(bin.len() > 0); } `} /> ### `Data::load_blob(bytes: blob, context: obj | str = self) -> data` The other direction — deserializes into a specific object, effectively copying the data onto it. str { 'hi' } #[main] fn main() { const func: fn = self.hi; const bin = func.data().blob(); const other = new {}; Data::load_blob(bin, other); pln(other.hi()); } `} /> ## Validity ### `Data::exists(ptr: data) -> bool` False once the data has been dropped — a pointer held onto after that point doesn't resurrect it. str { 'hi' } #[main] fn main() { const func: fn = self.hi; const ptr = func.data(); drop(func); pln(ptr.exists()); } `} /> ### `Data::invalidate(data: data, symbol: str = 'value') -> bool` Marks data as invalid under a symbol — a lightweight dirty-flag mechanism, throws if the data doesn't exist. str { 'hi' } #[main] fn main() { const func = self.hi; const ptr = func.data(); pln(ptr.invalidate('something_happened')); } `} /> ### `Data::validate(data: data, symbol?: str) -> bool` Clears an invalidation — returns `true` only if it was actually invalid under that symbol (or any symbol, if none given) beforehand. str { 'hi' } #[main] fn main() { const func = self.hi; const ptr = func.data(); ptr.invalidate('something_happened'); pln(ptr.validate('something_happened')); pln(ptr.validate('something_happened')); } `} /> --- ## Function Library (Fn) Linked to the `fn` type — the mechanics behind everything on the Functions and Attributes Learn pages. ## Calling ### `Fn::call(func: fn, ..) -> unknown` `func.call(...)`, `Fn::call(func, ...)`, and `func(...)` are all equivalent. "Hi, " + name; pln(func.call("Bob"), ' ', Fn::call(func, "Bob"), ' ', func("Bob")); } `} /> ### `Fn::call_expanded(func: fn, ..) -> unknown` Same as `call`, but a collection argument gets expanded into positional arguments instead of passed as one value. "Hi, " + name; pln(func.call_expanded(["Bob"])); } `} /> ## Binding ### `Fn::bind(func: fn, to: obj) -> bool` Moves a function onto a different object — `self` inside it now refers to `to`. self.msg ?? 'dne'; const to = new { msg: 'hi' }; func.bind(to); pln(func()); } `} /> ## Introspection ### `Fn::name(func: fn) -> str` str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.name()); } `} /> ### `Fn::params(func: fn) -> list` A list of `(name, type)` tuples. ### `Fn::return_type(func: fn) -> str` int { 42 } #[main] fn main() { const func: fn = self.hi; pln(func.return_type()); } `} /> ### `Fn::is_async(func: fn) -> bool` Shorthand for checking whether the `async` attribute is present. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.is_async()); } `} /> ### `Fn::attributes(func: fn) -> map` str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(str(func.attributes())); } `} /> ### `Fn::has_attribute(func: fn, name: str) -> bool` str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.has_attribute("greeting")); } `} /> ### `Fn::obj(func: fn) -> obj` The first object this function is attached to. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.obj() == self); } `} /> ### `Fn::objs(func: fn) -> list` Every object this function is attached to — usually just one, unless it's been shared via `Data.attach`. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.objs().len()); } `} /> ### `Fn::data(func: fn) -> data` The underlying `Data` handle — see the Data library. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.data().exists()); } `} /> ### `Fn::id(func: fn) -> str` Shorthand for `func.data().id()`. str { 'hi' } #[main] fn main() { const func: fn = self.hi; pln(func.id() == func.data().id()); } `} /> --- ## List Library (List) Linked to the `list` type — Collections covers the core operations already; this page is the complete reference. ## Query ### `List::len(array: list) -> int` ### `List::empty(array: list) -> bool` ### `List::any(array: list, func?: fn) -> bool` Without a function: does the list have any values? With one: does it return true for any value? x > 2)); } `} /> ### `List::contains(array: list, value: unknown) -> bool` ### `List::index_of(array: list, v: unknown) -> int` `-1` if not found. ### `List::is_uniform(array: list) -> bool` If every value has the exact same specific type, will return that type — prototype inheritance isn't considered. Otherwise, returns null. ## Access ### `List::at(array: list, index: int) -> unknown` `&array[i]` for a reference, so mutating the result mutates the list itself. ### `List::front(array: list) -> unknown` ### `List::back(array: list) -> unknown` ## Modify ### `List::push_back(array: list, ..) -> void` Takes any number of values. ### `List::push_front(array: list, ..) -> void` ### `List::pop_back(array: list) -> unknown` ### `List::pop_front(array: list) -> unknown` ### `List::insert(array: list, index: int, val: unknown) -> void` ### `List::replace(array: list, index: int, val: unknown) -> unknown` Returns the old value. ### `List::append(array: list, other: list) -> void` `other` is left unmodified. ### `List::clear(array: list) -> void` ### `List::join(array: list, sep: str) -> str` ## Removing ### `List::remove(array: list, index: int) -> unknown` Returns the removed value, or `null` if the index was out of bounds. ### `List::remove_first(array: list, val: unknown) -> unknown` ### `List::remove_last(array: list, val: unknown) -> unknown` ### `List::remove_all(array: list, val: unknown) -> bool` Returns whether anything was actually removed. ## Transform These call a function for each value, in order, and return a new list (the original isn't changed). ### `List::map(array: list, func: fn) -> list` The result of the function for each value. With an async function, the list holds its promises: await the list. p * 1.2)); } `} /> ### `List::filter(array: list, func: fn) -> list` The values the function returns true for. total >= 100)); } `} /> ## Sort & Reverse ### `List::sort(array: list) -> void` ### `List::sort_by(array: list, func: fn) -> void` The comparator returns negative/zero/positive, same convention as everywhere else. The sort is stable (values that compare equal keep their order), and an error in the comparator fails the sort. Written as a single ternary expression here rather than a chain of bare `if` statements — `if` can't produce a value on its own (see Control Flow), so a chain of them with no `else` wouldn't actually control the sort order the way it looks like it would. a < b ? 1 : (a > b ? -1 : 0)); pln(array); } `} /> ### `List::reverse(array: list) -> void` In place. ### `List::reversed(array: list) -> list` Returns a new reversed list, leaving the original untouched. ## Convert ### `List::to_uniform(array: list, type: str) -> void` Casts every value to the given type in place — throws if any value can't be cast. --- ## Map Library (Map) Linked to the `map` type. Maps stay ordered by key, which is why `first`/`last`/`at` are meaningful, not just `get`/`insert`. ## Query ### `Map::len(this: map) -> int` ### `Map::empty(this: map) -> bool` ### `Map::any(this: map) -> bool` ### `Map::contains(this: map, key: unknown) -> bool` ### `Map::keys(this: map) -> set` ### `Map::values(this: map) -> list` ## Access ### `Map::get(this: map, key: unknown) -> unknown` ### `Map::at(this: map, index: int) -> (unknown, unknown)` The `(key, value)` pair at a position in sorted order — not a lookup by key. ### `Map::first(this: map) -> (unknown, unknown)` ### `Map::last(this: map) -> (unknown, unknown)` ## Modify ### `Map::insert(this: map, key: unknown, value: unknown) -> unknown` Returns the old value if the key already existed, `null` otherwise. ### `Map::remove(this: map, key: unknown) -> unknown` ### `Map::pop_first(this: map) -> (unknown, unknown)` ### `Map::pop_last(this: map) -> (unknown, unknown)` ### `Map::append(this: map, other: map) -> void` ### `Map::clear(this: map) -> void` --- ## Number Library (Num) Linked to every numeric type — `int`, `float`, and units. Called as a method on the number itself (`val.abs()`) or through `Num` directly (`Num::abs(val)`) — both reach the same function. ## Basics ### `Num::abs(val: int | float) -> int | float` ### `Num::sqrt(val: int | float) -> float` ### `Num::cbrt(val: int | float) -> float` ### `Num::pow(val: int | float, to: int | float = 2) -> float` ### `Num::round(val: int | float, places: int = 0) -> int | float` ### `Num::floor(val: int | float) -> int | float` ### `Num::ceil(val: int | float) -> int | float` ### `Num::trunc(val: int | float) -> int | float` The integer part. ### `Num::fract(val: int | float) -> int | float` The fractional part. ### `Num::signum(val: int | float) -> int | float` `-1` or `1`. ## Trigonometry Angle-returning functions come back with `rad` units automatically. The examples below lean on identities with clean, predictable results. ### `Num::sin(val: int | float) -> float` ### `Num::cos(val: int | float) -> float` ### `Num::tan(val: int | float) -> float` ### `Num::asin(val: int | float) -> rad` ### `Num::acos(val: int | float) -> rad` ### `Num::atan(val: int | float) -> rad` ### `Num::atan2(y: int | float, x: int | float) -> rad` The four-quadrant arctangent — cast to `deg` to read it more naturally. ### `Num::sinh(val: int | float) -> float` ### `Num::cosh(val: int | float) -> float` ### `Num::tanh(val: int | float) -> float` ### `Num::asinh(val: int | float) -> float` ### `Num::acosh(val: int | float) -> float` ### `Num::atanh(val: int | float) -> float` ## Logarithms & Exponents ### `Num::exp(val: int | float) -> float` `e^val`. ### `Num::exp2(val: int | float) -> float` `2^val`. ### `Num::ln(val: int | float) -> float` Natural log. ### `Num::log(val: int | float, base: int | float = 10) -> float` ## Representations ### `Num::to_string(val: int | float) -> str` Same as `Std.str`, but a method on the number itself. ### `Num::bin(val: int) -> str` ### `Num::hex(val: int) -> str` ### `Num::oct(val: int) -> str` ## Units ### `Num::has_units(val: int | float) -> bool` ### `Num::is_length(val: int | float) -> bool` ### `Num::is_mass(val: int | float) -> bool` ### `Num::is_time(val: int | float) -> bool` ### `Num::is_temp(val: int | float) -> bool` ### `Num::is_angle(val: int | float) -> bool` ### `Num::is_memory(val: int | float) -> bool` ### `Num::remove_units(val: int | float) -> int | float` ### `Num::to_units(val: int | float, units: str | float) -> units` Doesn't modify the original. ## Iteration & Misc ### `Num::len(val: int | float) -> int` The number itself — makes a plain number iterable the same way a list is. ### `Num::at(val: int | float, index: int) -> int` Clamps to `val` if `index` is larger. ### `Num::inf(val: int | float) -> bool` ### `Num::nan(val: int | float) -> bool` ### `Num::min(..) -> unknown` Considers units if given. ### `Num::max(..) -> unknown` --- ## Object Library (Obj) Linked to the `obj` type. A lot of this library is already covered in depth elsewhere — Document Graph, Prototypes & Schemas, and Object Run — so those entries stay brief here and link back rather than re-explain; anything new to this page gets the full treatment. ## Navigation Covered in [The Document Graph](../../learn/document-graph). ### `Obj::name(obj: obj) -> str` 0); } `} /> ### `Obj::path(obj: obj) -> str` ### `Obj::id(obj: obj) -> str` 0); } `} /> ### `Obj::parent(obj: obj) -> obj` ### `Obj::root(obj: obj) -> obj` ### `Obj::is_root(obj: obj) -> bool` ### `Obj::children(obj: obj) -> list` ### `Obj::dist(obj: obj, other: obj) -> int` Number of edges separating two objects. ### `Obj::is_parent(obj: obj, other: obj) -> bool` ### `Obj::exists(obj: obj) -> bool` False once dropped — holding a reference afterward doesn't bring it back. ## Fields & Data ### `Obj::len(obj: obj) -> int` Number of fields. ### `Obj::any(obj: obj) -> bool` Has any data attached at all. ### `Obj::empty(obj: obj) -> bool` ### `Obj::contains(obj: obj, name: str) -> bool` ### `Obj::get(obj: obj, name: str) -> unknown` A field's value, a function, or a data pointer — by name. ### `Obj::at(obj: obj, index: int) -> (str, unknown)` The `(name, value)` field at a given index. ### `Obj::fields(obj: obj) -> list` Every `(name, value)` field, as a list. ### `Obj::funcs(obj: obj, attributes: str | list | set = null) -> list` Every function on this object, optionally filtered by attribute. ### `Obj::insert(obj: obj, path: str, value: unknown) -> void` Creates or assigns, using `obj` as the starting context — same as a normal field assignment. ### `Obj::move_field(obj: obj, source: str, dest: str) -> bool` Rename or move a field — like `mv` for a field path. ### `Obj::remove(obj: obj, path: str, shallow: bool = false) -> bool` Same operation as `Std.drop`, from `obj` as context — see Document Memory Management for the full `shallow` behavior. ### `Obj::set(obj: obj, name: str, value: unknown) -> void` Sets or creates the field with exactly this name. Unlike `insert`, the name is never a path, so it can have dots (the counterpart of `get`). A function with the name is an error, and so is a const field. ### `Obj::unset(obj: obj, name: str, shallow: bool = false) -> bool` Removes the field, function, or data with exactly this name, and returns whether there was one. Unlike `remove`, the name is never a path. A field's object value is dropped too, unless `shallow` is true. ### `Obj::attributes(obj: obj, path: str = null) -> map` This object's own attributes, or a specific field/func/object's if `path` is given. ### `Obj::to_map(obj: obj) -> map` ### `Obj::from_map(map: map) -> obj` ### `Obj::from_id(id: str) -> obj` Objects are references, same as data — reconstruct one from its ID. ## Prototypes Covered in depth in [Prototypes & Schemas](../../learn/prototypes-and-schemas). ### `Obj::create_type(obj: obj, typename: str) -> void` Programmatic equivalent of `#[type]`. ### `Obj::set_prototype(obj: obj, proto: obj | str) -> void` ### `Obj::remove_prototype(obj: obj) -> void` ### `Obj::prototype(obj: obj) -> obj` `null` if there isn't one. ### `Obj::instance_of(obj: obj, proto: str | obj) -> bool` ### `Obj::upcast(obj: obj) -> bool` Sets this object's prototype to its current prototype's own prototype — one step up an inheritance chain. ## Structural Operations ### `Obj::move(obj: obj, dest: obj) -> bool` Reparent — `dest` can't be a descendant of `obj` (no detaching a node from its own subtree). ### `Obj::run(obj: obj, prototype: str = null, steps: list = null) -> void` Runs the object's `#[run]` steps, in the order they're written. `prototype` is `'first'` (default), `'last'`, `'none'`, or `'only'`, and `steps` runs just the named steps, in that order. Covered in depth in [Object Run](../../learn/object-run). ### `Obj::schemafy(schema: obj, target: obj, remove_invalid: bool = false, remove_undefined: bool = false) -> bool` Covered in depth in [Prototypes & Schemas](../../learn/prototypes-and-schemas). target_value.len() > 2)] first: 'John' } target: { first: 'aj' } #[main] fn main() { pln(self.schema.schemafy(self.target)); } `} /> ### `Obj::diff(schema: obj, target: obj, symmetric: bool = false) -> void` Removes every field from `target` that matches `schema`, recursively — leaving only what's actually different. `symmetric` also copies over anything unique to `schema`. ### `Obj::dbg_graph() -> void` Dumps the entire graph — useful for debugging, expect dense output. For one specific node instead of the whole graph, `Std::dbg(node)` is the better tool. --- ## Prompt Library (Prompt) Linked to the `prompt` type — a tree of strings, each optionally tagged, that renders to an XML-like format. `Std::prompt(text, tag)` is how you build one to start with. ## Building ### `Prompt::push(prompt: prompt, other: prompt | str) -> void` Adds a sub-prompt. ### `Prompt::insert(prompt: prompt, index: int, other: prompt) -> void` ### `Prompt::replace(prompt: prompt, index: int, other: prompt) -> void` ### `Prompt::remove(prompt: prompt, index: int) -> prompt` ### `Prompt::pop(prompt: prompt) -> prompt` Removes from the end. ### `Prompt::clear(prompt: prompt) -> void` ### `Prompt::reverse(prompt: prompt) -> void` ## Reading ### `Prompt::str(prompt: prompt) -> str` Same conversion a plain `as str` cast would do. ### `Prompt::text(prompt: prompt) -> str` Just this prompt's own text, ignoring its tag and any sub-prompts. ### `Prompt::tag(prompt: prompt) -> str` ### `Prompt::set_text(prompt: prompt, text: str) -> void` ### `Prompt::set_tag(prompt: prompt, tag: str) -> void` `null` clears the tag entirely. ## Structure ### `Prompt::len(prompt: prompt) -> int` The number of sub-prompts — not the text length. ### `Prompt::empty(prompt: prompt) -> bool` ### `Prompt::any(prompt: prompt) -> bool` ### `Prompt::at(prompt: prompt, index: int) -> prompt` `p[0]` works the same way. ### `Prompt::prompts(prompt: prompt) -> list` Every direct sub-prompt, as a list. --- ## Set Library (Set) Linked to the `set` type. Ordered, and every value is unique — Types & Units and Collections both touch on this; this page is the complete reference. ## Query ### `Set::len(set: set) -> int` ### `Set::empty(set: set) -> bool` ### `Set::any(set: set) -> bool` ### `Set::contains(set: set, val: unknown) -> bool` ### `Set::is_uniform(set: set) -> bool` ## Access ### `Set::at(set: set, index: int) -> unknown` The element at a position in sorted order — `null` if out of bounds. ### `Set::first(set: set) -> unknown` The minimum value. ### `Set::last(set: set) -> unknown` The maximum value. ## Modify ### `Set::insert(set: set, val: unknown) -> bool` Returns whether the value was newly inserted — `false` if it was already there. ### `Set::remove(set: set, val: unknown) -> unknown` ### `Set::pop_first(set: set) -> unknown` ### `Set::pop_last(set: set) -> unknown` ### `Set::append(set: set, other: set) -> void` ### `Set::clear(set: set) -> void` ### `Set::split(set: set, val: unknown) -> (set, set)` Splits into everything smaller and everything larger than `val` — `val` itself isn't included in either half. ### `Set::to_uniform(set: set, type: str) -> void` Casts every value to a single type in place. ## Set Algebra ### `Set::union(set: set, other: set) -> set` ### `Set::intersection(set: set, other: set) -> set` ### `Set::difference(set: set, other: set) -> set` Everything in `set` that isn't in `other`. ### `Set::symmetric_difference(set: set, other: set) -> set` Everything in exactly one of the two sets, not both. ### `Set::disjoint(set: set, other: set) -> bool` True if the two sets share nothing. ### `Set::subset(set: set, other: set) -> bool` True if every value in `set` is also in `other`. ### `Set::superset(set: set, other: set) -> bool` True if every value in `other` is also in `set` — the reverse of `subset`. --- ## String Library (Str) Linked to the `str` type. ## Query ### `Str::len(val: str) -> int` ### `Str::contains(val: str, seq: str) -> bool` ### `Str::starts_with(val: str, seq: str) -> bool` ### `Str::ends_with(val: str, seq: str) -> bool` ### `Str::index_of(val: str, seq: str) -> int` `-1` if not found. ## Access ### `Str::at(val: str, index: int) -> str` Clamps to the last character if the index is out of bounds, rather than erroring. ### `Str::first(val: str) -> str` ### `Str::last(val: str) -> str` ### `Str::substring(val: str, start: int = 0, end: int = -1) -> str` `[start, end)` — default is the whole string. ## Case & Whitespace ### `Str::upper(val: str) -> str` ### `Str::lower(val: str) -> str` ### `Str::trim(val: str) -> str` Strips newlines, tabs, and spaces from both ends. ### `Str::trim_start(val: str) -> str` ### `Str::trim_end(val: str) -> str` ## Modify ### `Str::push(val: str, other: str) -> void` Mutates `val` in place — `other` is left untouched. ### `Str::replace(val: str, find: str, replace: str = "") -> str` Returns a new string — the original is unmodified. Default `replace` just removes every occurrence of `find`. ### `Str::split(val: str, sep: str = " ") -> list` ## Regex ### `Str::matches(val: str, regex: str) -> bool` ### `Str::find_matches(val: str, regex: str) -> list` Every match, as `(content, start, end)` tuples. --- ## Tuple Library (Tup) Linked to the tuple type — deliberately small, since a tuple's whole point is a fixed-length group of values, not a resizable collection. ## `Tup::at(tup: (..), index: int) -> unknown` Indexed access — `&tup[i]` also works, for a reference into the tuple. ## `Tup::len(tup: (..)) -> int` --- ## Semantic Version Library (Ver) `ver` is a base type in Stof — every field below operates on a version like `1.2.3-release+build` directly, no library prefix required on the value itself. ## Reading Components ### `Ver::major(ver: ver) -> int` ### `Ver::minor(ver: ver) -> int` ### `Ver::patch(ver: ver) -> int` ### `Ver::release(ver: ver) -> str` ### `Ver::build(ver: ver) -> str` ## Setting Components Every setter below mutates the version in place. ### `Ver::set_major(ver: ver, val: int) -> void` ### `Ver::set_minor(ver: ver, val: int) -> void` ### `Ver::set_patch(ver: ver, val: int) -> void` ### `Ver::set_release(ver: ver, val: str) -> void` ### `Ver::set_build(ver: ver, val: str) -> void` ### `Ver::clear_release(ver: ver) -> void` ### `Ver::clear_build(ver: ver) -> void` --- ## We Deserve Better Than JSON as a DSL It always starts simply. You need a config, or an endpoint with a little dynamic behavior, and you decide to keep it simple this time. What's simpler than JSON? Some keys, some nesting, done. Six months later you're hunting for a `$ref`, writing helper scripts to generate the file, and explaining to a new hire which strings are secretly expressions. {/* truncate */} Every team does this. GitHub Actions is YAML with an expression language inside it. Terraform invented HCL because JSON wasn't enough. OpenAPI is JSON Schema with extensions on top. Every AI framework has its own JSON format for tool definitions, each slightly different. JSON, YAML, and TOML are fine at what they do. The trouble starts when we ask them to carry logic, then build more and more scaffolding around them when they can't. ## Data That Can Do What It Needs To That's why I built [Stof](/what-is-stof): data that carries its own logic, and runs it safely. ```typescript const doc = await stofAsync` name: 'Stof' fn loaded() -> str { const source = await Ext::fetch(); parse(source, self); self.say_hello() } `; // your app decides what the document can reach doc.lib('Ext', 'fetch', async () => ` fn say_hello() -> str { 'Hello, ' + (self.name ?? 'World') + '!' } `, true); console.log(await doc.call('loaded')); // Hello, Stof! ``` The document starts without a `say_hello` function. It gets more Stof from somewhere (an API, another service, a model), parses it into itself, and calls the function that just arrived. A Stof document can't touch your files, your network, or your app unless you give it a way, like `Ext.fetch` above. So logic can arrive while your app runs, from somewhere you don't fully control, and still only do what you allowed. A service can share what it can do as Stof: not a description of the logic, the logic itself. No client library, no SDK, no redeploy. ## Works With What You Already Have Stof is a superset of JSON, so your data is already valid Stof. On top of it you get types, units, functions, and validation, as part of the document instead of a layer around it. It doesn't replace your formats. It sits between them: parse JSON, YAML, TOML, or Stof into one document, add the logic that belongs with the data, and export to whatever your app expects. ```typescript const doc = await stofAsync` #[type] Server: { int port: 8080 str host: 'localhost' bool secure: false MiB memory: 512MiB fn url() -> str { (self.secure ? 'https://' : 'http://') + self.host + ':' + self.port } } `; // JSON, typed as a Server doc.parse(`Server prod: { "host": "prod.example.com", "port": 443, "secure": true, "memory": "2GB" }`); await doc.call('prod.url'); // 'https://prod.example.com:443' doc.get('prod.memory'); // 1907.3486328125 (2GB in MiB) doc.stringify('toml', doc.get('prod') as string); // host = "prod.example.com" // port = 443 // secure = true // memory = 1907.3486328125 ``` The `Server` type gives the shape, the defaults, and the behavior. Data that comes in from anywhere and takes that type gets all of it. ## Rules That Can't Drift Worse than writing a JSON Schema is keeping it in sync with the data it checks. Here's one for a server config: ```json { "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "port": { "type": "integer", "exclusiveMinimum": 1024, "maximum": 65535 }, "address": { "type": "string", "minLength": 1 }, "memory": { "type": "string", "description": "Memory in MB, must be at least 256" } }, "required": ["address"] } ``` It's a separate file in a separate format. And look at `memory`: JSON Schema has no units, so the best it can do is a description and a hope that someone reads it. The real check lives in your app code. In Stof, the rule sits on the field it protects. This part is plain Stof, so you can run it here: target_val != null && target_val > 1024 && target_val <= 65535)] int port: 8080 #[schema((target_val: str): bool => target_val != null && target_val.len() > 0)] str address: 'localhost' #[schema((target_val: MiB): bool => target_val != null && target_val >= 256MB)] MiB memory: 2GB } #[main] fn main() { const good = new { port: 8080, address: 'prod.example.com', memory: '2GB' }; pln(.schemafy(good)); const bad = new { port: 80, address: 'prod.example.com', memory: '100MB' }; pln(.schemafy(bad)); } `} /> The memory rule means something because Stof knows units: `'2GB'` converts and passes, `'100MB'` fails. The rule can't drift from the data because it's in the data. [A Self-Validating Config](/cookbook/config) builds this out into a full recipe. ## In Production [Limitr](https://limitr.dev) runs on Stof. Every pricing policy (plans, credits, limits, and the rules that enforce them) is one Stof document, run in-process on every request. It's a good look at "data that carries its own logic" past the toy stage. ## Try It The [playground](/playground) runs Stof in your browser, with no install, and every page in the [docs](/what-is-stof) has runnable examples. ```bash npm i @formata/stof # TypeScript / JavaScript pip install stof # Python cargo install stof-cli # CLI ``` - [GitHub](https://github.com/dev-formata-io/stof): source and issues - [Discord](https://discord.gg/Up5kxdeXZt): tell us what you're building - [VS Code extension](https://marketplace.visualstudio.com/items?itemName=Formata.stof): highlighting, errors as you type, and go to definition Stof is open source under Apache 2.0. --- ## The Origin of Stof: From CAD Software to a Data Runtime Stof wasn't designed as a product. It came out of about a decade of building parametric and geometry formats for CAD and graphics systems at Siemens, Lockheed Martin, Configura, and Anark. On the surface, that work has nothing to do with web APIs or config files. Underneath, it has everything to do with them. {/* truncate */} ## What CAD Teaches You About Data A parametric CAD model isn't a static shape. It's a set of constraints and relationships (this face stays perpendicular to that one, this length depends on that parameter) that the software recomputes constantly as a designer drags things around. The geometry comes out of the logic. A model that couldn't recompute its own constraints wouldn't be a CAD model. It would be a picture of one. Build systems like that long enough, and you stop thinking of data and the logic that works on it as separate things. They never were, in the systems I worked on. The usual split, a data file here and an app that interprets it over there, started to look less like a law of nature and more like a habit that came from which tools got popular for which jobs. The idea that came out of that work: store everything as a flat graph of objects and data, connected by references instead of nested copies. An object isn't a deep tree you copy to move. It's an entry in a list, and its relationships are pointers that are cheap to change. Once geometry, constraints, and the functions that maintain them are all data on the same graph, moving or transforming any part of the document isn't special or expensive. It's updating a pointer. ## Trying to Send WebAssembly Over the Wire Stof started as a solo experiment. I was trying to send WebAssembly over the wire, and kept hitting the same wall. Wasm binaries are big, they come with tooling and dependencies, and whatever carried them had no good way to also carry the data that made them useful when they arrived. Every option meant picking a side: either inert data that the receiver already had to understand, or executable code with nowhere natural to keep the structured data it worked on. The flat graph from the CAD work made a different answer feel obvious. If a function is just another kind of data on an object, no different from a field except that it can be called, then one document can carry both without bolting one onto the other. Fields, functions, and later richer data like images and PDFs all became the same kind of thing on the graph. Let functions change the graph they're attached to, sandbox what else they can reach, and the rest of Stof follows. It's a superset of JSON because JSON was already the shape of plain data, and adding logic didn't require breaking that. ## From Experiment to Production Stof now runs in production as the engine underneath [Limitr](https://limitr.dev). Every plan, credit limit, and rule there is a live Stof document, not a config file that Limitr's code has to interpret separately. It's the same idea as the CAD work, aimed at pricing instead of geometry: the rule and the data it governs are the same object, so they can't drift apart the way a schema and the data it checks so often do. None of this started as a thesis about data and logic belonging together. It came from a decade of building systems where that was already true, and noticing that most software, for understandable reasons, ended up somewhere else. For where this is headed in more concrete terms, start with [What Is a Data Runtime?](/blog/what-is-a-data-runtime) --- ## What Is a Data Runtime? A **data runtime** runs data directly, the way a JavaScript runtime runs JavaScript. Instead of treating data as an inert value that some other program loads and interprets, it runs documents that carry their own logic: fields, types, and functions together, sandboxed, and the same on every platform that embeds it. [Stof](/what-is-stof) is a data runtime. It's a superset of JSON where data can validate itself, transform itself, and act. {/* truncate */} That's the definition. The rest of this post is what it means in practice, and why it needed a name. ## Not a Format, a Language, or a Database It helps to say what a data runtime isn't, since its three nearest neighbors each miss something. **It's not a data format.** JSON, YAML, and TOML describe a snapshot. It means nothing on its own until some program decides what to do with it. A Stof document is still data (any JSON is valid Stof), but it doesn't stop there. **It's not a general-purpose language.** There's no compile step, no executable, and no program apart from the data. The logic exists to serve the data it sits next to. **It's not a database.** There's no query engine and no storage layer. A document is one self-contained unit, closer to an object with methods than a table with rows. ## What Actually Runs Any JSON is already a Stof document: ```json { "host": "0.0.0.0", "port": 8080 } ``` Add types and a function, and it's still the same document: str { \`http://\${self.host}:\${self.port}\` } #[main] fn main() { pln(self.url()); } `} /> `url()` isn't a separate script that knows how to read this config. It's part of the document. Your app calls it with `doc.call('url')` and gets a real value, computed from data that sits right next to it. ## Why It Needed a Name Every interchange format describes a snapshot. Once it crosses a wire, it's inert on the other side, and whatever receives it has to already know what to do with it. That's where schema drift comes from: a rule in a separate file or service that quietly stops matching the data it describes. It's where "the API changed and nobody told the client" comes from. And it's why so many systems end up bolting a DSL onto JSON or YAML: Terraform invented HCL, GitHub Actions grew an expression language inside YAML, OpenAPI piled extensions onto JSON Schema. Those projects weren't wrong to need more than plain data. They didn't have a runtime built to hold both halves, so each built one for its own tool. A data runtime is the general version of that, with three properties: - **Sandboxed by default**, so logic from somewhere you don't control, like a model, a customer, or another service, is safe to run. - **Portable**, so a document behaves the same natively, in WebAssembly, in a browser, or embedded in Python. - **Able to grow**, so a document can take in new fields, types, and functions while it runs, instead of being a fixed shape you redeploy to change. That first property matters more every month. Models write more and more of the logic in our systems, and a data runtime is a safe place to run it. ## FAQ **Is a data runtime a database?** No. A database stores and queries many records. A data runtime document is one self-contained unit of data and logic, with no query engine or storage of its own. **How is it different from a templating engine or a DSL?** A templating engine or DSL is a separate tool that reads a document and produces something. In a data runtime, the document is the thing that runs. The logic lives inside it. **Is it safe to run a document from an untrusted source?** That's what the sandbox is for. A Stof document can only reach what the app running it provides: no files, network, or environment unless the app turns them on, and every call has a time limit. See [Safe by Default](/playground?example=safe-by-default) in the playground. **Do I have to migrate my data?** No. Any JSON is already a Stof document. Using Stof is a decision to add logic where you need it, not a migration. --- ## The Problem With JSON Schema (And What We Did Differently) JSON Schema's real limitation isn't its syntax. It's that a schema is always a second document describing a first one, and nothing keeps the two in agreement. Add a field to your data, forget the schema, and nothing tells you. Stof takes a different approach: `#[schema(...)]` puts the rule on the field it protects, in the same document. There's no second file to fall out of sync. {/* truncate */} ## Where JSON Schema Struggles **It's kept in sync by hand.** The schema and the data live in different places, maintained by whoever remembers to update both. Nothing enforces the connection except discipline, which is the first thing to go under a deadline. **It has no units.** "Memory in MB, at least 256" ends up as a string with a description. JSON Schema can check that a value is a number, but it can't know that `"2GB"` and `2000` MB are the same thing, or that a value in KB doesn't belong where MB was meant. **Rules across fields are awkward.** Checking one field against another takes `$data` references or vendor extensions that most tools only partly support. Something that's one line of real code turns into a fight with the format. **The checks get written twice anyway.** Most teams still write the same validation in app code: for better error messages, for logic the schema can't express, or because it's faster than fighting the format. None of this means JSON Schema is badly designed. It validates the shape of any JSON, in a language-neutral way that tools understand. The problem is structural: validation lives somewhere other than the data. ## Validation as Part of the Data In Stof, a rule is an attribute on the field it checks. Press **Run**: target_val != null && target_val.len() > 0)] str label: 'Growth' #[schema((target_val: hr): bool => target_val != null && target_val > 0hr)] hr reset_every: 1hr } #[main] fn main() { pln(.schemafy(new { label: 'Pro', reset_every: 60min })); pln(.schemafy(new { label: '', reset_every: 0s })); } `} /> `schemafy` checks an object against every rule on the type. The unit-typed field means something: `1hr` and `60min` are the same value as far as the rule is concerned. ## More Than One Rule A list of rules runs in order, and every one has to pass: ```stof fn not_empty(target_val: str) -> bool { target_val != null && target_val.len() > 0 } #[type] Contact: { #[schema([self.not_empty, (target_val: str): bool => target_val.contains('@')])] str email: '' } ``` A bare `#[schema]` on a field that holds an object checks that object against the rules inside it, so nested data is validated without writing the traversal yourself. And `schemafy` can clean data instead of only rejecting it. `remove_invalid = true` drops fields that fail their rules, and `remove_undefined = true` drops fields the type doesn't declare: target_val != null && target_val.len() > 0)] str name: '' #[schema((target_val: str): bool => target_val != null && target_val.len() <= 12)] str nickname: '' } #[main] fn main() { const input = new { name: 'Ada', nickname: 'a nickname far too long', admin: true }; .schemafy(input, remove_invalid = true, remove_undefined = true); pln(stringify('json', input)); } `} /> None of this needs a second file, a second syntax, or a build step to keep generated types in sync. It's the same document doing one more thing. [A Self-Validating Config](/cookbook/config) builds a complete example, and [Prototypes & Schemas](/learn/prototypes-and-schemas) is the full reference. ## FAQ **Does Stof replace JSON Schema?** For Stof documents, yes: `#[schema]` is built in. For checking plain JSON in a system that doesn't use Stof, JSON Schema is still a fine tool for that narrower job. **Can a Stof schema check data that didn't start as Stof?** Yes. `schemafy` works on any object, including one parsed from JSON, YAML, TOML, or form data. It doesn't need to have been created from the type. **What happens when validation fails?** `schemafy` returns `false` instead of throwing, so a failure is a value you check. With `remove_invalid`, it can also clean the object, and comparing before and after tells you which fields failed. **Is this like Zod, Yup, or Pydantic?** Same goal: the rule next to the type, not in a separate file. But those check data at the edge of one language's app. Stof's rules live in the document, so they run the same in TypeScript, Python, Rust, or a browser. --- ## How to Safely Give AI Agents New Tools at Runtime Most systems treat "let an agent gain new tools while it runs" and "stop an agent from doing damage" as two problems with two solutions: a tool registry plus an approval workflow, or a plugin marketplace plus a permission system added afterward. In Stof, they're the same mechanism. A document can only reach what it has been given, so a tool that arrives at runtime is limited the same way everything else in the document already is. Not because a permission check ran, but because there's nothing else there to reach. {/* truncate */} ## The Real Fear An agent that can pick up new tools at runtime, from an MCP server, a plugin, or another service, is powerful in exactly the way that should make you nervous. The fear is specific: an agent gets a "check the weather" tool, and that tool also reads environment variables, or sends data somewhere it shouldn't. What a tool says it does and what it can do are two different things, and most systems can't guarantee they match. The usual fix is a permission system on top of the tool loading: scopes, an approval queue, a sandbox that has to be set up correctly for each new tool. That's a second system to build, and a second system to get wrong. ## Tools Are Just Functions In Stof, a tool is a function with an attribute. An agent finds its own tools by asking which functions have it, so adding a capability is adding a function, and removing one is removing it. A good tool and a bad one arrive the same way, and what happens next depends only on what the document can reach. Press **Run**: str { \`order \${id}: shipped\` } /// The prompt: the role, and every tool the agent has right now. fn render() -> prompt { const context = prompt(tag = 'agent'); context.push(prompt(self.role, 'role')); for (const tool in self.funcs('tool')) { context.push(prompt(tool.name(), 'tool')); } context } } #[main] fn main() { // A new tool arrives as text: from an MCP server, an API, or the model itself. parse("#[tool] fn issue_refund(id: str, amount: float) -> str { \`refunded \${amount} on \${id}\` }", self.Agent); pln(self.Agent.render() as str); pln(self.Agent.issue_refund('ord_7', 20)); // A tool that tries to reach the network arrives the same way. parse("#[tool] fn exfiltrate() -> str { await Http::fetch('https://evil.example') }", self.Agent); try { self.Agent.exfiltrate(); } catch { pln('blocked: this document has no network'); } // Removing a tool is removing a function. self.Agent.remove('issue_refund'); self.Agent.remove('exfiltrate'); pln(self.Agent.funcs('tool').len()); } `} /> `issue_refund` works right away: real code doing real work, with nothing special needed to accept it. `exfiltrate` fails, and not because anything inspected it and decided it looked suspicious. It fails because this document has no `Http` library. The failure is an ordinary error you can catch, not a security incident. There was simply nothing for that function to call. Removing a tool is one line, whatever the reason, and there's no separate registry to keep in sync with what the agent actually has. The document is the source of truth. ## Giving Tools Real Power, Carefully Tools that do something useful need to reach your systems. Your app gives the document specific functions, scoped to the current customer or request, and puts its limits inside them: ```typescript doc.lib('App', 'refund', async (orderId: string, amount: number) => { if (amount > 100) throw new Error('refunds over $100 need a person'); return payments.refund(currentCustomer, orderId, amount); }, true); ``` Now a tool can call `await App::refund(id, amount)`, and so can any tool that arrives later, but only within that limit, and only for that customer. If a tool needs the network, `doc.allowHttp(['api.example.com'])` opens it to the hosts you name and no others. Every call also has a time limit, so a tool stuck in a loop fails on its own. ## Try It [Agent Tools at Runtime](/cookbook/agent-tools) builds the full version: tool definitions generated from the functions for any model API, a conversation loop, and an agent that writes and checks its own tools. You can also run it in the [playground](/playground?example=rw-agent-tools). [Rules Your Customers Write](/cookbook/customer-rules) uses the same idea for logic from customers instead of models. ## FAQ **Is this the same as an MCP server's permissions?** No, and they don't compete. MCP and similar protocols define how a client finds and calls tools. This is about what a tool's code can do once it runs. Use both: MCP for discovery and transport, Stof for what a tool is allowed to touch. **Does this replace API keys or OAuth scopes?** No. Those decide who can call a tool. This decides what a tool can do once it's running, however well authenticated the request to load it was. **What if a tool needs network access or other real capabilities?** Your app grants exactly that, with `doc.lib()` for its own functions or `doc.allowHttp()` for specific hosts. Nothing is out of reach forever, only by default. **Can an agent's tools shrink as well as grow?** Yes. Remove the function and the tool is gone, immediately. Nothing else tracks what the agent is supposed to have. --- ## The Part of Your App You Can Hand to Something You Don't Trust Sometime in the last year, "the agent will just edit the file" stopped being hypothetical and became a normal workflow. Point an agent at a config, a document, or a blob of state, and it reads, edits, and writes it back, with no UI in between. That's useful. It's also a trust problem most formats were never built to handle. {/* truncate */} ## Two Questions Most Formats Can't Answer When something you don't fully control (an agent, a plugin, logic from another service) edits your app's state directly, two questions come up right away: 1. **Can it leave the data broken?** Most formats are only structure. JSON doesn't know what a valid version of itself looks like. Validation lives somewhere else: a schema file, a Zod or Pydantic model, a check your app runs afterward. The data has no opinion. 2. **Can it reach further than it should?** If the edit includes logic, like a new rule or a new function, what stops that logic from touching the file system, the network, or anything else once it runs? Most systems built around "one portable file, edited by anything" answer both by trusting more, not less: bundle the whole app, sign the release, keep a rollback plan. That works. It's also a lot of machinery just to make an edit safe to accept. ## A Narrower Answer Stof goes the other way. Instead of trusting the whole bundle, it makes the data the part that can't misbehave. A Stof document [validates itself](/learn/prototypes-and-schemas). Fields carry their types and rules, so there's no separate schema to drift: target_val != null && target_val.len() > 1)] str name: 'Ada' #[schema((target_val: int): bool => target_val != null && target_val >= 0)] int age: 30 fn greet() -> str { \`Hello, \${self.name}!\` } } #[main] fn main() { // an edit from an agent: checked before it's accepted const edit = new { name: 'Grace', age: -4 }; pln(.schemafy(edit)); } `} /> And it's sandboxed by default. A document reaches only what your app gives it: ```typescript const doc = await StofDoc.new(state); // nothing outside the document is reachable until you allow it doc.allowHttp(['api.example.com']); // this host, and no other doc.lib('App', 'notify', (user: string, message: string) => notify(user, message)); doc.setMaxExecutionTime(50); // and every call has a time limit // logic from an agent goes in, and runs with exactly those limits doc.parse(logicFromTheAgent); await doc.run('on_update'); ``` If network access was never granted, a function in that document can't reach the network. Not because a permission check ran, but because there's nothing to call. Logic from an agent, a tool result, or another service can be parsed into a running document and called right away, and the damage it could do is limited by what you gave it, not by how much you trust where it came from. ## Not the Container, the Part Inside It You Can Trust It's tempting to read "portable, sandboxed, single document" and reach for something bigger: what if the whole app lived in a Stof file, UI and all? We don't think that's the right shape. The moment a document has to draw pixels or drive a UI, it needs broad, standing access to do its job, and you're back to trusting the whole bundle, the problem the sandbox was meant to avoid. The useful version is narrower. **The UI stays outside, doing what it's good at. The state, the rules, and what's allowed to happen live in a Stof document, because that's the part of the system that's safe to hand to something you don't fully trust.** An agent can edit it. A plugin can extend it while it runs. Neither requires trusting the editor, only that the document can't be made invalid and can't reach further than you allowed. More and more of an app's state is edited by something other than the app. The question isn't how to make the whole app safe to hand over. It's which part should be. [Run Rules a Model Writes](/cookbook/ai-rules) walks through the full loop: a model writes logic, the app checks it, and only then does it run. --- ## The Heap Is Just a Document: How Stof Actually Works Every document on your computer is a pile of bytes. A photo, a spreadsheet, a JSON file: underneath, it's all bytes sitting next to other bytes. Here's the idea behind Stof. What if some of those bytes were instructions? Small pieces that say "add one to that number over there" or "join those two words into a name." Not code stored somewhere else that reads the document, but instructions inside the document, right next to the data they're allowed to change. That's a Stof document: data, and the instructions for changing it, in the same pile of bytes. {/* truncate */} ## You've Already Used This If that sounds strange, it isn't. You've used a version of it for years: a spreadsheet. Put a formula in cell B2, and it can read and change other cells in the same sheet. It can't reach into another spreadsheet on your computer, and it can't touch your files. It only knows about the sheet it lives in. Nobody finds that surprising. Stof takes the same idea, instructions that live in the data and only affect the data around them, and applies it to documents in general, not just grids of numbers. ## Same Idea, Bigger Format Here's a plain JSON object: ```json { "name": { "first": "Bob", "last": "Jones" } } ``` Any JSON document is already a valid Stof document. Nothing to convert. But Stof lets you add instructions right next to the data: self.name.first + ' ' + self.name.last } #[main] fn main() { pln(self.full_name()); }`} /> `full_name` isn't stored somewhere else and pointed at this data. It's part of the document, the way a formula is part of the sheet. ## It's a Real Runtime Stof is a runtime, the same kind of thing as a JavaScript engine or a Python interpreter. Something has to read those instructions and carry them out, and that's what the Stof runtime does. Here's what's different. When JavaScript or Python run a program, they keep working memory (the heap) where the program's data lives while it runs. That memory belongs to that one running program, on that one machine. When the program ends, or you want to hand its data to something else, the memory is gone. What's left is a snapshot: a JSON file or a database row that some other program has to already understand. Stof doesn't work that way. Its memory is the document. There's no private place where the real data lives while the document sits there as a copy. The document is the memory the runtime uses, whether it's on your disk, in the middle of a calculation, or arriving somewhere else. ## You Can Send It Anywhere Because the memory is a document, you can do document things with it: save it, send it to another server, store it in a database. And when it arrives, it isn't inert. It's not a snapshot waiting for the other side to have the right code installed. It's the same document it was on your machine, instructions included, so it can pick up where it left off, check itself, or run whatever it needs to. ## Why It Stays Safe Remember that a formula can only touch its own sheet? Same rule here. A function in a Stof document can only read and change that document. It can't reach your files, your database, or another document next to it, unless the app running it hands it a way to. That isn't a feature added on top. It's what happens when the instructions and the data they change are the same pile of bytes: there's nothing outside it for the instructions to reach. ## A Small Example Say a customer has a credit limit, and a function that checks whether a new charge is allowed: bool { self.used + amount <= self.credit_limit } #[main] fn main() { pln(self.can_charge(100)); pln(self.can_charge(200)); }`} /> That document can be created on one server, sent across the network, and land on another machine. Nobody sets anything up on the receiving end. When it arrives, `can_charge` works exactly as it did before, checking only the numbers in its own document. ## Why It's Useful Documents grow and change as they move around, and Stof documents are no different. The part that defines an API or the validation rules can start separate from the part that holds working data, and the two can be combined or split as needed. Stof also works with other formats (JSON, YAML, TOML, form data, and more), so data from many places, and the logic that makes sense of it, can come together in one document, get worked on, and go back out in whatever format you need: bool { self.working.used + amount <= self.working.credit_limit } #[main] fn main() { self.load(); if (self.can_charge(100)) self.working.used += 100; pln(self.can_charge(100)); // ready for your app, in the format it wants pln(stringify('yaml', self.working)); }`} /> ## The Short Version Other runtimes run code that happens to use some data. A Stof document carries its own instructions, and running it is reading and writing the document itself. That's the whole idea. Everything else (the graph underneath, the sandbox, running the same in Rust, Python, or a browser) is built on that one shift: a program's memory isn't private anymore. It's a document. For the details of the graph, see [How Stof Works](/start/how-it-works) and [The Document Graph](/learn/document-graph). --- ## Stof 0.10: Safe by Default, Errors You Can Read, and a Language Server Stof 0.10 is out. It's the biggest release since Stof started, and almost all of it serves one goal: making it safe and easy to run logic you didn't write, whether it came from a model, a customer, or another team. That meant tightening the sandbox, making every error something a person or a model can read and fix, closing off behaviors that quietly produced wrong answers, and giving you better tools to write Stof yourself. This post covers 0.10.0 through 0.10.8, along with stof-cli 0.13 and version 0.3 of the VS Code extension. It was updated on October 20 for 0.10.7 and 0.10.8. {/* truncate */} ```bash npm i @formata/stof@latest # TypeScript / JavaScript pip install -U stof # Python cargo add stof # Rust cargo install stof-cli # CLI, including the language server ``` Most documents run unchanged, but a few fixes are breaking. Read [Breaking Changes](#breaking-changes) before you upgrade. ## Sandboxed by Default Before 0.10, an embedded document in Rust or Python could read and write files, read environment variables, and make network requests unless the host removed those libraries. That's the wrong default for logic you don't fully trust. Now an embedded document reaches nothing outside itself unless your app opts in: ```typescript const doc = await StofDoc.new(rulesFromAModel); doc.allowHttp(['api.example.com']); // the Http library, for these hosts only (0.10.4) doc.setMaxExecutionTime(50); // every call fails after 50ms (0.10.4) ``` - **Files and environment** are off until `allow_system()` (Rust, Python). - **Network** is off until `allow_http()` (Rust, Python) or `allowHttp()` (JavaScript). In JavaScript, `allowHttp()` also gives documents `Http.fetch`, backed by the platform's fetch, so you don't need your own. - **An allowlist** limits requests to the hosts you name. A request to any other host, or a redirect to one, fails. - **Time limits** are now yours to set. The default is still 2 minutes, and a runaway loop fails that one call and nothing else. The `stof` CLI turns everything on, since it runs documents on your own machine. See [Using Stof in Your App](/start/using-stof#what-a-document-can-reach). ## Errors You Can Read Errors were the most common complaint about Stof, and they're the thing that matters most when a model writes the code: an error it can read is an error it can fix. Runtime errors now say what went wrong in plain language, show the line, and give the Stof call stack: ```text error: cannot multiply null: the right value is null or missing (use ?? to give a default) --> pricing.stof:4:9 | 4 | units * self.rat | ^ at root.plan.total (pricing.stof:4:9) at root.main (pricing.stof:10:5) ``` Parse errors show the file, line, and column with a caret. Argument errors say exactly what's wrong: `missing argument 'units' for total(units: int, rate?: float)`. And names that can't be anything are caught when the document is parsed, before they turn into a confusing null: ```text warning: unknown name 'team' in fn root.main (main.stof:4:9): not a parameter, variable, or root, so it is always null (use self.team for a field) ``` Your app gets the same messages and call stacks from `call` and `run`, in every language. ## Fewer Silent Wrong Answers A few long-standing behaviors gave wrong results without complaint. They're now fixed or reported: - **Math with null is an error.** `3 * null` used to be `3`. Now it throws, so a misspelled field can't become a wrong price. Use `??` for defaults. - **Functions have their own scope.** A function could read its caller's variables, so a typo could quietly bind to the wrong value. Now named functions see only their parameters and their own variables. Arrow functions still see the function they're written in. - **Paths start at a root.** `team.name` used to find any object named `team` anywhere in the document. Now it means the root named `team`, and the warning above catches the mistake. - **Numbers behave.** Number methods return a new value instead of changing the original. Integer overflow becomes a float instead of wrapping negative, division by zero gives infinity instead of crashing, and unit conversions use exact factors. - **`typeof` binds tightly** (0.10.2). `typeof x == 'obj'` is now `(typeof x) == 'obj'`, as you'd expect. ## Temporary Objects with `using` Objects in a Stof document stay until something removes them. That's fine for a document you run once, but a policy that runs on every request for months can pile up temporary objects. 0.10.3 adds `using`: declare a temporary object with it, and it's dropped when its block or function ends, however it ends: bool { using order = new { total: total, tax: total * 0.2 } on self.orders; order.total + order.tax < 1000 } #[main] fn main() { pln(self.check(500)); pln(self.check(900)); pln(self.orders.children().len()); // 0: both were dropped } `} /> `self.handle(using new { .. })` does the same for an object passed straight into a call. And `stof test --leaks` runs each test on its own and fails any test that leaves objects behind, saying where they are. See [Memory Management](/learn/memory-management). ## Destructuring 0.10.5 adds destructuring, which a data language really should have. Take values apart into variables in one step, from objects, maps, lists, and tuples, and in loops: See [Variables & References](/learn/variables-and-references#destructuring). ## Workflows From Types `obj.run()` runs an object's `#[run]` steps, and in 0.10.6 it works the way you'd expect with types. Steps run in the order they're written. A type's steps run on every instance, each step once. An instance can replace a step by giving one the same name, remove one with `null`, or add its own, and a number moves any step before or after the rest: `run(steps = ['check'])` runs just the steps you name. See [Object Run](/learn/object-run). ## HTML and XML 0.10.7 adds `html` and `xml` formats, so a document can become a page, a component tree, or an XML file. An object is an element, and its fields, in order, are what's inside it. `#[tag]` names an element, `#[attr]` makes a field an attribute, and `#[text]` writes a value as text: The HTML is safe to put in a page, whatever the document holds, including a view a model wrote. Text and attribute values are always escaped, scripts, styles, frames, event handlers, and `javascript:` URLs are never written, and a host can allow exactly the elements it renders with `doc.allowHtmlTags([..])`. Functions marked `#[attr]` or `#[text]` are called when the object is exported, async ones included, so part of a view can be computed. XML parses back too: `parse(src, obj, 'xml')`. See [Formats](/learn/formats#markup-html-and-xml). ## Editing Stof Text in Place Exporting a document again rewrites all of it. When people also edit the text by hand (Ex. a settings page saving into a hand-written policy), 0.10.8 can change just what you name, by path, and leave everything else exactly as it was: comments, formatting, and every other definition. ```typescript const text = `// Pricing for the Pro plan pro: { price: 20 // per seat seats: 5 }`; const next = StofDoc.edit(text, [ { set: 'pro.price', value: 24 }, // a JSON value { set: 'pro.storage', stof: '50GB' }, // Stof source, added in the object's style ]); // // Pricing for the Pro plan // pro: { // price: 24 // per seat // seats: 5 // storage: 50GB // } ``` `set_fn`, `remove`, and `rename` work the same way, and `StofDoc.outline(text)` lists every field and function with its path and source. Nothing in the text runs while it's outlined or edited, so it's safe to use on text a user or a model sent. It's in Python (`Doc.edit`) and Rust (`stof::parser::edit`) too. See [Using Stof in Your App](/start/using-stof#editing-stof-text-in-place). ## Readable Stof Export `stringify('stof', obj)` now writes Stof a person or a model can read and edit, instead of the exact graph form: objects with their types and attributes, values as literals (`50GB`, `set(..)`, `map((k, v))`), and functions as their source: float { seats * self.price } } Plan pro: { name: 'Pro' price: 20 storage: 50GB features: set('sso', 'audit') } #[main] fn main() { pln(stringify('stof', self)); } `} /> Keeping each function's source costs memory, so apps turn it on before parsing with `doc.keepSource()`. Without it, functions are written as data blocks, which still load and run. The CLI and this site keep source. `stof:graph` is the exact form, with object IDs and compiled functions. See [Formats](/learn/formats#stof-stof-and-bstf). ## Smaller Additions - **`Lib::func()`** always calls the library, even when an object or variable has the same name: `Num::round(x, 2)`. `Lib?::func()` gives null when the library isn't there, which is handy for optional host libraries. - **Null-safe calls anywhere in a path:** `self.plugins?.on_save()`. - **Qualified type names:** `` finds the type `Point` under `Geometry`, from anywhere in the document. - **Assigning to computed targets:** `self.customer(id).plan = 'pro'` and `list[0].name = 'a'`. - **Trailing commas** in calls and parameter lists (0.10.3). - **New asserts:** `assert_null` and `assert_not_null`, plus `inf()` and `nan()`. - **JavaScript:** plain objects passed to `call` arrive as maps (like Python dicts), library functions can declare parameter names for named arguments, and a rejected promise in an async library function now throws inside Stof, so `try`/`catch` works. - **Python:** `Doc.get`, `set`, `call`, and `binary_export` have sensible defaults. - **`List.map`, `filter`, and `any`** take a function (0.10.7): `prices.map((p: int): float => p * 1.2)`. - **`Obj.set(name, value)` and `Obj.unset(name)`** (0.10.8) set or remove the field with exactly that name, so a name can have dots: `limits.set('api.calls', 1000)`. - **A variable declared without a type takes no type from its value** (0.10.8): `let x = self.price;` and then `x = 'text'` works. A declared type still holds. ## Fixed in 0.10.6 - **Left to right.** Operators and template strings evaluated their right side first, so side effects ran out of order. Everything now runs left to right. - **Arrow functions keep their variables.** An arrow passed to another function, or stored and called later, now sees the variables around where it was made, and can update them. - **Calls that return nothing give null** where a value is expected (`const x = self.log();`, `f(self.log())`), instead of an internal error. - **Parsing into an object again merges** into its nested objects instead of replacing them, so loading the same config twice, or a partial update, does what you'd expect. - **`pln` in the browser** writes to the console. ## Fixed in 0.10.7 - **`obj.run()` awaits async steps,** so each step finishes before the next one starts. - **Functions run in the order they're written** for `funcs(attributes)`, `#[main]`, `#[test]`, and `doc.run('attr')`. The order used to change between runs. - **`sort_by` is stable,** and an error in its compare function fails the sort instead of leaving the list unsorted. - **A cast that isn't possible says so,** with both types: "cannot cast (int, str) to tup". ## Faster - **Parsing:** a 5,000-line document (the Limitr pricing spec) parses 60 times faster, from 2.9s to 0.05s. Parsing is a runtime operation in Stof, so this matters every time logic is loaded or swapped. - **Calls:** function calls are about twice as fast (200,000 calls went from 2.9s to 1.5s), and loops are faster too (a million iterations went from 4.1s to 3.0s). - **Large documents** (0.10.4): `drop`, `funcs(attributes = ..)`, and method calls on objects with many fields no longer slow down as a document grows. ## Half the Memory 0.10.8 cuts what a running document holds by about half. An empty document went from 2.2 MB to 0.24 MB, and the Limitr pricing engine from 7.4 MB to 3.5 MB. That matters when an app keeps a document per customer or per request. Binary documents (BSTF) didn't change: files written by 0.10.7 load in 0.10.8, and the reverse. ## Safer, Too A fuzzing sweep over the test suite (40,000 parsed documents, 20,000 corrupted binary documents) found parser hangs, stack overflows, panics, and an out-of-memory crash in binary decoding. All are fixed: no known input crashes or hangs the runtime. All 13 `cargo audit` advisories in dependencies are fixed too. ## Tools - **A language server.** `stof lsp`, in stof-cli 0.13, gives editors errors and warnings as you type, an outline, go to definition, hover with doc comments and library docs, completion, find references, and rename. Files in a package are checked together. - **VS Code extension 0.3** uses it, and its highlighting is updated for the new syntax. [Get it on the Marketplace](https://marketplace.visualstudio.com/items?itemName=Formata.stof). - **The CLI** exits with an error code when a run, test, or parse fails, so it fits in CI, and `stof test --leaks` checks for leftover objects. - **A new agent skill.** The [Stof skill](https://github.com/dev-formata-io/stof/blob/main/content/SKILL.md) teaches AI coding agents to write correct Stof. It's rewritten from 1,900 lines to about 400, every example in it is tested. In our evaluation, agents using it solved all three tasks on the first try, with every test passing. - **New docs.** This site has been rewritten alongside the release, with a [cookbook](/cookbook/overview) of real uses and a [playground](/playground) with a scratch pad, a test runner, and shareable links. ## Breaking Changes | Change | Before | Now | What to do | |---|---|---|---| | Sandbox | Rust and Python documents could use files, environment variables, and the network by default | Nothing outside the document unless the host opts in | Call `allow_system()` or `allow_http()` (Rust, Python), or `allowHttp()` (JS) | | Paths | `a.b` found any object named `a` | `a` must be a root | Use `self.a.b`, `super.a.b`, or `root.a.b` | | Function scope | Functions could read their caller's variables | Only their own parameters and variables | Pass values as arguments | | Math with null | `3 * null == 3` | An error | Use `??`, and start totals at `0` | | Number methods | `x.round(2)` changed `x` | Returns a new value | `x = x.round(2)` | | Template strings | Could add numbers; null printed as empty | Always a string; null prints `null` | Format numbers explicitly | | Imports | Relative to the working directory | Relative to the importing file | Fix paths that assumed the working directory | | `return` | `return x` without `;` mis-parsed | `return x;` needs its semicolon | Add the `;` | | Integers | Overflow wrapped; `/ 0` crashed | Overflow becomes a float; `/ 0` is `inf` | | | `typeof` (0.10.2) | Took the rest of the expression | Binds to the next value | Use `typeof (a + b)` for an expression | | `stof` export (0.10.8) | The exact graph form, with IDs | Readable Stof | Use `stof:graph` for the exact form | | Rust `LibFunc` params (0.10.8) | `params: vector![..]` | `params: params![..]` | Change the macro | The [full changelog](https://github.com/dev-formata-io/stof/blob/main/CHANGELOG.md) has every change, including the Rust API details. ## Tell Us If something breaks when you upgrade, or you build something with Stof, tell us on [Discord](https://discord.gg/Up5kxdeXZt) or open an issue on [GitHub](https://github.com/dev-formata-io/stof).