Schemas & Validation

Schema definition

schema Invoice {
  invoice_number: string
  total: float
}

Fields are written one per line. A trailing comma is optional; putting two fields on the same line is a syntax error.

Schema-level metadata

schema Invoice
  @desc("Sales invoice")
  @prompt("Extract the invoice fields completely; set the ones you cannot find to null.") {
  invoice_number: string
  total: float
}

How to use @prompt

import schema from "std/schema"

schema InvoiceExtract
  @desc("Structured data extracted from invoice Markdown")
  @prompt("Produce JSON conforming to the InvoiceExtract schema from the Markdown below. Use the field descriptions as extraction instructions. Do not invent values you cannot find; use null in nullable fields and [] in array fields.") {
  invoice_number: string | null
  items: InvoiceItem[]
}

agent Extractor {
  fn extract(markdown: string) -> InvoiceExtract {
    instruction = schema.prompt(InvoiceExtract)
    result = ask "{instruction}\n\nINVOICE MARKDOWN:\n{markdown}" -> InvoiceExtract
    return result
  }
}

The split is deliberate: the extraction instruction is versioned together with the schema, but you decide where in the prompt it lands.

Field annotation reference

Field annotations are written after the field definition and can be chained. Long chains can be split across continuation lines that start with @.

Three commonly confused distinctions

1. Value range ≠ list length

2. @contains is for arrays

tags: string[] @contains("vip")
numbers: int[] @contains(2)

// To search for a substring in a scalar string:
name: string @valid(x => contains(x, "Ada"))

// @contains cannot be used on a scalar
// string — the parser rejects it:
name: string @contains("Ada")
// @contains usage: @contains(value)

3. @length(n) is an exact length

tax_id: string @length(10)      // exactly 10 characters (minLength = maxLength = 10)
name: string @length(2, 40)   // between 2 and 40 characters

@fmt values

@valid

@valid takes a deterministic lambda: @valid(x => expression). The supported forms:

// ✓ correct
code: string | null
  @valid(x => x == null || regex.matches(x, "^[A-Z]{3}-[0-9]+$"))

// ✗ blows up when null arrives
code: string | null @valid(x => regex.matches(x, "^[A-Z]{3}-[0-9]+$"))

@alias and @default

schema Profile {
  email: string @title("Email") @example("ada@example.com") @alias("e_mail") @alias("mail")
  active: bool   @default(true)
  old_code?: string @deprecated("Use new_code instead")
}

• If the incoming data contains e_mail, the value is moved to email and the alias key is removed.
• If both the canonical name and an alias arrive together it is an error: Profile.email cannot be given by its canonical name and an alias at the same time.
• If more than one alias arrives together it is an error: Profile.email was given more than one alias: e_mail, mail
• An alias cannot be the name of a real field in the schema, and two fields cannot share the same alias.
• A value filled in by @default then goes through normal type validation.

Automatic validations

Even if you write no annotation at all, the checks below run for every schema:

The special field: sources

If you define a field named sources: string[], every value inside it is required to be a context ID declared in the program. This is the direct way to stop the model from inventing sources. The name is fixed by the runtime — sources is Turkish for "sources" and is hardcoded, so it cannot be renamed or translated; English programs use sources too.

context contract = file("docs/contract.md")
context spec     = file("docs/spec.md")

schema Answer {
  summary: string
  sources: string[1..3]     // only "contract" or "spec" is allowed
  confidence: float [0..1]
}

Files supplied with input document: file count as context IDs as well.

Errors are collected before they are reported

Every error found in one validation pass is accumulated and reported in a single go — so the model can fix all of them at once on the retry:

Error: A.f.v.name must be at least 2 characters.
A.f.v.age must be an integer.
A.f.v.email must be in email format.
A.f.v.tags must contain unique items.
A.f.v.score must be at most 100.
A.f.v.role must be one of these values: admin, user.
A.f.v.extra is not defined in the schema.

Customizing messages with @feedback

The default error messages are sent back to the model. Use @feedback when you want to state more clearly what the model is supposed to do.

schema Order {
  subtotal: float
  tax: float
  total: float @feedback("The total must be the exact sum of the subtotal and the tax")

  where total == subtotal + tax @feedback("total == subtotal + tax does not hold")
}

Error: A.f.v: total == subtotal + tax does not hold

Error: BadRows.row[0]: total must equal subtotal + tax

where — deterministic invariants

where validates the relationship between fields. Use an annotation for a single field's own constraint and where for consistency across fields.

pure fn totalAmount(items: LineItem[]) -> float {
  total = reduce item in items with 0 => acc + item.amount
  return total
}

schema Invoice {
  items: LineItem[1..100]
  total_amount: float

  where totalAmount(items) == total_amount @feedback("The line-item total must match the invoice total")
}

Floating-point comparison

import num from "std/numeric"

schema LineItem {
  quantity: float
  unit_price: float
  total: float

  where num.approximatelyEqual(quantity * unit_price, total, 0.01)
}

schema LineItem {
  quantity: float
  unit_price: float
  total: float

  // 0.1 * 3 == 0.30000000000000004
  where quantity * unit_price == total
}

where quantity == null || unit_price == null
   || num.approximatelyEqual(quantity * unit_price, total, 0.01)
  @feedback("does not match the a * b total")

pure fn lineItemConsistent(quantity: float | null, unit_price: float | null, total: float | null) -> bool {
  if quantity == null {
    return true
  }
  if unit_price == null {
    return true
  }
  if total == null {
    return true
  }
  return num.approximatelyEqual(quantity * unit_price, total, 0.01)
}

schema LineItem {
  quantity: float | null
  unit_price: float | null
  total: float | null

  where lineItemConsistent(quantity, unit_price, total)
}

where with nullable fields

schema InvoiceItem {
  quantity: float | null
  unit_price: float | null
  total: float | null

  where quantity == null || unit_price == null || total == null || num.approximatelyEqual(quantity * unit_price, total, 0.01)
    @feedback("total does not match quantity * unit_price")
}

For nullable fields, write the "skip when there is no data" conditions first and add the real invariant afterwards. || short-circuits from left to right. @feedback can be moved to the next line; where itself cannot.

@severity — the third state

Until v0.8.0 a validated value was in one of two states: it satisfied the schema, or the run stopped. @severity("warn") adds a third — valid, but flagged. The value is accepted and the run continues; what the rule found travels with it as a finding.

The default is error, so a schema that does not mention @severity behaves exactly as it did before. The set is closed: error and warn, nothing else. A tier vocabulary of your own — review, reject, a numeric score — belongs in the code that reads the findings, not in the rule.

Where it is written

On a field, @severity modifies the @valid rule it follows; on a where line it modifies that invariant. Those are the only two places, and both are checked at parse time.

schema Scored {
  // the rule may fail without stopping the run
  score: int @valid(x => x >= 0) @severity("warn")

  // two rules, only the second one soft
  ratio: float @valid(x => x <= 1) @valid(x => x > 0) @severity("warn")
}

schema Line {
  qty: int
  total: int

  where total == qty * 5 @severity("warn")
}

Honoured on results, ignored on inputs

The same schema can describe a model's answer in one place and a tool's argument in another. Severity is not a property of the schema alone — it depends on which side of a boundary the value is being checked on.

The reason is the difference between the two sentences those paths speak. A flagged result says "this data is doubtful, and here is why" — what to do about it is the program's decision. A flagged input would say "we are calling your side-effecting code with data we know breaks its contract", which is the silent-corruption class Luma spends its rules closing.

Error: Sink.take.s.score does not satisfy its @valid rule: x => x >= 0

On the ask path, a soft rule still gets a retry

A soft violation is not a shrug. On the ask path it enters the retry loop exactly like a hard one: the failing rules are spoken back to the model with their paths, and the model gets its chance to correct. The first eight messages are quoted and the rest are counted ((+3 more)).

The difference is only at the end. When the attempts run out, a hard violation stops the run and a soft one does not: the last value is accepted, and the finding records the attempt number it was accepted on. Retry exists only where there is a model to feed — on the other result paths a soft violation goes straight to "accepted and flagged". This is the same asymmetry @feedback already has.

Reading the findings

std/findings is how a flag reaches the program. Both functions take the flagged value and are io, not pure:

import findings from "std/findings"

schema Scored {
  score: int @valid(x => x >= 0) @severity("warn")
}

schema Audit {
  score: int
  flaws: int
  first_rule: string
}

agent Maker {
  fn make(value: int) -> Scored {
    return { score: value }
  }
}

agent Auditor {
  uses maker: Maker

  fn audit() -> Audit {
    shaded = maker.make(-7)
    flaws = findings.of(shaded)
    return { score: shaded.score, flaws: length(flaws), first_rule: flaws[0].rule }
  }
}

Error: findings is io and cannot be used inside @valid or where rules: findings.count

Findings live per operation. Under parallel over each element collects its own and the results merge in input order, so a flag never lands on the wrong row.

Handing a flagged value to something that inspects it

A flagged value cannot cross a parameter typed with the schema whose soft rule it broke. That is deliberate, but it leaves a real question: how do you write the helper whose job is to look at flagged values? Typing the parameter any works and the flag survives the crossing — but it drops the type check for every other field too.

The typed way is to say what the boundary actually accepts: a candidate schema with the same shape that keeps the hard rules and leaves out the soft one being inspected.

schema Document {
  amount: int @valid(x => x >= 0) @severity("warn")
  name: string @length(2, 40)
}

// Same shape. Keeps @length; leaves out the rule under inspection.
schema DocumentCandidate {
  amount: int
  name: string @length(2, 40)
}

agent Reviewer {
  fn review(doc: DocumentCandidate) -> Report {
    flaws = findings.of(doc)
    return { amount: doc.amount, flaws: length(flaws), rule: flaws[0].rule }
  }
}

The value crosses, findings.of still returns its flags, and doc.amount and doc.name stay typed — @length is still enforced at that boundary. What you gave up is exactly one rule, in writing, at the place where it matters.

r = reviewer.review(doc)   // this
return { amount: r.amount }

return reviewer.review(doc)   // not this

A flag nobody read is still announced

Because reading is optional, an unread finding would be a value silently accepted — the very thing @severity is allowed to do only because it is visible. So the run does not go quiet. If findings were produced and never read, the run ends with a warning on standard error naming each producing operation and its count:

Warning: 1 finding accepted but never read: Maker.make.return (1). Read them with findings.of(...) from std/findings.

It is a warning, not an error: the exit code stays 0, because stopping the run would defeat the feature. Read the findings and the run is silent. This is the same bargain a bare call line makes — the language permits it and declines to be quiet about it.

The ceiling

A query returning two hundred thousand rows, multiplied by its rules, could accumulate findings without bound. So there is a ceiling of 1000 per operation. Past it, violations are counted instead of stored and a marker finding named findings.capped says exactly how many were cut:

Findings cap reached (1000); 412 more soft violations were not recorded.

Neither growing without bound nor shrinking without a trace — the count is always in the record.

when — conditional rules

when makes other fields required or forbidden according to a discriminator field. This is not a workflow if — it is for schema validation and JSON Schema generation.

where kind != "calc" || amount != null
  @feedback("amount is required for a calc operation")

schema Operation {
  kind: enum["calc", "text"]
  calc?: CalcInput
  text?: string

  when kind == "calc" require [calc] forbid [text]
    @feedback("The calc field is required in a calc operation")

  when kind == "text" require [text] forbid [calc]
    @feedback("The text field is required in a text operation")
}

else when / else chains

schema Envelope {
  payload: Payload

  when payload.kind == "calc" require [payload.calc] forbid [payload.text]
    @feedback("The calc field is required in a calc payload")
  else when payload.kind == "text" require [payload.text] forbid [payload.calc]
  else require [payload.text]
    @feedback("The text field is required for other payload kinds")
}

• when starts a new group; else when and else join the previous group.
• Within a group the first matching branch is applied; if none match, the else branch runs.
• A nested field path can be written with a dot: payload.calc.
• An else cannot be written without a when before it.
• Every field name that appears in the rules must exist in the schema, otherwise you get a parse error.

Conditional defaults

schema Operation {
  kind: enum["calc", "text", "other"]
  currency?: string
  description?: string

  when kind == "calc" default [currency = "TRY"]
  else when kind == "text" default [currency = "TEXT", description = "Text operation"]
  else default [currency = "GENERIC"]
}

A conditional default is applied only when the field is absent; an existing value is not overwritten. The filled-in value then goes through normal type validation. Defaults run before the requiredness checks.

How it maps to JSON Schema

When the condition has the form field == constant, an if/then/else structure is produced in JSON Schema:

"allOf": [
  { "if":   { "properties": { "kind": { "const": "company" } } },
    "then": { "allOf": [ { "required": ["tax_id"] } ] } }
]

when, or a tagged union?

Everything above is about a conditional requirement inside one schema: a field that becomes required, forbidden or pre-filled because another field holds a particular value. That is what when is for, and it stays the right tool for it.

The same rules were also carrying a second, quite different job: saying "this object is exactly one of these shapes" — a kind enum, one branch field per shape, and a require/forbid pair per branch. That job now has a declaration of its own:

schema Invoice {
  invoice_number: string
}

schema Receipt {
  reference: string
}

schema Document = Invoice | Receipt tagged by kind

The difference is not cosmetic. In the envelope, exclusivity is an invariant you wrote: forget the when rule for one branch and an object carrying both branches at once validates cleanly — nothing points at the rule that is missing. A tagged union cannot be wrong that way, because exclusivity is the type: a value carries one member and its tag, and the match that consumes it has to name every member. Leave one out and the program never runs — it is rejected while parsing, with Match for tagged union 'Document' must handle member 'Receipt'.

Nothing is taken away here. The envelope still validates exactly as it did, and it stays the answer when the branches are not schemas in their own right or when other fields have to sit beside the branch; a union member also keeps its own where, when and annotations, which run at full strength whenever a Document arrives as an Invoice. For the declaration itself, match, and what the tag holds → Type System · Tagged union.

Schema composition

schema User {
  id: string
  email: string @fmt("email")
}

schema AuditFields {
  created_at: string @fmt("date-time")
  created_by: string
}

schema Admin = User + AuditFields

A wrapper for any payload

Composition merges two schemas that both already exist. The other half of the reuse problem is a schema whose shape repeats while its payload changes — Page, Result, Envelope. Written out by hand, that is one copy per payload type:

schema LineItem {
  sku: string
}

schema Customer {
  name: string
}

schema Page<T> {
  items: T[]
  total: int
  next_cursor: string | null
}

schema Report {
  lines: Page<LineItem>
  customers: Page<Customer>
}

schema LineItemPage {
  items: LineItem[]
  total: int
  next_cursor: string | null
}

schema CustomerPage {
  items: Customer[]
  total: int
  // next_cursor was added on the left only
}

Today both halves validate the same values. The difference shows up on the day next_cursor is added: on the left it lands on every payload at once, on the right it lands wherever the person editing remembered to put it. Nothing reports the copy that was missed, because nothing is wrong — CustomerPage is a perfectly valid schema, just a different one from the wrapper it was supposed to be. That is what drift is, and one declaration is the only thing that rules it out.

Validation order

When an object is validated the steps run in exactly this order. The order matters: defaults, for example, are filled in before the requiredness check, while where is left until last because it assumes every field is already valid.

There are three entry points into this sequence and they all run the same steps, so every rule below — annotations, when, where, composition — applies equally to data that never touched a model. Only the error path prefix differs:

  1  alias resolution      alias → move to the canonical name
  2  @default fill-in      constant value for absent fields
  3  conditional defaults  when/else default [ … ]
       │
  4  for every field
       ├─ alias collision check
       ├─ requiredness check         "X.y missing."
       ├─ type validation            (nested schemas recursively)
       ├─ sources context check    (if the field name is "sources")
       ├─ collection rules           minItems · maxItems · unique · uniqueBy · contains
       ├─ value rules                fmt · length · match · range  (per element)
       └─ @valid expressions         run on null values too
       │
  5  unknown field check       "X.k is not defined in the schema."
  6  when require / forbid     the first matching branch in the group
  7  where invariants
       ↓
  every error is combined and reported in a single go

Schema reflection

Schemas are compile-time definitions; they cannot be modified at runtime. They can, however, be read deterministically through std/schema — to generate a prompt, produce a catalog or export the JSON Schema:

import schema from "std/schema"

name        = schema.name(Invoice)                 // "Invoice"
description = schema.description(Invoice)          // the @desc text or null
instruction = schema.prompt(Invoice)               // the @prompt text or null
fields      = schema.fields(Invoice)               // field descriptors
field       = schema.field(Invoice, "invoice_number")
var         = schema.hasField(Invoice, "seller.name")
rules       = schema.rules(Invoice)                // where + conditional rules
parts       = schema.components(Admin)             // ["User", "AuditFields"]
catalog     = schema.catalog([Invoice, Receipt])
js          = schema.toJsonSchema(Invoice)

For the full signature list and return shapes → Standard Library · std/schema.

schema Invoice {
  invoice_number: string
}

schema Receipt {
  reference: string
}

schema Document = Invoice | Receipt tagged by kind

agent Extractor {
  fn extract(kind: string, text: string) -> Document {
    switch kind {
      case "Invoice" { ask "Extract the invoice fields:\n{text}" -> Invoice }
      default { ask "Extract the receipt fields:\n{text}" -> Receipt }
    }
  }
}