IO & Multimodal

The project root and @ paths

project/                   ← io root (the nearest package.json lives here)
├── package.json
├── luma/
│   └── invoice.luma       the program being run
├── 0213/
│   ├── page-1.png
│   └── page-2.png
└── docs/
    └── document.md

  "@0213/page-1.png"    →  project/0213/page-1.png
  "@/docs/document.md"  →  project/docs/document.md
  "docs/document.md"    →  project/docs/document.md   (root-relative even without @)
  "../secret"           →  rejected

Reading and writing files

import io from "std/io"

agent FileWorker {
  fn run() -> Result {
    exists  = io.exists("@docs/document.md")
    text    = io.readText("@docs/document.md")
    mime    = io.mimeType("@0213/page-1.png")     // "image/png"
    file    = io.readFile("@0213/page-1.png")
    dataUrl = io.toDataUrl(file)
    written = io.writeText("@output/summary.md", text)
    return { path: written }
  }
}

The shape of an io.readFile result

{
  "path":      "0213/page-1.png",
  "mime_type": "image/png",
  "base64":    "iVBORw0KGgoAAAANSUhEUg…",
  "size":      184213
}

MIME mapping

io.writeText creates the intermediate directories of the target automatically — you do not have to prepare the output directory beforehand.

Listing files

pages = io.listFiles("@0213", "*.png", false)
// ["@0213/page-1.png", "@0213/page-2.png", "@0213/page-10.png"]

pages = io.listFiles("@0213", "*.png", false)
for page in pages {
  ...
}

// The pattern cannot contain a path:
pages = io.listFiles("@", "0213/*.png", false)
// io listFiles pattern expects
// only a file name glob.

Multimodal markers

Image and file content is embedded into the prompt as a marker. When the model profile has multimodal: true, those markers are turned into content parts.

{{image_url:<url or data URL>}}
{{file_url:<url or data URL>}}

// 1) A single file, by hand
file    = io.readFile("@0157.pdf")
dataUrl = io.toDataUrl(file)
text    = ask "Convert this invoice to Markdown. Document: {{file_url:{dataUrl}}}" -> Transcript

// 2) Many files, automatically
files   = io.readFiles(pages)
markers = io.toMultimodalMarkers(files)
text    = ask "Convert all the pages to Markdown, in order: {markers}" -> Transcript

The two forms are not interchangeable, and the rule that separates them is about memory. io.readFiles reads every path you hand it, base64-encodes each one, and returns the whole list — so all of the files are resident at the same time, at roughly 4/3 of their bytes on disk. Pass a list to it when the files are small and you want them in one prompt anyway; pass [one] at a time when they are pages, scans or PDFs. That is why the end-to-end pipeline below calls io.readFiles([page]) inside a loop rather than io.readFiles(pages) once.

Error: Result set exceeded the row limit (200000). Process in batches with col.chunk, or raise --max-rows.

io.toMultimodalMarkers looks at each file's MIME type to pick the right marker and joins them line by line:

Provider content mapping

End to end: multi-page invoice extraction

The pipeline below reads the PNG pages in a directory and converts them to Markdown, then extracts typed invoice data out of that Markdown and writes two files. It is the essence of the luma/064_invoice_multi_desc_prompt.luma example in the repository.

  @0213/*.png  (in natural order)
       ↓  a separate multimodal ask per page
  Markdown fragments  →  concatenate  →  @0213/pre.md
       ↓  schema.prompt(InvoiceExtract) + Markdown
  InvoiceExtract  (typed, validated)
       ↓
  @0213/ext.json

1. Declare the input and the models. Using a multimodal model for the images and a separate wide-context model for the text extraction improves both quality and cost.
   luma

   input folder: string
   
   import io     from "std/io"
   import json   from "std/json"
   import schema from "std/schema"
   import num    from "std/numeric"
   
   model Vision {
     provider: "openai-compatible"
     base_url: "https://gateway.example/v1"
     model: "vision-model"
     api_key: getEnv("api_key")
     max_tokens: 6000
     multimodal: true
   }
   
   model TextExtractor {
     provider: "openai-compatible"
     base_url: "https://gateway.example/v1"
     model: "text-model"
     api_key: getEnv("api_key")
     max_tokens: 24000
   }

2. Write the output schema together with its field descriptions. @desc texts go into the JSON Schema and act as extraction instructions for the model.
   luma

   schema InvoiceItem {
     line_no: int | null   @desc("Line item sequence number on the invoice")
     description: string | null @desc("Description of the goods or the service")
     quantity: float | null @desc("Quantity sold")
     unit_price: float | null @desc("Unit price before discount")
     total: float | null @desc("Line item total after discount, excluding VAT")
   
     where quantity == null || unit_price == null || total == null || num.approximatelyEqual(quantity * unit_price, total, 0.01)
       @feedback("does not match the quantity * unit_price total")
   }
   
   schema MetaEntry {
     key: string
     value: string
   }
   
   schema InvoiceExtract
     @desc("Structured data extracted from the invoice Markdown")
     @prompt("Produce JSON conforming to the InvoiceExtract schema from the invoice Markdown text below. Use the field descriptions as extraction instructions. For every line item that appears in every table, produce a separate object inside items; do not skip rows and do not summarise. Do not invent values that are not in the text; use null for nullable fields and [] for array fields. Do not lose any information that does not fit the schema fields; keep it as key/value pairs in the meta field.") {
     invoice_number: string | null @desc("Invoice number")
     invoice_date: string | null @desc("Invoice date, YYYY-MM-dd HH:mm")
     items: InvoiceItem[] @desc("All line items on the invoice")
     notes: string[]      @desc("Footnotes and bank details")
     meta: MetaEntry[]   @desc("All information that does not fit the schema fields")
   }

3. Transcribe the pages one by one. Processing each page with its own ask is markedly more reliable than sending all the pages in a single request: the token limit is not exceeded and the model does not mix the pages up.
   luma

   agent InvoiceReader {
     model: Vision
     retry: 2
   
     fn transcribe() -> InvoiceTranscript {
       root     = "@" + folder
       pages    = io.listFiles(root, "*.png", false)
       markdown = ""
   
       for page in pages {
         files   = io.readFiles([page])
         markers = io.toMultimodalMarkers(files)
         part    = ask "Convert this invoice image to Markdown in full. Keep the table as a Markdown table; carry over the company, the customer, the line items, the VAT, the totals and the footnotes. Fill in the markdown field only. Image: {markers}" -> InvoiceTranscript
         markdown = markdown + part.markdown + "\n\n"
       }
   
       written = io.writeText(root + "/pre.md", markdown)
       return { markdown: markdown }
     }
   }

4. Extract the typed data from the Markdown. The instruction is read from the schema itself.
   luma

   agent InvoiceExtractor {
     model: TextExtractor
     retry: 2
   
     fn extract(markdown: string) -> InvoiceExtract {
       instruction = schema.prompt(InvoiceExtract)
       result  = ask "{instruction}\n\nINVOICE MARKDOWN:\n\n{markdown}" -> InvoiceExtract
       root    = "@" + folder
       written = io.writeText(root + "/ext.json", json.pretty(result))
       return result
     }
   }

5. Wire the pipeline up and run it.
   luma

   reader     = InvoiceReader()
   extractor  = InvoiceExtractor()
   transcript = reader.transcribe()
   result     = extractor.extract(transcript.markdown)

   bun src/index.ts luma/invoice.luma \
     --folder 0213 \
     --all \
     --trace-file /tmp/invoice-trace.json

Why this way?

jq -r '.events[] | select(.type=="validation" and .ok==false)
        | "\(.name) [\(.metadata.rule // "-")] \(.error // "-")"' /tmp/invoice-trace.json