--- title: Format API description: Call a saved authoring recipe from your own backend. One HTTP call runs it in a sandbox, and a schema you supply shapes the result into typed JSON. --- A Format is a saved authoring recipe — a house style, an output contract, a production playbook. You call it over HTTP, Sume runs it, and you get back durable media plus a JSON object in a shape you defined. This is the surface most partners integrate. It is one call rather than a prompt you maintain, and one typed record rather than a transcript you have to parse. For a map of every Sume surface (Agents UI, Models, SDK, Dashboard), start with [Sume basics](/the-basics). ```bash curl -sS -X POST "https://api.sume.com/v1/formats/chase/product-promo/runs" \ -H "Authorization: Bearer $SUME_API_KEY" \ -H "Content-Type: application/json" \ -H "Idempotency-Key: $(uuidgen)" \ -d '{"instruction":"Make two ad images for the linked product.","input":{"product_url":"https://example.com/p"}}' ``` ## Why Formats exist Sume is a **video agent** platform. Image, Video, Avatar, TTS, timeline, and related tools are available as HTTP and MCP APIs, but stitching those calls yourself recreates the hard part: deciding which clip to make next, fetching product truth, writing VO, assembling a timeline, and failing cleanly when a step breaks. A Format is that judgement, saved. Partners invoke a vanity URL (`POST /v1/formats/{handle}/{slug}/runs`); Sume boots a sandbox Agent with the recipe and the generation tools. You get artifacts — and optional structured output — without owning the orchestration graph. That is why Formats can ship work raw [Video 1.0](/models/video) cannot: a single short clip is one model call; a live-commerce or product-promo deliverable is many tool calls plus assembly into a **post-ready** video. ```text Format run = fresh sandbox + recipe (SKILL) + instruction/input + tools → artifacts + optional structured output ``` ## The mental model Four pieces, and it is worth holding all four at once. | Piece | What it is | |---|---| | **The recipe** | The Format itself: a `SKILL.md` body plus reference files, authored in the Agents dashboard or by asking the Agent in chat. It is the *how* — house style, branch rules, quality bar. | | **The sandbox** | Every run gets a freshly created sandbox — a remote Linux workspace with a filesystem, a shell, and Sume's generation tools. The recipe's files are placed there, and the run works inside it. Nothing leaks from the previous caller's disk. | | **The call** | Your `instruction` and your `input`, supplied per request. That is the *what* — product URL, brief, locale, SKUs. | | **The schema** | An optional JSON Schema. Send one and the finished run is projected into it, so you get typed JSON back instead of prose. | Two properties follow, and both are load-bearing for an integration: **One run is one unit of work.** A fresh thread, one agent turn, one receipt. There is no partial delivery — a run that could not finish comes back `failed`, never `completed` with half the work done. A bulk request (`POST …/bulk-runs`) is a server-side queue of those runs, not a second execution engine. Each item is still one Format run; the queue only keeps `concurrency` of them in flight. Poll the list at `GET /v1/format-run-queues/{queue_id}`. See [Bulk runs](/formats/bulk-runs). **Sume steers the run to the authored recipe.** The Format body is composed ahead of your instruction, so the house style is established before the task is. You are not re-sending a system prompt on every call and hoping it holds. ## Runtime flow What happens after you `POST` a run: ```text POST /v1/formats/{handle}/{slug}/runs (+ optional communication.webhook_url) -> 202 with a run receipt (status: queued) -> sandbox boots; recipe files land on disk -> Agent reads SKILL.md (+ references it asks for) -> Agent uses tools as the recipe requires, for example: talking / avatar video, image-to-video B-roll, TTS / voiceover, captions, timeline assembly -> artifacts mirrored to media.sume.com -> completion (same receipt either way): • webhook: signed POST format.run.terminal (live on api.dev; not delivered on api.sume.com yet) • poll: GET …/status until terminal, then GET …/result -> read `output` (your schema), `artifacts[]`, `primary_output_url` ``` The Agent is not limited to one model endpoint. A live-commerce style Format may generate host takes, product B-roll clips, a voice track, then compose them on a timeline into a single export. Your integration still sees **one** Format run and **one** receipt. The run is asynchronous because the work is real — a promo or multi-minute video is minutes of wall clock, not milliseconds. **Webhook and poll are peer completion paths** when delivery is on for your environment: supply `communication.webhook_url` and verify the signed `format.run.terminal` POST ([run webhooks](/agents/run-webhooks)), or loop on `status_url` / `result_url`. Today that means **webhooks on `api.dev.sume.com`**, and **poll (or [`subscribeFormatRun`](/sdk/runs)) on `api.sume.com`** until production delivery is enabled. For progress while a run is in flight, poll `events_url` — a phase timeline, not a log stream. See [Watch a run progress](/formats/runs#watch-a-run-progress). Format `on_active_run` defaults to **`allow`** (concurrent runs). That differs from Actions, which default to **`skip`** — see [Calling a Format](/formats/call#request-body) before copying Action request bodies. In the TypeScript SDK, [`subscribeFormatRun`](/sdk/runs) polls for you and yields status updates until the run is terminal. Structured output is produced **after** the run reaches a terminal status, by a separate constrained pass over what the run actually made. That ordering is the whole design, and it is documented in full on [Structured output](/formats/structured-output). ## Why multi-minute, post-ready video is possible A raw [Video 1.0](/models/video) (or similar) call returns one generated clip within that model’s length and framing limits. Long-form, post-ready work — for example a ~5 minute live-commerce host video with product inserts — is not “ask Video 1.0 for five minutes.” It is orchestration: 1. Plan the beats from the product / brief (inside the recipe). 2. Generate host and B-roll segments with the right tools. 3. Synthesize or attach voiceover where the recipe requires it. 4. Assemble segments on a timeline into one export. 5. Hand back durable `media.sume.com` URLs (and optional typed `output`). The Format owns steps 1–5. Your backend owns the invoke, the completion path (webhook where delivered, otherwise poll), and what you do with the result. ## Formats vs raw video APIs | | [Video 1.0](/models/video) / [Avatar talking-video](/models/avatar-videos) | **Format** (e.g. live-commerce, [product promo](/formats/product-promo)) | |---|---|---| | Unit of work | One model job | One sandbox Agent run that may call many tools | | Typical output | A single clip | Assembled, post-ready deliverable (+ optional JSON) | | Style / playbook | Prompted per call | Saved in `SKILL.md` (versioned recipe) | | Who orchestrates | Your code | The Format (Agent + tools in a sandbox) | | Best for | Atomic generation you compose yourself | Partner products that need a packaged workflow | Drop to model APIs when you truly need one image or one clip and will assemble elsewhere. Prefer the Format vanity invoke when the product value is the finished package. Enterprise partners with provisioned Formats (for example Mobidoo) also get a portal page under `/enterprise/{slug}/formats` that lists those recipes and filled-in call examples. ## When to use the Format API | You want | Use | |---|---| | A packaged workflow where the style is fixed and the inputs vary | **Format API** — this page | | One model invocation, nothing else | A model endpoint: [Image 1.0](/models/image), [Video 1.0](/models/video), [Music 1.0](/models/music) | | The same saved task on a cadence | [Scheduled](/agents/actions) | | An ad-hoc task with nothing worth saving | [Agent Completions](/agents/completions) | | A person in the loop, approving as it goes | The Agents chat UI at [sume.com/agents](https://www.sume.com/agents) | The line against raw model endpoints is the useful one. `POST /v1/image-1.0/generate` makes an image; a Format decides *which* images to make, makes them, and hands you a labelled result. If your product's value sits in the judgement between those two points, that judgement belongs in a Format rather than in your own orchestration code. Formats are authored in the dashboard or in chat. The Developer API can list them, read them, start runs (one, or a bulk queue), and monitor runs and queues — it cannot create or edit them. ## Instruction composition On every Format run the server composes the agent instruction in this exact order: ````text [Format: product-promo v3] [Format attached: product-promo v3 → /workspace/skills/product-promo/SKILL.md] [Format run instruction] [Sume action input] ```` The Format comes **first** — it is the *how*, and it should be established before the task. Your `instruction` comes last, so where the two disagree the model follows what you asked for. Three rules worth knowing: - **The package is attached, never inlined.** `SKILL.md` *and* its reference files are written to `/workspace/skills//` on every run, and `[Format attached]` names that path — so a body that says *read `references/style.md`* works. The turn itself carries only that pointer: no `SKILL.md` text is copied into it, whatever the body's size. - **`input` is data, never instructions.** It is written to `/workspace/inputs/sume-action-input.json` — whole, at every size — and the turn carries a bounded pointer labelling it as caller-supplied data the agent must read before acting. - **Runs over the API are unattended.** A body written for interactive chat may pause to ask a person for approval. Over the API those approvals are pre-granted and the run carries on within its spend cap. See [Runs over the API are unattended](/formats/call#runs-over-the-api-are-unattended). ### How big `SKILL.md` should be No limit applies to the body beyond the ones every package file shares — 100 MiB per file and 100 MiB per package, checked when you save the Format rather than when it runs. A Format that saved will run. Keep `SKILL.md` a short index; the agent reads `references/` on demand. Size does not change how the body is delivered either: it is attached as a file on every run and the agent reads it. What size still changes is how reliably it is *followed*. A body the agent can read in one pass, with the rules stated once each, is followed more closely than the same rules buried in twenty pages. So: - **Write a spec the agent can hold at once.** Identity, the non-negotiable rules one line each, a phase index, the tool list — with the detail in `references/*` the body points at. - **Push detail into references rather than growing the body.** They are attached beside `SKILL.md` in the same directory and cost the turn nothing until they are opened. - **Do not pad to be sure a rule lands.** A rule stated once in a short body beats the same rule stated three times in a long one. Nothing rejects the long one — it is just followed less closely, which is the more expensive failure. ## Anatomy of a Format `GET /v1/formats` and `GET /v1/formats/{format_id}` return this shape. ```json { "id": "skl_...", "object": "format", "slug": "product-promo", "handle": "chase", "title": "Product promo", "description": "House style for product promo videos.", "source": "custom", "version": 3, "status": "active", "api_trigger_enabled": true, "model": "...", "generation_spend_cap_usd_micros": 1000000, "created_at": "2026-07-20T12:00:00.000Z", "updated_at": "2026-07-30T09:00:00.000Z", "invoke_url": "https://api.sume.com/v1/formats/skl_.../runs", "vanity_invoke_url": "https://api.sume.com/v1/formats/chase/product-promo/runs" } ``` | Field | Notes | |---|---| | `status` | `active` or `inactive`. `inactive` until the API-call trigger is provisioned, and an `inactive` Format rejects API runs. | | `api_trigger_enabled` | When `true`, `POST /v1/formats/{handle}/{slug}/runs` and `…/bulk-runs` (and the opaque twins) are allowed. | | `handle` / `vanity_invoke_url` | User-facing address. Prefer these in links and curls; `null` for first-party Formats. | | `invoke_url` | Opaque path. Permanent — persist this if a rename must not break a stored URL. | | `version` | Bumps on every edit. The version actually used is echoed in the run receipt's `format.version`. | | `source` | `first_party` marks the curated catalog. Formats by Sume are called at `sume/{slug}` by any key, and the run is billed to that key. | The `SKILL.md` body is deliberately not in this shape; it reaches the model, not the caller. Exact request and response schemas come from live OpenAPI (`https://api.sume.com/reference/json`). The tables on these pages are a readable summary, not a second schema. ## Attachments A Format run can carry up to 30 images the agent can actually look at. Send them as `attachments[]` on the run create body. ```bash curl -sS -X POST "https://api.sume.com/v1/formats/chase/product-promo/runs" \ -H "Authorization: Bearer $SUME_API_KEY" \ -H "Content-Type: application/json" \ -H "Idempotency-Key: $(uuidgen)" \ -d '{ "instruction": "Make a product hero from the attached photo.", "input": { "brand": "Acme" }, "attachments": [ { "type": "input_image", "image_url": "https://cdn.example.com/shot.jpg" } ], "output_schema": { "name": "hero", "schema": { "type": "object", "properties": { "hero_image_url": { "type": "string" }, "caption": { "type": "string" } }, "required": ["hero_image_url", "caption"], "additionalProperties": false } }, "generation_spend_cap_usd": 5 }' ``` Attachments and `output_schema` compose. The images reach the agent, and the run's `output` is still parsed against your schema after the run completes. ### The item shape | Field | Required | Notes | |---|---|---| | `type` | yes | `"input_image"`. It is the only type today. | | `image_url` | one of | Public HTTPS URL. Sume fetches it — it must be reachable without auth. | | `asset_id` | one of | An asset you uploaded through the [Assets API](/api/reference#assets). Must be a ready image asset in the same workspace. | | `filename` | no | Label the agent sees. Defaults to the URL's basename. | Send exactly one of `image_url` or `asset_id` per item. ### What happens to your image Sume fetches every attachment **when you create the run**, checks its real content type and size, and copies it into Sume storage. The agent then works from a durable `media.sume.com` copy, which is also what keeps the run reproducible after your own URL rotates or expires. The practical consequence: a broken, private, or oversized image fails the create call with a `4xx`/`5xx` you can act on, instead of killing the run several minutes later. An `asset_id` or a URL already on `media.sume.com` is not re-copied. ### Limits | | Limit | |---|---| | Types | JPEG, PNG, WebP, GIF, AVIF | | Images per run | 30 | | Bytes per image | 30 MB | | Bytes per run | 500 MB | Attachments do not count against the `input` size cap — they are never serialized into `input`. ### Media referenced from `input` Many Formats take their references through their own `input` fields instead of `attachments[]` — `host_image_url`, `input_reference_image_urls[]`, B-roll clip lists, narration URLs. Those count too, against **one shared budget** with `attachments[]`: | | Limit | |---|---| | Images, videos, and audio per run, combined | 30 | | Images per run | 30 | | Videos per run | 10 | | Audio files per run | 10 | So 10 images + 10 videos + 10 audio is fine; 5 images + 15 videos is not (videos cap at 10), and neither is 15 images + 15 audio. The check is by file type, not by field name: any HTTPS URL anywhere in `input` whose filename ends in an image, video, or audio extension counts, however deeply it is nested and whatever the Format calls the field. A product page URL, or any other link that is not a media file, does not. The same URL repeated in two fields counts once. Media the agent finds for itself while the run is going — product photos it crawls off a page you named — is not what you sent, so it does not count either. Over any of these is a `400 invalid_attachment` on the create call, naming the type and the count it saw. ### Attachment errors | Status | Code | Cause | |---|---|---| | `400` | `invalid_attachment` | Wrong `type`, missing or non-HTTPS URL, both `image_url` and `asset_id`, too many items, or a source that is not an allowed image type. | | `400` | `attachment_not_found` | `asset_id` is unknown in this workspace. | | `413` | `attachment_too_large` | An image is over 30 MB, or the set is over 500 MB total. | | `502` | `attachment_fetch_failed` | Sume could not fetch the image — unreachable host, hotlink protection, or a non-2xx response. | `Idempotency-Key` covers attachments: replaying a key with a different image list returns `409 idempotency_conflict`, and a true replay does not re-fetch your images. ## What the Format API does not support yet - **No push channel.** There is no SSE or WebSocket stream. `events_url` gives a polled phase timeline (`preparing` / `running` / `finalizing`), not agent output or logs. - **Pagination is keyset.** Format and run lists return `has_more` and `next_cursor`; pass the cursor back as `?cursor=` to walk the next page. - **No authoring over the API.** The Developer API cannot create, edit, or delete a Format — use the dashboard or ask the Agent in chat. - **Team Formats need a team (workspace) key.** Vanity and opaque invoke both work for a Format owned by a team workspace — but only with an API key **created in that workspace**. A personal key fails with `403 workspace_key_required`. See [Team Formats need a team key](/formats/call#team-formats-need-a-team-key). - **No non-image attachments.** `input_image` is the only attachment `type`; PDFs and other files follow later. OpenAI's `detail` hint is not supported — Sume gives the model the full-resolution image and sizes it itself. ## Next - [Sume basics](/the-basics) — product map of Agents, Formats, Models, and clients - [Calling a Format](/formats/call) — auth, scopes, the invoke contract, and every error - [Bulk runs](/formats/bulk-runs) — queue up to 100 runs; poll `GET /v1/format-run-queues/{id}` - [Structured output](/formats/structured-output) — schema rules, the projection, failure modes - [Runs and results](/formats/runs) — the receipt field by field, polling, cancelation - [Waiting for runs (SDK)](/sdk/runs) — `subscribeFormatRun` / `waitForRun` - [Format catalog](/formats/catalog) — ready-made first-party Formats - [Product promo](/formats/product-promo) · [Avatar UGC video](/formats/avatar-ugc) - [Embed a Format in your product](/cookbooks/embed-a-format) — the whole partner integration