---
title: Video filter
description: Dim, crop, or apply an allowlisted pixel filtergraph to one Sume-hosted clip. Returns a new MP4. Material preparation, not timeline placement.
---

> **Current SoT.** Media L2 pixel pass (`sume/video-filter-1.0`, #5820 /
> #5839). Dest and prod. This is **not** a range cut — that is
> [video trim](/models/video-trim). It is **not** assembly —
> sequencing, transitions, and the audio spine stay on
> [Timeline 1.0](/models/timeline).

Video filter 1.0 takes **one** workspace `media.sume.com` clip plus a
validated program (`ops[]` dim/crop and/or a filters-only `filtergraph`)
and returns a **new** MP4. The source is untouched. The server compiles
ffmpeg on the worker media runtime (`apps/api/src/routes.ts`
`createVideoFilterV1` / `submitSumeVideoFilterJob`;
`packages/timeline-compiler/src/filter.ts` `compileVideoFilterProgram`).

```text
POST /v1/video-filter
POST /v1/video-filter/check              # unbilled contract check
POST /v1/models/sume/video-filter-1.0/runs   # same body, no extra `model` field
```

There is **no** `GET /v1/video-filter/:id`. Poll the job envelope:

```text
GET /v1/jobs/:id/status
GET /v1/jobs/:id/result
```

Hosted MCP: `video_filter` (`packages/mcp-server/src/mcp.ts`). Writes
need `idempotency_key` (and `mcp:write` under OAuth). Flow:
`video_filter` with `check_only: true` (free) → `video_filter` →
`jobs_wait` → `jobs_result`.

## Check a program (unbilled)

`POST /v1/video-filter/check` runs the same schema, op whitelist,
filtergraph allowlist, and Sume-host / HEAD source preflight as the
encode, and returns diagnostics instead of a `400`. It does **not**
create a job, reserve credits, boot a box, or touch the encoder. A
program that passes here can still fail on the box (bad expression,
memory, time) — that comes back as a structured job error.

`Idempotency-Key` is **not** required on the check. A valid response is
`object: video_filter_check` with `valid`, `encode: "not_run"`,
`diagnostics[]`, compiled `program.filters` (names only, no argv), an
`estimate` when valid, and `next_action`
`submit_video_filter` | `fix_program_and_recheck`.

```bash
curl -X POST https://api.sume.com/v1/video-filter/check \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "ops": [{ "op": "dim", "amount": 0.45 }]
  }'
```

## Encode

Required: `video_url` (this workspace’s `media.sume.com` artifact or
asset) **and** a program — at least one of `ops[]` or a non-empty
`filtergraph`. There is no open-internet fetch — import first
(`POST /v1/media-imports`). `Idempotency-Key` is required.

Optional: `ops`, `filtergraph`, `metadata`, plus the usual `mode` /
`webhook_url` / `wait_timeout_seconds` communication fields.

Default `mode` is **`async`** (`readCommunicationOptions`). Pass
`mode: "sync"` to wait up to **30 seconds** for a `200` finished job, or
get `202` and poll.

```bash
curl -X POST https://api.sume.com/v1/video-filter \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-filter-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "ops": [{ "op": "dim", "amount": 0.45 }]
  }'
```

A successful submit returns a job (`request_id` is the job id). When
`result_ready`, `GET /v1/jobs/:id/result` is `kind: video_filter` with
`video_url` (new `artf_`, never the source), `duration_seconds`,
`ops_applied`, `filtergraph`, compiled `filters[]`, and optional
`warnings[]`. Drop that MP4 into `timeline_create` `video[]`.

Public rate: **$0.02 per encode job** (`VIDEO_FILTER_PUBLIC_PRICING`;
confirm live in `GET /v1/catalog`). The check is free. No provider
inference — worker ffmpeg only.

## Program

| Field | Effect |
|---|---|
| `ops[]` | Ordered program, applied **before** `filtergraph`. Max **8**. At least one op **or** a filtergraph is required. |
| `ops[].op: "dim"` | Whole-clip luma multiply. `amount` in **(0, 1]** — `0.45` is darker, `1` is unchanged. `0` and `>1` are refused. Black stays black; chroma does not shift. |
| `ops[].op: "crop"` | Rectangle as **fractions** of the source frame: `x`, `y` in `[0, 1]`; `width`, `height` in `[0.05, 1]`; `x+width ≤ 1` and `y+height ≤ 1`. The compiler even-rounds for yuv420p. |
| `filtergraph` | Filters-only ffmpeg graph, applied after `ops[]`. Max **2048** chars, **32** named filters. No inputs, no outputs, no paths — the server wraps `[0:v]…[vout]`. Internal labels (`split[a][b]`) are fine; stream specifiers (`[0:v]`) are not. |

Caps (from `packages/api-contract/src/index.ts`): source ≤ **300** s
(compose-band clip ceiling). Output inherits source geometry, frame
rate, and audio unless the program changes them.

Allowlisted filter names live in
`packages/timeline-compiler/src/filter.ts`
`VIDEO_FILTER_GRAPH_ALLOWED_FILTERS` (tone, blur, geometry, fade,
internal compositing). **Not** on that list: `trim` / `setpts` (use
[video trim](/models/video-trim)), `drawtext` / `subtitles` / `movie` /
`lut3d`, and anything that reads a file or a socket.

## Refusals (stable codes)

| Code | When |
|---|---|
| `video_filter_ops_empty` | Neither `ops[]` nor a non-empty `filtergraph`. |
| `video_filter_too_many_ops` | More than 8 ops. |
| `unsupported_filter_op` | `ops[].op` is not `dim` or `crop`. Other pixel work goes in `filtergraph`. |
| `unsupported_filter_op_field` | Extra key on an op (dim takes `{op, amount}` only; crop takes `{op, x, y, width, height}`). |
| `video_filter_amount_out_of_range` | dim `amount` not in `(0, 1]`. |
| `video_filter_crop_out_of_bounds` | Crop fractions miss the source frame or a side is under `0.05`. |
| `invalid_filtergraph` | Empty, too long, unknown filter, illegal alphabet, or stream specifier. `unknown_filter` names the token and the allowlist. |
| `ffmpeg_fields_rejected` | Client sent `vf` / `filter` / `ffmpeg` / `cmd` / `codec` / `crf` / `-i` / friends. The server compiles ffmpeg. |
| `unsupported_media_source` | `video_url` is not on the Sume media host. |
| `source_not_found` | Dead or foreign `media.sume.com` URL. |
| `unsupported_media_type` | HEAD is not a video. |
| `source_too_large` | Source exceeds the timeline download budget. |
| `output_duration_exceeded` | Source longer than 300 s (worker). |

Off-host URLs (`https://example.com/…`) are rejected at admit. Import first.

## Not this surface

| Need | Use |
|---|---|
| Probe / stills / optional STT | [Video inspect](/models/video-inspect) |
| A new MP4 cut | [Video trim](/models/video-trim) |
| Audio track as a durable wav / mp3 | [Audio detach](/models/audio-detach) |
| Sequence several clips | [Timeline 1.0](/models/timeline) |
| Caption plates | HyperFrames compose / caption assembler |
| Exact frame at `t`, source size | [Video frames](/models/video-frames) |

## Related

- [Video inspect](/models/video-inspect)
- [Video frames](/models/video-frames)
- [Video trim](/models/video-trim)
- [Audio detach](/models/audio-detach)
- [Timeline 1.0](/models/timeline)
- [Media inputs](/workflows/asset-library)
- [Jobs and results](/workflows/jobs-and-results)
- [MCP tools and gates](/mcp/tools-and-gates)