Video filter
Current SoT. Media L2 pixel pass (
sume/video-filter-1.0, #5820 / #5839). It is available on dest and prod. This API does not cut a range. For a range cut, use video trim. This API does not assemble clips. The clip sequence, transitions, and the audio spine stay on Timeline 1.0.
Video filter 1.0 uses one workspace media.sume.com clip and a
validated program (ops[] dim/crop and/or a filters-only filtergraph)
as input. It returns a new MP4. The source does not change. The server
compiles ffmpeg on the worker media runtime (apps/api/src/routes.ts
createVideoFilterV1 / submitSumeVideoFilterJob, and
packages/timeline-compiler/src/filter.ts compileVideoFilterProgram).
There is no GET /v1/video-filter/:id. Poll the job envelope:
Hosted MCP: video_filter (packages/mcp-server/src/mcp.ts). A write
must have idempotency_key (and mcp:write under OAuth). Flow:
video_filter with check_only: true (free) → video_filter →
jobs_wait → jobs_result.
Check a program (unbilled)
POST /v1/video-filter/check runs the same checks as the encode: the
schema, the op whitelist, the filtergraph allowlist, and the Sume-host /
HEAD source preflight. It returns diagnostics and not a 400. It does
not create a job, reserve credits, boot a box, or use the encoder. A
program that passes this check can still fail on the box (bad expression,
memory, time). In that case, you get a structured job error.
Idempotency-Key is not necessary for the check. A valid response is
object: video_filter_check. It contains valid, encode: "not_run",
diagnostics[], and the compiled program.filters (names only, no argv).
If the program is valid, it also contains an estimate. It also contains
next_action submit_video_filter | fix_program_and_recheck.
Encode
You must send video_url (a media.sume.com artifact or asset of this
workspace) and a program. The program must contain a minimum of one of
these: ops[] or a non-empty filtergraph. The API does not fetch from
the open internet. First, import the clip (POST /v1/media-imports). You
must send Idempotency-Key.
Optional fields: ops, filtergraph, metadata, and the usual mode /
webhook_url / wait_timeout_seconds communication fields.
The default mode is async (readCommunicationOptions). To wait a
maximum of 30 seconds for a 200 with the completed job, send
mode: "sync". If the job does not complete in that time, you get 202,
and then you poll the job.
If the submit is successful, the API returns a job (request_id is the
job ID). When the status is result_ready, GET /v1/jobs/:id/result
returns kind: video_filter. The result contains video_url (a new
artf_, never the source), duration_seconds, ops_applied,
filtergraph, the compiled filters[], and optional warnings[]. Put
that MP4 into timeline_create video[].
Each encode job uses a flat number of credits. The API rate card lists the current credits. The check is free. There is no provider inference. Only the worker ffmpeg runs.
Program
| Field | Effect |
|---|---|
ops[] | Ordered program. Sume applies it before filtergraph. Maximum 8. You must send a minimum of one op or a filtergraph. |
ops[].op: "dim" | Multiplies the luma of the full clip. amount is in (0, 1]. 0.45 makes the clip less bright, and 1 makes no change. The API refuses 0 and >1. Black stays black, and the chroma does not change. |
ops[].op: "crop" | Rectangle as fractions of the source frame. x and y are in [0, 1]. width and height are in [0.05, 1]. x+width ≤ 1 and y+height ≤ 1. The compiler rounds to even values for yuv420p. |
filtergraph | Filters-only ffmpeg graph. Sume applies it after ops[]. Maximum 2048 characters and 32 named filters. No inputs, no outputs, and no paths. The server adds [0:v]…[vout] around the graph. Internal labels (split[a][b]) are permitted. Stream specifiers ([0:v]) are not permitted. |
Limits (from packages/api-contract/src/index.ts): source ≤ 300 s
(compose-band clip ceiling). The output keeps the geometry, frame rate,
and audio of the source, unless the program changes them.
The allowlisted filter names are in
packages/timeline-compiler/src/filter.ts
VIDEO_FILTER_GRAPH_ALLOWED_FILTERS (tone, blur, geometry, fade,
internal compositing). These items are not on that list: trim /
setpts (use video trim), drawtext / subtitles
/ movie / lut3d, and all filters that read a file or a socket.
Refusals (stable codes)
| Code | When |
|---|---|
video_filter_ops_empty | The request has no ops[], and filtergraph is missing or empty. |
video_filter_too_many_ops | More than 8 ops. |
unsupported_filter_op | ops[].op is not dim or crop. Put other pixel work in filtergraph. |
unsupported_filter_op_field | An op has an extra key (dim accepts only {op, amount}, and crop accepts only {op, x, y, width, height}). |
video_filter_amount_out_of_range | The dim amount is not in (0, 1]. |
video_filter_crop_out_of_bounds | The crop fractions are outside the source frame, or a side is less than 0.05. |
invalid_filtergraph | The graph is empty or too long, or it has an unknown filter, an illegal alphabet, or a stream specifier. unknown_filter gives the token and the allowlist. |
ffmpeg_fields_rejected | The client sent vf / filter / ffmpeg / cmd / codec / crf / -i / related fields. The server compiles ffmpeg. |
unsupported_media_source | video_url is not on the Sume media host. |
source_not_found | The media.sume.com URL is dead or is not from this workspace. |
unsupported_media_type | The HEAD result is not a video. |
source_too_large | The source is larger than the timeline download budget. |
output_duration_exceeded | The source is longer than 300 s (worker). |
The API rejects off-host URLs (https://example.com/…) at admit. Import the
clip first.
Not this surface
| Need | Use |
|---|---|
| Probe / stills / optional STT | Video inspect |
| A new MP4 cut | Video trim |
| Audio track as a durable wav / mp3 | Audio detach |
| Put several clips in a sequence | Timeline 1.0 |
| Caption plates | HyperFrames compose / caption assembler |
The accurate frame at t, at the source size | Video frames |