Timeline 1.0
Current SoT. Timeline 1.0 render (
sume/timeline-1.0). Dest and prod. This is assembly — sequencing, transitions, and the audio spine. Material prep stays on video trim, audio detach, video filter, and timeline compose.
Timeline 1.0 takes a declarative document (one audio spine + ordered
video[] slots) and returns one MP4. The server compiles ffmpeg on
the worker media runtime (apps/api/src/routes.ts renderTimelineV1 /
submitSumeTimelineRenderJob). Callers never send filtergraphs, codecs,
or shell fragments.
There is no GET /v1/timeline-1.0/:id. Poll the job envelope:
Hosted MCP: timeline_create then jobs_wait then timeline_get
(packages/mcp-server/src/mcp.ts). timeline_get is
GET /v1/jobs/:id/result. Writes need idempotency_key (and mcp:write
under OAuth).
Plan (unbilled)
POST /v1/timeline-1.0/plan (planTimelineV1) runs schema + Sume-host
URL checks + the pure compiler and returns
object: timeline_plan with duration_seconds, segment_count,
billable_minutes, estimated_cost_usd_micros, and a
filtergraph_summary. It does not create a job, reserve credits, or
download media. Idempotency-Key is not required. A plan cannot predict
short-source pad/loop warnings.
Render
Required: audio.duration_seconds (1–1800) plus either audio.url
or audio.parts[] (unless audio.mode is "silence"), and video[]
(1–200 slots). Every URL must already be this workspace’s
media.sume.com artifact or asset. Import first
(POST /v1/media-imports). Idempotency-Key is required.
Default mode is async (readCommunicationOptions). Pass
mode: "sync" to wait up to 30 seconds for a 200 finished job, or
get 202 and poll.
A successful submit returns a job (type: timeline_render,
model: sume/timeline-1.0). When result_ready,
GET /v1/jobs/:id/result is kind: timeline_render with video_url,
duration_seconds, segment_count, billable_minutes, and optional
warnings[]. Soft warnings (padded/looped short sources, snapped
transitions, ignored still motion) are not failures.
Public rate: $0.10 per ceil(output minute)
(TIMELINE_PUBLIC_PRICING; confirm live in GET /v1/catalog). Reserve
is ceil(audio.duration_seconds / 60) minutes. No provider inference —
worker ffmpeg only.
Default output is 1080×1920 MP4. An omitted output.fps renders at the
rate the sources already run at (the longest video sources decide; stills
have no rate; 30 only when nothing has one). A rate that differs from a
source's is met by repeating or dropping a frame every few frames — judder
on motion — and is reported as output_fps_resamples_sources with the rate
the sources wanted.
Program
| Field | Effect |
|---|---|
audio.duration_seconds | Output length. Required. 1–1800 s. |
audio.url | One Sume-hosted spine. Exclusive with parts. |
audio.parts[] | ≤20 gapless slices (url + optional source_in / duration). Sample-domain join, no re-TTS. Exclusive with url. |
audio.mode | "silence" — declared length with no spine file. Then no url / parts / gain_db / source_in. |
audio.source_in | In-point into a single url spine. Illegal with parts. Output length is still duration_seconds. |
audio.gain_db | −60…12. Illegal with silence. |
video[].source_url | Sume-hosted clip or still. Stills are static holds (motion is accepted and ignored with motion_ignored). |
video[].start | On-spine start. video[0].start must be 0. Later starts must increase. Declared starts are authoritative — the compiler compensates xfade, it never pre-shifts. |
video[].duration | On-screen length, ≥ 0.2 s. Coverage may trail the spine by at most 0.5 s. |
video[].source_in | In-point into the file. |
video[].fit | cover (default) | contain | stretch | blur. |
video[].transition | On slots after the first. type ∈ fade | wipeleft | wiperight | slideup | slidedown | dissolve. Duration ≤ 1 s, ≤ 50% of the shorter neighbour, and at least one output frame. |
output.width / height | Even integers 256–2160. |
output.fps | 24 | 25 | 30 | 60. Omit to match the sources. |
output.fade_in_seconds / fade_out_seconds | 0–5 s; sum ≤ output length. |
soundtrack | Optional bed: url, gain_db, loop, fade_out_seconds ≤ 10, duck_db 0–20 (needs a real spine, not silence). |
render.strategy | auto (default; chunks past 12 segments) | chunked | single (single refused above 12 slots: render_strategy_unsafe). |
Sliced VO that is only needed inside this render belongs on
audio.parts[]. A reusable merged file is
timeline audio. Two sources on screen at once
is timeline compose, then drop that MP4 into
video[].
Refusals (stable codes)
| Code | When |
|---|---|
audio_url_required | No url / parts and not silence. |
audio_url_and_parts_exclusive | Both url and parts. |
silent_audio_takes_no_url / _parts / _gain / _source_in | Silence plus a spine field. |
audio_source_in_requires_single_spine | source_in with parts. |
audio_parts_shorter_than_duration | Declared part lengths sum to less than duration_seconds. |
timeline_must_start_at_zero | video[0].start ≠ 0. |
transition_on_first_segment | video[0].transition. |
invalid_segment_timing / segment_overlap | Starts not increasing, or overlap past the xfade. |
transition_too_long / transition_not_frame_aligned | Duration vs neighbours / fps. |
too_many_chained_transitions | More than 8 adjacent fades. Insert a hard cut. |
edge_fades_exceed_output / soundtrack_fade_exceeds_output | Fade longer than the spine. |
duck_requires_audio_spine | soundtrack.duck_db with silence. |
render_strategy_unsafe | strategy: "single" with more than 12 slots. |
unsupported_media_source / source_not_found | Off-host or dead URL. |
| Provider / ffmpeg keys | 400 — model, filtergraph, ffmpeg_args, codec, crf, friends (TIMELINE_REJECTED_PROVIDER_KEYS). |
Off-host URLs (https://example.com/…) are rejected at admit. Import first.
A dim / crop / pixel pass is not a render option — use video filter.
Not this surface
| Need | Use |
|---|---|
[start, end) of one clip | Video trim |
| Audio track as a durable wav / mp3 | Audio detach |
| Concat / split audio into reusable files | Timeline audio |
| Still + video in one frame (반배너) | Timeline compose |
| Pixel pass (dim / crop) | Video filter |
| Probe / stills / optional STT | Video inspect |