Video analyses

Video analyses take a public HTTPS video URL and return a job-backed scene-by-scene breakdown. Prefer this when an agent needs to read an existing clip (structure, hooks, B-roll insertion points). This is not trend discovery and not virality prediction, and it does not generate video — pair it with a generation Format when you want a structure-matched remix.

Under the hood a video-understanding model (TwelveLabs Pegasus) watches the MP4 and segments it into typed scenes with descriptions and transcript; ffmpeg then extracts one JPEG still per scene (keyframe_url). Current analysis_version is 1.1.

Remote MCP wrappers: video-analyses_create, video-analyses_get, video-analyses_list. Poll with jobs_wait (typical runtime 3–5 minutes; poll every 30–60s).

Accuracy caveat: longer videos are less accurate; short clips give the most reliable results.

Create an analysis job

Required: video_url. Optional: max_scenes (2–40, default 24), include_transcript (when true, each scene's audio carries the spoken speech for that scene; otherwise audio is null), plus the usual mode / webhook_url / wait_timeout_seconds communication fields.

A successful submit returns 202 with a vana_… resource id, a video_analysis job (request_id), and status_url / result_url.

Prefer a durable media.sume.com URL from media imports (POST /v1/media-imports) or a completed Sume generation artifact. There is no Higgsfield-style media_id intake gate.

Pricing note

Each accepted analysis reserves and captures $0.30 USD of Sume usage under the current fixed estimate. Confirm live pricing in GET /v1/catalog and OpenAPI.

Duration limits

CapBehavior
Soft (~90s)Job succeeds; response may include warning low_confidence_long_video.
Hard (300s)Rejected after probe with a stable duration error.

Unsupported inputs

InputResult
YouTube URLs422 unsupported_source
Non-HTTPS / private / localhostRejected at admit
Non-video assetsFail with a stable, agent-legible code

Poll and read

When ready, the resource includes whole-video metadata plus a typed scenes[] array. Each scene has contiguous start_seconds / end_seconds, duration_seconds, summary, visual, optional shot_type / camera_motion, on_screen_text, audio ({ speech, has_speech, has_music } when include_transcript was requested, otherwise null), keyframe_url (a per-scene JPEG still, or null with a keyframe_mirror_failed:scene_N warning), and confidence (0–1).

Scopes

ScopeOperations
video_analyses:writePOST /v1/video-analyses
video_analyses:readGET /v1/video-analyses, GET /v1/video-analyses/:id