Skip to content
Fondazione ITS INCOM

Examples · Programming

A video clip from Python

Ask for a clip, check on it until it is ready, and download the MP4. Then a storyboard of three shots, made one after the other and joined with ffmpeg.

Intermediate · 40 min

What you need

A key, as in Your first API call in Python, and the requests library:

pip install requests

For the exercise, ffmpeg, to join clips.

The model

wan-2.2-5b-fast, the Fast model of the Studio: from a description to a 5-second MP4 without sound, 24 frames a second, 1280x704 (landscape) or 704x1280 (portrait). About two minutes a clip.

Video models are not listed by GET /v1/models: the documentation lists them.

Three requests, not one

A chat answer arrives in a few seconds, on the request you sent. A clip takes two minutes: too long to keep a request open. So the API gives you a job:

  1. POST /v1/videos creates the job and answers at once, with its id;
  2. GET /v1/videos/{id} says how it is going: queued, in_progress, then completed or failed;
  3. GET /v1/videos/{id}/content downloads the MP4, once the job is completed.

The program

Save it as clip.py:

import os
import time

import requests

API = "https://api.ai.itsincom.org/v1"
# The same key you use for the chat models, in a header.
HEADERS = {"Authorization": f"Bearer {os.environ['API_KEY']}"}


def make_clip(prompt, path, seed=None):
    # 1. Create the job. The answer comes back at once, without the video:
    #    it only says that the job exists, with an id to ask about it later.
    body = {"model": "wan-2.2-5b-fast", "prompt": prompt, "size": "1280x704"}
    if seed is not None:
        body["seed"] = seed  # the documentation: same seed, prompt and size, same clip
    r = requests.post(f"{API}/videos", headers=HEADERS, json=body, timeout=60)
    if r.status_code != 200:
        # A refused prompt, or a second job while the first is not done:
        # the message says what went wrong.
        raise RuntimeError(r.json()["error"]["message"])
    job = r.json()
    print("job", job["id"])

    # 2. Poll: ask how the job is going until it is completed or failed.
    #    A clip takes about two minutes: every ten seconds is plenty.
    while job["status"] in ("queued", "in_progress"):
        time.sleep(10)
        job = requests.get(f"{API}/videos/{job['id']}", headers=HEADERS, timeout=30).json()
        # "\r" goes back to the start of the line: the progress stays on one line.
        print(f"\r{job['status']} {job['progress']}%   ", end="", flush=True)
    print()
    if job["status"] == "failed":
        # A failed clip costs nothing. The first line of the message says why.
        raise RuntimeError(f"{job['error']['code']}: {job['error']['message'].splitlines()[0]}")

    # 3. Download the MP4: a separate request, now that the clip exists.
    #    A clip is one or two MB, so it fits in memory in one go.
    r = requests.get(f"{API}/videos/{job['id']}/content", headers=HEADERS, timeout=300)
    r.raise_for_status()
    with open(path, "wb") as f:
        f.write(r.content)
    print("saved", path)
    return job["id"]


# This part runs only when you start clip.py itself. When another program
# imports make_clip, as in the exercise below, no balloon is made.
if __name__ == "__main__":
    make_clip(
        "A red hot-air balloon rises slowly above green hills and a small lake, "
        "at sunrise, warm golden light, light mist over the water, "
        "wide shot from the ground, static camera",
        "balloon.mp4",
    )

Run it with python3 clip.py.

What you should see

job video_01m49rdhg1fr6zzsrmc3wavewd
completed 100%
saved balloon.mp4

While the clip is made, the middle line changes in place: in_progress 0%, in_progress 12%, in_progress 19%… The progress is an estimate from the time a clip usually takes, and it waits at 95% until the clip is really done. When we ran it, it took 123 seconds from the request to the file.

The clip showed the red balloon, the hills, the lake with its mist and the golden light. As in the Studio, the camera was a wish: the view came from the air, not from the ground, and it drifted slowly towards the balloon instead of standing still.

Check the file with ffprobe, which comes with ffmpeg:

ffprobe -v error -show_entries stream=width,height,r_frame_rate,nb_frames:format=duration -of default=nw=1 balloon.mp4

width=1280
height=704
r_frame_rate=24/1
nb_frames=121
duration=5.041667

121 frames at 24 a second: five seconds. The file also says where it comes from:

ffprobe -v error -show_entries format_tags=comment -of default=nw=1 balloon.mp4

TAG:comment=Generated by artificial intelligence with ITS INCOM AI (EU AI Act, art. 50).

Delete it from the server

A clip stays on the server for 7 days, and it shows up in the Studio too, under Your videos. When you have the file, you can delete it there. Save this as delete.py:

import sys

import requests

from clip import API, HEADERS  # the address and the key, from clip.py

video_id = sys.argv[1]  # the id that clip.py printed
r = requests.delete(f"{API}/videos/{video_id}", headers=HEADERS, timeout=30)
print(r.json())

python3 delete.py video_01m49rdhg1fr6zzsrmc3wavewd

{'id': 'video_01m49rdhg1fr6zzsrmc3wavewd', 'object': 'video.deleted', 'deleted': True}

Run it again and the job is gone:

{'error': {'message': "No video 'video_01m49rdhg1fr6zzsrmc3wavewd' among yours.", 'type': 'invalid_request_error', 'param': None, 'code': 'video_not_found'}}

Exercise: a storyboard in three shots

A clip is one shot. A story is several shots, one after the other. Since you can have only one job at a time, a storyboard is a loop: each request starts when the previous clip is saved. Save it as storyboard.py, in the same folder as clip.py:

import time

from clip import make_clip  # the function from clip.py, in the same folder

# One sentence per shot. The model does not remember the previous clip,
# so every shot describes the robot again, with the same words.
shots = [
    "A small round white robot with one blue eye rolls down a ramp from a spaceship "
    "onto red desert sand, at sunset, warm orange light, wide shot",
    "Close-up of a small round white robot with one blue eye looking at a glowing "
    "green crystal half buried in red sand, at sunset, warm orange light",
    "A small round white robot with one blue eye rolls across red desert sand "
    "towards a spaceship, at night under two moons, cold blue light, wide shot",
]

# One clip at a time: you can have only one job running, so each request
# starts when the previous clip is saved. Three clips take about six minutes.
for n, shot in enumerate(shots, 1):
    print(f"shot {n} of {len(shots)}")
    for attempt in (1, 2):
        try:
            make_clip(shot, f"shot-{n}.mp4", seed=7)
            break
        except RuntimeError as e:
            # A clip can fail on the machine: try it once more, a minute later.
            print("failed:", e)
            if attempt == 2:
                raise
            time.sleep(60)

# The list of the files, in order, for ffmpeg.
with open("shots.txt", "w") as f:
    for n in range(1, len(shots) + 1):
        f.write(f"file 'shot-{n}.mp4'\n")
print("Join them: ffmpeg -f concat -i shots.txt -c copy storyboard.mp4")

Run it with python3 storyboard.py, then join the three clips into one video of 15 seconds:

ffmpeg -f concat -i shots.txt -c copy storyboard.mp4

-c copy puts the clips one after the other without encoding them again: it takes a second, and it works because the three clips have the same size and format.

The model does not know that the shots belong together: each clip is made from its own sentence. That is why every shot describes the robot again, with the same words. The seed makes a shot repeatable: the documentation says that the same seed, prompt and size give the same clip, file for file.

Now make it yours:

Good to know

All examples