Commercial Video Production Pipeline with AI Tools

TL;DR / Quick Summary & Key Takeaways

Agencies charging clients for 30–60 second commercials no longer need a camera crew, a voice actor’s studio booking, or a week in the edit bay. A four-stage AI pipeline — script → voice → visuals → assembly — can take a coffee shop, SaaS, or real estate brief from creative concept to a client-ready cut in under a day, at a fraction of traditional production cost.

• Script generation (Claude): structured scene-by-scene scripts with shot direction baked into the prompt, not just dialogue.
• Voice generation (ElevenLabs): brand-consistent voiceover using saved voice IDs, with pacing/emotion tags embedded directly in the script text.
• Visual rendering (Veo 3 / Runway): scene-level prompts derived programmatically from the script, keeping visual style and pacing consistent across shots.
• Assembly (Descript): automated rough-cut assembly using its API/composition layer, syncing voice timing to generated scenes and adding captions.
• This is a pipeline, not four disconnected tools — the real engineering work is in the JSON contract that passes scene data between each stage, not in any single tool’s UI.
• Client vertical matters: a coffee shop spot needs warm, handheld-feeling B-roll; a SaaS explainer needs UI-screen composites; real estate needs consistent property geography across scenes — the prompt templates below are built per-vertical.

---

Core Technical / Conceptual Deep Dive

Why a pipeline beats tool-hopping

Most agencies that “use AI for video” are still manually copy-pasting between ChatGPT, a voice tool, and a video generator — which breaks consistency at every handoff. The fix is treating the whole production as a data pipeline where each stage consumes and produces a shared JSON schema (a “shot list” object), so scene count, pacing, and style descriptors stay locked across every tool.

{
  "brand": "Roast & Reel Coffee Co.",
  "duration_target_seconds": 30,
  "tone": "warm, artisanal, unhurried",
  "scenes": [
    {
      "scene_id": 1,
      "duration_seconds": 4,
      "voiceover": "Every morning starts the same way here.",
      "visual_prompt": "Close-up, steam rising off a pour-over coffee, warm morning light, shallow depth of field, handheld camera feel",
      "camera": "static close-up"
    }
  ]
}

This shot-list JSON is the contract between all four stages. Claude produces it; ElevenLabs consumes the voiceover fields; Veo 3/Runway consume the visual_prompt and camera fields; Descript consumes the final rendered assets plus the original timing data for sync.

Stage responsibilities

Stage	Tool	Input	Output
Script + shot list	Claude	Client brief, brand voice guide	Structured JSON shot list
Voiceover	ElevenLabs	voiceover text per scene	Per-scene MP3/WAV with timestamps
Visual scenes	Veo 3 / Runway Gen-3	visual_prompt + camera per scene	Per-scene video clips
Assembly	Descript	All clips + audio + timing	Rough cut with captions

---

Practical Tutorial / Step-by-Step Implementation

Step 1: Generate the shot list with Claude

Prompt Claude to return the shot list as strict JSON so it’s directly machine-consumable by the next stage — this is the single most important engineering decision in the pipeline.

import anthropic
import json

client = anthropic.Anthropic()

brief = """
Client: Roast & Reel Coffee Co. (independent coffee shop)
Goal: 30-second Instagram/Reels ad, warm and artisanal tone
Key message: Small-batch roasted, made by hand, every morning
CTA: Visit us on Main Street
"""

system_prompt = """You are a commercial video scriptwriter and director.
Return ONLY valid JSON matching this schema, no prose, no markdown fences:
{
  "brand": string,
  "duration_target_seconds": number,
  "tone": string,
  "scenes": [
    {"scene_id": number, "duration_seconds": number, "voiceover": string,
     "visual_prompt": string, "camera": string}
  ]
}
Keep total scene duration equal to duration_target_seconds. 5-7 scenes."""

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=2048,
    system=system_prompt,
    messages=[{"role": "user", "content": brief}],
)

shot_list = json.loads(response.content[0].text)
print(json.dumps(shot_list, indent=2))

Step 2: Generate voiceover per scene with ElevenLabs

Use a saved, brand-consistent voice ID (cloned or selected from the library once per client) so every future spot for that brand sounds identical without re-selecting a voice.

import requests

ELEVENLABS_API_KEY = "your-api-key"
VOICE_ID = "your-saved-brand-voice-id"

def generate_voiceover(text: str, scene_id: int) -> str:
    resp = requests.post(
        f"https://api.elevenlabs.io/v1/text-to-speech/{VOICE_ID}",
        headers={
            "xi-api-key": ELEVENLABS_API_KEY,
            "Content-Type": "application/json",
        },
        json={
            "text": text,
            "model_id": "eleven_multilingual_v2",
            "voice_settings": {"stability": 0.5, "similarity_boost": 0.75},
        },
    )
    resp.raise_for_status()
    out_path = f"scene_{scene_id}_voice.mp3"
    with open(out_path, "wb") as f:
        f.write(resp.content)
    return out_path

for scene in shot_list["scenes"]:
    scene["voice_file"] = generate_voiceover(scene["voiceover"], scene["scene_id"])

Step 3: Render each scene’s visuals

Both Veo 3 and Runway accept a text prompt plus duration; route the visual_prompt and camera fields directly from the shot list so visual style never drifts between scenes written days apart.

def render_scene_veo(scene: dict) -> str:
    # Pseudocode — actual endpoint/auth depends on your Veo 3 API access tier
    prompt = f"{scene['visual_prompt']}, {scene['camera']}, cinematic, 4K"
    job = veo_client.generate_video(
        prompt=prompt,
        duration_seconds=scene["duration_seconds"],
        aspect_ratio="9:16",   # vertical for Reels/TikTok
    )
    return job.download(f"scene_{scene['scene_id']}_visual.mp4")

for scene in shot_list["scenes"]:
    scene["visual_file"] = render_scene_veo(scene)

Consistency tip: append a fixed style suffix (e.g., ", warm 35mm film grain, natural light, no text overlays") to every visual_prompt at render time — this is what actually keeps a multi-scene AI-generated ad from looking like five unrelated stock clips stitched together.

Step 4: Assemble the rough cut

Descript’s project format accepts audio and video tracks with explicit timing. Build a simple assembly manifest and either import it through Descript’s UI or drive it via its API/composition layer if you’re on an Enterprise plan with API access.

def build_assembly_manifest(shot_list: dict) -> dict:
    timeline = []
    cursor = 0.0
    for scene in shot_list["scenes"]:
        timeline.append({
            "start": cursor,
            "duration": scene["duration_seconds"],
            "video": scene["visual_file"],
            "audio": scene["voice_file"],
            "caption_text": scene["voiceover"],
        })
        cursor += scene["duration_seconds"]
    return {"brand": shot_list["brand"], "timeline": timeline}

manifest = build_assembly_manifest(shot_list)
with open("assembly_manifest.json", "w") as f:
    json.dump(manifest, f, indent=2)

Import assembly_manifest.json into your Descript project (or your NLE of choice) to auto-populate the timeline, then apply auto-captions and a brand LUT/color pass as the final manual polish step — this is intentionally the one stage where a human editor should still review before client delivery.

---

Tool / Solution Comparison Table

Stage	Primary Tool	Alternative	Strength	Watch-out
Script/shot list	Claude (Sonnet 5)	GPT-4o	Strong structured-JSON output, long-context brand guides	Always validate JSON before passing downstream
Voiceover	ElevenLabs	MiniMax Speech, Play.ht	Best-in-class naturalness, stable brand voice cloning	Emotion/pacing tags need per-model syntax tuning
Visual scenes	Google Veo 3	Runway Gen-3, Luma Dream Machine	Strong physics/motion coherence, longer native clips	Higher cost per generation than Runway for iteration-heavy work
Assembly	Descript	Adobe Premiere + Frame.io, CapCut	Fast auto-caption + rough cut, screen-recording native	Deep color grading still needs Premiere/DaVinci

---

Actionable Checklist / Next Steps

• Lock the shot-list JSON schema first — every downstream tool call depends on it staying stable across script revisions.
• Save a client’s ElevenLabs voice ID once, reuse it across every future spot for brand consistency — don’t regenerate a new voice per project.
• Append a fixed visual style suffix to every scene prompt sent to Veo 3/Runway to prevent scene-to-scene visual drift.
• Render at the client’s actual delivery aspect ratio (9:16 for Reels/TikTok, 16:9 for YouTube/website) — don’t crop after the fact, it destroys framing.
• Budget one manual review pass before delivery — auto-assembly gets you 80% of the way; color, pacing trims, and music bed placement remain a human polish step.
• Track per-scene generation cost (Veo/Runway credits are the biggest line item) so client quotes reflect real production economics, not guesswork.
• Version your prompt templates per vertical (coffee shop, SaaS, real estate) so a new client brief in an existing vertical takes minutes, not a rebuild from scratch.