Knowledge Vault

The Code-as-Video Compiler — Procedural Media and RSI Pipelines

2 min read

Overview

AP-generated architectural blueprint for a paradigm shift from artisanal video editing to procedural asset generation. Treats video as a compilation target defined by code (React/TypeScript/Python), enabling content generation at CI/CD speed rather than human editing speed.

Architecture

I. Code-as-Video Engine (Remotion & Revideo)

Framework decision matrix:

FrameworkBest ForMode
RemotionData-driven content (charts, code walkthroughs)React-based production
RevideoHeadless CI/CD rendering (Motion Canvas fork)Automated nightly builds
Motion CanvasDesign-time iteration (timeline scrubbing)Interactive development

Key paradigm: Videos defined as React components. A change in theme.ts propagates to all intro/outro assets across the organization.

II. Hybrid Assembly Pipeline (Kdenlive & Blender)

  • Kdenlive as scriptable assembly station — Python generates .kdenlive XML project files, pre-populating timelines with Remotion-rendered assets
  • Blender as physics engine — headless bpy execution for 3D asset rendering (lossless EXR), imported into Kdenlive for final assembly
  • Whisper integration for AI-driven subtitle generation directly in timeline

III. Adversarial Training Data Flywheel (RSI Integration)

Strategic differentiator: the video pipeline manufactures training data for the local AI system.

  1. Generate — Render videos, 50% clean, 50% with programmatic corruption (Gaussian noise, pixel shifts, bitrate starvation)
  2. Manifest — Generate manifest.json answer key with ground truth
  3. Co-Training — Feed to local AI; failures to detect known corruption trigger targeted training updates

IV. Advanced Geometry

  • Dynamic SDFs (Signed Distance Fields) — geometry defined as GLSL math functions instead of heavy 3D meshes, enabling audio-reactive morphing that’s version-control friendly (KB vs GB)
  • CRDTs for local-first collaboration — mathematical convergence over centralized locking
  • dr-zero-local — The local AI system that consumes adversarial training data from this pipeline
  • procedural-generation-workflows — Multi-tier procedural generation patterns for treating creative assets as compilation targets

Graph View

Backlinks