AI Video Studio with Claude Code + Remotion
🛠 Build it: 1. Setup & your first skill · 2. The script-writing skill · 3. The editing skill · 4. The local AI story skill · 5. The comedy script skill
🎬 Use it: Talking-head video · AI story video · Deadpan comedy video
🔬 Under the hood: Architecture · Subtitles · Jump cuts · Music & SFX · Local AI pipeline
This guide is for using the studio, not building it. You have the project (from the build series, or cloned from the repository), and you want to make a 30–45 second talking-head video: you on camera, telling a short story around a saying, with jump cuts, animated subtitles, music and sound effects.
The whole process is three steps, and you're only needed for the middle one:
/tiktok-script "<a saying>" → you film 4–6 short clips → /tiktok-video <slug>
~1 minute ~20 minutes a few minutes
Before your first video: one-time setup
brew install ffmpeg
cd remotion && pnpm install && cd ..
claude # start Claude Code in the project root
The first edit downloads the speech-recognition engine and its model (about 1.5 GB) into .whisper/. It's reused by every video after that, so do your first run on a good connection.
Step 1: Get a script
Type /tiktok-script followed by what the video is about:
/tiktok-script "Nhân chi sơ, tính bản thiện. Không cà khịa, rất ngứa mồm."
You can give it any of these, and you can add plain-language instructions after the quote:
| You give | What happens |
|---|---|
| A proverb or saying | The most common input. It's read word for word in the punchline, then twisted |
| An opinion | "Đi làm sớm 30 phút là để có 30 phút yên tĩnh" — treated the same way |
| Just a topic | /tiktok-script deadline — the skill coins a short saying and tells you it did |
| Several quotes | It picks the sharpest one and writes one script, not a mashup |
| Extra instructions | /tiktok-script "…" — 40 seconds, filmed in a café, from a team lead's point of view |
Defaults if you don't say: 38 seconds, filmed at a desk with a computer, from the point of view of someone who works in IT or an office.
A few seconds later you get the script as a table — time, what to do on camera, what to say, and the text that appears on screen — and three files in a new folder such as content/2026-08-15-21-54-nhan-chi-so-tinh-ban-thien/:
| File | What it's for |
|---|---|
script.json | The script itself. The editor reads this. |
script.md | The same script as a readable table |
shoot.md | Open this one when you film. |
Changing the script
Just ask in the same conversation: "make the story about a client instead of a coworker", "the hook is too long", "make the CTA a question about weekends". The skill edits script.json, re-checks it, and regenerates both Markdown files. Never edit script.md or shoot.md directly — they're regenerated from the JSON and your edits would be lost. If you prefer to edit the JSON yourself, re-run the check and the generator afterwards:
S=.claude/skills/tiktok-script/scripts
node $S/validate-script.mjs content/<dir>/script.json && node $S/build-docs.mjs content/<dir>/script.json
The checker will tell you if you've made a scene too dense to say comfortably (it counts words per second), broken the timing, or highlighted a word that isn't in the on-screen text.
Step 2: Film
Open shoot.md. It's written to be followed with a phone in your hand.
Setup (five minutes, once per session)
- Phone vertical and fixed — a tripod, a clamp, or a stack of books. Handheld shake is obvious.
- Lens at eye level or slightly above, 50–70 cm from your face.
- 4K if your phone offers it. The editor zooms in slightly at every cut to fake a camera change, and needs the extra pixels.
- Beauty filter off. Lock exposure and focus by holding your finger on your face on screen.
- Light in front of you — face a window. Never sit with a window behind you.
- Lapel mic clipped below the collarbone, 15–20 cm from your mouth. Close windows, turn off fans and air conditioning. Airplane mode on.
- Record a 5-second test and play it back. The most common way to lose a session is filming everything with a disconnected microphone.
The one rule that makes automatic editing possible
One clip per scene, named by the scene's id. shoot.md lists them — s1, s2, s3… — and when you're done, the best take of each goes into the video's raw/ folder:
content/2026-08-15-21-54-nhan-chi-so-tinh-ban-thien/raw/
├── s1.mov
├── s2.mov
├── s3.mov
├── s4.mov
└── s5.mov
.mov and .mp4 both work. Keep spare takes in raw/takes/; the editor ignores that folder.
Filming tips that make a real difference
- Film the hook last.
shoot.mdputs it at the end of the order. By then you've told the story and you're warmed up. - One second of silence at each end of every clip. Press record, wait a beat, speak, wait a beat, stop. The editor trims these automatically.
- Don't memorize. Read the lines two or three times, then say them in your own words, glancing at the cue card — the three to five short phrases
shoot.mdgives for each scene. Write them on a sticky note next to the lens. Small stumbles are fine; they're what makes it sound real. - Look at the lens, not the screen.
- Speak about 20% louder than feels natural. Video always sounds flatter than the room.
- Do two or three takes per scene. The first is almost always a warm-up.
- Change the angle slightly between scenes — turn your body 15° or move the phone a little closer.
The subtitles will always show the script's exact wording, correctly spelled, even when you paraphrase. Paraphrasing a lot just makes the subtitle timing less precise.
Step 3: Edit
Copy the clips into raw/, then in Claude Code:
/tiktok-video nhan-chi-so-tinh-ban-thien
You can type just the end of the folder name; the skill finds the newest folder that matches. It runs six stages in order and stops at the first problem:
| Stage | What it does |
|---|---|
| Preflight | Checks every clip exists, has sound, and is vertical |
| Extract audio | Pulls out the voice and evens out the volume between clips |
| Transcribe | Finds when each word is spoken (text still comes from your script) |
| Detect cuts | Finds the silences to cut |
| Build props | Lays everything out on one timeline; creates any missing music or sound effects |
| Render | Produces out/<folder>.mp4 |
A 40-second video takes a few minutes, mostly rendering. The finished MP4 is 1080×1920 at 30 fps: jump cuts, a slight zoom change at each cut, a big title at the start of each block, word-by-word subtitles with the spoken word highlighted, a progress bar, background music and sound effects.
Read the report
When it finishes, the skill tells you three kinds of things worth paying attention to:
| The skill says | Meaning | What to do |
|---|---|---|
| "match below 70%" for a scene | You strayed far from the script there; subtitle timing may slip | Usually fine. If the timing looks off, edit that scene's voiceover to what you actually said, then re-run transcribe.mjs <slug> --force, build-props.mjs and render.mjs |
| "cut more than 50%" of a scene | The mic was too quiet, so speech looked like silence | Re-run cuts with a lower threshold (below) |
| "fell back" | Subtitles are spelled right but evenly spaced, not following your voice | Check that scene's audio — wrong file, very noisy room |
The finished video will almost never be exactly the length the script planned. That's normal; the timeline follows your real footage.
Fine-tuning
You don't need to start over to fix something. Each stage can be re-run on its own:
S=.claude/skills/tiktok-video/scripts
# Jump cuts too aggressive (words clipped) or too gentle:
node $S/detect-cuts.mjs <slug> --silence-db=-40 # quiet mic: treat more as speech
node $S/detect-cuts.mjs <slug> --pad=0.15 # words sound clipped at the edges
node $S/detect-cuts.mjs <slug> --min-silence=0.5 # edit feels choppy
node $S/detect-cuts.mjs <slug> --no-cut # keep every clip intact
# then rebuild and render:
node $S/build-props.mjs <slug> && node $S/render.mjs <slug>
If only one cut is wrong, open content/<dir>/cuts.json. Each scene lists the segments kept, in seconds of the original clip:
"s3": { "durationSec": 11.2, "keep": [[0.82, 4.18], [4.62, 10.28]], "removedSec": 1.98 }
Change one number, then run build-props and render again. No other stage needs re-running.
To scrub through the result before rendering, open the preview:
node .claude/skills/tiktok-video/scripts/render.mjs <slug> --studio
Music and sound effects
The script picks a background track (lofi by default) and sound effects per scene. They're generated by code the first time they're needed and saved in content/_assets/, so you own them outright — no licensing, no copyright claims. They sound like a decent synthesizer, not a band.
To use your own music instead, put an MP3 with the same name in content/_assets/bgm/ (for example lofi.mp3). Your file always wins; the generator never overwrites an existing file. To see what's available or regenerate one:
node .claude/skills/tiktok-video/scripts/audio-synth.mjs --list
node .claude/skills/tiktok-video/scripts/audio-synth.mjs lofi --force
Common problems
| Problem | Fix |
|---|---|
| "file HAS NO AUDIO" | The mic wasn't connected. That scene must be re-filmed |
| "missing footage for scene: s3" | Name the file exactly raw/s3.mov (or .mp4) |
| "filmed in LANDSCAPE" | Re-film with the phone vertical; cropping a landscape clip to 9:16 loses most of the frame |
| The first run is very slow | It's downloading the 1.5 GB model; later runs reuse it |
| You want a draft before downloading the model | node $S/transcribe.mjs <slug> --fallback, then continue — subtitles spelled right, evenly spaced |
| Black video, no footage | Everything was cut as silence. Re-run cuts with --silence-db=-40 or --no-cut |
| Something stranger | Run each stage by hand and read its output. Delete the intermediate files to start clean: rm -rf content/<dir>/{media,timing,cuts,props}.json content/<dir>/audio |
A dry run before you film
Want to see what the finished video will look like before setting up the camera? Generate fake clips — colored backgrounds with a tone where your voice would be — and run the whole edit:
S=.claude/skills/tiktok-video/scripts
node $S/make-test-footage.mjs <slug>
node $S/preflight.mjs <slug> && node $S/extract-audio.mjs <slug> && node $S/transcribe.mjs <slug> --fallback
node $S/detect-cuts.mjs <slug> && node $S/build-props.mjs <slug> && node $S/render.mjs <slug>
This overwrites raw/. Never run it after you've copied your real clips in.
Posting
script.md has the post caption and hashtags the skill wrote with the script. Copy them, upload out/<folder>.mp4, and you're done.
How it all works under the hood is covered in the build series (script skill, editing skill) and the deep dives on subtitles and jump cuts.

