H3 video editing: what works, what breaks

Object removal, general instruction-based editing and character swap, tested on our own 15 s, 768p clips.

Edit type

Slides

What we see

Small, local edits

Replace an object, change clothes or a sign, swap one person in a still shot.

21 · 22 · 32 · 37

Long clips lose the edit

Removed hands and cars return after 7 to 12 s, swaps start late, the rest of the scene drifts.

8 · 10 · 25 · 39

Global changes

Night to day or a new style: either the whole clip changes, or the style stays as it was.

27 · 28 · 29

Sound is never edited

A removed man keeps talking; new lines and sounds never appear; some voices sound wrong.

11 · 34 · 35

FastH3 holds the edit best

Swaps and rewrites from the first frame more often than base H3 or Pika: right in ~9 of 18 hard swaps.

33 · 40

Our first removal LoRA

Our very first finetune for mask-based removal helps in some cases, but there is a lot more to improve if we go in this direction.

12 · 13 · 15

Part 1

Object removal

Every removal clip: source, base H3, FastH3, Pika H3 Flex. 15 s, 768p.

Open on Drive: 22 clips, base H3, FastH3, Pika H3 Flex

Part 1

Our removal LoRA

Base H3 vs our LoRA, top row exact silhouette mask, bottom row box mask. 5 s, 480p.

Open on Drive: all 22 clips, base H3 vs our LoRA, exact and box masks

Part 2

General editing

Every edit: source, base H3, FastH3, Pika H3 Flex. 15 s, 768p.

Open on Drive: 23 edits, base H3, FastH3, Pika H3 Flex

Part 3

Character swap

Every swap, reference picture in the corner of the source. 15 s, 768p.

Open on Drive: 18 swaps, base H3, FastH3, Pika H3 Flex

Character swap

One clear, repeated failure: the new person often appears only late in the clip.

Object removal

The largest use case. Our very first mask-based LoRA helps in some cases, with much left to improve.

General editing

Fails in many different ways: global changes, audio, drift.

Voice stays

The person is gone from the picture, their lines are still heard.

Object stays or returns

Hands and cars come back after 7 to 12 s; a person stays in a mirror.

Wrong change

A different object goes, or the rest of the scene changes.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>. In the target video, the dog playing on the lawn (the ring toy stays) is removed completely. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

Base H3 and FastH3 remove the dog cleanly; with Pika the ring toy keeps flying on its own.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>. In the target video, the bookseller (the older woman) is removed completely, and so is the bookseller's voice and her lines. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

She goes, but her hands stay or come back (FastH3 from 11 s, Pika from 12 s), and her lines are still heard.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>. In the target video, the woman, and so also her reflection in the mirror is removed completely. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

The reflection is the hard part: base H3 and FastH3 remove only the reflection, Pika neither.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>. In the target video, the leading drifting car, with its smoke is removed completely. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

Base H3 brings both cars back after 7 s, FastH3 removes both; only Pika keeps the chase car.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>. In the target video, the man in the green shirt sitting in the middle is removed completely, and so is the man's voice and his lines. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

The man is deleted, but his voice remains

In all three, his lines are still heard. Play with sound.

What we trained

A LoRA (rank 32) on H3’s attention layers. H3 gets the clip with the object’s area blacked out, the mask as a second video, and a fixed “fill with background” prompt.

On what data

About 1,700 synthetic pairs: clean stock clips with holes cut in them, shaped like people, cars and animals from other clips, or brush blobs.

How long, on what

900 steps (of 2,000 planned), about 2 days on 4 B200 GPUs. 480p, 5 s clips.

How we tested

The same 5 clips, against base H3 with exactly the same input. Each with two masks: the object’s exact silhouette, and a box around it.

All 22 clips on Drive, both masks

Prompt · the same for every clip and mask

“The target video is <Video 1> with its black masked region filled in. In every frame, the region marked white in <Video 2> is filled by extending the surrounding background, so the scene looks natural and complete, with no new objects. Everything outside the mask stays exactly as in <Video 1>, frame by frame, with the same timing.”

What to see

With the box, our LoRA leaves a clean lawn; base H3 paints the dog back or leaves the box.

Prompt · the same for every clip and mask

“The target video is <Video 1> with its black masked region filled in. In every frame, the region marked white in <Video 2> is filled by extending the surrounding background, so the scene looks natural and complete, with no new objects. Everything outside the mask stays exactly as in <Video 1>, frame by frame, with the same timing.”

What to see

Our LoRA empties the shelf with either mask; base H3 paints a new woman or leaves a white box.

Prompt · the same for every clip and mask

“The target video is <Video 1> with its black masked region filled in. In every frame, the region marked white in <Video 2> is filled by extending the surrounding background, so the scene looks natural and complete, with no new objects. Everything outside the mask stays exactly as in <Video 1>, frame by frame, with the same timing.”

What to see

Only our LoRA with the box removes her; with the silhouette, both paint her back.

Prompt · the same for every clip and mask

“The target video is <Video 1> with its black masked region filled in. In every frame, the region marked white in <Video 2> is filled by extending the surrounding background, so the scene looks natural and complete, with no new objects. Everything outside the mask stays exactly as in <Video 1>, frame by frame, with the same timing.”

What to see

With the silhouette, both paint the car back; with the box, our LoRA shows the empty track.

Prompt · the same for every clip and mask

“The target video is <Video 1> with its black masked region filled in. In every frame, the region marked white in <Video 2> is filled by extending the surrounding background, so the scene looks natural and complete, with no new objects. Everything outside the mask stays exactly as in <Video 1>, frame by frame, with the same timing.”

What to see

Still failing with both masks. In part 1, text-only editing did remove him from the picture.

What improved

With a box mask, our LoRA removes cleanly in 4 of the 5 clips shown. Base H3 with the same input removes in none: it paints the object back or leaves the box.

What still fails

The dinner table fails with both masks. With the exact silhouette, objects still come back. The sound is not touched, so removed people keep talking.

How small this run was

900 steps, about 2 days on 4 GPUs, synthetic data only, 480p and 5 s. It also needs a mask, where H3 and Pika take only text.

Works

Replace an object, change clothes or colour, a new location, sign text.

Breaks

Night to day, snow, style; the rest of the clip drifts; audio edits never happen.

Side effects

Fast methods mix up who speaks; voices sometimes sound wrong.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the smartphone in her hand becomes a yellow banana. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

All keep her hand movements; Pika’s banana turns back into a phone at the end, and base H3’s voice sounds strange.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the dog becomes an orange tabby cat. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

All three replace the dog and keep its path around the ring toy.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the man's grey hoodie becomes a brown leather jacket. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

All three change the hoodie and keep both people moving and talking as before.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the bookseller's grey hair becomes dark brown. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

The hair changes in all three, but she moves her mouth where she should be silent.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: a golden retriever walks slowly across the office floor in the background, from left to right. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

In a mostly still scene the dog is added and the rest stays.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: a deer stands on the grass at the roadside in the foreground, grazing. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

The deer is added, but the train changes and passes twice.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the waterfront boardwalk becomes a wide sandy beach at sunset. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

All three move the walk to a beach and keep the five women’s motion.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the night becomes a bright sunny day. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

Instead of only the light, the whole video changes.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the live-action video becomes a Japanese anime illustration. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

Base H3 and FastH3 stay live action; Pika comes closest, but the man is not in anime style.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the anime episode becomes photorealistic live action. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

The whole clip changes: the start and the shots differ from the source.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the woman puts her cup down and waves cheerfully at someone outside the window. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

She waves in all three, but each result looks unnatural somewhere.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the man peels a carrot with a vegetable peeler instead of chopping vegetables. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

Only Pika shows the carrot, but the knife changes during the video.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the neon sign reads "BAKERY" instead of "TATTOO". Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

All three rewrite the neon sign and keep the shop front and camera drift.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the shop sign reads "PICSART" instead of "GAP". Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

FastH3 shows PICSART from the first frame; base H3 and Pika start as GAP and change later.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the man in the green shirt answers "Sure, I'll cook on Friday." instead of "Not me. I did the last two.". Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

None says the new line. Play with sound: each result plays in turn.

Prompt · base H3 and FastH3

“The target video is an edited version of <Video 1>: the train sounds a long horn blast, and birds sing. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

No horn, no birds, and the picture changes too. Play with sound.

Late swap

Base H3 and Pika start with the original person and switch midway.

FastH3 ~9 of 18

Swaps from the first frame, even from a character sheet.

Still hard

Keeping dance moves, two people touching, extra changes.

Prompt · base H3 and FastH3 · Picture 1 = inset

“The target video is an edited version of <Video 1>: the man in the green shirt is replaced by the man in <Picture 1>. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

One person in a still shot: all three swap him cleanly.

Prompt · base H3 and FastH3 · Picture 1 = inset

“The target video is an edited version of <Video 1>: the bookseller is replaced by the woman in <Picture 1>. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

All three swap her in, but with FastH3 and Pika the man’s lines come from the new woman.

At 1 s: all three still show the original woman

At 13 s: now the man from the reference

Prompt · base H3 and FastH3 · Picture 1 = inset

“The target video is an edited version of <Video 1>: the young woman with long dark hair and large glasses, in the shots where she appears is replaced by the person in <Picture 1>. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

Prompt · base H3 and FastH3 · Picture 1 = inset

“The target video is an edited version of <Video 1>: the young man with short dark hair in the black shirt is replaced by the person in <Picture 1>. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

FastH3 swaps from the first frame and keeps the tears; base H3 and Pika swap late.

Prompt · base H3 and FastH3 · Picture 1 = inset

“The target video is an edited version of <Video 1>: the young woman with an afro in the orange top is replaced by the person in <Picture 1>. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

The person is swapped, but the dance is a different one: the motion is not kept.

Prompt · base H3 and FastH3 · Picture 1 = inset

“The target video is an edited version of <Video 1>: the young woman with short brown hair and round glasses is replaced by the person in <Picture 1>. Everything else in <Video 1> stays exactly as it is, frame by frame, with the same timing.”

What to see

Close contact breaks the swap: no method makes a clean one.

Area

What exists

The gap

General editing

OpenVE-3M: 3M synthetic pairs, 720p, 8 edit types; older sets (Señorita-2M, Ditto-1M, InsViE-1M) are smaller in scope.

About 5 s at most, no sound, quality capped by the models that made the edits.

Object removal

VPData: 392K real clips with masks, no clean target. ROSE: rendered pairs with shadows and reflections.

No real paired video of the same scene with and without the object; ROSE is non-commercial.

Character swap

Phantom-Data and OpenS2V-5M: a reference image and a video of the same subject.

No full-body swap dataset; paired swaps would have to be made.

Benchmarks

OpenVE-Bench: 431 pairs in 8 types, the one recent papers report on.

None checks sound or goes to 15 s, the two places H3 failed most.