Face and voice tools#

These tools come from plugins you install first:

Plugin	Effects it adds	Systems	Video memory
Face Toolkit	Face Region, Face Swap, Live Portrait	Linux, Windows, macOS	6 GB (also runs on the processor, slowly)
Lip Sync (LatentSync)	Lip Region, Lip Sync	Linux, Windows	12 GB, graphics card required
Voice Toolkit	Clone Voice, Separator	Linux, Windows, macOS	2 GB (also runs on the processor)

How these effects work#

AI models are too slow to run while the timeline plays, so these effects work differently from ordinary ones:

You add the effect to a clip and set it up like any other effect: pick the options, keyframe what should change over time.
While you edit, the effect only holds those settings; the preview shows the clip unchanged.
When you are ready, click Generate into the bin in the effect. The plugin processes the clip in the background; the button shows the progress.
The result is added to the Project Bin as a new clip, and the original clip on the timeline is left as it was. After a change, the button reads Generate again.

Region and effect#

Face and lip tools come in pairs: a region effect that says where (which face, where it is in each frame) and the effect that says what to do there. When you add the effect from a detected face, both are added together and linked, and the region is filled with that face’s position through the clip.

Each keeps its own keyframes, so you can fix the region where face tracking went wrong without touching the expression or voice you set up.

Presets: faces, expressions, voices and audio#

Several effects take a source registered in advance: the photo for Face Swap, a recorded expression for Live Portrait, a voice for Clone Voice, an audio track for Lip Sync. Click the pen next to that field to open the panel that registers a new one. Registered sources are saved with the project, and the panel can export and import them to reuse one in another project.

Faces#

Start in the monitor: Wunjo detects faces in the frame and draws a box around each. Click a box to open its menu, and pick the tool.

Face Swap: Replaces the face in the Face Region with the face from a photo.
Live Portrait: Animates the head in the region with a recorded expression. Record one from a video or a photo with the pen next to Expression source; picking it fills the effect with keyframes, one per frame where the expression changes. On top of that, sliders such as Smile, Wink, Eyebrow and Grin can be keyframed by hand.
Lip Sync (LatentSync): Makes the face in the Lip Region speak a registered audio track. It is slow and needs a large graphics card, and keeps the mouth close to the original.

Face Toolkit skips explicit frames by default. The option is Refuse explicit frames in the plugin’s settings.

Voices#

Right-click a clip that has sound on the timeline and open Artificial Intelligence.

Clone Voice: Says what the clip says in a registered voice. Register a voice from a recording with the pen next to Voice.
Separator: Splits the voice from what is behind it. With Extract voice on, you get the voice alone; off, you get everything except the voice.

Both put what they make in the Project Bin and on an audio track of its own, and leave the clip on the timeline alone.