VoiceandFaceCloning

Method

This section is in four parts. Part 1 describes the method most people try first, and why it fails. Parts 2 through 4 describe the method that works. Each part uses the result of the one before it.

Part 1: Deepfake à la Nope

This method calls for a real person who has not agreed to it, as much video of them as can be found, and an AI agent. It does not get past the first step. The steps are listed so that the reader does not waste time on them.

1. Ask the agent to collect every YouTube video of the person.

   I will not collect footage of someone so that they can be impersonated. If you are the person, or have their written permission, I can help.

2. Ask the agent to cut the person’s voice out of each clip and save it as audio files.

   There are no clips. See step 1.

3. Ask the agent to check the audio files for other voices.

   There are no audio files.

4. Ask the agent to generate speech in the person’s voice, saying things they never said.

   This is the step where someone gets hurt. I can help write a parody that is clearly labeled as one.

5. Ask the agent to post the result.

   There is nothing to post.

The result is nothing. This is the correct result.

Part 2: Consent Consommé

A voice clone can only be made from a voice you own or have written permission to use. Every service below makes you prove this, either by verifying the voice or by agreeing that you have permission. Choose one.

1. Hosted services. Upload a short sample, or record one in the browser. A usable clone is ready in minutes.

   • ElevenLabs. Instant clones from about one minute of audio. Professional clones from longer recordings, which require a voice check to confirm the voice is yours.
   • Resemble AI. Fast clones and speech-to-speech conversion. Adds an audio watermark and includes a deepfake detector.
   • PlayHT. Instant and high-fidelity clones, intended for narration and voiceover.

2. Editors. For people who already edit podcasts or video and need to correct a line.

   • Descript. Trains a clone of your own voice. You can then type into the transcript to correct or add words. You must read a consent statement first.

3. Enterprise services, by application. For brands and products that need a custom voice and a paper trail.

   • Microsoft Azure Custom Neural Voice. Access is restricted. You apply, describe how the voice will be used, and the speaker records a consent statement.
   • Google Cloud and Amazon Polly custom voices. Similar. Arranged through sales and contracts.

4. Voice banking on your own device. For people who may lose the ability to speak.

   • Apple Personal Voice. Built into iPhone, iPad and Mac. You read prompted phrases for about 15 minutes. The voice is generated and stored on the device.
   • Voice-banking services such as ModelTalker and Acapela my-own-voice. Intended for people with ALS and similar conditions.

5. Open source. Free and private. Requires a GPU, some setup, and patience. You are responsible for reading the license and for how the voice is used.

   • XTTS-v2 (Coqui). Clones from a few seconds of audio, in many languages.
   • OpenVoice. Copies tone color from a short clip, with control over style.
   • F5-TTS. Good zero-shot quality from short reference clips.
   • RVC (Retrieval-based Voice Conversion). Speech-to-speech and singing conversion. Train it on your own recordings.

Wherever the cloned voice is used, label it as AI-generated.

Part 3: Talking Head Cheese

This part makes a short video of you speaking, from one photograph and the voice from Part 2. MiniMax H3 generates the picture, the sound and the mouth movement together, so no separate lip-sync step is needed. It takes about 30 minutes, plus render time.

1. Choose a photograph. It should be sharp and well lit, facing the camera, with the mouth closed and nothing covering the face. Crop it to the shape of the finished video: 9:16 for phones, 16:9 for everything else.

2. Make a voice reference. H3 does not load a trained voice. It copies the sound of a short clip. Use the clone from Part 2 to generate 10 to 15 seconds of clean speech, with one speaker and no music. Export it as a WAV or MP3 file under 15 MB. Keep this file for Part 4.

   • The clip should say different words from the script. The clip sets the sound of the voice. The words come from the prompt.
   • A clean recording of your actual voice also works.

3. Write the prompt. Each clip is 4 to 15 seconds long, which is one or two sentences. Put the exact words in quotation marks and describe how they are said. For example:

   Image 1 is the first frame. The person speaks to camera in the voice from Audio 1, warm and a little amused: "This whole video was made from one photo and fifteen seconds of my voice." Soft room tone, no music.

4. Generate the video. In H3, either start an image-to-video job with the photograph as the first frame, or start a reference-to-video job with the photograph as Image 1 and the voice clip as Audio 1. Use 768P for drafts and 2K for the final version.

5. Check the result at full size. If the voice is wrong, use a cleaner or longer reference clip. If the mouth is blurred, use a sharper photograph or a slower line. Generate it again rather than trying to repair the output.

6. For longer videos, join several clips in a video editor. Add captions. Label the video as AI-generated.

Show it only to people who know it is you.

Part 4: Remixed Leftovers

This part puts new words into existing video of you. Because it starts from real footage, the movement, framing and camera work are already done. It takes about 30 minutes, plus render time.

Each H3 job accepts up to 3 video clips, 3 audio clips and 9 images, with no more than 12 files in total. Each video or audio clip must be 2 to 15 seconds long, and the clips of each type must not add up to more than 15 seconds.

1. Choose a clip. Your face should be visible, the light steady, and the background simple. Trim it to 15 seconds or less. If you are recording a new clip, look at the camera and talk normally. The words do not matter. The model needs real mouth movement to work from.

2. Prepare the audio. Use the voice reference from Part 3. For an exact reading of the new line, also generate the line itself with the clone from Part 2 and save it as a separate file.

3. Choose one of the following. Each is a single H3 reference-to-video job. Refer to the files in the prompt as Video 1, Image 1 and Audio 1.

   • To change the words, add the clip as Video 1 and the new line, spoken in your voice, as Audio 1:

     Keep the person, framing and camera from Video 1. Replace what they say with the line in Audio 1, and adjust the mouth and expression to match.

   • To change the setting, add the clip as Video 1, a photograph of yourself as Image 1 to keep your face consistent, and the voice reference as Audio 1:

     The person from Image 1 makes the same movements and camera moves as Video 1, now at a kitchen counter at night. They say in the voice from Audio 1: "This was filmed somewhere else."

   • To change the style, add the clip as Video 1 and the voice reference as Audio 1. Describe the new look in the prompt, for example a film stock, a lighting setup or an illustration style.

4. Generate the video. Use 768P for drafts and 2K for the final version. Each job produces 4 to 15 seconds of video.

5. Check the result. If your face changes, add one or two more photographs of yourself as Image 2 and Image 3. If the timing is off, trim the clip closer to the length of the line. If the camera moves too much, write “locked-off camera” in the prompt.

6. Join the clips, add captions, and label the video as AI-generated.