--- url: https://docs.seen.io/platform/projects/ai-voices.md description: >- Covers recording or designing an AI voice clone, preparing training samples, and using personalised voiceover with properties in Studio. --- # AI Voices AI Voices lets you clone your own voice, or design a synthetic one, for use in personalised video voiceovers in Studio. ## Recording the voice Follow these guidelines to maximise the quality of the AI voice: ::: details Record at least 1 minute of audio Avoid recording more than 3 minutes, this will yield little improvement and can, in some cases, even be detrimental to the clone. How the audio was recorded is more important than the total length (total runtime) of the samples. The number of samples you use doesn't matter; it is the total combined length (total runtime) that is the important part. Approximately 1-2 minutes of clear audio without any reverb, artifacts, or background noise of any kind is recommended. When we speak of "audio or recording quality," we do not mean the codec, such as MP3 or WAV; we mean how the audio was captured. However, regarding audio codecs, using MP3 at 128 kbps and above is advised. Higher bitrates don't have a significant impact on the quality of the clone. ::: ::: details Keep the audio consistent The AI will attempt to mimic everything it hears in the audio. This includes the speed of the person talking, the inflections, the accent, tonality, breathing pattern and strength, as well as noise and mouth clicks. Even noise and artefacts which can confuse it are factored in. Ensure that the voice maintains a consistent tone throughout, with a consistent performance. Also, make sure that the audio quality of the voice remains consistent across all the samples. Even if you only use a single sample, ensure that it remains consistent throughout the full sample. Feeding the AI audio that is very dynamic, meaning wide fluctuations in pitch and volume, will yield less predictable results. ::: ::: details Replicate your performance Another important thing to keep in mind is that the AI will try to replicate the performance of the voice you provide. If you talk in a slow, monotone voice without much emotion, that is what the AI will mimic. On the other hand, if you talk quickly with much emotion, that is what the AI will try to replicate. It is crucial that the voice remains consistent throughout all the samples, not only in tone but also in performance. If there is too much variance, it might confuse the AI, leading to more varied output between generations. ::: ::: details Find a good balance for the volume Find a good balance for the volume so the audio is neither too quiet nor too loud. The ideal would be between -23 dB and -18 dB RMS with a true peak of -3 dB. ::: ### How to prepare your voice training samples When creating your custom voice you will need one or several training samples (Recordings). These should be recorded using the same equipment (microphone etc) and on the same set that the rest of the video is. **Must haves:** * Needs to have the same audio grading/mixing that the final video will have. * Needs to have emotional range that reflects the rest of your video, can't be overly monotone. **Nice to haves:** * If the intention is to use AI-voice for name reading, the training data should aim to include a small set of actual name readings, e.g. 5-10 greetings. This helps increase consistency. ::: info Training sample requirements * Accepted file-types: `wav`, `mp3` * Minimum length (per sample): `10 seconds` * Maximum length (per sample): `3 minutes` ::: ### Assuring high quality voice generation The quality of the generated audio is mainly dependent on two factors: * The quality of the original training data * The creative itself (e.g. how the generated audio is implemented in the script) The more weaknesses there are in these two areas, the less life-like the end result will be. For ideal implementation especially the creative/script should be considered closely. ### Language support Audio generation is supported in multiple languages, although quality may vary across. Currently, we have the most confidence in English scripts. ::: details List of supported languages * Arabic (`ar`) * Bulgarian (`bg`) * Chinese (`zh`) * Croatian (`hr`) * Czech (`cs`) * Danish (`da`) * Dutch (`nl`) * English (`en`) * Filipino (`fil`) * Finnish (`fi`) * French (`fr`) * German (`de`) * Greek (`el`) * Hungarian (`hu`) * Hindi (`hi`) * Indonesian (`id`) * Italian (`it`) * Japanese (`ja`) * Korean (`ko`) * Malay (`ms`) * Norwegian (`no`) * Polish (`pl`) * Portuguese (`pt`) * Romanian (`ro`) * Russian (`ru`) * Slovak (`sk`) * Spanish (`es`) * Swedish (`sv`) * Tamil (`ta`) * Turkish (`tr`) * Ukrainian (`uk`) * Vietnamese (`vi`) ::: ::: info A note on languages and QA (Only applicable for managed projects) Using scripts in languages that are foreign to Seen's CS-team will reduce their capability to perform quality assurance. In these cases you as a customer would be informed and invited to participate in the QA to ensure the best possible outcome. ::: ## Cloning the voice ::: info Make sure to have an approval from the voice actor before cloning any voice. ::: ### Creating a custom voice inside of the platform 1. Navigate to **AI Voices** through the left sidebar. 2. Click **Clone Voice** in the top right hand corner. 3. Upload the recorded training samples you want to use to instruct the cloning. Multiple samples can be used for a single cloning. Read more on how to prepare your voice training samples above. ![Add Voice dialog with a drop zone for uploading training samples](/images/ai-voices-upload-samples.png) 4. Add a name, description and tags to your new voice to make it easier to find in the future. ![Add Voice dialog with name, tags, and description fields](/images/ai-voices-add-metadata.png) ::: info You can upload two different audio files with different tonality, for example "conversational" and "energised", and mix between them inside the same Studio project. ::: ## Designing the voice If you are not able to record a voice, but still want a unique voice for your project, you can use our **Design Voice** tool. ### Prompt the voice Give it a name, select a language, and write a description following these guidelines: ![Design Voice dialog with name, language, and description fields](/images/ai-voices-design-voice-prompt.png) * **Be specific:** Include details like age, tone, gender, and accent. For example, instead of "old woman," use "sarcastic old woman with a thick New York accent". * **Combine attributes:** Use multiple traits together for better control (e.g., "A sarcastic old woman with a thick New York accent, speaking slowly"). * **Use emotional tags:** Incorporate emotional tags like `[excited]` or `[calm]` directly into your text to influence delivery. * **Avoid vague descriptors:** Steer clear of imprecise terms like "foreign" or "exotic". * **Specify audio quality:** You can ask for a specific quality, like "studio quality microphone" or "old phone". Select one of the previews and press **Create Voice**. ![Design Voice dialog showing generated preview options](/images/ai-voices-select-preview.png) Your voice is now ready to be used in your next Seen Studio project. ## Using the voice ::: info Use personalised audio as early in the video as possible for a great hook! ::: Your new voice can be found under **Select Voice** in your **Text to Speech** panel on the left side of your Seen Studio panel. ![Text To Speech panel with the Select Voice dropdown open](/images/ai-voices-select-voice-panel.png) Select the correct **language** for your voice. ![Language dropdown in the Text To Speech panel](/images/ai-voices-select-language.png) We have three different models supporting different languages: * **Swift**: the fastest one * **Flexible**: natural tone * **Max**: highest quality ![Model selection showing Swift, Flexible, and Max options](/images/ai-voices-model-selection.png) You can generate a new version of the same text until you are happy with it. To finish, press **Add to Project**. ![Generate a new variation button in the Text To Speech panel](/images/ai-voices-generate-new-variation.png) ## Writing the text Start with a test name, to understand the tone of voice and the length of your AI voice. Use both shorter and longer names, to understand how long your block needs to be. The full sentence will be generated for each variable, so you are only listening to one of the output examples. ### Sentence structure * **Replace periods**: For a more natural and conversational tone, replace periods at the end of sentences with exclamation marks. * **Add emphasis**: For extra emphasis, you can add multiple exclamation marks (e.g., `!!!`). * **Combine with capitalization**: Use capitalization for words you want to emphasize, like `THIS`. * Keep each sentence as short as possible, for example "Hi, {first\_name}", "Hello, {first\_name}" or "Welcome, {first\_name}". * Keep each personalised text as a full sentence, sentences that end with a period or exclamation mark work well. ![Personalised audio tag next to blocks in the timeline](/images/ai-voices-personalised-audio-timeline.png) ### Use it in the timeline Keep enough room in the timeline for longer variables. "Hi, Victoria-Helena" takes longer to read than "Hi, Bob". Keeping enough room before/after a personalised audio will also make it sound more natural between the sentences. ![Personalised audio tag positioned between timeline blocks](/images/ai-voices-personalised-audio-timeline-2.png) ## Use properties To finalize your project, go to the Personalisation panel and **Add existing** or create **New** properties for your project, for example `first_name`. ![Personalisation panel with the first\_name property](/images/ai-voices-personalisation-panel.png) Change your test data with the correct **property**, **generate** and **replace** it in the timeline. ![Text To Speech panel with the first\_name property highlighted in the script](/images/ai-voices-text-to-speech-property.png) Your audio layer will now have a purple outline to indicate that you have personalised the layer. ![Personalised audio layer with a purple outline in the timeline](/images/ai-voices-personalised-layer-outline.png)