VALL E Neural Network

VALL E Neural Network is an Android utility based on a mathematical model from Microsoft. The current version of the algorithm allows to imitate another person's speech quite accurately. A three-second recording is enough to fully train the algorithm.

Working principle

The service is based on a classical artificial intelligence algorithm. The main functionality is tied to the LibriLight library, which uses over 60 hours of speech (English only). Unlike most similar services, the model used for VALL E Neural Network does not use sound waves. It uses speech partitioning into special tokens to identify key features of the speech. The scheme is very efficient, which explains the possibility of training with the help of short recordings. The algorithm can also additionally add different emotions (anger, joy, sadness, and so on) to the speech. It is possible to overlay the sounds of the environment.

Availability

At the time of publication, Microsoft has not yet opened access to the new algorithm. The company motivates this by the fact that the capabilities of the service can be massively used by attackers. For familiarization, users are invited to go to the official page, where demo samples of generated speech are published. This is quite enough to assess the capabilities of the AI.

Features

• The algorithm is able to accurately mimic the speech of a specific person;
• free access to audio demos;
• a 3-second recording is sufficient for input data;
• EnCodec technology was actively used in the process of model creation;
• the ability to add certain emotions to speech.