DeepTomCruise breakdown

Chris Ume, with the help of Tom Cruise impersonator Miles Fisher, created highly believable deepfakes of Tom Cruise and posted the videos to TikTok. Ume showed the breakdown of the arduous process of training the A.I. model and editing each frame.

The Verge talked to Ume more about the process:

“You can’t do it by just pressing a button,” says Ume. “That’s important, that’s a message I want to tell people.” Each clip took weeks of work, he says, using the open-source DeepFaceLab algorithm as well as established video editing tools. “By combining traditional CGI and VFX with deepfakes, it makes it better. I make sure you don’t see any of the glitches.”

The results are both entertaining and worrisome.

Tags: , , ,

Making a convincing deepfake

For MIT Technology Review, Karen Hao looks into the process of artists Francesca Panetta and Halsey Burgund to produce a deepfake of Richard Nixon reading an alternate history of the moon landing:

This is how Lewis D. Wheeler, a Boston-based white male actor, found himself holed up in a studio for days listening to and repeating snippets of Nixon’s audio. There were hundreds of snippets, each only a few seconds long, “some of which weren’t even complete words,” he says.

The snippets had been taken from various Nixon speeches, much of it from his resignation. Given the grave nature of the moon disaster speech, Respeecher needed training materials that captured the same somber tone.

Wheeler’s job was to re-record each snippet in his own voice, matching the exact rhythm and intonation. These little bits were then fed into Respeecher’s algorithm to map his voice to Nixon’s. “It was pretty exhausting and pretty painstaking,” he says, “but really interesting, too, building it brick by brick.”

Sounds like a lot of work, luckily.

Tags: , ,

Text-to-speech models trained on celebrity voices

The Vocal Synthesis channel on YouTube trains text-to-speech models using publicly available celebrity voices. Then using this new computer-generated voice, the celebrities “recite” various scripts. For example, the above is Jay-Z rapping the “To be, or not to be” soliloquy from Hamlet, but it’s not him.

Find out more about the voice generation here, which was developed in 2017. Maybe more interesting, Jay-Z recently filed a copyright claim against the videos.

Tags: , , ,