The Technology Behind “Deepfakes”: How Video’s Most Controversial New Trend Works

“Deepfake” has become the most common new term for an artificial intelligence process that can digitally swap one face for another in just about any video. Naturally, the first area this technology found widespread use was in porn.

The name stems from Reddit user “deepfakes,” who appears to have developed the basic AI algorithm that powers the process. The user first started posting about their algorithm in December of 2017, using altered porn videos as their proof of concept. The first deepfake videos overlaid the faces of female celebrities like Gal Gadot, Kristen Bell and Megan Fox on the performers. The result was far from perfect, but still lifelike enough to be impressive. Someone viewing a low-resolution video or not paying close attention might be convinced that the video was real. Plenty of non-pornographic examples shortly followed on YouTube, with Nicholas Cage seeming to be the most popular subject of video memes.

This sort of thing was common with CGI for years, but that required access to costly technology as well as advanced training. What makes deepfakes revolutionary (and frightening) is how easy it to create fake videos with this technique; nearly anybody with a computer can do it.

How Are Deepfakes Made?

The deepfakes algorithm, shared on Github, requires a moderate amount of Python programming knowledge to use. However, shortly after its launch, a piece of Windows software called FakeApp was created. It streamlines the process so that the user needs no programming knowledge to create a video.

The original algorithm had users collect photos of the two performers whose faces they wanted to swap, usually by doing a Google image search for them. After users built an extensive library of facial shots of both subjects, the algorithm would use face detection AI to isolate the faces in each picture and screen out background elements. With this refined database of face images created, it would then match shots in which they have similar facial expressions. The comparisons throughout this whole process, done at the individual pixel level, make them highly accurate.

FakeApp simplifies things even further by asking users to feed it several videos that prominently features each subject’s face. It then captures all the facial data it needs directly from the videos.

Given how easy it is to use FakeApp, the only real limitation at this point is making sure the computer is good enough to render video. The process is very resource-intensive and requires a good GPU and CPU to execute quickly. If you try to create a deepfake on the standard budget laptop, your computer may be tied up with it for days.

Issues With Deepfakes

Technical issues still make it relatively easy to spot a deepfake. They’re often too stiff, move unnaturally, the skin tone doesn’t match the rest of the body, or background artifacts from the source will appear. Occasionally, the face will go entirely blank for a second in areas where the algorithm didn’t have enough facial data to render a match. There’s also a particular problem with matching mouth movements accurately to audio (the trickiest part from a tech perspective). And of course, there’s the inherent inconsistency of someone like Taylor Swift being in a Brazzers video.

The technology will be refined, of course, because it’s too tempting not to. And the more it improves, the more worrying questions emerge. There’s the emotional and reputation damage that fake porn can create, for starters. Perhaps an even more significant consideration is how this technology might be applied to create fake news.

via GIPHY

Some social media platforms have acted surprisingly quickly to head off this potential threat. Reddit has already banned the nascent deepfakes subreddit, as well as any others related to it. Twitter, Imgur, Discord, and Gfycat have also prohibited the sharing of such videos or images from them. Even Pornhub has banned them, though they are still readily available on nearly all of the other major “tube” sites.

However, platforms that are banning deepfakes are thus far focusing on the “involuntary porn” aspect for the most part. What they’ll do about potential “deepfake news” remains to be seen, especially given they have been slow to respond to the less technologically advanced methods of spreading it.

How Much Harm Can Deepfakes Cause?

The implications for both fake porn and fake news are worrying, particularly in conjunction with the new “Photoshop Voiceovers” feature of the Adobe Creative Cloud (which allows voice samples to be rearranged merely by typing). The creation of a realistic fake video, complete with credible phony audio, could soon be available to anyone with a decent enough computer.

Personal and business brand stakeholders should be terrified of the potential threat to their reputation. As a result, I believe that forensic evaluation of videos will become a necessity and we’ll see more private investigators and technologists get into the business of identifying deepfakes.

Deepfake videos currently are still more of an entertaining novelty than anything else, as they’re so easy to spot with just a little scrutiny. The quality is no better than what is achievable with existing CGI and similar tools; it’s just much easier for untrained and unskilled individuals to do. As quality improves, however, so will the potential for significant problems (and legitimate business opportunities).

https://videotoorder.com/blog/markets-that-deepfake-videos-create/