What is a Deepfake and How Are They Made?

You may have seen a video of Nicholas Cage’s face transposed onto another character doing the rounds on the internet recently. No, we don’t mean a clip from the movie Face/Off. These are scenes from various movies in which Nicholas Cage did definitely not play, yet they look very convincing indeed. How did they pull this off?

Well, the answer isn’t “witchcraft”, but a technology that has been dubbed “deepfake” and it’s causing quite a ruckus in various industries and communities.

What is a “Deepfake”?

The word “deepfake” comes from the term “deep learning” and of course “fake”. Deep learning is a specialized branch of machine learning,  which is again part of the overall area of Artificial Intelligence.

With the dramatic rise in computing power and newly-discovered ways for computers to process and analyze massive amount of data from the real world, computers can now suddenly do things most of us could never imagine. Deepfakes apply this technology to synthesize human images, creating photos or videos of things those people never did and never said. 

How Does Deepfake Technology Work?

Deep learning, which underpins deepfake methods, describes the modern application of neural net simulation to massive data sets. Neural nets are not a new concept or technology, but until modern times they have been pretty rudimentary.

An artificial neural net simulates the learning process that happens in biological brains, at least to some extent. When you learn or otherwise deal with the outside world, the connections between your brain cells change.

They form circuits and logical structures, strengthening some connections and weakening others. As you master something, like learning to drive or play tennis, those brain circuits become fast and efficient. Eventually you are so good at something that you don’t even have to think about doing it.

That’s essentially the same thing that happens with a deep learning system. It looks at heaps of examples of something and then becomes progressively better at “understanding it”.

In the case of deepfakes the software looks at examples of the face you want to transpose as well as the video you want to transpose it to. With enough training it can eventually synthesize a face that matches the training data and then seamlessly overlay it on any other face.

What Software is Used to Make Deepfakes?

There are a number of applications that allow people to make deepfakes. FakeApp was the first app we know of aimed at giving normal people a shot at making deepfake content. The website it now defunct and finding a copy is not easy at all.

Makers of deepfakes have now largely moved on to an application called DeepFaceLab, which is hosted on GitHub and has spawned an endless stream of tutorials on places like Reddit.

Making a Deepfake

This article is not meant to be a tutorial, so we are going to provide an overview of how deepfakes are made in practice, but not provide exact steps on how to make one yourself.

There are a few reasons for this, but the main one is that the legality of making deepfakes is highly-contentious. Just as we wouldn’t provide exact steps to help you pirate software or do other illegal activities, we won’t give step-by-step instructions for making deepfakes.

Besides, the actual creator of DeepFaceLab has released a step-by-step video tutorial for the software which anyone can follow along, should they wish to take responsibility for such a risk.

To get a gist of how good this stuff has become, check out this website where a new photograph of a person who does not exist is generated when you refresh the page.

Our goal is to help you understand the technology, since you are bound to encounter it more and more as time goes by. With that being said, these are the broad phases of creating a deepfake with DeepFaceLab.

After downloading and unzipping DeepFaceLab you are faced with a folder containing many other folders and a series of batch files.

There’s a folder called “workspace” which contains the training models, the source videos and the output. DeepFaceLab works with specific file names and locations, so that the batch files can work. For example, the source file is always named “data_src” and the destination is named “data_dst”.

There is no software interface as most people know it. Just a list of numbered batch files representing the steps of the process. First the source and destination videos have their frames extracted as picture files.

Then several analysis steps are run, followed by GPU-based training, where the neural net learns what it needs about the faces in the two videos to build a model. Finally a new composite video is produced.

Misuse of Deepfakes

As we mentioned above, deepfakes are very controversial. It may violate existing laws in some countries and new laws are in the works to deal with the technology and its applications.

Deepfakes can be used to create hoaxes, such as a video of a president saying crazy things. It can be used to insert people into pornographic films with the purpose or harassing or otherwise harming them.

As you can probably imagine, you could do a lot of damage with a convincing deepfake and the consequences if you are caught may become much more severe in the near future.

Question Everything

Now that this technology exists and works as well as it does, it means we have to look at media like videos in a whole new light. If someone circulates a video on social media of a famous person or politician saying or doing something controversial, you’ll first have to ask if the video is even real.

Most poorly-done deepfakes are obviously fake, once you know what to look for and have seen a few. However, in some cases even a trained eye may have trouble telling that some sort of CG manipulation is happening and as the tech improves it will eventually become impossible.