What is a deepfake and what’s the deepfake technology?
Deep fake (also spelled as deepfake) is a type of AI (artificial intelligence) used to create fake images, audios and videos. You might have encountered a deep fake and not recognised it to be fake. This technology has its roots attached to deep learning. Deep learning is nothing but a subset of machine learning in AI. It is capable of learning data that is unstructured or unlabelled without any supervision.
How is a deep fake made?
There are two ways in which deep fakes can be made:
1. Using GAN
What actually happens in deep fake is it uses two competing AI algorithms.
The generator is responsible for creating phoney multimedia content. This content is further passed to discriminator to determine whether the content is real or artificial. Each time the discriminator identifies as a content as being fabricated, it provides the generator with valuable information about how to improve the next deep fake. For this process to take place accurately, first the generator is trained with the training dataset which would help it identify the desired output.
Once the generator starts creating an acceptable level of output, this output is fed to a discriminator for identifying. Here, as the generator gets better at creating fake content, the discriminator gets better at spotting the areas of improvement. Conversely, as the discriminator gets better at spotting the areas which seem fake and needs improvement, the generator considering the discriminator’s feedback works on it and gets better at creating the content. Together these two algorithms form a generative adversarial network (GAN).
2. Using encoder and decoder
To understand this we’ll take an example of person A’s face being swapped with person B’s face in a video. First thing to do is to run thousands of face shots of person A and B through an AI algorithm called an encoder. The encoder finds and learns similarities between the two faces and reduces them to their common features, compressing the image in the process.
The second AI algorithm is made to recover the faces from the compressed images. Now, because the faces are different, we train one decoder to recover person A’s face and another decoder to recover person B’s face. To perform the face swap, we feed the encoded images to the wrong decoder, i.e. we’ll feed person A’s (compressed) encoded images to the decoder trained on person B’s face. As a result, the decoder reconstructs the face of person B with the orientation and expressions of person A. To apply this on a video, this process has to be done on every frame.
Who can make a deep fake?
However, deep fakes are created through AI, they don’t require any considerable skills which are required to create a real video. This means that anyone can create a deep fake. In fact, there’s even a mobile phone app, Zao, that lets users ass their faces to a list of TV and movie characters on which the system has trained.
The danger which comes with this is anyone can use deep fakes to promote their chosen agenda and people will take it at face value. Hence trusting on the validity of a multimedia content is questionable as the potential for scams is clear.
How to spot a deep fake?
Spotting a deep fake gets harder as the technology improves. Researchers discovered that deep fake faces don’t blink normally and the reason behind this can be understood considering the encoder-decoder method of creating a deep fake, i.e. majority of images which are fed to the encoder show people with their eyes open and so the algorithm never really learn about blinking. But soon as this research was published it was found that deep fakes started appearing with blinking.
Yet there are some poor quality deep fakes which are easier to spot paying attention to minor details like bad lip-syncing, flickering around the edges of transposed faces, to fine details like hair strands especially visible on the fringe. Strange lighting effects like inconsistent illumination and reflections on iris can be an identifying factor as well.