Repository files navigation

Deep Learning algorithms are being used in several fields nowadays, such as Self-drivingcars, Healthcare or Voice-Activated Assistants. Programs developed with these algorithms aregetting easier for not experienced users to use and their features are constantly growing. Despitethis being positive for contributing to previous sectors, deep learning-powered applications thatcould cause threats in politics, privacy and security are being developed too. An example of thisis the so-called "deep fakes". This research has the goal of introducing this type of algorithms,showing state-of-the-art implementations in the area and examples of what this technology iscapable of.

Below is presented a summary of the actual research, the full paper can be found atDeepfakes: Introduction and Latest Implementation orDeepfakes: Introduction and Applications in Digital Health for a more focused report on Digital Health. Also, here is a video summarizing the research presented, in case you prefer the visual to the textual 😄

Deepfake is a deep learning technique used to create convincing image, audio and video hoaxes.The most common application is to swap a person in an existing image or video with another one,generating a fake synthetic media. The difference with traditional techniques such as Photoshopis that deepfakes consists of powerful techniques for generating visual or audio content with highpotential to deceive. Some of the popular areas where these algorithms are used are fake news,financial fraud, revenge porn and celebrity pornographic videos. As these are critical areas, most ofthe academic research surrounding Deepfake seeks to detect this content [10]. Later some techniquesfor deepfake detection will be presented. According to the legal response, some countries have beenintroducing legislation to tackle deepfake content. For instance, in the United States, some statessuch as California, Texas and New York have introduced "The Malicious Deep Fake ProhibitionAct", which will make a federal crime to create or distribute a deepfake when doing so will facilitateillegal conduct [4].

The technology used by deepfakes algorithms includes previous techniques such as autoencodersand generative adversarial networks. A drawback in these algorithms would be the high quantityof data needed to train models to create realistic synthetic content. For this reason, most of theinitial targets for the deepfakes are public celebrities or politicians, which have a large number ofvideos and images available online. However, these techniques are improving significantly over timeand there are some techniques available to generate deepfake content with few images, through anapproach called few shot learning.

The first deepfake video was produced in 2017 where a popular face was swapped in pornographiccontent and published on Reddit. In 2018, the famous website banned this type of content classifiedas fake porn, which was followed by Twitter or Gfycat [6]. On top of that, these algorithms can beused to threaten the national security of an entire country, for instance publishing a video of thepresident of the United States giving a fake speech [14], which could affect election campaigns ormilitary actions. It could be used as part of social engineering scams through the branch of audiodeepfakes, fooling people into thinking they are receiving instructions from a trusted source. In2019, a UK energy firm CEO was scammed $243, 000 when he was ordered by a supposed chiefexecutive from the parent’s company to transfer the money to a Hungarian bank [5].

Despite all these dark applications of deepfake, there are many positive ones such as creatingvoices for those who have lost theirs, recording the same character in a film through different ages,such as in the Netflix film ’The Irishman’ [16] or updating episodes in movies without shootingthem again [11].

Nowadays there exist multiple open source projects to generate deepfakes that any user with no additional experience in the field could use. After researching the state-of-the-art ones, they have beenrepresented in table 1. The purpose of this list is to enhance efforts into research and developmentof deepfakes and their detection and not to support the creation of nefarious content.

Regarding face swapping, by using the Google Colab demo from the repository Motion Supervised co-part Segmentation [8], the results represented in the figure below were generated. The firstresult was produced through unsupervised segmentation, while the second one was formed usingsupervised part-swaps.

On top of that, below are displayed some examples from the First Order MotionModel for Image Animation Demo [13]. It was made in Google Colab modifying the original versionof the demo and generating deepfakes with a picture of myself as the source image. It was testedin different driving videos, two of them represented in the figure.

The current state in deepfakes is complicated. Every time a new deepfake technology is released,users will try to develop a technology to detect the images generated with that technology, andthen hackers would try to bypass that detection and ideate a new approach for generating them,and this will continue with a virus/anti-virus dynamic. On top of that, despite achieving highaccuracy in deepfake detection, for instance, 99%, sometimes that 1% of no detected deepfakescould compromise users of platforms such as Instagram or Twitter.

Another idea in the area for tackling this issue is to use programs that can automatically watermark and identifyimages taken on cameras or implementing blockchain technology to verify content from trusted sources [15].However, the likely future is that none of these approaches would help ’solve’ this issue, it is an endless competition that can have very critical results in terms of privacy, safety and political concerns. People need to be aware of these technologies as well as verifyeach source they read or watch. Besides, governments should take imminent measures for peoplegenerating or sharing deepfakes for unethical uses.

[1] Deep convolutional generative adversarial network : Tensorflow core.

[2] Agarwal, S., Farid, H., Gu, Y., He, M., Nagano, K., and Li, H. Protecting worldleaders against deep fakes. In Proceedings of the IEEE Conference on Computer Vision andPattern Recognition Workshops (2019), pp. 38–45.

[3] Badr, W. Auto-encoder: What is it? and what is it used for? (part 1), Jul 2019.

[4] Brown, N. I. Congress wants to solve deepfakes by 2020. that should worry us., Jul 2019.

[5] Damiani, J. A voice deepfake was used to scam a ceo out of $243,000, Sep 2019.

[6] Kelion, L. Reddit bans deepfake porn videos, Feb 2018.

[7] Korshunov, P., and Marcel, S. Vulnerability assessment and detection of deepfake videos.In The 12th IAPR International Conference on Biometrics (ICB) (2019), pp. 1–6.

[8] Lathuilière, S., Tulyakov, S., Ricci, E., Sebe, N., et al. Motion-supervised co-partsegmentation. arXiv preprint arXiv:2004.03234 (2020).

[9] Li, Y., Chang, M.-C., and Lyu, S. In ictu oculi: Exposing ai created fake videos bydetecting eye blinking. In 2018 IEEE International Workshop on Information Forensics andSecurity (WIFS) (2018), IEEE, pp. 1–7.

[10] Manke, K., and Manke, K. Researchers use facial quirks to unmask ’deepfakes’, Jun 2019.

[11] Marr, B. The best (and scariest) examples of ai-enabled deepfakes, Jul 2019.

[12] Nguyen, T. T., Nguyen, C. M., Nguyen, D. T., Nguyen, D. T., and Nahavandi, S.Deep learning for deepfakes creation and detection. arXiv preprint arXiv:1909.11573 (2019).

[13] Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., and Sebe, N. First ordermotion model for image animation. In Advances in Neural Information Processing Systems32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds.Curran Associates, Inc., 2019, pp. 7137–7147.

[14] Suwajanakorn, S., Seitz, S. M., and Kemelmacher-Shlizerman, I. Synthesizingobama: learning lip sync from audio. ACM Transactions on Graphics (TOG) 36, 4 (2017),1–13.

[15] Vincent, J. Deepfake detection algorithms will never be enough, Jun 2019.

[16] Wood, C. A deepfake artist’s attempt to make robert de niro look younger in ’the irishman’is being hailed as superior to netflix’s cgi, Jan 2020.

[17] Xuan, X., Peng, B., Wang, W., and Dong, J. On the generalization of gan imageforensics. In Chinese Conference on Biometric Recognition (2019), Springer, pp. 134–141.8

[18] Zakharov, E., Shysheya, A., Burkov, E., and Lempitsky, V. Few-shot adversariallearning of realistic neural talking head models. In Proceedings of the IEEE InternationalConference on Computer Vision (2019), pp. 9459–9468.