1691Accesses
29Citations
Abstract
An upsurge of false information revolves around the internet. Social media and websites are flooded with unverified news posts. These posts are comprised of text, images, audio, and videos. There is a requirement for a system that detects fake content in multiple data modalities. We have seen a considerable amount of research on classification techniques for textual fake news detection, while frameworks dedicated to visual fake news detection are very few. We explored the state-of-the-art methods using deep networks such as CNNs and RNNs for multi-modal online information credibility analysis. They show rapid improvement in classification tasks without requiring pre-processing. To aid the ongoing research over fake news detection using CNN models, we build textual and visual modules to analyze their performances over multi-modal datasets. We exploit latent features present inside text and images using layers of convolutions. We see how well these convolutional neural networks perform classification when provided with only latent features and analyze what type of images are needed to be fed to perform efficient fake news detection. We propose a multi-modal Coupled ConvNet architecture that fuses both the data modules and efficiently classifies online news depending on its textual and visual content. We thence offer a comparative analysis of the results of all the models utilized over three datasets. The proposed architecture outperforms various state-of-the-art methods for fake news detection with considerably high accuracies.
This is a preview of subscription content,log in via an institution to check access.
Access this article
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
Price includes VAT (Japan)
Instant access to the full article PDF.











Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Yang Y, Zheng L, Zhang J, Cui Q, Li Z, Yu, PS (2018) TI-CNN: Convolutional Neural Networks for fake news detection
Wu K, Yang S, Zhu KQ (2015) False rumors detection on sina weibo by propagation structures. In: 2015 IEEE 31st international conference on data engineering, pp 651–662
Ma J, Gao W, Mitra P, Kwon S, Jansen BJ, Wong KF, Cha M (2016) Detecting rumors from microblogs with recurrent neural networks
Ma J, Gao W, Wong KF (2019) Detect rumors on twitter by promoting information campaigns with generative adversarial learning. In: The world wide web conference, pp 3049–3055
Jin Z, Cao J, Zhang Y, Zhou J, Tian Q (2016) Novel visual and statistical image features for microblogs news verification. IEEE Transactions on Multimedia 19(3):598–608
Boididou C, Andreadou K, Papadopoulos S, Dang-Nguyen DT, Boato G, Riegler M, Kompatsiaris Y (2015) Verifying multimedia use at MediaEval 2015. MediaEval 3(3):7
Boididou C, Papadopoulos S, Dang-Nguyen D-T, Boato G, Riegler M, Middleton SE, Petlund A, Kompatsiaris Y et al (2016) Verifying multimedia use at MediaEval 2016. MediaEval
Jin Z, Cao J, Guo H, Zhang Y, Luo J (2017) Multi-modal fusion with recurrent neural networks for rumor detection on microblogs. In: Proceedings of the 25th ACM international conference on multimedia, pp 795–816
Qi P, Cao J, Yang T, Guo J, Li J (2019) Exploiting multi-domain visual information for fake news detection. In: 2019 IEEE International Conference on Data Mining (ICDM), pp 518–527
Chen T, Li X, Yin H, Zhang J (2018) Call attention to rumors: deep attention based recurrent neural networks for early rumor detection. In: Pacific-Asia conference on knowledge discovery and data mining, pp 40–52
Liu Y, Wu YFB (2018) Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. Thirty-Second AAAI Conference on Artificial Intelligence
Wang Y, Ma F, Jin Z, Yuan Y, Xun G, Jha K et al (2018) Eann: event adversarial neural networks for multi-modal fake news detection. In: Proceedings of the 24th acm sigkdd international conference on knowledge discovery & data mining, pp 849–857
Khattar D, Goud JS, Gupta M, Varma V (2019) Mvae: multi-modal variational autoencoder for fake news detection. In: The world wide web conference, pp 2915–2921
Ajao O, Bhowmik D, Zargari S (2018) Fake news identification on twitter with hybrid cnn and rnn models. In: Proceedings of the 9th international conference on social media and society, pp 226–230
Jindal S, Sood R, Singh R, Vatsa M, Chakraborty T (2020) NewsBag: a multi-modal benchmark dataset for fake news detection
Singhal S, Shah RR, Chakraborty T, Kumaraguru P, Satoh SI (2019) SpotFake: a multi-modal framework for fake news detection. In: 2019 IEEE fifth international conference on multimedia big data (BigMM), pp 39–47
Marra F, Gragnaniello D, Cozzolino D, Verdoliva L (2018) Detection of Gan-generated fake images over social networks. In: 2018 IEEE conference on multimedia information processing and retrieval (MIPR), pp 384–389
Sabir E, AbdAlmageed W, Wu Y, Natarajan P (2018) Deep multi-modal image-repurposing detection. In: Proceedings of the 26th ACM international conference on multimedia, pp 1337–1345
Pomari T, Ruppert G, Rezende E, Rocha A, Carvalho T (2018) Image splicing detection through illumination inconsistencies and deep learning. 2018 25th IEEE International Conference on Image Processing (ICIP):3788–3792
Maigrot C, Claveau V, Kijak E, Sicre R (2016) Mediaeval 2016: a multi-modal system for verifying multimedia use task
Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM workshop on information hiding and multimedia security, pp 5–10
Lago F, Phan QT, Boato G (2019) Visual and textual analysis for image trustworthiness assessment within online news. Security and Communication Networks
Cui L, Wang S, Lee D (2019) SAME: sentiment-aware multi-modal embedding for detecting fake news. In: Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining, pp 41–48
Volkova S, Ayton E, Arendt DL, Huang Z, Hutchinson B (2019) Explaining multi-modal deceptive news prediction models. In: Proceedings of the international AAAI conference on web and social media, vol 13, pp 659–662
Tariq S, Lee S, Kim H, Shin Y, Woo SS (2018) Detecting both machine and human created fake face images in the wild. In: Proceedings of the 2nd international workshop on multimedia privacy and security, pp 81–87
Sabir E, Cheng J, Jaiswal A, AbdAlmageed W, Masi I, Natarajan P (2019) Recurrent convolutional strategies for face manipulation detection in videos. Interfaces (GUI) 3(1)
Güera D, Delp EJ (2018) Deepfake video detection using recurrent neural networks. In: 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 1–6
Papadopoulou O, Zampoglou M, Papadopoulos S, Kompatsiaris Y (2017) Web video verification using contextual cues. In: Proceedings of the 2nd international workshop on multimedia forensics and security, pp 6–10
Zhou P, Han X, Morariu VI, Davis LS (2017) Two-stream neural networks for tampered face detection. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1831–1839
Rahmouni N, Nozick V, Yamagishi J, Echizen I (2017) Distinguishing computer graphics from natural images using convolution neural networks. In: 2017 IEEE workshop on information forensics and security (WIFS), pp 1–6
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
Ferreira W, Vlachos A (2016) Emergent: a novel dataset for stance classification. In: Proceedings of the 2016 conference of the north American chapter of the association for computational linguistics: human language technologies, pp 1163–1168
Amerini I, Ballan L, Caldelli R, Del Bimbo A, Serra G (2011) A sift-based forensic method for copy–move attack detection and transformation recovery. IEEE Transactions on Information Forensics and Security 6(3):1099–1110
Conforti C, Pilehvar MT, Collier N (2018) Towards automatic fake news detection: cross-level stance detection in news articles. In: Proceedings of the first workshop on fact extraction and VERification (FEVER), pp 40–49
Bourgonje P, Schneider JM, Rehm G (2017) From clickbait to fake news detection: an approach based on detecting the stance of headlines to articles. In: Proceedings of the 2017 EMNLP workshop: natural language processing meets journalism, pp 84–89
Thorne J, Chen M, Myrianthous G, Pu J, Wang X, Vlachos A (2017) Fake news stance detection using stacked ensemble of classifiers. In Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism, pp 80–83
Doegar A, Dutta M, Gaurav K (2019) CNN based image forgery detection using pre-trained AlexNet model. Int J Comput Intell & IoT 2(1)
Uliyan DM, Jalab HA, Wahab AWA (2015) Copy move image forgery detection using hessian and center symmetric local binary pattern. In: 2015 IEEE Conference on Open Systems (ICOS), pp 7–11
Uliyan DM, Jalab HA, Wahab AWA, Shivakumara P, Sadeghi S (2016) A novel forged blurred region detection system for image forensic applications. Expert Syst Appl 64:1–10
Author information
Authors and Affiliations
Department of Information Technology, Delhi Technological University, Delhi, India
Chahat Raj & Priyanka Meel
- Chahat Raj
You can also search for this author inPubMed Google Scholar
- Priyanka Meel
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toPriyanka Meel.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Raj, C., Meel, P. ConvNet frameworks for multi-modal fake news detection.Appl Intell51, 8132–8148 (2021). https://doi.org/10.1007/s10489-021-02345-y
Accepted:
Published:
Issue Date:
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative