multimodal-fusion

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

multimodal-sentiment-analysis multimodal-deep-learning multimodal-fusion

UpdatedMar 14, 2023
Python

thuiar /MIntRec

Star117

MIntRec: A New Dataset for Multimodal Intent Recognition (ACM MM 2022)

artificial-intelligence speaker-recognition acm-mm multimodal-deep-learning multimodal-fusion acm-mm-22 multimodal-intent-analysis

UpdatedMay 2, 2025
Python

icey-zhang /E2E-MFD

Star92

E2E-MFD-OOD

object-detection oriented-object-detection multimodal-fusion

UpdatedDec 10, 2024
Jupyter Notebook

akashe /Multimodal-action-recognition

Star73

Code on selecting an action based on multimodal inputs. Here in this case inputs are voice and text.

multimodality multimodal-learning multimodal-deep-learning multimodal-data multimodal-fusion multimodal-action-recognition cross-attention

UpdatedJun 7, 2021
Python

ai-forever /fusion_brain_aij2021

Star50

Creating multimodal multitask models

handwritten-text-recognition bilingual visual-question-answering multitask multimodal-fusion java-to-python zero-shot-object-detection

UpdatedOct 26, 2022
Jupyter Notebook

declare-lab /hfusion

Star44

Multimodal sentiment analysis using hierarchical fusion with context modeling

sentiment-analysis fusion emotion-analysis emotion-detection emotion-recognition data-fusion multimodal-interactions multimodal-sentiment-analysis multimodal-fusion

UpdatedMar 14, 2023
Python

gholste /breast_mri_fusion

Star43

[CVAMD 2021] "End-to-End Learning of Fused Image and Non-Image Feature for Improved Breast Cancer Classification from MRI"

deep-learning breast-cancer multimodal-fusion

UpdatedJul 27, 2023
Python

Asichurter /MalFusionFSL

Star34

Few-Shot malware classification using fused features of static analysis and dynamic analysis （基于静态+动态分析的混合特征的小样本恶意代码分类框架）

simple malware few-shot-learning malware-classification multimodal-fusion fused-features few-shot-malware-classification

UpdatedDec 27, 2021
Python

open-edge-platform /scenescape

Star31

Multimodal object tracking and scene analytics for highly actionable, real-world contextualized data

spatial-analysis scene-graph multimodal-fusion scene-analytics multimodal-object-tracking

UpdatedNov 5, 2025
Python

imadhou /multimodal-sentiment-analysis

Star23

Multimodal sentiment analysis

sentiment-analysis twitter-sentiment-analysis sentiment-classification multimodal-learning multimodal-sentiment-analysis multimodal-fusion multimodal-classification

UpdatedJul 17, 2023
Jupyter Notebook

shengyangsun /MSBT

Star19

Official implementation of "Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection"

weakly-supervised-learning anomaly-detection multimodal-fusion

UpdatedNov 6, 2024
Python

declare-lab /M2H2-dataset

Star18

This repository contains the dataset and baselines explained in the paper: M2H2: A Multimodal Multiparty Hindi Dataset For HumorRecognition in Conversations

humor-detection multimodal-deep-learning multimodal-fusion emotion-recognition-in-conversation