Researchers propose MIDAS framework for incomplete multimodal sentiment analysis

Researchers from multiple institutions have proposed a unified framework called MIDAS to address a persistent challenge in multimodal sentiment analysis: handling incomplete or corrupted input modalities. Most existing approaches assume access to complete data, which is unrealistic in real-world deployments. The authors argue that prior methods often rely on data imputation or heuristic coordination constraints, which fail to effectively extract task-relevant information from incomplete multimodal data.

MIDAS introduces a variational modeling strategy that represents each modality using multivariate Gaussian latent variables. These variables are further decomposed into shared and exclusive factors, enabling the model to separate modality-specific information from information that is shared across modalities. To ensure reliable representations, the authors design a minimax objective that minimizes mutual information between shared and exclusive spaces for stable disentanglement. Simultaneously, the objective maximizes mutual information among shared spaces across modalities to enhance semantic alignment.

The framework also incorporates an uncertainty-aware fusion mechanism. This mechanism uses posterior variance as a reliability indicator to adaptively weight latent features during fusion, ensuring robust integration even when modalities are incomplete. The approach avoids brittle imputation strategies and instead focuses on restructuring representations to be resilient to missing or corrupted inputs.

The researchers evaluate MIDAS on three widely used datasets and report strong, consistent performance gains over competitive baselines across a wide range of incomplete settings. The results demonstrate the framework’s effectiveness and robustness for scenarios where input modalities are missing or corrupted, a common occurrence in real-world applications.