multimodal learning

2 items

RESEARCHarXiv CS.LG·11d ago

Balancing Multimodal Learning through Label Space Reshaping

The paper addresses modality imbalance in multimodal learning, where some modalities dominate optimization while others remain undertrained. It proposes that this discrepancy stems from differing mapping difficulties between modality-specific feature space and the shared label space, introducing BMLR to equalize this difficulty.

multimodal learning Optimization learning machine learning

RESEARCHarXiv CS.LG·12d ago

Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey

This paper presents a survey addressing multimodal learning challenges with the Mixture-of-Experts (MoE) architecture. The study explores how MoE functions as an efficient engine and a representation learner for integrating diverse data modalities. It fills a gap in the literature by offering a comprehensive and systematic review on the topic.

multimodal learning Survey Mixture of Experts AI