📝 Publications

Multimodal Transformers are Hierarchical Modal-wise Heterogeneous Graphs

Yijie Jin, Junjie Peng, Xuanchao Lin, Haochen Yuan, Lan Wang, Cangzhi Zheng

Project

• Efficiency and Effectiveness: 1/3 number of parameters without additional computational overhead compared with traditional MulTs.
• Block-Sparse Attention (Triton Kernel) and All-Modal-In-One Fusion.