Hardware-Software Co-design

Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation Models

This research presents a multi-layered methodology to accelerate multimodal foundation models (MFMs) through hardware and software co-design. It employs optimization techniques like hierarchy-aware mixed-precision quantization, structural pruning, speculative decoding, and model cascading to reduce computational and memory requirements.