UniMamba is a new unified spatial-temporal forecasting framework that integrates efficient state-space dynamics with attention-based dependency learning to tackle multivariate time series challenges. It employs a Mamba Variate-Channel Encoding Layer and a Spatial Temporal Attention Layer to capture both global temporal dependencies and inter-variate correlations.
This paper introduces sparse prefix caching, an optimization for LLM serving that stores recurrent states at checkpoint positions rather than requiring the entire token history. The method consistently improves the Pareto frontier compared to standard heuristics, especially for use cases where requests share a non-trivial prefix.
This research systematically studies structured state space models (SSMs) for time-series classification, comparing complex Mamba-based architectures with simpler diagonal SSMs (S4D). Surprisingly, S4D consistently outperforms Mamba variants in accuracy and efficiency on large-scale benchmarks, challenging the assumption that increased model complexity leads to better performance in this domain.
StateSMix is a self-contained lossless compressor that couples an online-trained Mamba-style State Space Model (SSM) with sparse n-gram context mixing and arithmetic coding. It is initialized from scratch and trained token-by-token on the file, requiring no pre-trained weights, GPU, or external dependencies, achieving competitive results on the enwik8 benchmark.
This content discusses State Space Models, featuring Albert Gu and Karan Goel from Cartesia AI. It delves into a relevant technical topic within the field of artificial intelligence.
