We simply explain and illustrate Mamba, State Space Models (SSMs) and Selective SSMs.
SSMs match performance of transformers, but are faster and more memoryefficient than them. This is crucial for long sequences!

AI Coffee Break Merch! https://aicoffeebreak.creatorspring.... Celebrating our merch launch, here is a limited time offer! Get 25% discount on AI Coffee Break Merch with the code MAMBABEAN.

Thanks to our Patrons who support us in Tier 2, 3, 4:
Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael

Outline:
00:00 Mamba to replace Transformers!?
02:04 State Space Models (SSMs) – high level
03:09 State Space Models (SSMs) – more detail
05:45 Discretization step in SSMs
08:14 SSMs are fast! Here is why.
09:55 SSM training: Convolution trick
12:01 Selective SSMs
15:44 MAMBA Architecture
17:57 Mamba results
20:15 Building on Mamba
21:00 Do RNNs have a comeback?
21:42 AICoffeeBreak Merch

Gu, Albert, and Tri Dao. "Mamba: Lineartime sequence modeling with selective state spaces." arXiv preprint arXiv:2312.00752 (2023). https://arxiv.org/abs/2312.00752
MoEMamba https://arxiv.org/abs/2401.04081
Vision Mamba https://arxiv.org/abs/2401.09417
MambaByte https://arxiv.org/abs/2401.13660
Mamba rejected from ICLR:   / 1750526956452577486  
Prefix sum (Scan) with Cuda: https://developer.nvidia.com/gpugems/...
Transformer explained:    • The Transformer explained by Ms. Coff...  

Great resources to learn about Mamba:
Mamba: https://jameschen.io/jekyll/update/20...
The Annotated S4: https://srush.github.io/annotateds4/
Mamba The Easy Way: https://jackcook.com/2024/02/23/mamba...

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon:   / aicoffeebreak  
Kofi: https://kofi.com/aicoffeebreak
Join this channel to get access to perks:
   / @aicoffeebreak  
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

Links:
AICoffeeBreakQuiz:    / aicoffeebreak  
Twitter:   / aicoffeebreak  
Reddit:   / aicoffeebreak  
YouTube:    / aicoffeebreak  

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​

Scientific advising by Mara Popescu
Video editing: Nils Trost
Music : Sunny Days – Anno Domini Beats