Dense vs. MoE gap is shrinking fast with the 3.6-27B release

Dense AI models currently outperform MoE overall, but MoE is rapidly catching up, particularly in coding benchmarks. For users with 24GB VRAM and a need for large context windows, MoE is becoming a more appealing option.