Skymizer Taiwan Inc. Unveils Breakthrough Architecture Enabling Ultra-Large LLM Inference on a Single Card

Skymizer Taiwan Inc. has unveiled a breakthrough architecture, the HTX301 card, that allows 700B-parameter LLM inference on a single PCIe card with 384GB memory and low power consumption (~240W). This approach offloads decoding to the HTX301 while GPUs handle prefill, enabling ultra-large LLM inference locally without massive GPU VRAM.