On-prem AI

DGX Spark just arrived — planning to run vLLM + local models, looking for advice

A new DGX Spark owner is seeking advice on configuring it for local LLM inference, planning to use vLLM, PyTorch, and Hugging Face models for a private API backend. They are looking for recommendations on efficient models, tuning tips for vLLM on unified memory systems, and real-world throughput insights.