Local LLM with Google Gemma: On-Device Inference Between Theory and Practice

This article explores the feasibility and challenges of running LLMs locally on smartphones, using Google Gemma and LiteRT-LM within a Flutter app. It focuses on the trade-offs in model format, runtime, and performance for on-device inference, highlighting the shift from 'if it can be done' to 'how it's done'.