Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning

The paper addresses the challenge of training large language models (LLMs) on private, distributed data, especially in regulated sectors like healthcare and finance. It proposes a practical approach to leverage this valuable, yet unsharable, non-IID data, aiming for LLMs with deeper domain expertise.