Abstract: The rapid advancement of generative artificial intelligence and large language models is propelling artificial intelligence into a new era of “ubiquitous intelligence”. However, traditional cloud-based large models, due to their centralized deployment paradigm, face significant challenges in real-time responsiveness, privacy protection, and offline availability. Achieving efficient and secure localized intelligence on resource-constrained terminal devices has therefore become a core concern for both academia and industry. Based on the discussions from the technical forum “Embracing the Mountain in a Mustard Seed: Exploring the Development Path of On-Device Large Models” hosted by CCF YOCSEF Hangzhou, this article systematically constructs an analytical framework for on-device large models. First, it defines the concept of “on-device large models” from a conceptual perspective and elaborates on their irreplaceability in low-latency, high-privacy, and offline scenarios, along with the accompanying ethical and governance challenges. Next, it delves into the key technological pathways for achieving efficient on-device intelligence, covering model architecture innovations, algorithmic optimizations, software-hardware co-design, and data engineering. Subsequently, it objectively examines the capability boundaries imposed by computing power, storage, and sensing hardware, and analyzes the resulting societal challenges such as fairness and safety accountability. Finally, it proposes a future evolution paradigm centered on “collaborative intelligence”, encompassing task scheduling, automated model generation, and the construction of a cloud-edge-device collaborative ecosystem, while also discussing ecological development paths including open-source collaboration, standardization, and multi-stakeholder governance. This article aims to provide a systematic academic reference for the research, development, application, and governance of on-device intelligence technologies, promoting the scalable and responsible development of lightweight, efficient, secure, and collaborative terminal intelligence.