Abstract: At present, machine learning methods and technologies represented by deep learning and large-scale models have experienced rapid and vigorous development, achieving remarkable success across a wide range of application domains such as computer vision and natural language processing. However, a striking contrast has emerged between the rapid advancement of machine learning technologies and the relatively lagging development of their mathematical foundations. Traditional theoretical frameworks—including approximation theory, statistical learning theory, and optimization theory—are insufficient to fully explain key phenomena observed in large models, such as emergence, hallucination, robustness, and vulnerability. As a result, artificial intelligence is currently facing a developmental bottleneck characterized by engineering progress outpacing theoretical understanding. This article, grounded in the historical evolution of machine learning and its theoretical development, systematically reviews the core mathematical challenges faced by next-generation machine learning. It focuses on fundamental scientific questions, including the upper limits of model capability, reliability boundaries, and underlying architectural mechanisms. Furthermore, it attempts to propose research directions and development strategies toward a new theoretical framework for next-generation machine learning, with the aim of providing theoretical support for the transition of artificial intelligence from empirically driven approaches to theory-driven paradigms, and from engineering practice to scientific discipline.