Photo by Qingbao Meng on UnsplashTENG: Time-Evolving Natural Gradient for Solving PDEs with Deep Neural Net(arXiv)Author : Zhuo Chen, Jacob McCarran, Esteban Vizcaino, Marin Soljačić, Di LuoAbstract : Partial differential equations (PDEs) are instrumental for modeling dynamical programs in science and engineering. The introduction of neural networks has initiated a big shift in tackling these complexities although challenges in accuracy persist, particularly for preliminary worth issues. In this paper, we introduce the Time-Evolving Natural Gradient (TENG), generalizing time-dependent variational rules and optimization-based time integration, leveraging pure gradient optimization to acquire excessive accuracy in neural-network-based PDE options. Our complete growth contains algorithms like TENG-Euler and its high-order variants, similar to TENG-Heun, tailor-made for enhanced precision and effectivity. TENG’s effectiveness is additional validated via its efficiency, surpassing present main strategies and reaching machine precision in step-by-step optimizations throughout a spectrum of PDEs, together with the warmth equation, Allen-Cahn equation, and Burgers’ equation.
https://medium.com/@monocosmo77/working-with-natural-gradients-part1-machine-learning-2024-15ba1e18c8d1?responsesOpen=true&sortBy=REVERSE_CHRON
Working with Natural Gradients part1(Machine Learning 2024) | by Monodeep Mukherjee | Apr, 2024
