Different applications of the Jacobian matrix for Machine Learning models part4 | by Monodeep Mukherjee | Mar, 2024

Different applications of the Jacobian matrix for Machine Learning models part4 | by Monodeep Mukherjee | Mar, 2024

Photo by James Yarema on UnsplashCurves on powers of hyperelliptic Jacobians(arXiv)Author : Olivier de Gaay Fortman, Stefan SchreiederAbstract : For a curve of genus no less than 4 which is both very normal or very normal hyperelliptic, we classify all methods by which an influence of its Jacobian might be isogenous to a product of Jacobians of curves. As an utility, we present that, for a really normal principally polarized abelian selection of dimension no less than 4, or the intermediate Jacobian of a really normal cubic threefold, no energy is isogenous to a product of Jacobians of curves. This confirms some circumstances of the Coleman-Oort conjecture. We additional deduce from our outcomes some progress on the query whether or not the integral Hodge conjecture fails for such abelian varieties2.Gradient Flossing: Improving Gradient Descent by Dynamic Control of Jacobians (arXiv)Author : Rainer EngelkenAbstract : Training recurrent neural networks (RNNs) stays a problem on account of the instability of gradients throughout very long time horizons, which may result in exploding and vanishing gradients. Recent analysis has linked these issues to the values of Lyapunov exponents for the forward-dynamics, which describe the progress or shrinkage of infinitesimal perturbations. Here, we suggest gradient flossing, a novel method to tackling gradient instability by pushing Lyapunov exponents of the ahead dynamics towards zero throughout studying. We obtain this by regularizing Lyapunov exponents by backpropagation utilizing differentiable linear algebra. This permits us to “floss” the gradients, stabilizing them and thus bettering community coaching. We reveal that gradient flossing controls not solely the gradient norm but additionally the situation quantity of the long-term Jacobian, facilitating multidimensional error suggestions propagation. We discover that making use of gradient flossing previous to coaching enhances each the success charge and convergence pace for duties involving very long time horizons. For difficult duties, we present that gradient flossing throughout coaching can additional improve the time horizon that may be bridged by backpropagation by time. Moreover, we reveal the effectiveness of our method on varied RNN architectures and duties of variable temporal complexity. Additionally, we offer a easy implementation of our gradient flossing algorithm that can be utilized in observe. Our outcomes point out that gradient flossing by way of regularizing Lyapunov exponents can considerably improve the effectiveness of RNN coaching and mitigate the exploding and vanishing gradient downside.

https://medium.com/@monocosmo77/different-applications-of-the-jacobian-matrix-for-machine-learning-models-part4-649faed87c1b?responsesOpen=true&sortBy=REVERSE_CHRON

Recommended For You