Document type:
Zeitschriftenaufsatz
Author(s):
Chao Chen; Severin Reiz; Chenhan Yu; Hans-Joachim Bungartz; George Biros
Title:
Fast Evaluation and Approximation of the Gauss-Newton Hessian Matrix for the Multilayer Perceptron
Abstract:
We introduce a fast algorithm for entry-wise evaluation of the Gauss-Newton Hessian (GNH) matrix for the multilayer perceptron. The algorithm has a precomputation step and a sampling step. While it generally requires $O(Nn)$ work to compute an entry (and the entire column) in the GNH matrix for a neural network with $N$ parameters and $n$ data points, our fast sampling algorithm reduces the cost to $O(n+d/\epsilon^2)$ work, where $d$ is the output dimension of the network and $\epsilon$ is...    »
Journal title:
SIAM Journal on Matrix Analysis and Applications
Year:
2021
Year / month:
2021-01
Quarter:
1. Quartal
Month:
Jan
Covered by:
Web of Science
Reviewed:
ja
Language:
en
Publisher:
SIAM
Status:
Verlagsversion / published
Submitted:
20.08.2020
TUM Institution:
Department of Informatics
BibTeX