KWIM & OS Numerical Optimization SS24: Adaptive Deep Neural Network Architectures: Time-adaptive pruning and sensitivity-based layer insertion

Time
Thursday, 13. June 2024
15:15 - 16:45

Location
F423

Organizer
B. Azmi & S. Volkwein & KWIM

Speaker:
Dr. Evelyn Herberg

This event is part of an event series „KWIM Lecture Series“.

We propose adaptive neural network architecture approaches for Feedforward Neural Networks (FNNs) and Residual Neural Networks (ResNets). On the one hand we propose an adaptive pruning approach, and on the other hand we provide a sensitivity-based layer insertion technique. In both approaches, the hyperparameter search is simplified and network complexity can be reduced without compromising expressiveness, while simultaneously decreasing training time. The results are illustrated by numerical examples.