Seminar abstracts: SoSe 2026

Neural Galerkin is a nonlinear counterpart of classical dynamical approximation strategies for time-dependent PDEs. It approximates solutions by neural networks whose weights evolve in time. At first glance, the method is general, elegant, easy to implement, and it indeed has enjoyed notable empirical success. A closer look, however, quickly leads to funny behaviors and troubles: Formulating Neural Galerkin at a proper functional level and connecting it rigorously to practical algorithms raises a number of very subtle challenges. Some stem from sampling issues; others arise from the fact that neural network classes are, mathematically speaking, not particularly well-behaved sets. In this talk, I will illustrate several of these troubles, explain where they come from, and discuss why they are not mere technical nuisances but key conceptual elements to understand when working with neural network approximations.

---

Different from artificial neural networks, biological neurons communicate using brief electrical pulses, so-called spikes or action potentials. But is it sufficient for spiking neurons to spike at most once, or is there any benefit of spiking multiple times? Intuitively, one might assume that spiking only once represents a large constraint. However, we find a rather surprising (and compact) result that shows that the opposite is true: every bound on the approximation rate of single-spike neural networks also holds for multi-spike neural networks (and vice versa). Consequently, using this result, many approximation results in the literature for single-spike neural networks can be elevated to the multi-spike case.

---

Neural network methods for PDEs require reliable error control in function space norms. However, trained neural networks can typically only be probed at a finite number of point values. Without strong assumptions, point evaluations alone do not provide enough information to derive tight deterministic and guaranteed bounds on function space norms. In this work, we move beyond a purely black-box setting and exploit the neural network structure directly. We present a framework for the certified and accurate computation of integral quantities of neural networks, including Lebesgue and Sobolev norms, by combining interval arithmetic enclosures on axis-aligned boxes with adaptive marking/refinement and quadrature-based aggregation. On each box, we compute guaranteed lower and upper bounds for function values and derivatives, and propagate these local certificates to global lower and upper bounds for the target integrals. Our analysis provides a general convergence theorem for such certified adaptive quadrature procedures and instantiates it for function values, Jacobians, and Hessians, yielding certified computation of Lp, W 1,p, and W 2,p norms. We further show how these ingredients lead to practical certified bounds for PINN interior residuals. Numerical experiments illustrate the accuracy and practical behavior of the proposed methods.

---

In this talk, we give a survey of various results concerning minimal depth required for ReLU neural network expressivity. We would highlight the new connections amongst study of ReLU networks, polyhedral geometry and tropical geometry. The central question is to understand what is the minimal number of hidden layers required in a ReLU neural network to exactly represent a continous piecewise linear function and we would be discussing recent results in this direction.

---

It is yet unclear how AI tools will re-shape the research workflow of the pure math researcher. One source of excitement is found in the combination of genetic algorithms and LLMs, like OpenEvolve and AlphaEvolve. But the success of these methods seems to be limited to certain subareas of math. We report on the strategies we tried to use OpenEvolve to solve a problem in algebraic combinatorics: what we thought could work, what we thought would fail, and what the outcome has been so far. Joint with R. Orellana, F. Saliola, A. Schilling, M. Zabrocki. Partially supported by ICERM and CohereLabs

---

In this talk, we present per-sample gradient clipping as a method for improving the robustness of neural network training. In the first part, we take a look at deep-learning variational Monte Carlo (VMC) for solving the electronic Schrödinger equation. We present novel results showing that the nodal surface of the wave function gives rise to a naturally heavy-tailed optimization problem. We derive sharp moment bounds for the relevant estimators and show that per-sample clipped VMC guarantees convergence both in expectation and with high probability under these weak moment assumptions. In the second part, we demonstrate that per-sample gradient clipping provides analogous convergence guarantees for standard heavy-tailed stochastic optimization problems. Our theoretical analysis is complemented by numerical experiments showing that per-sample gradient clipping outperforms standard regularization methods, even after accounting for its additional computational cost.

---

This talk discusses binary classification problems in o-minimal expansions of the ordered field of real numbers. Using cell decomposition of definable sets, we analyze the approximation capabilities of ReLU neural networks for characteristic functions of the corresponding decision regions. Under suitable assumptions, we obtain approximation rates that essentially match those known for locally smooth decision boundaries. We then combine these approximation results with entropy estimates for classes of ReLU neural networks to derive statistical learning rates for empirical risk minimization with hinge loss.

---