Seminar abstracts: SoSe 2025

We introduce a novel concept for spiking neural networks (SNNs) derived from the idea of "linear pieces" used to analyse the expressiveness and trainability of artificial neural networks (ANNs). We prove that the input domain of SNNs decomposes into distinct causal regions where its output spike times are locally Lipschitz continuous with respect to the input spike times and network parameters. The number of such regions - which we call "causal pieces" - is a measure of the approximation capabilities of SNNs. In particular, we demonstrate in simulation that parameter initialisations which yield a high number of causal pieces on the training set strongly correlate with SNN training success. Moreover, we find that feedforward SNNs with purely positive weights exhibit a surprisingly high number of causal pieces, allowing them to achieve competitive performance levels on benchmark tasks. We believe that causal pieces are not only a powerful and principled tool for improving SNNs, but might also open up new ways of comparing SNNs and ANNs in the future.

---

Starting with the seminal work of Andrew Barron in 1993, spectral Barron spaces have become a focal point in the study of neural network approximation, as they characterize function classes that can be approximated without suffering from the curse of dimensionality across various topologies. While the existing literature primarily addresses approximation in Lebesgue and Hilbert–Sobolev spaces, both considered over bounded domains, numerical methods for partial differential equations often benefit from employing weighted spaces to account for singularities in the model or non-uniform data distributions. Moreover, many real-world systems naturally evolve in unbounded domains, where traditional bounded-domain analysis is insufficient. In this presentation, I will discuss two extensions of existing theory that address these challenges. The first considers bounded domains equipped with possibly unbounded Muckenhoupt weights; the second focuses on unbounded domains with suitably decaying weights. In both settings, the analysis proceeds in two steps: first, establishing embedding results for Fourier–Lebesgue spaces into the relevant weighted Sobolev spaces; and second, deriving asymptotic neural network approximation rates, including conditions under which these rates remain free from the curse of dimensionality.

---

Solving the Schrödinger Equation allows in principle to compute any property of any molecule, but solving it in practice is hard, due to the high dimensionality of the problem and the required accuracy. Due to these challenges there are still no numerical solvers, which can efficiently and reliably solve it even for relatively small systems. Recently a method known as Neural Network Wavefunctions have emerged as a promising approach: For many small molecules they yield highly accurate solutions, outperforming all conventional solvers, and provide in principle favorable scaling with system size n. In this talk I will give a brief informal introduction into Neural Network Wavefunctions and then focus on our latest preprint, which reduces the complexity of Neural Wavefunction by O(n) and yields more than 10x speedups in practice. arxiv.org/abs/2504.06087

---

One of the main selling points of modern GPUs is their support of low-precision numbers: the lower the precision, the faster the computation. "The more efficient the training of my neural network," one may conclude immediately, forgetting to acknowledge the effects of rounding errors due to low-precision floating-point computations. In this talk, I will discuss the basics of floating-point numbers and the numerical analysis of rounding errors. They are at the centre of our research project with Philipp Petersen, and I will share some of the results obtained so far.

---

Contrastive learning leverages data augmentation to develop feature representation without relying on large labeled datasets. However, despite its empirical success, the theoretical foundations of contrastive learning remain incomplete, with many essential guarantees left unaddressed, particularly the realizability assumption concerning neural approximability of an optimal spectral contrastive loss solution. We overcome these limitations by analyzing the pointwise and spectral consistency of the augmentation graph Laplacian. We establish that, under specific conditions for data generation and graph connectivity, as the augmented dataset size increases, the augmentation graph Laplacian converges to a weighted Laplace-Beltrami operator on the natural data manifold. These consistency results in turn give way to a robust framework for establishing neural approximability, directly resolving the realizability assumption in a current paradigm. The talk is based on the paper of the same title: arxiv.org/pdf/2502.04312. I will begin with brief introductions to manifold learning and the current paradigm of contrastive learning. I will then present our manifold learning analysis used to address the realizability assumption in contrastive learning.

---

I will give a brief and biased introduction to Diffusion models and present some of my own work on the topic, specifically, a modification of the inference process. The mathematical basis is rather heuristic, but it seems to yield some interesting empirical results.

---

We bound the minimax error of estimating binary classifiers when the decision boundary of the learning set can be described by horizon functions in a general class and a margin condition is satisfied. Then, we apply these bounds to particular cases of some classes of functions, such as convex functions or Barron-regular functions. In addition, we show that classifiers with a Barron-regular decision boundary can be approximated by ReLU neural networks, with an optimal rate independent of the dimension that is close to $n^{-1}(1+\log n)$ when the margin is large compared to around $n^{-1/3}$ when no margin condition is assumed, here $n$ is the number of training samples. To conclude, we present some numerical experiments that confirm the theoretical results. This is a joint work with Philipp Petersen.

---

In recent years, there has been growing interest in applying neural networks to the data-driven approximation of partial differential equations (PDEs). This talk will discuss a generative AI approach for fast, accurate, and robust statistical computation of three-dimensional turbulent fluid flows. On a set of challenging fluid flows, this approach provides an accurate approximation of relevant statistical quantities of interest while also efficiently generating high-quality realistic samples of turbulent fluid flows. This stands in stark contrast to ensemble forecasts from deterministic machine learning models, which are observed to fail on these challenging tasks. This talk will highlight theoretical results that reveal the underlying mechanisms by which generative AI models can succeed in capturing key physical properties where deterministic ML approaches fall short.

---