Artificial Audio

Research

Differentiable Audio Processing & Deep Learning

Bridging classical signal processing with modern machine learning for audio.

Concept

Classical audio signal processing offers transparent, interpretable algorithms — but tuning their parameters to match complex acoustic targets remains an open challenge. Deep learning brings powerful optimization, but often at the cost of interpretability and efficiency.

Our research bridges these worlds through differentiable signal processing (DDSP): embedding classical audio structures (filters, delays, feedback networks) into differentiable computation graphs that can be optimized end-to-end with gradient descent. Alongside this, we develop neural network approaches for tasks where traditional methods fall short.

Differentiable Feedback Delay Networks

Making FDN parameters differentiable allows reverberation to be optimized toward target decay, coloration, or perceptual objectives using gradient-based training. We showed that even tiny FDN configurations produce high-quality colorless reverberation when optimized this way[1][2], and developed RIR2FDN[3] for automatically synthesizing FDN configurations that match measured room impulse responses.

Code: diff-fdn-colorless — optimize FDN parameters for spectrally flat reverberation via gradient descent.

Demo: Colorless FDN examples — audio comparisons.

Code: rir2fdn — analyze measured RIRs and synthesize matching FDN configurations.

Demo: RIR2FDN project page — listening examples of RIR-to-FDN conversion.

FLAMO: Differentiable Audio Systems Library

FLAMO (Frequency-sampling Library for Audio-Module Optimization)[4] is a PyTorch library for building and optimizing differentiable linear time-invariant audio systems. It provides differentiable gains, filters (biquads, state variable filters, graphic EQs), delays, and transforms that can be chained into complex architectures and trained end-to-end.

Documentation · PyPI

Differentiable Active Acoustics

Reverberation enhancement systems form an electro-acoustic feedback loop whose stability is critical. We treat this loop as a differentiable system and optimize stability and performance via gradient descent[5], opening new possibilities for automated active acoustics design.

Demo: Differentiable active acoustics project page — demonstrations of stability optimization.

Room Impulse Response Completion

Rendering immersive audio in VR and games requires fast RIR generation. DECOR (Deep Exponential Completion Of Room impulse responses)[6] predicts late reverberation from only the early 50 ms of a measured response — an encoder-decoder network that synthesizes multi-exponential decay envelopes of filtered noise.

Demo: RIR completion project page — interactive examples.

Neural Decay Analysis

DecayFitNet[7] is a lightweight neural network that replaces brittle iterative fitting for multi-exponential energy decay estimation. Trained on synthetic data, it provides deterministic inference without manual tuning, validated on over 20,000 real acoustic measurements.

Neural Physical Modeling

Recurrent neural networks with fast convolutional layers[8] learn the dynamics of physical models of vibrating systems, such as strings, from simulated data.

Demo: FNO for physical modeling — sound examples.

KLANN: Koopman-Linearised Audio Networks

KLANN[9] models nonlinear audio effects with long-term dynamics by lifting the signal into a high-dimensional, approximately linear state space and predicting there with differentiable biquad filters. It matches state-of-the-art black-box models with far fewer parameters.

Demo: KLANN examples — audio processing results.

References

  1. Gloria Dal Santo, Karolina Prawda, Sebastian J. Schlecht et al. (2023). Differentiable Feedback Delay Network for Colorless Reverberation. International Conference on Digital Audio Effects (DAF 23).
  2. Gloria Dal Santo, Karolina Prawda, Sebastian J. Schlecht et al. (2025). Optimizing Tiny Colorless Feedback Delay Networks. EURASIP Journal on Audio, Speech, and Music Processing.
  3. Gloria Dal Santo, Benoit Alary, Karolina Prawda et al. (2024). RIR2FDN: An Improved Room Impulse Response Analysis and Synthesis. Proceedings of the 27-Th Int. Conf. on Digital Audio Effects (Dafx24).
  4. Gloria Dal Santo, Gian Marco De Bortoli, Karolina Prawda et al. (2025). FLAMO: An Open-Source Library for Frequency-Domain Differentiable Audio Processing. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP).
  5. Gian Marco De Bortoli, Gloria Dal Santo, Karolina Prawda et al. (2024). Differentiable Active Acoustics - Optimizing Stability via Gradient Descent. Proceedings of the 27-Th Int. Conf. on Digital Audio Effects (Dafx24).
  6. Jackie Lin, Georg Götz & Sebastian J. Schlecht (2025). Deep Room Impulse Response Completion. EURASIP Journal on Audio, Speech, and Music Processing.
  7. Georg Götz, Ricardo Falcón Pérez, Sebastian J. Schlecht et al. (2022). Neural Network for Multi-Exponential Sound Energy Decay Analysis. The Journal of the Acoustical Society of America.
  8. Julian D. Parker, Sebastian J. Schlecht, Rudolf Rabenstein et al. (2022). Physical Modeling Using Recurrent Neural Networks with Fast Convolutional Layers. Proceedings of the 25-Th Int. Conf. on Digital Audio Effects (Dafx20in22).
  9. Ville Huhtala, Lauri Juvela & Sebastian J. Schlecht (2024). KLANN: Linearising Long-Term Dynamics in Nonlinear Audio Effects Using Koopman Networks. IEEE Signal Processing Letters.