Bridging classical signal processing with modern machine learning for audio.

Concept

Classical audio signal processing offers transparent, interpretable algorithms — but tuning their parameters to match complex acoustic targets remains an open challenge. Deep learning brings powerful optimization, but often at the cost of interpretability and efficiency.

Our research bridges these worlds through differentiable signal processing (DDSP): embedding classical audio structures (filters, delays, feedback networks) into differentiable computation graphs that can be optimized end-to-end with gradient descent. Alongside this, we develop neural network approaches for tasks where traditional methods fall short.

Differentiable Feedback Delay Networks

Making FDN parameters differentiable allows reverberation to be optimized toward target decay, coloration, or perceptual objectives using gradient-based training. We showed that even tiny FDN configurations produce high-quality colorless reverberation when optimized this way[3][10], and developed RIR2FDN[6] for automatically synthesizing FDN configurations that match measured room impulse responses.

Code: diff-fdn-colorless — optimize FDN parameters for spectrally flat reverberation via gradient descent.

Demo: Colorless FDN examples — audio comparisons.

Code: rir2fdn — analyze measured RIRs and synthesize matching FDN configurations.

Demo: RIR2FDN project page — listening examples of RIR-to-FDN conversion.

FLAMO: Differentiable Audio Systems Library

FLAMO (Frequency-sampling Library for Audio-Module Optimization)[9] is a PyTorch library for building and optimizing differentiable linear time-invariant audio systems. It provides differentiable gains, filters (biquads, state variable filters, graphic EQs), delays, and transforms that can be chained into complex architectures and trained end-to-end.

Documentation · PyPI

Differentiable Active Acoustics

Reverberation enhancement systems form an electro-acoustic feedback loop whose stability is critical. We treat this loop as a differentiable system and optimize stability and performance via gradient descent[5], opening new possibilities for automated active acoustics design.

Demo: Differentiable active acoustics project page — demonstrations of stability optimization.

Room Impulse Response Completion

Rendering immersive audio in VR and games requires fast RIR generation. DECOR (Deep Exponential Completion Of Room impulse responses)[8] predicts late reverberation from only the early 50 ms of a measured response — an encoder-decoder network that synthesizes multi-exponential decay envelopes of filtered noise.

Demo: RIR completion project page — interactive examples.

Neural Decay Analysis

DecayFitNet[1] is a lightweight neural network that replaces brittle iterative fitting for multi-exponential energy decay estimation. Trained on synthetic data, it provides deterministic inference without manual tuning, validated on over 20,000 real acoustic measurements.

Physical Modeling with Neural Operators

Fourier neural operators[2] learn to approximate PDE solutions for physical models of musical instruments, enabling real-time sound synthesis that captures the physics of vibrating strings and resonant bodies.

Demo: FNO for physical modeling — Fourier neural operator examples.

KLANN: Knowledge-Leveraging Audio Networks

KLANN[7] integrates domain knowledge into neural network architectures for audio processing, combining the efficiency of classical signal processing structures with the flexibility of learned parameters.

Demo: KLANN examples — audio processing results.

References