Generative Stereo Upmixing

The aim of this project is to build and evaluate a stereo upmixing system by combining generative audio models with classical spatial audio processing.

Classical upmixers rely on simple signal statistics, such as level differences and correlation between the left and right channels. They cannot tell overlapping sources apart, so voices, instruments, and reverberation often end up smeared across the wrong loudspeakers. Generative audio models, such as recent diffusion- and flow-based separation models, take a different approach: having learned what real sources sound like, they can isolate individual instruments, voices, and ambience even when these overlap, and can be steered towards arbitrary sources instead of a fixed set of stems. Classical spatial audio methods, such as amplitude panning or Directional Audio Coding (DirAC), can then place these sounds anywhere in a loudspeaker setup. Combined, they enable a new approach to upmixing: separate the mix into its sources, then place each source and the remaining ambience around the listener.

Tasks

• Install and calibrate a 5.1 loudspeaker setup in the lab.
• Review diffusion and flow-matching models and use a pretrained generative model to separate stereo mixes into individual sources and ambience.
• Spatialize the separated sources by distributing them across the 5.1 setup, and compare the result with a classical parametric upmixer.
• Evaluate the results with objective measures and a listening test in the lab.

A the end of the project you will deliver a working upmixing pipeline with documented code, a comparison against classical baselines, listening examples, and the results of a listening test. It can be taken as a research project or as a master thesis.

Prerequisites

Audio signal processing and programming, preferably Python. Experience with deep learning and PyTorch is helpful, as is an interest in spatial audio and sound perception.

Starting references

• [1] Holderrieth, P. & Erives, E. An Introduction to Flow Matching and Diffusion Models. arXiv:2506.02070 (2025). Paper, course website
• [2] Shi, B. et al. SAM Audio: Segment Anything in Audio. arXiv:2512.18099 (2025). Paper, code
• [3] Pulkki, V. & Faller, C. Directional Audio Coding in Spatial Sound Reproduction and Stereo Upmixing. Proc. AES 28th International Conference (2006). Paper
• [4] Liang, Z., Wang, R., Xiao, Y., Ye, X. & Kong, Q. ImmersiveFlow: Stereo-to-7.1.4 Spatial Audio Generation with Flow Matching. arXiv:2601.12950 (2026). Paper

Contact

Cristobal Andrade(cristobal.andrade@fau.de).