ML-processing of Media Streams in the Browser in the Context of SberJazz Video Conferences

Speakers will share how they implemented video and audio processing in SberJazz using the example of background replacement technology. We will learn about capturing and working with video/audio in a web browser, integrating high-performance modules (WASM, ML runtime), as well as training and optimising ML models to work on the client.

Dmitry and Roman will show how you can build your own media processing pipeline for a product. We will discuss in detail why they developed the in-house engine to run ML models and processing on GPU using WebGL.