Conversation - Background substraction

Here is a description of the different steps to add background substraction on conference : blur, transparent, or image.

Connect to the conference and select media devices

Check our tutorial on conversation and devices selection to have details about the conversation bases and media devices selection.

Add needed filter parameters on createStream

When you use createStream() function, you can add different options thanks to createStreamOptions parameter.

        var createStreamOptions = {};
        createStreamOptions.audioInputId = selectMic.value;
        createStreamOptions.videoInputId = selectCamera.value;
    

For background filtering, we can choose the kind of filtering between : blur, transparent or image. Here is the code to do it :

        //For Blur
        createStreamOptions.filters = [
            { type: 'backgroundSubtraction', options: { backgroundMode: 'blur' } },
        ];

        //For Transparent
        createStreamOptions.filters = [
            { type: 'backgroundSubtraction', options: { backgroundMode: 'transparent' } },
        ];

        //For Image
        createStreamOptions.filters = [
            { type: 'backgroundSubtraction', options: { backgroundMode: 'image', image: imageData } },
        ];
    

Then simply use createStreamOptions on createStream()

        ua.createStream(createStreamOptions)
        .then(function (stream) {
            [...]
        })
        .catch(function (err) {
            [...]
        });
    

Switch Stream

Use call.replacePublishedStream() function update the published stream after a background modification. Check the code of our sample Github repository for a complete description

        call.replacePublishedStream(null, callbacks);
    

How to get imageData ?

Here is a sample code to get imageData using a canvas :

        var imageData = null;
        const image = new Image();
        image.src = window.location.href + 'img/background_apiRTC.jpg';

        image.onload = () => {
            var canvas = document.createElement('canvas');
            var context = canvas.getContext('2d');
            canvas.width = image.width;
            canvas.height = image.height;
            context.drawImage(image, 0, 0 );
            imageData = context.getImageData(0, 0, 640, 480);
            console.debug("image is loaded");
        };
    

Live Demo