Canvas streaming issue - Android Chrome

[marko.nastic]

Recording the canvas stream doesn't work on the newest Android Chrome version(104.0.5112.69).
Session and stream are established and stream preview works properly in the video element flashphoner library creates, but the recording never starts.
In other browsers and on other devices all work as intended.

Just to mention, canvas stream preview doesn't work on some iOS and macOS devices in the Safari browser. It just presents a black preloader video that is used in the playFirstVideo method.

 

[Max]

Good day.

Recording the canvas stream doesn't work on the newest Android Chrome version(104.0.5112.69).

Seems like latest Chrome browser builds (104.0.5112.81 on desktop and 104.0.5112.69 on Androd) does not encode H264 frames properly when capturing from canvas. In desktop Chrome, you can switch to VP8 codec when publishing

    session.createStream({
        name: streamName,
        display: localVideo,
        constraints: constraints,
        stripCodecs: "H264"
        ...
    }).publish();

But in Android Chrome this does not work. We raised the ticket WCS-3569 to investigate it. As workaround, use Firefox browser on Android devices.

Just to mention, canvas stream preview doesn't work on some iOS and macOS devices in the Safari browser.

We do not confirm this (iOS Safari 15.5, MacOS Safari 15.5 works properly using WebSDK 2.0.221). Please clarify what Safari versions do you test?

 

[marko.nastic]

Hi, Max,
Thank you for your replies.

We do not confirm this (iOS Safari 15.5, MacOS Safari 15.5 works properly using WebSDK 2.0.221). Please clarify what Safari versions do you test?

macOS Monterey, version 12.5, Safari version 15.6
We are using the playFirstVideo method for the Safari browser and after that, we create a canvas stream,
You can see how it looks in the browser and in developer tools in the attached images

We found a workaround by presenting the canvas element and hiding the flashphoner video element, but it will be better to have a real presentation of the stream.

 

[Max]

We are using the playFirstVideo method for the Safari browser and after that, we create a canvas stream,

Since WebSDK build 2.0.212 you do not need to use playFirstVideoMethod. Please try the Canvas Streaming example on demo server (see also Github source code). Please read details here: Video publishing and playback on mobile devices

 

[Max]

We tested a various Chromium based browsers. Unfortunately, some of them (Chrome, MS Edge) cannot encode video frames if hardare encoding acceleration is enabled. So stream cannot be published from canvas. There are two unresolved Chromium bugs:
https://bugs.chromium.org/p/chromium/issues/detail?id=1156408
https://bugs.chromium.org/p/chromium/issues/detail?id=1307780
The only recommended workaround is to disable hardware encoding acceleration by switching the browser flag chrome://flags/#disable-accelerated-video-encode to Disable

or use other browser: Samsung Internet browser or Yandex browser on Android or Firefox browser on desktop for example. Please note that Android Firefox cannot capture canvas at all, so it is not an option.