WebRTC to RTMP, optimize quality

[kindco]

Hello,
We have a WCS instance running on Digital Ocean droplet, and we use it as a "proxy" for live streams from browser to Mux (https://mux.com), that accepts RTMP input. We are aiming 1280x720 resolution, and right now we don't like video quality much.

I tried both H.264 and VP8 codecs (changing their order in codecs setting).

H.264 sends stable frame size 1280x720 from the start, but it takes it about a minute or even more to get good image quality. In other words, it takes H.264 about a minute to reach bitrate of ~2.2 Mb/s. And during this minute video quality is very poor.

VP8 has dynamic frame resizing and it starts from 480x360 or something like this, and then gets required 1280x720 framesize in the same minute of so. During this minute its picture is better than of H.264, but the problem is that Mux resizes the whole stream by its first frame, so as a result on Mux size we have a 480x360 video, even if WebRTC provides 1280x720. Moreover, VP8 adds ugly colored artifacts to the picture sometimes.

I also tried to force server transcoding using disable_streaming_proxy=true and force_client_requested_video_resolution=false and encoder_default_video_resolution=1280x720 settings, but I didn't notice any change. Maybe I made something wrong.

So, my question is: are there any client- or serverside options to increase WebRTC video quality, making it smooth from the start and having the desired framesize?

 

[kindco]

Another question: is there a possibility to start RTMP restreaming on a running stream by demand? Is this case we could start a stream, wait a minute and only then start RTMP translation.

 

[Max]

Hello!

are there any client- or serverside options to increase WebRTC video quality, making it smooth from the start and having the desired framesize?

For WCS there is a possibility of buffering outgoing RTMP stream. This increases the broadcast latency, but allows you to get smoother playback of the stream on the server, where the stream is relayed. Buffering is enabled using a parameter in the flashphoner.properties file

rtmp_out_buffer_enabled = true

More about buffering and available options here.

Also, to improve the quality of the picture, you can enable transport via the TCP protocol and increase the bitrate. This can be done at the server level using the settings in the flashphoner.properties file:

ice_tcp_transport = true

webrtc_cc_min_bitrate = 3000000
webrtc_cc_max_bitrate = 10000000

For the Google Chrome browser, you can force the bitrate increase by specifying in SDP by replacing the x-google-max-bitrate and x-google-min-bitrate parameters.
Settings for replacing SDP parameters in the Chrome browser can be set on the server side using parameters in the flashphoner.properties file:

webrtc_sdp_min_bitrate_bps = 3000000
webrtc_sdp_max_bitrate_bps = 7000000

These parameters are specified in bits per second.

Or on the browser side by specifying parameters when publishing the stream.

For example, like this:
For transport protocol

session.createStream ({
    name: streamName,
    display: localVideo,
    transport: "TCP"
}). on (STREAM_STATUS.PUBLISHING, function (stream) {
...
}), publish ();

For bitrate

function publishStream () {
    constraints = {
        audio: true,
        video: {
            minBitrate: 1000,
            maxBitrate: 2000,
        }
    };
    stream = session.createStream ({
        name: "stream",
        display: "localVideo",
        constraints: constraints,
    });
    stream.publish ();
}

At the same time, on the browser side, the bitrate is set in kilobits per second, and on the server side, in bits per second.

More: https://docs.flashphoner.com/displa...ement+when+capturing+WebRTC+stream+in+browser

Publishing and playing stream via WebRTC over TCP - Web Call Server 5.2 - EN - Flashphoner Documentation

docs.flashphoner.com

is there a possibility to start RTMP restreaming on a running stream by demand?

You can control WebRTC relaying to an RTMP stream using both REST calls and JavaScript APIs.
More: https://docs.flashphoner.com/display/WCS52EN/To+another+RTMP+server#ToanotherRTMPserver-RESTqueries

 

[kindco]

@Max thanks for your answer!

Unfortunately, with buffer enabled we get "invalid_input" error on Mux side. We'll reach their support to understand what exactly is wrong.

With TCP transport video has noticeable freezes, so this isn't an option too.

 

[Max]

With TCP transport video has noticeable freezes, so this isn't an option too.

Freezes with TCP transport usually means bitrate is too high for publishers channel bandwidth. Please clarify what bitrate settings do you use. Usually, 2000 kbps should be enough for 720p stream with acceptable quality.
You can check channel quality using Media Devices example as described here

but the problem is that Mux resizes the whole stream by its first frame

To workaroud this, you can use REST API to enable stream transcoding to desirable resolution while republishing it as RTMP, for example

POST /rest-api/push/startup HTTP/1.1
Host: 192.168.1.101:8081
Content-Type: application/json
 
{
    "streamName": "test",
    "rtmpUrl":"rtmps://global-live.mux.com:443/app/your-secret-stream-key",
    "width": 1280,
    "height": 720
}

This query pushes a stream to Mux with transcoding it to 720p.
Please note that your DO instance should be powerful enough to transcode 720p stream (at least 4 CPU and 8 Gb RAM is recommended if there is only one such stream)

 

[kindco]

Thanks @Max
Just setting min bitrate appeared to be enough in our case. Videos are streamed in good quality now.

 

[kindco]

Hello @Max
I remade our app to use REST method /push/startup to start RTMP translation. All works great except for one thing: now I don't get any notification in the app if RTMP stream fails. I suppose this is because new stream is created for RTMP translation. But it's crucial to know if something is wrong with RTMP translation. So, the question is: is there any way to get RTMP stream errors in the web SDK?

 

[Max]

Good day.

So, the question is: is there any way to get RTMP stream errors in the web SDK?

First, you can check a response to /push/startup query. If stream cannot be republished for some reason, this query returns an error.
If /push/startup returned 200 OK, you can use REST queries /push/find

POST /rest-api/push/find HTTP/1.1
Host: 192.168.1.101:8081
Content-Type: application/json
 
{
    "rtmpUrl":"rtmp://localhost:1935/live"
}

to find a certain RTMP stream by RTMP URL or /push/find_all to find all RTMP streams at the moment. You'll receive RTMP streams list

[
    {
        "mediaSessionId":"6dk1l0m3cm17c68232bccm52f1",
        "streamName":"rtmp_cf58",
        "rtmpUrl":"rtmp://rtmp.flashphoner.com:1935/live"
        },{
        "mediaSessionId":"bung2cu1npigp8u4bd75ha4c5r",
        "streamName":"rtmp_4be7",
        "rtmpUrl":"rtmp://localhost:1935/live"
        },{
        "mediaSessionId":"hs4khbl7g3abmnkeli217lrcmf",
        "streamName":"rtmp_f85d",
        "rtmpUrl":"rtmp://localhost:1935/live"
    }
]

If the stream is in list, it is successfully re-published to RTMP server. If not, there was some error, and re-publishing is failed. There's no way to check the certain error message unless server logs parsing.