Rtmp republish quality

Stian Eide · Nov 4, 2021

Hi there

We´r seeing some random issues with the rtmp republished quality, we use the room api and republish both streams.

If we use VP8 then looking at webrtc internals bandwidth and the bandwidth on the republished streams we can see that there's almost a 50% reduction so everything looks good as webrtc but quality is bad on the rtmp republished stream.. If we strip VP8 on the publishing side the bandwidth of webrtc and rtmp matches, i guess that's because there's no need for transcoding if we use H264 ..

However we sometimes see a deprecated quality using H264 as well. One of our clients report that when they test using firefox as guest and host in the room API, everything looks fine but rtmp republished stream is pixelated..

Any inputs on how we can make this more stable, our setup will possible involve several concurrent rtmp republishing streams

We run 5.2.1050 on an instance with 4 CPUs and 8GB ram , centos 7 and test is done with only 2 streams

Code:

codecs                   =opus,alaw,ulaw,g729,speex16,g722,mpeg4-generic,telephone-event,h264,flv,mpv
codecs_exclude_sip       =mpeg4-generic,flv,mpv
codecs_exclude_streaming =flv,telephone-event
codecs_exclude_sip_rtmp  =opus,g729,g722,mpeg4-generic,vp8,mpv

webrtc_cc_min_bitrate=1000000
webrtc_cc_max_bitrate=10000000
webrtc_sdp_min_bitrate_bps=3000000
webrtc_sdp_max_bitrate_bps=7000000

periodic_fir_request=true
video_encoder_second_thread_threshold=777000
video_encoder_max_threads=2
rtmp.server_buffer_enabled=true

Max · Nov 4, 2021

Hello

First of all try to record metrics of the published WebRTC stream using CLI v2.

Code:

ssh -p 2002 localhost
>stream record-metrics-start

Streams management - Web Call Server 5.2 - EN - Flashphoner Documentation

docs.flashphoner.com

Once you recorded metrics, you will see metrics causing pixels: VIDEO_RATE, VIDEO_WIDTH, VIDEO_HEIGHT, FPS

If VIDEO_RATE (bitrate) is too low, you will see pixels on playback.
If VIDEO_WIDTH x VIDEO_HEIGHT is low (i.e. 320x240), you will see pixels too.

How to fix low resolution and low bitrate:

See: content hints

From a web camera in a browser via WebRTC - Web Call Server 5.2 - EN - Flashphoner Documentation

docs.flashphoner.com

Chrome browser may decrease bitrate and resolution to keep FPS high (the "motion" hint).

Update Web SDK version to 2.0.180 (or newer) and set hint "detail" to keep resolution and bitrate high.

Stian Eide · Nov 5, 2021

HI Max

Thank you for the quick reply. There's no issues with the published stream, constraints on our webapp is 1920x1080 and bitrate quality is good and playback in chrome is smooth and clear, the stream show-metrics --mediaSessionId reflects what we see in webrtc internals

What we need to know is how to compare the publishing quality and the republished quality to figure out if that's where it goes wrong and hopefully be able to tweak it to be equal ..

if we run 8081/rest-api/push/find it shows the to republished rtmp streams , but it does not show any bitrate or framerate, using the mediasession id from the rest api reply like "show-metrics --1ecib4nlk8if3dbtl3cfhdt8e" it says it does not exist

Code:

[
    {
        "mediaSessionId": "1ecib4nlk8if3dbtl3cfhdt8e",
        "streamName": "room-wOCrV-host-9f33",
        "rtmpUrl": "rtmp://ourip:1935/live",
        "rtmpFlashVersion": "LNX 9,0,124,2",
        "rtmpTransponderStreamNamePrefix": "",
        "width": 1920,
        "height": 1080,
        "fps": 0,
        "bitrate": 0,
        "keyFrameInterval": 0,
        "rtmpTransponderKframeInterval": 60,
        "muted": false,
        "soundEnabled": false,
        "rtmpTransponderForceKframeInterval": true,
        "rtmpTransponderFullUrl": false
    },
    {
        "mediaSessionId": "vhjma50r4du6dbrub86f0rsdmb",
        "streamName": "room-wOCrV-guest-a5c2",
        "rtmpUrl": "rtmp://ourip:1935/live",
        "rtmpFlashVersion": "LNX 9,0,124,2",
        "rtmpTransponderStreamNamePrefix": "",
        "width": 1920,
        "height": 1080,
        "fps": 0,
        "bitrate": 0,
        "keyFrameInterval": 0,
        "rtmpTransponderKframeInterval": 60,
        "muted": false,
        "soundEnabled": false,
        "rtmpTransponderForceKframeInterval": true,
        "rtmpTransponderFullUrl": false
    }
]

Stian Eide · Nov 5, 2021

It´seems like a Firefox issue, if we have a video-chat in our webbapp testing with to tabs in firefox on same pc using a webcam as host and obs virtual cam as guest , both guest and host republished signal looks pixelated , using chrome this does not happen

Max · Nov 8, 2021

Stian Eide said:
It´seems like a Firefox issue, if we have a video-chat in our webbapp testing with to tabs in firefox on same pc using a webcam as host and obs virtual cam as guest , both guest and host republished signal looks pixelated , using chrome this does not happen

Please check if hardware acceleration enabled in Firefox. Check also CPU load on PC you test.
On server side, please check if there's no video transcoding while republishing: open statistics page http://wcs:8081/?action=stat and check native_resource section, the following parameters should be zero:

Code:

native_resources.video_transcoders=0
native_resources.video_decoders=0
native_resources.video_encoders=0

If not, there is some transcoding on server. In this case, make sure you've not republishing with resolution explicit settings in REST API /push/startup query (there should not be width and height parameters)

Stian Eide · Nov 8, 2021

HI Max

Code:

Chrome in both ends

-----Native Resources-----
native_resources=140695900712848,mpeg4-generic,-285326;140696016083600,opus,3425280;140695897400304,RESAMPLER:48000/44100,786744;140695896357360,mpeg4-generic,-285326;140695903726816,RESAMPLER:48000/44100,786744;140696028192480,opus,3425280
native_resources.audio_codecs=4
native_resources.audio_resamplers=2
native_resources.video_transcoders=0
native_resources.video_decoders=0
native_resources.video_encoders=0
native_resources.writers=0

Code:

Firefox in both ends

-----Native Resources-----
native_resources=140695883405536,opus,11339520;140696494562864,mpeg4-generic,-944773;140696484946544,opus,11339520;140696520394208,mpeg4-generic,-944774;140696520356832,RESAMPLER:48000/44100,2604546;140696521239872,RESAMPLER:48000/44100,2604546
native_resources.audio_codecs=4
native_resources.audio_resamplers=2
native_resources.video_transcoders=0
native_resources.video_decoders=0
native_resources.video_encoders=0
native_resources.writers=0

Hardware acceleration is enabled in Firefox , i can reproduce everytime using firefox in to tabs on my PC, Cpu load is much more heavy compared to same test in chrome.. We don't have any width and height parameters in the /push/startup

We done some more testing using to computers as in a real life scenario

firefox in both ends , guest video looks better, host video still pixilates
Firefox on guest and chrome on host , guest video is perfect host video still pixilates
Chrome in both ends , everything looks good

Really strange this one

Stian Eide · Nov 8, 2021

It seems like its a scaling issue, it starts out with a resolution of 1920x1080 but then changes it to 1440x810 it jumps up and down from !920x1080 to 1440x810

we use the following in our video web app

Code:

constraints.video = {
            width: 1920,
            height: 1080,
            deviceId: selected_camera_id,
            frameRate: 25
        }

We have a transcoding service connected at the receiving end of the republished stream and that scales everything to 1920x1080 but should put black bars if resolution on the received stream i smaller then 1920x1080

Stian Eide · Nov 8, 2021

Our websdk is version 2.0.199, we tried to upgrade to latest flashphoner and latest websdk but we got an error "RoomApi undefined" so we downgraded again

Stian Eide · Nov 8, 2021

Looking at webrtc internals when it scales down to 1440x810 it says

Code:

frameWidth    1440
frameHeight    810
framesPerSecond    24
framesSent    11835
[framesSent/s]    25.02502320941455
hugeFramesSent    107
totalPacketSendDelay    6492.003
[totalPacketSendDelay/packetsSent_in_ms]    30.818318318317772
qualityLimitationReason    bandwidth
qualityLimitationDurations    {bandwidth:191701,cpu:0,none:319667,other:0}
qualityLimitationResolutionChanges    0
encoderImplementation    OpenH264

Max · Nov 9, 2021

Stian Eide said:
It seems like its a scaling issue, it starts out with a resolution of 1920x1080 but then changes it to 1440x810 it jumps up and down from !920x1080 to 1440x810

This is normal WebRTC behaviour: if channel bandwidth is not enough for streaming in desired resolution, the resolution may be scaled down. Webrtc internals confirm that

Stian Eide said:
qualityLimitationReason bandwidth

Seems like Firefox checks bandwidth more strictly than Chrome.
You can also try to use TCP transport if not:

Code:

session.createStream({
    name: streamName,
    display: localVideo,
    constraints: {
         ...
    }
    transport: "TCP"
    ...
}).publish();

May be TCP channel quality is better than UDP.
If this does not help, we recommend to use Chrome. We tested VR streaming a couple of years ago, and Chrome is the best choice for "fat" streams publishing.

Stian Eide said:
Our websdk is version 2.0.199, we tried to upgrade to latest flashphoner and latest websdk but we got an error "RoomApi undefined" so we downgraded again

Since WebSDK build 2.0.201 RoomApi is built as separate bundle to prevent a cyclic import when using NPM package. So you should include flashphoner-room-api.js to HTML page:

Code:

<script type="text/javascript" src="../../../../flashphoner-room-api.js"></script>

In this case, RoomApi.sdk object should be used to access Flashphoner methods

Code:

var Flashphoner = RoomApi.sdk;
...
Flashphoner.init();

Stian Eide · Nov 22, 2021

Hi Max

Sorry for the late reply on this, and thank you for the reply and support . In the end we found out that there's´a different behaviour on our transcoding server when the republished stream comes from a firefox webrtc session, we use a special built ffmepg together with some mnve hardware transcoding modules and when one of the ffmpeg parameters is to set bitrate to a fixed value for the transcoded video, but using that parameter when incoming stream is webrtc from firefox, republished as rtmp results in a 70% reduction in bitrate .. We´r still investigating the main reason for this but we have a workaround so far.. Still a bit weird that this is only a firefox issue but maybee it´s somehow related to the openh264 firefox Video Codec/webrtc plugin

Max · Nov 23, 2021

Stian Eide said:
when incoming stream is webrtc from firefox, republished as rtmp results in a 70% reduction in bitrate

If there is no transcoding on WCS while republishing a WebRTC stream as RTMP, outgoing RTMP stream bitrate is always the same as published strem bitrate. Our tests with the latest Firefox build confirm that.
So please check WebRTC incoming bitrate on WCS using REST API queries /stream/find or /stream/metrics. Perhaps Firefox may drop publishing bitrate.

Rtmp republish quality

Stian Eide

Member

Max

Administrator

Streams management - Web Call Server 5.2 - EN - Flashphoner Documentation

From a web camera in a browser via WebRTC - Web Call Server 5.2 - EN - Flashphoner Documentation

Stian Eide

Member

Stian Eide

Member

Max

Administrator

Stian Eide

Member

Stian Eide

Member

Stian Eide

Member

Stian Eide

Member

Max

Administrator

Stian Eide

Member

Max

Administrator