Re-stream RTP desktop capture to browser for 20-30 users

[Neil Canham]

Hi
I have a need to capture and broadcast desktop with low latency to 20-30 users. The end users need to be able to just use HTML5 video tag to view the desktop. Currently I've managed to get a low latency, high quality desktop capture to server using FFMPEG to generate and send h264 encoded RTP stream. I tried using VLC to transcode and restream to HTTP/ogg/theora but this works badly.

Is this something I could do using Flashphoner and if so is there a good example of how to set that up so I can try it?

Thanks

 

[Max]

Hello.
You can capture WebRTC stream from desktop to WCS server as described on this page. By default, it is H264 stream if browser supports this codec. Note that extension is requred to share screen from Chrome browsers.
Then you can play WebRTC stream using HTML5 video tag as described here. We use VR player as an example, but it should work with any custom player that use HTML5.
If you have any questions please ask.

 

[Neil Canham]

Hello.
You can capture WebRTC stream from desktop to WCS server as described on this page. By default, it is H264 stream if browser supports this codec. Note that extension is requred to share screen from Chrome browsers.
Then you can play WebRTC stream using HTML5 video tag as described here. We use VR player as an example, but it should work with any custom player that use HTML5.
If you have any questions please ask.

Thank you for the reply. My question was more specific - I already have the desktop capture working, it can not use the browser - it is a command line invocation of ffmpeg that generates an RTP stream of h264 encoded packets which I send to a fixed port on the server. The server would need to listen to this port, accept the stream and re-package it for display in the browsers of the connected clients. Is this something that I can do with Flashphoner? And if so can you help me to understand how I would do that?

Thanks
Neil Canham
CTO vComm Solutions AG

 

[Max]

Flashphoner WCS server does not support raw H.264 RTP stream.
So to implement your case you have to use RTMP protocol with ffmpeg.
If you publish ffmpeg stream to rtmp://host:1935/live/stream1 it should work.
Please check out our ffmpeg related docs https://docs.flashphoner.com/display/WCS5EN/Using+ffmpeg

 

[Neil Canham]

Thanks! So now I have my own trial server working on EC2 and I'm able to stream a sample mp4 from my PC to the admin demo panel. A big step forward. However when I try to replace the source movie with capture from desktop, the stream player connects but just spins. The ffmpeg line I use is:

ffmpeg -f gdigrab -i "title=notepad.exe" -r 10 -framerate 10 -preset ultrafast -vcodec h264 -strict -2 -f flv rtmp://<hostip>:1935/live/stream_ffmpeg

I can see that ffmpeg is streaming, and the web player shows that connection is established, and then shows 'playing' but the wheel just spins. As far as I can see ffmpeg is streaming similar data to the mp4 example - or am I missing something important here?

 

[Max]

Hello.
By default, Player example waits for video and audio tracks in stream to play, but your stream has only video. You should either to add empty audio to your stream or to switch off audio in player via constraints. Please modify Player example source code as follows:

function playStream(session) {
    var streamName = $('#streamName').val();
    var options = {
        name: streamName,
        display: remoteVideo,
        flashShowFullScreenButton: true,
        constraints: {audio: false, video: true}
    };
    ...

 

[Neil Canham]

Ah - excellent - thanks

 

[Neil Canham]

Had a chance to try this now - still no luck with the above change to the player - still spinning wheel in the player with 'PLAYING' shown beside the Start button. In the server log I see 'not enough bandwidth' being reported:

OBJECT:
{
"nodeId" : "DbnxXbU4HjGDDasKOHLUpHEaG895Za1A@127.0.0.1",
"appKey" : "defaultApp",
"sessionId" : "/**********:56996/**********:8443",
"mediaSessionId" : "48842180-287e-11e9-af3e-8bfe97a3d59c",
"name" : "stream_ffmpeg",
"published" : false,
"hasVideo" : true,
"hasAudio" : true,
"status" : "NOT_ENOUGH_BANDWIDTH",
"videoCodec" : "H264",
"info" : "3854480/11056",
"record" : false,
"width" : 944,
"height" : 682,
"bitrate" : 0,
"minBitrate" : 0,
"maxBitrate" : 0,
"quality" : 0,
"history" : false,
"createDate" : 1549285851204,
"mediaProvider" : "WebRTC",
"origin" : "https://*************:8888",
"constraints" : {
"audio" : false,
"video" : true
}
}
which is odd as the stream is 10FPS desktop capture of a single window 952x693 - not demanding.

 

[Max]

Hello.
Please try to use lower stream resolution. It looks like your channel bandwith is not enough for 944x682 stream, especially if you test streaming and playback from the same client PC.
You can check the bandwidth using iperf utility:
1. Install iperf to client and server (it should be same major version on both ends, 2.* for example)
2. Run iperf on server

iperf -s -p 5201

3. Run iperf on client

iperf -c server_ip -t 120 -p 5201 -i 5

This test the channel bandwidth from client to server for 120 seconds printing results every 5 seconds.

 

[Neil Canham]

I've tried reducing the client streaming massively - now 562x350 (too small to be useful) and also stepped up te EC2 server instance size to t2.large - now the server.log in Web Call Server no longer reports bandwidth issue, but the Javascript console reports bandwidth issue and still the wheel spins. So I think here my journey with Web Call Server must end. Thanks for the support.

 

[Max]

Please send us your server SSH credentials to support@flashphoner.com. We will check.

 

[Max]

Hello.
We've checked your settings and reproduced the problem.
The issue is not in bandwidth. When you streaming desktop with ffmpeg, video traffic goes unevenly, sometimes it can stop at all: when screen does not chenge, ffmpeg sends nothing.
"Not enough bandwith" event is raising for that reason.
So, we recommend to set the following WCS parameters:

# To switch off RTP activity timer
rtp_activity_detecting=false,0
# To switch keep alive off for RTMP publishsing
keep_alive.enabled=websocket,rtmfp
# To switch off 'Not enough bandwidth' event
webrtc_cc2_bitrate_overuse_event=false

and you should use transcoding. To enable it for client, set the resolution explicitly, in Player example it can be done via page parameters:

https://your_server:8888/client2/examples/demo/streaming/player/player.html?resolution=952x693

In production, set it in stream options:

function playStream(session) {
    var streamName = $('#streamName').val();
    var options = {
        name: streamName,
        display: remoteVideo,
        flashShowFullScreenButton: true,
        constraints: {audio: false, video: true},
        playWidth: 952
        playHeight: 693
    };
    ...
    session.createStream(options).play()

You can also enable transcoding on server level with the following parameter

disable_streaming_proxy=true

Note that transcoding requires much CPU resources on FullHD streams.

 

[Neil Canham]

Thanks for the detailed analysis. I've tried the first set of config changes - which as far as I can see require changing two properties files - flashphoner.properties and server.properties (is that right?). Then restarted the server. This has not helped, still spinning wheel. I didn't understand the need for transcoding - the client is sending the stream sized according to the current size of the application window for the application being streamed. Does the player have to match exactly those dimensions before it will work?

 

[Max]

which as far as I can see require changing two properties files - flashphoner.properties and server.properties (is that right?)

No, you have build 5.1.3702, all the settings should be placed to flashphoner.properties only

I didn't understand the need for transcoding

Transcoding is needed to make the stream more even to play it via WebRTC.
Transcoding may be enabled for all the streams on server by setting

disable_streaming_proxy=true

or per subscriber by explicitly setting stream resolution (example that we've provided above) or stream bitrate.
Another possible solution can be MSE playback, it can be tested using Player page parameter

https://your_server:8888/client2/examples/demo/streaming/player/player.html?mediaProvider=MSE

In this case, if stream is played normally, transcoding is not required.

 

[Neil Canham]

Ah finally! This was the key:
disable_streaming_proxy=true

Now I have something working - thanks for the support.

One last question - in my intended use, the client running ffmpeg will initiate the stream. Where should I start looking to understand how to set the web player to automatically detect that stream and start playing? Currently as soon as the stream goes away the player stops and won't restart without pressing 'start'

 

[Neil Canham]

Seems I was too quick to say that worked - it sort of works. But after 10s the live stream in the player freezes. sometimes making a big change like moving a window on the client unfreezes it. But mostly not, it just is stuck. The sending stream is fine. So still not reliable enough to be used. I guess some other setting I must change?

 

[Max]

Hello Neil.

I guess some other setting I must change?

Try to set ffmpeg to send keyframes regularly with keyint option, for example every 1 second

ffmpeg -f gdigrab -i "title=Untitled - Notepad" -r 10 -framerate 10 -preset ultrafast -vcodec h264 -x264-params keyint=10:scenecut=0 -strict -2 -f flv rtmp://your_server:1935/live/stream_ffmpeg

Note that this option should be set in frames, not in seconds, and in this case its value should be equal to framerate.

as soon as the stream goes away the player stops and won't restart without pressing 'start'

You can handle STREAM_STATUS.STOPPED event in player script, set the timeout and try to restart playback periodically, for example

    stream = session.createStream(options).on(STREAM_STATUS.PENDING, function(stream) {
    ...
    }).on(STREAM_STATUS.STOPPED, function () {
        console.log("streamStatus",stream.status());

        setStatus(STREAM_STATUS.STOPPED);
        onStopped();
        //try to restart
        if (!isManualStopped) {

            retryToRestart();
        }
    }).on(STREAM_STATUS.FAILED, function () {
    ...
    });
    stream.play();

function retryToRestart(){
    if (retryCount < retryMaxTimes){
        setTimeout(function(){
            if (stream
                && (stream.status() != STREAM_STATUS.PLAYING)
            ){
                start();
                retryToRestartTimeout = retryToRestartTimeout + addMilesecondsToRestartTryOnEveryFailed;
                retryCount++;
            }
        },retryToRestartTimeout);
    }
}

 

[Neil Canham]

I'm very grateful for your continued support on this. Adding the keyframes option as you suggested had the immediate benefit of much faster start for playback. However, it has increased the bandwidth required by a factor of 10! I can see the outgoing stream for ffmpeg which settles around 200kbits/s without keyframe set now never drops below 2500kbits/s with those params added. This in turn means that the stream quickly freezes again

 

[Max]

You can limit outgoing stream bitrate with the following ffmpeg options, for example:

-b:v 100K -minrate 50K -maxrate 150K -bufsize 1M

In this case, ffmpeg will try to hold average bitrate near 100 kbit/s, but not higher than 150 kbit/s. bufsize option sets the buffer size to calculate average bitrate.
Even more, if you specify bitreate for ffmpeg, you do not need to set keyframes option, because ffmpeg send packets regularly to conform bitrate specified.
So try to use ffmpeg with this settings:

ffmpeg -f gdigrab -i "title=Untitled - Notepad" -r 10 -framerate 10 -preset ultrafast -vcodec h264 -b:v 100K -minrate 50K -maxrate 150K -bufsize 1M -strict -2 -f flv rtmp://your_server:1935/live/stream_ffmpeg

Try to change ffmpeg bitrate parameters to conform your bandwith limits and to let stream flow smoothly.

 

[Neil Canham]

Thanks - that has helped to control the bandwidth