Re-stream RTP desktop capture to browser for 20-30 users

Discussion in 'Web Call Server 5' started by Neil Canham, Jan 21, 2019.

  1. Neil Canham

    Neil Canham New Member

    Hi
    I have a need to capture and broadcast desktop with low latency to 20-30 users. The end users need to be able to just use HTML5 video tag to view the desktop. Currently I've managed to get a low latency, high quality desktop capture to server using FFMPEG to generate and send h264 encoded RTP stream. I tried using VLC to transcode and restream to HTTP/ogg/theora but this works badly.

    Is this something I could do using Flashphoner and if so is there a good example of how to set that up so I can try it?

    Thanks
  2. Max

    Max Administrator Staff Member

    Hello.
    You can capture WebRTC stream from desktop to WCS server as described on this page. By default, it is H264 stream if browser supports this codec. Note that extension is requred to share screen from Chrome browsers.
    Then you can play WebRTC stream using HTML5 video tag as described here. We use VR player as an example, but it should work with any custom player that use HTML5.
    If you have any questions please ask.
  3. Neil Canham

    Neil Canham New Member

    Thank you for the reply. My question was more specific - I already have the desktop capture working, it can not use the browser - it is a command line invocation of ffmpeg that generates an RTP stream of h264 encoded packets which I send to a fixed port on the server. The server would need to listen to this port, accept the stream and re-package it for display in the browsers of the connected clients. Is this something that I can do with Flashphoner? And if so can you help me to understand how I would do that?

    Thanks
    Neil Canham
    CTO vComm Solutions AG
  4. Max

    Max Administrator Staff Member

    Flashphoner WCS server does not support raw H.264 RTP stream.
    So to implement your case you have to use RTMP protocol with ffmpeg.
    If you publish ffmpeg stream to rtmp://host:1935/live/stream1 it should work.
    Please check out our ffmpeg related docs https://docs.flashphoner.com/display/WCS5EN/Using ffmpeg
  5. Neil Canham

    Neil Canham New Member

    Thanks! So now I have my own trial server working on EC2 and I'm able to stream a sample mp4 from my PC to the admin demo panel. A big step forward. However when I try to replace the source movie with capture from desktop, the stream player connects but just spins. The ffmpeg line I use is:

    ffmpeg -f gdigrab -i "title=notepad.exe" -r 10 -framerate 10 -preset ultrafast -vcodec h264 -strict -2 -f flv rtmp://<hostip>:1935/live/stream_ffmpeg

    I can see that ffmpeg is streaming, and the web player shows that connection is established, and then shows 'playing' but the wheel just spins. As far as I can see ffmpeg is streaming similar data to the mp4 example - or am I missing something important here?
  6. Max

    Max Administrator Staff Member

    Hello.
    By default, Player example waits for video and audio tracks in stream to play, but your stream has only video. You should either to add empty audio to your stream or to switch off audio in player via constraints. Please modify Player example source code as follows:
    Code:
    function playStream(session) {
        var streamName = $('#streamName').val();
        var options = {
            name: streamName,
            display: remoteVideo,
            flashShowFullScreenButton: true,
            constraints: {audio: false, video: true}
        };
        ...
    
  7. Neil Canham

    Neil Canham New Member

    Ah - excellent - thanks
  8. Neil Canham

    Neil Canham New Member

    Had a chance to try this now - still no luck with the above change to the player - still spinning wheel in the player with 'PLAYING' shown beside the Start button. In the server log I see 'not enough bandwidth' being reported:

    OBJECT:
    {
    "nodeId" : "DbnxXbU4HjGDDasKOHLUpHEaG895Za1A@127.0.0.1",
    "appKey" : "defaultApp",
    "sessionId" : "/**********:56996/**********:8443",
    "mediaSessionId" : "48842180-287e-11e9-af3e-8bfe97a3d59c",
    "name" : "stream_ffmpeg",
    "published" : false,
    "hasVideo" : true,
    "hasAudio" : true,
    "status" : "NOT_ENOUGH_BANDWIDTH",
    "videoCodec" : "H264",
    "info" : "3854480/11056",
    "record" : false,
    "width" : 944,
    "height" : 682,
    "bitrate" : 0,
    "minBitrate" : 0,
    "maxBitrate" : 0,
    "quality" : 0,
    "history" : false,
    "createDate" : 1549285851204,
    "mediaProvider" : "WebRTC",
    "origin" : "https://*************:8888",
    "constraints" : {
    "audio" : false,
    "video" : true
    }
    }
    which is odd as the stream is 10FPS desktop capture of a single window 952x693 - not demanding.
    Last edited: Feb 4, 2019
  9. Max

    Max Administrator Staff Member

    Hello.
    Please try to use lower stream resolution. It looks like your channel bandwith is not enough for 944x682 stream, especially if you test streaming and playback from the same client PC.
    You can check the bandwidth using iperf utility:
    1. Install iperf to client and server (it should be same major version on both ends, 2.* for example)
    2. Run iperf on server
    Code:
    iperf -s -p 5201
    3. Run iperf on client
    Code:
    iperf -c server_ip -t 120 -p 5201 -i 5
    This test the channel bandwidth from client to server for 120 seconds printing results every 5 seconds.
  10. Neil Canham

    Neil Canham New Member

    I've tried reducing the client streaming massively - now 562x350 (too small to be useful) and also stepped up te EC2 server instance size to t2.large - now the server.log in Web Call Server no longer reports bandwidth issue, but the Javascript console reports bandwidth issue and still the wheel spins. So I think here my journey with Web Call Server must end. Thanks for the support.
  11. Max

    Max Administrator Staff Member

  12. Max

    Max Administrator Staff Member

    Hello.
    We've checked your settings and reproduced the problem.
    The issue is not in bandwidth. When you streaming desktop with ffmpeg, video traffic goes unevenly, sometimes it can stop at all: when screen does not chenge, ffmpeg sends nothing.
    "Not enough bandwith" event is raising for that reason.
    So, we recommend to set the following WCS parameters:
    Code:
    # To switch off RTP activity timer
    rtp_activity_detecting=false,0
    # To switch keep alive off for RTMP publishsing
    keep_alive.enabled=websocket,rtmfp
    # To switch off 'Not enough bandwidth' event
    webrtc_cc2_bitrate_overuse_event=false
    
    and you should use transcoding. To enable it for client, set the resolution explicitly, in Player example it can be done via page parameters:
    Code:
    https://your_server:8888/client2/examples/demo/streaming/player/player.html?resolution=952x693
    In production, set it in stream options:
    Code:
    function playStream(session) {
        var streamName = $('#streamName').val();
        var options = {
            name: streamName,
            display: remoteVideo,
            flashShowFullScreenButton: true,
            constraints: {audio: false, video: true},
            playWidth: 952
            playHeight: 693
        };
        ...
        session.createStream(options).play()
    
    You can also enable transcoding on server level with the following parameter
    Code:
    disable_streaming_proxy=true
    Note that transcoding requires much CPU resources on FullHD streams.
  13. Neil Canham

    Neil Canham New Member

    Thanks for the detailed analysis. I've tried the first set of config changes - which as far as I can see require changing two properties files - flashphoner.properties and server.properties (is that right?). Then restarted the server. This has not helped, still spinning wheel. I didn't understand the need for transcoding - the client is sending the stream sized according to the current size of the application window for the application being streamed. Does the player have to match exactly those dimensions before it will work?
  14. Max

    Max Administrator Staff Member

    No, you have build 5.1.3702, all the settings should be placed to flashphoner.properties only
    Transcoding is needed to make the stream more even to play it via WebRTC.
    Transcoding may be enabled for all the streams on server by setting
    Code:
    disable_streaming_proxy=true
    or per subscriber by explicitly setting stream resolution (example that we've provided above) or stream bitrate.
    Another possible solution can be MSE playback, it can be tested using Player page parameter
    Code:
    https://your_server:8888/client2/examples/demo/streaming/player/player.html?mediaProvider=MSE
    In this case, if stream is played normally, transcoding is not required.
  15. Neil Canham

    Neil Canham New Member

    Ah finally! This was the key:
    disable_streaming_proxy=true

    Now I have something working - thanks for the support.

    One last question - in my intended use, the client running ffmpeg will initiate the stream. Where should I start looking to understand how to set the web player to automatically detect that stream and start playing? Currently as soon as the stream goes away the player stops and won't restart without pressing 'start'
  16. Neil Canham

    Neil Canham New Member

    Seems I was too quick to say that worked - it sort of works. But after 10s the live stream in the player freezes. sometimes making a big change like moving a window on the client unfreezes it. But mostly not, it just is stuck. The sending stream is fine. So still not reliable enough to be used. I guess some other setting I must change?
  17. Max

    Max Administrator Staff Member

    Hello Neil.
    Try to set ffmpeg to send keyframes regularly with keyint option, for example every 1 second
    Code:
    ffmpeg -f gdigrab -i "title=Untitled - Notepad" -r 10 -framerate 10 -preset ultrafast -vcodec h264 -x264-params keyint=10:scenecut=0 -strict -2 -f flv rtmp://your_server:1935/live/stream_ffmpeg
    Note that this option should be set in frames, not in seconds, and in this case its value should be equal to framerate.
    You can handle STREAM_STATUS.STOPPED event in player script, set the timeout and try to restart playback periodically, for example
    Code:
        stream = session.createStream(options).on(STREAM_STATUS.PENDING, function(stream) {
        ...
        }).on(STREAM_STATUS.STOPPED, function () {
            console.log("streamStatus",stream.status());
    
            setStatus(STREAM_STATUS.STOPPED);
            onStopped();
            //try to restart
            if (!isManualStopped) {
    
                retryToRestart();
            }
        }).on(STREAM_STATUS.FAILED, function () {
        ...
        });
        stream.play();
    
    function retryToRestart(){
        if (retryCount < retryMaxTimes){
            setTimeout(function(){
                if (stream
                    && (stream.status() != STREAM_STATUS.PLAYING)
                ){
                    start();
                    retryToRestartTimeout = retryToRestartTimeout + addMilesecondsToRestartTryOnEveryFailed;
                    retryCount++;
                }
            },retryToRestartTimeout);
        }
    }
    
  18. Neil Canham

    Neil Canham New Member

    I'm very grateful for your continued support on this. Adding the keyframes option as you suggested had the immediate benefit of much faster start for playback. However, it has increased the bandwidth required by a factor of 10! I can see the outgoing stream for ffmpeg which settles around 200kbits/s without keyframe set now never drops below 2500kbits/s with those params added. This in turn means that the stream quickly freezes again
  19. Max

    Max Administrator Staff Member

    You can limit outgoing stream bitrate with the following ffmpeg options, for example:
    Code:
    -b:v 100K -minrate 50K -maxrate 150K -bufsize 1M
    In this case, ffmpeg will try to hold average bitrate near 100 kbit/s, but not higher than 150 kbit/s. bufsize option sets the buffer size to calculate average bitrate.
    Even more, if you specify bitreate for ffmpeg, you do not need to set keyframes option, because ffmpeg send packets regularly to conform bitrate specified.
    So try to use ffmpeg with this settings:
    Code:
    ffmpeg -f gdigrab -i "title=Untitled - Notepad" -r 10 -framerate 10 -preset ultrafast -vcodec h264 -b:v 100K -minrate 50K -maxrate 150K -bufsize 1M -strict -2 -f flv rtmp://your_server:1935/live/stream_ffmpeg
    Try to change ffmpeg bitrate parameters to conform your bandwith limits and to let stream flow smoothly.
  20. Neil Canham

    Neil Canham New Member

    Thanks - that has helped to control the bandwidth

Share This Page