Twilio group sceen to RTMP

vanarie

New Member
Hello,

I've been testing with WCS and setup a test server. I was able to get webRTC -> WCS -> RTMP working from the install and read the rtmp://myserver stream with VLC media player. I assume that's a good test.

My goal now is to find a good, scalable way to port a video meeting conference screen created with Twilio to do the same thing.

The first probable solution is to use a headless Chrome instance as a "bot" and then do something like:
JavaScript:
...
navigator.mediaDevices.getUserMedia({audio: true, video: true})
    .then(function(localStream) {
    document.getElementById("local-video").srcObject = localStream;
    localStream.getTracks().forEach(track => clientPeerconnection.addTrack(track, localStream));
    return clientPeerconnection.createAnswer();
})
...
from https://github.com/muxinc/chromium_broadcast_demo/blob/master/static/client.html

Someone else suggested SIP -> WCS -> RTMP. I haven't gone down that road yet.

Thanks for any suggestions you may have with examples if possible.
 
Last edited:

Max

Administrator
Staff member
Hello

We didn't test such a scenario.

You can address to our canvas streaming example

Perhaps you can place Twilio video element(s) on the canvas.

Regarding SIP. It would be working if Twilio operates as a mixing unit. This means if we do a SIP call towards Twilio gateway, it should reply with exactly 1 audio and 1 video RTP. You can give it a try using Desktop SIP softphone like Linphone or Bria. If you able to call to Twilio from the Desktop softphone and if you see all conference participants, this indicates that Twilio operates as a mixing unit. Wireshark traffic dump needed for verification. It should contain two single RTP streams: audio and video towards the softphone.

If you have successfully established video call Desktop Softphone - Twilio, then you can establish SIP video call WCS - Twilio using REST API.
Established call can be converted to a stream. Stream can be played or republished as RTMP.

SIP as Stream feature:
 

vanarie

New Member
Let's forget SIP for the moment since Twilio doesn't provide a mixing unit. The goal is for the final "mix" is done client-side via standard html/js and is composed on the fly based on session details. So we're really just talking about sending what everyone else sees on the screen down the line, via RTMP as a passive stream with 1 audio/video signal. In the most basic sense, it would be like pointing a webcam to the meeting screen and just sending the whole composition to canvas to then capture and serve to RTMP consumer. Reminds me of using a cassette recorder to record songs off the radio speaker in the old, old days. No-one around here probably knows what I'm talking about. :D

Still learning about video services and asking (sometimes) dumb questions.

It looks like OBS supports webpage capture and streaming, so this might be an option as well. Still looking at solutions.
 

Max

Administrator
Staff member
It looks like OBS supports webpage capture and streaming, so this might be an option as well. Still looking at solutions.
WCS supports screen, application window or browser tab capturing and publishing to server as WebRTC. Please read the details here. In latest build of WCS Web SDK, a Chrome extension does not needed, just a little Javascript code. Read also this article with step by step example.
In this case, you can capture meeting screen from browser tab and publish it to WCS via WebRTC. Then, you can play the stream from server, record it or republish it as RTMP to third party server.
 
Top