RTSP to Webrtc to stream, but can i send text to the broswer at the same time? Thanks

[tonyportersec]

I am using Live555 to generate the rtsp link for the WCS to connect (RSTP to Webrtc), but i also want to send the coordinate of a Rectangle or some text, so that i can draw it on the video on broswer, how could i do it? and the drawing need to synchronized with the video in time?

 

[Max]

Good day.
You can add watermark to the stream on server:
1. Capture RTSP stream
2. Create a transcoder using REST API:

POST /rest-api/transcoder2/startup HTTP/1.1
HOST: wcs:8081
content-type: application/json
 
{
    "uri": "transcoder2://tcode2",
    "localStreamName": "watermarked_rtsp_stream",
    "remoteStreamName": "rtsp://rtsp_server:554/stream",
    "encoder": {
      "videoCodec": "H264",
      "watermark": "/opt/watermark_picture.png" 
    }
}

3. Viewers are playing watermarked_rtsp_stream
Please read the details about transcoding here. Note that it requires a lot of server resources: 1 CPU core per 2 720p streams, or 2 CPU cores per 1 1080p stream, server RAM should be at least 64 Gb (Java heap 32 Gb)

You can also draw the picture in clients browser using HTML5 canvas element. Please see this example which shows how to draw the picture from HTML5 video element on canvas, and this article about canvas streaming.
In this case, you can send the text (or picture id) to the client using /data/send REST API query in conjuction with /connect REST hook which is needed to detect client session Id. You can also implement your own signaling to send a text to the client.

 

[tonyportersec]

Thanks, i tried /data/send API, it return 200, but how can i receive the text on the web sdk? Any example? Thanks

 

[Max]

Please read the following thread. The case is to switch to another stream by receiving a message from server. In your case, you will draw something on canvas when message is received.

 

[tonyportersec]

Thanks, i can recieve the text sent by /data/send API.
But is it possible to add something on the rtsp stream and receive it when drawing the video on broswer, so that i can draw or show text on the same time with the specific video frame?

 

[Max]

But is it possible to add something on the rtsp stream and receive it when drawing the video on broswer, so that i can draw or show text on the same time with the specific video frame?

No, this is not possible yet. We plan to add custom data sending from publisher to subscribers for WebRTC translations (ticket WCS-3105), but for RTSP stream captured it will still require a REST API query to the server. And it will never be synchronized with a certain video frame.

 

[Max]

You can use server-side decoded frames interceptor custom class. In this case, you can add any picture or text to every stream frame, or even call some external software (face recognotion etc). Please note that this feature requires transcoding to decode and then encode frames modified.

 

[tonyportersec]

Is it possible to get the raw video byte data (encoded video data) from the session on the browser WCS SDK?

 

[Max]

Is it possible to get the raw video byte data (encoded video data) from the session on the browser WCS SDK?

No, it is impossible