Delay after aging

[ksko]

I'm testing aging, and I checked it 16 hours later, and there's a delay of about 8 seconds.

Browser : Chrome 90.0.4430.72
Test app : "Player" in the demo

The left one started 16 hours ago, and the right one just started.

Is there anything I can look for to solve the problem?

 

[Max]

Good day.
Please clarify how do you publish a test stream (WebRTC, RTMP) and what parameters do you use (resolution, bitrate).
Also, is this delay reproducing during stress test like mentioned in this thread?
Please enable stream distribution optimization to decrease stream playback delay in case "one publisher to many subscribers"

 

[ksko]

I'm testing in the thread environment you mentioned, and I think I've applied everything you recommend.

This is a test environment

WCS#1 : 16 Core 32 Thread CPU, 64Gb RAM, 32Gb Heap, 10Gb Fiber publish WebRTC (demo->Streaming->Media Devices) resolution : 1280 x 720 (no mixing) min : 1Mbps, max : 3Mbps WCS#2 : pull 1000 WebRTC (demo->Console->Pull Streams) PC #1 : play WebRTC (demo->Streaming->Player)

 

[Max]

Please clarify: do you play the stream with delay from WCS #2 or from WCS #1? If from WCS #2, do you play the same stream in both cases?
Please note that we recommended to enable either stream distribution optimization or traffic encryption in a separate thread for each client session on WCS server under test. In the first case, the delay should be less, but CPU load will be more. In the second case, CPU load will be less, but delay will be more.
It is also possible that your playback channel is not clean enough to play FullHD with 3 Mbps bitrate. In this case, enabling TCP transport for the client will help to prevent delays caused by quality issues:

session.createStream({
    name: streamName,
    display: remoteVideo,
    transport: "TCP"
}).on(STREAM_STATUS.PENDING, function (stream) {
...
}).play();

 

[ksko]

PC #1 : play WebRTC (demo->Streaming->Player) from WCS#1
ISP bandwidth : 1Gbps

and I enabled "stream distribution optimization" and "traffic encryption in a separate thread for each client session" at WCS#1
The left one in the first image only has a delay, and a new attempt to access(the right one) it is normal. I've screen captured both player at the same time.

The difference between the two sides is eight seconds, as shown in the image. (10:39:43, 10:39:51)
Is it better to use tcp for live broadcasting, as I thought using udp would drop packets and reduce latency?

 

[Max]

Is it better to use tcp for live broadcasting, as I thought using udp would drop packets and reduce latency?

TCP transport allows to prevent packet loss, so it may be useful if there are some channel issues.

and I enabled "stream distribution optimization" and "traffic encryption in a separate thread for each client session" at WCS#1

Please try to enable one of the tweaks, but not both.
The delay seems to have a same reason as high LA in this post. Please try to test with lower resolution/bitrate (480p/1 Mbps max for example). If load average and delay will be lower in this case, it shows that more powerful server needed.

 

[Max]

Good day.
We made a load test with 1000 subscribers per 720p stream on the following server: 24 CPU cores (48 HT cores), 192 Gb RAM (24 Gb for Java heap), 10 Gbps. The test passed succesfully with no delay while stream playback under the following conditions:
- Stream distribution optimization enabled
- ZGC enabled
- Traffic encryption hardware acceleration enabled
- The second server imitating the subscribers should be at least twice as powerful as tested server, or at least two servers should be used to distribute a testing load, 500 subscibers on each server.