mixer drop frame

[sangsoo]

Hello.
I have a question about the Stream Mixer feature.
Mixer startup, creation and deletion work fine.
• WCS Version: v.0.5.28.2753-5.2.417-e05c9d7e8db5c35e81c30d4d9061825ccf78a7b9
• Test Server (for Test): AWS EC2 C5.2xlarge (8 vCPU, 16G MEM)

If you add streams to the mixer one by one, "Drop Video Frame ~" occurs.
In the same time server log, the following is also confirmed:
02:59:20,381 WARN D - MIXER-AGENT-mixer://mixer1-41500aed-0d3a-4406-bc6d-b1e9911bb56b Drop video frame from mixer's buffer on ready check
02:59:20,422 WARN D - VideoProcessor-723e8554-682c-44cb-bdc4-9d15d9969813 Drop video frame from mixer's buffer
On htop, CPU is usually 50-60% and memory is 4G used.
(*)mixer-fhd4_DropVideoFrame2.png

>>> Question.
1. What does the log mean?
2. Is there a limit to the c5.2xlarget EC2 Instance Spec?
3. Below is the test environment.
Mixer streams are created in FHD, 4mbps
-Remote Stream also adds FHD, 4mbps one by one
-The problem log can be checked when 3 ~ 4 remote streams are added.
-The same problem occurs when adding 15 ~ 16 RemoteStreams of HD and 2mbps instead.
4. Is there a stable way to use Mixer? (*For example)
-Mixer stream uses 720p resolution
-EC2 instances use at least m5.4xlarge
-Mixer out buffer, custom lossless videoprocessor server configuration
(https://docs.flashphoner.com/display/WCS52EN/Stream mixer#Streammixer-Mixeroutputstreambufferization, https://docs.flashphoner.com/display/WCS52EN/Stream mixer#Streammixer-Usingcustomlosslessvideoprocessorforincomingstreamshandling
-In my experience, streaming quality has improved a lot after the following setup:
mixer_out_buffer_enabled = true
mixer_out_buffer_start_size = 400
mixer_out_buffer_polling_time = 20
*Please let me know if there are any considerations when setting the values.
-Approximate method of calculating mixer performance (understand load test guide)

>>> Additional question. You have configured a Custom Mixer Layout.
I used the same TestLayout code in the Doc Guide.
(https://docs.flashphoner.com/display/WCS52EN/Stream mixer#Streammixer-Custommixerlayoutimplementation)
I can hear sound, but I can't see the screen. (testlayout.jar file link.)
- https://drive.google.com/file/d/1D7OH_BAZr06_wd2O1jh2mVJ-6npWRh4A/view?usp=sharing

Then please comment. Thank you.
Best Regards.

 

[Max]

Good day.
The log means that mixer drops incoming frames. Your server configuration seems to be good enough for your test case. The reason for frames dropping can be one of:
1. The publisher channel bandwidth from client to server is not enough for the streams published. Amazon declares 10 Gbit for C5.2xlarge, but this is only on server side. So please check the channel quality using this method for WebRTC (set up the server and test) or iperf for RTMP
2. There are not enough encoding threads. By default, WCS uses 2 threads to encode 720p and higher streams. You can try to increase this value to 3 threads for example:

video_encoder_max_threads=3

And yes, if mixer output bufferization helps you, using custom losless videoprocessor as described here should also help to improve mixer output stream quality, if you're experiencing some issues with it. Note that this may degrade realtime performance: the more buffer size, the more latency.
Stream mixer uses transcoding, so mixer perfomance will be equal to transcoding perfomance on the same system. Approximately, 1 CPU core can transcode 2 720p streams (and your test confirms this, 16 HD streams requires 4 vCPUs of 8).
About custom mixer layout, we raised internal ticket WCS-2419 to check it.

 

[Max]

Also please collect a report as described here, we will check GC logs.

 

[sangsoo]

Dear Max.
You are right.
Using obs to lower the setting of one of the publish streams (resolution, bitrate) has greatly improved the symptoms.
The cpu usage is stable and no unnecessary server logs are shown.
There was not enough media traffic for the stream mixer.
Attached video is 6 stream mixing and flashphoner recording video. Add for reference purposes.
- https://drive.google.com/open?id=1It-pFhPgn1xgDrZ2WiuDLSpCsTNs8Rd0

Now the next task is to improve picture quality and control the sound of each media.
We plan to use custom layout to improve image quality.(Minimize resolution change)
The audio control seems to be only an RTMP stream. Please fix the media audio controls.

I will test the GC log for further analysis.
Please comment if there is more tuning available.
Thanks.

Best Regards.

 

[Max]

Good day.

Please comment if there is more tuning available.

Look at your gc_core.log. If Full GC works often, consider to use an instance with more RAM (32 Gb) and set Java heap size to 16 Gb (it is 8 Gb now as we see in screenshots).

We plan to use custom layout to improve image quality

Mixer custom layout is intended only for incoming streams pictures arrange. You cannot change picture resolution here, you can only resize viewable image.
So, to improve quality, you can:
1. Use good channels for streaming FullHD pictures (note that mixer resolution should be equal to incoming streams resolution for best quality).
2. Reduce incoming streams picture resolution to maximum allowed for channels quality (for example, 848x480).
3. Use mixer tweaks (bufferization and custom losless videoprocessor). Note that you cannot make good picture from bad source.

The audio control seems to be only an RTMP stream. Please fix the media audio controls.

Audio volume can be changed by publishing or playing client only. On server side, you can only change audio codec and/or samplerate.