WebRTC Multi-Part Recording Error

AlanM

Member
When recording with Flashphoner and using file splitting, the server sometimes will start spitting out unusable files every couple seconds. We have observed this issue since ~5.0.31xx, up until the newest version of 5.1.3415. Sometimes it takes a couple minutes for this issue to happen, sometimes it takes days. It seems to happen at random.

To reproduce this issue, we ran on Flashphoner 5.1.3415. We ran the broadcast from the Stream Recording Demo page. It was configured to split the recorded file every 5 minutes.

Streaming began at Aug 10 18:24. Recording worked fine until Aug 11 03:29. It continued dumping files until the stream was stopped at Aug 11 12:48. In 18 hours, 24 minutes, Flashphoner created 31,594 files. (1 file every 2.09 seconds).

The server had plenty of storage. After we stopped the stream, only 9% of the disk was used.
There was nothing abnormal in any of the log files. The internet connection to the server / broadcast computer stayed consistent throughout the test.

Here is a look at the recordings right as it failed:
upload_2018-8-13_8-51-18.png


Here is a look at the Flashphoner.properties file:
upload_2018-8-13_8-49-9.png


Here is a look at the server logs at the point that the recording failed:
upload_2018-8-13_9-22-44.png
 

Max

Administrator
Staff member
Thanks for detailed report.
We are working on fix. Internal case is WCS-1307.
We will inform you once we have any progress.
 

AlanM

Member
Tested on 3482 and it is still broken.
We setup 2 brand new servers to test with. The only config option that we used was:
record_rotation=120
Besides that, all other config files are using default settings. There was 1 stream broadcast to each server, and there were no subscribers.
We broadcast using the Stream Recording Demo page.
Both servers randomly began spitting out the small files, and then randomly stopped recording any files at all. One server had 59903 files, the other had 47107 files.

We will test again using the debug option and send you the log files.
 

Max

Administrator
Staff member
Hello
Try to set
Code:
webrtc_cc2_twcc=false
It should fix recording error.
 

AlanM

Member
Tested on Flashphoner 3496 with the following config:

record_rotation=120
webrtc_cc2_twcc=false

Recording still appears to be broken. We ran a stream for 112 minutes, and recorded 2007 files (new file every ~3.3 seconds).
 

Max

Administrator
Staff member
Hello
We have fixed this issue in build 3502
https://docs.flashphoner.com/display/WCS5EN/Release+notes
So it should work properly in latest 3511 too.

We found another issue related lost packets. On some rotations recording files can be empty.
To avoid this issue you have to add setting:
Code:
periodic_fir_request=true
Eventually you need for good recording:
1. Latest 5.1 build, i.e. 3511.
2. webrtc_cc2_twcc=false
3. periodic_fir_request=true
 
Top