After 2K connections no more users able to connect stream

[Shark Skill]

Dear sir,

We already tested same server with 5K+ users before 4 to 5 months but now we are not able to get connections more than 2K.
We have following server configuration:
CPU: 2× AMD Epyc 7402 - 24c/48t - 2.8 GHz/3.35 GHz
RAM : 256 GB ECC 2400 MHz
DISK: 2×480 GB SSD SATA
Public Bandwidth: 4 GBPS Out going / Incoming

We have Flashphoner monthly subscription.

Also we setup proper flashphoner.property file as well as java heap base on your documentation of https://docs.flashphoner.com/display/WCS52EN/Before+moving+to+production
and another one for thousands of users:

What kind of server do you need to run a thousand WebRTC streams? | Flashphoner Streaming & Calls for Web

flashphoner.com

We cover almost all things of above two links.

 

[Max]

Good day.

We already tested same server with 5K+ users before 4 to 5 months but now we are not able to get connections more than 2K.

If no setup was changed, and server load is the same as in previous tests, it seems like channel bandwidth is decreasing. Please try to switch to TCP transport and check the player channel as described here while testing.

 

[Shark Skill]

Hi,

Thank you for quick response but I do not think channel bandwidth has any issue as we have enough bandwidth to handle users and stream. Also we already setup TCP transport.
Please review following configuration of flashphoner.properties

#webrtc ports range
media_port_from =20001
media_port_to =40000

streaming_distributor_subgroup_enabled=true
streaming_distributor_subgroup_size=50
streaming_distributor_audio_subgroup_size=500
streaming_distributor_subgroup_queue_size=300
streaming_distributor_subgroup_queue_max_waiting_time=5000
streaming_distributor_audio_subgroup_queue_size=300
streaming_distributor_audio_subgroup_queue_max_waiting_time=5000
enable_extended_logging=false

rtp_receive_buffer_size=131072
rtp_send_buffer_size =131072
streaming_video_decoder_fast_start=false
rtp_paced_sender=true
rtp_paced_sender_initial_rate=200000
rtp_paced_sender_increase_interval=50
rtp_paced_sender_k_up=0.9
webrtc_cc_min_bitrate=8000
webrtc_cc_max_bitrate=10000
webrtc_cc2_twcc=false

#codecs
codecs =opus,alaw,ulaw,g729,speex16,g722,mpeg4-generic,telephone-event,h264,vp8,flv,mpv
codecs_exclude_sip =mpeg4-generic,flv,mpv
codecs_exclude_streaming =flv,telephone-event
codecs_exclude_sip_rtmp =opus,g729,g722,mpeg4-generic,vp8,mpv

#websocket ports
ws.port =8080
wss.port =8443

#client_mode=false
rtc_ice_add_local_component=true
rtp_force_synchronization=true
webrtc_aes_crypto_provider=JCE
ice_tcp_transport=true

ice_tcp_send_buffer_size=1048576
ice_tcp_receive_buffer_size=1048576

#api auth
disable_rest_auth=false

 

[Shark Skill]

Hi,

Any update base on configuration, can you please provide your suggestion as soon as possible.

 

[Max]

1. Remove the following lines

rtp_paced_sender=true
rtp_paced_sender_initial_rate=200000
rtp_paced_sender_increase_interval=50
rtp_paced_sender_k_up=0.9

because we do not recommend to use this obsoleted feature.
2. Seems you limit a maximum video bitrate to 10 kbps

webrtc_cc_min_bitrate=8000
webrtc_cc_max_bitrate=10000

The values are very low. In this case, server will send REMB to a publishing client permanently, this increase publisher channel load. Anyway, 10 kbps gives an extremely low playback quality.
You possibly mean 8-10 Mbps:

webrtc_cc_min_bitrate=8000000
webrtc_cc_max_bitrate=10000000

However, in this case any publisher and any viewer must have an effective channel bandwidth from client to server at least 20 Mbps.
Also we do not recommend to use one server for more than 2000 subscribers. If you want more, consider a CDN (1 origin + 2 edge servers for 4000-5000 subscribers). See this article about deploying a CDN and this doc about technical details.

 

[Shark Skill]

Hi,

Thank you for your response.
We changed it as per your suggestion, I hope now we have more users there.

We already tested 6K + users with same server without CDN implementation so what do you think we need to create another nodes for CDN? and if we want to do CDN then we can use same server with different node or we require to setup new servers for the same?

 

[Max]

We already tested 6K + users with same server without CDN implementation so what do you think we need to create another nodes for CDN?

You should use CDN to accept a maximum viewers number per one server that can be stable handled. 2000 subscribers per server is a common recommendation in this case.

and if we want to do CDN then we can use same server with different node or we require to setup new servers for the same?

One server (physical or virtual) is one CDN node (origin, transcoder or edge). You cannot combine two CDN roles at one server. Please read the article to clarify.