WCS Hangs... or not?

[serviciosenlared]

Hi!

We've got a WCS server working correctly (in all functions). All is perfect during beta time, but today we move it to production and starts the problems.
The service is working but suddenly the service don't respond... imposible to connect via websocks, or web panel... nothing works. In our server Ubuntu 20.04 via SSH the service shows as ACTIVE without any error. If we restart service, then all come back to normal function, for the next error, in loop. This occur every 3 or 4 hours.

We've tried to read and understand the logs, but... we haven't been able to find what the problem is. We do not understand that during the beta it has always worked and has not been stopped, and now in production, when our clients were connecting, it failed.

It may not be the same problem but we have received some complaints from customers that they have not been able to connect to the video server, both for streaming and for viewing. We have detected that the problem originated when they tried to connect via wss to create session, but it has only happened with some clients, others have worked perfectly. Until the total fall of the service described before. Can it be the same problem?

Thank you!

 

[Max]

Hello

If you send ssh access, we will check logs.

Also you can make report and send us the tar archive using the same form:

cd /usr/local/FlashphonerWebCallServer/tools
sudo ./report.sh --sysinfo --conf --stats --tar

Regarding websocket connection. What is server version?
If server does not handle ws connection properly, do the following:

jstack 11111 > jstack.txt

here 11111 is PID of your WCS server process
jstack is a tool supplied with JDK
Send us the jstack.txt file

 

[serviciosenlared]

Hi!
I've sent SSH ACCESS, if you need any other thing, please, request me.
Thank you very much for your help!

 

[Max]

We checked your server. Seems like streaming is working at the time of test:

But you're using the default Java heap settings (1024 M). This may be not enough in production environment. Please set Java heap to 1/2 servers RAM (16 Gb in your case) in the wcs-core.properries file:

-Xmx16g
-Xms16g

Also, you're already using JDK 15, so it's strongly recommended to set up ZGC garbage collector by adding to the wcs-core.properries file:

-XX:+UnlockExperimentalVMOptions -XX:+UseZGC -Xms16g -Xmx16g

We also noticed in server logs attempts to play a stream which is not published on server every 200 ms:

00:00:51,552 ERROR         MediaHandler - API-ASYNC-pool-13-thread-237 playStream actualSession doesn't exists, session name 1642377691599465-2
00:00:51,752 ERROR         MediaHandler - API-ASYNC-pool-13-thread-235 playStream actualSession doesn't exists, session name 1642377691599465-2
00:00:51,901 ERROR         MediaHandler - API-ASYNC-pool-13-thread-242 playStream actualSession doesn't exists, session name 1642377691599465-2

We recommend to check the stream availability less often, every 1 second for example. Or, use /publishStream REST hook to collect the current state on backend and to notify the client about stream availability to play.

 

[serviciosenlared]

Hi!
Thank you so much for the info. I'll change it ASAP.

Viewing the screenshot, I've got a question: How you connect to IP and not to domain if SSL certificate is only attached to a domain? You don't receive a connection error? or only you change it for the screenshot? Thank you!

 

[Max]

Hello

Viewing the screenshot, I've got a question: How you connect to IP and not to domain if SSL certificate is only attached to a domain? You don't receive a connection error? or only you change it for the screenshot?

You can manually accept not trusted certificate in browser if you open page directly:

https://ip:8443

Once you accepted server's certificates, you will be able to connect by IP.

wss://ip:8443

Example:

 

[serviciosenlared]

Oh perfect!
Thank you very much for your support!
Greetings

 

[serviciosenlared]

Hi Max!

Server hang is solved, thank you very much, but now I've found another problem.
I've got this javascript code to connect to stream:

session.createStream({
name: %%roomname%%,
display: remoteVideo,
constraints:{audio:true, video:videoplayer}
}).on(STREAM_STATUS.PLAYING, function(stream){
openvideoroom();
}).on(STREAM_STATUS.STOPPED, function(){
closevideoroom();
}).on(STREAM_STATUS.FAILED, function(){
errorvideoroom();
}).play();

Almost 90% of users connects correctly, but some of them always goes to "STREAM_STATUS.FAILED". How can I do to know why?
But can I receive any error description or message in "STREAM_STATUS.FAILED"? How to receive it?

And another question, is possible to stop or kill "event listeners" created in "createStream"?

Sorry if my questions are too obvius, this world is quite new for me.

Thank you again!!

 

[Max]

Almost 90% of users connects correctly, but some of them always goes to "STREAM_STATUS.FAILED". How can I do to know why?
But can I receive any error description or message in "STREAM_STATUS.FAILED"? How to receive it?

Please read here about receiving an error info: WebSDK error handling
Also, please red this article What kind of server do you need to run a thousand WebRTC streams? about tuning and testing WCS for a huge number of subscribers. See the "Preparing for testing" chapter for a useful tweaks for high server load.

And another question, is possible to stop or kill "event listeners" created in "createStream"?

Your code will not receive any stream related events after STREAM_STATUS.STOPPED or STREAM_STATUS.FAILED, so you should not kill anything. If you've created an event listener in your custom code with addEventListener, you should remove it with removeEventListener when you don't need it any more.