Edge servers failed to load streams

M

manikandan

Member
Hi,

I have a origin and edge server configuration and it worked very well. But from today, I can't able to play a stream from edge servers

It is getting stopped automatically after few minutes. When I gone through the logs, I got following errors with following message

"info" : "Failed by RTP activity",
"description" : "Audio RTP activity",
Click to expand...
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(Unknown Source)
at org.jboss.netty.channel.Channels.fireMessageReceived(Unknown Source)
at org.jboss.netty.channel.Channels.fireMessageReceived(Unknown Source)
at org.jboss.netty.channel.socket.nio.NioWorker.read(Unknown Source)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(Unknown Source)
at org.jboss.netty.channel.socket.nio.DeadlockAwareNioWorker.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
11:32:22,361 INFO HLSStreamManager - HLSStreamProviderKeepaliveThread-66 Remove hls channel [id: 0x00a790a7, /172.31.26.233:61292 :> /172.31.77.69:8082]
11:34:55,356 INFO WSServerHandler - WS-pool-20-thread-36 Orgign: null
11:34:55,356 WARN WSServerHandler - WS-pool-20-thread-36 Close channel [id: 0x4f57bedf, /18.132.46.167:21345 => /172.31.77.69:8080] because: org.jboss.netty.handler.codec.http.websocketx.WebSocketHandshakeException 'not a WebSocket handshake request: missing upgrade'
11:36:49,261 WARN RtpActivityTimerTask - FScheduling-pool-1-thread-3 RTP ACTIVITY EVENT DETECTED!
11:36:49,261 WARN RtpActivityTimerTask - FScheduling-pool-1-thread-1 RTP ACTIVITY EVENT DETECTED!
11:36:49,261 INFO RtcMediaClient - FScheduling-pool-1-thread-3 Stop media session 554921f6-2484-4f24-a978-b20506a5ce90
11:36:49,261 INFO Agent - FScheduling-pool-1-thread-3 ICE state changed from Completed to Terminated. Local ufrag 554921f6/2484/4f24/a978/b20506a5ce9038g7k1ff54cgv5
11:36:49,261 INFO ergingDatagramSocket - FScheduling-pool-1-thread-3 Closing.
11:36:49,261 INFO StunUdpSocket - FScheduling-pool-1-thread-3 Close socket
11:36:49,261 INFO MediaSession - FScheduling-pool-1-thread-3 Stop MediaSession id: 554921f6-2484-4f24-a978-b20506a5ce90
11:36:49,262 INFO MediaSession - FScheduling-pool-1-thread-3 Stop MediaSession id: bf363345-9ebd-4017-a555-37f7e23e14e8-38-1631186066-337654-HLS
11:36:49,262 INFO MediaSession - FScheduling-pool-1-thread-3 'bf363345-9ebd-4017-a555-37f7e23e14e8-38-1631186066-337654-HLS' has been terminated
11:36:49,264 WARN MediaPortManager - FScheduling-pool-1-thread-3 Port -1 doesn't exists
11:36:49,264 INFO MediaSession - FScheduling-pool-1-thread-3 '554921f6-2484-4f24-a978-b20506a5ce90' has been terminated
11:36:49,265 INFO RestClient - API-ASYNC-pool-13-thread-75 SEND REST OBJECT ==>


INFO HLSStreamProvider - API-ASYNC-pool-13-thread-77 Provider 38-1631186066-337654 notifyStreamStatusEvent: Stream{mediaSessionId='bf363345-9ebd-4017-a555-37f7e23e14e8-38-1631186066-337654-HLS', remoteMediaElementId='null', name='38-1631186066-337654', published=false, hasVideo=false, hasAudio=true, status=FAILED, sdp='null', audioCodec='null', videoCodec='null', info='Stopped by publisher stop', record=false, recordName='null', width=0, height=0, bitrate=0, minBitrate=0, maxBitrate=0, audioBitrate=0, gop=0, quality=0, rtmpUrl='null', parentMediaSessionId='null', createDate=null, endDate=null, streamInfo=Context{custom={}, nodeId='null', appKey='null', sessionId='null'}, mediaProvider='null', timeShift='null', transport='UDP', profile='null', level='null', preset='null', profiles='null', audioState='MediaState{muted=false}', videoState='MediaState{muted=false}'} Context{custom={}, nodeId='null', appKey='null', sessionId='null'}
11:36:49,321 INFO WCS4Handler - DISCONNECT-CLIENT-pool-6-thread-62 Disconnect client: com.flashphoner.server.client.MediaWCSClient@39a6d6ec
11:36:49,321 INFO WCS4Handler - DISCONNECT-CLIENT-pool-6-thread-62
[-- BEGIN CLIENT LOG --]
CONF: ClientConfig{rtspMedia=falsertmpAgent=false, wsTunnel=false, login='3v4rabok4ve7o6s11j5strvlga', authenticationName='null', password='null', domain='null', outboundProxy='null', port=0, visibleName='null', regRequired=false, applicationName='null', swfUrl='null', qValue='null', pAssociatedUri='null', contactParams='null', authDate=Thu Sep 09 11:16:47 UTC 2021, authToken='38-1631186066-337654-HLS', logsFolderName='null', clientVersion='null', clientOSVersion='null', clientBrowserVersion='null', wsTunnelPacketization2='false', custom='{aclAuth=d2568945-1c6a-40c3-956a-c5ee5425e27e}'}
IMPL: com.flashphoner.server.H.H@22089bd3
Click to expand...
I am using AWS Marketplace AMI with version 5.2.983

This affects our production very badly.
 
Max

Max

Administrator
Staff member
Hello

All the streams are traced in SDR logs

WCS Core logs - Web Call Server 5.2 - EN - Flashphoner Documentation

docs.flashphoner.com docs.flashphoner.com

1. Reproduce your issue
2. Find affected stream in the latest sdr.log
3. Attach full sdr.log and logs/server/flashphoner.log

Report form

From your logs we can see that stream between Origin and Edge is stopped by RTP Activity

Code: 
11:36:49,261 WARN RtpActivityTimerTask - FScheduling-pool-1-thread-3 RTP ACTIVITY EVENT DETECTED!
11:36:49,261 WARN RtpActivityTimerTask - FScheduling-pool-1-thread-1 RTP ACTIVITY EVENT DETECTED!
This means no audio or no video component on the stream. You can try to disable RTP activity detecting on Edge server.
rtp_activity_audio=false
rtp_activity_video=false

However, disabling RTP Activity is not a fix. To fix it we should find out why no data (audio or video) received from Origin.
You can dump metrics from Origin server and from Edge server and check why no audio or video data on the stream.

Streams management - Web Call Server 5.2 - EN - Flashphoner Documentation

docs.flashphoner.com docs.flashphoner.com

If you see audio or video metrics like audio sync, video sync, timestamp. This means you have data on the stream.
 
M

manikandan

Member
After checking further, it seems Master Origin server ( we have one master server origin and slave origin servers) fails to connect to edge server. I guess that is why the edge servers can't stream smoothly.
Is that be a reason?

I just launched a new set of servers and replaced the faulty ones. It works fine now.

But I would like to know the possible causes for this issue?
 
Last edited:
You must log in or register to reply here.
Top