Web SDK Snapshot issue

[fsacca]

Hello everyone,
I'm coming across an issue using Flashphoner snapshot feature with web SDK library on my sample app.
Differentry from Flashphoner website snapshot demo, I'm trying to get snapshot from a Player only Web app, while another Web app is streaming on a smartphone device.
Like that demo, everytime snapshot is requested, I'm creating a new streaming from session (using createStream method), calling snapshot() method and listening for SNAPSHOT_COMPLETE event .
Now the issue itself: I'm able to get the snapshot base64 every time but if i do another request, it seems that i get the same image as before, and next requests will have a curious pattern.
Let's say I'm trying to get this 3 subjects snapshot in this sequence: a pen , a mouse, a keyboard. If I keep asking for snapshots, I'll get this pattern: pen, pen (same), pen (same), mouse, keyboard and so on.
I already checked if it could be my software problem, but the thing is that this pattern is directly returned by Websocket response which contains image base64.
I also checked snapshot demo, which works perfectly, so i don't think the problem could be streamer's device.
Another fun fact: if I switch device orientation on streamer app (which is also managed with a video resize on player side) in between snapshots , I will always get the right image.
Some more info about my environments:

• streamer app
  • device: Xiaomi redmi 9 se
  • os: Android 10
  • browser: Chrome 81.0.4044.138
  • web SDK version: latest
• player app
  • device: PC
  • os: Windows 10
  • browser: Chrome 81.0.4044.138
  • web SDK version: latest

Hope you can help me, cause snapshot will be a key feature for our next client, thanks in advance.

Francesco.

 

[Max]

Upon snapshot request WCS server raises a video decoder for requested stream.
The decoder should receive K video frame from streamer to take a snapshot.

Please try this server-side setting in WCS_HOME/conf/flashphoner.properties

periodic_fir_request=true

This will request K frames periodically and should fix snapshot issues.
This setting is applied on our demo server.

 

[fsacca]

Thank you Max for the reply.
Unfortunately after setting that property and restarting the server, the problem still persist.
The property has been surely updated as seen on server log:

10:16:38,975 INFO SettingsLoader - main Override setting periodic_fir_request: from false to true

Strange thing on server log is this error after taking the snapshot:


11:34:38,725 ERROR HttpServerHandler - HTTP-pool-2-thread-8 HTTP error
org.jboss.netty.handler.codec.frame.TooLongFrameException: HTTP content length exceeded 1048576 bytes.
at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(Unknown Source)
at org.jboss.netty.channel.Channels.fireMessageReceived(Unknown Source)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(Unknown Source)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(Unknown Source)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(Unknown Source)
at org.jboss.netty.channel.Channels.fireMessageReceived(Unknown Source)
at org.jboss.netty.channel.Channels.fireMessageReceived(Unknown Source)
at org.jboss.netty.channel.socket.nio.NioWorker.read(Unknown Source)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(Unknown Source)
at org.jboss.netty.channel.socket.nio.DeadlockAwareNioWorker.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


I'm attaching the full log stack trace. I only took the most important log's parts starting from stream PLAYING status.
Unfortunately for this project we will have to deliver the most high quality snapshots as possible to our clients, that's why I set the high resolution on the stream.

 

[Max]

Unfortunately after setting that property and restarting the server, the problem still persist.

By default, key frames will be requested from browser every 5 seconds if periodic FIR request is enabled. So, the picture will remains the same in this 5 seconds.
Please add the following parameters

periodic_fir_request_interval=1000
rtcp_pli_request_interval=1000

to request key frames every 1 second.
But in this case publisher channel bandwidth and phone resources may be not enough to send big frame (key frames are bigger than pucture frames) every second, this may lead to stream quality drop.

Unfortunately for this project we will have to deliver the most high quality snapshots as possible to our clients, that's why I set the high resolution on the stream.

In this case, we recommend to use REST API to take snapshots than download them using separate web server (Apache, nginx etc).

 

[Max]

Please add the following setting

streaming_video_decoder_fast_start=true

In this case, every published stream will be decoded on server, and snapshot decoder will start much faster. No need to reduce keyframe interval, 5 second default will be enough.

 

[fsacca]

Wait a moment...too many things here.
Let's say we want to use snapshots from Flashphoner Rest API from now on, to be sure it's not a RTC issue and in order for the streamer app to perform better.
So now my configuration is only with theese two new properties (probably the first not needed anymore):

periodic_fir_request=true
streaming_video_decoder_fast_start=true

I tried to make REST API calls now (with "Cache-Control: no-cache" if could be useful), instead of calling for WSS snapshot, with the same subject pattern as I mentioned on the first post I wrote.
First, with api calls now I don't get those "TooLongFrameException " I had before.
Second, the pattern still remains, and it seems it is not a matter of 5 or more seconds in order to have an updated snapshot.
I tried this absurdly: I hold for 1 minute the stream on the same subject and than called snapshot API. The result was the image about the "previous 2nd subject" I snapped before.
So the thing is: snapshots seem are stored good but are returned badly, with a delay that is unlocked only asking for the snapshot two times.

 

[Max]

We reproduced the issue with previous snapshot return and raised the ticket WCS-2709 to fix it. We'll let you know in this topic.

 

[fsacca]

Hello, any news about the fix?
Thank you again.

 

[Max]

Good day.
We still working on it. Please be patient, we'll let you know when fix it.

 

[Max]

Good day.
We fixed snapshot series taking issue (WCS-2709) in build 5.2.706. Please update and check.