Snapshot timeout

[wyvasi]

Hello,
I am getting this warning every few times I try to get a snapshot and it returns status 500.
Can I increase the timeout or do something about this?

WARN  RestStreamController - HTTPS-pool-5-thread-347 Snapshot response timeout

 

[Max]

Good day.
The response timeout in this case is 3 seconds (fixed value). Usually, this is enough to take a stream snapshot. Please check your server configuration: for example, if you're using a minimal supported server config (1 CPU core) and trying to take 1080p stream snapshot, the CPU is not enough to do this. Or, on cloud server, if vrtual storage is used, dick I/O may be not enough for a continious stream snapshotting (for example, to snapshot every frame).
We raised the ticket WCS-3351 to add the setting for stream snapshot timeout, but strongly recommend to review server configuration at this point.

 

[Max]

Good day.
We added a maximum snapshot taking duration configuration in build 5.2.1116, by default 30 attempts to check if snapshot file is ready will be performed during 3000 ms

snapshot_taking_interval_ms=3000
snapshot_taking_attempts=30

You can expand this interval if CPU or disk I/O is not enough to take 1080p snapshots.

 

[wyvasi]

Good day, does this settings help in case of POST request to /rest-api/stream/snapshot
We take like 15 snapshots in an interval of 30 seconds, we do one request at time.
Every 15 requests 1-3 fails, at least.
We have like 5 streams(720p) and 5 streams(320p) on 10cpu 64gb ram machine, we don't use transcoding, is the server to weak?
This is the Load Average when I tested: 0.22 0.33 0.35
This is the response I get:

{
    "exception": "com.flashphoner.rest.server.exception.InternalErrorException",
    "reason": "com.flashphoner.rest.server.exception.InternalErrorException, Internal Server Error, Snapshot response timeout, ts: 1648354288600, path: /rest-api/stream/snapshot",
    "path": "/rest-api/stream/snapshot",
    "error": "Internal Server Error",
    "message": "Snapshot response timeout",
    "timestamp": 1648354288600,
    "status": 500
}

 

[Max]

Good day, does this settings help in case of POST request to /rest-api/stream/snapshot

Yes, the parameters are applied to REST API query.

We have like 5 streams(720p) and 5 streams(320p) on 10cpu 64gb ram machine, we don't use transcoding, is the server to weak?

The CPU is not a bottleneck in this case, but disk I/O is. You can set up RAM drive and assign snapshots dir to it:

snapshot_auto_dir=/ramdrive

to prevent disk I/O issues.

 

[wyvasi]

Does it work with docker? I still get timeout sometimes after this changes:

sudo mount -t tmpfs -o rw,size=2G tmpfs /mnt/snapshots
sudo docker run \
    -e PASSWORD=PASSWORD\
    -e LICENSE=LICENSE \
    -e WCS_FD_LIMIT=100000 \
    -e WCS_NON_ROOT=false \
    -v /usr/local/wcs/bin/setenv.sh:/usr/local/FlashphonerWebCallServer/bin/setenv.sh \
    -v /usr/local/wcs/conf/flashphoner.properties:/usr/local/FlashphonerWebCallServer/conf/flashphoner.properties \
    -v /usr/local/wcs/conf/database.yml:/usr/local/FlashphonerWebCallServer/conf/database.yml \
    -v /usr/local/wcs/conf/wss.jks:/usr/local/FlashphonerWebCallServer/conf/wss.jks \
    -v /usr/local/wcs/conf/wcs-core.properties:/usr/local/FlashphonerWebCallServer/conf/wcs-core.properties \
    -v /usr/local/wcs/bin/on_record_hook.sh:/usr/local/FlashphonerWebCallServer/bin/on_record_hook.sh \
    -v /usr/local/wcs/logs:/usr/local/FlashphonerWebCallServer/logs \
    -v /usr/local/wcs/hls:/usr/local/FlashphonerWebCallServer/hls \
    -v /usr/local/wcs/media:/usr/local/FlashphonerWebCallServer/media \
    -v /usr/local/wcs/records:/usr/local/FlashphonerWebCallServer/records \
    -v /mnt/snapshots:/usr/local/FlashphonerWebCallServer/snapshots \
    --restart unless-stopped \
    --net host \
    --name wcs -d flashphoner/webcallserver:5.2.1146

 

[Max]

Does it work with docker? I still get timeout sometimes after this changes:

This should work in any environment.
But all the filesystems are virtual in docker, so any disk I/O operations may be slower, and this is impredictable. In docker environment, we recoddems to inscrease a maximum snapshot taking interval, for example twice

snapshot_taking_interval_ms=6000

Also, please check the recommendations from this article: if you run a number of containers simultaneously, it's recommended to bind the container to a certain host CPU cores.