IDecodedFrameInterceptor for all streams

[Petro O. Bochan]

Hello, the IDecodedFrameInterceptor works only for transcoded frames. Is there a way to capture frames from all streams regardless if the frame has been transcoded or not?
Thanks,
P

 

[Max]

Good day.

Is there a way to capture frames from all streams regardless if the frame has been transcoded or not?

You should enable decoding for all the streams and set the full interceptor class name (for example, com.flashphoner.frameInterceptor.TestInterceptor)

streaming_video_decoder_fast_start=true
decoded_frame_interceptor=com.flashphoner.frameInterceptor.TestInterceptor

WCS should be restarted to apply the changes.
But any picture changes you made in the interceptor class may be visible only for transcoded streams subscribers, because the YUV picture must be encoded. So without transcoding it may be useful only to extract and save the picture on servers disk.

 

[Petro O. Bochan]

Thanks for a reply. Interceptor is now being called for all streams and I can see logs. However, nothing is being drawn on the output stream. When I play the stream there are no pixel changes. What might be the reason?

 

[Max]

This is normal behavior if you do not transcode stream.

This is the chain with output:

Publish (input) -> Decoding -> intercepting -> Encoding -> Subscribing (output)

Transcoding == Decoding -> intercepting -> Encoding

This is the chain without output
Publish (input) -> Decoding -> intercepting -> Recognition
Note. This is decoding only case. No transcoding. No output. You can just analyse decoded frames for example apply Image recognition algorithms or any other video processing without output.

 

[Max]

You can analyse (read) frames.
But you can't change it on the fly (write changes) without transcoding.

 

[Petro O. Bochan]

Thanks for a reply. We need to be able to change/write any frames of any stream with or without transcoding. How do we do that?

 

[Max]

Thanks for a reply. We need to be able to change/write any frames of any stream with or without transcoding. How do we do that?

You can forcefully enable transcoding for all the streams on the server

disable_streaming_proxy=true

In this case, all the streams will be decoded, then encoded. A picture changes made by interceptor class will be w=visible for all the stream subscribers. But this will increase the server CPU load: you must have at least one CPU core to encode two 720p streams, or two CPU cores to encode one 1080p stream. And we recommend to use a dedicated hardware server, not virtual, because vCPU perfomance may be lower.

 

[Petro O. Bochan]

This approach worked. Thanks for an updated.

 

[Petro O. Bochan]

We are able to successfully save a decoded frame in YUV format. However, when trying to open it in OpenCV, we are getting a triple image. The question is: there are many YUV formats, which one do we use to convert it to OpenCV BGR format and then convert it back to WCS YUV format after processing?

 

[Max]

The question is: there are many YUV formats, which one do we use to convert it to OpenCV BGR format and then convert it back to WCS YUV format after processing?

WCS uses YUV420 internally, so you should use YUV420 to BGR and back to YUV420 conversion.

 

[Petro O. Bochan]

Correct, yuv420 worked out. Thanks.

 

[Petro O. Bochan]

For some reason, inside the "public void frameDecoded(String streamName, YUVFrame frame)" method, the "streamName" parameter is always "Stream_NAME" instead of being the variable stream name. What could be the reason?

 

[Max]

the "streamName" parameter is always "Stream_NAME" instead of being the variable stream name

We confirm the issue, the ticket WCS-3718 is raised. Will inform you about progress here.

 

[Max]

The issue was fixed in build 5.2.1574. Please update and check.

 

[Petro O. Bochan]

Thanks for an update. It's now working as expected.