IDecodedFrameInterceptor for all streams

Hello, the IDecodedFrameInterceptor works only for transcoded frames. Is there a way to capture frames from all streams regardless if the frame has been transcoded or not?
Thanks,
P
 

Max

Administrator
Staff member
Good day.
Is there a way to capture frames from all streams regardless if the frame has been transcoded or not?
You should enable decoding for all the streams and set the full interceptor class name (for example, com.flashphoner.frameInterceptor.TestInterceptor)
Code:
streaming_video_decoder_fast_start=true
decoded_frame_interceptor=com.flashphoner.frameInterceptor.TestInterceptor
WCS should be restarted to apply the changes.
But any picture changes you made in the interceptor class may be visible only for transcoded streams subscribers, because the YUV picture must be encoded. So without transcoding it may be useful only to extract and save the picture on servers disk.
 
Thanks for a reply. Interceptor is now being called for all streams and I can see logs. However, nothing is being drawn on the output stream. When I play the stream there are no pixel changes. What might be the reason?
 

Max

Administrator
Staff member
This is normal behavior if you do not transcode stream.

This is the chain with output:

Publish (input) -> Decoding -> intercepting -> Encoding -> Subscribing (output)
Code:
Transcoding == Decoding -> intercepting -> Encoding
This is the chain without output
Publish (input) -> Decoding -> intercepting -> Recognition
Note. This is decoding only case. No transcoding. No output. You can just analyse decoded frames for example apply Image recognition algorithms or any other video processing without output.
 

Max

Administrator
Staff member
You can analyse (read) frames.
But you can't change it on the fly (write changes) without transcoding.
 

Max

Administrator
Staff member
Thanks for a reply. We need to be able to change/write any frames of any stream with or without transcoding. How do we do that?
You can forcefully enable transcoding for all the streams on the server
Code:
disable_streaming_proxy=true
In this case, all the streams will be decoded, then encoded. A picture changes made by interceptor class will be w=visible for all the stream subscribers. But this will increase the server CPU load: you must have at least one CPU core to encode two 720p streams, or two CPU cores to encode one 1080p stream. And we recommend to use a dedicated hardware server, not virtual, because vCPU perfomance may be lower.
 
We are able to successfully save a decoded frame in YUV format. However, when trying to open it in OpenCV, we are getting a triple image. The question is: there are many YUV formats, which one do we use to convert it to OpenCV BGR format and then convert it back to WCS YUV format after processing?
 

Attachments

Max

Administrator
Staff member
The question is: there are many YUV formats, which one do we use to convert it to OpenCV BGR format and then convert it back to WCS YUV format after processing?
WCS uses YUV420 internally, so you should use YUV420 to BGR and back to YUV420 conversion.
 
For some reason, inside the "public void frameDecoded(String streamName, YUVFrame frame)" method, the "streamName" parameter is always "Stream_NAME" instead of being the variable stream name. What could be the reason?
 
Top