streaming from ReactNative app

[SlavaCh]

Hi
We experience black screen instead of stream while seting the react native app
and tried with iframe in the app.
Iframe is not loaded from the phone (iOS 13.5.1)
when using WebView and loading html page
and the link to the stream itself (https://IP:8888/embed_player?urlServer=wss://IP:8443&streamName=d398c256&mediaProviders=WebRTC) does not load when using RTCView (react-native-webrtc plugin)

we bought Amazon EC2 instance type

does anybody have a successful react-native implementation?

 

[Max]

Good day.

We experience black screen instead of stream while seting the react native app

WCS WebSDK is intended to use in browser. React Native provides no DOM, so WebSDK needs to be reworked to integrate with React Native webrtc implementation. You can try to modify WebSDK source code to solve it.

and the link to the stream itself (https://IP:8888/embed_player?urlServer=wss://IP:8443&streamName=d398c256&mediaProviders=WebRTC) does not load when using RTCView (react-native-webrtc plugin)

This seems like SSL certificate problem. Please import SSL certificate as described here and check the domain name you're using to establish connection to the server.

 

[SlavaCh]

thank you!
it helped

We would like to integrate some ML algorithms on the video stream and would like to use python.
Do you have a protocol description? or a library for python client?

 

[Max]

Do you have a protocol description?

Please read WebRTC protocol specification here.

or a library for python client?

We have no library for python client, but there is low level Websocket API. You can try to integrate it with this python WebRTC library.

 

[SlavaCh]

Please read WebRTC protocol specification here.

We have no library for python client, but there is low level Websocket API. You can try to integrate it with this python WebRTC library.

Thank you for the explanation.

However we have a few additional questions:

Question 1. How to approach a bi-directional streaming for the following case?

WebRTC agent 1 (native mobile SDK) initiates a streaming.
WebRTC agent 2 (python script) needs to consume this stream, process it and return the processed stream to the agent

2. What is the operational flow for both agents? What is the order and how they should connect to the server?

Quiestion 2. Is it possible to configure WebRTC SDP in the WCS? Python aiortc library can't parse Flashphoner SDP well in regard to fingerprints. https://github.com/aiortc/aiortc/issues/391

Quiestion 3.
we also experience a poor quilty of the video stream. 15fps and smartphone screen resolution is set on the mobile SKD side.
AWS instance from the "box" 4Gb ram purchased on Flashphoner website.

thank you

 

[Max]

Question 1. How to approach a bi-directional streaming for the following case?

WebRTC agent 1 (native mobile SDK) initiates a streaming.
WebRTC agent 2 (python script) needs to consume this stream, process it and return the processed stream to the agent

For this case, you can use another approach. You can impelement Java class to receive decoded frames on server side, then you can call python scripts from Java to handle the picture, and pass the picture to WCS again. This is not documented yet, so we raised the ticket WCS-2777 to prepare Java inteface implementation example.

Quiestion 2. Is it possible to configure WebRTC SDP in the WCS? Python aiortc library can't parse Flashphoner SDP well in regard to fingerprints. https://github.com/aiortc/aiortc/issues/391

SDP offer/answer may contain fingerprint on session level according to specification, but this is not mandatory. Most browsers supports fingerprint on media level. So it seems like third party library issue. We raised the ticket WCS-2795 to check if we can do something on our side.

Quiestion 3.
we also experience a poor quilty of the video stream. 15fps and smartphone screen resolution is set on the mobile SKD side.
AWS instance from the "box" 4Gb ram purchased on Flashphoner website.

Please try to raise publishing bitrate with the following settings on server side (assuming you're publishing HD+ resolution)

webrtc_cc_min_bitrate=500000
webrtc_cc_max_bitrate=2500000

 

[SlavaCh]

Code:
webrtc_cc_min_bitrate=500000
webrtc_cc_max_bitrate=2500000

thank you, we tried this settings but still are experiencing some freeze and video deformation (on both landscape and portrait camera mode on iPhone 10)

I assume there are some parameters on video resolution or size (camera settings) on the mobile SKD side (we use flashphoner native iOS SDK).

 

[Max]

thank you, we tried this settings put still are experiencing some freeze and video deformation (on both landscape and portrait camera mode on iPhone 10)

Please try to use WebSDK to test publishing channel bandwidth as described here. Then, choose optimal resolution and bitrate, or consider to migrate server closer to publisher, or check publisher last mile.

I assume there are some parameters on video resolution or size (camera settings) on the mobile SKD side (we use flashphoner native iOS SDK).

You can use constraints to set video publishing resolution and bitrate. Please read the MediaDevices example description (and check source code here)

 

[SlavaCh]

we set this settings on the mobile side and flashphoner.properties file
but still experience freezes

 

[Max]

You should set bitrate on publisher side, not only frameRate. Please look at source code

- (FPWCSApi2MediaConstraints *)toMediaConstraints {
    FPWCSApi2MediaConstraints *ret = [[FPWCSApi2MediaConstraints alloc] init];
    ...
    if ([_sendVideo.control isOn]) {
        FPWCSApi2VideoConstraints *video = [[FPWCSApi2VideoConstraints alloc] init];
        ...
        video.bitrate = [_videoBitrate.input.text integerValue];
        ret.video = video;
    }
    return ret;
}

On server side, please remove the following parameter

encoder_default_video_resolution=736x414

because it affects transcoding, not publishing.
Also, you're trying to raise bitrate too much: the setting

webrtc_cc_max_bitrate=10000000

means 10 Mbps per stream. Please set this value to 2500000 (2,5 Mbps, this is Chrome browser default maximum bitrate setting)
Please also try to check player and publisher channel as we recommeded in this post

 

[Max]

We added the documentation about decoded frames interception and changing on server side. The Java interface class implementation is required, but you can also invoke python from Java if needed.
Please note that pictures will be decoded only if stream transcoding is enabled, so you shoudl consider this choosing instance configuration. Usually, 1 CPU core is needed to encode 2 720p streams. Also, the frames decoded require a lot of memory, so we recommend at least 8 CPU and 16 Gb RAM, and more in production environment.

 

[SlavaCh]

Thank you for your answers.
So we should replace native ios SDK on the mobile side to WebSDK ? then send stream to web page and play with settings. correct?
and when we know the settings, we can configure ios SDK and replace it back on mobile side?

Can somebody assist us with this? beacuse we chose flashphoner AWS solution in hope it works from the box. however I see mobile streaming doesn't work from the box. and the roadmap is not clear as for now.

(decoding to frames is a separate task. for now we would like to have a stable stream with good quality - as a 1 step. and video chat in mobile app - as a step 2)

thank you

 

[Max]

So we should replace native ios SDK on the mobile side to WebSDK ? then send stream to web page and play with settings. correct?
and when we know the settings, we can configure ios SDK and replace it back on mobile side?

Yes. You have to test a channel, adopt bitrate/resolution to channel quality, then use adopted parameters in mobile application (you can add application settings rather than hardcode them).

beacuse we chose flashphoner AWS solution in hope it works from the box. however I see mobile streaming doesn't work from the box. and the roadmap is not clear as for now.

Server itself works out of the box. But there is a channel between client and server. Channels are different, client can use mobile 3G networks or even 2G that is definitely not suitable to stream a video. So you have to tweak server and client settings to a channel.

 

[SlavaCh]

actually it doesn't help.

in the code i see this parametrs

however iphone front camera has 1080*720. maybe the problem is here.
but the stream foe now works unacceptable. even 352*288 freezes on good wifi internet channel

Which parametrs should be set in Quality field? I didn't find documentation for it. 0 or GOOD?

We use native ios app on SWIFT.

if you can assist and make it work fine with fullscreen streaming then we can move further with flashphoner
thank you

 

[Max]

but the stream foe now works unacceptable. even 352*288 freezes on good wifi internet channel

Did you tested a channel quality using Web SDK in Safari browser as described here? If not, how do you determine good channel?

Which parametrs should be set in Quality field? I didn't find documentation for it. 0 or GOOD?

In the example on screenshot the quality parameter is stream encoding quality (from 5 to 20). This will enable transcoding on server, more quality, more CPU, and, therefore, more latency.

if you can assist and make it work fine with fullscreen streaming then we can move further with flashphoner

We trying to assist, please just follow the recommendations.
Please provide us SSH and publishing access to your server, we will check the server settings. Send credentials using this link.

 

[SlavaCh]

ok. we got a stable picture and the quality is good.
the only issue - on a full screen it deforms image proportion.
Could be a problem in aspect ratio.



How we can change these parameters to have ratio like on smartphone full screen?

 

[Max]

ok. we got a stable picture and the quality is good.
the only issue - on a full screen it deforms image proportion.

Please look at this post.