Implementing iOS Client without SDK - Failed by DTLS error

[fbajanik]

Hello,

I am trying to implement a simple wrapper for iOS App, to be able to distribute the app for more platforms that you currently support. My colleague has been asking you questions recently about whether this solution is possible and he got a response, that you have few clients like that.

So I started. I opened https://demo.flashphoner.com/admin/demo.html# (Streamer) and intercepted a few messages. But after I received `setRemoteDescription` and set `remoteDescription` I got `"info":"Failed by DTLS error"` from notifyStreamStatusEvent message.

Is there any way how to easily reverse-engineer your SDK to be able to connect to the demo webpage?

We are considering your solution to convert WebRTC to HLS, but we need help.

I am testing it with iOS 16.1 with the latest stable WebRTC framework with some patches from our company.

Thank you

 

[Max]

Good day.
Check if the problem is reproducing with Two Way Streaming example from the same device. Make sure you opening the page via HTTPS. Also please try to use TCP transport for WebRTC (some providers may block UDP DTLS traffic from iOS devices), this may be tested in Media Devices example.

I am testing it with iOS 16.1 with the latest stable WebRTC framework with some patches from our company.

Please use our native iOS SDK 2.6 which is available in CocoaPods (it includes a specially patched WebRTC framework for Screen capturing and some other features to work correctly). We can't guarantee other WebRTC implementations to work.
If you're using WebView or browser, please use our WebSDK 2.0 (also available as NPM package).

 

[fnuky]

Hi, We wanted to implement you SDK, but we have app based on Webrtc. We have our own version of WebRTC with our updates and PeerConnectonFactory setup so it is not possible to use your SDK. I discuss it with sales of flashphoner and they told me that It is possible to use raw websockets api and don't use you framework. You don't support arm simulator slice, mac catalyst and tvos (device and simulator). Architectures exclude is just workaround and not solving problem. We cannot switch fully to your SDK and your server. We want to provide Webrtc -> HLS Live broadcast functionality to our users and with it we wanted to start pasying your licence. Sales told me that that we can track communication in browser and implement it based on this communication but it is not working as my colleague said.

 

[Max]

You don't support arm simulator slice

Arm64 simulator slice is supported since iOS SDK 2.6.105. Please see details here: ARM64 architecture support to build on ARM based Mac

I discuss it with sales of flashphoner and they told me that It is possible to use raw websockets api and don't use you framework

Yes, this is possible. See the specification: Raw WebSocket API. In this case, you should set the following parameters in connect message

...
"clientVersion" : "2.6",
"clientOSVersion": "iOS",
"clientBrowserVersion": "iOS SDK; 2.6"
...

 

[Max]

Regarding reverse engineering of iOS SDK 2.6.105

Try the following way

1. Launch default sample, i.e. Two Way Streaming.
2. Use non-secure (because if connection is secure, you will not be able to decrypt https wss traffic) ws connection url, like ws://demo.flashphoner.com:8080
3. Do "Connect" then "Publish".
Therefore you are using unsecure ws connection which can be parsed in Wireshark
4. Capture traffic by tcpdump on server-side
tcpdump port 8080 -i ens192 -w log.pcap
Here ens192 is a network interface (do ifconfig to list all network interfaces in your system)

As a result, you have all websocket traffic in the log.pcap and you have to send exactly the same message format from your custom application.
See also Websocket Traffic Analyzing in Wireshark

Websocket - Web Call Server 5.2 - EN - Flashphoner Documentation

docs.flashphoner.com

 

[fbajanik]

In order to capture server-side logs I need a license and 30 days of trial is not enough, so I cannot do it. Is there another way?

-------------------------------------------------------------------

However...

I accidentally managed to connect successfully, I don't know why. I included logs with the flow.

One case is non-working - resulting in `lastStatus=FAILED` and another resulting in `lastStatus=PUBLISHING`.

I compared SDPs and there is no difference. So either it is the server's fault or I am doing something wrong. Could you help me with that? The flow is identical in both cases, SDPs and candidates are the same. One works, but the second one doesn't.

It looks like there is some timing issue that I can't properly reproduce.

Thank you @Max

 

[Max]

In order to capture server-side logs I need a license and 30 days of trial is not enough, so I cannot do it. Is there another way?

30 days trial is fully functional with two limits:
1. A voice watermark in all the streams
2. A logo watermark in transcoded pictures (for example, when you're publishing and playing a stream with a different resolution. So you can collect server side logs needed.
If you have any troubles to deploy a WCS instance, you can request a server setup: Server setup request
We still need server side logs and traffic dump collected as described here to help you to resolve the issue.

 

[fbajanik]

Video and audio works, but every 1-3 minutes the Flashphoner demo Player video got stucked and after a while a get `notifyStreamStatusEvent` with this data:

{"message":"notifyStreamStatusEvent","data":[{"logger":{"name":"Stream","debugEnabled":false,"infoEnabled":true,"warnEnabled":true,"errorEnabled":true,"traceEnabled":false},"protocols":["ws","wss","vod","vod-live","rtmp","rtsp","pcap"],"mediaSessionId":"C277C9AD-51F9-426F-891A-6C725CE55AF6","name":"myStream","published":true,"hasVideo":true,"hasAudio":true,"status":"FAILED","audioCodec":"opus","videoCodec":"H264","info":"Failed by RTP activity","description":"Video RTP activity","record":false,"width":864,"height":1158,"bitrate":0,"minBitrate":0,"maxBitrate":0,"quality":0,"history":false,"gop":0,"fps":0,"audioBitrate":0,"codecImpl":"","transport":"UDP","cvoExtension":false,"createDate":1673801535874,"mediaType":"publish","audioState":{"muted":false},"videoState":{"muted":false},"mixerStream":false,"namedParams":{},"streamInfo":{"nodeId":null,"appKey":null,"sessionId":null,"mediaSessionId":"C277C9AD-51F9-426F-891A-6C725CE55AF6","name":"myStream","samplingTime":null,"recordTimestamp":null,"recordStarted":false},"mediaProvider":"WebRTC","lastStatus":"FAILED"}]}

I attached a zipped log.pcap and uploaded all the logs (3.5MB) because I couldn't upload them here directly because of the limit.

https://drive.google.com/file/d/1EKE4EqWL1i1UQi46TFOK8T0flHkOBa9w/view?usp=share_link

I hope you can help me with my issue.

The phone on which is it running is iPhone 14 Pro and the server is running on Ubuntu 64-bit 20.04.5 LTS.

 

[Max]

Good day.

"info":"Failed by RTP activity"

This means there is no video data sent to the server in last 60 seconds. Unfortunately, your traffic dump contains only websocket packets, but not the media data, but the client debug logs you collected show that the last incoming video packet was received in 19:23:46 by server time

After that, the server receives only audio packets. Then, the stream publishing is stopped by inactivity.
This may happen due to packets loss on the channel between client and server, or the channel bandwidth is not enough. In this case try to use
TCP transport

           let options = FPWCSApi2StreamOptions()
           options.name = publishName.text
           options.display = localDisplay.videoView
           ...
           options.transport = kFPWCSTransport.fpwcsTransportTCP
           do {
               try publishStream = session!.createStream(options)
           } catch {
               print(error);
           }
           ...
           do {
               try publishStream?.publish()
           } catch {
               print(error);
           }

or use lower publishing resolution/bitrate to fit the channel.
Please also test the case in Two Way Streaming or Media Devices examples build from sources. If publishing is not stopping in these examples, please modify the example code minimally to reproduce the problem and send using this form.