Stream black screen on safari mobile

[Tigran]

Hi. I am testing FlashPhoner stream on safari mobile. Everything is ok.
When, during streaming on safari, i get phone call, stream video become black and not recovered.
Or when I minimize browser, use Viber, WhatsApp or other applications, then maximize browser, stream screen is black and not recovered automatically.
How can I fix this issue ?

Device: iPhone (ex. 11 Pro Max).
Browser: Safari.

 

[Max]

Good day.
According to Apple policy, camera usage is forbidden in background mode, both for browsers and native applications. The only way to proceed streaming after maximizing the browser again is to restart stream publishing.

 

[Tigran]

Hi Max.
I use below applications without camera or video-call, only text-chat, then return to my stream and its already black and not recovered.
How can i handle this event to restart stream publishing ?

 

[Max]

I use below applications without camera or video-call, only text-chat, then return to my stream and its already black and not recovered.
How can i handle this event to restart stream publishing ?

Pkease read this thread on StackOverflow and Page Visibility API documentation.

 

[Tigran]

kease read this thread on StackOverflow

In this example there are several answers`

1. window visibilitychange event
2. window focus event
3. window pageshow event

1.Here i can handle browser minimize/maximize event.
2.Here i can handle this event only clicking somewhere in browser. This event doesn't triggered onVisibilityChange event.
3.This event is not helpfull for my project.

This is my main problem.
When i am watching stream in browser and during this time i`m having phone incoming call, my stream stops automatically without closing or minimizing browser. So it is not recovered automatically. At this time there is not focus - visibilitychange events or stream any type of error.

 

[Max]

Please check the example from this page. Did it work in iOS Safari? If yes, use handleVisibilityChange to stop stream playback and then resume:

function handleVisibilityChange() {
  if (document[hidden]) {
    stream.stop();
  } else {
    playStream();
  }
}

You shouls handle also STREAM_STATUS.FAILED, STREAM_STATUS.STOPPED and SESSION_STATUS.FAILED, SESSION_STATUS.DISCONNECTED events to catch any issue which may occur. See Player example description for events handling details.

 

[Tigran]

window.addEventListener(visibilityChange, function() {
        if (!document[visibilityHidden]) {
          if(STREAM_WEBRTC){
            STREAM_WEBRTC.stop();
            START_SESSION()
          }
        }
      })

I already have this part in my code, but its fires when there is browser minimize/maximize.

I think , maybe during stream watching, when I`m having incoming phone call without minimizing safari browser, phone-call calls getUserMedia() for audio and my stream brakes, because there is happening double getUserMedia() call ?
I can`t handle this error no where.

And I tested on`

1. 11 Pro Max - 14.6(v)
2. 12 Pro - 14.6(v)
3. XS Max - 14.6(v)
4. X

First 3 devices passed my test.
But iPhone X doesn`t.

 

[Max]

Please make a test with AppRTC:
- join a conference
- play a stream
- make an incoming call to the device
If the behaviour is the same, this means we also cannot handle this event. In this case, user action is required to restart stream playback.

First 3 devices passed my test.
But iPhone X doesn`t.

If the problem is reproduced on one model only, this can be the model issue.

 

[Tigran]

I use React.js and here is my code. Please help if smth is wrong in my code.

import Flashphoner from "@flashphoner/websdk/flashphoner.js";

FlashPhonePlayer() called in componentDidMount(document.load).

 FlashPhonePlayer = () => {
    let self = this;

    function init(){
      try {
        Flashphoner.init({
          flashMediaProviderSwfLocation: 'https://stream.beticus.com:8444/shared/media-provider.swf',
          receiverLocation: 'https://stream.beticus.com:8444/shared/examples/demo/dependencies/websocket-player/WSReceiver2.js',
          decoderLocation: 'https://stream.beticus.com:8444/shared/examples/demo/dependencies/websocket-player/video-worker2.js',
          preferredMediaProvider: ["webrtc"]
        });

        self.setState({ showPlayButton: true })

        // This part is for desktop, autoPlay
        if(!isSafari && !isMobileOnly){
          self.START_SESSION()
        }
      } catch (e) {
        console.log(e)
        return;
      }
    }

    init()
  }

START_SESSION = () => {
    let self = this;
    let remoteVideo = document.getElementById("remoteVideo");

   // There are some cases, when video is loaded twice and here before start i check and remove video tags if they are in html.
    let VID = document.querySelectorAll("video");
    if(VID.length > 0) {
      for (let i = 0; i < VID.length; i++) {
        VID[i].remove()
      }
    }

    let PRELOADER_URL = "https://stream.beticus.com:8444/shared/examples/demo/dependencies/media/preloader.mp4";

    self.setState({
      showPlayButton: false
    }, () => {
      Flashphoner.playFirstVideo(remoteVideo, false, PRELOADER_URL).then(function() {
        self.CREATE_SESSION();
      }).catch(function() {
        self.setState({
          showPlayButton: true
        }, () => {
          let WEBRTC_PLAYER = self.STREAM_WEBRTC;
          if(WEBRTC_PLAYER) {
            self.STREAM_WEBRTC.stop()
            self.CREATE_SESSION();
          }
        })
      });
    })
  }

CREATE_SESSION = () => {
    let self = this;
    let { hlsStreamURL } = this.state;
    let SESSION_STATUS = Flashphoner.constants.SESSION_STATUS;

    if (Flashphoner.getSessions().length > 0) {
      let session = Flashphoner.getSessions()[0];
      self.PLAY_STREAM(session);

      // Hide play button
      self.SHOW_HIDE_PLAY_BUTTON(false)
      return;
    }

    Flashphoner.createSession({urlServer: hlsStreamURL}).on(SESSION_STATUS.ESTABLISHED, function(session){
      self.PLAY_STREAM(session);
      // Hide play button
      self.SHOW_HIDE_PLAY_BUTTON(false)
    }).on(SESSION_STATUS.DISCONNECTED, function(){
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(SESSION_STATUS.UNREGISTERED, function(){
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(SESSION_STATUS.INCOMING_CALL, function(){
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(SESSION_STATUS.FAILED, function(){
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    });
  }

PLAY_STREAM = (session) => {
    let { hlsStreamId } = this.state;
    let self = this;
    let stream;
    let streamName = hlsStreamId;
    let remoteVideo = document.getElementById("remoteVideo");
    let STREAM_STATUS = Flashphoner.constants.STREAM_STATUS;

    let options = {
      name: streamName,
      display: remoteVideo,
      flashShowFullScreenButton: true
    };

    stream = session.createStream(options).on(STREAM_STATUS.PENDING, function(stream) {
      console.log("PENDING")
    }).on(STREAM_STATUS.PLAYING, function (stream) {
      console.log("PLAYING")
    }).on(STREAM_STATUS.STOPPED, function () {
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(STREAM_STATUS.PAUSED, function () {
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(STREAM_STATUS.UNPUBLISHED, function () {
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(STREAM_STATUS.PLAYBACK_PROBLEM, function () {
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(STREAM_STATUS.FAILED, function () {
      // Show play button
      self.SHOW_HIDE_PLAY_BUTTON(true)
    }).on(STREAM_STATUS.NOT_ENOUGH_BANDWIDTH, function (stream) {
      console.log("Not enough bandwidth, consider using lower video resolution or bitrate. Bandwidth " + (Math.round(stream.getNetworkBandwidth() / 1000)) + " bitrate " + (Math.round(stream.getRemoteBitrate() / 1000)));
    });
    stream.play();

    this.STREAM_WEBRTC = stream
    window.STREAM_WEBRTC = stream
  }

 

[Max]

I use React.js and here is my code. Please help if smth is wrong in my code.

Your code seems correct, but you're trying to restart playback only if exception occured while playFirstVideo promise is resolving. When the promise is successfully resolved, you'll never reach this peace of code:

          let WEBRTC_PLAYER = self.STREAM_WEBRTC;
          if(WEBRTC_PLAYER) {
            self.STREAM_WEBRTC.stop()
            self.CREATE_SESSION();
          }

Also, websocket connection break or stream playback failure raise a corresponding event, but not an exception. If you prefer try-catch style, you should throw exception from your custom code upon receiving STREAM_STATUS.FAILED for example.
Please also look at this example to resume stream playback automatically. Note that we recommend to reconnect websocket session in this case to be sure to workaround a network problem.

 

[Tigran]

Hi Max. Hope you`re good.
Witch property and where should I use or how can I set buffering for incoming stream, for example to have 2s difference between local stream and real ?

 

[Max]

Witch property and where should I use or how can I set buffering for incoming stream, for example to have 2s difference between local stream and real ?

WebRTC is supposed to minimize delay. So you have two options:
1. Use playoutDelay stream option when playing:

session.createStream({
    name: streamName,
    display: remoteVideo,
    playoutDelay: 10
}).on(STREAM_STATUS.PENDING, function (stream) {
    ...
}).play();

This works for video only, but audio will play without delay. Plase read details here.
2. Use WebRTC as RTMP republishing to local server with RTMP buffering enabled. Please read this thread. In this case, both audio and video tracks will be delayed.

 

[Tigran]

Thanks Max for answer ,but i want this`
I want buffering for smooth video, because when my network is slower for 0.1-2.0 seconds, stream gets cutten.

playoutDelay: 10

- works only for web, not for IOS.

 

[Max]

I want buffering for smooth video, because when my network is slower for 0.1-2.0 seconds, stream gets cutten.

If you have a network problems (packet loss), you can use TCP transport. Please read details here.

 

[Tigran]

Hello Max. How are you?
I`ve a question.
Can I change connection url during playing stream without getting black screen ?

 Flashphoner.createSession({
     urlServer: streamUrl
 })

 

[Max]

Good day.

Can I change connection url during playing stream without getting black screen ?

You need to re-establish Websocket connection to a new URL, so the previous session must be closed.

 

[Tigran]

Hello Max. Hope you`re well.

Stream starts, and during playing it gone black (safari ios-15).
How can I fix this ?

 

[Max]

Stream starts, and during playing it gone black (safari ios-15).

Please clarify how do you test:
1. How do you publish a stream?
2. How do you play the stream?
3. What exactly video becomes black: local video while publishing or remote video while playing?
4. What example do you use to test? Is the problem reproducing in Two Way Streaming example? If not, please modify Two Way Streaming example code minimally to reproduce the issue, and send this code using this form.