iphone video freeze

Dani · Aug 31, 2020

Hi,
we're using an almost year old version of the sever/ api.
the reason is that whenever we tried to upgrade - iPhone video is frozen.
No issues on MAC and no issue on android.
I can also confirm that the OTHER side see the "frozen" video as live - the problem looks local to iphone (trying on Iphone XR)

We want to be able to keep up with you - so it's about time to figure out what we do wrong and fix it
Any ideas where to start ? how can we debug this issue ?
I've installed the latest versions on both client and server of server and api.
since we're using a development environment I can do whatever I won't without interfering with production.

Max · Sep 1, 2020

Good day.

Dani said:
I can also confirm that the OTHER side see the "frozen" video as live - the problem looks local to iphone (trying on Iphone XR)

Please clarify: a local video is frozen while publishing from iOS Safary?
Is the issue reproduced in Two Way Streaming example on our demo server? If not, please modify Two Way Streaming application code minimally to reproduce the issue, and send us using this link.
Also please provide SSH access and web interface access to your development server, we try to reproduce the issue in your environment. Please use this link to send credentials.

Dani · Sep 2, 2020

I think the answer is in this log....
how do I fix this ?

Log] connectFphoner() ESTABLISHED (.aspx, line 2330)
[Warning] RTCIceServer.url is deprecated, please use RTCIceServer.urls instead. (flashphoner.min.js, line 12)
[Warning] 09:06:31 WARN playout delay unsupported - – undefined (flashphoner.min.js, line 25)
[Log] 09:06:32 INFO webrtc - – "Autoplay detected! Trying to play a video with a muted sound..." (flashphoner.min.js, line 25)
[Log] showVidW: 480 - showVidH: 270 (.aspx, line 2476)
[Error] Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.
(anonymous function)
rejectPromise
play
(anonymous function) (flashphoner.min.js:25:178225)
promiseReactionJob
[Log] 09:06:34 INFO webrtc - – {audio: true, video: {width: {min: 320, max: 640}, height: {min: 240, max: 480}, frameRate: {ideal: 30}}, customStream: undefined} (flashphoner.min.js, line 25)
[Log] 09:06:37 INFO webrtc - – "FOUND WEBRTC CACHED INSTANCE, id 75b7b840-ece2-11ea-9ca1-e91a8149fe87-LOCAL_CACHED_VIDEO" (flashphoner.min.js, line 25)
[Log] startStreaming kind: cam (.aspx, line 2747)
[Log] PUBLISHING startStreaming (.aspx, line 2758)
[Error] Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.
(anonymous function)
rejectPromise
play
(anonymous function) (flashphoner.min.js:25:169381)

Max · Sep 2, 2020

Good day.
Please check if you're using playFirstVideo() function as described here. Please see the source code of Two Way Streaming example, for publishing and for playback:

Code:

function publishBtnClick() {
    if (validateForm("streamerForm")) {
       ...
       if (Browser.isSafariWebRTC()) {
            Flashphoner.playFirstVideo(localVideo, true, PRELOADER_URL).then(function() {
                publishStream();
            });
            return;
        }
        publishStream();
    }
}
...
function playBtnClick() {
    if (validateForm("playerForm")) {
        ...
        if (Flashphoner.getMediaProviders()[0] === "WSPlayer") {
            Flashphoner.playFirstSound();
        } else if (Browser.isSafariWebRTC() || Flashphoner.getMediaProviders()[0] === "MSE") {
            Flashphoner.playFirstVideo(remoteVideo, false, PRELOADER_URL).then(function () {
                playStream();
            });
            return;
        }
        playStream();
    }
}

Dani · Jan 3, 2021

What is PRELOADER_URL ? I tried using it as is - and it says it can't find the variable.

Max · Jan 4, 2021

PRELOADER_URL is the URI to get preloader video (black screen with silence). The preloader used in WebSDK is in /usr/local/FlashphonerWebCallServer/client2/examples/demo/dependencies/media/preloader.mp4 file.
All Web SDK examples refer to this preloader by relative path

Code:

var PRELOADER_URL = "../../dependencies/media/preloader.mp4";

Dani · Jan 6, 2021

the call for playFirstVideo pass.
Then I create the stream session.createStream(optionsPlayVid).on(STREAM_STATUS.PLAYING, function (prvStream)

and get this on the console:
Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

The local stream is showing on the other side, but the remote stream doesn't show on the iPhone.

Max · Jan 6, 2021

and get this on the console ...

To reproduce the issue, please send us the application code using this link.

Dani · Jan 6, 2021

the Rejection notice is on play function I believe.

this is my code:

from the log I can see this text: console.log("remote pass playfirst video, calling start");

I can see the AddEventListener and I can even see once the
showVidW: 480 - showVidH: 270 log message
but then I get the NotAllowedError - b/c of user denied permission.
the very same code - on a year old version of flashphoner works on the new iphones.

remoteVideo.innerHTML = "";

var constPlayVid = {
audio: false,
video: true
};

console.log("Browser.IsWebRTC: " + Browser.isSafariWebRTC().toString());
if (Browser.isSafariWebRTC() || mediaProvidFP === "MSE")
{
console.log("trying to run playFirstVideo Remote: ");
Flashphoner.playFirstVideo(remoteVideo, false, PRELOADER_URL).then(function () {
console.log("remote pass playfirst video, calling start");

});

if (Browser.isSafariWebRTC() && isMobile == "1")
{
remoteVideo.style.width = showVidW + "px";
remoteVideo.style.height = TDshowVidID.style.height = "270px";

if (vidSideCut > 0) {

if (langID == "1") { remoteVideo.style.marginRight = "-" + vidSideCut + "px"; }
else { remoteVideo.style.marginLeft = "-" + vidSideCut + "px"; }
}
else {
remoteVideo.className = "display center";
}
}
}

var optionsPlayVid;
if (isWebRTCbrowser == "1")
{
optionsPlayVid = {
name: streamInName,
display: remoteVideo,
constraints: constPlayVid,
mediaProvider: mediaProvid,
cacheLocalResources: false,
flashShowFullScreenButton: true
};

if (didConAgain == false && mediaProvid == "WebRTC") {
timerConAgainWS = setTimeout(function () { doConAgainWS(); }, 4000);
}
}
else
{
optionsPlayVid = {
name: streamInName,
display: remoteVideo,
constraints: constPlayVid,
cacheLocalResources: false,
flashShowFullScreenButton: true
};
}
var session = Flashphoner.getSessions()[0];
prvStream = session.createStream(optionsPlayVid).on(STREAM_STATUS.PLAYING, function (prvStream) {
streamInVid = prvStream;
clearTimeout(timerConAgainWS);
didConAgain = false;
DIVshowVidLoadID.style.display = "none";
TDshowVidID.style.display = "table-cell";

videoIN = document.getElementById(streamInVid.id());

var vidCanH = showVidH;
var vidCanW = showVidW;

console.log("Add Event Listener");
videoIN.addEventListener("loadedmetadata", function (e) {
vidWidth = this.videoWidth, vidHeight = this.videoHeight;

if (isMobile == "1") {
vidCanH = Math.round(vidCanW / (vidWidth / vidHeight));
}
else {
vidCanW = Math.round(vidCanH * (vidWidth / vidHeight));
}

showVidW = vidCanW;
showVidH = vidCanH;
proportional = showVidW / showVidH; // w/h
console.log("863 - showVidW: " + showVidW + " - showVidH: " + showVidH);

if (vidSideCut > 0)
{

if (langID == "1") { remoteVideo.style.marginRight = "-" + vidSideCut + "px"; }
else { remoteVideo.style.marginLeft = "-" + vidSideCut + "px"; }
}
else
{
remoteVideo.className = "display center";
}

videoIN.style.width = remoteVideo.style.width = vidCanW.toString() + "px";
videoIN.style.height = remoteVideo.style.height = TDshowVidID.style.height = vidCanH.toString() + "px";

}, false);
console.log("After ADd EVent Listener");

console.log("BrowserName: " + browserName + " isMobile: " + isMobile + " safariWebRTC: " + Browser.isSafariWebRTC().toString());
if (isSound == "1" && (mediaProvid == "WebRTC" || mediaProvid == "Flash") && (browserName != "safari" || isMobile == "0") && !Browser.isSafariWebRTC())
{
console.log("need to go to playSteamS.");
playStreamS();
}
console.log("doUserConInfo");
doUserConInfo(mediaProvid, "0");
console.log("After doUserConInfo");

}).on(STREAM_STATUS.STOPPED, function () {
console.log("Play show Stream STOPPED");
if (didConAgain == false || mediaProvid != "WebRTC") {
doDisconnect(langArrOn[37], 2000, "3");
}
}).on(STREAM_STATUS.FAILED, function () {
console.log("Play show Stream FAILED");
if (didConAgain == false || mediaProvid != "WebRTC") {
doDisconnect(langArrOn[37], 2000, "4");
}
}).play();

Max · Jan 7, 2021

You should call Session.createStream() and Stream.play() only when promise returned by playFirstVideo() is resolved, and in context of the promise. Please see Two Way Streaming example, playing section

Code:

function playBtnClick() {
    if (validateForm("playerForm")) {
        ...
        if (Flashphoner.getMediaProviders()[0] === "WSPlayer") {
            Flashphoner.playFirstSound();
        } else if (Browser.isSafariWebRTC() || Flashphoner.getMediaProviders()[0] === "MSE") {
            Flashphoner.playFirstVideo(remoteVideo, false, PRELOADER_URL).then(function () {
                /// The stream is created and played here, when promise is resolved successfully
                playStream();
            });
            return;
        }
        playStream();
    }
}

function playStream() {
    var session = Flashphoner.getSessions()[0];
    var streamName = $('#playStream').val();

    session.createStream({
        name: streamName,
        display: remoteVideo
    }).on(STREAM_STATUS.PENDING, function (stream) {
        ...
    }).play();
}

In your code you just log something to browser console

Code:

    Flashphoner.playFirstVideo(remoteVideo, false, PRELOADER_URL).then(function () {
        console.log("remote pass playfirst video, calling start");
        
    });

So, you call flow looks like this:
1. playFirstVideo() is called (thread started)
2. Stream object is created (another thread and context)
3. Trying to play stream (and get exception)
4. Promise is resolved (in its own thread)
and should be like this:
1. playFirstVideo() is called (thread started)
2. Promise is resolved
2.1. Stream object is created (in the same thred)
2.2. Trying to play stream (in the same thread)

Dani · Jan 7, 2021

I will try to push the code inside. but this code works on android. doesn't work only on iphone. I will update

Max · Jan 8, 2021

Dani said:
I will try to push the code inside. but this code works on android. doesn't work only on iphone. I will update

The playFirstVideo() cratch is intended only for Safari browser (both for MacOS and iOS). This function is not necessary on Windows, Linux and Android browsers (but it does not break something if called).

Dani · Jan 10, 2021

I've pushed the code inside the promise of the play first viedo - but I still get the same error message:

console.log("Browser.IsWebRTC: " + Browser.isSafariWebRTC().toString());
if (Browser.isSafariWebRTC() || mediaProvidFP === "MSE") {
console.log("trying to run playFirstVideo Remote: ");
Flashphoner.playFirstVideo(remoteVideo, false, PRELOADER_URL).then(function () {
console.log("remote pass playfirst video, calling start");

if (Browser.isSafariWebRTC() && isMobile == "1") {
remoteVideo.style.width = showVidW + "px";
remoteVideo.style.height = TDshowVidID.style.height = "270px";

if (vidSideCut > 0) {
//לסדר פה שזה יקח ממשתנה קבוע ב Dict - marginRight or marginLeft
if (langID == "1") { remoteVideo.style.marginRight = "-" + vidSideCut + "px"; }
else { remoteVideo.style.marginLeft = "-" + vidSideCut + "px"; }
}
else {
remoteVideo.className = "display center";
}
}

/// moving the code into the promise:

if (Browser.isSafariWebRTC() && isMobile == "1") {
remoteVideo.style.width = showVidW + "px";
remoteVideo.style.height = TDshowVidID.style.height = "270px";

if (vidSideCut > 0) {
//לסדר פה שזה יקח ממשתנה קבוע ב Dict - marginRight or marginLeft
if (langID == "1") { remoteVideo.style.marginRight = "-" + vidSideCut + "px"; }
else { remoteVideo.style.marginLeft = "-" + vidSideCut + "px"; }
}
else {
remoteVideo.className = "display center";
}
}

var optionsPlayVid;
if (isWebRTCbrowser == "1") {
optionsPlayVid = {
name: streamInName,
display: remoteVideo,
constraints: constPlayVid,
mediaProvider: mediaProvid,
cacheLocalResources: false,
flashShowFullScreenButton: true
};

if (didConAgain == false && mediaProvid == "WebRTC") {
timerConAgainWS = setTimeout(function () { doConAgainWS(); }, 4000);
}
}
else {
optionsPlayVid = {
name: streamInName,
display: remoteVideo,
constraints: constPlayVid,
cacheLocalResources: false,
flashShowFullScreenButton: true
};
}

var session = Flashphoner.getSessions()[0];
console.log("calling create stream from withn the promise: ");
prvStream = session.createStream(optionsPlayVid).on(STREAM_STATUS.PLAYING, function (prvStream) {
streamInVid = prvStream;
clearTimeout(timerConAgainWS);
didConAgain = false;
DIVshowVidLoadID.style.display = "none";
TDshowVidID.style.display = "table-cell";

//remoteVideo.style.height = "" + showVidH.toString() + "px";

videoIN = document.getElementById(streamInVid.id());

//מארגן את נתוני הגובה והרוחב של הוידאו
var vidCanH = showVidH;
var vidCanW = showVidW;
//stackoverflow.com/questions/4129102/html5-video-dimensions

console.log("Add Event Listener");
videoIN.addEventListener("loadedmetadata", function (e) {
vidWidth = this.videoWidth, vidHeight = this.videoHeight;

if (isMobile == "1") {
vidCanH = Math.round(vidCanW / (vidWidth / vidHeight));
}
else {
vidCanW = Math.round(vidCanH * (vidWidth / vidHeight));
}

showVidW = vidCanW;
showVidH = vidCanH;
proportional = showVidW / showVidH; // w/h
console.log("863 - showVidW: " + showVidW + " - showVidH: " + showVidH);
videoIN.style.width = remoteVideo.style.width = vidCanW.toString() + "px";
videoIN.style.height = remoteVideo.style.height = TDshowVidID.style.height = vidCanH.toString() + "px";

}, false);
console.log("After ADd EVent Listener");

console.log("BrowserName: " + browserName + " isMobile: " + isMobile + " safariWebRTC: " + Browser.isSafariWebRTC().toString());
if (isSound == "1" && (mediaProvid == "WebRTC" || mediaProvid == "Flash") && (browserName != "safari" || isMobile == "0") && !Browser.isSafariWebRTC()) {
console.log("need to go to playSteamS.");
playStreamS();
}
console.log("doUserConInfo");
doUserConInfo(mediaProvid, "0");
console.log("After doUserConInfo");
//console.log(streamInVid.status());
//var streamResolution = streamInVid.videoResolution(); לברר על זה
//console.log("streamInVid size width: " + streamResolution.width + " height: " + streamResolution.height);
}).on(STREAM_STATUS.STOPPED, function () {
console.log("Play show Stream STOPPED");
if (didConAgain == false || mediaProvid != "WebRTC") {
doDisconnect(langArrOn[37], 2000, "3");
}
}).on(STREAM_STATUS.FAILED, function () {
console.log("Play show Stream FAILED");
if (didConAgain == false || mediaProvid != "WebRTC") {
doDisconnect(langArrOn[37], 2000, "4");
}
}).play();

});

}

and the console log:
[Log] 16:51:22 INFO webrtc - – "Initialized" (flashphoner.min.js, line 25)
[Log] 16:51:22 INFO websocket - – "Initialized" (flashphoner.min.js, line 25)
[Log] 16:51:22 INFO core - – "Initialized" (flashphoner.min.js, line 25)
[Log] userAskChat() (aaswuChatMTest.aspx, line 3436)
[Log] userAskChat() counterUAskChat: 1 (aaswuChatMTest.aspx, line 3517)
[Log] userAskChat() (aaswuChatMTest.aspx, line 3436)
[Log] userAskChat() counterUAskChat: 2 (aaswuChatMTest.aspx, line 3517)
[Log] userAskChat() (aaswuChatMTest.aspx, line 3436)
[Log] aaswuChatScriptTest.ascx - connectFphoner (aaswuChatMTest.aspx, line 2291)
[Log] mediaProvider: WebRTC (aaswuChatMTest.aspx, line 2292)
[Log] creating session: (aaswuChatMTest.aspx, line 2293)
[Log] connectFphoner() ESTABLISHED (aaswuChatMTest.aspx, line 2325)
[Log] Runnign playStream (aaswuChatMTest.aspx, line 2350)
[Log] Browser.IsWebRTC: true (aaswuChatMTest.aspx, line 2382)
[Log] trying to run playFirstVideo Remote: (aaswuChatMTest.aspx, line 2384)
[Log] remote pass playfirst video, calling start (aaswuChatMTest.aspx, line 2386)
[Log] calling create stream from withn the promise: (aaswuChatMTest.aspx, line 2447)
[Warning] RTCIceServer.url is deprecated, please use RTCIceServer.urls instead. (flashphoner.min.js, line 12)
[Log] Add Event Listener (aaswuChatMTest.aspx, line 2464)
[Log] After ADd EVent Listener (aaswuChatMTest.aspx, line 2496)
[Log] BrowserName: safari isMobile: 1 safariWebRTC: true (aaswuChatMTest.aspx, line 2537)
[Log] doUserConInfo (aaswuChatMTest.aspx, line 2542)
[Log] After doUserConInfo (aaswuChatMTest.aspx, line 2544)
[Log] 16:51:40 INFO webrtc - – "Autoplay detected! Trying to play a video with a muted sound..." (flashphoner.min.js, line 25)
[Log] 863 - showVidW: 480 - showVidH: 270 (aaswuChatMTest.aspx, line 2478)
[Error] Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.
(anonymous function)
rejectPromise
play
(anonymous function) (flashphoner.min.js:25:178628)
promiseReactionJob

Max · Jan 11, 2021

Please do the following:
1. Check if stream playback on iPhone works in Two Way Streaming example shipped with WCS
2. Carefully modify the TwoWay Streaming code to reproduce the issue
3. Send us the code modified using this private form

Dani · Jan 11, 2021

thanks I will. you sample works on the iphone.

Dani · Jan 11, 2021

I am not sure why - but changing this code:

Code:

Flashphoner.init({
   flashMediaProviderSwfLocation: 'flash/flashphoner/media-provider.swf',
   receiverLocation: 'scripts/flashphoner/dependencies/websocket-player/WSReceiver2.js',
   decoderLocation: 'scripts/flashphoner/dependencies/websocket-player/video-worker2.js',
   preferredMediaProviders: mediaProviders.split(',')
});

With this:

Flashphoner.init({ flashMediaProviderSwfLocation: 'flash/flashphoner/media-provider.swf' });

Solve the problem.

any idea why ?

Max · Jan 12, 2021

Dani said:
any idea why ?

By default, localion parameters are relative to example source code:

Code:

        Flashphoner.init({
            flashMediaProviderSwfLocation: '../../../../media-provider.swf',
            receiverLocation: '../../dependencies/websocket-player/WSReceiver2.js',
            decoderLocation: '../../dependencies/websocket-player/video-worker2.js',
            preferredMediaProviders: mediaProviders && mediaProviders !== "" ? mediaProviders.split(','): []
        });

If you moved Embed Player code (and whole SDK) to another location on your webserver, you should set it to absolute paths.

Dani · Jan 14, 2021

Well. looks like I almost solve everything. The only issue left - Iphone doesn't play the sound of the remote stream.
it works on android and on pc, but not on iphone. any idea how canI fix it ?

Max · Jan 15, 2021

Dani said:
The only issue left - Iphone doesn't play the sound of the remote stream.
it works on android and on pc, but not on iphone. any idea how canI fix it ?

The autoplay policy in Safari requires a user action to start sound playback. Unfortunately, there is no workaround.

iphone video freeze

Dani

Member

Max

Administrator

Dani

Member

Attachments

Max

Administrator

Dani

Member

Max

Administrator

Dani

Member

Max

Administrator

Dani

Member

Max

Administrator

Dani

Member

Max

Administrator

Dani

Member

Max

Administrator

Dani

Member

Dani

Member

Max

Administrator

Dani

Member

Max

Administrator