Fixing Video Frame Size in WebRTC

J

jasonkc

Member
I'm using the demo code from webrtc streaming example to build our streaming page, and I realized if I open the streaming page on mobile device, the capture window will appear in portrait mode. How do I fix the capture window in landscape mode and also the ratio aspect (eg: 16:9 with min width of 1200)?
 
Max

Max

Administrator
Staff member
Good day.
jasonkc said:
How do I fix the capture window in landscape mode
Click to expand...
It is impossible, the mobile device camera always returns frames in portrait orientation
You can resize the displayed video, but it still will be published as 9:16 or 3:4 depending on publishing resolution. Please use resizeLocalVideo() functions as shown in Media Devices example code.
Code: 
    ...
    }).on(STREAM_STATUS.PUBLISHING, function (stream) {
        var video = document.getElementById(stream.id());
        //resize local if resolution is available
        if (video.videoWidth > 0 && video.videoHeight > 0) {
            resizeLocalVideo({target: video});
        }
        //remove resize listener in case this video was cached earlier
        video.removeEventListener('resize', resizeLocalVideo);
        video.addEventListener('resize', resizeLocalVideo);
        ...
    });
 
J

jasonkc

Member
reason I'm asking is that i want the video to remain in portrait mode be it whether the user stream in portrait or landscape mode on their device (eg. mobile phone, desktop etc)
 
Max

Max

Administrator
Staff member
jasonkc said:
btw, how do i set the video size?
Click to expand...
You should use constraints
Code: 
    session.createStream({
        name: streamName,
        display: localVideo,
        ...
        constraints: {
               audio: true,
               video: {
                     width: 640,
                     height: 480
               }
        }
    }).publish();
jasonkc said:
reason I'm asking is that i want the video to remain in portrait mode be it whether the user stream in portrait or landscape mode on their device (eg. mobile phone, desktop etc)
Click to expand...
To stream in portrait mode from desktop, you should revert width and height:
Code: 
        constraints: {
               audio: true,
               video: {
                     width: 480,
                     height: 640
               }
        }
 
J

jasonkc

Member
still no luck in getting it work...what we want to achieve is like attached:

when open our app in mobile device in portrait mode, not only that the screen (circled in RED) would appear in landscape mode, the captured/streamed video would be in the same aspect too with the portrait video appeared in center of the video frame.
 

Attachments

Max

Max

Administrator
Staff member
You can achive this using constraints. For example, this is landascape mode in Android Chrome browser:
Code: 
    session.createStream({
        name: streamName,
        display: localVideo,
        ...
        constraints: {
            audio: true,
            video: {
                width: 360,
                height: 640
            }
        }
        ...
    }).publish();
1623138734839.png

And this is portrait mode
Code: 
    session.createStream({
        name: streamName,
        display: localVideo,
        ...
        constraints: {
            audio: true,
            video: {
                width: 640,
                height: 360
            }
        }
        ...
    }).publish();
1623138801674.png
 
J

jasonkc

Member
i hv added the following:

session.createStream({
name: streamName,
display: localVideo,
cacheLocalResources: true,
receiveVideo: false,
receiveAudio: false,
transport: "UDP",
constraints: {
audio: true,
video: {
width: 1280,
height: 720
}
},
..........
}).publish();
runShellScript();




but when the stream was forwarded to FB, I still noticed the video frame was in portrait.
 
J

jasonkc

Member
Also, is it possible to achieve the attached - input in portrait frame, and output in landscape with the portrait video appeared in the middle?
 
Max

Max

Administrator
Staff member
jasonkc said:
in above samples, did you set the dimension of video frame in CSS too (both local and player)?
Click to expand...
Yes, this is Two Way Streaming example (look at its code on GitHub)
jasonkc said:
Also, is it possible to achieve the attached - input in portrait frame, and output in landscape with the portrait video appeared in the middle?
Click to expand...
This can be done with stream transcoding only. For example: publish stream with constraints
Code: 
width: 1280,
height: 720
play with constraints
Code: 
width: 720,
height: 1280
and disable aspect ratio preserving in flashphoner.properties file:
Code: 
video_transcoder_preserve_aspect_ratio=false
But, in this case, more server resources required: at least 1 CPU core per 2 720p streams, at least 32 Gb RAM (16 Gb for Java heap).
 
Last edited:
J

jasonkc

Member
I tried this below setting in constraints:

function startStreaming(session) {
var streamName = urlParams.get('user_id');
var rtmpUrl = "rtmp://localhost:1935/live";
session.createStream({
name: streamName,
display: localVideo,
cacheLocalResources: true,
receiveVideo: false,
receiveAudio: false,
transport: "UDP",
constraints: {
audio: true,
video: {
width: 720,
height: 1280
}
},
.........

and I added an picture of about the size of 200x200 pixels con top of the video, then i realized the video is no way to be 720x1280 since the picture has taken up almost 1/3 of the width (attached).
 

Attachments

J

jasonkc

Member
FYI, the RTMP stream was re-processed with below ffmpeg command, I dont see any resize of the video frame:

apache 21325 21317 72 08:46 ? 00:00:56 /snap/ffmpeg/1286/usr/bin/ffmpeg -threads 1 -i rtmp://127.0.0.1:1935/live/dc78 -stream_loop -1 -i /home/apache/dc78/logo.png -f image2 -stream_loop -1 -i /home/apache/dc78/prod.jpg -filter_complex [0][1]overlay=x=30:y=10[v1];[v1][2]overlay=x=main_w-overlay_w-5:y=10[v2] -map [v2] -map 0:a -framerate 25 -ar 44100 -c:a copy -b:a 20000k -movflags +faststart -preset medium -crf 1 -tune zerolatency -maxrate 20000k -vcodec libx264 -fflags nobuffer -g 60 -max_muxing_queue_size 1024 -f flv pipe:1
apache 21326 21317 31 08:46 ? 00:00:24 /snap/ffmpeg/1286/usr/bin/ffmpeg -threads 1 -i pipe:0 -vf drawtext=textfile=/home/apache/dc78/rolling.txt: reload=1: fontcolor=white: fontsize=20: box=1: boxcolor=black@0.5: boxborderw=5: y=((h)-20):x=w-(mod(4*n\,w+tw)-tw/40) -nostats -loglevel 0 -framerate 25 -ar 44100 -c:a libfdk_aac -b:a 20000k -movflags +faststart -preset medium -crf 1 -tune zerolatency -maxrate 20000k -vcodec libx264 -fflags nobuffer -g 60 -max_muxing_queue_size 1024 -bsf:a aac_adtstoasc -map 0 -flvflags no_duration_filesize -flags +global_header -f tee [f=flv:eek:nfail=ignore]rtmp://127.0.0.1:1935/live/rpdc78|[f=flv:eek:nfail=ignore]rtmps://live-api-s.facebook.com:443/rtmp/330104032121395?s_bl=1&s_psm=1&s_sw=0&s_vt=api-s&a=AbzKWClVN_Sz0Gbi
 
Max

Max

Administrator
Staff member
WebRTC may publish lower resolution than set in constraints depending on channel quality. So video contraints strictly define aspect ratio only.
You can check actual publishing resolution in stream metrics:
Code: 
POST /rest-api/stream/find HTTP/1.1
Host: localhost:8081
Content-Type: application/json
 
{
    "name":"stream1",
    "published":true,
    "display":["metrics"]
}
This query returns stream parameters, including metrics. Check VIDEO_WIDTH and VIDEO_HEIGHT metrics for actual width and height respectively.
 
J

jasonkc

Member
Max said:
WebRTC may publish lower resolution than set in constraints depending on channel quality. So video contraints strictly define aspect ratio only.
You can check actual publishing resolution in stream metrics:
Code: 
POST /rest-api/stream/find HTTP/1.1
Host: localhost:8081
Content-Type: application/json

{
    "name":"stream1",
    "published":true,
    "display":["metrics"]
}
This query returns stream parameters, including metrics. Check VIDEO_WIDTH and VIDEO_HEIGHT metrics for actual width and height respectively.
Click to expand...

Is this the right command via curl:

curl -X POST "https://127.0.0.1:8081/rest-api/stream/find" -H "Content-Type:application/json" -d "name=100000000000003&published=true&display=metrics" --insecure

I'm getting "curl: (35) Encountered end of file" error.
 
Last edited:
You must log in or register to reply here.
Top