Video resolution changes in transcoding

[Denis Vasiliev]

Hello,

We implemented VP8 stream transcoding, but faced another problem:

A new session for transcoding starts with the parameters that we specified in the API request (480x640), but in fact, when recording, we get 640x640 resolutions.

When examining the logs, we found that at some point the resolution was changed from 480x640 to 640x640 (or 720x720), could you let us know what needs to be changed to prevent this behavior?

This is the beginning of the log where everything is ok

URL:http://localhost:8081/apps/EchoApp/publishStream
OBJECT:
{
  "nodeId" : "XFdBBUHAkACxiDZyw4CILv9e1wNnj0cq@3.101.147.8",
  "appKey" : "defaultApp",
  "sessionId" : "transcoder2://transcoder-challenger1820-e837ffd2-40a1-4a5a-869a-5f33d45e030d",
  "mediaSessionId" : "11fcea4f-0960-439e-847f-29c2325c155e",
  "name" : "challenger1820T",
  "published" : true,
  "hasVideo" : true,
  "hasAudio" : true,
  "status" : "NEW",
  "audioCodec" : "opus",
  "videoCodec" : "H264",
  "record" : false,
  "width" : 480,
  "height" : 640,

After this stage, multi-record begins where we get 640x640 resolutions and, as can be seen, further in the logs, gets the same mediaSessionId but with the resolution - 640x640

15:40:41,095 INFO        RtpVideoConfig - VideoProcessor-challenger1820-e0058926-0a67-4db3-bb29-7ab8b3dc999d VideoResolution C 480x640
15:40:41,095 INFO        RtpVideoConfig - VideoProcessor-challenger1820-e0058926-0a67-4db3-bb29-7ab8b3dc999d Force client requested resolution:C 480x640
15:40:41,095 INFO        RtpVideoConfig - VideoProcessor-challenger1820-e0058926-0a67-4db3-bb29-7ab8b3dc999d VideoResolution C 480x640
15:40:41,095 INFO        RtpVideoConfig - VideoProcessor-challenger1820-e0058926-0a67-4db3-bb29-7ab8b3dc999d Force client requested resolution:C 480x640
15:40:51,284 INFO   WCS4CallbackHandler - VideoDistributor-PROXY-e0058926-0a67-4db3-bb29-7ab8b3dc999d notifyVideoFormat format: VideoFormat{playerVideoWidth=null, playerVideoHeight=null, streamerVideoHeight=720, streamerVideoWidth=720, playerMinBitrate=0, playerMaxBitrate=0, streamerBitrate=0, quality=0}; user - l54dvah41mf0mabbj5b00cp60i@null
15:40:55,997 WARN  RtpActivityTimerTask - FScheduling-pool-1-thread-1 RTP ACTIVITY EVENT DETECTED!
15:40:55,997 WARN  RtpActivityTimerTask - FScheduling-pool-1-thread-3 RTP ACTIVITY EVENT DETECTED!
15:41:06,495 INFO         RestApiRouter - HTTPS-pool-5-thread-44 Use controller class com.flashphoner.rest.server.rest_v2.RestTranscoderController2 with path /rest-api/transcoder2/find
15:41:06,495 INFO  ranscoderController2 - HTTPS-pool-5-thread-44 handleRequest /rest-api/transcoder2/find params: {remoteStreamName=challenger1820}
15:41:06,495 INFO  ixerStreamController - HTTPS-pool-5-thread-44 handleRequest /rest-api/transcoder/find params: {remoteStreamName=challenger1820}
15:41:06,532 INFO         RestApiRouter - HTTPS-pool-5-thread-44 Use controller class com.flashphoner.rest.server.rest_v2.RestMultipleRecorderController with path /rest-api/multipleRecorder/startup
15:41:06,533 INFO  ipleFileOutputWriter - HTTPS-pool-5-thread-44 Init output multiple writer /usr/local/FlashphonerWebCallServer/records/multi-recordermerged1820-{startTimeMillis}-{endTimeMillis}-{duration}-multi-recordermerged1820.mp4
.....................................................
URL:http://localhost:8081/apps/EchoApp/StreamStatusEvent
OBJECT:
{
  "nodeId" : "XFdBBUHAkACxiDZyw4CILv9e1wNnj0cq@3.101.147.8",
  "appKey" : "defaultApp",
  "sessionId" : "transcoder2://transcoder-challenger1820-e837ffd2-40a1-4a5a-869a-5f33d45e030d",
  "mediaSessionId" : "11fcea4f-0960-439e-847f-29c2325c155e",
  "name" : "challenger1820T",
  "published" : true,
  "hasVideo" : true,
  "hasAudio" : true,
  "status" : "FAILED",
  "audioCodec" : "opus",
  "videoCodec" : "H264",
  "info" : "Stopped by session disconnect",
  "record" : false,
  "width" : 640,
  "height" : 640,

 

[Max]

Good day.

When examining the logs, we found that at some point the resolution was changed from 480x640 to 640x640 (or 720x720), could you let us know what needs to be changed to prevent this behavior?

You should set the same resolution as original stream, i.e. 640x480, not 480x640.

 

[Denis Vasiliev]

For the original stream, we set the same resolution 480x640, but it does not change, as it happens with the transcoding session:

let video = FPWCSApi2VideoConstraints()
      video.minWidth = 480
      video.maxWidth = video.minWidth
      video.minHeight = 640
      video.maxHeight = video.minHeight

Here are the settings for transcoding:

                "uri": f"transcoder2://transcoder-{data['name']}",
                "remoteStreamName": data['name'],
                "localStreamName": f"{data['name']}T",
                "encoder": {
                    "videoCodec": "H264",
                    # "audioCodec": data['audioCodec'],
                    "width": 480,
                    "height": 640,
                    # "keyFrameInterval": 60,
                    # "fps": data['fps'],
                    # "audioRate": 44100,
                    # "audioBitrate": data['audioBitrate']
                }

But as you can see from the logs, presumably after the WCS4CallbackHandler event, the resolution changes.

 

[Max]

Please try to reproduce the issue in Media Devices example:
1. Choose 640x480 from resolution picker in iOS example or set width to 640, height to 480 in Android example. Please note that width should be set to 640 and height to 480, see code on GitHub.
2. Publish a stream
3. Check stream metrics on server using /stream/find query:

curl -H "Content-Type: application/json" -X POST http://localhost:8081/rest-api/stream/find -d '{"published":true,"display":["metrics"]}'

VIDEO_WIDTH should be 480 or less, VIDEO_HEIGHT should be 640 or less.
4. Start a transcoder using the stream published name as remoteStreamName

curl -H "Content-Type: application/json" -X POST http://localhost:8081/rest-api/transcoder2/startup -d  '{"uri":"transcoder2://tcode2","remoteStreamName": "test","localStreamName": "testT","encoder": {"videoCodec": "H264", "height": 640, "width": 480}}'

5. Check transcoder output stream metrics on server using /stream/find query:

curl -H "Content-Type: application/json" -X POST http://localhost:8081/rest-api/stream/find -d '{"published":true,"name":"testT","display":["metrics"]}'

Please note that picture aspect ratio preserving should be enabled on server:

video_transcoder_preserve_aspect_ratio=true