Recherche avancée

Sur d’autres sites (11976)

• FFMPEG changes pixel values when reading and saving png without modification

  25 janvier 2023, par walrus

  This is a toy problem that is the result of my trying to identify a bug within a video pipeline I'm working on. The idea is that I want to take a frame from a YUV420 video, modify it as an RGB24 image, and reinsert it. To do this I convert YUV420 -> YUV444 -> RGB -> YUV444 -> YUV420. Doing this without any modification should result in the same frame however I noticed slight color transformations.

  


  I tried to isolate the problem using a toy 3x3 RGB32 png image. The function read_and_save_image reads the image and then saves it as new file. It returns the read pixel array. I run this function thrice successively using the output of the previous run as the input of the next. This is to demonstrate a perplexing fact. While passing an image through the function once causes the resulting image to have different pixel values, doing it twice does not change anything. Perhaps more confusing is that the pixel values returned by the function are all the same.

  


  tldr ; How can I load and save the toy image below using ffmpeg as a new file such that the pixel values of the new and original files are identical ?

  


  Here is the original image followed by the result from one and two passes through the function. Note that the pixel value displayed by when reading these images with Preview has changed ever so slightly. This becomes noticeable within a video.

  


  Test image (very small) ->&#xA; <-

  &#xA;

  Here are the pixel values read (note that after being loaded and saved there is a change) :

  &#xA;

  original test image

  &#xA;

  test image after one pass

  &#xA;

  test image after two passes

  &#xA;

  Edit : here is an RGB24 frame extracted from a video I am using to test my pipeline. I had the same issue with pixel values changing after loading and saving with ffmpeg.

  &#xA;

  frame from video I was testing pipeline on

  &#xA;

  Here is a screenshot showing how the image is noticeably darker after ffmpeg. Same pixels on the top right corner of the image.

  &#xA;

  

  &#xA;

  Here is the code of the toy problem :

  &#xA;

  import os
import ffmpeg
import numpy as np


def read_and_save_image(in_file, out_file, width, height, pix_fmt='rgb32'):
    input_data, _ = (
        ffmpeg
        .input(in_file)
        .output('pipe:', format='rawvideo', pix_fmt=pix_fmt)
        .run(capture_stdout=True)
    )
  
    frame = np.frombuffer(input_data, np.uint8)
    print(in_file,'\n', frame.reshape((height,width,-1)))
    
    save_data = (
        ffmpeg
            .input('pipe:', format='rawvideo', pix_fmt=pix_fmt, s='{}x{}'.format(width, height))
            .output(out_file, pix_fmt=pix_fmt)
            .overwrite_output()
            .run_async(pipe_stdin=True)
    )
    
    

    save_data.stdin.write(frame.tobytes())
    save_data.stdin.close()
    #save_data.wait()

    return frame

try:
    test_img = "test_image.png"
    test_img_1 = "test_image_1.png"
    test_img_2 = "test_image_2.png"
    test_img_3 = "test_image_3.png"

    width, height, pix_fmt = 3,3,'rgb32'
    #width, height, pix_fmt = video_stream['width'], video_stream['height'],  'rgb24'
    test_img_pxls = read_and_save_image(test_img,test_img_1, width, height, pix_fmt)
    test_img_1_pxls = read_and_save_image(test_img_1,test_img_2, width, height, pix_fmt)
    test_img_2_pxls = read_and_save_image(test_img_2,test_img_3, width, height, pix_fmt)

    print(np.array_equiv(test_img_pxls, test_img_1_pxls))
    print(np.array_equiv(test_img_1_pxls, test_img_2_pxls))

except ffmpeg.Error as e:
    print('stdout:', e.stdout.decode('utf8'))
    print('stderr:', e.stderr.decode('utf8'))
    raise e


!mediainfo --Output=JSON --Full $test_img
!mediainfo --Output=JSON --Full $test_img_1
!mediainfo --Output=JSON --Full $test_img_2


  &#xA;

  Here is the console output of the program that shows that the pixel arrays read by ffmpeg are the same despite the images being different.

  &#xA;

  test_image.png 
 [[[253 218 249 255]
  [252 213 248 255]
  [251 200 244 255]]

 [[253 227 250 255]
  [249 209 236 255]
  [243 169 206 255]]

 [[253 235 251 255]
  [245 195 211 255]
  [226 103 125 255]]]
test_image_1.png 
 [[[253 218 249 255]
  [252 213 248 255]
  [251 200 244 255]]

 [[253 227 250 255]
  [249 209 236 255]
  [243 169 206 255]]

 [[253 235 251 255]
  [245 195 211 255]
  [226 103 125 255]]]
test_image_2.png 
 [[[253 218 249 255]
  [252 213 248 255]
  [251 200 244 255]]

 [[253 227 250 255]
  [249 209 236 255]
  [243 169 206 255]]

 [[253 235 251 255]
  [245 195 211 255]
  [226 103 125 255]]]
True
True
{
"media": {
"@ref": "test_image.png",
"track": [
{
"@type": "General",
"ImageCount": "1",
"FileExtension": "png",
"Format": "PNG",
"FileSize": "4105",
"StreamSize": "0",
"File_Modified_Date": "UTC 2023-01-19 13:49:00",
"File_Modified_Date_Local": "2023-01-19 13:49:00"
},
{
"@type": "Image",
"Format": "PNG",
"Format_Compression": "LZ77",
"Width": "3",
"Height": "3",
"BitDepth": "32",
"Compression_Mode": "Lossless",
"StreamSize": "4105"
}
]
}
}

{
"media": {
"@ref": "test_image_1.png",
"track": [
{
"@type": "General",
"ImageCount": "1",
"FileExtension": "png",
"Format": "PNG",
"FileSize": "128",
"StreamSize": "0",
"File_Modified_Date": "UTC 2023-01-24 15:31:58",
"File_Modified_Date_Local": "2023-01-24 15:31:58"
},
{
"@type": "Image",
"Format": "PNG",
"Format_Compression": "LZ77",
"Width": "3",
"Height": "3",
"BitDepth": "32",
"Compression_Mode": "Lossless",
"StreamSize": "128"
}
]
}
}

{
"media": {
"@ref": "test_image_2.png",
"track": [
{
"@type": "General",
"ImageCount": "1",
"FileExtension": "png",
"Format": "PNG",
"FileSize": "128",
"StreamSize": "0",
"File_Modified_Date": "UTC 2023-01-24 15:31:59",
"File_Modified_Date_Local": "2023-01-24 15:31:59"
},
{
"@type": "Image",
"Format": "PNG",
"Format_Compression": "LZ77",
"Width": "3",
"Height": "3",
"BitDepth": "32",
"Compression_Mode": "Lossless",
"StreamSize": "128"
}
]
}
}



  &#xA;

• `ffmpet -f concat` don't work when all input streams appear to have the same spec

  9 mars 2023, par Roy

  My ffmpeg command :

  &#xA;

  ffmpeg -safe 0 -f concat -i list.txt -c copy out.mp4


  &#xA;

  My 1st input file :

  &#xA;

  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\Applications\ffmpeg_6.0_full\a.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Duration: 00:00:04.97, start: 0.000000, bitrate: 40 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 2 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 27 kb/s, 30 fps, 30 tbr, 30k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 libx264


  &#xA;

  My 2nd input file :

  &#xA;

  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\Applications\ffmpeg_6.0_full\b.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp41isom
    creation_time   : 2023-03-08T06:47:13.000000Z
    artist          : Microsoft Game DVR
    title           : PUBG: BATTLEGROUNDS
  Duration: 00:10:00.16, start: 0.000000, bitrate: 20885 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 20739 kb/s, 30 fps, 30 tbr, 30k tbn (default)
    Metadata:
      creation_time   : 2023-03-08T06:47:13.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 131 kb/s (default)
    Metadata:
      creation_time   : 2023-03-08T06:47:13.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]


  &#xA;

  The above command outputs some warning signals :

  &#xA;

  [mov,mp4,m4a,3gp,3g2,mj2 @ 0000025239902d40] Auto-inserting h264_mp4toannexb bitstream filter
[mp4 @ 00000252396fe5c0] Non-monotonous DTS in output stream 0:1; previous: 218112, current: 150024; changing to 218113. This may result in incorrect timestamps in the output file.
...
a lot of them
...
frame=25992 fps=21754 q=-1.0 Lsize= 1519621kB time=00:14:49.39 bitrate=13996.8kbits/s speed= 744x
video:9649kB audio:1519216kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown


  &#xA;

  The resultant video can play the first part of the video correctly, then the video players either skips directly to the end of the video (MPC-HC), or don't render anything at all while timer passes as normal (VLC).

  &#xA;

  My impression of the concat is that it requires all videos to have the same spec, which I think my input achieved (all the "Steam #0:0", etc, line matches). I only see the following difference, which I assumed that should be okay :

  &#xA;

  &#xA;
  1. Metadata are different both for the whole input (e.g. "major_brand") and for each stream (e.g. "encoder"). I assumed that metadata won't affect the processing.

  &#xA;

  2. The order of video/audio streams are different in the two inputs : the 1st input file has audio then video ; the 2nd input file has video then audio. I assumed that ffmpeg knows the difference and won't concat a video stream to an audio stream.

  &#xA;

  &#xA;

  The full output of the command can be found in this pastebin : https://pastebin.com/Z5q97Uyg

  &#xA;

• `ffmpeg -f concat` don't work when all input streams appear to have the same spec

  2 octobre 2024, par Roy

  My ffmpeg command :

  &#xA;

  ffmpeg -safe 0 -f concat -i list.txt -c copy out.mp4


  &#xA;

  My 1st input file :

  &#xA;

  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\Applications\ffmpeg_6.0_full\a.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Duration: 00:00:04.97, start: 0.000000, bitrate: 40 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 2 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 27 kb/s, 30 fps, 30 tbr, 30k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 libx264


  &#xA;

  My 2nd input file :

  &#xA;

  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\Applications\ffmpeg_6.0_full\b.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp41isom
    creation_time   : 2023-03-08T06:47:13.000000Z
    artist          : Microsoft Game DVR
    title           : PUBG: BATTLEGROUNDS
  Duration: 00:10:00.16, start: 0.000000, bitrate: 20885 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 20739 kb/s, 30 fps, 30 tbr, 30k tbn (default)
    Metadata:
      creation_time   : 2023-03-08T06:47:13.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 131 kb/s (default)
    Metadata:
      creation_time   : 2023-03-08T06:47:13.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]


  &#xA;

  The above command outputs some warning signals :

  &#xA;

  [mov,mp4,m4a,3gp,3g2,mj2 @ 0000025239902d40] Auto-inserting h264_mp4toannexb bitstream filter
[mp4 @ 00000252396fe5c0] Non-monotonous DTS in output stream 0:1; previous: 218112, current: 150024; changing to 218113. This may result in incorrect timestamps in the output file.
...
a lot of them
...
frame=25992 fps=21754 q=-1.0 Lsize= 1519621kB time=00:14:49.39 bitrate=13996.8kbits/s speed= 744x
video:9649kB audio:1519216kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown


  &#xA;

  The resultant video can play the first part of the video correctly, then the video players either skips directly to the end of the video (MPC-HC), or don't render anything at all while timer passes as normal (VLC).

  &#xA;

  My impression of the concat is that it requires all videos to have the same spec, which I think my input achieved (all the "Steam #0:0", etc, line matches). I only see the following difference, which I assumed that should be okay :

  &#xA;

  &#xA;
  1. Metadata are different both for the whole input (e.g. "major_brand") and for each stream (e.g. "encoder"). I assumed that metadata won't affect the processing.

  &#xA;

  2. The order of video/audio streams are different in the two inputs : the 1st input file has audio then video ; the 2nd input file has video then audio. I assumed that ffmpeg knows the difference and won't concat a video stream to an audio stream.

  &#xA;

  &#xA;

  The full output of the command can be found in this pastebin : https://pastebin.com/Z5q97Uyg

  &#xA;