Recherche avancée

Sur d’autres sites (5073)

• Basic Video Palette Conversion

  20 août 2011, par Multimedia Mike — General, Python

  How do you take a 24-bit RGB image and convert it to an 8-bit paletted image for the purpose of compression using a codec that requires 8-bit input images ? Seems simple enough and that’s what I’m tackling in this post.

  Ask FFmpeg/Libav To Do It
  Ideally, FFmpeg / Libav should be able to handle this automatically. Indeed, FFmpeg used to be able to, at least at the time I wrote this post about ZMBV and was unhappy with FFmpeg’s default results. Somewhere along the line, FFmpeg and Libav lost the ability to do this. I suspect it got removed during some swscale refactoring.

  Still, there’s no telling if the old system would have computed palettes correctly for QuickTime files.

  Distance Approach
  When I started writing my SMC video encoder, I needed to convert RGB (from PNG files) to PAL8 colorspace. The path of least resistance was to match the pixels in the input image to the default 256-color palette that QuickTime assumes (and is hardcoded into FFmpeg/Libav).

  How to perform the matching ? Find the palette entry that is closest to a given input pixel, where "closest" is the minimum distance as computed by the usual distance formula (square root of the sum of the squares of the diffs of all the components).

  
  
  

  That means for each pixel in an image, check the pixel against 256 palette entries (early termination is possible if an acceptable threshold is met). As you might imagine, this can be a bit time-consuming. I wondered about a faster approach...

  Lookup Table
  I think this is the approach that FFmpeg used to use, but I went and derived it for myself after studying the default QuickTime palette table. There’s a pattern there— all of the RGB entries are comprised of combinations of 6 values — 0x00, 0x33, 0x66, 0x99, 0xCC, and 0xFF. If you mix and match these for red, green, and blue values, you come up with 6 * 6 * 6 = 216 different colors. This happens to be identical to the web-safe color palette.

  The first (0th) entry in the table is (FF, FF, FF), followed by (FF, FF, CC), (FF, FF, 99), and on down to (FF, FF, 00) when the green component gets knocked down and step and the next color is (FF, CC, FF). The first 36 palette entries in the table all have a red component of 0xFF. Thus, if an input RGB pixel has a red color closest to 0xFF, it must map to one of those first 36 entries.

  I created a table which maps indices 0..215 to values from 5..0. Each of the R, G, and B components of an input pixel are used to index into this table and derive 3 indices ri, gi, and bi. Finally, the index into the palette table is given by :

    index = ri * 36 + gi * 6 + bi
  

  For example, the pixel (0xFE, 0xFE, 0x01) would yield ri, gi, and bi values of 0, 0, and 5. Therefore :

    index = 0 * 36 + 0 * 6 + 5
  

  The palette index is 5, which maps to color (0xFF, 0xFF, 0x00).

  Validation
  So I was pretty pleased with myself for coming up with that. Now, ideally, swapping out one algorithm for another in my SMC encoder should yield identical results. That wasn’t the case, initially.

  One problem is that the regulation QuickTime palette actually has 40 more entries above and beyond the typical 216-entry color cube (rounding out the grand total of 256 colors). Thus, using the distance approach with the full default table provides for a little more accuracy.

  However, there still seems to be a problem. Let’s check our old standby, the Big Buck Bunny logo image :

  
  
  Distance approach using the full 256-color QuickTime default palette
  
  
  Distance approach using the 216-color palette
  
  
  Table lookup approach using the 216-color palette
  

  I can’t quite account for that big red splotch there. That’s the most notable difference between images 1 and 2 and the only visible difference between images 2 and 3.

  To prove to myself that the distance approach is equivalent to the table approach, I wrote a Python script to iterate through all possible RGB combinations and verify the equivalence. If you’re not up on your base 2 math, that’s 2²⁴ or 16,777,216 colors to run through. I used Python’s multiprocessing module to great effect and really maximized a Core i7 CPU with 8 hardware threads.

  So I’m confident that the palette conversion techniques are sound. The red spot is probably attributable to a bug in my WIP SMC encoder.

  Source Code
  Update August 23, 2011 : Here’s the Python code I used for proving equivalence between the 2 approaches. In terms of leveraging multiple CPUs, it’s possibly the best program I have written to date.

  PLAIN TEXT

  PYTHON :

  1. # !/usr/bin/python
  2.  
  3. from multiprocessing import Pool
  4.  
  5. palette = []
  6. pal8_table = []
  7.  
  8. def process_r(r) :
  9.  counts = []
  10.  
  11.  for i in xrange(216) :
  12.   counts.append(0)
  13.  
  14.  print "r = %d" % (r)
  15.  for g in xrange(256) :
  16.   for b in xrange(256) :
  17.    min_dsqrd = 0xFFFFFFFF
  18.    best_index = 0
  19.    for i in xrange(len(palette)) :
  20.     dr = palette[i][0] - r
  21.     dg = palette[i][1] - g
  22.     db = palette[i][2] - b
  23.     dsqrd = dr * dr + dg * dg + db * db
  24.     if dsqrd <min_dsqrd :
  25.      min_dsqrd = dsqrd
  26.      best_index = i
  27.    counts[best_index] += 1
  28.  
  29.    # check if the distance approach deviates from the table-based approach
  30.    i = best_index
  31.    r = palette[i][0]
  32.    g = palette[i][1]
  33.    b = palette[i][2]
  34.    ri = pal8_table[r]
  35.    gi = pal8_table[g]
  36.    bi = pal8_table[b]
  37.    table_index = ri * 36 + gi * 6 + bi ;
  38.    if table_index != best_index :
  39.     print "(0x%02X 0x%02X 0x%02X) : distance index = %d, table index = %d" % (r, g, b, best_index, table_index)
  40.  
  41.  return counts
  42.  
  43. if __name__ == ’__main__’ :
  44.  counts = []
  45.  for i in xrange(216) :
  46.   counts.append(0)
  47.  
  48.  # initialize reference palette
  49.  color_steps = [ 0xFF, 0xCC, 0x99, 0x66, 0x33, 0x00 ]
  50.  for r in color_steps :
  51.   for g in color_steps :
  52.    for b in color_steps :
  53.     palette.append([r, g, b])
  54.  
  55.  # initialize palette conversion table
  56.  for i in range(0, 26) :
  57.   pal8_table.append(5)
  58.  for i in range(26, 77) :
  59.   pal8_table.append(4)
  60.  for i in range(77, 128) :
  61.   pal8_table.append(3)
  62.  for i in range(128, 179) :
  63.   pal8_table.append(2)
  64.  for i in range(179, 230) :
  65.   pal8_table.append(1)
  66.  for i in range(230, 256) :
  67.   pal8_table.append(0)
  68.  
  69.  # create a pool of worker threads and break up the overall job
  70.  pool = Pool()
  71.  it = pool.imap_unordered(process_r, range(256))
  72.  try :
  73.   while 1 :
  74.    partial_counts = it.next()
  75.    for i in xrange(216) :
  76.     counts[i] += partial_counts[i]
  77.  except StopIteration :
  78.   pass
  79.  
  80.  print "index, count, red, green, blue"
  81.  for i in xrange(len(counts)) :
  82.   print "%d, %d, %d, %d, %d" % (i, counts[i], palette[i][0], palette[i][1], palette[i][2])

• FFmpeg generated video length doesn't match expected length [on hold]

  21 juin 2018, par BentCoder

  I am using command below to generate video out of images. The problem is the length out the video generated. It is always 5 seconds less than expected. If I have 4 images, I expect to see 20 seconds of video but I am getting 15 seconds instead. The very last image img-03.jpg just appears at the end of the video as if it is a cover image. Any idea why and solution ?

  Command

  ffmpeg -y -framerate 1/5 -f image2 \
  
  -i img-%2d.jpg \
  
  -c:v libvpx-vp9 \
  
  -r 25 \
  
  -crf 30 -b:v 0 \
  
  video.webm

  Images

  img-00.jpg
  
  img-01.jpg
  
  img-02.jpg
  
  img-03.jpg

  Output

  ffmpeg version 3.2.10-1~deb9u1~bpo8+1 Copyright (c) 2000-2018 the FFmpeg developers
  
    built with gcc 4.9.2 (Debian 4.9.2-10)
  
    configuration: --prefix=/usr --extra-version='1~deb9u1~bpo8+1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --disable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  
    libavutil      55. 34.101 / 55. 34.101
  
    libavcodec     57. 64.101 / 57. 64.101
  
    libavformat    57. 56.101 / 57. 56.101
  
    libavdevice    57.  1.100 / 57.  1.100
  
    libavfilter     6. 65.100 /  6. 65.100
  
    libavresample   3.  1.  0 /  3.  1.  0
  
    libswscale      4.  2.100 /  4.  2.100
  
    libswresample   2.  3.100 /  2.  3.100
  
    libpostproc    54.  1.100 / 54.  1.100
  
  Input #0, image2, from 'img-%2d.jpg':
  
    Duration: 00:00:20.00, start: 0.000000, bitrate: N/A
  
      Stream #0:0: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 854x480 [SAR 1:1 DAR 427:240], 0.20 fps, 0.20 tbr, 0.20 tbn, 0.20 tbc
  
  [swscaler @ 0x55f2af60e800] deprecated pixel format used, make sure you did set range correctly
  
  [libvpx-vp9 @ 0x55f2af6276a0] v1.3.0
  
  Output #0, webm, to 'video.webm':
  
    Metadata:
  
      encoder         : Lavf57.56.101
  
      Stream #0:0: Video: vp9 (libvpx-vp9), yuv420p, 854x480 [SAR 1:1 DAR 427:240], q=-1--1, 25 fps, 1k tbn, 25 tbc
  
      Metadata:
  
        encoder         : Lavc57.64.101 libvpx-vp9
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mjpeg (native) -> vp9 (libvpx-vp9))
  
  Press [q] to stop, [?] for help
  
  Input stream #0:0 frame changed from size:854x480 fmt:yuvj444p to size:854x480 fmt:yuvj420p
  
  [swscaler @ 0x55f2af602b60] deprecated pixel format used, make sure you did set range correctly
  
  frame=    4 fps=0.0 q=0.0 Lsize=     214kB time=00:00:15.00 bitrate= 117.0kbits/s speed=  16x    
  
  video:214kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.234810%

• FFMPEG parameter -sseof position is not “cutting” as expected

  9 mai 2020, par Vitalii

  I have a regular recording of the desktop screen, indefinite in time. I want to keep in memory, only the last 30 seconds. My command looks like this, where I try to cut the video of the last 30 seconds, using -sizeof -30. But the entire length of the video is saved.
  &#xA;avfoundation -i '1' it's my macbook screen

  &#xA;&#xA;

  ffmpeg -rtbufsize 300M -sseof -30 -f avfoundation -i '1' -c:v libx264 -preset ultrafast -tune zerolatency -crf 25 ~/Desktop/output.mkv


  &#xA;&#xA;

  System Macos 10.15
  &#xA;ffmpeg version 4.2.2

  &#xA;&#xA;

  My full log

  &#xA;&#xA;

  ffmpeg -rtbufsize 300M -sseof -30  -f avfoundation -i '1' -c:v libx264 -preset ultrafast -tune zerolatency -crf 25 ~/Desktop/output.mkv
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.59)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_3 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags=-fno-stack-check --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[AVFoundation input device @ 0x7fec8551cec0] Configuration of video device failed, falling back to default.
[avfoundation @ 0x7fec8600a200] Selected pixel format (yuv420p) is not supported by the input device.
[avfoundation @ 0x7fec8600a200] Supported pixel formats:
[avfoundation @ 0x7fec8600a200]   uyvy422
[avfoundation @ 0x7fec8600a200]   yuyv422
[avfoundation @ 0x7fec8600a200]   nv12
[avfoundation @ 0x7fec8600a200]   0rgb
[avfoundation @ 0x7fec8600a200]   bgr0
[avfoundation @ 0x7fec8600a200] Overriding selected pixel format to use uyvy422 instead.
[avfoundation @ 0x7fec8600a200] Stream #0: not enough frames to estimate rate; consider increasing probesize
Cannot use -sseof, duration of 1 not known
Input #0, avfoundation, from '1':
  Duration: N/A, start: 107388.118500, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 2560x1600, 1000k tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x7fec869bee00] MB rate (16000000000) > level limit (16711680)
[libx264 @ 0x7fec869bee00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fec869bee00] profile High 4:2:2, level 6.2, 4:2:2 8-bit
[libx264 @ 0x7fec869bee00] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=25.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, matroska, to '/Users/vitalii-artec3d/Desktop/output.mkv':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264) (H264 / 0x34363248), yuv422p, 2560x1600, q=-1--1, 1000k fps, 1k tbn, 1000k tbc
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame=  322 fps= 30 q=19.0 Lsize=     947kB time=00:00:10.80 bitrate= 718.0kbits/s speed=0.998x    
video:943kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.382723%
[libx264 @ 0x7fec869bee00] frame I:2     Avg QP:19.00  size:465715
[libx264 @ 0x7fec869bee00] frame P:320   Avg QP:19.30  size:   105
[libx264 @ 0x7fec869bee00] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x7fec869bee00] mb P  I16..4:  0.0%  0.0%  0.0%  P16..4:  0.1%  0.0%  0.0%  0.0%  0.0%    skip:99.9%
[libx264 @ 0x7fec869bee00] coded y,uvDC,uvAC intra: 36.9% 24.9% 21.0% inter: 0.0% 0.1% 0.0%
[libx264 @ 0x7fec869bee00] i16 v,h,dc,p: 66% 28%  4%  2%
[libx264 @ 0x7fec869bee00] i8c dc,h,v,p: 70% 16% 12%  1%
[libx264 @ 0x7fec869bee00] kb/s:23977740.00


  &#xA;