Recherche avancée

Sur d’autres sites (13760)

• The Big VP8 Debug

  20 novembre 2010, par Multimedia Mike — VP8

  I hope my previous walkthrough of the VP8 4x4 intra coding process was educational. Today, I’ll be walking through an example of what happens when my toy VP8 encoder encodes an intra 16x16 block. This may prove educational to those who have never been exposed to the deep details of this or related algorithms. Also, I wanted to illustrate where I think my VP8 encoder process is going bad and generating such grotesque results.

  Before I start, let me give a shout-out to Google Docs’ Drawing tool which I used to generate these diagrams. It works quite well.

  Results

  (Always cut to the chase in a blog post ; results first.) I’m glad I composed this post. In the course of doing so, I found the problem, fixed it, and am now able to present this image that was decoded from the bitstream encoded by my toy working VP8 encoder :

  
  
  

  Yeah, I know that image doesn’t look like anything you haven’t seen before. The difference is that it has made a successful trip through my VP8 encoder.

  Follow along through the encoding process and learn of the mistake...

  Original Block and Subblocks

  Here is the 16x16 block to be encoded :

  
  
  

  The block is broken down into 16 4x4 subblocks for further encoding :

  
  
  

  Prediction
  
  The first step is to pick a prediction mode, generate a prediction block, and subtract the predictors from the macroblock. In this case, we will use DC prediction which means the predictor will be the same for each element.

  In 4x4 VP8 DC intra prediction, samples outside of the frame are assumed to be 128. It’s a little different in 16x16 DC intra prediction— samples above the top row are assumed to be 127 while samples left of the leftmost column are assumed to be 129. For the top left macroblock, this still works out to 128.

  Subtract 128 from each of the samples :

  
  
  

  Forward Transform

  Run each of the 16 prediction-removed subblocks through the forward transform. This example uses the forward transform from libvpx 0.9.5 :

  
  
  

  I have highlighted the DC coefficients in each subblock. That’s because those receive special consideration in 16x16 intra coding.

  Quantization

  The Y plane AC quantizer is 4 in this example, the minimum allowed. (The Y plane DC quantizer is also 4 but doesn’t come into play for intra 16x16 coding since the DC coefficients follow a different process.) Thus, quantize (integer divide) each AC element in each subblock (we’ll ignore the DC coefficient for this part) :

  
  
  

  The Y2 Round Trip

  Those highlighted DC coefficients from each of the 16 subblocks comprise the Y2 block. This block is transformed with a slightly different algorithm called the Walsh-Hadamard Transform (WHT). The results of this transform are then quantized (using 8 for both Y2 DC and AC in this example, as those are the smallest Y2 quantizers that VP8 allows), then zigzagged and entropy-coded along with the rest of the macroblock coefficients.

  On the decoder side, the Y2 coefficients are decoded, de-zigzagged, dequantized and run through the inverse WHT.

  And this is where I suspect that most of the error is creeping into my VP8 encoder. Observe the round-trip through the Y2 process :

  
  
  

  As intimated, this part causes me consternation due to the wide discrepancy between the original and the reconstructed Y2 blocks. Observe the absolute difference between the 2 vectors :

  
  
  

  That’s really significant and leads me to believe that this is where the big problem is.

  What’s Wrong ?

  My first suspicion is that the quantization is throwing off the process. I was disabused of this idea when I removed quantization from the equation and immediately reversed the transform :

  
  
  

  So perhaps there is a problem with the forward WHT. Just like with the usual subblock transform, the VP8 spec doesn’t define how to perform the forward WHT, only the inverse WHT. Do I need to audition different forward WHTs from various versions of libvpx, similar to what I did with the other transform ? That doesn’t make a lot of sense— libvpx doesn’t seem to have so much trouble with basic encoding.

  The Punchline

  I reviewed the forward WHT code, the stuff that I plagiarized from libvpx 0.9.0. The function takes, among other parameters, a pitch value. There are 2 loops in the code. The first iterates through the rows of the input matrix— which I assumed was a 4x4 matrix. I was puzzled that during each iteration of the row loop, the input pointer was only being advanced by (pitch/2). I removed the division by 2 and the problem went away. I.e., the encoded image looks correct.

  What’s up with the (pitch/2), anyway ? It seems that the encoder likes to pack 2 4x4 subblocks into an 8x4 block data structure. In fact, the forward DCTs in the libvpx encoder have the same artifact. Remember how I surveyed several variations of forward DCT from different versions of libvpx ? The one that proved most accurate in that test was the one I had already modified to advance the input pointer properly. Fixing the other 2 candidates yields similar results :

  input :   92 91 89 86 91 90 88 86 89 89 89 88 89 87 88 93
  short 0.9.0 : -311 6 2 0 0 11 -6 1 2 -3 3 0 0 0 -2 1
  inverse : 92 91 89 86 91 90 88 87 90 89 89 88 89 87 88 93
  fast  0.9.0 : -313 5 1 0 1 11 -6 1 3 -3 4 0 0 0 -2 1
  inverse : 91 91 89 86 90 90 88 86 89 89 89 88 89 87 88 93
  short 0.9.5 : -312 7 1 0 1 12 -5 2 2 -3 3 -1 1 0 -2 1
  inverse : 92 91 89 86 91 90 88 86 89 89 89 88 89 87 88 93
  

  Code cribber beware !

  Corrected Y2 Round Trip

  Let’s look at that Y2 round trip one more time :

  
  
  

  And another look at the error between the original and the reconstruction :

  
  
  

  Better.

  Dequantization, Prediction, Inverse Transforms, and Reconstruction

  To be honest, now that I solved the major problem, I’m getting a little tired of making these pictures. Long story short, all elements of the original 16 subblocks are dequantized and their DC coefficients are filled in with the appropriate item from the reconstructed Y2 block. A base predictor block is generated (all 128 values in this case). And each Y block is run through the inverse transform and added to the predictor block. The following is the reconstruction :

  
  
  

  And if you compare that against the original luma macroblock (I don’t feel like doing it right now), you’ll find that it’s pretty close.

  I can’t believe how close I was all this time, and how long that pitch bug held me up.

• Could not write header for output file #0 (incorrect codec parameters ?) : No such file or directory Error initializing output stream 0:5 —

  13 septembre 2022, par M E

  I am trying to create a DASH manifest and segments from an .mp4 file. I have created a bash files with the commands below, and calling bbb.mp4, but the output gives the error on the subject line. This is the script I am using :

  


  ffmpeg -re -i bbb.mp4 -map 0 -map 0 -c:a aac -c:v libx264 \
-b:v:0 800k -b:v:1 300k -s:v:1 320x170 -profile:v:1 baseline \
-profile:v:0 main -bf 1 -keyint_min 120 -g 120 -sc_threshold 0 \
-b_strategy 0 -ar:a:1 22050 -use_timeline 1 -use_template 1 \
-window_size 5 -adaptation_sets "id=0,streams=v id=1,streams=a" \
-f dash /path/to/out.mpd


  


  After running the command for my bash file dash-demo I get :

  


  $ dash-demo.sh
ffmpeg version 5.1.1-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bbb.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 1
    compatible_brands: isomavc1
    creation_time   : 2013-12-16T17:59:32.000000Z
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
    composer        : Sacha Goedegebure
  Duration: 00:10:34.53, start: 0.000000, bitrate: 4486 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 4001 kb/s, 60 fps, 60 tbr, 60k tbn (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:32.000000Z
      handler_name    : GPAC ISO Video Handler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s (default)
    Metadata:
      creation_time   : 2013-12-16T17:59:37.000000Z
      handler_name    : GPAC ISO Audio Handler
      vendor_id       : [0][0][0][0]
    Side data:
      audio service type: main
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (mp3 (mp3float) -> aac (native))
  Stream #0:2 -> #0:2 (ac3 (native) -> aac (native))
  Stream #0:0 -> #0:3 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:4 (mp3 (mp3float) -> aac (native))
  Stream #0:2 -> #0:5 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 00000172d0783040] using SAR=1/1
[libx264 @ 00000172d0783040] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000172d0783040] profile Main, level 4.2, 4:2:0, 8-bit
[libx264 @ 00000172d0783040] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=1 b_pyramid=0 b_adapt=0 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=120 keyint_min=61 scenecut=0 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=800 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[libx264 @ 00000172d011dac0] using SAR=17/18
[libx264 @ 00000172d011dac0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000172d011dac0] profile Constrained Baseline, level 2.1, 4:2:0, 8-bit
[libx264 @ 00000172d011dac0] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=5 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=120 keyint_min=61 scenecut=0 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=300 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[aac @ 00000172d011d2c0] Using a PCE to encode channel layout "5.1(side)"
[aac @ 00000172d0136580] Using a PCE to encode channel layout "5.1(side)"
[dash @ 00000172d0a82880] Opening 'C:/Program Files/Git/path/to/init-stream0.m4s' for writing
Could not write header for output file #0 (incorrect codec parameters ?): No such file or directory
Error initializing output stream 0:5 --
[libx264 @ 00000172d0783040] final ratefactor: 41.41
[aac @ 00000172d04ffc00] Qavg: 8678.318
[aac @ 00000172d04ffc00] 2 frames left in the queue on closing
[aac @ 00000172d011d2c0] Qavg: nan
[libx264 @ 00000172d011dac0] final ratefactor: 34.26
[aac @ 00000172d0136180] Qavg: 8678.318
[aac @ 00000172d0136180] 2 frames left in the queue on closing
[aac @ 00000172d0136580] Qavg: nan
Conversion failed!


  


  What am I doing wrong ?

  


  Thank you !

  


• Revision 7ab9a9587b : Remove Wextra warnings from vp9_sad.c As a side-effect, the max_sad check is re

  13 mai 2014, par Deb Mukherjee

  Changed Paths :
   Modify /test/sad_test.cc

  
   Modify /vp8/common/rtcd_defs.pl

  
   Modify /vp8/common/sad_c.c

  
   Modify /vp8/common/variance.h

  
   Modify /vp8/encoder/mcomp.c

  
   Modify /vp8/encoder/rdopt.c

  
   Modify /vp9/common/vp9_rtcd_defs.pl

  
   Modify /vp9/encoder/vp9_encodeframe.c

  
   Modify /vp9/encoder/vp9_mbgraph.c

  
   Modify /vp9/encoder/vp9_mcomp.c

  
   Modify /vp9/encoder/vp9_rdopt.c

  
   Modify /vp9/encoder/vp9_sad.c

  
   Modify /vp9/encoder/vp9_variance.h

  
  
  Remove Wextra warnings from vp9_sad.c

  As a side-effect, the max_sad check is removed from the
  C-implementation of VP8, for consistency with VP9, and to
  ensure that the SAD tests common to VP8/VP9 pass.
  That will make the VP8 C implementation of sad a little slower
  but given that is rarely used in practice, the impact will be
  minimal.

  Change-Id : I7f43089fdea047fbf1862e40c21e4715c30f07ca