Recherche avancée

Sur d’autres sites (6822)

• Using ffmpeg for streaming video composition from Python : why the sound is cut only when streaming into a mp4 file ?

  4 décembre 2017, par redantlabs

  I am using ffmpeg within Python for automatizing some video compositions. To do so, I am using the subprocess module of Python and run a ffmpeg command with a pipe input. This allows me to stream frame by frame the video composition from my array of raw images (numpy matrices) into ffmpeg. In the following example, I reduced the video composition to a simple video of a duration of 12 seconds. The produced ffmpeg command is :

  ffmpeg -i stream_audio.mp3 -re -f rawvideo -vcodec rawvideo -s 1280x720 -pix_fmt rgb24 -r 25 -i - -map 0:a -map 1:v -y -strict -2 -f mp4 -pix_fmt yuv420p out.mp4

  The file stream_audio.mp3 is a well formed mp3 file corresponding to the audio output of the output mp4 file. However, the sound in the output file out.mp4 is cut 3 seconds before the end.

  If I am trying to output any other format (for example avi), I do not observe this problem. The following command produces a well formed avi file.

   ffmpeg -i ./stream_audio.mp3 -re -f rawvideo -vcodec rawvideo -s 1280x720 -pix_fmt rgb24 -r 25 -i - -map 0:a -map 1:v -y -b 4096k -f avi -pix_fmt yuv420p out.avi

  I tried to reproduce the bug with the most simple ffmpeg command without passing by Python, but did not succeed.The closest I have done is the following :

  Building a raw video file with the correct specifications

  ffmpeg -i samplevideo.mp4 -f rawvideo -vcodec rawvideo -acodec none -s 1280x720 -pix_fmt rgb24 -r 25 samplevideo.raw

  Streaming the raw video file into the ffmpeg command

  cat samplevideo.raw | ffmpeg -i /tmp/stream_audio.mp3 -re -f rawvideo -vcodec rawvideo -s 1280x720 -pix_fmt rgb24 -r 25 -i - -map 0:a -map 1:v 
  
  -y -strict -2 -f mp4 -pix_fmt yuv420p  -ss 00:00:00 -t 00:00:12 out.mp4

  Here is the version of ffmpeg I am using :

  ffmpeg version 3.1.9 Copyright (c) 2000-2017 the FFmpeg developers
  
    built with gcc 6.3.1 (GCC) 20161221 (Red Hat 6.3.1-1)
  
    configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --extra-ldflags='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libcdio --enable-indev=jack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-nvenc --extra-cflags=-I/usr/include/nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-libmfx --enable-runtime-cpudetect
  
    libavutil      55. 28.100 / 55. 28.100
  
    libavcodec     57. 48.101 / 57. 48.101
  
    libavformat    57. 41.100 / 57. 41.100
  
    libavdevice    57.  0.101 / 57.  0.101
  
    libavfilter     6. 47.100 /  6. 47.100
  
    libavresample   3.  0.  0 /  3.  0.  0
  
    libswscale      4.  1.100 /  4.  1.100
  
    libswresample   2.  1.100 /  2.  1.100
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Hyper fast Audio and Video encoder

  The file samplevideo.mp4 can be found here : sample videos, the stream_audio.mp3 file is a simple extraction of the audio track of the samplevideo.mp4 file :

  ffmpeg -i samplevideo.mp4 stream_audio.mp3

  Thanks for your help.

  p.s : Here are the different logs of the ffmpeg commands :

  Command generated by my Python script with mp4 :

  Input #0, mp3, from './stream_audio.mp3':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf57.41.100
  
    Duration: 00:00:29.59, start: 0.023021, bitrate: 128 kb/s
  
      Stream #0:0: Audio: mp3, 48000 Hz, stereo, s16p, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.48
  
  Input #1, rawvideo, from 'pipe:':
  
    Duration: N/A, start: 0.000000, bitrate: 552960 kb/s
  
      Stream #1:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1280x720, 552960 kb/s, 25 tbr, 25 tbn, 25 tbc
  
  [libx264 @ 0xf5cb748ac0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
  
  [libx264 @ 0xf5cb748ac0] profile High, level 3.1
  
  [libx264 @ 0xf5cb748ac0] 264 - core 148 r2708 86b7198 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  [mp4 @ 0xf5cb7462a0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
  
      Last message repeated 1 times
  
  Output #0, mp4, to 'out.mp4':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf57.41.100
  
      Stream #0:0: Audio: aac (LC) ([64][0][0][0] / 0x0040), 48000 Hz, stereo, fltp, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.48.101 aac
  
      Stream #0:1: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=-1--1, 25 fps, 12800 tbn, 25 tbc
  
      Metadata:
  
        encoder         : Lavc57.48.101 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mp3 (native) -> aac (native))
  
    Stream #1:0 -> #0:1 (rawvideo (native) -> h264 (libx264))
  
  [rawvideo @ 0xf5cb722860] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  
  frame=   14 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   27 fps= 27 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   40 fps= 26 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   52 fps= 26 q=28.0 size=     113kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   65 fps= 26 q=28.0 size=     188kB time=00:00:00.49 bitrate=3145.8kbits/s speed=0.194x    
  
  frame=   77 fps= 25 q=28.0 size=     320kB time=00:00:01.00 bitrate=2615.3kbits/s speed=0.331x    
  
  frame=   90 fps= 25 q=28.0 size=     450kB time=00:00:01.51 bitrate=2431.3kbits/s speed=0.428x    
  
  frame=  103 fps= 25 q=28.0 size=     592kB time=00:00:02.00 bitrate=2418.9kbits/s speed=0.496x    
  
  frame=  116 fps= 25 q=28.0 size=     731kB time=00:00:02.52 bitrate=2376.1kbits/s speed=0.554x    
  
  frame=  128 fps= 25 q=28.0 size=     849kB time=00:00:03.02 bitrate=2295.6kbits/s speed=0.599x    
  
  frame=  141 fps= 25 q=28.0 size=     931kB time=00:00:03.54 bitrate=2153.0kbits/s speed=0.637x    
  
  frame=  153 fps= 25 q=28.0 size=     983kB time=00:00:04.03 bitrate=1996.7kbits/s speed=0.665x    
  
  frame=  166 fps= 25 q=28.0 size=    1067kB time=00:00:04.56 bitrate=1914.5kbits/s speed=0.695x    
  
  frame=  179 fps= 25 q=28.0 size=    1123kB time=00:00:05.04 bitrate=1824.8kbits/s speed=0.712x    
  
  frame=  191 fps= 25 q=28.0 size=    1213kB time=00:00:05.54 bitrate=1791.0kbits/s speed=0.732x    
  
  frame=  204 fps= 25 q=28.0 size=    1271kB time=00:00:06.05 bitrate=1718.5kbits/s speed=0.749x    
  
  frame=  217 fps= 25 q=28.0 size=    1346kB time=00:00:06.57 bitrate=1678.1kbits/s speed=0.764x    
  
  frame=  230 fps= 25 q=28.0 size=    1452kB time=00:00:07.08 bitrate=1678.9kbits/s speed=0.778x    
  
  frame=  242 fps= 25 q=28.0 size=    1567kB time=00:00:07.59 bitrate=1690.3kbits/s speed=0.79x    
  
  frame=  255 fps= 25 q=28.0 size=    1660kB time=00:00:08.08 bitrate=1682.4kbits/s speed=0.799x    
  
  frame=  267 fps= 25 q=28.0 size=    1739kB time=00:00:08.59 bitrate=1657.1kbits/s speed=0.81x    
  
  frame=  280 fps= 25 q=28.0 size=    1751kB time=00:00:09.08 bitrate=1578.1kbits/s speed=0.817x    
  
  frame=  291 fps= 24 q=-1.0 Lsize=    1864kB time=00:00:11.52 bitrate=1325.7kbits/s speed=0.954x    
  
  video:1707kB audio:149kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.437266%
  
  [aac @ 0xf5cb747540] Qavg: 202.422
  
  [libx264 @ 0xf5cb748ac0] frame I:2     Avg QP:15.83  size: 78710
  
  [libx264 @ 0xf5cb748ac0] frame P:183   Avg QP:21.13  size:  7293
  
  [libx264 @ 0xf5cb748ac0] frame B:106   Avg QP:26.82  size:  2406
  
  [libx264 @ 0xf5cb748ac0] consecutive B-frames: 47.8% 10.3%  2.1% 39.9%
  
  [libx264 @ 0xf5cb748ac0] mb I  I16..4: 22.1% 32.5% 45.4%
  
  [libx264 @ 0xf5cb748ac0] mb P  I16..4:  0.8%  1.4%  0.2%  P16..4: 25.3%  6.3%  3.1%  0.0%  0.0%    skip:62.9%
  
  [libx264 @ 0xf5cb748ac0] mb B  I16..4:  0.1%  0.2%  0.0%  B16..8: 39.0%  1.7%  0.2%  direct: 0.4%  skip:58.5%  L0:47.3% L1:49.6% BI: 3.1%
  
  [libx264 @ 0xf5cb748ac0] 8x8 transform intra:50.3% inter:57.9%
  
  [libx264 @ 0xf5cb748ac0] coded y,uvDC,uvAC intra: 49.5% 65.3% 23.0% inter: 7.8% 10.2% 0.5%
  
  [libx264 @ 0xf5cb748ac0] i16 v,h,dc,p: 30% 32% 11% 27%
  
  [libx264 @ 0xf5cb748ac0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 21% 23%  4%  5%  7%  5%  5%  5%
  
  [libx264 @ 0xf5cb748ac0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 15% 11%  8%  8%  9%  7% 12%  7%
  
  [libx264 @ 0xf5cb748ac0] i8c dc,h,v,p: 51% 20% 21%  8%
  
  [libx264 @ 0xf5cb748ac0] Weighted P-Frames: Y:0.0% UV:0.0%
  
  [libx264 @ 0xf5cb748ac0] ref P L0: 79.8% 11.4%  7.6%  1.2%
  
  [libx264 @ 0xf5cb748ac0] ref B L0: 96.9%  2.8%  0.4%
  
  [libx264 @ 0xf5cb748ac0] ref B L1: 97.7%  2.3%
  
  [libx264 @ 0xf5cb748ac0] kb/s:1200.72
  
  Exiting normally, received signal 15.

  Command generated by my Python script with avi :

  Input #0, mp3, from './stream_audio.mp3':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf57.41.100
  
    Duration: 00:00:29.59, start: 0.023021, bitrate: 128 kb/s
  
      Stream #0:0: Audio: mp3, 48000 Hz, stereo, s16p, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.48
  
  Input #1, rawvideo, from 'pipe:':
  
    Duration: N/A, start: 0.000000, bitrate: 552960 kb/s
  
      Stream #1:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1280x720, 552960 kb/s, 25 tbr, 25 tbn, 25 tbc
  
  Please use -b:a or -b:v, -b is ambiguous
  
  [avi @ 0x82147c30e0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
  
      Last message repeated 1 times
  
  Output #0, avi, to 'out.avi':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      ISFT            : Lavf57.41.100
  
      Stream #0:0: Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 48000 Hz, stereo, s16p
  
      Metadata:
  
        encoder         : Lavc57.48.101 libmp3lame
  
      Stream #0:1: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1280x720, q=2-31, 4096 kb/s, 25 fps, 25 tbn, 25 tbc
  
      Metadata:
  
        encoder         : Lavc57.48.101 mpeg4
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/4096000 buffer size: 0 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mp3 (native) -> mp3 (libmp3lame))
  
    Stream #1:0 -> #0:1 (rawvideo (native) -> mpeg4 (native))
  
  [rawvideo @ 0x821479f820] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  
  frame=   14 fps=0.0 q=2.6 size=     553kB time=00:00:00.56 bitrate=8082.6kbits/s speed= 1.1x    
  
  frame=   27 fps= 27 q=5.4 size=     858kB time=00:00:01.08 bitrate=6507.5kbits/s speed=1.06x    
  
  frame=   40 fps= 26 q=5.9 size=    1089kB time=00:00:01.60 bitrate=5574.9kbits/s speed=1.05x    
  
  frame=   52 fps= 26 q=5.5 size=    1332kB time=00:00:02.08 bitrate=5225.6kbits/s speed=1.03x    
  
  frame=   65 fps= 26 q=4.9 size=    1582kB time=00:00:02.60 bitrate=4985.6kbits/s speed=1.03x    
  
  frame=   78 fps= 26 q=4.3 size=    1816kB time=00:00:03.12 bitrate=4768.1kbits/s speed=1.03x    
  
  frame=   90 fps= 25 q=2.8 size=    2035kB time=00:00:03.60 bitrate=4631.7kbits/s speed=1.02x    
  
  frame=  103 fps= 25 q=2.3 size=    2288kB time=00:00:04.12 bitrate=4549.8kbits/s speed=1.02x    
  
  frame=  116 fps= 25 q=2.4 size=    2558kB time=00:00:04.64 bitrate=4516.3kbits/s speed=1.02x    
  
  frame=  128 fps= 25 q=2.3 size=    2835kB time=00:00:05.12 bitrate=4535.4kbits/s speed=1.01x    
  
  frame=  141 fps= 25 q=3.1 size=    3103kB time=00:00:05.64 bitrate=4506.6kbits/s speed=1.01x    
  
  frame=  154 fps= 25 q=2.8 size=    3381kB time=00:00:06.16 bitrate=4495.9kbits/s speed=1.01x    
  
  frame=  166 fps= 25 q=3.4 size=    3648kB time=00:00:06.64 bitrate=4494.9kbits/s speed=1.01x    
  
  frame=  179 fps= 25 q=3.3 size=    3894kB time=00:00:07.16 bitrate=4455.7kbits/s speed=1.01x    
  
  frame=  192 fps= 25 q=3.2 size=    4128kB time=00:00:07.68 bitrate=4402.9kbits/s speed=1.01x    
  
  frame=  204 fps= 25 q=3.4 size=    4404kB time=00:00:08.16 bitrate=4420.9kbits/s speed=1.01x    
  
  frame=  217 fps= 25 q=2.0 size=    4592kB time=00:00:08.68 bitrate=4334.0kbits/s speed=1.01x    
  
  frame=  230 fps= 25 q=2.0 size=    4657kB time=00:00:09.20 bitrate=4147.2kbits/s speed=1.01x    
  
  frame=  242 fps= 25 q=2.0 size=    4736kB time=00:00:09.68 bitrate=4007.9kbits/s speed=1.01x    
  
  frame=  255 fps= 25 q=2.0 size=    4830kB time=00:00:10.20 bitrate=3879.0kbits/s speed=1.01x    
  
  frame=  267 fps= 25 q=2.0 size=    4953kB time=00:00:10.68 bitrate=3799.5kbits/s speed=1.01x    
  
  frame=  280 fps= 25 q=2.0 size=    5093kB time=00:00:11.20 bitrate=3722.3kbits/s speed=1.01x    
  
  frame=  291 fps= 25 q=2.0 Lsize=    5209kB time=00:00:11.68 bitrate=3650.9kbits/s speed=1.01x    
  
  video:4998kB audio:183kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.542552%
  
  Exiting normally, received signal 15.

  Pure command line for mp4 format :

  Input #0, mp3, from './stream_audio.mp3':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf57.41.100
  
    Duration: 00:00:29.59, start: 0.023021, bitrate: 128 kb/s
  
      Stream #0:0: Audio: mp3, 48000 Hz, stereo, s16p, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.48
  
  Input #1, rawvideo, from 'pipe:':
  
    Duration: N/A, start: 0.000000, bitrate: 552960 kb/s
  
      Stream #1:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1280x720, 552960 kb/s, 25 tbr, 25 tbn, 25 tbc
  
  [libx264 @ 0x75fc583aa0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
  
  [libx264 @ 0x75fc583aa0] profile High, level 3.1
  
  [libx264 @ 0x75fc583aa0] 264 - core 148 r2708 86b7198 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  [mp4 @ 0x75fc581280] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
  
      Last message repeated 1 times
  
  Output #0, mp4, to 'out.mp4':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf57.41.100
  
      Stream #0:0: Audio: aac (LC) ([64][0][0][0] / 0x0040), 48000 Hz, stereo, fltp, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.48.101 aac
  
      Stream #0:1: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=-1--1, 25 fps, 12800 tbn, 25 tbc
  
      Metadata:
  
        encoder         : Lavc57.48.101 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mp3 (native) -> aac (native))
  
    Stream #1:0 -> #0:1 (rawvideo (native) -> h264 (libx264))
  
  [rawvideo @ 0x75fc55d840] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  
  frame=   14 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   27 fps= 27 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   39 fps= 26 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   52 fps= 26 q=28.0 size=     113kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   65 fps= 26 q=28.0 size=     188kB time=00:00:00.48 bitrate=3209.4kbits/s speed=0.191x    
  
  frame=   77 fps= 26 q=28.0 size=     320kB time=00:00:01.00 bitrate=2615.3kbits/s speed=0.332x    
  
  frame=   90 fps= 26 q=28.0 size=     450kB time=00:00:01.51 bitrate=2431.3kbits/s speed=0.43x    
  
  frame=  102 fps= 25 q=28.0 size=     591kB time=00:00:01.98 bitrate=2442.1kbits/s speed=0.493x    
  
  frame=  115 fps= 25 q=28.0 size=     731kB time=00:00:02.51 bitrate=2377.6kbits/s speed=0.556x    
  
  frame=  128 fps= 25 q=28.0 size=     845kB time=00:00:03.00 bitrate=2307.7kbits/s speed=0.596x    
  
  frame=  140 fps= 25 q=28.0 size=     930kB time=00:00:03.52 bitrate=2164.6kbits/s speed=0.636x    
  
  frame=  153 fps= 25 q=28.0 size=     983kB time=00:00:04.03 bitrate=1996.7kbits/s speed=0.668x    
  
  frame=  165 fps= 25 q=28.0 size=    1063kB time=00:00:04.52 bitrate=1925.2kbits/s speed=0.691x    
  
  frame=  178 fps= 25 q=28.0 size=    1122kB time=00:00:05.03 bitrate=1826.2kbits/s speed=0.714x    
  
  frame=  191 fps= 25 q=28.0 size=    1203kB time=00:00:05.52 bitrate=1783.6kbits/s speed=0.732x    
  
  frame=  203 fps= 25 q=28.0 size=    1270kB time=00:00:06.03 bitrate=1723.7kbits/s speed=0.749x    
  
  frame=  216 fps= 25 q=28.0 size=    1339kB time=00:00:06.52 bitrate=1680.5kbits/s speed=0.763x    
  
  frame=  228 fps= 25 q=28.0 size=    1443kB time=00:00:07.06 bitrate=1674.1kbits/s speed=0.779x    
  
  frame=  241 fps= 25 q=28.0 size=    1558kB time=00:00:07.55 bitrate=1690.0kbits/s speed=0.789x    
  
  frame=  254 fps= 25 q=28.0 size=    1654kB time=00:00:08.04 bitrate=1685.0kbits/s speed=0.799x    
  
  frame=  266 fps= 25 q=28.0 size=    1738kB time=00:00:08.55 bitrate=1664.6kbits/s speed=0.809x    
  
  frame=  279 fps= 25 q=28.0 size=    1750kB time=00:00:09.06 bitrate=1581.2kbits/s speed=0.818x    
  
  frame=  291 fps= 25 q=28.0 size=    1765kB time=00:00:09.55 bitrate=1512.7kbits/s speed=0.825x    
  
  frame=  300 fps= 25 q=28.0 size=    1779kB time=00:00:11.73 bitrate=1242.1kbits/s speed=0.971x    
  
  frame=  300 fps= 24 q=-1.0 Lsize=    1917kB time=00:00:12.01 bitrate=1307.2kbits/s speed=0.955x    
  
  video:1720kB audio:188kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.488004%
  
  [aac @ 0x75fc582520] Qavg: 221.585
  
  [libx264 @ 0x75fc583aa0] frame I:2     Avg QP:15.83  size: 78710
  
  [libx264 @ 0x75fc583aa0] frame P:185   Avg QP:21.02  size:  7259
  
  [libx264 @ 0x75fc583aa0] frame B:113   Avg QP:26.59  size:  2298
  
  [libx264 @ 0x75fc583aa0] consecutive B-frames: 46.3% 10.0%  1.0% 42.7%
  
  [libx264 @ 0x75fc583aa0] mb I  I16..4: 22.1% 32.5% 45.4%
  
  [libx264 @ 0x75fc583aa0] mb P  I16..4:  0.8%  1.4%  0.2%  P16..4: 25.2%  6.3%  3.1%  0.0%  0.0%    skip:63.0%
  
  [libx264 @ 0x75fc583aa0] mb B  I16..4:  0.1%  0.2%  0.0%  B16..8: 37.3%  1.6%  0.2%  direct: 0.4%  skip:60.2%  L0:47.3% L1:49.6% BI: 3.0%
  
  [libx264 @ 0x75fc583aa0] 8x8 transform intra:50.5% inter:58.1%
  
  [libx264 @ 0x75fc583aa0] coded y,uvDC,uvAC intra: 49.4% 65.2% 22.9% inter: 7.7% 10.0% 0.5%
  
  [libx264 @ 0x75fc583aa0] i16 v,h,dc,p: 29% 32% 11% 28%
  
  [libx264 @ 0x75fc583aa0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 26% 21% 23%  4%  5%  6%  5%  5%  5%
  
  [libx264 @ 0x75fc583aa0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 15% 11%  8%  8%  9%  7% 12%  7%
  
  [libx264 @ 0x75fc583aa0] i8c dc,h,v,p: 51% 20% 21%  8%
  
  [libx264 @ 0x75fc583aa0] Weighted P-Frames: Y:0.0% UV:0.0%
  
  [libx264 @ 0x75fc583aa0] ref P L0: 79.7% 11.4%  7.7%  1.2%
  
  [libx264 @ 0x75fc583aa0] ref B L0: 96.8%  2.9%  0.4%
  
  [libx264 @ 0x75fc583aa0] ref B L1: 97.7%  2.3%
  
  [libx264 @ 0x75fc583aa0] kb/s:1173.39

• FFmpeg - resampling from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16 got very bad sound quality (slow, out of tune, noise)

  13 janvier 2015, par kaienfr

  I was confused with resampling result in new ffmpeg.
  I decode an AAC audio into PCM, the ffmpeg show audio information as :

  Stream #0:0: Audio: aac, 44100 Hz, stereo, fltp, 122 kb/s

  In new ffmpeg, the output samples are fltp format, so I have to convert it from AV_SAMPLE_FMT_FLTP to AV_SAMPLE_FMT_S16

  PS : in old ffmpeg as libavcodec 54.12.100, it is directly S16, so do not need resampling and without any sound quality problem.

  Then I’ve tried three ways to resampling,

  1. using swr_convert

  2. using avresample_convert

  3. convert manualy

  But all of them yield the same result, the sound quality is really bad, very slow and out of tune, with some noise too.

  My resampling code is as follows :

  void resampling(AVFrame* frame_, AVCodecContext* pCodecCtx, int64_t want_sample_rate, uint8_t* outbuf){
  
      SwrContext      *swrCtx_ = 0;
  
      AVAudioResampleContext *avr = 0;
  
  
  
      // Initializing the sample rate convert. We only really use it to convert float output into int.
  
      int64_t wanted_channel_layout = AV_CH_LAYOUT_STEREO;
  
  
  
  #ifdef AV_SAMPLEING
  
      avr = avresample_alloc_context();
  
      av_opt_set_int(avr, "in_channel_layout", frame_->channel_layout, 0);
  
      av_opt_set_int(avr, "out_channel_layout", wanted_channel_layout, 0);
  
      av_opt_set_int(avr, "in_sample_rate", frame_->sample_rate, 0);
  
      av_opt_set_int(avr, "out_sample_rate", 44100, 0);
  
      av_opt_set_int(avr, "in_sample_fmt", pCodecCtx->sample_fmt, 0); //AV_SAMPLE_FMT_FLTP
  
      av_opt_set_int(avr, "out_sample_fmt", AV_SAMPLE_FMT_S16, 0);
  
      av_opt_set_int(avr, "internal_sample_fmt", pCodecCtx->sample_fmt, 0);
  
      avresample_open(avr);
  
      avresample_convert(avr, &outbuf, frame_->linesize[0], frame_->nb_samples, frame_->extended_data, frame_->linesize[0], frame_->nb_samples);
  
      avresample_close(avr);
  
      return;
  
  #endif
  
  
  
  #ifdef USER_SAMPLEING
  
      if (pCodecCtx->sample_fmt == AV_SAMPLE_FMT_FLTP)
  
      {
  
              int nb_samples = frame_->nb_samples;
  
              int channels = frame_->channels;
  
              int outputBufferLen = nb_samples & channels * 2;
  
              auto outputBuffer = (int16_t*)outbuf;
  
  
  
              for (int i = 0; i < nb_samples; i++)
  
              {
  
                      for (int c = 0; c < channels; c++)
  
                      {
  
                              float* extended_data = (float*)frame_->extended_data[c];
  
                              float sample = extended_data[i];
  
                              if (sample < -1.0f) sample = -1.0f;
  
                              else if (sample > 1.0f) sample = 1.0f;
  
                              outputBuffer[i * channels + c] = (int16_t)round(sample * 32767.0f);
  
                      }
  
              }
  
              return;
  
      }
  
  #endif
  
      swrCtx_ = swr_alloc_set_opts(
  
              NULL, //swrCtx_,
  
              wanted_channel_layout,
  
              AV_SAMPLE_FMT_S16,
  
              want_sample_rate,
  
              pCodecCtx->channel_layout,
  
              pCodecCtx->sample_fmt,
  
              pCodecCtx->sample_rate,
  
              0,
  
              NULL);
  
  
  
      if (!swrCtx_ || swr_init(swrCtx_) < 0) {
  
              printf("swr_init: Failed to initialize the resampling context");
  
              return;
  
      }
  
  
  
      // convert audio to AV_SAMPLE_FMT_S16
  
      int swrRet = swr_convert(swrCtx_, &outbuf, frame_->nb_samples, (const uint8_t **)frame_->extended_data, frame_->nb_samples);
  
      if (swrRet < 0) {
  
              printf("swr_convert: Error while converting %d", swrRet);
  
              return;
  
      }
  
  }

  What should to do ?

  PS1 : playing with ffplay is just all right.

  PS2 : save resample S16 PCM into file and playing it will have the same sound quality problem.

  Thanks a lot for your help and suggestions !

  ---

  I’ve also noticed that, in old ffmpeg, aac is recongized as FLT format and directly decoded into 16-bit PCM, while in new ffmpeg, aac is counted as FLTP format and produce still 32-bit IEEE float output.

  Thus the same code will produce quite different outputs with different versions of ffmpeg.
  Then, I’d like to ask what is the right way to convert a AAC audio to 16-bit PCM in new version ?

  Thanks a lot in advance !

• example copy sound for MP4 linked files

  27 septembre 2017, par HenrykVIII

  Is it possible to copy a soundtrack to a file linked to a script by a file ?

  If so, you can ask for an example of such a slot machine.
  Files in MP4 format.