Recherche avancée

Sur d’autres sites (6011)

• How to fix laggy ffmpeg screen and audio capture ?

  26 juillet 2022, par Wh0r00t

  I am using ffmpeg to capture the screen along with audio.

  


  The ffmpeg command that i tried is

  


  ffmpeg -y \
    -f x11grab \
    -framerate 60 \
    -s 1366x768 \
    -i :0.0 \
    -f alsa -i default -ac 2 \
    -r 30 \
    -c:v h264 -crf 0 -preset ultrafast -c:a vorbis -strict experimental  \
    "$HOME/Videos/$fname-$(date '+%y%m%d-%H%M-%S').mkv"


  


  The stdout of the ffmpeg https://pastebin.com/Qmi5TMKv

  


  ffmpeg version n5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
[x11grab @ 0x561faf77eb00] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, x11grab, from ':0.0':
  Duration: N/A, start: 1658814267.169414, bitrate: 2014248 kb/s
  Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1366x768, 2014248 kb/s, 60 fps, 1000k tbr, 1000k tbn
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'default':
  Duration: N/A, start: 1658814267.230653, bitrate: 1536 kb/s
  Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> vorbis (native))
Press [q] to stop, [?] for help
[libx264 @ 0x561faf7d4300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA3 BMI1
[libx264 @ 0x561faf7d4300] profile High 4:4:4 Predictive, level 3.2, 4:4:4, 8-bit
[libx264 @ 0x561faf7d4300] 264 - core 164 r3081 19856cc - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=cqp mbtree=0 qp=0
[alsa @ 0x561faf78a940] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
Output #0, matroska, to '/home/earth/Videos/-220726-1114-27.mkv':
  Metadata:
    encoder         : Lavf59.16.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuv444p(tv, progressive), 1366x768, q=2-31, 30 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.18.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: vorbis (oV[0][0] / 0x566F), 48000 Hz, stereo, fltp
    Metadata:
      encoder         : Lavc59.18.100 vorbis
[vorbis @ 0x561faf7d5500] Queue input is backward in time0 bitrate=N/A speed=   0x
frame=  153 fps= 31 q=-1.0 Lsize=    2295kB time=00:00:05.06 bitrate=3709.5kbits/s dup=0 drop=150 speed=1.01x
video:2282kB audio:7kB subtitle:0kB other streams:0kB global headers:3kB muxing overhead: 0.281689%
[libx264 @ 0x561faf7d4300] frame I:1     Avg QP: 0.00  size:381729
[libx264 @ 0x561faf7d4300] frame P:152   Avg QP: 0.00  size: 12857
[libx264 @ 0x561faf7d4300] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x561faf7d4300] mb P  I16..4: 56.3%  0.0%  0.0%  P16..4:  0.1%  0.0%  0.0%  0.0%  0.0%    skip:43.6%
[libx264 @ 0x561faf7d4300] coded y,u,v intra: 1.6% 1.6% 1.6% inter: 0.2% 0.2% 0.2%
[libx264 @ 0x561faf7d4300] i16 v,h,dc,p: 99%  1%  0%  0%
[libx264 @ 0x561faf7d4300] kb/s:3664.27
Exiting normally, received signal 15.


  


  I am using the preset ultrafast because I read that it helps not to compress the video too much.
The output of the recorded test file using ffmpeg is as below.

  


   (+) Video --vid=1 (h264 1366x768 30.000fps)
 (+) Audio --aid=1 (vorbis 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1366x768 yuv444p
AV: 00:00:03 / 00:00:19 (17%) A-V:  0.000
[mkv] Discarding potentially broken or useless index.
AV: 00:00:14 / 00:00:19 (73%) A-V:  0.000

Exiting... (Quit)


  


  The recording works but there is a audio lag. If I record the same using simplescreenrecorder with the same settings like,

  


  audio backend - alsa

  


  source - default

  


  audio codec - vorbis

  


  video codec - h.264

  


  container - matroska

  


  preset - superfast

  


  The simplescreenrecorder log https://pastebin.com/83hMMRQF

  


  [PageRecord::StartPage] Starting page ...
[PageRecord::StartPage] Started page.
[PageRecord::StartOutput] Starting output ...
[PageRecord::StartOutput] Output file: /home/earth/Videos/simplescreenrecorder-2022-07-26_11.18.13.mkv
[Muxer::Init] Using format matroska (Matroska).
[Muxer::AddStream] Using codec libx264 (libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10).
[VideoEncoder::PrepareStream] Using pixel format nv12.
[libx264 @ 0x563436cbfd40] using SAR=1/1
[libx264 @ 0x563436cbfd40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA3 BMI1
[libx264 @ 0x563436cbfd40] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 0x563436cbfd40] 264 - core 164 r3081 19856cc - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00
[Muxer::AddStream] Using codec libvorbis (libvorbis).
[BaseEncoder::EncoderThread] Encoder thread started.
[AudioEncoder::PrepareStream] Using sample format f32p.
[BaseEncoder::EncoderThread] Encoder thread started.
[Muxer::MuxerThread] Muxer thread started.
[PageRecord::StartOutput] Started output.
[Synchronizer::SynchronizerThread] Synchronizer thread started.
[PageRecord::StartInput] Starting input ...
[X11Input::Init] Using X11 shared memory.
[X11Input::Init] Detecting screen configuration ...
[X11Input::Init] Screen 0: x1 = 0, y1 = 0, x2 = 1366, y2 = 768
[X11Input::InputThread] Input thread started.
[ALSAInput::InputThread] Using sample format s16.
[PageRecord::StartInput] Started input.
[ALSAInput::InputThread] Input thread started.
[FastResampler::Resample] Resample ratio is 1.0000 (was 0.0000).
[PageRecord::StopOutput] Stopping output ...
[PageRecord::StopOutput] Stopped output.
[PageRecord::StopInput] Stopping input ...
[X11Input::~X11Input] Stopping input thread ...
[X11Input::InputThread] Input thread stopped.
[ALSAInput::~ALSAInput] Stopping input thread ...
[ALSAInput::InputThread] Input thread stopped.
[PageRecord::StopInput] Stopped input.


  


  It works perfectly without any lag whatsoever. The output of the recorded test file using simplescreenrecorder is as below.

  


   (+) Video --vid=1 (h264 1366x768)
 (+) Audio --aid=1 (vorbis 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1366x768 yuv420p
AV: 00:00:01 / 00:00:17 (7%) A-V:  0.000
[mkv] Discarding potentially broken or useless index.
AV: 00:00:08 / 00:00:17 (47%) A-V:  0.000

Exiting... (Quit)


  


  The only difference that I saw between these two recordings is VO: [gpu] 1366x768 yuv444p
VO: [gpu] 1366x768 yuv420p for ffmpeg and simplescreenrecorder receptively.
I do not know if this matters but is there something that I could tweak to make ffmpeg to capture the screen and audio without any lag.
Like answered here https://unix.stackexchange.com/questions/675436/ffmpeg-recording-slows-down-when-audio-inputs-are-added
I do open pavucontrol but its not much of a help.

  


  The reason that I going with ffmpeg is because I can kill the process using pid at a particular time using cronjobs.
These are my system information, in case if it helps

  


  System:
  Host: taco Kernel: 5.18.12-arch1-1 arch: x86_64 bits: 64 Desktop: dwm
    v: 6.2 Distro: Arch Linux
Machine:
  Type: Desktop Mobo: Acer model: A75F2-M v: P21-A1 serial: N/A BIOS: Acer
    v: P21-A1 date: 02/07/2014
CPU:
  Info: quad core model: AMD A8-5500B APU with Radeon HD Graphics bits: 64
    type: MT MCP cache: L2: 4 MiB
  Speed (MHz): avg: 1400 min/max: 1400/3200 cores: 1: 1400 2: 1400 3: 1400
    4: 1400
Graphics:
  Device-1: AMD Trinity [Radeon HD 7560D] driver: radeon v: kernel
  Display: server: X.Org v: 21.1.4 driver: X: loaded: modesetting
    gpu: radeon resolution: 1366x768~60Hz
  OpenGL: renderer: AMD ARUBA (DRM 2.50.0 / 5.18.12-arch1-1 LLVM 14.0.6)
    v: 4.3 Mesa 22.1.3
Audio:
  Device-1: AMD FCH Azalia driver: snd_hda_intel
  Sound Server-1: ALSA v: k5.18.12-arch1-1 running: yes
  Sound Server-2: PulseAudio v: 16.1 running: yes
  Sound Server-3: PipeWire v: 0.3.56 running: yes


  


  Any help is much appreciated.

  


• Dash output with ffmpeg not producing durations specified with -seg_duration

  30 juillet 2022, par Codie

  There is a .mp4 file of 35 MB and 51 seconds. I have to create 51 chunks, each corresponding to 1 second with a size of less than 1MB (the total size should be almost the same as the original file). Please note that I have to implement lossless converting.

  


  I've tried many times, but it just produces about 10 files above 10 MB.

  


  ---

  


  Command :

  


  ffmpeg -re -i input.mp4 -map 0:v -c:v libx264 -crf 0 -bf 1 -keyint_min 120 -g 120 -sc_threshold 0 -b_strategy 0 -use_template 1 -seg_duration 1 -window_size 60 -adaptation_sets "id=0,streams=v id=1,streams=a" -f dash ./dashTest/out.mpd


  


  ---

  


  Command line log :

  


  ffmpeg version 5.1-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41
    creation_time   : 2022-07-27T09:13:31.000000Z
  Duration: 00:00:50.03, start: 0.000000, bitrate: 5716 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 5396 kb/s, 25 fps, 25 tbr, 25k tbn (default)
    Metadata:
      creation_time   : 2022-07-27T09:13:31.000000Z
      handler_name    : ?Mainconcept Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      creation_time   : 2022-07-27T09:13:31.000000Z
      handler_name    : #Mainconcept MP4 Sound Media Handler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 000001d13acb0380] using SAR=1/1
[libx264 @ 000001d13acb0380] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 000001d13acb0380] profile High 4:4:4 Predictive, level 3.1, 4:2:0, 8-bit
[libx264 @ 000001d13acb0380] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=0 mixed_ref=1 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=120 keyint_min=61 scenecut=0 intra_refresh=0 rc=cqp mbtree=0 qp=0
[dash @ 000001d13a2a4680] No bit rate set for stream 0
[dash @ 000001d13a2a4680] Opening './dashTest/init-stream0.m4s' for writing
Output #0, dash, to './dashTest/out.mpd':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41
    encoder         : Lavf59.27.100
  Stream #0:0(eng): Video: h264, yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 12800 tbn (default)
    Metadata:
      creation_time   : 2022-07-27T09:13:31.000000Z
      handler_name    : ?Mainconcept Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc59.37.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00001.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.849x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00002.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.918x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00003.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.942x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00004.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.957x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00005.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.964x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00006.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.971x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00007.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.975x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00008.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.978x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00009.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.981x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00010.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.982x
[dash @ 000001d13a2a4680] Opening './dashTest/chunk-stream0-00011.m4s.tmp' for writing
[dash @ 000001d13a2a4680] Opening './dashTest/out.mpd.tmp' for writing0.983x
frame= 1250 fps= 25 q=-1.0 Lsize=N/A time=00:00:49.96 bitrate=N/A speed=0.992x
video:171641kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 000001d13acb0380] frame I:11    Avg QP: 0.00  size:255122
[libx264 @ 000001d13acb0380] frame P:1239  Avg QP: 0.00  size:139591
[libx264 @ 000001d13acb0380] mb I  I16..4: 52.8%  8.8% 38.4%
[libx264 @ 000001d13acb0380] mb P  I16..4:  3.9%  0.7%  1.3%  P16..4: 28.0% 13.9% 11.3%  0.0%  0.0%    skip:40.8%
[libx264 @ 000001d13acb0380] 8x8 transform intra:11.9% inter:33.4%
[libx264 @ 000001d13acb0380] coded y,uvDC,uvAC intra: 68.6% 80.1% 78.9% inter: 38.0% 47.1% 46.5%
[libx264 @ 000001d13acb0380] i16 v,h,dc,p: 65% 28%  5%  2%
[libx264 @ 000001d13acb0380] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33% 33% 28%  2%  1%  1%  1%  1%  1%
[libx264 @ 000001d13acb0380] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 36% 36%  7%  3%  5%  3%  4%  3%  2%
[libx264 @ 000001d13acb0380] i8c dc,h,v,p: 17% 38% 44%  1%
[libx264 @ 000001d13acb0380] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 000001d13acb0380] ref P L0: 86.4%  7.6%  4.6%  1.4%
[libx264 @ 000001d13acb0380] kb/s:28121.58


  


  ---

  


  .mpd file :

  


  &lt;?xml version="1.0" encoding="utf-8"?>&#xA;<mpd xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" mediapresentationduration="PT50.0S" maxsegmentduration="PT1.0S" minbuffertime="PT9.6S">&#xA;    <programinformation>&#xA;    </programinformation>&#xA;    <servicedescription>&#xA;    </servicedescription>&#xA;    <period start="PT0.0S">&#xA;        <adaptationset contenttype="video" startwithsap="1" segmentalignment="true" bitstreamswitching="true" framerate="25/1" maxwidth="1280" maxheight="720" par="16:9" lang="eng">&#xA;            <representation mimetype="video/mp4" codecs="avc1.f4001f" bandwidth="28122926" width="1280" height="720" sar="1:1">&#xA;                <segmenttemplate timescale="12800" initialization="init-stream$RepresentationID$.m4s" media="chunk-stream$RepresentationID$-$Number%05d$.m4s" startnumber="1">&#xA;                    <segmenttimeline>&#xA;                        <s t="0" d="61440" r="9"></s>&#xA;                        <s d="25600"></s>&#xA;                    </segmenttimeline>&#xA;                </segmenttemplate>&#xA;            </representation>&#xA;        </adaptationset>&#xA;        <adaptationset contenttype="audio" startwithsap="1" segmentalignment="true" bitstreamswitching="true">&#xA;        </adaptationset>&#xA;    </period>&#xA;</mpd>&#xA;

  


  ---

  


  Please, if you want to put a negative point, mention your reason in the comments !

  


  ---

  


• FFmpeg saturates memory + CPU usage drops to 0% during very basic conversion of PNG files to MP4 video

  7 août 2022, par mattze_frisch

  I have this Python function that runs ffmpeg with minimal options from the Windows command line :

  


  def run_ffmpeg(frames_path, ffmpeg_path=notebook_directory):
    '''
    This function runs ffmpeg.exe to convert PNG image files into a MP4 video.
    
    Parameters
    ----------
    frames_path : string
        Absolute path to the PNG files
    ffmpeg_path : string
        Absolute path to the FFmpeg executable (ffmpeg.exe)
    '''
    
    from subprocess import check_call
    
    
    check_call(
        [
            os.path.join(ffmpeg_path, 'ffmpeg'),
            '-y',    # Overwrite output files without asking
            '-report',    # Write logfile to current working directory
            '-framerate', '60',    # Input frame rate
            '-i', os.path.join(frames_path, 'frame%05d.png'),    # Path to input frames
            os.path.join(frames_path, 'video.mp4')    # Path to store output video
        ]
    )


  


  When running it from a Jupyter notebook over 2500 PNG files (RGBA, ca. 600-700 kB each, 9000 x 13934 pixels), CPU usage briefly peaks to 100% before dropping to 0%, while memory usage quickly saturates to 100% and stays there, slowing the system down almost to a freeze, so I need to terminate ffmpeg from the task manager :

  


  Screenshot

  


  The generated video file has a size of only 48 bytes and contains just a black frame when viewed in the VLC player.

  


  This is the ffmpeg log output :

  


  ffmpeg started on 2022-08-05 at 17:17:55
Report written to "ffmpeg-20220805-171755.log"
Log level: 48
Command line:
"C:\\Users\\Username\\Desktop\\folder\\ffmpeg" -y -report -framerate 60 -i "C:\\Users\\Username\\Desktop\\e\\frame%05d.png" "C:\\Users\\Username\\Desktop\\e\\video.mp4"
ffmpeg version 2022-07-14-git-882aac99d2-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --ena  libavutil      57. 29.100 / 57. 29.100
  libavcodec     59. 38.100 / 59. 38.100
  libavformat    59. 28.100 / 59. 28.100
  libavdevice    59.  8.100 / 59.  8.100
  libavfilter     8. 45.100 /  8. 45.100
  libswscale      6.  8.100 /  6.  8.100
  libswresample   4.  8.100 /  4.  8.100
  libpostproc    56.  7.100 / 56.  7.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-framerate' ... matched as AVOption 'framerate' with argument '60'.
Reading option '-i' ... matched as input url with argument 'C:\Users\Username\Desktop\e\frame%05d.png'.
Reading option 'C:\Users\Username\Desktop\e\video.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url C:\Users\Username\Desktop\e\frame%05d.png.
Successfully parsed a group of options.
Opening an input file: C:\Users\Username\Desktop\e\frame%05d.png.
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00000.png' for reading
[file @ 0000000000425680] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042d800] Statistics: 668318 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00001.png' for reading
[file @ 000000000042dac0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042d6c0] Statistics: 668371 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00002.png' for reading
[file @ 000000000042d6c0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042dac0] Statistics: 669177 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00003.png' for reading
[file @ 000000000042dac0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437a40] Statistics: 684594 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00004.png' for reading
[file @ 0000000000437a40] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437c00] Statistics: 703014 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00005.png' for reading
[file @ 0000000000437c00] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437d00] Statistics: 721604 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00006.png' for reading
[file @ 0000000000437cc0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000437f40] Statistics: 739761 bytes read, 0 seeks
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00007.png' for reading
[file @ 0000000000437f40] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000000000438040] Statistics: 757327 bytes read, 0 seeks
[image2 @ 000000000041ff80] Probe buffer size limit of 5000000 bytes reached
Input #0, image2, from 'C:\Users\Username\Desktop\e\frame%05d.png':
  Duration: 00:00:41.67, start: 0.000000, bitrate: N/A
  Stream #0:0, 8, 1/60: Video: png, rgba(pc), 9000x13934 [SAR 29528:29528 DAR 4500:6967], 60 fps, 60 tbr, 60 tbn
Successfully opened the file.
Parsing a group of options: output url C:\Users\Username\Desktop\e\video.mp4.
Successfully parsed a group of options.
Opening an output file: C:\Users\Username\Desktop\e\video.mp4.
[file @ 000000002081e3c0] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
detected 12 logical cores
Stream mapping:
  Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00008.png' for reading
[file @ 00000000024ad980] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 00000000004379c0] Statistics: 767857 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00009.png' for reading
[file @ 000000000042d600] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 00000000004379c0] Statistics: 774848 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00010.png' for reading
[file @ 00000000004379c0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042da00] Statistics: 787178 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00011.png' for reading
[file @ 00000000004379c0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042da00] Statistics: 797084 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00012.png' for reading
[file @ 0000000000437a80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000042da00] Statistics: 802870 bytes read, 0 seeks
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'video_size' to value '9000x13934'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'pix_fmt' to value '26'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'time_base' to value '1/60'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'pixel_aspect' to value '29528/29528'
[graph 0 input from stream 0:0 @ 00000000208bf800] Setting 'frame_rate' to value '60/1'
[graph 0 input from stream 0:0 @ 00000000208bf800] w:9000 h:13934 pixfmt:rgba tb:1/60 fr:60/1 sar:29528/29528
[format @ 00000000025ef840] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16|nv21|yuv420p10le|yuv422p10le|yuv444p10le|nv20le|gray|gray10le'
[auto_scale_0 @ 00000000025efe40] w:iw h:ih flags:'' interl:0
[format @ 00000000025ef840] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 000000000042da00] query_formats: 4 queried, 3 merged, 1 already done, 0 delayed
[auto_scale_0 @ 00000000025efe40] picking yuv444p out of 13 ref:rgba alpha:1
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[auto_scale_0 @ 00000000025efe40] w:9000 h:13934 fmt:rgba sar:29528/29528 -> w:9000 h:13934 fmt:yuv444p sar:1/1 flags:0x0
[libx264 @ 000000002081d280] using mv_range_thread = 376
[libx264 @ 000000002081d280] using SAR=1/1
[libx264 @ 000000002081d280] frame MB size (563x871) > level limit (139264)
[libx264 @ 000000002081d280] DPB size (4 frames, 1961492 mbs) > level limit (1 frames, 696320 mbs)
[libx264 @ 000000002081d280] MB rate (29422380) > level limit (16711680)
[libx264 @ 000000002081d280] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 000000002081d280] profile High 4:4:4 Predictive, level 6.2, 4:4:4, 8-bit
[libx264 @ 000000002081d280] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'C:\Users\Username\Desktop\e\video.mp4':
  Metadata:
    encoder         : Lavf59.28.100
  Stream #0:0, 0, 1/15360: Video: h264 (avc1 / 0x31637661), yuv444p(tv, progressive), 9000x13934 [SAR 1:1 DAR 4500:6967], q=2-31, 60 fps, 15360 tbn
    Metadata:
      encoder         : Lavc59.38.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Clipping frame in rate conversion by 0.000008
frame=    1 fps=0.8 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00013.png' for reading
[file @ 000000000a6a2180] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 810395 bytes read, 0 seeks
frame=    2 fps=0.8 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00014.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 818213 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00015.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 817936 bytes read, 0 seeks
frame=    4 fps=1.2 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00016.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 817014 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00017.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 828088 bytes read, 0 seeks
frame=    6 fps=1.5 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00018.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 831007 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00019.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 845203 bytes read, 0 seeks
frame=    8 fps=1.7 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00020.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 851548 bytes read, 0 seeks
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00021.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 847629 bytes read, 0 seeks
frame=   10 fps=1.8 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00022.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 860169 bytes read, 0 seeks
frame=   11 fps=1.4 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00023.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 857243 bytes read, 0 seeks
frame=   12 fps=1.2 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[image2 @ 000000000041ff80] Opening 'C:\Users\Username\Desktop\e\frame00024.png' for reading
[file @ 000000001ec86c80] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 000000000b38de80] Statistics: 835155 bytes read, 0 seeks


  


  What is the problem ?

  


  By the way, the color model of the image files was confirmed by doing

  


  from PIL import Image


img = Image.open('C:\\Users\\EPI-SMLM\\Desktop\\e\\frame00000.png')
img.mode
-------------------------------------------------------------------
C:\Program Files\Python38\lib\site-packages\PIL\Image.py:3035: DecompressionBombWarning: Image size (125406000 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
  warnings.warn(

'RGBA'


  


  The "decompression bomb warning" appears to be a false alarm/bug.

  


  UPDATE : I can confirm that this happens even when there are only 50 image files, i.e. 50 x 700 kB = 35 MB in total size. ffmpeg still gobbles up all available memory (almost 60 GB of private bytes !!!).

  


  And it also happens if ffmpeg is run from the command line.

  


  This must be a bug !