Recherche avancée

Sur d’autres sites (12017)

• ffmpeg memory leak when opening libx264 encoder

  18 octobre 2023, par ksb496

  I have spotted a memory leak issue when I use the libx264 encoder in the FFmpeg C API. Specifically, when it comes to deallocate memory after encoding a video. After tracking the factor that causes it, I realized that it happens after invoking avcodec_open2, which allocates some memory that afterwards cannot be freed. Once the video is processed, calling avcodec_close and then avcodec_free_context does not entirely free all the allocated memory.

  


  After some investigation, I found out that the problem could be located in AVCodecContext::priv_data being allocated but not being freed afterwards. In this question a solution to the issue is proposed. However, I tried to implement it without success (the memory being leaked seems to be exactly the same).

  


  As a matter of fact, the following simple code (which includes the patch that was proposed in the aforementioned question), in which the codec is being opened and closed multiple times without even writing a single frame or allocating an AVFormatContext, illustrates the memory leak.

  


  #include &#xA;extern "C"{&#xA;#include &#xA;#include <libavcodec></libavcodec>avcodec.h>&#xA;#include <libavutil></libavutil>opt.h>&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    avcodec_register_all();&#xA;&#xA;    AVCodec *codec;&#xA;    AVCodecContext *c;&#xA;    for (int n=0; n&lt;2000; n&#x2B;&#x2B;)&#xA;    {&#xA;        codec = avcodec_find_encoder_by_name("libx264");&#xA;        c=avcodec_alloc_context3(codec);&#xA;        c->pix_fmt=AV_PIX_FMT_YUV420P;&#xA;        c->width=1920;&#xA;        c->height=1080;&#xA;        c->time_base=(AVRational){1, 30};&#xA;        c->framerate=(AVRational){30, 1};&#xA;        avcodec_open2(c, codec, NULL);&#xA;        avcodec_close(c);&#xA;        av_opt_free(c->priv_data);&#xA;        av_freep(&amp;c->priv_data);&#xA;        avcodec_free_context(&amp;c);&#xA;    }&#xA;    return 0;&#xA;}&#xA;

  


  It must be remarked that if the line codec = avcodec_find_encoder_by_name("libx264") is replaced to an invocation to an internal/native encoder, e.g., codec = avcodec_find_encoder(AV_CODEC_ID_MPEG4), then the memory leak issue completely disappears. Hence, it certainly seems to be an issue related to some private data of the external encoder not being properly freed.

  


  It is also worth mentioning that I am using an old version of ffmpeg and libx264. To be more precise, ffmpeg version 2.8git and libx264 version 0.136.x. For technical reasons that are beyond the scope of this question, it is not possible to upgrade the libraries to newer versions onto the project in which these are being used. I am fully aware that most of the involved ffmpeg/libx264 code has been probably changed along the years and many functions became deprecated or fixed, and thus reporting this as a possible bug in the ffmpeg developer's mailbox is out of the question.

  


  Nevertheless, I am still asking this here because I would like to know whether it is just some mistake on my end and/or something I am not taking into account when it comes to free all the memory relative to an external encoder (best case scenario). Otherwise, I would like to know whether there can be some reasonably cheap solution through some custom code or function that can be implemented as a patch (assuming it is indeed an issue related to ffmpeg/libx264), no matter if it makes the whole deallocation code less elegant or concise. If someone is still working on these older versions of ffmpeg and can come up with a workaround, that would be highly appreciated.

  


• ffmpeg memory leak when opening libx264 encoder

  18 octobre 2023, par ksb496

  I have spotted a memory leak issue when I use the libx264 encoder in the FFmpeg C API. Specifically, when it comes to deallocate memory after encoding a video. After tracking the factor that causes it, I realized that it happens after invoking avcodec_open2, which allocates some memory that afterwards cannot be freed. Once the video is processed, calling avcodec_close and then avcodec_free_context does not entirely free all the allocated memory.

  


  After some investigation, I found out that the problem could be located in AVCodecContext::priv_data being allocated but not being freed afterwards. In this question a solution to the issue is proposed. However, I tried to implement it without success (the memory being leaked seems to be exactly the same).

  


  As a matter of fact, the following simple code (which includes the patch that was proposed in the aforementioned question), in which the codec is being opened and closed multiple times without even writing a single frame or allocating an AVFormatContext, illustrates the memory leak.

  


  #include &#xA;extern "C"{&#xA;#include &#xA;#include <libavcodec></libavcodec>avcodec.h>&#xA;#include <libavutil></libavutil>opt.h>&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    avcodec_register_all();&#xA;&#xA;    AVCodec *codec;&#xA;    AVCodecContext *c;&#xA;    for (int n=0; n&lt;2000; n&#x2B;&#x2B;)&#xA;    {&#xA;        codec = avcodec_find_encoder_by_name("libx264");&#xA;        c=avcodec_alloc_context3(codec);&#xA;        c->pix_fmt=AV_PIX_FMT_YUV420P;&#xA;        c->width=1920;&#xA;        c->height=1080;&#xA;        c->time_base=(AVRational){1, 30};&#xA;        c->framerate=(AVRational){30, 1};&#xA;        avcodec_open2(c, codec, NULL);&#xA;        avcodec_close(c);&#xA;        av_opt_free(c->priv_data);&#xA;        av_freep(&amp;c->priv_data);&#xA;        avcodec_free_context(&amp;c);&#xA;    }&#xA;    return 0;&#xA;}&#xA;

  


  It must be remarked that if the line codec = avcodec_find_encoder_by_name("libx264") is replaced to an invocation to an internal/native encoder, e.g., codec = avcodec_find_encoder(AV_CODEC_ID_MPEG4), then the memory leak issue completely disappears. Hence, it certainly seems to be an issue related to some private data of the external encoder not being properly freed.

  


  It is also worth mentioning that I am using an old version of ffmpeg and libx264. To be more precise, ffmpeg version 2.8git and libx264 version 0.136.x. For technical reasons that are beyond the scope of this question, it is not possible to upgrade the libraries to newer versions onto the project in which these are being used. I am fully aware that most of the involved ffmpeg/libx264 code has been probably changed along the years and many functions became deprecated or fixed, and thus reporting this as a possible bug in the ffmpeg developer's mailbox is out of the question.

  


  Nevertheless, I am still asking this here because I would like to know whether it is just some mistake on my end and/or something I am not taking into account when it comes to free all the memory relative to an external encoder (best case scenario). Otherwise, I would like to know whether there can be some reasonably cheap solution through some custom code or function that can be implemented as a patch (assuming it is indeed an issue related to ffmpeg/libx264), no matter if it makes the whole deallocation code less elegant or concise. If someone is still working on these older versions of ffmpeg and can come up with a workaround, that would be highly appreciated.

  


• FFMPEG : RTSP re-stream dies randomly

  14 mai 2018, par stevendesu

  I have a security camera streaming RTSP, and I wish to re-stream this to an RTMP ingest server. For now I’m using my laptop as an ffmpeg proxy, but eventually I’ll use a raspberry pi or something similar (cheap/small)

  Here’s the command I’m using (pretty simple) :

  ffmpeg -i rtsp://@10.0.0.16:554/1/h264major -c:v libx264 -c:a none -f flv rtmp://output/camera_stream

  This works but after a minute or two the stream dies. Here’s the output :

  ffmpeg version N-90057-g7c82e0f Copyright (c) 2000-2018 the FFmpeg developers
  
    built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.6) 20160609
  
    configuration: --prefix=/home/sbarnett/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/sbarnett/ffmpeg_build/include --extra-ldflags=-L/home/sbarnett/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/sbarnett/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libspeex --enable-nonfree
  
    libavutil      56.  7.101 / 56.  7.101
  
    libavcodec     58. 11.101 / 58. 11.101
  
    libavformat    58.  9.100 / 58.  9.100
  
    libavdevice    58.  1.100 / 58.  1.100
  
    libavfilter     7. 12.100 /  7. 12.100
  
    libswscale      5.  0.101 /  5.  0.101
  
    libswresample   3.  0.101 /  3.  0.101
  
    libpostproc    55.  0.100 / 55.  0.100
  
  Input #0, rtsp, from 'rtsp://@10.0.0.16:554/1/h264major':
  
    Metadata:
  
      title           : h264major
  
      comment         : h264major
  
    Duration: N/A, start: 0.360000, bitrate: N/A
  
      Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 720x480, 25 fps, 25 tbr, 90k tbn, 50 tbc
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  
  Press [q] to stop, [?] for help
  
  [libx264 @ 0x38843c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
  
  [libx264 @ 0x38843c0] profile High, level 3.0
  
  [libx264 @ 0x38843c0] 264 - core 155 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  Output #0, flv, to 'rtmp://output/camera_stream':
  
    Metadata:
  
      title           : h264major
  
      comment         : h264major
  
      encoder         : Lavf58.9.100
  
      Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuvj420p(pc), 720x480, q=-1--1, 25 fps, 1k tbn, 25 tbc
  
      Metadata:
  
        encoder         : Lavc58.11.101 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
  Past duration 0.999992 too large
  
      Last message repeated 29 times
  
  [rtsp @ 0x3847600] max delay reached. need to consume packet
  
  [rtsp @ 0x3847600] RTP: missed 48 packets
  
  Past duration 0.999992 too large
  
      Last message repeated 4 times
  
  frame=   44 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A dup=0 drop=5 speed=   0x    
  
  frame=   57 fps= 54 q=28.0 size=      43kB time=00:00:00.16 bitrate=2186.4kbits/s dup=0 drop=5 speed=0.153x    
  
  ... (lots of similar messages) ...   
  
  frame= 1163 fps= 26 q=28.0 size=    1341kB time=00:00:44.84 bitrate= 245.0kbits/s dup=0 drop=5 speed=0.99x    
  
  frame= 1177 fps= 26 q=28.0 size=    1353kB time=00:00:45.40 bitrate= 244.2kbits/s dup=0 drop=5 speed=0.99x    
  
  [rtsp @ 0x3847600] max delay reached. need to consume packet
  
  [rtsp @ 0x3847600] RTP: missed 2 packets
  
  frame= 1190 fps= 26 q=28.0 size=    1370kB time=00:00:45.92 bitrate= 244.4kbits/s dup=0 drop=5 speed=0.99x    
  
  [h264 @ 0x38c08c0] Increasing reorder buffer to 1
  
  frame= 1201 fps= 26 q=28.0 size=    1381kB time=00:00:46.36 bitrate= 244.0kbits/s dup=0 drop=5 speed=0.989x    
  
  frame= 1214 fps= 26 q=28.0 size=    1393kB time=00:00:46.88 bitrate= 243.4kbits/s dup=0 drop=5 speed=0.989x    
  
  ... (lots of similar messages) ...    
  
  frame= 1761 fps= 25 q=28.0 size=    2030kB time=00:01:08.80 bitrate= 241.7kbits/s dup=0 drop=5 speed=0.993x    
  
  frame= 1774 fps= 25 q=28.0 size=    2041kB time=00:01:09.32 bitrate= 241.2kbits/s dup=0 drop=5 speed=0.993x    
  
  [flv @ 0x3884900] Failed to update header with correct duration.
  
  [flv @ 0x3884900] Failed to update header with correct filesize.
  
  frame= 1782 fps= 25 q=-1.0 Lsize=    2127kB time=00:01:11.64 bitrate= 243.2kbits/s dup=0 drop=5 speed=1.02x    
  
  video:2092kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.679417%
  
  [libx264 @ 0x38843c0] frame I:8     Avg QP:16.89  size: 42446
  
  [libx264 @ 0x38843c0] frame P:1672  Avg QP:19.54  size:  1065
  
  [libx264 @ 0x38843c0] frame B:102   Avg QP:23.00  size:   205
  
  [libx264 @ 0x38843c0] consecutive B-frames: 92.4%  0.0%  0.0%  7.6%
  
  [libx264 @ 0x38843c0] mb I  I16..4: 12.9% 36.2% 50.9%
  
  [libx264 @ 0x38843c0] mb P  I16..4:  0.2%  0.2%  0.0%  P16..4: 16.7%  0.7%  1.0%  0.0%  0.0%    skip:81.1%
  
  [libx264 @ 0x38843c0] mb B  I16..4:  0.1%  0.1%  0.0%  B16..8: 11.7%  0.1%  0.0%  direct: 1.5%  skip:86.5%  L0:62.2% L1:35.3% BI: 2.5%
  
  [libx264 @ 0x38843c0] 8x8 transform intra:40.8% inter:47.4%
  
  [libx264 @ 0x38843c0] coded y,uvDC,uvAC intra: 46.5% 53.0% 17.2% inter: 3.9% 8.7% 0.0%
  
  [libx264 @ 0x38843c0] i16 v,h,dc,p: 21% 56%  8% 15%
  
  [libx264 @ 0x38843c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 33% 31%  1%  2%  3%  2%  2%  3%
  
  [libx264 @ 0x38843c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 39%  9%  3%  3%  4%  5%  3%  8%
  
  [libx264 @ 0x38843c0] i8c dc,h,v,p: 43% 33% 21%  3%
  
  [libx264 @ 0x38843c0] Weighted P-Frames: Y:0.0% UV:0.0%
  
  [libx264 @ 0x38843c0] ref P L0: 88.0%  1.4%  6.6%  4.0%
  
  [libx264 @ 0x38843c0] ref B L0: 99.4%  0.5%  0.1%
  
  [libx264 @ 0x38843c0] ref B L1: 99.4%  0.6%
  
  [libx264 @ 0x38843c0] kb/s:238.73

  The camera is pretty cheap (from China) so it’s likely I’m getting bad data from it or it’s cutting out for a few seconds at a time. Ideally I would need ffmpeg to handle this well (ignore bad data, wait as long as necessary for good data to resume encoding)

  Using ffplay to check out the RTSP stream, I get output like the following :

  $> ffplay -i rtsp://@10.0.0.16:554/1/h264major
  
  ffplay version N-90057-g7c82e0f Copyright (c) 2003-2018 the FFmpeg developers
  
    built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.6) 20160609
  
    configuration: --prefix=/home/sbarnett/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/sbarnett/ffmpeg_build/include --extra-ldflags=-L/home/sbarnett/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/sbarnett/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libspeex --enable-nonfree
  
    libavutil      56.  7.101 / 56.  7.101
  
    libavcodec     58. 11.101 / 58. 11.101
  
    libavformat    58.  9.100 / 58.  9.100
  
    libavdevice    58.  1.100 / 58.  1.100
  
    libavfilter     7. 12.100 /  7. 12.100
  
    libswscale      5.  0.101 /  5.  0.101
  
    libswresample   3.  0.101 /  3.  0.101
  
    libpostproc    55.  0.100 / 55.  0.100
  
  Input #0, rtsp, from 'rtsp://@10.0.0.16:554/1/h264major':0B f=0/0
  
    Metadata:
  
      title           : h264major
  
      comment         : h264major
  
    Duration: N/A, start: 0.320000, bitrate: N/A
  
      Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 720x480, 25 fps, 25 tbr, 90k tbn, 50 tbc
  
  [swscaler @ 0x7f6bbc093180] deprecated pixel format used, make sure you did set range correctly
  
  [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
  
  [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets
  
  [h264 @ 0x7f6bc0041080] error while decoding MB 44 28, bytestream -37
  
  [h264 @ 0x7f6bc0041080] concealing 95 DC, 95 AC, 95 MV errors in I frame
  
  [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
  
  [rtsp @ 0x7f6bc0000940] RTP: missed 1 packets
  
  [h264 @ 0x7f6bc0041080] error while decoding MB 43 29, bytestream -49
  
  [h264 @ 0x7f6bc0041080] concealing 51 DC, 51 AC, 51 MV errors in I frame
  
  [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
  
  [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets
  
  [h264 @ 0x7f6bc0041080] Increasing reorder buffer to 1
  
  [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
  
  [rtsp @ 0x7f6bc0000940] RTP: missed 3 packets
  
  [h264 @ 0x7f6bc02c3600] error while decoding MB 27 29, bytestream -24
  
  [h264 @ 0x7f6bc02c3600] concealing 67 DC, 67 AC, 67 MV errors in I frame
  
  [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
  
  [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets
  
  [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
  
  [rtsp @ 0x7f6bc0000940] RTP: missed 42 packets
  
  [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
  
  [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets

  Then eventually the video just freezes. The first time it froze after around 5 minutes, but I wasn’t able to say definitively if it froze the instant 44 packets were dropped or if it froze randomly later. So the second time I stared intently.... for 21 minutes. Then I got bored of it not freezing, turned to pet my cat, and when I looked back 15 seconds later it was frozen. I think it only breaks when no one is watching it.

  What I can say definitively is :

  • While running normally, M-V hovers around 0 (anywhere between -0.01 and +0.01)
  • Once frozen, M-V begins to count down into negative numbers without stopping - although at a rate slower than -1 per second
  • While running normally, aq is 0KB and vq is a positive number (I think it was 30KB or so ?)
  • Once frozen, vq is also 0KB

  It’s a really cheap camera with a crummy power supply that goes out if you breathe on it, so it’s likely the camera is going temporarily offline during this time — but I’d like ffmpeg to wait out a timeout and resume streaming when it sees the camera again.