Sur d’autres sites (13133)

• How does FFmpeg determine the dispositions of an MP4 track ?

  5 mars, par obskyr

  The Issue

  


  FFmpeg has a concept of “dispositions” – a property that describes the purpose of a stream in a media file. For example, here are the streams in a file I have lying around, with the dispositions emphasized :

  


    Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,&#xA;fltp, 251 kb/s <strong><em>(default)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Audio&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:1[0x2](und): Video: mjpeg (Baseline) (jpeg / 0x6765706A),&#xA;yuvj420p(pc, bt470bg/unknown/unknown), 1024x1024, 0 kb/s, 0.0006 fps, 3.08 tbr,&#xA;600 tbn <strong><em>(default) (attached pic) (timed thumbnails)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Video&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:2[0x3](und): Data: bin_data (text / 0x74786574)&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Text&#xA;&#xA;  Stream #0:3[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/&#xA;unknown), 1024x1024 [SAR 144:144 DAR 1:1], 90k tbr, 90k tbn <strong><em>(attached pic)</em></strong>

  


  However, if I make any modification to this file’s chapter markers using the C++ library MP4v2 (even just re-saving the existing ones : auto f = MP4Modify("test.m4a"); MP4Chapter_t* chapterList; uint32_t chapterCount; MP4GetChapters(f, &chapterList, &chapterCount); MP4SetChapters(f, chapterList, chapterCount); MP4Close(f);), some of these dispositions are removed :

  


    Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,&#xA;fltp, 251 kb/s <strong><em>(default)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Audio&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:1[0x2](und): Video: mjpeg (Baseline) (jpeg / 0x6765706A),&#xA;yuvj420p(pc, bt470bg/unknown/unknown), 1024x1024, 0 kb/s, 0.0006 fps, 3.08 tbr,&#xA;600 tbn <strong><em>(default)</em></strong> <kbd>← “attached pic” and “timed thumbnails” removed!</kbd>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Video&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:2[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/&#xA;unknown), 1024x1024 [SAR 144:144 DAR 1:1], 90k tbr, 90k tbn <strong><em>(attached pic)</em></strong>&#xA;&#xA;  Stream #0:3[0x4](und): Data: bin_data (text / 0x74786574)&#xA;  <kbd>This stream was moved to the end, but that’s intended behavior. It contains chapter titles, and we just edited the chapters.</kbd>&#xA;      Metadata:&#xA;        creation_time   : 2025-03-05T09:56:31.000000Z

  


  It also renders the file unplayable in MPC-HC (but not in VLC !), which is apparently a bug in MP4v2. I’m currently investigating that bug to report and potentially fix it, but that’s a separate issue – in my journey there, I’m wracking my brain trying to understand what it is that MP4v2 changes to make FFmpeg stop reporting the “attached pic” and “timed thumbnails” dispositions. I’ve explored the before-and-afters in MP4 Box, and I can’t for the life of me find which atom it is that differs in a relevant way.

  


  (I’d love to share the files, but unfortunately the contents are under copyright – if anyone knows of a way to remove the audio from an MP4 file without changing anything else, let me know and I’ll upload dummied-out versions. Without them, I can’t really ask about the issue directly. I can at least show you the files’ respective atom trees, but I’m not sure how relevant that is.)

  


  The Question

  


  I thought I’d read FFmpeg’s source code to find out how it determines dispositions for MP4 streams, but of course, FFmpeg is very complex. Could someone who’s more familiar with C and/or FFmpeg’s codebase help me sleuth out how FFmpeg determines dispositions for MP4 files (in particular, “attached pic” and “timed thumbnails”) ?

  


  Some Thoughts…

  


  

  • I figure searching for “attached_pic” might be a good start ?

  


  • Could the MP4 muxer movenc.c be helpful ?

  


  • I’d imagine what we’d really like to look at is the MP4 demuxing process, as it’s during demuxing that FFmpeg determines dispositions from the data in the file. After poring over the code for hours, however, I’ve been utterly unable to find where that happens.

  


  


• How does one correctly avoid the avcodec_alloc_context3 leak from avformat_new_stream ?

  14 avril 2017, par Tom Seddon

  This maddening thread describes the problem I’m having : a memory leak on shutdown due to some stuff allocated when avformat_new_stream is called.

  Here’s the valgrind stack trace from the leak :

  • 1,447 (1,440 direct, 7 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 4
  • at 0x4C2FFC6 : memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
  • by 0x4C300D1 : posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
  • by 0x690DEFF : av_malloc (mem.c:87)
  • by 0x690E09D : av_mallocz (mem.c:224)
  • by 0x533D28A : init_context_defaults (options.c:128)
  • by 0x533D325 : avcodec_alloc_context3 (options.c:164)
  • by 0x663D09E : avformat_new_stream (utils.c:4384)
  • by 0x4204B6 : main (test_ffmpeg.cpp:918)

  So clearly the problem is that when a AVFormatContext’s stream’s codec context’s priv_data field is somehow not being freed.

  My code frees the AVFormatContext with avformat_free_context. This calls ff_free_stream, which calls free_stream, which frees a few of the stream’s codec context fields itself - but not the priv_data field !

  Compared and contrast with the corresponding code in avcodec_close.

  The suggested solution to the problem from the thread : "close the codec firstly before calling av_format_free_context". Presumably this refers to calling avcodec_free_context ? - but I’m already doing this ! Roughly following the structure in the muxing example, I have an encoder context created by my code, that’s used to track the uncompressed input data. Then there’s another encoder context created internally by avformat_new_stream (as above), which is used internally by FFmpeg. I close the former, because it was opened using avcodec_open2, but I don’t close the latter, because it wasn’t. I am following the maddening thread’s advice, and yet here I am.

  Furthermore, reading between the lines, using avcodec_free_context to free the AVStream’s codec context is no good anyway, because when doing this (a) AVStream’s codec field is deprecated, so this gives a bunch of warnings, and (b) there are no NULL checks in free_stream, so this crashes at runtime.

  What I have done for now is drag in the appropriate bit of code from avcodec_close, and put it in my own code just ahead of the call to avformat_free_context :

  #ifdef __GNUC__
  
  #pragma GCC diagnostic push
  
  #pragma GCC diagnostic ignored "-Wdeprecated-declarations"
  
  #endif
  
      for(unsigned i=0;inb_streams;++i) {
  
          AVStream *st=avf_context->streams[i];
  
  
  
          if(st->codec->priv_data&&
  
             st->codec->codec&&
  
             st->codec->codec->priv_class)
  
          {
  
              av_opt_free(st->codec->priv_data);
  
          }
  
  
  
          av_freep(&st->codec->priv_data);
  
      }
  
  #ifdef __GNUC__
  
  #pragma GCC diagnostic pop
  
  #endif

  So that fixes the leak, and it’s clearly (to my eyes) a bug, and I need to file a bug report or something.

  However the corresponding bug report is marked as fixed and closed... along with a link back to the maddening thread, and no further explanation. (This is why it is maddening !) So maybe I’m just using FFmpeg wrongly ?

  Can anybody confirm whether this is actually a bug in FFmpeg or not ?

  If it isn’t a bug, what’s the right sequence of calls to make ?

  (I’m using FFmpeg built locally from commit 03eb0515c12637dbd20c2e3ca8503d7b47cf583a. I had similar-looking problems with the one you get from the Ubuntu 16 package manager, which prompted me to build it myself with symbols and so on.)

• png : Only init zlib once

  29 mars 2015, par Donny Yang

  png : Only init zlib once
  Signed-off-by : Donny Yang <work@kota.moe>
  
  Signed-off-by : Michael Niedermayer <michaelni@gmx.at>
  

  • [DH] libavcodec/pngenc.c