Recherche avancée

Sur d’autres sites (5830)

• Glitchy audio or broken video in fragmented MP4

  6 septembre 2022, par PookyFan

  I'm working on small C++ library for muxing audio and video. This is basically a facade for FFMPEG functions and structures. The code is here with minimal reproduction testing code here and as of now it seems like it's working fine... almost.

  


  For the record - my MP4 file is so-called "fragmented MP4", with headers moved at the beginning of the file in a way that would allow to stream that file (ie. play it in a browser while it's being buffered). That's what these movflags I'm setting in Mp4Muxer::writeHeader() are for.

  


  While testing this library with raw H264 video stream and MP3 file (video is <1 min long, MP3 - a few minutes long), I observed that :

  &#xA;

  &#xA;
  • if I don't limit muxing audio stream when it's way "ahead" of the video (and it will be since MP3 is longer, so eventually video frames stop coming but audio frames still come in), all muxes just fine with no errors, but playing output MP4 with ffplay after just a few seconds results in the following log (and also frozen video, while audio keeps playing) :

  &#xA;

  &#xA;

  [h264 @ 0x7f90a40ae2c0] Invalid NAL unit size (2162119 > 76779).0
[h264 @ 0x7f90a40ae2c0] Error splitting the input into NAL units.
[mp3float @ 0x7f90a4009540] Header missing  515KB sq=    0B f=0/0
[h264 @ 0x7f90a40cb0c0] Invalid NAL unit size (-860010620 > 17931).
[h264 @ 0x7f90a40cb0c0] Error splitting the input into NAL units.
[h264 @ 0x7f90a42bf440] Invalid NAL unit size (-168012642 > 8000).
[h264 @ 0x7f90a42bf440] Error splitting the input into NAL units.
[h264 @ 0x7f90a42fa780] Invalid NAL unit size (-1843711407 > 5683).
[ and it repeats...]


  &#xA;

  &#xA;
  • even if I limit how much a stream can be "ahead" of the other, limiting it too much results in no muxed data in the output

  &#xA;

  • any other intermediate level of limiting how much one stream can be buffered in muxer relative to the other stream results in glitchy audio, with the following errors popping out every now and then in ffplay (the more strict limit is, the more often they are printed) :

  &#xA;

  &#xA;

  [mp3float @ 0x7f744c01b640] overread, skip -6 enddists: -1 -1=0/0 


  &#xA;

  Not limitting muxed audio (at all or enough) relative to muxed video also results in following messages in my muxing application :

  &#xA;

  [mp4 @ 0x55d0c6c21940] Delay between the first packet and last packet in the muxing queue is 10004898 > 10000000: forcing output


  &#xA;

  For now, the fix is quite ugly and I don't even understand why it works, but before writting MP4 header I manually set a limit for frames buffered by muxer, like so :

  &#xA;

  formatCtxt->max_interleave_delta = 10000000LL * 10LL;


  &#xA;

  This way the muxer can store more packets of one stream that's way "ahead" of the other (maximum difference between DTS of the packets at the beginning and at the end of queue is set to 10x larger than default ; it also gets rid of information log mentioned above). Obviously, I'd like to resolve it more properly, without hacking things like that.

  &#xA;

  I was trying various things, including manual skipping of ID3 tags in MP3 file (but seems like FFMPEG handles them just fine and it didn't change anything). I was also experimenting with FLAC in MP4 instead of MP3. and while I know it's rather experimental thing, I encountered very similar problems with glitching audio (no problem with video being frozen when lots of audio data gets muxed, though). It also seems that problem with glitching audio or frozen video varies in scale depending on how large are input data chunks that I feed muxer with. For now, honestly, I'm out of ideas.

  &#xA;

• The ffmpeg webm_dash_manifest demuxer fails with webvtt subtitles

  30 janvier 2020, par Hyldrean

  I am trying to create a DASH manifest using ffmpeg and I encounter some difficulties with the subtitles. I get the error

  [webm_dash_manifest @ 0x55e19db48180] EBML header parsing failed
  
  [webm_dash_manifest @ 0x55e19db48180] Failed to read file headers
  
  subtitles.vtt: Operation not permitted

  The subtitles are encoded in the plain text webvtt format. I first assume that the EBML header are missing because the subtitles were not contained in a webm container. I then tried to create a webm file containing only the subtitles. It appears impossible.

  I think the problem comes from the webm_dash_manifest demuxer. It seeks metadata in a file with no metadata. However neither Google nor Stack Overflow give me an answer.

  The command I used :

  ffmpeg \
  
  -f webm_dash_manifest -i video.webm \
  
  -f webm_dash_manifest -i audio.webm \
  
  -f webm_dash_manifest -i subtitles.vtt \
  
  -map 0 -map 1 -map 2 \
  
  -c copy \
  
  -f webm_dash_manifest \
  
  -adaptation_sets "id=0,streams=0 id=1,streams=1 id=2,stream=2" \
  
  minimal.mpd

  and the ffmpeg output :

  ffmpeg version 3.2.14-1~deb9u1 Copyright (c) 2000-2019 the FFmpeg developers
  
    built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  
    configuration: --prefix=/usr --extra-version='1~deb9u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  
    libavutil      55. 34.101 / 55. 34.101
  
    libavcodec     57. 64.101 / 57. 64.101
  
    libavformat    57. 56.101 / 57. 56.101
  
    libavdevice    57.  1.100 / 57.  1.100
  
    libavfilter     6. 65.100 /  6. 65.100
  
    libavresample   3.  1.  0 /  3.  1.  0
  
    libswscale      4.  2.100 /  4.  2.100
  
    libswresample   2.  3.100 /  2.  3.100
  
    libpostproc    54.  1.100 / 54.  1.100
  
  [webm_dash_manifest @ 0x55e19db17160] Could not find codec parameters for stream 0 (Video: vp9, none, 1280x534): unspecified pixel format
  
  Consider increasing the value for the 'analyzeduration' and 'probesize' options
  
  Input #0, webm_dash_manifest, from 'video.webm':
  
    Metadata:
  
      title           : Cloud Atlas
  
      ENCODER         : Lavf58.35.104
  
    Duration: 00:05:00.01, bitrate: 998 kb/s
  
      Chapter #0:0: start 0.000000, end 300.000000
  
      Metadata:
  
        title           : Chapter 01
  
      Stream #0:0(eng): Video: vp9, none, 1280x534, SAR 1:1 DAR 640:267, 23.98 fps, 23.98 tbr, 1k tbn, 1k tbc (default) (forced)
  
      Metadata:
  
        title           : Video @ 6.9 Mbps [frame B:188517 Avg QP:20.66]
  
        ENCODER         : Lavc58.66.100 libvpx-vp9
  
        DURATION        : 00:05:00.008000000
  
        webm_dash_manifest_duration: 300008
  
        webm_dash_manifest_initialization_range: 900
  
        webm_dash_manifest_file_name: video.webm
  
        webm_dash_manifest_track_number: 1
  
        webm_dash_manifest_cues_start: 37455991
  
        webm_dash_manifest_cues_end: 37457486
  
        webm_dash_manifest_bandwidth: 1420427
  
        webm_dash_manifest_cluster_keyframe: 1
  
        webm_dash_manifest_cue_timestamps: 298715
  
  Input #1, webm_dash_manifest, from 'audio.webm':
  
    Metadata:
  
      title           : Cloud Atlas
  
      ENCODER         : Lavf58.35.104
  
    Duration: 00:05:00.01, bitrate: 126 kb/s
  
      Chapter #1:0: start 0.000000, end 300.000000
  
      Metadata:
  
        title           : Chapter 01
  
      Stream #1:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
  
      Metadata:
  
        title           : Audio Eng DTS 5.1 channels 24 bits @ 1509 Kbps cbr
  
        ENCODER         : Lavc58.66.100 libopus
  
        DURATION        : 00:05:00.007000000
  
        webm_dash_manifest_duration: 300007
  
        webm_dash_manifest_initialization_range: 838
  
        webm_dash_manifest_file_name: audio.webm
  
        webm_dash_manifest_track_number: 1
  
        webm_dash_manifest_cues_start: 4744091
  
        webm_dash_manifest_cues_end: 4745279
  
        webm_dash_manifest_bandwidth: 137531
  
        webm_dash_manifest_cluster_keyframe: 1
  
        webm_dash_manifest_cue_timestamps: 295000
  
  [webm_dash_manifest @ 0x55e19db48180] EBML header parsing failed
  
  [webm_dash_manifest @ 0x55e19db48180] Failed to read file headers
  
  subtitles.vtt: Operation not permitted

  I noticed that ffmpeg -f webm_dash_manifest -i subtitles.vtt suffices to reproduce the error.

  I have also tried without the -f webm_dash_manifest flag, with no success :

  ffmpeg \
  
  -f webm_dash_manifest -i video.webm \
  
  -f webm_dash_manifest -i audio.webm \
  
  -i subtitles.vtt \
  
  -map 0 -map 1 -map 2 \
  
  -c copy \
  
  -f webm_dash_manifest \
  
  -adaptation_sets "id=0,streams=0 id=1,streams=1 id=2,stream=2" minimal.mpd

  Output :

  ffmpeg version 3.2.14-1~deb9u1 Copyright (c) 2000-2019 the FFmpeg developers
  
    built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  
    configuration: --prefix=/usr --extra-version='1~deb9u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  
    libavutil      55. 34.101 / 55. 34.101
  
    libavcodec     57. 64.101 / 57. 64.101
  
    libavformat    57. 56.101 / 57. 56.101
  
    libavdevice    57.  1.100 / 57.  1.100
  
    libavfilter     6. 65.100 /  6. 65.100
  
    libavresample   3.  1.  0 /  3.  1.  0
  
    libswscale      4.  2.100 /  4.  2.100
  
    libswresample   2.  3.100 /  2.  3.100
  
    libpostproc    54.  1.100 / 54.  1.100
  
  [webm_dash_manifest @ 0x557f55dd1160] Could not find codec parameters for stream 0 (Video: vp9, none, 1280x534): unspecified pixel format
  
  Consider increasing the value for the 'analyzeduration' and 'probesize' options
  
  Input #0, webm_dash_manifest, from 'video.webm':
  
    Metadata:
  
      title           : Cloud Atlas
  
      ENCODER         : Lavf58.35.104
  
    Duration: 00:05:00.01, bitrate: 998 kb/s
  
      Chapter #0:0: start 0.000000, end 300.000000
  
      Metadata:
  
        title           : Chapter 01
  
      Stream #0:0(eng): Video: vp9, none, 1280x534, SAR 1:1 DAR 640:267, 23.98 fps, 23.98 tbr, 1k tbn, 1k tbc (default) (forced)
  
      Metadata:
  
        title           : Video @ 6.9 Mbps [frame B:188517 Avg QP:20.66]
  
        ENCODER         : Lavc58.66.100 libvpx-vp9
  
        DURATION        : 00:05:00.008000000
  
        webm_dash_manifest_duration: 300008
  
        webm_dash_manifest_initialization_range: 900
  
        webm_dash_manifest_file_name: video.webm
  
        webm_dash_manifest_track_number: 1
  
        webm_dash_manifest_cues_start: 37455991
  
        webm_dash_manifest_cues_end: 37457486
  
        webm_dash_manifest_bandwidth: 1420427
  
        webm_dash_manifest_cluster_keyframe: 1
  
        webm_dash_manifest_cue_timestamps: 298715
  
  Input #1, webm_dash_manifest, from 'audio.webm':
  
    Metadata:
  
      title           : Cloud Atlas
  
      ENCODER         : Lavf58.35.104
  
    Duration: 00:05:00.01, bitrate: 126 kb/s
  
      Chapter #1:0: start 0.000000, end 300.000000
  
      Metadata:
  
        title           : Chapter 01
  
      Stream #1:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
  
      Metadata:
  
        title           : Audio Eng DTS 5.1 channels 24 bits @ 1509 Kbps cbr
  
        ENCODER         : Lavc58.66.100 libopus
  
        DURATION        : 00:05:00.007000000
  
        webm_dash_manifest_duration: 300007
  
        webm_dash_manifest_initialization_range: 838
  
        webm_dash_manifest_file_name: audio.webm
  
        webm_dash_manifest_track_number: 1
  
        webm_dash_manifest_cues_start: 4744091
  
        webm_dash_manifest_cues_end: 4745279
  
        webm_dash_manifest_bandwidth: 137531
  
        webm_dash_manifest_cluster_keyframe: 1
  
        webm_dash_manifest_cue_timestamps: 295000
  
  Input #2, webvtt, from 'subtitles.vtt':
  
    Duration: N/A, bitrate: N/A
  
      Stream #2:0: Subtitle: webvtt
  
  Could not write header for output file #0 (incorrect codec parameters ?): Operation not permittedStream mapping:
  
    Stream #0:0 -> #0:0 (copy)
  
    Stream #1:0 -> #0:1 (copy)
  
    Stream #2:0 -> #0:2 (copy)
  
      Last message repeated 1 times

  Note : [webm_dash_manifest @ 0x557f55dd1160] Could not find codec parameters for stream 0 (Video: vp9, none, 1280x534): unspecified pixel format is harmless according to this post.

  The subtitles have been extracted with ffmpeg :

  ffmpeg -i "Cloud Atlas.mkv" -map 0:6 -t 00:05:00 -c:s webvtt -dash 1 subtitles.vtt

  Thank you for reading my long post, I have no idea what to do next.

  EDIT : The WebM Project website says : ’Initial WebM release does not support subtitles’. Has someone heard of a new release ? Or a convenient way to create DASH manifest with subtitles ?

• Call to avformat_find_stream_info prevents decoding of simple PNG image ?

  10 avril 2014, par kloffy

  I am running into a problem decoding a simple PNG image with libav. The decode_ok flag after the call to avcodec_decode_video2 is set to 0, even though the packet contains the entire image. Through some experimentation, I have managed to pinpoint the issue and it seems related to calling avformat_find_stream_info. If the call is removed, the example runs successfully. However, I would like to use the same code for other media, and calling avformat_find_stream_info is recommended in the documentation.

  The following minimal example illustrates the behavior (unfortunately still a bit lengthy) :

  #include <iostream>
  
  
  
  extern "C"
  
  {
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libavcodec></libavcodec>avcodec.h>
  
  }
  
  
  
  // Nothing to see here, it&#39;s just a helper function
  
  AVCodecContext* open(AVMediaType mediaType, AVFormatContext* formatContext)
  
  {
  
      auto ret = 0;
  
      if ((ret = av_find_best_stream(formatContext, mediaType, -1, -1, nullptr, 0)) &lt; 0)
  
      {
  
          std::cerr &lt;&lt; "Failed to find video stream." &lt;&lt; std::endl;
  
          return nullptr;
  
      }
  
  
  
      auto codecContext = formatContext->streams[ret]->codec;
  
      auto codec = avcodec_find_decoder(codecContext->codec_id);
  
      if (!codec)
  
      {
  
          std::cerr &lt;&lt; "Failed to find codec." &lt;&lt; std::endl;
  
          return nullptr;
  
      }
  
  
  
      if ((ret = avcodec_open2(codecContext, codec, nullptr)) != 0)
  
      {
  
          std::cerr &lt;&lt; "Failed to open codec context." &lt;&lt; std::endl;
  
          return nullptr;
  
      }
  
  
  
      return codecContext;
  
  }
  
  
  
  // All the interesting bits are here
  
  int main(int argc, char* argv[])
  
  {
  
      auto path = "/path/to/test.png"; // Replace with valid path to PNG
  
      auto ret = 0;
  
  
  
      av_log_set_level(AV_LOG_DEBUG);
  
  
  
      av_register_all();
  
      avcodec_register_all();
  
  
  
      auto formatContext = avformat_alloc_context();
  
      if ((ret = avformat_open_input(&amp;formatContext, path, NULL, NULL)) != 0)
  
      {
  
          std::cerr &lt;&lt; "Failed to open input." &lt;&lt; std::endl;
  
          return -1;
  
      }
  
      av_dump_format(formatContext, 0, path, 0);
  
  
  
  //*/ Info is successfully found, but interferes with decoding
  
      if((ret = avformat_find_stream_info(formatContext, nullptr)) &lt; 0)
  
      {
  
          std::cerr &lt;&lt; "Failed to find stream info." &lt;&lt; std::endl;
  
          return -1;
  
      }
  
      av_dump_format(formatContext, 0, path, 0);
  
  //*/
  
  
  
      auto codecContext = open(AVMEDIA_TYPE_VIDEO, formatContext);
  
  
  
      AVPacket packet;
  
      av_init_packet(&amp;packet);
  
  
  
      if ((ret = av_read_frame(formatContext, &amp;packet)) &lt; 0)
  
      {
  
          std::cerr &lt;&lt; "Failed to read frame." &lt;&lt; std::endl;
  
          return -1;
  
      }
  
  
  
      auto frame = av_frame_alloc();
  
      auto decode_ok = 0;
  
      if ((ret = avcodec_decode_video2(codecContext, frame, &amp;decode_ok, &amp;packet)) &lt; 0 || !decode_ok)
  
      {
  
          std::cerr &lt;&lt; "Failed to decode frame." &lt;&lt; std::endl;
  
          return -1;
  
      }
  
  
  
      av_frame_free(&amp;frame);
  
      av_free_packet(&amp;packet);
  
  
  
      avcodec_close(codecContext);
  
  
  
      avformat_close_input(&amp;formatContext);
  
      av_free(formatContext);
  
  
  
      return 0;
  
  }
  
  </iostream>

  The format dump before avformat_find_stream_info prints :

  Input #0, image2, from '/path/to/test.png' :
    Duration : N/A, bitrate : N/A
      Stream #0:0, 0, 1/25 : Video : png, 25 tbn

  The format dump after avformat_find_stream_info prints :

  Input #0, image2, from '/path/to/test.png' :
    Duration : 00:00:00.04, start : 0.000000, bitrate : N/A
      Stream #0:0, 1, 1/25 : Video : png, rgba, 512x512 [SAR 3780:3780 DAR 1:1], 1/25, 25 tbr, 25 tbn, 25 tbc

  So it looks like the search yields potentially useful information. Can anybody shed some light on this problem ? Other image formats seem to work fine. I assume that this is a simple user error rather than a bug.

  Edit : Debug logging was already enabled, but the PNG decoder does not produce a lot of output. I have also tried setting a custom logging callback.

  Here is what I get without the call to avformat_find_stream_info, when decoding succeeds :

  Statistics : 52125 bytes read, 0 seeks

  And here is what I get with the call to avformat_find_stream_info, when decoding fails :

  Statistics : 52125 bytes read, 0 seeks
  detected 8 logical cores
  

  The image is 52125 bytes, so the whole file is read. I am not sure what the logical cores are referring to.