Recherche avancée

Sur d’autres sites (7753)

• How to compensate frame rate underrun while muxing video to mp4 container with libav

  17 septembre 2021, par Nuno Santos

  I have a process that generates video frames in real time. I’m muxing the generated video frames stream in a video file (x264 codec on a mp4 container).

  


  I'm using ffmpeg-libav and I'm basing myself on the muxing.c example. The problem with the example is that isn't a real world scenario as frames are being generated on a while loop for a given stream duration, never missing a frame.

  


  On my program, frames are supposed to be generated at FPS, however, depending on the hardware capacity it might produce less than FPS. When I initialize the video stream context I declare that frame rate is FPS :

  


  AVRational r = { 1, FPS };
ost->st->time_base = r;


  


  This specifies that the video is going to have FPS frame rate but if less frames are produced, the playback will be faster because it will still reproduce the video as it if had all the declared frames per second.

  


  After googling a lot about this topic I understand that the key to fix this is to manipulate pts and dts but I still haven't found a solution that works.

  


  There are two key functions when writing video frames in the muxing.c example, routines that I'm using in my program :

  


  AVFrame* get_video_frame(int timestamp, OutputStream *ost, const QImage &image)
{
    /* when we pass a frame to the encoder, it may keep a reference to it
     * internally; make sure we do not overwrite it here */
    if (av_frame_make_writable(ost->frame) < 0)
        exit(1);

    av_image_fill_arrays(ost->tmp_frame->data, ost->tmp_frame->linesize, image.bits(), AV_PIX_FMT_RGBA, ost->frame->width, ost->frame->height, 8);
    libyuv::ABGRToI420(ost->tmp_frame->data[0], ost->tmp_frame->linesize[0], ost->frame->data[0], ost->frame->linesize[0], ost->frame->data[1], ost->frame->linesize[1], ost->frame->data[2], ost->frame->linesize[2], ost->tmp_frame->width, -ost->tmp_frame->height);

    #if 1 // this is my attempt to rescale pts, but crashes with ptsframe->pts = av_rescale_q(timestamp, AVRational{1, 1000}, ost->st->time_base);
    #else
    ost->frame->pts = ost->next_pts++;
    #endif

    return ost->frame;
}


  


  On the original code, the pts is simply an incremeting integer for each frame. What I'm trying to do is to pass a timestamp in ms since the beggining of the recording so that I can rescale the pts. When I rescale pts the program crashes complaining that pts is lower then dts.

  


  From what I've been reading, the pts/dts manipulation is supposed to be done at the packet level so I have also tried to manipulate things on write_frame routine without success.

  


  int write_frame(AVFormatContext *fmt_ctx, AVCodecContext *c, AVStream *st, AVFrame *frame)
{
    int ret;

    // send the frame to the encoder
    ret = avcodec_send_frame(c, frame);

    if (ret<0)
    {
        fprintf(stderr, "Error sending a frame to the encoder\n");
        exit(1);
    }

    while (ret >= 0)
    {
        AVPacket pkt = { 0 };

        ret = avcodec_receive_packet(c, &pkt);

        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
        {
            break;
        }
        else if (ret<0)
        {
            //fprintf(stderr, "Error encoding a frame: %s\n", av_err2str(ret));
            exit(1);
        }

        /* rescale output packet timestamp values from codec to stream timebase */
        av_packet_rescale_ts(&pkt, c->time_base, st->time_base);
        pkt.stream_index = st->index;

        /* Write the compressed frame to the media file. */
        //log_packet(fmt_ctx, &pkt);
        ret = av_interleaved_write_frame(fmt_ctx, &pkt);
        av_packet_unref(&pkt);

        if (ret < 0)
        {
            //fprintf(stderr, "Error while writing output packet: %s\n", av_err2str(ret));
            exit(1);
        }
    }

    return ret == AVERROR_EOF ? 1 : 0;
}


  


  How should I manipulate dts and pts so that I can achieve a video at certain frame that does not have all the frames as specified in the stream initialization ? Where should I do that manipulation ? On get_video_frame ? On write_frame ? On both ?

  


  Am I heading in the right direction ? What am I missing ?

  


• Multimedia Exploration Journal : The Past Doesn’t Die

  12 juillet 2011, par Multimedia Mike — Game Hacking

  New haul of games, new (old) multimedia formats.

  Lords of Midnight
  Check out the box copy scan for Lords of Midnight in MobyGames. In particular, I’d like to call your attention to this little blurb :

  
  
  

  Ahem, "Journey through an immense world — the equivalent of 8 CD-ROMs." Yet, when I procured the game, it only came on a single CD-ROM. It’s definitely a CD-ROM (says so on the disc) and, coming from 1995, certainly predates the earliest DVD-ROMs (which can easily store 8 CD-ROMs on a disc). Thus, I wanted to jump in a see if they were using some phenomenal compression in order to squeeze so much info into 600 or so megabytes.

  I was surprised to see the contents of the disc clocking in at just under 40 megabytes. An intro movie and an outro movie account for 75% of that. Format ? None other than that curious ASCII anomaly, ARMovie/RPL with Escape 122 codec data.

  Cyclemania

  
  
  

  Cyclemania is one of those FMV backdrop action games, but with a motorcycle theme. I had a good feeling I would find some odd multimedia artifacts here and the game didn’t disappoint. The videos are apparently handled using 3-4 discrete files per animation. I’ve documented my cursory guesses and linked some samples at the new MultimediaWiki page.

  Interplay ACMP
  This is unrelated to this particular acquistion, but I was contacted today about audio files harvested from the 1993 DOS game Star Trek : Judgment Rites. The files begin with the ASCII signature "Interplay ACMP Data". This reminds me of Interplay MVE files which begin with the similar string "Interplay MVE File". My theory is that these files use the ACOMP compression format, though I’m still trying to make it fit.

  Wiki and samples are available as usual if you’d like to add your own research.

• FFMPEG conversion error from MKV to MP4 using libfdk_aac codec

  23 août 2014, par Charlie Brown

  The purpose of doing this is because my "Smart" samsung TV doesn’t play MKV files.

  This thread mentions that there is no need for video conversion, just changing the container might suffice. After reading multiple threads on how to change the container of MKV to MP4, I downloaded and compiled the FFMPEG with libfdk_aac support from here. I tried converting a sample MKV file using this command :

  ffmpeg -i "C:\VideoTest\Sample1.mkv" -c:v copy -c:a libfdk_aac -b:a 384k "C:\VideoTest\Sample2.mp4"

  but getting the following errors (copied from command prompt) :

  [hevc @ 0000000000308ae0] Could not find ref with POC 113
  
  [hevc @ 0000000000308ae0] Could not find ref with POC 111
  
  [hevc @ 0000000000308ae0] Could not find ref with POC 109
  
  [hevc @ 0000000000308ae0] Could not find ref with POC 107
  
  Input #0, matroska,webm, from 'C:\VideoTest\Sample-1.mk
  
  v':
  
    Metadata:
  
      encoder         : libebml v1.3.0 + libmatroska v1.4.1
  
      creation_time   : 2014-07-16 08:54:02
  
    Duration: 00:00:58.68, start: 0.042000, bitrate: 3520 kb/s
  
      Chapter #0.0: start 0.000000, end 58.717000
  
      Metadata:
  
        title           : Chapter 4
  
      Stream #0:0: Video: hevc (Main), yuv420p(tv), 1920x816, SAR 1:1 DAR 40:17, 2
  
  4 fps, 24 tbr, 1k tbn, 1k tbc (default)
  
      Metadata:
  
        _STATISTICS_WRITING_APP: mkvmerge v7.0.0 ('Where We Going') 64bit built on
  
   Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_APP-eng: mkvmerge v7.0.0 ('Where We Going') 64bit buil
  
  t on Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_DATE_UTC: 2014-07-16 08:54:02
  
        _STATISTICS_WRITING_DATE_UTC-eng: 2014-07-16 08:54:02
  
        _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        BPS             : 2733260
  
        BPS-eng         : 2733260
  
        DURATION        : 00:00:58.667000000
  
        DURATION-eng    : 00:00:58.667000000
  
        NUMBER_OF_FRAMES: 1408
  
        NUMBER_OF_FRAMES-eng: 1408
  
        NUMBER_OF_BYTES : 20044023
  
        NUMBER_OF_BYTES-eng: 20044023
  
      Stream #0:1: Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 768 kb/s (default)
  
  
  
      Metadata:
  
        _STATISTICS_WRITING_APP: mkvmerge v7.0.0 ('Where We Going') 64bit built on
  
   Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_APP-eng: mkvmerge v7.0.0 ('Where We Going') 64bit buil
  
  t on Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_DATE_UTC: 2014-07-16 08:54:02
  
        _STATISTICS_WRITING_DATE_UTC-eng: 2014-07-16 08:54:02
  
        _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        BPS             : 767986
  
        BPS-eng         : 767986
  
        DURATION        : 00:00:58.625000000
  
        DURATION-eng    : 00:00:58.625000000
  
        NUMBER_OF_FRAMES: 5496
  
        NUMBER_OF_FRAMES-eng: 5496
  
        NUMBER_OF_BYTES : 5627904
  
        NUMBER_OF_BYTES-eng: 5627904
  
      Stream #0:2(eng): Subtitle: hdmv_pgs_subtitle, 1920x1080 (default)
  
      Metadata:
  
        _STATISTICS_WRITING_APP: mkvmerge v7.0.0 ('Where We Going') 64bit built on
  
   Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_APP-eng: mkvmerge v7.0.0 ('Where We Going') 64bit buil
  
  t on Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_DATE_UTC: 2014-07-16 08:54:02
  
        _STATISTICS_WRITING_DATE_UTC-eng: 2014-07-16 08:54:02
  
        _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        BPS             : 17485
  
        BPS-eng         : 17485
  
        DURATION        : 00:00:56.750000000
  
        DURATION-eng    : 00:00:56.750000000
  
        NUMBER_OF_FRAMES: 40
  
        NUMBER_OF_FRAMES-eng: 40
  
        NUMBER_OF_BYTES : 124036
  
        NUMBER_OF_BYTES-eng: 124036
  
  Output #0, mp4, to 'C:\VideoTest\Sample.mp4':
  
    Metadata:
  
      encoder         : Lavf56.1.100
  
      Chapter #0.0: start 0.000000, end 58.675000
  
      Metadata:
  
        title           : Chapter 4
  
      Stream #0:0: Video: hevc ([35][0][0][0] / 0x0023), yuv420p, 1920x816 [SAR 1:
  
  1 DAR 40:17], q=2-31, 24 fps, 16k tbn, 1k tbc (default)
  
      Metadata:
  
        _STATISTICS_WRITING_APP: mkvmerge v7.0.0 ('Where We Going') 64bit built on
  
   Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_APP-eng: mkvmerge v7.0.0 ('Where We Going') 64bit buil
  
  t on Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_DATE_UTC: 2014-07-16 08:54:02
  
        _STATISTICS_WRITING_DATE_UTC-eng: 2014-07-16 08:54:02
  
        _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        BPS             : 2733260
  
        BPS-eng         : 2733260
  
        DURATION        : 00:00:58.667000000
  
        DURATION-eng    : 00:00:58.667000000
  
        NUMBER_OF_FRAMES: 1408
  
        NUMBER_OF_FRAMES-eng: 1408
  
        NUMBER_OF_BYTES : 20044023
  
        NUMBER_OF_BYTES-eng: 20044023
  
      Stream #0:1: Audio: aac (libfdk_aac) ([64][0][0][0] / 0x0040), 48000 Hz, 5.1
  
  , s16, 128 kb/s (default)
  
      Metadata:
  
        _STATISTICS_WRITING_APP: mkvmerge v7.0.0 ('Where We Going') 64bit built on
  
   Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_APP-eng: mkvmerge v7.0.0 ('Where We Going') 64bit buil
  
  t on Jun  9 2014 15:16:27
  
        _STATISTICS_WRITING_DATE_UTC: 2014-07-16 08:54:02
  
        _STATISTICS_WRITING_DATE_UTC-eng: 2014-07-16 08:54:02
  
        _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  
        BPS             : 767986
  
        BPS-eng         : 767986
  
        DURATION        : 00:00:58.625000000
  
        DURATION-eng    : 00:00:58.625000000
  
        NUMBER_OF_FRAMES: 5496
  
        NUMBER_OF_FRAMES-eng: 5496
  
        NUMBER_OF_BYTES : 5627904
  
        NUMBER_OF_BYTES-eng: 5627904
  
        encoder         : Lavc56.0.101 libfdk_aac
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (copy)
  
    Stream #0:1 -> #0:1 (dts (dca) -> aac (libfdk_aac))
  
  Press [q] to stop, [?] for help
  
  [mp4 @ 0000000003be0500] Invalid DTS: 656 PTS: -672 in output stream 0:0
  
  av_interleaved_write_frame(): Invalid argument
  
  [libfdk_aac @ 00000000042fa640] Trying to remove 1024 samples, but the queue is
  
  empty
  
  [mp4 @ 0000000003be0500] Encoder did not produce proper pts, making some up.
  
  [libfdk_aac @ 00000000042fa640] Trying to remove 1024 samples, but the queue is
  
  empty
  
  frame=    2 fps=0.0 q=-1.0 Lsize=      96kB time=00:00:00.04 bitrate=18415.0kbit
  
  s/s
  
  video:95kB audio:1kB subtitle:0kB other streams:0kB global headers:0kB muxing ov
  
  erhead: 0.570364%
  
  Conversion failed!

  Also tried using this command :

  ffmpeg -i input.wav -c:a libfdk_aac -b:a 128k output.mp4

  But this screws up the video quality and I start seeing some bad pixelated patches during the playback.

  Any help would be appreciated.

  Just an FYI : I got this far by reading blogs and posts. I have 0 knowledge about video conversion.