Recherche avancée

Sur d’autres sites (8963)

• How to compensate frame rate underrun while muxing video to mp4 container with libav

  17 septembre 2021, par Nuno Santos

  I have a process that generates video frames in real time. I’m muxing the generated video frames stream in a video file (x264 codec on a mp4 container).

  


  I'm using ffmpeg-libav and I'm basing myself on the muxing.c example. The problem with the example is that isn't a real world scenario as frames are being generated on a while loop for a given stream duration, never missing a frame.

  


  On my program, frames are supposed to be generated at FPS, however, depending on the hardware capacity it might produce less than FPS. When I initialize the video stream context I declare that frame rate is FPS :

  


  AVRational r = { 1, FPS };
ost->st->time_base = r;


  


  This specifies that the video is going to have FPS frame rate but if less frames are produced, the playback will be faster because it will still reproduce the video as it if had all the declared frames per second.

  


  After googling a lot about this topic I understand that the key to fix this is to manipulate pts and dts but I still haven't found a solution that works.

  


  There are two key functions when writing video frames in the muxing.c example, routines that I'm using in my program :

  


  AVFrame* get_video_frame(int timestamp, OutputStream *ost, const QImage &image)
{
    /* when we pass a frame to the encoder, it may keep a reference to it
     * internally; make sure we do not overwrite it here */
    if (av_frame_make_writable(ost->frame) < 0)
        exit(1);

    av_image_fill_arrays(ost->tmp_frame->data, ost->tmp_frame->linesize, image.bits(), AV_PIX_FMT_RGBA, ost->frame->width, ost->frame->height, 8);
    libyuv::ABGRToI420(ost->tmp_frame->data[0], ost->tmp_frame->linesize[0], ost->frame->data[0], ost->frame->linesize[0], ost->frame->data[1], ost->frame->linesize[1], ost->frame->data[2], ost->frame->linesize[2], ost->tmp_frame->width, -ost->tmp_frame->height);

    #if 1 // this is my attempt to rescale pts, but crashes with ptsframe->pts = av_rescale_q(timestamp, AVRational{1, 1000}, ost->st->time_base);
    #else
    ost->frame->pts = ost->next_pts++;
    #endif

    return ost->frame;
}


  


  On the original code, the pts is simply an incremeting integer for each frame. What I'm trying to do is to pass a timestamp in ms since the beggining of the recording so that I can rescale the pts. When I rescale pts the program crashes complaining that pts is lower then dts.

  


  From what I've been reading, the pts/dts manipulation is supposed to be done at the packet level so I have also tried to manipulate things on write_frame routine without success.

  


  int write_frame(AVFormatContext *fmt_ctx, AVCodecContext *c, AVStream *st, AVFrame *frame)
{
    int ret;

    // send the frame to the encoder
    ret = avcodec_send_frame(c, frame);

    if (ret<0)
    {
        fprintf(stderr, "Error sending a frame to the encoder\n");
        exit(1);
    }

    while (ret >= 0)
    {
        AVPacket pkt = { 0 };

        ret = avcodec_receive_packet(c, &pkt);

        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
        {
            break;
        }
        else if (ret<0)
        {
            //fprintf(stderr, "Error encoding a frame: %s\n", av_err2str(ret));
            exit(1);
        }

        /* rescale output packet timestamp values from codec to stream timebase */
        av_packet_rescale_ts(&pkt, c->time_base, st->time_base);
        pkt.stream_index = st->index;

        /* Write the compressed frame to the media file. */
        //log_packet(fmt_ctx, &pkt);
        ret = av_interleaved_write_frame(fmt_ctx, &pkt);
        av_packet_unref(&pkt);

        if (ret < 0)
        {
            //fprintf(stderr, "Error while writing output packet: %s\n", av_err2str(ret));
            exit(1);
        }
    }

    return ret == AVERROR_EOF ? 1 : 0;
}


  


  How should I manipulate dts and pts so that I can achieve a video at certain frame that does not have all the frames as specified in the stream initialization ? Where should I do that manipulation ? On get_video_frame ? On write_frame ? On both ?

  


  Am I heading in the right direction ? What am I missing ?

  


• Multimedia Exploration Journal : The Past Doesn’t Die

  12 juillet 2011, par Multimedia Mike — Game Hacking

  New haul of games, new (old) multimedia formats.

  Lords of Midnight
  Check out the box copy scan for Lords of Midnight in MobyGames. In particular, I’d like to call your attention to this little blurb :

  
  
  

  Ahem, "Journey through an immense world — the equivalent of 8 CD-ROMs." Yet, when I procured the game, it only came on a single CD-ROM. It’s definitely a CD-ROM (says so on the disc) and, coming from 1995, certainly predates the earliest DVD-ROMs (which can easily store 8 CD-ROMs on a disc). Thus, I wanted to jump in a see if they were using some phenomenal compression in order to squeeze so much info into 600 or so megabytes.

  I was surprised to see the contents of the disc clocking in at just under 40 megabytes. An intro movie and an outro movie account for 75% of that. Format ? None other than that curious ASCII anomaly, ARMovie/RPL with Escape 122 codec data.

  Cyclemania

  
  
  

  Cyclemania is one of those FMV backdrop action games, but with a motorcycle theme. I had a good feeling I would find some odd multimedia artifacts here and the game didn’t disappoint. The videos are apparently handled using 3-4 discrete files per animation. I’ve documented my cursory guesses and linked some samples at the new MultimediaWiki page.

  Interplay ACMP
  This is unrelated to this particular acquistion, but I was contacted today about audio files harvested from the 1993 DOS game Star Trek : Judgment Rites. The files begin with the ASCII signature "Interplay ACMP Data". This reminds me of Interplay MVE files which begin with the similar string "Interplay MVE File". My theory is that these files use the ACOMP compression format, though I’m still trying to make it fit.

  Wiki and samples are available as usual if you’d like to add your own research.

• FFMPEG = I tried resizing a video, but got different resolution than I wanted [closed]

  10 janvier 2024, par wakanasakai

  I downloaded a video that had some black bars (left & right), so I used the following command in FFmpeg to make various changes to it. I tested it on a 10 second clip to see what the result would look like.

  


  -ss 00:04:44 -to 00:04:54 -vf "crop=1870:20:20:0","scale=640x480:flags=lanczos","eq=gamma=1.5:saturation=1.3:contrast=1.2"

  


  The original video is an mp4, with a resolution of 1920 x 1080. Besides trying to crop it & adjust the gamma, saturation, & contrast, I also tried to resize it to 640 x 480. Instead, it's resulting resolution is 44880 x 480 ! I have a link to it for anybody who wants to examine it directly. (It's only 487 kb.)
text

  


  I've tried using FFmpeg before, & it never did anything so insane. (It cropped it, & adjusted the gamma a saturation (I didn't test the contrast until THIS time), but it did not resize it at all.)

  


  Here is FFmpeg's log file for it. Guesses as to the cause of the insane result, & advice on how to achieve the DESIRED result (in 1 pass, if possible) are requested.

  


  ffmpeg -hwaccel auto -y -i "/storage/emulated/0/bluetooth/Barbie & the Rockers=1080-Out of this world (1987).mp4" -ss 00:04:44 -to 00:04:54 -vf "crop=1870:20:20:0","scale=640x480:flags=lanczos","eq=gamma=1.5:saturation=1.3:contrast=1.2" "/storage/emulated/0/Movies/Barbie.mp4"

ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 4.9.x (GCC) 20150123 (prerelease)
  configuration: --enable-version3 --enable-gpl --enable-nonfree --disable-indev=v4l2 --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libvpx --enable-libvorbis --enable-libtheora --enable-libopus --enable-libfdk-aac --enable-libfreetype --enable-libass --enable-libfribidi --enable-fontconfig --enable-pthreads --enable-libxvid --enable-filters --enable-openssl --enable-librtmp --disable-protocol='udp,udplite' --enable-libopencore-amrwb --enable-libopencore-amrnb --enable-libvo-amrwbenc --enable-libspeex --enable-libsoxr --enable-libwebp --enable-libxml2 --enable-libopenh264 --enable-jni --prefix=/home/silentlexx/AndroidstudioProjects/ffmpeg/ffmpeg/build/arm-api18-r13b --sysroot=/home/silentlexx/Android/android-ndk-r13b/platforms/android-18/arch-arm --arch=arm --disable-shared --enable-static --enable-pic --enable-ffmpeg --disable-ffplay --disable-ffprobe --disable-ffnvcodec --disable-avdevice --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-symver --cross-prefix=/home/silentlexx/Android/android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi- --target-os=android --enable-cross-compile --pkg-config-flags=--static --extra-libs='-lgnustl_static -lm -lpng -l:libz.so -lpthread' --enable-asm --enable-neon --enable-small
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/bluetooth/Barbie & the Rockers=1080-Out of this world (1987).mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: mp41isomiso2
    creation_time   : 2024-01-04T01:46:07.000000Z
  Duration: 00:45:33.10, start: 0.000000, bitrate: 3404 kb/s
  Stream #0:0[0x1](und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 3272 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0xf38cd180] using SAR=561/8
[libx264 @ 0xf38cd180] using cpu capabilities: ARMv6 NEON
[libx264 @ 0xf38cd180] profile High, level 3.0, 4:2:0, 8-bit
[libx264 @ 0xf38cd180] 264 - core 158 r2984 3759fcb - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/storage/emulated/0/Movies/Barbie.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: mp41isomiso2
    encoder         : Lavf60.3.100
  Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 640x480 [SAR 561:8 DAR 187:2], q=2-31, 30 fps, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 aac
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
frame=    0 fps=0.0 q=0.0 size=       0kB time=00:00:00.16 bitrate=   2.4kbits/s speed=0.00197x    
frame=    0 fps=0.0 q=0.0 size=       0kB time=00:00:00.71 bitrate=   0.5kbits/s speed=0.00867x    
frame=   13 fps=0.2 q=29.0 size=       0kB time=00:00:01.48 bitrate=   0.3kbits/s speed=0.0178x    
frame=   45 fps=0.5 q=29.0 size=       0kB time=00:00:02.55 bitrate=   0.2kbits/s speed=0.0304x    
frame=   78 fps=0.9 q=29.0 size=       0kB time=00:00:03.66 bitrate=   0.1kbits/s speed=0.0434x    
frame=  114 fps=1.3 q=29.0 size=       0kB time=00:00:04.85 bitrate=   0.1kbits/s speed=0.057x    
frame=  146 fps=1.7 q=29.0 size=       0kB time=00:00:05.92 bitrate=   0.1kbits/s speed=0.0692x    
frame=  178 fps=2.1 q=29.0 size=       0kB time=00:00:07.03 bitrate=   0.1kbits/s speed=0.0817x    
frame=  209 fps=2.4 q=29.0 size=     256kB time=00:00:08.03 bitrate= 261.1kbits/s speed=0.0928x    
frame=  240 fps=2.8 q=29.0 size=     256kB time=00:00:09.07 bitrate= 231.0kbits/s speed=0.104x    
frame=  300 fps=3.4 q=-1.0 Lsize=     445kB time=00:00:09.98 bitrate= 365.2kbits/s speed=0.114x    
video:275kB audio:159kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.692692%
[libx264 @ 0xf38cd180] frame I:10    Avg QP:20.34  size:  2434
[libx264 @ 0xf38cd180] frame P:129   Avg QP:21.89  size:  1292
[libx264 @ 0xf38cd180] frame B:161   Avg QP:21.69  size:   555
[libx264 @ 0xf38cd180] consecutive B-frames: 20.0% 18.7% 20.0% 41.3%
[libx264 @ 0xf38cd180] mb I  I16..4: 30.2% 66.5%  3.2%
[libx264 @ 0xf38cd180] mb P  I16..4: 14.3% 17.7%  0.2%  P16..4: 12.7%  2.7%  0.4%  0.0%  0.0%    skip:52.1%
[libx264 @ 0xf38cd180] mb B  I16..4:  2.1%  1.1%  0.0%  B16..8: 21.9%  1.7%  0.0%  direct: 1.5%  skip:71.6%  L0:46.0% L1:53.0% BI: 1.0%
[libx264 @ 0xf38cd180] 8x8 transform intra:54.9% inter:98.2%
[libx264 @ 0xf38cd180] coded y,uvDC,uvAC intra: 10.3% 14.9% 1.5% inter: 2.2% 5.4% 0.0%
[libx264 @ 0xf38cd180] i16 v,h,dc,p: 93%  2%  2%  4%
[libx264 @ 0xf38cd180] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 69%  1% 28%  0%  0%  1%  0%  0%  0%
[libx264 @ 0xf38cd180] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 76%  3% 17%  1%  1%  2%  0%  1%  0%
[libx264 @ 0xf38cd180] i8c dc,h,v,p: 45%  2% 53%  1%
[libx264 @ 0xf38cd180] Weighted P-Frames: Y:0.8% UV:0.8%
[libx264 @ 0xf38cd180] ref P L0: 57.0%  8.7% 24.0% 10.4%
[libx264 @ 0xf38cd180] ref B L0: 79.7% 17.3%  3.0%
[libx264 @ 0xf38cd180] ref B L1: 95.6%  4.4%
[libx264 @ 0xf38cd180] kb/s:224.32
[aac @ 0xf38cd880] Qavg: 457.489