Recherche avancée

Sur d’autres sites (7215)

• Why does every encoded frame's size increase after I had use to set one frame to be key in intel qsv of ffmpeg

  22 avril 2021, par TONY

  I used intel's qsv to encode h264 video in ffmpeg. My av codec context settings is like as below :

  


   m_ctx->width = m_width;
    m_ctx->height = m_height;
    m_ctx->time_base = { 1, (int)fps };
    m_ctx->qmin = 10;
    m_ctx->qmax = 35;
    m_ctx->gop_size = 3000;
    m_ctx->max_b_frames = 0;
    m_ctx->has_b_frames = false;
    m_ctx->refs = 2;
    m_ctx->slices = 0;
    m_ctx->codec_id = m_encoder->id;
    m_ctx->codec_type = AVMEDIA_TYPE_VIDEO;
    m_ctx->pix_fmt = m_h264InputFormat;
    m_ctx->compression_level = 4;
    m_ctx->flags &= ~AV_CODEC_FLAG_CLOSED_GOP;
    AVDictionary *param = nullptr;
    av_dict_set(&param, "idr_interval", "0", 0);
    av_dict_set(&param, "async_depth", "1", 0);
    av_dict_set(&param, "forced_idr", "1", 0);


  


  and in the encoding, I set the AVFrame to be AV_PICTURE_TYPE_I when key frame is needed :

  


    if(key_frame){
        encodeFrame->pict_type = AV_PICTURE_TYPE_I;
    }else{
        encodeFrame->pict_type = AV_PICTURE_TYPE_NONE;
    }
    avcodec_send_frame(m_ctx, encodeFrame);
    avcodec_receive_packet(m_ctx, m_packet);
   std::cerr<<"packet size is "<size<<",is key frame "<code>

  


  The strange phenomenon is that if I had set one frame to AV_PICTURE_TYPE_I, then every encoded frame's size after the key frame would increase. If I change the h264 encoder to x264, then it's ok.

  


  The packet size is as below before I call "encodeFrame->pict_type = AV_PICTURE_TYPE_I" :

  


  packet size is 26839
packet size is 2766
packet size is 2794
packet size is 2193
packet size is 1820
packet size is 2542
packet size is 2024
packet size is 1692
packet size is 2095
packet size is 2550
packet size is 1685
packet size is 1800
packet size is 2276
packet size is 1813
packet size is 2206
packet size is 2745
packet size is 2334
packet size is 2623
packet size is 2055


  


  If I call "encodeFrame->pict_type = AV_PICTURE_TYPE_I", then the packet size is as below :

  


  packet size is 23720,is key frame 1
packet size is 23771,is key frame 0
packet size is 23738,is key frame 0
packet size is 23752,is key frame 0
packet size is 23771,is key frame 0
packet size is 23763,is key frame 0
packet size is 23715,is key frame 0
packet size is 23686,is key frame 0
packet size is 23829,is key frame 0
packet size is 23774,is key frame 0
packet size is 23850,is key frame 0


  


• Mixing various audio and video sources into a single video

  18 février 2021, par Basj

  I've already read FFmpeg - Overlay one video onto another video ?, How to overlay 2 videos at different time over another video in single ffmpeg command ?, FFmpeg - Multiple videos with 4 areas and different play times (and many similar questions tagged [ffmpeg] about setpts), and the following code is working, but I'm sure we can simplify it, and have a more elegant solution.

  


  I'd like to mix multiple sources (image and sound) , with different starting points :

  


  t (seconds)           0   1   2   3   4   5   6   7   8   9  10  11  12  13    
test.png              [-------------------------------]
a.mp3                         [-------]
without_sound.mp4                                 [-------------------]        (overlay at x,y=200,200)
b.mp3                                     [---]
with_sound.mp4                    [---------------------------------------]    (overlay at x,y=100,100)


  


  This works :

  


  ffmpeg -i test.png 
       -t 2 -i a.mp3 
       -t 5 -i without_sound.mp4 
       -t 1 -i b.mp3 
       -t 10 -i with_sound.mp4 
       -filter_complex "
            [0]setpts=PTS-STARTPTS[s0];
            [1]adelay=2000^|2000[s1];
            [2]setpts=PTS-STARTPTS+7/TB[s2];
            [3]adelay=5000^|5000[s3];
            [4]setpts=PTS-STARTPTS+3/TB[s4];
            [4:a]adelay=3000^|3000[t4];
            [s1][s3][t4]amix=inputs=3[outa];
            [s0][s4]overlay=100:100[o2];
            [o2][s2]overlay=200:200[outv]
       " -map [outa] -map [outv]
       out.mp4 -y


  


  but :

  


  


  • is it normal that we have to use both setpts and adelay ? I have tried without adelay and then the sound is not shifted. Said differently, is there a way to simplify :

    


    [4]setpts=PTS-STARTPTS+3/TB[s4];
[4:a]adelay=3000^|3000[t4];


    


     ?

    


  


  • is there a way to do it with setpts and asetpts only ? When I replaced adelay=5000|5000 with asetpts=PTS-STARTPTS+5/TB and also for the other one, it didn't give the expected time-shifting (see below)

    


  


  • in similar questions/answers I often see overlay=...:enable='between(t,...,...)', here it seems it is not needed, why ?

    


  


  


  More generally, how would you simplify this "mix multiple audio and video" ffmpeg code ?

  


  ---

  


  More details about the second bullet point : if we replace adelay by asetpts,

  


  -filter_complex "
            [0]setpts=PTS-STARTPTS[s0];
            [1]asetpts=PTS-STARTPTS+2/TB[s1];
            [2]setpts=PTS-STARTPTS+7/TB[s2];
            [3]asetpts=PTS-STARTPTS+5/TB[s3];
            [4]setpts=PTS-STARTPTS+3/TB[s4];
            [4:a]asetpts=PTS-STARTPTS+3/TB[t4];
            [s1][s3][t4]amix=inputs=3[outa];
            [s0][s4]overlay=100:100[o2];
            [o2][s2]overlay=200:200[outv]


  


  it doesn't work : [3] should begin at 0'05", and [4:a] at 0'03" but they all begin at the same time than [1], i.e. at 0'02".

  


  It seems that amix only takes the first asetpts in consideration, and discards the others ; is it true ?

  


• ffmpeg stream input sdp shows warning keyframe missing

  27 mars 2021, par Doua Beri

  I'm using ffmpeg 4.3.2.
I'm trying to forward a stream to a rtmp server having sdp file as input

  


  Opening the sdp file with VLC everything is working great. The same thing happens when I use ffplay

  


  ffplay -i rtp-forwarder.sdp -protocol_whitelist file,udp,rtp


  


  The problem starts when I start streaming to a rtmp server. The audio is good but the video is just a black screen. I event tried to stream to youtube rtmp server but it didn't work.

  


  I'm new to ffmpeg. Let me know if I'm missing something.

  


  I'm using this command

  


  ffmpeg -protocol_whitelist file,crypto,udp,rtp -re -i rtp-forwarder.sdp -c:v libx264 -b:v 3000k -maxrate 3000k -bufsize 6000k -pix_fmt yuv420p -g 50 -c:a aac -b:a 160k -ac 2 -ar 44100 -f flv rtmp://localhost/live/test


  


  The sdp file content is like this

  


  v=0
o=- 0 0 IN IP4 192.168.1.49
s=Pion WebRTC
c=IN IP4 192.168.1.49
t=0 0
m=audio 4000 RTP/AVP 111
a=rtpmap:111 OPUS/48000/2
m=video 4002 RTP/AVP 96
a=rtpmap:96 VP8/90000


  


  Here is a full log

  


    libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[sdp @ 0000015a81ede6c0] Keyframe missing
Input #0, sdp, from 'rtp-forwarder.sdp':
  Metadata:
    title           : Pion WebRTC
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
    Stream #0:1: Video: vp8, yuv420p(tv, bt470bg/unknown/unknown), 640x480, 90k tbr, 90k tbn, 90k tbc
Stream mapping:
  Stream #0:1 -> #0:0 (vp8 (native) -> h264 (libx264))
  Stream #0:0 -> #0:1 (opus (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0000015a81f9e880] MB rate (108000000) > level limit (16711680) -0.0kbits/s speed=N/A
[libx264 @ 0000015a81f9e880] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000015a81f9e880] profile High, level 6.2, 4:2:0, 8-bit
[libx264 @ 0000015a81f9e880] 264 - core 161 r3048 b86ae3c - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=50 keyint_min=5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=cbr mbtree=1 bitrate=3000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=3000 vbv_bufsize=6000 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
Output #0, flv, to 'rtmp://localhost/live/test':
  Metadata:
    title           : Pion WebRTC
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 640x480, q=-1--1, 3000 kb/s, 90k fps, 1k tbn, 90k tbc
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 3000000/0/3000000 buffer size: 6000000 vbv_delay: N/A
    Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, 160 kb/s
    Metadata:
      encoder         : Lavc58.91.100 aac
[flv @ 0000015a820dd200] Failed to update header with correct duration..1kbits/s speed=   1x
[flv @ 0000015a820dd200] Failed to update header with correct filesize.
frame=   13 fps=0.1 q=-1.0 Lsize=    4233kB time=00:03:07.12 bitrate= 185.3kbits/s speed=0.999x
video:440kB audio:3658kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.281771%
[libx264 @ 0000015a81f9e880] frame I:1     Avg QP: 1.61  size: 59263
[libx264 @ 0000015a81f9e880] frame P:3     Avg QP: 9.10  size: 44500
[libx264 @ 0000015a81f9e880] frame B:9     Avg QP: 9.80  size: 28605
[libx264 @ 0000015a81f9e880] consecutive B-frames:  7.7%  0.0%  0.0% 92.3%
[libx264 @ 0000015a81f9e880] mb I  I16..4: 36.5% 19.5% 44.0%
[libx264 @ 0000015a81f9e880] mb P  I16..4: 10.7% 50.3% 12.3%  P16..4: 11.6%  9.6%  5.5%  0.0%  0.0%    skip: 0.1%
[libx264 @ 0000015a81f9e880] mb B  I16..4:  2.2% 10.4%  3.6%  B16..8: 45.7% 21.2%  6.1%  direct:10.6%  skip: 0.1%  L0:60.0% L1:25.5% BI:14.5%
[libx264 @ 0000015a81f9e880] final ratefactor: 13.03
[libx264 @ 0000015a81f9e880] 8x8 transform intra:56.7% inter:41.9%
[libx264 @ 0000015a81f9e880] coded y,uvDC,uvAC intra: 90.9% 99.6% 98.9% inter: 66.1% 99.0% 93.2%
[libx264 @ 0000015a81f9e880] i16 v,h,dc,p: 23% 22% 18% 38%
[libx264 @ 0000015a81f9e880] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 26% 23%  4%  2%  3%  3%  4%  4%
[libx264 @ 0000015a81f9e880] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 36% 34% 18%  2%  2%  2%  2%  2%  2%
[libx264 @ 0000015a81f9e880] i8c dc,h,v,p: 55% 29% 11%  5%
[libx264 @ 0000015a81f9e880] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000015a81f9e880] ref P L0: 59.2%  5.0% 13.4% 22.4%
[libx264 @ 0000015a81f9e880] ref B L0: 87.1%  7.5%  5.4%
[libx264 @ 0000015a81f9e880] ref B L1: 95.1%  4.9%
[libx264 @ 0000015a81f9e880] kb/s:4028.74
[aac @ 0000015a81f9a880] Qavg: 7535.380


  


  I get this warning [sdp @ 0000015a81ede6c0] Keyframe missing
and this one after a stop it

  


  [flv @ 0000015a820dd200] Failed to update header with correct duration..1kbits/s speed=   1x
[flv @ 0000015a820dd200] Failed to update header with correct filesize.