Recherche avancée

Sur d’autres sites (10133)

• Fixing A/V sync issues with RTP/RTCP sent from mediasoup

  10 décembre 2017, par artushin

  A little background : I’m attempting to record a webrtc call being made through the mediasoup v2 SFU. I’m using mediasoup’s room.createRtpStreamer() method to generate a stream which mirrors RTP/RTCP to ffmpeg. Two streamers are created for audio and video within 30ms of each other and begin broadcasting. FFmpeg then spins up and starts accepting. Pretty sure RTCP is working since ffmpeg is always starting with a keyframe despite being started after the streamer begins broadcasting.

  The problem is that I encounter audio/video desynchronization with seemingly random offsets. My current theory is that this offset is based on how old the last keyframe is that RTCP requests to start the stream. See below for ffmpeg configuration and output but my question is : what ffmpeg arguments can I use to adjust the video frame timestamps to match the audio frame timestamps or vice versa ? I’ve messed around with -map 0:0,0:1 -map 0:1,0:1 but it doesn’t seem to do what I’m looking for.

  ffmpeg flags :

  '-y',
  
  '-loglevel',
  
  'debug',
  
  '-dump',
  
  '-protocol_whitelist',
  
  'file,crypto,udp,rtp,data',
  
  '-analyzeduration',
  
  '20M',
  
  '-probesize',
  
  '20M',
  
  '-i',
  
  `data:text/plain;base64,${sdp.toString('base64')}`,
  
  '-fflags',
  
  '+genpts',
  
  '-vcodec',
  
  'copy',
  
  '-acodec',
  
  'aac',
  
  '-bsf:v',
  
  'h264_mp4toannexb',
  
  '-start_number',
  
  '0',
  
  '-hls_list_size',
  
  '2147480000',
  
  '-hls_wrap',
  
  '0',
  
  '-hls_time',
  
  '10',

  SDP used for input (template) :

  v=0
  
  o=- 0 0 IN IP4 <%=ip %>
  
  s=title
  
  c=IN IP4 <%=ip %>
  
  m=audio <%=audioPort %> RTP/AVPF <%=audioPayload %>
  
  a=sendrecv
  
  a=rtcp-mux
  
  a=rtpmap:<%=audioPayload %> opus/48000/2
  
  a=fmtp:<%=audioPayload %> minptime=10; useinbandfec=1
  
  m=video <%=videoPort %> RTP/AVPF <%=videoPayload %>
  
  a=sendrecv
  
  a=rtcp-mux
  
  a=rtpmap:<%=videoPayload %> H264/90000
  
  a=rtcp-fb:<%=videoPayload %> ccm fir
  
  a=rtcp-fb:<%=videoPayload %> nack
  
  a=rtcp-fb:<%=videoPayload %> nack pli
  
  a=rtcp-fb:<%=videoPayload %> goog-remb
  
  a=rtcp-fb:<%=videoPayload %> transport-cc
  
  a=fmtp:<%=videoPayload %> level-asymmetry-allowed=1;packetization-mode=1

  ffmpeg output - garbled with some timestamps

  1512775954585 - stderr: ffmpeg version 3.4 Copyright (c) 2000-2017 the FFmpeg developers
  
    built with Apple LLVM version 9.0.0 (clang-900.0.38)
  
    configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-ffplay --enable-libmp3lame --enable-libvpx --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  
  1512775954587 - stderr:   libavutil      55. 78.100 / 55. 78.100
  
    libavcodec     57.107.100 / 57.107.100
  
    libavformat    57. 83.100 / 57. 83.100
  
    libavdevice    57. 10.100 / 57. 10.100
  
  1512775954587 - stderr:   libavfilter     6.107.100 /  6.107.100
  
    libavresample   3.  7.  0 /  3.  7.  0
  
    libswscale      4.  8.100 /  4.  8.100
  
    libswresample   2.  9.100 /  2.  9.100
  
    libpostproc    54.  7.100 / 54.  7.100
  
  Splitting the commandline.
  
  Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
  
  Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
  
  Reading option '-dump' ... matched as option 'dump' (dump each input packet) with argument '1'.
  
  Reading option '-protocol_whitelist' ...1512775954589 - stderr:  matched as AVOption 'protocol_whitelist' with argument 'file,crypto,udp,rtp,data'.
  
  Reading option '-analyzeduration' ...1512775954590 - stderr:  matched as AVOption 'analyzeduration' with argument '20M'.
  
  Reading option '-probesize' ...1512775954590 - stderr:  matched as AVOption 'probesize' with argument '20M'.
  
  Reading option '-i' ... matched as input url with argument 'data:text/plain;base64,dj0wCm89LSAwIDAgSU4gSVA0IDEyNy4wLjAuMQpzPWUwYzkyZmEwLWRhZDUtMTFlNy04Njg3LTA5MGRkYTk1YjFhNCBmb29ib2FyCmM9SU4gSVA0IDEyNy4wLjAuMQptPWF1ZGlvIDIwMDAwIFJUUC9BVlBGIDEwMAphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAwIG9wdXMvNDgwMDAvMgphPWZtdHA6MTAwIG1pbnB0aW1lPTEwOyB1c2VpbmJhbmRmZWM9MQptPXZpZGVvIDIwMDAyIFJUUC9BVlBGIDEwMQphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAxIEgyNjQvOTAwMDAKYT1ydGNwLWZiOjEwMSBjY20gZmlyCmE9cnRjcC1mYjoxMDEgbmFjawphPXJ0Y3AtZmI6MTAxIG5hY2sgcGxpCmE9cnRjcC1mYjoxMDEgZ29vZy1yZW1iCmE9cnRjcC1mYjoxMDEgdHJhbnNwb3J0LWNjCmE9Zm10cDoxMDEgbGV2ZWwtYXN5bW1ldHJ5LWFsbG93ZWQ9MTtwYWNrZXRpemF0aW9uLW1vZGU9MTtwcm9maWxlLWxldmVsLWlkPTQyZTAxZgo='.
  
  Reading option '-fflags' ...1512775954591 - stderr:  matched as AVOption 'fflags' with argument '+genpts'.
  
  Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'copy'.
  
  Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'aac'.
  
  Reading option '-vsync' ... matched as option 'vsync' (video sync method) with argument '0'.
  
  Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:0,0:1'.
  
  Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:1,0:1'.
  
  Reading option '-bsf:v' ... matched as option 'bsf' (A comma-separated list of bitstream filters) with argument 'h264_mp4toannexb'.
  
  Reading option '-start_number' ...1512775954591 - stderr:  matched as AVOption 'start_number' with argument '0'.
  
  Reading option '-hls_list_size' ...1512775954591 - stderr:  matched as AVOption 'hls_list_size' with argument '2147480000'.
  
  Reading option '-hls_wrap' ... matched as AVOption 'hls_wrap' with argument '0'.
  
  Reading option '-hls_time' ...1512775954591 - stderr:  matched as AVOption 'hls_time' with argument '10'.
  
  Reading option '/tmp/archive/e0c92fa0-dad5-11e7-8687-090dda95b1a4_10e1c990-dc70-11e7-888d-9f39ca0c79bc/1512775954465.m3u8' ... matched as output url.
  
  Finished splitting the commandline.
  
  Parsing a group of options: global .
  
  Applying option y (overwrite output files) with argument 1.
  
  1512775954592 - stderr: Applying option loglevel (set logging level) with argument debug.
  
  Applying option dump (dump each input packet) with argument 1.
  
  Applying option vsync (video sync method) with argument 0.
  
  Successfully parsed a group of options.
  
  Parsing a group of options: input url data:text/plain;base64,dj0wCm89LSAwIDAgSU4gSVA0IDEyNy4wLjAuMQpzPWUwYzkyZmEwLWRhZDUtMTFlNy04Njg3LTA5MGRkYTk1YjFhNCBmb29ib2FyCmM9SU4gSVA0IDEyNy4wLjAuMQptPWF1ZGlvIDIwMDAwIFJUUC9BVlBGIDEwMAphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAwIG9wdXMvNDgwMDAvMgphPWZtdHA6MTAwIG1pbnB0aW1lPTEwOyB1c2VpbmJhbmRmZWM9MQptPXZpZGVvIDIwMDAyIFJUUC9BVlBGIDEwMQphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAxIEgyNjQvOTAwMDAKYT1ydGNwLWZiOjEwMSBjY20gZmlyCmE9cnRjcC1mYjoxMDEgbmFjawphPXJ0Y3AtZmI6MTAxIG5hY2sgcGxpCmE9cnRjcC1mYjoxMDEgZ29vZy1yZW1iCmE9cnRjcC1mYjoxMDEgdHJhbnNwb3J0LWNjCmE9Zm10cDoxMDEgbGV2ZWwtYXN5bW1ldHJ5LWFsbG93ZWQ9MTtwYWNrZXRpemF0aW9uLW1vZGU9MTtwcm9maWxlLWxldmVsLWlkPTQyZTAxZgo=.
  
  Successfully parsed a group of options.
  
  Opening an input file: data:text/plain;base64,dj0wCm89LSAwIDAgSU4gSVA0IDEyNy4wLjAuMQpzPWUwYzkyZmEwLWRhZDUtMTFlNy04Njg3LTA5MGRkYTk1YjFhNCBmb29ib2FyCmM9SU4gSVA0IDEyNy4wLjAuMQptPWF1ZGlvIDIwMDAwIFJUUC9BVlBGIDEwMAphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAwIG9wdXMvNDgwMDAvMgphPWZtdHA6MTAwIG1pbnB0aW1lPTEwOyB1c2VpbmJhbmRmZWM9MQptPXZpZGVvIDIwMDAyIFJUUC9BVlBGIDEwMQphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAxIEgyNjQvOTAwMDAKYT1ydGNwLWZiOjEwMSBjY20gZmlyCmE9cnRjcC1mYjoxMDEgbmFjawphPXJ0Y3AtZmI6MTAxIG5hY2sgcGxpCmE9cnRjcC1mYjoxMDEgZ29vZy1yZW1iCmE9cnRjcC1mYjoxMDEgdHJhbnNwb3J0LWNjCmE9Zm10cDoxMDEgbGV2ZWwtYXN5bW1ldHJ5LWFsbG93ZWQ9MTtwYWNrZXRpemF0aW9uLW1vZGU9MTtwcm9maWxlLWxldmVsLWlkPTQyZTAxZgo=.
  
  1512775954592 - stderr: [NULL @ 0x7f81fd000000] Opening 'data:text/plain;base64,dj0wCm89LSAwIDAgSU4gSVA0IDEyNy4wLjAuMQpzPWUwYzkyZmEwLWRhZDUtMTFlNy04Njg3LTA5MGRkYTk1YjFhNCBmb29ib2FyCmM9SU4gSVA0IDEyNy4wLjAuMQptPWF1ZGlvIDIwMDAwIFJUUC9BVlBGIDEwMAphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAwIG9wdXMvNDgwMDAvMgphPWZtdHA6MTAwIG1pbnB0aW1lPTEwOyB1c2VpbmJhbmRmZWM9MQptPXZpZGVvIDIwMDAyIFJUUC9BVlBGIDEwMQphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAxIEgyNjQvOTAwMDAKYT1ydGNwLWZiOjEwMSBjY20gZmlyCmE9cnRjcC1mYjoxMDEgbmFjawphPXJ0Y3AtZmI6MTAxIG5hY2sgcGxpCmE9cnRjcC1mYjoxMDEgZ29vZy1yZW1iCmE9cnRjcC1mYjoxMDEgdHJhbnNwb3J0LWNjCmE9Zm10cDoxMDEgbGV2ZWwtYXN5bW1ldHJ5LWFsbG93ZWQ9MTtwYWNrZXRpemF0aW9uLW1vZGU9MTtwcm9maWxlLWxldmVsLWlkPTQyZTAxZgo=' for reading
  
  1512775954593 - stderr: [data @ 0x7f81fca001a0] Content-type: text/plain
  
  {"level":"info","time":"Dec 8, 2017 11:32 PM","message":"ffmpeg started"}
  
  1512775954595 - stderr: [sdp @ 0x7f81fd000000] Format sdp probed with size=2048 and score=50
  
  1512775954598 - stderr: [sdp @ 0x7f81fd000000] audio codec set to: opus
  
  [sdp @ 0x7f81fd000000] audio samplerate set to: 48000
  
  [sdp @ 0x7f81fd000000] audio channels set to: 2
  
  1512775954639 - stderr: [sdp @ 0x7f81fd000000] video codec set to: h264
  
  [sdp @ 0x7f81fd000000] RTP Packetization Mode: 1
  
  [sdp @ 0x7f81fd000000] RTP Profile IDC: 42 Profile IOP: e0 Level: 1f
  
  [udp @ 0x7f81fcb007e0] end receive buffer size reported is 65536
  
  [udp @ 0x7f81fbe00180] end receive buffer size reported is 65536
  
  [sdp @ 0x7f81fd000000] setting jitter buffer size to 500
  
  [udp @ 0x7f81fbe00680] end receive buffer size reported is 65536
  
  [udp @ 0x7f81fbe00740] end receive buffer size reported is 65536
  
  [sdp @ 0x7f81fd000000] setting jitter buffer size to 500
  
  [sdp @ 0x7f81fd000000] Before avformat_find_stream_info() pos: 479 bytes read:479 seeks:0 nb_streams:2
  
  {"level":"info","time":"Dec 8, 2017 11:32 PM","message":"new active speaker","activePeer":"9f05d96a-9641-4c63-8f0e-486b98e48eb5"}
  
  1512775954773 - stderr: [AVBSFContext @ 0x7f81fc8018e0] nal_unit_type: 7, nal_ref_idc: 3
  
  [AVBSFContext @ 0x7f81fc8018e0] nal_unit_type: 8, nal_ref_idc: 3
  
  [AVBSFContext @ 0x7f81fc8018e0] nal_unit_type: 5, nal_ref_idc: 3
  
  1512775954774 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 7, nal_ref_idc: 3
  
  1512775954774 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 8, nal_ref_idc: 3
  
  1512775954774 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 5, nal_ref_idc: 3
  
  1512775954774 - stderr: [h264 @ 0x7f8200000c00] Reinit context to 640x480, pix_fmt: yuv420p
  
  1512775954801 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 1, nal_ref_idc: 3
  
  1512775954939 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 7, nal_ref_idc: 3
  
  1512775954939 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 8, nal_ref_idc: 3
  
  1512775954940 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 5, nal_ref_idc: 3
  
  1512775955003 - stderr: [h264 @ 0x7f8200000c00] nal_unit_type: 1, nal_ref_idc: 3
  
  1512775955999 - stderr:     Last message repeated 3 times
  
  [sdp @ 0x7f81fd000000] All info found
  
  1512775955999 - stderr: [sdp @ 0x7f81fd000000] rfps: 29.750000 0.019566
  
  [sdp @ 0x7f81fd000000] rfps: 29.833333 0.015263
  
  [sdp @ 0x7f81fd000000] rfps: 29.916667 0.011503
  
  1512775955999 - stderr: [sdp @ 0x7f81fd000000] rfps: 30.000000 0.008285
  
  [sdp @ 0x7f81fd000000] rfps: 31.000000 0.011990
  
      Last message repeated 1 times
  
  [sdp @ 0x7f81fd000000] rfps: 29.970030 0.009380
  
  [sdp @ 0x7f81fd000000] After avformat_find_stream_info() pos: 479 bytes read:479 seeks:0 frames:98
  
  1512775956000 - stderr: Input #0, sdp, from 'data:text/plain;base64,dj0wCm89LSAwIDAgSU4gSVA0IDEyNy4wLjAuMQpzPWUwYzkyZmEwLWRhZDUtMTFlNy04Njg3LTA5MGRkYTk1YjFhNCBmb29ib2FyCmM9SU4gSVA0IDEyNy4wLjAuMQptPWF1ZGlvIDIwMDAwIFJUUC9BVlBGIDEwMAphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAwIG9wdXMvNDgwMDAvMgphPWZtdHA6MTAwIG1pbnB0aW1lPTEwOyB1c2VpbmJhbmRmZWM9MQptPXZpZGVvIDIwMDAyIFJUUC9BVlBGIDEwMQphPXNlbmRyZWN2CmE9cnRjcC1tdXgKYT1ydHBtYXA6MTAxIEgyNjQvOTAwMDAKYT1ydGNwLWZiOjEwMSBjY20gZmlyCmE9cnRjcC1mYjoxMDEgbmFjawphPXJ0Y3AtZmI6MTAxIG5hY2sgcGxpCmE9cnRjcC1mYjoxMDEgZ29vZy1yZW1iCmE9cnRjcC1mYjoxMDEgdHJhbnNwb3J0LWNjCmE9Zm10cDoxMDEgbGV2ZWwtYXN5bW1ldHJ5LWFsbG93ZWQ9MTtwYWNrZXRpemF0aW9uLW1vZGU9MTtwcm9maWxlLWxldmVsLWlkPTQyZTAxZgo=':
  
  1512775956000 - stderr:   Metadata:
  
      title           : e0c92fa0-dad5-11e7-8687-090dda95b1a4 fooboar
  
    Duration: N/A, start: 0.000000, bitrate: N/A
  
      Stream #0:0, 70, 1/48000: Audio: opus, 48000 Hz, stereo, fltp1512775956000 - stderr:
  
      Stream #0:1, 28, 1/90000: Video: h264 (Constrained Baseline), 1 reference frame, yuv420p(progressive, left), 640x480, 0/1, 30 tbr, 90k tbn, 180k tbc
  
  1512775956000 - stderr: Successfully opened the file.
  
  Parsing a group of options: output url /tmp/archive/e0c92fa0-dad5-11e7-8687-090dda95b1a4_10e1c990-dc70-11e7-888d-9f39ca0c79bc/1512775954465.m3u8.
  
  Applying option vcodec (force video codec ('copy' to copy stream)) with argument copy.
  
  Applying option acodec (force audio codec ('copy' to copy stream)) with argument aac.
  
  Applying option map (set input stream mapping) with argument 0:0,0:1.
  
  Applying option map (set input stream mapping) with argument 0:1,0:1.
  
  Applying option bsf:v (A comma-separated list of bitstream filters) with argument h264_mp4toannexb.
  
  Successfully parsed a group of options.
  
  1512775956000 - stderr: Opening an output file: /tmp/archive/e0c92fa0-dad5-11e7-8687-090dda95b1a4_10e1c990-dc70-11e7-888d-9f39ca0c79bc/1512775954465.m3u8.
  
  1512775956000 - stderr: Successfully opened the file.
  
  1512775956001 - stderr: [AVBSFContext @ 0x7f81fcb020a0] The input looks like it is Annex B already
  
  Stream mapping:
  
  1512775956001 - stderr:   Stream #0:0 -> #0:0 [sync #0:1] (opus (native) -> aac (native))
  
    Stream #0:1 -> #0:1 (copy)
  
  Press [q] to stop, [?] for help
  
  cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.000  pts=0.000
  
    size=82
  
  1512775956001 - stderr: [SWR @ 0x7f820001e600] Using fltp internally between filters
  
  1512775956002 - stderr: detected 8 logical cores
  
  1512775956003 - stderr: [graph_0_in_0_0 @ 0x7f81fc90dfc0] Setting 'time_base' to value '1/48000'
  
  [graph_0_in_0_0 @ 0x7f81fc90dfc0] Setting 'sample_rate' to value '48000'
  
  1512775956003 - stderr: [graph_0_in_0_0 @ 0x7f81fc90dfc0] Setting 'sample_fmt' to value 'fltp'
  
  [graph_0_in_0_0 @ 0x7f81fc90dfc0] Setting 'channel_layout' to value '0x3'
  
  [graph_0_in_0_0 @ 0x7f81fc90dfc0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
  
  [format_out_0_0 @ 0x7f81fc914da0] Setting 'sample_fmts' to value 'fltp'
  
  [format_out_0_0 @ 0x7f81fc914da0] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
  
  1512775956004 - stderr: [AVFilterGraph @ 0x7f81fbd02200] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
  
  1512775956007 - stderr: [hls @ 0x7f81fd80f000] Opening '/tmp/archive/e0c92fa0-dad5-11e7-8687-090dda95b1a4_10e1c990-dc70-11e7-888d-9f39ca0c79bc/15127759544650.ts' for writing
  
  [file @ 0x7f81fbf01ee0] Setting default whitelist 'file,crypto'
  
  1512775956007 - stderr: [mpegts @ 0x7f81fd877800] muxrate VBR, pcr every 9000 pkts, sdt every 2147483647, pat/pmt every 2147483647 pkts
  
  Output #0, hls, to '/tmp/archive/e0c92fa0-dad5-11e7-8687-090dda95b1a4_10e1c990-dc70-11e7-888d-9f39ca0c79bc/1512775954465.m3u8':
  
    Metadata:
  
      title           : e0c92fa0-dad5-11e7-8687-090dda95b1a4 fooboar
  
      encoder         : Lavf57.83.100
  
  1512775956007 - stderr:     Stream #0:0, 0, 1/90000: Audio: aac (LC), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.107.100 aac
  
      Stream #0:1, 0, 1/90000: Video: h264 (Constrained Baseline), 1 reference frame, yuv420p(progressive, left), 640x480 (0x0), 0/1, q=2-31, 30 tbr, 90k tbn, 90k tbc
  
  cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  1512775956007 - stderr:     Last message repeated 1 times
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.020  pts=0.020
  
    size=79
  
  1512775956007 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
      Last message repeated 1 times
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
  1512775956007 - stderr:   dts=0.040  pts=0.040
  
    size=75
  
  1512775956014 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.0601512775956014 - stderr:   pts=0.060
  
    size=81
  
  1512775956017 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.080  pts=0.080
  
    size=76
  
  1512775956022 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.1001512775956022 - stderr:   pts=0.100
  
    size=79
  
  1512775956023 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  stream #0:
  
    keyframe=1
  
  1512775956023 - stderr:   duration=0.000
  
    dts=0.120  pts=0.120
  
    size=95
  
  1512775956024 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  1512775956024 - stderr: stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.140  pts=0.140
  
    size=93
  
  1512775956025 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  stream #0:
  
    keyframe=1
  
  1512775956025 - stderr:   duration=0.000
  
    dts=0.160  pts=0.160
  
    size=94
  
  1512775956026 - stderr: cur_dts is invalid (this is harmless if it occurs once at the start per stream)
  
  stream #1:
  
    keyframe=1
  
  1512775956026 - stderr:   duration=0.000
  
    dts=N/A  pts=N/A
  
    size=992
  
  [hls @ 0x7f81fd80f000] Timestamps are unset in a packet for stream 1. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
  
  1512775956026 - stderr: stream #0:
  
    keyframe=1
  
    duration=0.000
  
  1512775956026 - stderr:   dts=0.180  pts=0.180
  
    size=108
  
  1512775956027 - stderr: stream #1:
  
    keyframe=0
  
    duration=0.000
  
  1512775956027 - stderr:   dts=0.002  pts=0.002
  
    size=3047
  
  [hls @ 0x7f81fd80f000] pkt->duration = 0, maybe the hls segment duration will not precise
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.200  pts=0.200
  
    size=89
  
  1512775956060 - stderr: stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.220  pts=0.220
  
    size=73
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.240  pts=0.240
  
    size=78
  
  stream #0:
  
    keyframe=1
  
    duration=0.000
  
    dts=0.260  pts=0.260

  Notice how the first frame for stream #1 (video) starts after a number of audio frames ? Specifically, it starts at stream #0 dts/pts 0.18. In this situation, the a/v sync issue is pretty much unnoticeable, but with a bunch of repros, I’ve determined that the a/v sync offset is always the duration of however long audio frames were sent before the first video frame (sometimes seconds). I’m consistently starting the RTP streams only tens of ms apart, so I can’t control for this variance on the input side.

  After the initial audio frames come in, the first video frame has a dts/pts around 0. What ffmpeg setting would I use to adjust the timestamps accordingly ? I don’t care about losing the starting audio that doesn’t have video, so any solution that would adjust the timestamps works.

• How i can add logo on any location on ffmpeg stream ? This is my command :

  9 décembre 2017, par Jond David

  ffmpeg -y -i input.mp4 -i "logo2.png" -filter_complex "[0:v]setpts=PTS/1.15,boxblur=2:1,scale=iw/1.75 :-1,pad=iw+26:ih+26:13:13:color=blue [v1] ; movie=bgmu.mp4:loop=999,setpts=N/(FRAME_RATE*TB) [v2] ; [v2][v1]overlay=shortest=1:x=W-w-30:y=H-h-22 [v3] ; [v3][1:v]overlay=0:0,setdar=16/9 ; [0:a]atempo=1.15, aecho=0.4:0.66:2:0.2, chorus=0.5:0.9:50|80:0.4|0.42:0.25|0.4:2|1.4, firequalizer=gain_entry=’entry(100,0) ; entry(400, -4) ; entry(1000, -6) ; entry(2000, 0)’,equalizer = f = 1000 : width_type = q : width = 1 : g = 2, equalizer = f = 100 : width_type = q : width = 2 : g = 5,pan=stereo|c0

• ffmpeg - merge mp3 and mp4 (duration difference)

  8 décembre 2017, par dfionov

  I’m trying to merge mp4 and mp3 files with ffmpeg.
  mp4 duration - 9.800 sec, mp3 - 58.540 sec. So i using -shortest key.
  Code :

  ffmpeg -i video.mp4 -i audio.mp3 -c:v libx264 -c:a aac -strict experimental -shortest output.mp4

  After that i got output.mp4 with duration 9.846. Where is my error ? Why output video longer than source ? (9.846 sec and 9.800 sec).

  Source mp4 MediaInfo :

  General
  
  Complete name                  : F:\video test\video.mp4
  
  Format                         : MPEG-4
  
  Format profile                 : Base Media
  
  Codec ID                       : iso5 (iso5/dash)
  
  File size                      : 3.19 MiB
  
  Duration                       : 9 s 800 ms
  
  Overall bit rate               : 2 732 kb/s
  
  Encoded date                   : UTC 2017-11-24 20:53:53
  
  Tagged date                    : UTC 2017-11-24 20:53:53
  
  
  
  Video
  
  ID                             : 1
  
  Format                         : AVC
  
  Format/Info                    : Advanced Video Codec
  
  Format profile                 : High@L3.1
  
  Format settings                : CABAC / 4 Ref Frames
  
  Format settings, CABAC         : Yes
  
  Format settings, ReFrames      : 4 frames
  
  Codec ID                       : avc1
  
  Codec ID/Info                  : Advanced Video Coding
  
  Duration                       : 9 s 800 ms
  
  Bit rate                       : 2 729 kb/s
  
  Maximum bit rate               : 3 766 kb/s
  
  Width                          : 1 280 pixels
  
  Height                         : 720 pixels
  
  Display aspect ratio           : 16:9
  
  Frame rate mode                : Constant
  
  Frame rate                     : 25.000 FPS
  
  Color space                    : YUV
  
  Chroma subsampling             : 4:2:0
  
  Bit depth                      : 8 bits
  
  Scan type                      : Progressive
  
  Bits/(Pixel*Frame)             : 0.118
  
  Stream size                    : 3.19 MiB (100%)
  
  Writing library                : x264 core 146
  
  Encoding settings              : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
  
  Tagged date                    : UTC 2017-11-24 20:53:53

  Source mp3 Mediainfo :

  General
  
  Complete name                  : F:\video test\audio.mp3
  
  Format                         : MPEG Audio
  
  File size                      : 1.19 MiB
  
  Duration                       : 58 s 540 ms
  
  Overall bit rate mode          : Variable
  
  Overall bit rate               : 170 kb/s
  
  Writing library                : LAME3.99r
  
  
  
  Audio
  
  Format                         : MPEG Audio
  
  Format version                 : Version 1
  
  Format profile                 : Layer 3
  
  Format settings                : Joint stereo / MS Stereo
  
  Duration                       : 58 s 540 ms
  
  Bit rate mode                  : Variable
  
  Bit rate                       : 170 kb/s
  
  Minimum bit rate               : 32.0 kb/s
  
  Channel(s)                     : 2 channels
  
  Sampling rate                  : 44.1 kHz
  
  Frame rate                     : 38.281 FPS (1152 SPF)
  
  Compression mode               : Lossy
  
  Stream size                    : 1.19 MiB (100%)
  
  Writing library                : LAME3.99r
  
  Encoding settings              : -m j -V 2 -q 0 -lowpass 18.5 --vbr-new -b 32

  Console output :

  ffmpeg version 3.4 Copyright (c) 2000-2017 the FFmpeg developers
  
    built with gcc 7.2.0 (GCC)
  
    configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  
    libavutil      55. 78.100 / 55. 78.100
  
    libavcodec     57.107.100 / 57.107.100
  
    libavformat    57. 83.100 / 57. 83.100
  
    libavdevice    57. 10.100 / 57. 10.100
  
    libavfilter     6.107.100 /  6.107.100
  
    libswscale      4.  8.100 /  4.  8.100
  
    libswresample   2.  9.100 /  2.  9.100
  
    libpostproc    54.  7.100 / 54.  7.100
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
  
    Metadata:
  
      major_brand     : iso5
  
      minor_version   : 1
  
      compatible_brands: iso5dash
  
      creation_time   : 2017-11-24T20:53:53.000000Z
  
    Duration: 00:00:09.80, start: 0.000000, bitrate: 2732 kb/s
  
      Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 2259 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
  
      Metadata:
  
        handler_name    : VideoHandler
  
  Input #1, mp3, from 'audio.mp3':
  
    Duration: 00:00:58.54, start: 0.025057, bitrate: 170 kb/s
  
      Stream #1:0: Audio: mp3, 44100 Hz, stereo, s16p, 170 kb/s
  
      Metadata:
  
        encoder         : LAME3.99r
  
      Side data:
  
        replaygain: track gain - -2.200000, track peak - unknown, album gain - unknown, album peak - unknown, 
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  
    Stream #1:0 -> #0:1 (mp3 (native) -> aac (native))
  
  Press [q] to stop, [?] for help
  
  [libx264 @ 00000000005ab440] using SAR=1/1
  
  [libx264 @ 00000000005ab440] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
  
  [libx264 @ 00000000005ab440] profile High, level 3.1
  
  [libx264 @ 00000000005ab440] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  Output #0, mp4, to 'output.mp4':
  
    Metadata:
  
      major_brand     : iso5
  
      minor_version   : 1
  
      compatible_brands: iso5dash
  
      encoder         : Lavf57.83.100
  
      Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
  
      Metadata:
  
        handler_name    : VideoHandler
  
        encoder         : Lavc57.107.100 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
      Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.107.100 aac
  
      Side data:
  
        replaygain: track gain - -2.200000, track peak - unknown, album gain - unknown, album peak - unknown, 
  
  frame=   54 fps=0.0 q=28.0 size=       0kB time=00:00:00.04 bitrate=   8.3kbits/s speed=0.0927x    
  
  frame=   80 fps= 80 q=28.0 size=       0kB time=00:00:01.09 bitrate=   0.4kbits/s speed=1.09x    
  
  frame=   98 fps= 65 q=28.0 size=     256kB time=00:00:01.83 bitrate=1143.5kbits/s speed=1.21x    
  
  frame=  119 fps= 59 q=28.0 size=     512kB time=00:00:02.67 bitrate=1570.9kbits/s speed=1.32x    
  
  frame=  144 fps= 56 q=28.0 size=     768kB time=00:00:03.66 bitrate=1715.0kbits/s speed=1.42x    
  
  frame=  167 fps= 52 q=28.0 size=    1024kB time=00:00:04.57 bitrate=1833.9kbits/s speed=1.44x    
  
  frame=  190 fps= 51 q=28.0 size=    1280kB time=00:00:05.50 bitrate=1905.5kbits/s speed=1.47x    
  
  frame=  218 fps= 51 q=28.0 size=    1792kB time=00:00:06.64 bitrate=2210.6kbits/s speed=1.56x    
  
  frame=  242 fps= 50 q=28.0 size=    2048kB time=00:00:07.56 bitrate=2216.4kbits/s speed=1.58x    
  
  frame=  245 fps= 41 q=-1.0 Lsize=    3045kB time=00:00:09.82 bitrate=2539.6kbits/s speed=1.65x    
  
  video:2880kB audio:156kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.298058%
  
  [libx264 @ 00000000005ab440] frame I:14    Avg QP:20.01  size: 39750
  
  [libx264 @ 00000000005ab440] frame P:106   Avg QP:23.85  size: 14578
  
  [libx264 @ 00000000005ab440] frame B:125   Avg QP:24.63  size:  6770
  
  [libx264 @ 00000000005ab440] consecutive B-frames: 22.9% 22.0% 15.9% 39.2%
  
  [libx264 @ 00000000005ab440] mb I  I16..4: 16.7% 80.3%  3.0%
  
  [libx264 @ 00000000005ab440] mb P  I16..4: 10.2% 36.2%  1.1%  P16..4: 25.0%  7.9%  2.5%  0.0%  0.0%    skip:17.1%
  
  [libx264 @ 00000000005ab440] mb B  I16..4:  2.3%  5.8%  0.2%  B16..8: 31.4%  6.5%  0.9%  direct: 3.7%  skip:49.2%  L0:51.8% L1:44.5% BI: 3.7%
  
  [libx264 @ 00000000005ab440] 8x8 transform intra:76.1% inter:86.3%
  
  [libx264 @ 00000000005ab440] coded y,uvDC,uvAC intra: 38.3% 52.1% 9.0% inter: 12.3% 20.1% 0.2%
  
  [libx264 @ 00000000005ab440] i16 v,h,dc,p: 30% 28%  9% 33%
  
  [libx264 @ 00000000005ab440] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 36% 23% 19%  3%  3%  4%  4%  4%  4%
  
  [libx264 @ 00000000005ab440] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33% 21% 14%  5%  7%  7%  6%  5%  3%
  
  [libx264 @ 00000000005ab440] i8c dc,h,v,p: 45% 24% 25%  6%
  
  [libx264 @ 00000000005ab440] Weighted P-Frames: Y:13.2% UV:6.6%
  
  [libx264 @ 00000000005ab440] ref P L0: 71.7% 12.5% 12.9%  2.7%  0.2%
  
  [libx264 @ 00000000005ab440] ref B L0: 92.8%  6.3%  0.9%
  
  [libx264 @ 00000000005ab440] ref B L1: 98.3%  1.7%
  
  [libx264 @ 00000000005ab440] kb/s:2406.56
  
  [aac @ 00000000005adde0] Qavg: 511.420

  ffprobe -show_packets output too big, so I loaded to pastebin
  https://pastebin.com/TYSMdceS