Recherche avancée

Sur d’autres sites (11057)

• FFmpeg C audio video streams (mic, webcam) sync to mp4

  22 février 2020, par NIAZ

  I am trying to capture audio and video using a microphone and a webcam into an mp4 file. The recorded file is playable but over the time the audio starts drifting away from video and for a longer period of time the gap increases. Both the audio and video is handled in separate threads, for the audio I am using audiofifo adapted from the transcode_acc.c example https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/transcode_aac.c
  Here is how I am setting up the streams
  Video :

  video_output_codec_ctx = video_stream->codec;
  
  video_output_codec_ctx->bit_rate = 2000000;
  
  video_output_codec_ctx->codec_id = AV_CODEC_ID_MPEG4;
  
  video_output_codec_ctx->width = 640;
  
  video_output_codec_ctx->height = 480;
  
  video_stream->time_base = (AVRational){1, fps};
  
  video_output_codec_ctx->time_base = video_stream->time_base;
  
  video_output_codec_ctx->pix_fmt = AV_PIX_FMT_YUV420P;
  
  video_output_codec_ctx->codec_type = AVMEDIA_TYPE_VIDEO;

  Audio :

  audio_output_codec_ctx->channels       = OUTPUT_CHANNELS; // 2
  
  audio_output_codec_ctx->channel_layout = av_get_default_channel_layout(OUTPUT_CHANNELS);
  
  audio_output_codec_ctx->sample_rate    = audio_input_codec_ctx->sample_rate;
  
  audio_output_codec_ctx->sample_fmt     = audio_output_codec->sample_fmts[0];
  
  audio_output_codec_ctx->bit_rate       = OUTPUT_BIT_RATE; // 96000
  
  audio_output_codec_ctx->strict_std_compliance = FF_COMPLIANCE_EXPERIMENTAL;
  
  
  
  /* Set the sample rate for the container. */
  
  audio_stream->time_base.den = audio_input_codec_ctx->sample_rate;
  
  audio_stream->time_base.num = 1;

  For the video pts, I increment the index by 1 once the frame is encoded and before sending the frame to the encoder I use rescale and also after receiving the frame,afterwards the packets are written via av_interleaved_write_frame().

  output_frame->pts = av_rescale_q(video_frame_index, video_output_codec_ctx->time_base, video_input_format_ctx->streams[0]->time_base);
  
  
  
  error = avcodec_send_frame(video_output_codec_ctx, output_frame);
  
  
  
  error = avcodec_receive_packet(video_output_codec_ctx, &output_packet);
  
  
  
  output_packet.stream_index = video_index;
  
  
  
  output_packet.pts = av_rescale_q_rnd(output_packet.pts, video_input_format_ctx->streams[0]->time_base, output_format_context->streams[video_index]->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
  
  
  
  output_packet.dts = av_rescale_q_rnd(output_packet.dts, video_input_format_ctx->streams[0]->time_base, output_format_context->streams[video_index]->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
  
  
  
  output_packet.duration = ((output_format_context->streams[0]->time_base.den / output_format_context->streams[0]->time_base.num) / video_output_codec_ctx->time_base.den);
  
  
  
  output_packet.pos = -1;  
  
  
  
  video_frame_index++;

  For the audio pts, I increment by frame->nb_samples once the frame is encoded I use rescale, afterwards the packets are written via av_interleaved_write_frame().

  frame->pts = aud_pts;
  
  aud_pts += frame->nb_samples;
  
  
  
  error = avcodec_send_frame(audio_output_codec_ctx, frame);
  
  
  
  error = avcodec_receive_packet(audio_output_codec_ctx, &output_packet);
  
  
  
  output_packet.stream_index = audio_index;
  
  
  
  output_packet.pts = av_rescale_q_rnd(output_packet.pts, audio_output_codec_ctx->time_base, output_format_context->streams[audio_index]->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
  
  
  
  output_packet.dts = av_rescale_q_rnd(output_packet.dts, audio_output_codec_ctx->time_base, output_format_context->streams[audio_index]->time_base, (AVRounding)(AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
  
  
  
  output_packet.duration = av_rescale_q(output_packet.duration, audio_output_codec_ctx->time_base, output_format_context->streams[audio_index]->time_base);

  I am new to the FFmpeg C API, have tried various resources / posts over the internet but still haven’t been able to sync the audio and video in a robust way. Here are few question that I want to understand which will help me to resolve this. Any thoughts are really appreciated on this.

  1. Can the FFmpeg C API handle sync internally or this is something which needs to be handled from the caller side ?

  2. Am I setting up the PTS correctly both for audio and video in the first place ? I have noticed that when I use fps lower than 20, I get Invalid pts (66667) <= last (66667) Operation not permitted from the encoder. This must be something wrong with the way I am currently setting the video PTS. How can I set up the video PTS to handle lower fps ?

  3. I am also trying to adopt the idea of clocks sync from dranger’s tutorial, not sure whether this would be suitable for my use case, things like where to setup the audio and video clocks as he used only decoders, for the audio I am using fifo and not sure how to adjust the samples based on the clocks sync, also the way to call and setup the refresh timer ?

  4. Is there a better mechanism for creating a robust sync for my use case which can handle both audio and video if they go out of sync, would be great to have an idea about the samples and frames adjustment based on that ?

• How to fix EXT-X-MAP:URI in m3u8 when specifying absolute hls_fmp4_init_filename using ffmpeg ?

  29 février 2020, par Ryan

  Today I found this blog post teaching me about fragmented mp4s (fmp4s).

  Using the ffmpeg docs, I’ve made great progress.

  I run this :

  '/usr/bin/ffmpeg' '-y' '-i' '/myproject/storage/app/sample_media/2020-02-27/Sample Videos 5.mp4' \
  
  '-c:v' 'libx264' '-s:v' '1920x1080' \
  
   '-crf' '20' '-sc_threshold' '0' '-g' '48' '-keyint_min' '48' \
  
  '-hls_list_size' '0' '-hls_time' '10' '-hls_allow_cache' '0' '-b:v' '4889k' \
  
  '-maxrate' '5866k' '-hls_segment_type' 'fmp4' \
  
  '-hls_fmp4_init_filename' '/myproject/public/storage/000000002/init.mp4' \
  
  '-hls_segment_filename' '/myproject/public/storage/000000002/list_1080p_%04d.m4s' \
  
  '-hls_key_info_file' '/myproject/hls_hls.keyInfo' '-strict' '-2' '-threads' '12' \
  
  '/myproject/public/storage/000000002/list_1080p.m3u8'

  And then I use hls.js on my webpage to try to show the video.

  Initially, it won’t load.

  But then I can get the video to load properly if I refresh the page after making this edit to list_1080p.m3u8 :

  Change from :

  #EXT-X-MAP:URI="/myproject/public/storage/000000002/init.mp4"

  To :

  #EXT-X-MAP:URI="init.mp4"

  How can I change my ffmpeg command so that it knows the absolute path of where to save the init.mp4 but also knows to write in the m3u8 only the filename init.mp4 without any path ?

  (I’ve tried using -hls_fmp4_init_filename init.mp4 without the abosolute path, but then it creates the init.mp4 at the root, which also doesn’t work. I need each video’s init.mp4 to be in its own folder.)

• avformat/hlsenc : add hls_fmp4_init_resend option

  7 avril 2020, par Steven Liu

  avformat/hlsenc : add hls_fmp4_init_resend option
  add option for resend init file after m3u8 refresh everytime.
  Signed-off-by : Steven Liu <lq@chinaffmpeg.org>
  

  • [DH] doc/muxers.texi
  • [DH] libavformat/hlsenc.c