Recherche avancée

Sur d’autres sites (2736)

• Pion WebRTC Audio stream cutting out while video works

  19 août 2021, par Twometer

  I am trying to send an MP4 video through Pion WebRTC to the browser.

  


  Using FFmpeg, I split it into an Opus OGG stream and an Annex-B H.264 video stream. While the video works fine, the audio keeps cutting in and out. It plays fine for a few seconds, then stops for a second, and continues.

  


  This is the FFmpeg command I use for audio :

  


  ffmpeg -i demo.mp4 -c:a libopus -vn -page_duration 20000 demo.ogg


  


  And this is my transmitter (shortened) :

  


  var lastGranule uint64
for {
    pageData, pageHeader, err := ogg.ParseNextPage() // Uses Pion OggReader

    // Taken from the play-from-disk example
    sampleCount := float64(pageHeader.GranulePosition - lastGranule)
    lastGranule = pageHeader.GranulePosition
    sampleDuration := time.Duration((sampleCount/48000)*1000) * time.Millisecond

    err = audioTrack.WriteSample(media.Sample{Data: pageData, Duration: sampleDuration})
    util.HandleError(err)

    time.Sleep(sampleDuration)
}


  


  I tried hardcoding the delay to 15ms, which fixes the issue that it's cutting out, but then it randomly plays way too fast or starts skipping. Since I had glitchy video before updating my FFmpeg command (add keyframes and remove b-frames), I assume this is also an encoder problem.

  


  What could be the cause for this ?

  


  Update : Using WebRTC logging in Chrome, I discovered the following log lines that occurred frequently :

  


  [27216:21992:0809/141533.175:WARNING:rtcp_receiver.cc(452)] 30 RTCP blocks were skipped due to being malformed or of unrecognized/unsupported type, during the past 10 second period.


  


  This is probably the reason for the cutouts, although I can't figure out why it receives malformed data.

  


• How to decode video in ffmpeg in real time ?

  18 juillet 2021, par Guerlando OCs

  Basic ffmpeg decoding consists of avcodec_send_packet to send encoded packets and avcodec_receive_frame to receive the decoded frames.

  


  avcodec_send_packet returns EAGAIN in case it's full. That is, in case no one is calling avcodec_receive_frame.

  


  Suppose I'm reading an mp4 file and rendering on the screen. That is, I want real time playback of the video. One way to do that would be to tell the renderer the exact fps the file has, for example :

  


  If the file should play at 30 fps, then the renderer calls avcodec_receive_frame 30 times per second, and the avcodec_send_packet only sends new packets when it can. If it receives EAGAIN, it waits and sends again in 1 millisecond, instead of discarding the packet.

  


  Is this the right way to do ? Because getting the fps of the file is not trivial. At least I've never seen this in ffmpeg. Maybe a .mp4 file has this information, but what about a raw .h264 file ? VLC player can play raw .h264 in the right time, but I don't know how it does that

  


• How do you properly mux an mp4 file with FFmpeg 4 ?

  21 juin 2021, par TheNeuronalCoder

  How do you properly mux an mp4 file with FFmpeg 4 ? I have an issue with the time base, I'm pretty sure. Using ffprobe, I found encoding 24 frames for a 24 FPS output video which should theoretically produce a 1-second video, instead produces a 4 millisecond, 142.61 FPS video with a bitrate 157326 kb/s. I have been stuck on this for months. What's the issue ? How do I fix this ?

  


  void Camera::Encode(Frame frame) {
  if (av_frame_make_writable(frame_) < 0)
    throw std::runtime_error("failed to write to frame");

  frame_->data = frame.data;
  frame_->pts += av_rescale_q(1, context_->time_base, stream_->time_base);

  if (avcodec_send_frame(context_, frame_) < 0)
    throw std::runtime_error("failed to send a frame for encoding");

  int ret = 0;
  AVPacket packet = { 0 };
  while (ret >= 0) {
    ret = avcodec_receive_packet(context_, &packet);
    if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
      break;
    else if (ret < 0)
      throw std::runtime_error("failed to encode frame");

    ret = av_interleaved_write_frame(format_, &packet);
    av_packet_unref(&packet);
  }
}

void Camera::Start() {
  avformat_alloc_output_context2(&format_, NULL, "mp4", output_.c_str());
  if (!format_)
    throw std::runtime_error("failed to find output format");

  stream_ = avformat_new_stream(format_, NULL);
  if (!stream_)
    throw std::runtime_error("failed to allocate video stream");

  stream_->id = format_->nb_streams-1;
  stream_->codecpar->codec_id = AV_CODEC_ID_H264;
  stream_->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
  stream_->codecpar->width = frame.width;
  stream_->codecpar->height = frame.height;
  stream_->codecpar->bit_rate = 0.28 * fps_ * stream_->codecpar->width *
                                stream_->codecpar->height;
  stream_->codecpar->format = AV_PIX_FMT_YUV420P;
  stream_->time_base = av_make_q(1, fps_);
  stream_->r_frame_rate = av_make_q(fps_, 1);
  stream_->avg_frame_rate = av_make_q(fps_, 1);

  AVCodec* codec = avcodec_find_encoder(AV_CODEC_ID_H264);
  if (!codec)
    throw std::runtime_error("failed to find codec");

  context_ = avcodec_alloc_context3(codec);
  if (!context_)
    throw std::runtime_error("failed to allocate video codec context");

  context_->width = stream_->codecpar->width;
  context_->height = stream_->codecpar->height;
  context_->bit_rate = stream_->codecpar->bit_rate;
  context_->time_base = av_make_q(1, fps_);
  context_->framerate = av_make_q(fps_, 1);
  context_->gop_size = 10;
  context_->max_b_frames = 1;
  context_->pix_fmt = AV_PIX_FMT_YUV420P;

  if (avcodec_open2(context_, codec, NULL) < 0)
    throw std::runtime_error("failed to open codec");

  if (format_->oformat->flags & AVFMT_GLOBALHEADER)
    context_->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

  if (!(format_->flags & AVFMT_NOFILE))
    if (avio_open(&format_->pb, output_.c_str(), AVIO_FLAG_WRITE) < 0)
      throw std::runtime_error("failed to open video file");

  if (avformat_write_header(format_, NULL) < 0)
    throw std::runtime_error("failed to write headers");

  frame_ = av_frame_alloc();
  if (!frame_)
    throw std::runtime_error("failed to allocate video frame");

  frame_->pts = 0;
  frame_->width = context_->width;
  frame_->height = context_->height;
  frame_->format = AV_PIX_FMT_YUV420P;

  if (av_frame_get_buffer(frame_, 32) < 0)
    throw std::runtime_error("failed to allocate the video frame data");
}