Newest 'ffmpeg' Questions - Stack Overflow

Les articles publiés sur le site

• How to fetch live video frame and its timestamp from ffmpeg to python

  15 février 2017, par vijiboy

  Searching for an alternative as OpenCV would not provide timestamps for live camera stream, which are required in my computer vision algorithm, I found this excellent article https://zulko.github.io/blog/2013/09/27/read-and-write-video-frames-in-python-using-ffmpeg/

  Working up the code on windows I still could'nt get the frame timestamps. I recollected seeing on ffmpeg forum somewhere that the video filters like showinfo are bypassed when redirected. Is this why the following code does not work as expected?

  Expected: It should write video frames to disk as well as print timestamp details.
  Actual: It writes video files but does not get the timestamp (showinfo) details.

  Here's the code I tried:

  import subprocess as sp
  import numpy
  import cv2
  
  command = [ 'ffmpeg', 
              '-i', 'e:\sample.wmv',
              '-pix_fmt', 'rgb24',
              '-vcodec', 'rawvideo',
              '-vf', 'showinfo', # video filter - showinfo will provide frame timestamps
              '-an','-sn', #-an, -sn disables audio and sub-title processing respectively
              '-f', 'image2pipe', '-'] # we need to output to a pipe
  
  pipe = sp.Popen(command, stdout = sp.PIPE, stderr = sp.STDOUT) # TODO someone on ffmpeg forum said video filters (e.g. showinfo) are bypassed when stdout is redirected to pipes??? 
  
  for i in range(10):
      raw_image = pipe.stdout.read(1280*720*3)
      img_info = pipe.stdout.read(244) # 244 characters is the current output of showinfo video filter
      print "showinfo output", img_info
      image1 =  numpy.fromstring(raw_image, dtype='uint8')
      image2 = image1.reshape((720,1280,3))  
  
      # write video frame to file just to verify
      videoFrameName = 'Video_Frame{0}.png'.format(i)
      cv2.imwrite(videoFrameName,image2)
  
      # throw away the data in the pipe's buffer.
      pipe.stdout.flush()
  

  So how to still get the frame timestamps from ffmpeg into python code so that it can be used in my computer vision algorithm...

• av_write_frame throws AccessViolationException

  15 février 2017, par iamyz

  I am creating Cli/Cpp wrapper over ffmpeg. I am trying to encode video frames and audio to H.264 format. When only Video stream is there then its fine. But if i add audio stream then on writing of first video frame av_write_frame function throws access violation exception.

  But the same scenario works fine with the MJEPG and WMV encoding.

  Can some one guide me.

• ffmpeg - Have troubling syncing up audio and video together

  15 février 2017, par zyeek

  I have a webcam and a separate mic. I want to record what is happening.

  It almost works, however the audio seems to play quickly and parts missing while playing over the video.

  This is the command I am currently using to get it partially working

  ffmpeg -thread_queue_size 1024 -f alsa -ac 1 -i plughw:1,0 -f video4linux2 -thread_queue_size 1024 -re -s 1280x720 -i /dev/video0 -r 25 -f avi -q:a 2 -acodec libmp3lame -ab 96k out.mp4

  I have tried other arguments, but unsure if it has to do with the formats I am using or incorrect parameter settings.

  Also, the next part would be how to stream it. Everytime I try going through rtp it complains about multiple streams. I tried doing html as well, but didn't like the format. html html://localhost:50000/live_feed or rts rts://localhost:5000

  edit:

  I am running this on a rpi 3.

• Pipe raw OpenCV images to FFmpeg

  15 février 2017, par BrianTheLion

  Here's a fairly straightforward example of reading off a web cam using OpenCV's python bindings:

  '''capture.py'''
  import cv, sys
  cap = cv.CaptureFromCAM(0)                    # 0 is for /dev/video0
  while True :
      if not cv.GrabFrame(cap) : break
      frame = cv.RetrieveFrame(cap)
      sys.stdout.write( frame.tostring() )
  

  Now I want to pipe the output to ffmpeg as in:

  $ python capture.py | ffmpeg -f image2pipe -pix_fmt bgr8 -i - -s 640x480 foo.avi

  Sadly, I can't get the ffmpeg magic incantation quite right and it fails with

    libavutil     50.15. 1 / 50.15. 1
    libavcodec    52.72. 2 / 52.72. 2
    libavformat   52.64. 2 / 52.64. 2
    libavdevice   52. 2. 0 / 52. 2. 0
    libavfilter    1.19. 0 /  1.19. 0
    libswscale     0.11. 0 /  0.11. 0
    libpostproc   51. 2. 0 / 51. 2. 0
  Output #0, avi, to 'out.avi':
      Stream #0.0: Video: flv, yuv420p, 640x480, q=2-31, 19660 kb/s, 90k tbn, 30 tbc
  [image2pipe @ 0x1508640]max_analyze_duration reached
  [image2pipe @ 0x1508640]Estimating duration from bitrate, this may be inaccurate
  Input #0, image2pipe, from 'pipe:':
    Duration: N/A, bitrate: N/A
      Stream #0.0: Video: 0x0000, bgr8, 25 fps, 25 tbr, 25 tbn, 25 tbc
  swScaler: 0x0 -> 640x480 is invalid scaling dimension
  

  • The captured frames are definitely 640x480.
  • I'm pretty sure the pixel order for the OpenCV image type (IplImage) is GBR, one byte per channel. At least, that's what seems to be coming off the camera.

  I'm no ffmpeg guru. Has anyone done this successfully?

• FFMPEG AAC encoding causes audio to be lower in pitch

  14 février 2017, par Paul Knopf

  I built a sample application that encodes AAC (from PortAudio) into a MP4 container (no video stream).

  The resulting audio is lower in pitch.

  #include "stdafx.h"
  #include "TestRecording.h"
  #include "libffmpeg.h"
  
  TestRecording::TestRecording()
  {
  }
  
  
  TestRecording::~TestRecording()
  {
  }
  
  struct RecordingContext
  {
      RecordingContext()
      {
          formatContext = NULL;
          audioStream = NULL;
          audioFrame = NULL;
          audioFrameframeNumber = 0;
      }
  
      libffmpeg::AVFormatContext* formatContext;
      libffmpeg::AVStream* audioStream;
      libffmpeg::AVFrame* audioFrame;
      int audioFrameframeNumber;
  };
  
  static int AudioRecordCallback(const void *inputBuffer, void *outputBuffer,
      unsigned long framesPerBuffer,
      const PaStreamCallbackTimeInfo* timeInfo,
      PaStreamCallbackFlags statusFlags,
      void *userData)
  {
      RecordingContext* recordingContext = (RecordingContext*)userData;
  
      libffmpeg::avcodec_fill_audio_frame(recordingContext->audioFrame,
          recordingContext->audioFrame->channels,
          recordingContext->audioStream->codec->sample_fmt,
          static_cast(inputBuffer),
          (framesPerBuffer * sizeof(float) * recordingContext->audioFrame->channels),
          0);
  
      libffmpeg::AVPacket pkt;
      libffmpeg::av_init_packet(&pkt);
      pkt.data = NULL;
      pkt.size = 0;
  
      int gotpacket;
      int result = avcodec_encode_audio2(recordingContext->audioStream->codec, &pkt, recordingContext->audioFrame, &gotpacket);
  
      if (result < 0)
      {
          LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't encode the audio frame to acc");
          return paContinue;
      }
  
      if (gotpacket)
      {
          pkt.stream_index = recordingContext->audioStream->index;
          recordingContext->audioFrameframeNumber++;
  
          // this codec requires no bitstream filter, just send it to the muxer!
          result = libffmpeg::av_write_frame(recordingContext->formatContext, &pkt);
          if (result < 0)
          {
              LOG(ERROR) << "Couldn't write the encoded audio frame";
              libffmpeg::av_free_packet(&pkt);
              return paContinue;
          }
  
          libffmpeg::av_free_packet(&pkt);
      }
  
      return paContinue;
  }
  
  static bool InitializeRecordingContext(RecordingContext* recordingContext)
  {
      int result = libffmpeg::avformat_alloc_output_context2(&recordingContext->formatContext, NULL, NULL, "C:\\Users\\Paul\\Desktop\\test.mp4");
      if (result < 0)
      {
          LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't create output format context");
          return false;
      }
  
      libffmpeg::AVCodec *audioCodec;
      audioCodec = libffmpeg::avcodec_find_encoder(libffmpeg::AV_CODEC_ID_AAC);
      if (audioCodec == NULL)
      {
          LOG(ERROR) << "Couldn't find the encoder for AAC";
      }
  
      recordingContext->audioStream = libffmpeg::avformat_new_stream(recordingContext->formatContext, audioCodec);
      if (!recordingContext->audioStream)
      {
          LOG(ERROR) << "Couldn't create the audio stream";
          return false;
      }
  
      recordingContext->audioStream->codec->bit_rate = 64000;
      recordingContext->audioStream->codec->sample_fmt = libffmpeg::AV_SAMPLE_FMT_FLTP;
      recordingContext->audioStream->codec->sample_rate = 48000;
      recordingContext->audioStream->codec->channel_layout = AV_CH_LAYOUT_STEREO;
      recordingContext->audioStream->codec->channels = libffmpeg::av_get_channel_layout_nb_channels(recordingContext->audioStream->codec->channel_layout);
  
      recordingContext->audioStream->codecpar->bit_rate = recordingContext->audioStream->codec->bit_rate;
      recordingContext->audioStream->codecpar->format = recordingContext->audioStream->codec->sample_fmt;
      recordingContext->audioStream->codecpar->sample_rate = recordingContext->audioStream->codec->sample_rate;
      recordingContext->audioStream->codecpar->channel_layout = recordingContext->audioStream->codec->channel_layout;
      recordingContext->audioStream->codecpar->channels = recordingContext->audioStream->codec->channels;
  
      result = libffmpeg::avcodec_open2(recordingContext->audioStream->codec, audioCodec, NULL);
      if (result < 0)
      {
          LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't open the audio codec");
          return false;
      }
  
      // create a new frame to store the audio samples
      recordingContext->audioFrame = libffmpeg::av_frame_alloc();
      if (!recordingContext->audioFrame)
      {
          LOG(ERROR) << "Couldn't alloce the output audio frame";
          return false;
      }
  
      recordingContext->audioFrame->nb_samples = recordingContext->audioStream->codec->frame_size;
      recordingContext->audioFrame->channel_layout = recordingContext->audioStream->codec->channel_layout;
      recordingContext->audioFrame->channels = recordingContext->audioStream->codec->channels;
      recordingContext->audioFrame->format = recordingContext->audioStream->codec->sample_fmt;
      recordingContext->audioFrame->sample_rate = recordingContext->audioStream->codec->sample_rate;
  
      result = libffmpeg::av_frame_get_buffer(recordingContext->audioFrame, 0);
      if (result < 0)
      {
          LOG(ERROR) << "Coudln't initialize the output audio frame buffer";
          return false;
      }
  
      // some formats want video_stream headers to be separate  
      if (!strcmp(recordingContext->formatContext->oformat->name, "mp4") || !strcmp(recordingContext->formatContext->oformat->name, "mov") || !strcmp(recordingContext->formatContext->oformat->name, "3gp"))
      {
          recordingContext->audioStream->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;
      }
  
      // open the ouput file
      if (!(recordingContext->formatContext->oformat->flags & AVFMT_NOFILE))
      {
          result = libffmpeg::avio_open(&recordingContext->formatContext->pb, recordingContext->formatContext->filename, AVIO_FLAG_WRITE);
          if (result < 0)
          {
              LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't open the output file");
              return false;
          }
      }
  
      // write the stream headers
      result = libffmpeg::avformat_write_header(recordingContext->formatContext, NULL);
      if (result < 0)
      {
          LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't write the headers to the file");
          return false;
      }
  
      return true;
  }
  
  static bool FinalizeRecordingContext(RecordingContext* recordingContext)
  {
      int result = 0;
  
      // write the trailing information
      if (recordingContext->formatContext->pb)
      {
          result = libffmpeg::av_write_trailer(recordingContext->formatContext);
          if (result < 0)
          {
              LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't write the trailer information");
              return false;
          }
      }
  
      // close all the codes
      for (int i = 0; i < (int)recordingContext->formatContext->nb_streams; i++)
      {
          result = libffmpeg::avcodec_close(recordingContext->formatContext->streams[i]->codec);
          if (result < 0)
          {
              LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't close the codec");
              return false;
          }
      }
  
      // close the output file
      if (recordingContext->formatContext->pb)
      {
          if (!(recordingContext->formatContext->oformat->flags & AVFMT_NOFILE))
          {
              result = libffmpeg::avio_close(recordingContext->formatContext->pb);
              if (result < 0)
              {
                  LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't close the output file");
                  return false;
              }
          }
      }
  
      // free the format context and all of its data
      libffmpeg::avformat_free_context(recordingContext->formatContext);
  
      recordingContext->formatContext = NULL;
      recordingContext->audioStream = NULL;
  
      if (recordingContext->audioFrame)
      {
          libffmpeg::av_frame_free(&recordingContext->audioFrame);
          recordingContext->audioFrame = NULL;
      }
  
      return true;
  }
  
  int TestRecording::Test()
  {
      PaError result = paNoError;
  
      result = Pa_Initialize();
      if (result != paNoError) LOGINT_WITH_MESSAGE(ERROR, result, "Error initializing audio device framework");
  
      RecordingContext recordingContext;
      if (!InitializeRecordingContext(&recordingContext))
      {
          LOG(ERROR) << "Couldn't start recording file";
          return 0;
      }
  
      auto defaultDevice = Pa_GetDefaultInputDevice();
      auto deviceInfo = Pa_GetDeviceInfo(defaultDevice);
  
      PaStreamParameters  inputParameters;
      inputParameters.device = defaultDevice;
      inputParameters.channelCount = 2;
      inputParameters.sampleFormat = paFloat32;
      inputParameters.suggestedLatency = deviceInfo->defaultLowInputLatency;
      inputParameters.hostApiSpecificStreamInfo = NULL;
  
      PaStream* stream = NULL;
      result = Pa_OpenStream(
          &stream,
          &inputParameters,
          NULL,
          48000,
          1024,
          paClipOff,
          AudioRecordCallback,
          &recordingContext);
      if (result != paNoError)LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't open the audio stream");
  
      result = Pa_StartStream(stream);
      if (result != paNoError)LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't start the audio stream");
  
      Sleep(1000 * 5);
  
      result = Pa_StopStream(stream);
      if (result != paNoError)LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't stop the audio stream");
  
      if (!FinalizeRecordingContext(&recordingContext)) LOG(ERROR) << "Couldn't stop recording file";
  
      result = Pa_CloseStream(stream);
      if (result != paNoError)LOGINT_WITH_MESSAGE(ERROR, result, "Couldn't stop the audio stream");
  
      return 0;
  }
  

  Here is the stdout, in case it helps.

  https://gist.github.com/pauldotknopf/9f24a604ce1f8a081aa68da1bf169e98

  Why is the audio lower in pitch? I assume I am overlooking a parameter that needs to be configured between PortAudio and FFMPEG. Is there something super obvious that I am missing?