Recherche avancée

Sur d’autres sites (11560)

• Writing opencv frames to avi container using libavformat Custom IO

  16 juillet 2017, par Aryan

  I have to write OpenCV cv::Mat frames to an AVI container. I cannot use OpenCV’s VideoWriter because I do not intend to write the AVI file to disk directly, instead I want to send it to a custom stream, so I have to use ffmpeg/libav. As I have never used ffmpeg before I have taken help from solutions provided here and here, alongwith the ffmpeg documentation.

  I am able to send AVI container packets to my custom output stream as required but the performance is very bad. Specifically, the call to avcodec_encode_video2 is taking too long.

  First, I suspect that due to my inexperience I have misconfigured or wrongly coded something. I am currently working wit 640*480 grayscale frames. On my i.MX6 platform the call to avcodec_encode_video2 is taking about 130ms on average per frame, which is unacceptably slow. Any pointers to an obvious performance killer ??? (i know sws_scale looks useless but it takes negligible time, and might be useful for me later).

  Second, I am using PNG encoder, but that is not required, I would be happy to write uncompressed data if I know how to : If the slowdown is not due to my bad programming, can we just get rid of the encoder and generate uncompressed packets for the avi container ? Or use some encoder that accepts grayscale images and is not that slow ?

  For Initialization and writing of header I am using :

  void MyWriter::WriteHeader()  
  
  {
  
      av_register_all();
  
  
  
      // allocate output format context
  
      if (avformat_alloc_output_context2(&avFmtCtx, NULL, "avi", NULL) < 0) { printf("DATAREC: avformat_alloc_output_context2 failed"); exit(1); }
  
  
  
      // buffer for avio context    
  
      bufSize = 640 * 480 * 4; // Don't know how to derive, but this should be big enough for now
  
      buffer = (unsigned char*)av_malloc(bufSize);
  
      if (!buffer) { printf("DATAREC: Buffer alloc failed"); exit(1); }
  
  
  
      // allocate avio context
  
      avIoCtx = avio_alloc_context(buffer, bufSize, 1, this, NULL, WriteCallbackWrapper, NULL);
  
      if (!avIoCtx) { printf("DATAREC: avio_alloc_context failed"); exit(1); }
  
  
  
      // connect avio context to format context
  
      avFmtCtx->pb = avIoCtx;
  
  
  
      // set custom IO flag
  
      avFmtCtx->flags |= AVFMT_FLAG_CUSTOM_IO;
  
  
  
      // get encoder
  
      encoder = avcodec_find_encoder(AV_CODEC_ID_PNG);
  
      if (!encoder) { printf("DATAREC: avcodec_find_encoder failed"); exit(1); }
  
  
  
      // create new stream
  
      avStream = avformat_new_stream(avFmtCtx, encoder);
  
      if (!avStream) { printf("DATAREC: avformat_new_stream failed"); exit(1); }
  
  
  
      // set stream codec defaults
  
      if (avcodec_get_context_defaults3(avStream->codec, encoder) < 0)  { printf("DATAREC: avcodec_get_context_defaults3 failed"); exit(1); }
  
  
  
     // hardcode settings for now
  
      avStream->codec->pix_fmt = AV_PIX_FMT_GRAY8;
  
      avStream->codec->width = 640;
  
      avStream->codec->height = 480;
  
      avStream->codec->time_base.den = 15;
  
      avStream->codec->time_base.num = 1;
  
      avStream->time_base.den = avStream->codec->time_base.den;
  
      avStream->time_base.num = avStream->codec->time_base.num;
  
      avStream->r_frame_rate.num = avStream->codec->time_base.den;
  
      avStream->r_frame_rate.den = avStream->codec->time_base.num;
  
  
  
      // open encoder    
  
      if (avcodec_open2(avStream->codec, encoder, NULL) < 0) {
  
          printf("DATAREC: Cannot open codec\n");
  
          exit(1);
  
      }
  
  
  
      // write header
  
      if(avformat_write_header(avFmtCtx, NULL) < 0) { printf("DATAREC: avformat_write_header failed\n"); exit(1);}
  
  
  
      // prepare for first frame
  
      framePts = 0;
  
      firstFrame = true; 
  
  }

  After writing the header, the following function is called in a loop for each cv::Mat frame :

  void MyWriter::WriteFrame(cv::Mat& item)
  
  {
  
      if (firstFrame) // do only once, before writing the first frame
  
      {
  
          // allocate frame
  
          frame = av_frame_alloc();
  
          if (!frame) { printf("DATAREC: av_frame_alloc failed"); exit(1); }
  
  
  
          // get size for framebuffer
  
          int picsz = av_image_get_buffer_size(avStream->codec->pix_fmt, avStream->codec->width, avStream->codec->height, 1);
  
  
  
          // allocate frame buffer
  
          framebuf = (unsigned char*)av_malloc(picsz);
  
          if (!framebuf) { printf("DATAREC: fail to alloc framebuf"); exit(1); }
  
  
  
          // set frame width, height, format
  
          frame->width = avStream->codec->width;
  
          frame->height = avStream->codec->height;
  
          frame->format = static_cast<int>(avStream->codec->pix_fmt);
  
  
  
          // set up data pointers and linesizes
  
          if (av_image_fill_arrays(frame->data, frame->linesize, framebuf, avStream->codec->pix_fmt, avStream->codec->width, avStream->codec->height, 1) &lt; 0) { printf("DATAREC: av_image_fill_arrays failed\n"); exit(1);}
  
  
  
          // get sws context
  
          swsctx = sws_getCachedContext(nullptr, avStream->codec->width, avStream->codec->height, avStream->codec->pix_fmt, avStream->codec->width, avStream->codec->height, avStream->codec->pix_fmt, SWS_BICUBIC, nullptr, nullptr, nullptr);
  
          if (!swsctx) { printf("DATAREC: fail to sws_getCachedContext"); exit(1); }
  
  
  
          // done initializing
  
          firstFrame = false; // don't repeat this for the following frames
  
      }
  
  
  
      // call sws scale
  
      const int stride[] = { static_cast<int>(item.step[0]) };
  
      sws_scale(swsctx, &amp;item.data, stride, 0, item.rows, frame->data, frame->linesize);
  
  
  
      // set presentation timestamp
  
      frame->pts = framePts++;
  
  
  
      // initialize packet
  
      av_init_packet(&amp;pkt);
  
      pkt.data = NULL;
  
      pkt.size = 0;
  
  
  
      // THIS TAKES VERY LONG TO EXECUTE
  
      // call encoder, convert frame to packet
  
      if (avcodec_encode_video2(avStream->codec, &amp;pkt, frame, &amp;got_pkt) &lt; 0) { printf("DATAREC: fail to avcodec_encode_video2");  exit(1); }
  
  
  
      write packet if available
  
      if (got_pkt)
  
      {
  
          pkt.duration = 1;
  
          av_write_frame(avFmtCtx, &amp;pkt);
  
      }
  
  
  
      // wipe packet
  
      av_packet_unref(&amp;pkt);
  
  }
  
  </int></int>

  After writing required frames, trailer is written :

  void MyWriter::WriteTrailer()
  
  {
  
      // prepare packet for trailer
  
      av_init_packet(&pkt);
  
      pkt.data = NULL;
  
      pkt.size = 0;
  
  
  
      // encode trailer packet
  
      if (avcodec_encode_video2(avStream->codec, &pkt, nullptr, &got_pkt) < 0) { printf("DATAREC: fail to avcodec_encode_video2");    exit(1); }
  
  
  
      // write trailer packet
  
      if (got_pkt)
  
      {
  
          pkt.duration = 1;
  
          av_write_frame(avFmtCtx, &pkt);
  
      }
  
  
  
      // free everything
  
      av_packet_unref(&pkt);
  
      av_write_trailer(avFmtCtx);
  
      av_frame_free(&frame);
  
      avcodec_close(avStream->codec);
  
      av_free(avIoCtx);
  
      sws_freeContext(swsctx);
  
      avformat_free_context(avFmtCtx);
  
      av_free(framebuf);
  
      av_free(buffer);
  
      firstFrame = true; // for the next file
  
  }

  Many many thanks to everyone who made it down to this line !

• Running ffmpeg with multiple pipes

  13 août 2017, par baci

  Is there a way to let ffmpeg read from multiple buffers or multiple named pipes ?

  Basically I want to output the first loop to different pipes instead of .jpg files, and run the video filter command using those pipes as input. Right now enormous disk IO is needed to create and read jpg tiles.

  // heic -> tiles -> jpeg tiles
  
  #pragma omp parallel for
  
      for(uint i = 0; i < data.tiles.size(); ++i) {
  
          char buff[100];
  
          snprintf(buff, sizeof(buff), "./tmp/%03d.jpg", data.tiles[i].idx);
  
          std::string cmdStr = "ffmpeg -i - -loglevel warning -frames:v 1 -vsync vfr -an -q:v 1 " + std::string(buff);
  
          FILE * pipe_in;
  
          if(pipe_in = popen(cmdStr.c_str(), "w")) {
  
              fwrite((char*)&data.tiles[i].data[0], 1, data.tiles[i].data.size(), pipe_in);
  
              fflush(pipe_in);
  
              fclose(pipe_in);
  
          }
  
      }
  
  
  
  ... some processing
  
  
  
  // jpeg tiles -> big jpeg
  
  std::string inputStr = "./tmp/%03d.jpg";
  
  std::string cmdStr = "ffmpeg -i " + inputStr +
  
          " -loglevel warning -threads 4 -q:v 1 -vf \"tile=" + std::to_string(data.cols) + "x" + std::to_string(data.rows) +
  
          ",crop=" + std::to_string(data.width) + ":" + std::to_string(data.height) + ":0:0" +
  
          transStr + "\" -y " + outputStr;
  
  ret = std::system(cmdStr.c_str());

• stream flv over tcp using ffmpeg

  16 août 2017, par Naseeb Panghal

  I am using FC23 machine and FFMPEG version is 2.8.10

  Using below command, I can stream flv over tcp successfully and able to receive it successfully at receiver side as well. I play it using vlc player. Both audio and video are played well.

  ./ffmpeg -f x11grab -s 1920x1080 -framerate 15 -i :0.0 -f alsa -ac 2 -i hw:1 -vcodec libx264 -r 30 -pix_fmt yuv420p -tune zerolatency -preset ultrafast -acodec aac -strict -2 -ar 48000 -ab 96k -f flv -metadata streamName=naseeb.sdp tcp ://127.0.0.1:6666

  But in actually i need to do this using an application. So i wrote an application in ’C’ language.
  I have done following things in the application.

  1. open `AVOutputFormat` using below API
  
      fmt = av_guess_format("flv", NULL , NULL);
  
   2. Get `AVFormatContext` using below API
  
      avformat_alloc_output_context2(&oc, fmt, NULL, NULL);
  
   3. Then added streams(audio and video) using required APIs
  
   4. Then open codecs using required APIs
  
   5. Then set the output using below API
  
      ret = avio_open(&oc->pb, filename, AVIO_FLAG_WRITE);

  When i need to write flv into a file on system then i provide filename in step 5 as muxer.flv.
  When i need to stream flv over tcp then i provide filename as tcp://127.0.0.1:6666.

  when i play muxer.flv using vlc player both video and audio are played well but when it is streamed over network then Green frames are shown for video and audio works very well. I am wondering what extra setting is required when i need to stream it over network ?

  I have no doubt on Receiver side as it works very well with ffmpeg utility(command mentioned above).
  Currently i have downloaded ffmpeg 2.8.10 source code and looking into ffmpeg.c file for some extra setting. Till now i have not found anything helpful. Please suggest something why Green frames shown when flv is sent on network where it works well when dump on system hard disk.