Recherche avancée

Sur d’autres sites (12179)

• ffmpeg/libx264 C API : frames dropped from end of short MP4

  19 juillet 2017, par Blake McConnell

  In my C++ application, I am taking a series of JPEG images, manipulating their data using FreeImage, and then encoding the bitmaps as H264 using the ffmpeg/libx264 C API. The output is an MP4 which shows the series of 22 images at 12fps. My code is adapted from the "muxing" example that comes with ffmpeg C source code.

  My problem : no matter how I tune the codec parameters, a certain number of frames at the end of the sequence which are passed to the encoder do not appear in the final output. I’ve set the AVCodecContext parameters like this :

  //set context params
  
  ctx->codec_id = AV_CODEC_ID_H264;
  
  ctx->bit_rate = 4000 * 1000;
  
  ctx->width = _width;
  
  ctx->height = _height;
  
  ost->st->time_base = AVRational{ 1, 12 };
  
  ctx->time_base = ost->st->time_base;
  
  ctx->gop_size = 1;
  
  ctx->pix_fmt = AV_PIX_FMT_YUV420P;

  I have found that the higher the gop_size the more frames are dropped from the end of the video. I can also see from the output that, with this gop size (where I’m essentially directing that all output frames be I frames) that only 9 frames are written.

  I’m not sure why this is occurring. I experimented with encoding duplicate frames and making a much longer video. This resulted in no frames being dropped. I know with the ffmpeg command line tool there is a concatenation command that accomplishes what I am trying to do, but I’m not sure how to accomplish the same goal using the C API.

  Here’s the output I’m getting from the console :

  [libx264 @ 026d81c0] using cpu capabilities : MMX2 SSE2Fast SSSE3
  SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 026d81c0] profile High, level
  3.1 [libx264 @ 026d81c0] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Cop yleft 2003-2017 - http://www.videolan.org/x264.html -
  options : cabac=1 ref=1 deb lock=1:0:0 analyse=0x3:0x113 me=hex subme=7
  psy=1 psy_rd=1.00:0.00 mixed_ref=0 m e_range=16 chroma_me=1 trellis=1
  8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chro ma_qp_offset=-2
  threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1
  interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0
  keyint=1 ke yint_min=1 scenecut=40 intra_refresh=0 rc=abr mbtree=0
  bitrate=4000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4
  ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to
  ’....\images\c411a991-46f6-400c-8bb0-77af3738559a.mp4’ :
  Stream #0:0 : Video : h264, yuv420p, 700x700, q=2-31, 4000 kb/s, 12 tbn

  [libx264 @ 026d81c0] frame I:9 Avg QP:17.83 size:111058 [libx264
  @ 026d81c0] mb I I16..4 : 1.9% 47.7% 50.5% [libx264 @ 026d81c0] final
  ratefactor : 19.14 [libx264 @ 026d81c0] 8x8 transform intra:47.7%
  [libx264 @ 026d81c0] coded y,uvDC,uvAC intra : 98.4% 96.9% 89.5%
  [libx264 @ 026d81c0] i16 v,h,dc,p : 64% 6% 2% 28% [libx264 @
  026d81c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu : 32% 15% 9% 5% 5% 6% 8%
  10% 10% [libx264 @ 026d81c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu : 28% 18%
  7% 6% 8% 8% 8% 9% 8% [libx264 @ 026d81c0] i8c dc,h,v,p : 43% 22%
  25% 10% [libx264 @ 026d81c0] kb/s:10661.53

  Code included below :

  MP4Writer.h

  #ifndef MPEG_WRITER
  
  #define MPEG_WRITER
  
  
  
  #include <iostream>
  
  #include <string>
  
  #include <vector>
  
  #include 
  
  extern "C" {
  
      #include <libavformat></libavformat>avformat.h>
  
      #include <libswscale></libswscale>swscale.h>
  
      #include <libswresample></libswresample>swresample.h>
  
      #include <libswscale></libswscale>swscale.h>
  
  }
  
  
  
  typedef struct OutputStream 
  
  {
  
      AVStream *st;
  
      AVCodecContext *enc;
  
  
  
      //pts of the next frame that will be generated
  
      int64_t next_pts;
  
      int samples_count;
  
  
  
      AVFrame *frame;
  
      AVFrame *tmp_frame;
  
  
  
      float t, tincr, tincr2;
  
  
  
      struct SwsContext *sws_ctx;
  
      struct SwrContext *swr_ctx;
  
  };
  
  
  
  class MP4Writer {
  
      public:
  
          MP4Writer();
  
          void Init();
  
          int16_t SetOutput( const std::string &amp; path );
  
          int16_t AddFrame( uint8_t * imgData );
  
          int16_t Write( std::vector<imgdata> &amp; imgData );
  
          int16_t Finalize();
  
          void SetHeight( const int height ) { _height = _width = height; } //assuming 1:1 aspect ratio
  
  
  
      private:
  
          int16_t AddStream( OutputStream * ost, AVFormatContext * formatCtx, AVCodec ** codec, enum AVCodecID codecId );
  
          int16_t OpenVideo( AVFormatContext * formatCtx, AVCodec *codec, OutputStream * ost, AVDictionary * optArg );
  
          static AVFrame * AllocPicture( enum AVPixelFormat pixFmt, int width, int height );
  
          static AVFrame * GetVideoFrame( uint8_t * imgData, OutputStream * ost, const int width, const int height );
  
          static int WriteFrame( AVFormatContext * formatCtx, const AVRational * timeBase, AVStream * stream, AVPacket * packet );
  
  
  
          int _width;
  
          int _height;
  
          OutputStream _ost;
  
          AVFormatContext * _formatCtx;
  
          AVDictionary * _dict;
  
  };
  
  
  
  #endif //MPEG_WRITER
  
  </imgdata></vector></string></iostream>

  MP4Writer.cpp

  #include 
  
  #include <algorithm>
  
  
  
  MP4Writer::MP4Writer()
  
  {
  
      _width = 0;
  
      _height = 0;
  
  }
  
  
  
  void MP4Writer::Init()
  
  {
  
      av_register_all();
  
  }
  
  
  
  /**
  
   sets up output stream for the specified path.
  
   note that the output format is deduced automatically from the file extension passed
  
   @param path: output file path
  
   @returns: -1 = output could not be deduced, -2 = invalid codec, -3 = error opening output file,
  
             -4 = error writing header
  
  */
  
  int16_t MP4Writer::SetOutput( const std::string &amp; path )
  
  {
  
      int error;
  
      AVCodec * codec;
  
      AVOutputFormat * format;
  
  
  
      _ost = OutputStream{}; //TODO reset state in a more focused way?
  
  
  
      //allocate output media context
  
      avformat_alloc_output_context2( &amp;_formatCtx, NULL, NULL, path.c_str() );
  
      if ( !_formatCtx ) {
  
          std::cout &lt;&lt; "could not deduce output format from file extension.  aborting" &lt;&lt; std::endl;
  
          return -1;
  
      }
  
      //set format
  
      format = _formatCtx->oformat;
  
      if ( format->video_codec != AV_CODEC_ID_NONE ) {
  
          AddStream( &amp;_ost, _formatCtx, &amp;codec, format->video_codec );
  
      }
  
      else {
  
          std::cout &lt;&lt; "there is no video codec set.  aborting" &lt;&lt; std::endl;
  
          return -2;
  
      }
  
  
  
      OpenVideo( _formatCtx, codec, &amp;_ost, _dict );
  
  
  
      av_dump_format( _formatCtx, 0, path.c_str(), 1 );
  
  
  
      //open output file
  
      if ( !( format->flags &amp; AVFMT_NOFILE )) {
  
          error = avio_open( &amp;_formatCtx->pb, path.c_str(), AVIO_FLAG_WRITE );
  
          if ( error &lt; 0 ) {
  
              std::cout &lt;&lt; "there was an error opening output file " &lt;&lt; path &lt;&lt; ".  aborting" &lt;&lt; std::endl;
  
              return -3;
  
          }
  
      }
  
  
  
      //write header
  
      error = avformat_write_header( _formatCtx, &amp;_dict );
  
      if ( error &lt; 0 ) {
  
          std::cout &lt;&lt; "an error occurred writing header. aborting" &lt;&lt; std::endl;
  
          return -4;
  
      }
  
  
  
      return 0;
  
  }
  
  
  
  /**
  
   initialize the output stream
  
   @param ost: the output stream
  
   @param formatCtx: the context format
  
   @param codec: the output codec
  
   @param codec: the ffmpeg enumerated id of the codec
  
   @returns: -1 = encoder not found, -2 = stream could not be allocated, -3 = encoding context could not be allocated
  
  */
  
  int16_t MP4Writer::AddStream( OutputStream * ost, AVFormatContext * formatCtx, AVCodec ** codec, enum AVCodecID codecId )
  
  {
  
      AVCodecContext * ctx; //TODO not sure why this is here, could just set ost->enc directly
  
      int i;
  
  
  
      //detect the encoder
  
      *codec = avcodec_find_encoder( codecId );
  
      if ( (*codec) == NULL ) {
  
          std::cout &lt;&lt; "could not find encoder.  aborting" &lt;&lt; std::endl;
  
          return -1;
  
      }
  
  
  
      //allocate stream
  
      ost->st = avformat_new_stream( formatCtx, NULL );
  
      if ( ost->st == NULL ) {
  
          std::cout &lt;&lt; "could not allocate stream.  aborting" &lt;&lt; std::endl;
  
          return -2;
  
      }
  
  
  
      //allocate encoding context
  
      ost->st->id = formatCtx->nb_streams - 1;
  
      ctx = avcodec_alloc_context3( *codec );
  
      if ( ctx == NULL ) {
  
          std::cout &lt;&lt; "could not allocate encoding context.  aborting" &lt;&lt; std::endl;
  
          return -3;
  
      }
  
  
  
      ost->enc = ctx;
  
  
  
      //set context params
  
      ctx->codec_id = AV_CODEC_ID_H264;
  
      ctx->bit_rate = 4000 * 1000;
  
      ctx->width = _width;
  
      ctx->height = _height;
  
      ost->st->time_base = AVRational{ 1, 12 };
  
      ctx->time_base = ost->st->time_base;
  
      ctx->gop_size = 1;
  
      ctx->pix_fmt = AV_PIX_FMT_YUV420P;
  
  
  
      //if neccesary, set stream headers and formats separately
  
      if ( formatCtx->oformat->flags &amp; AVFMT_GLOBALHEADER ) {
  
          std::cout &lt;&lt; "setting stream and headers to be separate" &lt;&lt; std::endl;
  
          ctx->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
  
      }
  
  
  
      return 0;
  
  }
  
  
  
  /**
  
   open the video for writing
  
   @param formatCtx: the format context
  
   @param codec: output codec
  
   @param ost: output stream
  
   @param optArg: dictionary
  
   @return: -1 = error opening codec, -2 = allocate new frame, -3 = copy stream params
  
  */
  
  int16_t MP4Writer::OpenVideo( AVFormatContext * formatCtx, AVCodec *codec, OutputStream * ost, AVDictionary * optArg )
  
  {
  
      int error;
  
      AVCodecContext * ctx = ost->enc;
  
      AVDictionary * dict = NULL;
  
      av_dict_copy( &amp;dict, optArg, 0 );
  
  
  
      //open codec
  
      error = avcodec_open2( ctx, codec, &amp;dict );
  
      av_dict_free( &amp;dict );
  
      if ( error &lt; 0 ) {
  
          std::cout &lt;&lt; "there was an error opening the codec.  aborting" &lt;&lt; std::endl;
  
          return -1;
  
      }
  
  
  
      //allocate new frame
  
      ost->frame = AllocPicture( ctx->pix_fmt, ctx->width, ctx->height );
  
      if ( ost->frame == NULL ) {
  
          std::cout &lt;&lt; "there was an error allocating a new frame.  aborting" &lt;&lt; std::endl;
  
          return -2;
  
      }
  
  
  
      //copy steam params
  
      error = avcodec_parameters_from_context( ost->st->codecpar, ctx );
  
      if ( error &lt; 0 ) {
  
          std::cout &lt;&lt; "could not copy stream parameters.  aborting" &lt;&lt; std::endl;
  
          return -3;
  
      }
  
  
  
      return 0;
  
  }
  
  
  
  /**
  
   allocate a new frame
  
   @param pixFmt: ffmpeg enumerated pixel format
  
   @param width: output width
  
   @param height: output height
  
   @returns: an inititalized frame
  
  */
  
  AVFrame * MP4Writer::AllocPicture( enum AVPixelFormat pixFmt, int width, int height )
  
  {
  
      AVFrame * picture;
  
      int error;
  
  
  
      //allocate the frame
  
      picture = av_frame_alloc();
  
      if ( picture == NULL ) {
  
          std::cout &lt;&lt; "there was an error allocating the picture" &lt;&lt; std::endl;
  
          return NULL;
  
      }
  
  
  
      picture->format = pixFmt;
  
      picture->width = width;
  
      picture->height = height;
  
  
  
      //allocate the frame's data buffer
  
      error = av_frame_get_buffer( picture, 32 );
  
      if ( error &lt; 0 ) {
  
          std::cout &lt;&lt; "could not allocate frame data" &lt;&lt; std::endl;
  
          return NULL;
  
      }
  
      picture->pts = 0;
  
      return picture;
  
  }
  
  
  
  /**
  
   convert raw RGB buffer to YUV frame
  
   @return: frame that contains image data
  
  */
  
  AVFrame * MP4Writer::GetVideoFrame( uint8_t * imgData, OutputStream * ost, const int width, const int height )
  
  {
  
      int error;
  
      AVCodecContext * ctx = ost->enc;
  
  
  
      //prepare the frame
  
      error = av_frame_make_writable( ost->frame );
  
      if ( error &lt; 0 ) {
  
          std::cout &lt;&lt; "could not make frame writeable" &lt;&lt; std::endl;
  
          return NULL;
  
      }
  
  
  
      //TODO set this context one time per run, or even better, one time at init
  
      //convert RGB to YUV
  
      struct SwsContext* fooContext = sws_getContext( width, height, AV_PIX_FMT_BGR24, 
  
          width, height, AV_PIX_FMT_YUV420P, SWS_BICUBIC, NULL, NULL, NULL );
  
      int inLinesize[1] = { 3 * width }; // RGB stride
  
      uint8_t * inData[1] = { imgData };
  
      int sliceHeight = sws_scale( fooContext, inData, inLinesize, 0, height, ost->frame->data, ost->frame->linesize );
  
      sws_freeContext( fooContext );
  
  
  
      ost->frame->pts = ost->next_pts++;
  
      //TODO does the frame need to be returned here as it is available at the class level?
  
      return ost->frame;
  
  }
  
  
  
  /**
  
   write frame to file
  
   @param formatCtx: the output format context
  
   @param timeBase: the framerate
  
   @param stream: output stream
  
   @param packet: data packet
  
   @returns: see return values for av_interleaved_write_frame
  
  */
  
  int MP4Writer::WriteFrame( AVFormatContext * formatCtx, const AVRational * timeBase, AVStream * stream, AVPacket * packet )
  
  {
  
      av_packet_rescale_ts( packet, *timeBase, stream->time_base );
  
      packet->stream_index = stream->index;
  
  
  
      //write compressed file to media file
  
      return av_interleaved_write_frame( formatCtx, packet );
  
  }
  
  
  
  int16_t MP4Writer::Write( std::vector<imgdata> &amp; imgData )
  
  {
  
      int16_t errorCount = 0;
  
      int16_t retVal = 0;
  
      bool countingUp = true;
  
      size_t i = 0;
  
      while ( true ) {
  
          //don't show first frame again when counting back down
  
          if ( !countingUp &amp;&amp; i == 0 ) {
  
              break;
  
          }
  
          uint8_t * pixels = imgData[i].GetBits( imgData[i].mp4Input );
  
          AddFrame( pixels );
  
  
  
          //handle inc/dec without repeating last frame
  
          if ( countingUp ) {
  
              if ( i == imgData.size() -1 ) {
  
                  countingUp = false;
  
                  i--;
  
              }
  
              else {
  
                  i++;
  
              }
  
          }
  
          else {
  
              i--;
  
          }
  
      }
  
      Finalize();
  
      return 0; //TODO return error code
  
  }
  
  
  
  /**
  
   add another frame to output video
  
   @param imgData: the raw image data
  
   @returns -1 = error encoding video frame, -2 = error writing frame
  
  */
  
  int16_t MP4Writer::AddFrame( uint8_t * imgData )
  
  {
  
      int error;
  
      AVCodecContext * ctx;
  
      AVFrame * frame;
  
      int gotPacket = 0;
  
      AVPacket pkt = { 0 };
  
  
  
      ctx = _ost.enc;
  
      av_init_packet( &amp;pkt );
  
  
  
      frame = GetVideoFrame( imgData, &amp;_ost, _width, _height );
  
  
  
      //encode the image
  
      error = avcodec_encode_video2( ctx, &amp;pkt, frame, &amp;gotPacket );
  
      if ( error &lt; 0 ) {
  
          std::cout &lt;&lt; "there was an error encoding the video frame" &lt;&lt; std::endl;
  
          return -1;
  
      }
  
  
  
      //write the frame.  NOTE: this doesn't kick in until the encoder has received a certain number of frames
  
      if ( gotPacket ) {
  
          error = WriteFrame( _formatCtx, &amp;ctx->time_base, _ost.st, &amp;pkt );
  
          if ( error &lt; 0 ) {
  
              std::cout &lt;&lt; "the video frame could not be written" &lt;&lt; std::endl;
  
              return -2;
  
          }
  
      }
  
      return 0;
  
  }
  
  
  
  /**
  
   finalize output video and cleanup
  
  */
  
  int16_t MP4Writer::Finalize()
  
  {
  
      av_write_trailer( _formatCtx );
  
      avcodec_free_context( &amp;_ost.enc );
  
      av_frame_free( &amp;_ost.frame);
  
      av_frame_free( &amp;_ost.tmp_frame );
  
      avio_closep( &amp;_formatCtx->pb );
  
      avformat_free_context( _formatCtx );
  
      sws_freeContext( _ost.sws_ctx );
  
      swr_free( &amp;_ost.swr_ctx);
  
      return 0;
  
  }
  
  </imgdata></algorithm>

  usage

  #include 
  
  #include 
  
  #include <vector>
  
  
  
  struct ImgData
  
  {
  
      unsigned int width;
  
      unsigned int height;
  
      std::string path;
  
      FIBITMAP * mp4Input;
  
  
  
      uint8_t * GetBits( FIBITMAP * bmp ) { return FreeImage_GetBits( bmp ); }
  
  };
  
  
  
  int main()
  
  {
  
       std::vector<imgdata> imgDataVec;
  
       //load images and push to imgDataVec
  
       MP4Writer mp4Writer;
  
       mp4Writer.SetHeight( 1200 ); //assumes 1:1 aspect ratio
  
       mp4Writer.Init();
  
       mp4Writer.SetOutput( "test.mp4" );
  
       mp4Writer.Write( imgDataVec );
  
  }
  
  </imgdata></vector>

• How to extract elementary video from mp4 using ffmpeg programmatically ?

  4 juillet 2013, par epipav

  I have started learning ffmpeg few weaks ago. At the moment I am able to transcode any video to mp4 using h264/AVC codec. The main scheme is something like that :

  -open input
   demux
   decode
   encode
   mux

  The actual code is below :

  #include <iostream>
  
  #include 
  
  extern "C" 
  
  {
  
  
  
  #ifndef __STDC_CONSTANT_MACROS
  
  #undef main /* Prevents SDL from overriding main() */
  
  #  define __STDC_CONSTANT_MACROS
  
  #endif
  
  
  
  
  
  
  
  #pragma comment (lib,"avcodec.lib")
  
  #pragma comment (lib,"avformat.lib")
  
  #pragma comment (lib,"swscale.lib")
  
  #pragma comment(lib,"avutil.lib")
  
  
  
  #include 
  
  #include 
  
  #include 
  
  #include 
  
  #include <libavutil></libavutil>opt.h>
  
  #include 
  
  #include 
  
  #include 
  
  #include 
  
  #include 
  
  
  
  }
  
  
  
  
  
  using namespace std;
  
  
  
  
  
  void open_video(AVFormatContext*oc , AVCodec *codec, AVStream * st)
  
  {
  
  int ret;
  
  AVCodecContext *c ;
  
  c = st->codec;
  
  
  
  /*open codec */
  
  
  
  cout &lt;&lt; "probably starts here" &lt;&lt; endl;
  
  ret = avcodec_open2(c,codec,NULL);
  
  cout &lt;&lt; "and ends here" &lt;&lt; endl;
  
  
  
  if ( ret &lt; 0)
  
  {
  
      cout &lt;&lt; ("Could not open video codec") &lt;&lt; endl;
  
  }
  
  
  
  
  
  }
  
  
  
  
  
  
  
  
  
  
  
  
  
  /*This function will add a new stream to our file.
  
  @param
  
  oc -> Format context that the new stream will be added.
  
  codec -> codec of the stream, this will be passed.
  
  codec_id -> 
  
  chWidth->
  
  chHeight->
  
  */
  
  
  
  AVStream * addStream(AVFormatContext * oc, AVCodec **codec, enum AVCodecID codec_id, int chWidth,        int chHeight, int fps)
  
  {
  
  AVCodecContext *c;
  
  AVStream *st;
  
  
  
  //find encoder of the stream, it passes this information to @codec, later on
  
  //it will be used in encoding the video @ avcodec_encode_video2 in loop.
  
  *codec = avcodec_find_encoder(AV_CODEC_ID_H264);
  
  
  
  if ( (*codec) == NULL)
  
      cout &lt;&lt; "ERROR CAN NOT FIND ENCODER! ERROR! ERROR! AVCODEC_FIND_ENCODER FAILED !!!1 """ &lt;&lt; endl;
  
  
  
  if(!(*codec))
  
      printf ("Could not find encoder for &#39; %s &#39; ", avcodec_get_name(codec_id));
  
  
  
  
  
  //create a new stream with the found codec inside oc(AVFormatContext).
  
  st = avformat_new_stream ( oc, *codec);
  
  
  
  if (!st)
  
      cout &lt;&lt; " Cannot allocate stream " &lt;&lt; endl;
  
  
  
  //Setting the stream id.
  
  //Since, there can be other streams in this AVFormatContext,
  
  //we should find the first non used index. And this is oc->nb_streams(number of streams) - 1
  
  st ->id = oc ->nb_streams - 1;
  
  
  
  c = st->codec;
  
  
  
  //setting the stream&#39;s codec&#39;s properties.
  
  c-> codec_id = codec_id;
  
  c->bit_rate = 4000000;
  
  c->width = chWidth;
  
  c->height = chHeight;
  
  c->time_base.den = fps;
  
      //fps;
  
  c->time_base.num = 1;
  
  c->gop_size = 12;
  
  c->pix_fmt = AV_PIX_FMT_YUV420P;
  
  
  
  if (c->codec_id == AV_CODEC_ID_MPEG2VIDEO) {
  
          /* just for testing, we also add B frames */
  
          c->max_b_frames = 2;
  
      }
  
  
  
  
  
  if (c->codec_id == AV_CODEC_ID_MPEG1VIDEO) {
  
          /* Needed to avoid using macroblocks in which some coeffs overflow.
  
           * This does not happen with normal video, it just happens here as
  
           * the motion of the chroma plane does not match the luma plane. */
  
          c->mb_decision = 2;
  
      }
  
  
  
  /* Some formats want stream headers to be separate. */
  
   if (oc->oformat->flags &amp; AVFMT_GLOBALHEADER)
  
      c->flags |= CODEC_FLAG_GLOBAL_HEADER;
  
  
  
  //returning our lovely new brand stream.
  
  return st;
  
  
  
  
  
  }
  
  
  
  int changeResolution ( string source, int format )
  
  {
  
  //Data members
  
  struct SwsContext   *sws_ctx = NULL;
  
  AVFrame             *pFrame = NULL;
  
  AVFrame             *outFrame = NULL;   
  
  AVPacket            packet;
  
  uint8_t             *buffer = NULL;
  
  uint8_t endcode[] = { 0, 0, 1, 0xb7 };
  
  AVDictionary        *optionsDict = NULL;
  
  AVFormatContext     *pFormatCtx = NULL; 
  
  AVFormatContext     *outputContext = NULL;
  
  AVCodecContext      *pCodecCtx;
  
  AVCodec             *pCodec ;
  
  AVCodec             *codec;
  
  AVCodec             *videoCodec;
  
  AVOutputFormat      *fmt;
  
  AVStream            *video_stream;
  
  int                 changeWidth;
  
  int                  changeHeight;
  
  int                 frameFinished;
  
  int                 numBytes;
  
  int                 fps;
  
  
  
  
  
  int lock = 0;
  
  
  
  //Register all codecs &amp; other important stuff. Vital!..
  
  av_register_all();
  
  
  
  
  
  //Selects the desired resolution.
  
  if (format == 0)
  
  {
  
      changeWidth = 320;
  
      changeHeight = 180;
  
  }
  
  
  
  else if (format == 1)
  
  {
  
      changeWidth = 640;
  
      changeHeight = 480;
  
  
  
  }
  
  else if (format == 2)
  
  {
  
      changeWidth = 960;
  
      changeHeight = 540;
  
  
  
  }
  
  else if (format == 3)
  
  {
  
      changeWidth = 1024;
  
      changeHeight = 768;
  
  
  
  }
  
  else
  
  {
  
      changeWidth = 1280;
  
      changeHeight = 720;
  
  }
  
  
  
  
  
  // Open video file
  
  int aaa;
  
  aaa = avformat_open_input(&amp;pFormatCtx, source.c_str(), NULL, NULL) ;
  
  if(aaa !=0)
  
  {
  
      cout &lt;&lt; " cannot open input file \n" &lt;&lt; endl;
  
      cout &lt;&lt; "aaa = " &lt;&lt; aaa  &lt;&lt; endl;
  
       return -1; // Couldn&#39;t open file   
  
  }
  
  
  
  // Retrieve stream information
  
  if(av_find_stream_info(pFormatCtx)&lt;0)
  
      return -1; // Couldn&#39;t find stream information
  
  
  
  //just checking duration casually for no reason
  
  /*int64_t duration = pFormatCtx->duration;
  
  
  
  cout &lt;&lt; "the duration is " &lt;&lt; duration &lt;&lt; " " &lt;&lt; endl;*/
  
  
  
  //this writes the info about the file
  
  av_dump_format(pFormatCtx, 0, 0, 0);
  
  cin >> lock;
  
  
  
  // Find the first video stream
  
  int videoStream=-1;
  
  int i;
  
  
  
  for(i=0; i&lt;3; i++)
  
       if(pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO) {
  
        videoStream=i;
  
        cout &lt;&lt; " lel \n " ;
  
       break;
  
  
  
   }
  
  
  
  
  
  if(videoStream==-1)
  
      return -1; // Didn&#39;t find a video stream
  
  
  
  // Get a pointer to the codec context for the video stream
  
  pCodecCtx=pFormatCtx->streams[videoStream]->codec;
  
  fps = pCodecCtx -> time_base.den;
  
  
  
  
  
  
  
  //Find the decoder of the input file, for the video stream
  
  pCodec=avcodec_find_decoder(pCodecCtx->codec_id);
  
  
  
  if(pCodec==NULL) {
  
      fprintf(stderr, "Unsupported codec!\n");
  
      return -1; // Codec not found
  
  }
  
  
  
  
  
  // Open codec, you must open it first, in order to use it.
  
  if(avcodec_open2(pCodecCtx, pCodec, &amp;optionsDict)&lt;0)
  
  return -1; // Could not open codec
  
  
  
  
  
  // Allocate video frame ( pFrame for taking the packets into, outFrame for processed frames to packet.)
  
   pFrame=avcodec_alloc_frame();
  
   outFrame = avcodec_alloc_frame();
  
  
  
  
  
  i=0;
  
  
  
  int ret;
  
  int video_frame_count = 0;
  
  
  
  //Initiate the outFrame set the buffer &amp; fill the properties
  
  numBytes=avpicture_get_size(PIX_FMT_YUV420P, changeWidth, changeHeight);
  
  buffer=(uint8_t *)av_malloc(numBytes*sizeof(uint8_t));
  
      avpicture_fill((AVPicture *)outFrame, buffer, PIX_FMT_YUV420P, changeWidth, changeHeight );  
  
  
  
  
  
  int pp;
  
  int frameNo = 0;
  
  
  
  //allocate the outputContext, it will be the AVFormatContext of our output file. 
  
  //It will try to find the format by giving the file name.
  
  avformat_alloc_output_context2(&amp;outputContext,NULL,NULL, "myoutput.mp4");
  
  
  
  //Cant find the file extension, using MPEG as default.
  
  if (!outputContext) {
  
      printf("Could not deduce output format from file extension: using MPEG.\n");
  
      avformat_alloc_output_context2(&amp;outputContext, NULL, "mpeg", "myoutput.mp4");
  
  }
  
  
  
  //Still cant set file extension, exit.
  
  if (!outputContext) {
  
      return 1;
  
  }
  
  
  
  //set AVOutputFormat fmt to our outputContext&#39;s format.
  
  fmt = outputContext -> oformat;
  
  video_stream = NULL;
  
  
  
  //If fmt has a valid codec_id, create a new video stream.
  
  //This function will set the streams codec &amp; codecs desired properties.
  
  //Stream&#39;s codec will be passed to videoCodec for later usage.
  
  if (fmt -> video_codec != AV_CODEC_ID_NONE)
  
      video_stream = addStream(outputContext, &amp;videoCodec, fmt ->video_codec, changeWidth, changeHeight,fps);
  
  
  
  
  
  //open the video using videoCodec. by avcodec_open2() i.e open the codec.
  
  if (video_stream)   
  
      open_video(outputContext, videoCodec, video_stream);
  
  
  
  //Creating our new output file.
  
  if (!(fmt->flags &amp; AVFMT_NOFILE)) {
  
      ret = avio_open(&amp;outputContext->pb, "toBeStreamed.264", AVIO_FLAG_WRITE);
  
      if (ret &lt; 0) {
  
          cout &lt;&lt; " cant open file " &lt;&lt; endl;
  
          return 1;
  
        }
  
     }
  
  
  
  //Writing the header of format context.
  
  //ret = avformat_write_header(outputContext, NULL);
  
  
  
  if (ret >= 0) {
  
      cout &lt;&lt; "writing header success !!!"  &lt;&lt; endl;
  
  }
  
  
  
  
  
    //Start reading packages from input file.
  
   while(av_read_frame(pFormatCtx, &amp;packet)>=0  ) {
  
  
  
  // Is this a packet from the video stream?  
  
  if(packet.stream_index==videoStream) {
  
  
  
  // Decode video package into frames
  
  ret = avcodec_decode_video2(pCodecCtx, pFrame, &amp;frameFinished, &amp;packet);
  
  
  
  if( ret &lt; 0)
  
  {
  
      printf ( " Error decoding frame !!.." );
  
      return ret;
  
  }
  
  
  
  if (frameFinished){
  
      printf("video_frame n:%d    coded_n:%d\n" , video_frame_count++, pFrame->coded_picture_number);
  
  }
  
  
  
  av_free_packet(&amp;packet);
  
  
  
  //do stuff with frame, in this case we are changing the resolution.
  
  static struct SwsContext *img_convert_ctx_in = NULL;
  
  if (img_convert_ctx_in == NULL)
  
  {
  
      img_convert_ctx_in =sws_getContext( pCodecCtx->width,
  
          pCodecCtx->height, 
  
          pCodecCtx->pix_fmt,
  
          changeWidth,
  
          changeHeight,
  
          PIX_FMT_YUV420P,
  
          SWS_BICUBIC,
  
          NULL,
  
          NULL, 
  
          NULL ); 
  
  
  
  }
  
  //scale the frames
  
  sws_scale(img_convert_ctx_in,
  
      pFrame->data, 
  
      pFrame->linesize,
  
      0, 
  
      pCodecCtx->height, 
  
      outFrame->data,
  
      outFrame->linesize);
  
  
  
  //initiate the pts value
  
  if ( frameNo == 0)
  
      outFrame->pts = 0;
  
  
  
  
  
      //calculate the pts value &amp; set it.
  
      outFrame->pts += av_rescale_q(1, video_stream->codec->time_base, video_stream->time_base);
  
  
  
      //encode frames into packages. Package passed in @packet.
  
      if(avcodec_encode_video2(outputContext->streams[0]->codec, &amp;packet, outFrame, &amp;pp) &lt; 0 )
  
        cout &lt;&lt; "Encoding frames into packages, failed. " &lt;&lt;  endl;
  
  
  
      frameNo++;
  
  
  
      //write the packages into file, resulting in creating a video file.
  
      av_interleaved_write_frame(outputContext,&amp;packet);
  
  
  
  
  
    }
  
  
  
  }
  
  
  
  
  
  
  
  av_free_packet(&amp;packet);
  
  //av_write_trailer(outputContext);
  
  
  
  avio_close(outputContext->pb);
  
  
  
  
  
  
  
   // Free the RGB image
  
   av_free(buffer);
  
   av_free(outFrame);
  
  
  
   // Free the YUV frame
  
   av_free(pFrame);
  
  
  
  // Close the codec
  
  avcodec_close(video_stream->codec);
  
  avcodec_close(pCodecCtx);
  
  
  
  // Close the video file
  
  avformat_close_input(&amp;pFormatCtx);
  
  
  
  return 0;
  
  
  
  
  
  }
  
  </iostream>

  at the end of the process I get my desired file with desired codec & container & resolution.

  My problem is, in a part of our project I need to get elementary video streams IN file. Such as example.264. However I can not add a stream without creating an AVFormatContext. I can not create an AVFormatContext because 264 files does not have a container,they are just raw video ?, as far as I know.

  I have tried the way in decoding_encoding.c which uses fwrite. However that example was for mpeg-2 codec and when I try to adapt that code to H264/AVC codec, I got "floating point division by zero" error from mediainfo and moreover, some of the properties of the video was not showing (such as FPS & playtime & quality factor). I think it has to do with the "endcode" the example adds at the end of the code. It is for mpeg-2. ( uint8_t endcode[] = 0, 0, 1, 0xb7  ; )

  Anyway, I would love to get a startpoint for this task. I have managed to come this far by using internet resources ( quite few & outdated for ffmpeg) but now I'm stuck a little.

• The 11th Hour RoQ Variation

  12 avril 2012, par Multimedia Mike — Game Hacking, dreamroq, Reverse Engineering, roq, Vector Quantization

  I have been looking at the RoQ file format almost as long as I have been doing practical multimedia hacking. However, I have never figured out how the RoQ format works on The 11th Hour, which was the game for which the RoQ format was initially developed. When I procured the game years ago, I remember finding what appeared to be RoQ files and shoving them through the open source decoders but not getting the right images out.

  I decided to dust off that old copy of The 11th Hour and have another go at it.

  
  
  

  Baseline
  The game consists of 4 CD-ROMs. Each disc has a media/ directory that has a series of files bearing the extension .gjd, likely the initials of one Graeme J. Devine. These are resource files which are merely headerless concatenations of other files. Thus, at first glance, one file might appear to be a single RoQ file. So that’s the source of some of the difficulty : Sending an apparent RoQ .gjd file through a RoQ player will often cause the program to complain when it encounters the header of another RoQ file.

  I have uploaded some samples to the usual place.

  However, even the frames that a player can decode (before encountering a file boundary within the resource file) look wrong.

  Investigating Codebooks Using dreamroq
  I wrote dreamroq last year– an independent RoQ playback library targeted towards embedded systems. I aimed it at a gjd file and quickly hit a codebook error.

  RoQ is a vector quantizer video codec that maintains a codebook of 256 2×2 pixel vectors. In the Quake III and later RoQ files, these are transported using a YUV 4:2:0 colorspace– 4 Y samples, a U sample, and a V sample to represent 4 pixels. This totals 6 bytes per vector. A RoQ codebook chunk contains a field that indicates the number of 2×2 vectors as well as the number of 4×4 vectors. The latter vectors are each comprised of 4 2×2 vectors.

  Thus, the total size of a codebook chunk ought to be (# of 2×2 vectors) * 6 + (# of 4×4 vectors) * 4.

  However, this is not the case with The 11th Hour RoQ files.

  Longer Codebooks And Mystery Colorspace
  Juggling the numbers for a few of the codebook chunks, I empirically determined that the 2×2 vectors are represented by 10 bytes instead of 6. Now I need to determine what exactly these 10 bytes represent.

  I should note that I suspect that everything else about these files lines up with successive generations of the format. For example if a file has 640×320 resolution, that amounts to 40×20 macroblocks. dreamroq iterates through 40×20 8×8 blocks and precisely exhausts the VQ bitstream. So that all looks valid. I’m just puzzled on the codebook format.

  Here is an example codebook dump :
  

  ID 0x1002, len = 0x0000014C, args = 0x1C0D
    0 : 00 00 00 00 00 00 00 00 80 80
    1 : 08 07 00 00 1F 5B 00 00 7E 81
    2 : 00 00 15 0F 00 00 40 3B 7F 84
    3 : 00 00 00 00 3A 5F 18 13 7E 84
    4 : 00 00 00 00 3B 63 1B 17 7E 85
    5 : 18 13 00 00 3C 63 00 00 7E 88
    6 : 00 00 00 00 00 00 59 3B 7F 81
    7 : 00 00 56 23 00 00 61 2B 80 80
    8 : 00 00 2F 13 00 00 79 63 81 83
    9 : 00 00 00 00 5E 3F AC 9B 7E 81
    10 : 1B 17 00 00 B6 EF 77 AB 7E 85
    11 : 2E 43 00 00 C1 F7 75 AF 7D 88
    12 : 6A AB 28 5F B6 B3 8C B3 80 8A
    13 : 86 BF 0A 03 D5 FF 3A 5F 7C 8C
    14 : 00 00 9E 6B AB 97 F5 EF 7F 80
    15 : 86 73 C8 CB B6 B7 B7 B7 85 8B
    16 : 31 17 84 6B E7 EF FF FF 7E 81
    17 : 79 AF 3B 5F FC FF E2 FF 7D 87
    18 : DC FF AE EF B3 B3 B8 B3 85 8B
    19 : EF FF F5 FF BA B7 B6 B7 88 8B
    20 : F8 FF F7 FF B3 B7 B7 B7 88 8B
    21 : FB FF FB FF B8 B3 B4 B3 85 88
    22 : F7 FF F7 FF B7 B7 B9 B7 87 8B
    23 : FD FF FE FF B9 B7 BB B7 85 8A
    24 : E4 FF B7 EF FF FF FF FF 7F 83
    25 : FF FF AC EB FF FF FC FF 7F 83
    26 : CC C7 F7 FF FF FF FF FF 7F 81
    27 : FF FF FE FF FF FF FF FF 80 80
  

  Note that 0x14C (the chunk size) = 332, 0x1C and 0x0D (the chunk arguments — count of 2×2 and 4×4 vectors, respectively) are 28 and 13. 28 * 10 + 13 * 4 = 332, so the numbers check out.

  Do you see any patterns in the codebook ? Here are some things I tried :

  • Treating the last 2 bytes as U & V and treating the first 4 as the 4 Y samples :
    
    
    
  • Treating the last 2 bytes as U & V and treating the first 8 as 4 16-bit little-endian Y samples :
    
    
    
  • Disregarding the final 2 bytes and treating the first 8 bytes as 4 RGB565 pixels (both little- and big-endian, respectively, shown here) :
    
    
    
  • Based on the type of data I’m seeing in these movies (which appears to be intended as overlays), I figured that some of these bits might indicate transparency ; here is 15-bit big-endian RGB which disregards the top bit of each pixel :
    
    
    

  These images are taken from the uploaded sample bdpuz.gjd, apparently a component of the puzzle represented in this screenshot.

  Unseen Types
  It has long been rumored that early RoQ files could contain JPEG images. I finally found one such specimen. One of the files bundled early in the uploaded fhpuz.gjd sample contains a JPEG frame. It’s a standard JFIF file and can easily be decoded after separating the bytes from the resource using ‘dd’. JPEGs serve as intraframes in the coding scheme, with successive RoQ frames moving objects on top.

  However, a new chunk type showed up as well, one identified by 0×1030. I have never encountered this type. Where could I possibly find data about this ? Fortunately, iD Games recently posted all of their open sourced games at Github. Reading through the code for their official RoQ decoder, I see that this is called a RoQ_PACKET. The name and the code behind it are both supremely unhelpful. The code is basically a no-op. The payloads of the various RoQ_PACKETs from one sample are observed to be either 8784, 14752, or 14760 bytes in length. It’s very likely that this serves the same purpose as the JPEG intraframes.

  Other Tidbits
  I read through the readme.txt on the first game disc and found this nugget :

          g)      Animations displayed normally or in SPOOKY MODE
                  SPOOKY MODE is blue-tinted grayscale with color cursors, puzzle
  
                  and game pieces.  It is the preferred display setting of the
  
                  developers at Trilobyte.  Just for fun, try out the SPOOKY
  
                  MODE.
  

  The MobyGames screenshot page has a number of screenshots labeled as being captured in spooky mode. Color tricks ?

  Meanwhile, another twist arose as I kept tweaking dreamroq to deal with more RoQ weirdness : After modifying my dreamroq code to handle these 10-byte vectors, it eventually chokes on another codebook. These codebooks happen to have 6-byte vectors again ! Fortunately, I was already working on a scheme to automatically detect which codebook is in play (plugging the numbers into a formula and seeing which vector size checks out).