Recherche avancée

Sur d’autres sites (11304)

• RoQ on Dreamcast

  18 mars 2011, par Multimedia Mike — Sega Dreamcast

  I have been working on that challenge to play back video on the Sega Dreamcast. To review, I asserted that the RoQ format would be a good fit for the Sega Dreamcast hardware. The goal was to play 640x480 video at 30 frames/second. Short version : I have determined that it is possible to decode such video in real time. However, I ran into certain data rate caveats.

  First off : Have you ever wondered if the Dreamcast can read an 80mm optical disc ? It can ! I discovered this when I only had 60 MB of RoQ samples to burn on a disc and a spindle full of these 210MB-capacity 80mm CD-Rs that I never have occasion to use.

  
  
  

  New RoQ Library
  There are open source RoQ decoders out there but I decided to write a new one. A few reasons : 1) RoQ is so simple that I didn’t think it would take too long ; 2) it would be nice to have a RoQ library that is license-compatible (BSD-like) with the rest of the KallistiOS distribution ; 3) the idroq.tar.gz distribution, while license-compatible, has enough issues that I didn’t want to correct it.

  Thankfully, I was correct about the task not being too difficult : I put together a new RoQ decoder in short order. I’m a bit embarrassed to admit that the part I had the most trouble with was properly converting YUV -> RGB.
  
  About the approach I took : While the original idroq.tar.gz decoder maintains YUV 4:2:0 codebooks (which led to chroma bugs during motion compensation) and FFmpeg’s decoder maintains YUV 4:4:4 codebooks, this decoder is built to convert the YUV 4:2:0 vectors into RGB565 vectors during the vector unpacking phase. Thus, the entire frame is rendered in RGB565 — no lengthy YUV -> RGB conversion after decoding — and all pixels are shuffled around as 16-bit units (minor speedup vs. shuffling everything as bytes).

  I also entertained the idea of maintaining YUYV codebooks (since the DC supports that colorspace as a texture format). But I scrapped that idea when I remembered it would lead to the same chroma bleeding problem seen in the original idroq.tar.gz decoder.

  Onto The Dreamcast
  I developed the library on a Linux computer, allowing it to output a series of PNM files for visual verification and debugging. Dropping it into a basic DC/KOS-compatible program was trivial and the first order of business was profiling.

  At first, I profiled the entire decode operation : open file, then read and decode each chunk while tossing away the results. I was roundly disappointed to see that, e.g., an 8.5-second RoQ sample needed a little more than 20 seconds to complete. Not real time. I performed a series of optimizations on the decoding library that netted notable performance gains when profiling on Linux. When I brought these same optimizations over to the DC, decoding time didn’t improve at all. This was my first suspicion that perhaps my assumptions regarding the DC’s optical drive’s data rate were not correct.

  Dreamcast Data Rate Profiling
  Let’s start with some definitions : In terms of data rate, an ’X’, i.e., 1X is the minimum data rate needed to read CD quality audio from a disc. At that speed, a drive should be able to stream 75 sectors each second. When reading mode 1/form 1 CD-ROM data, each sector has 2048 bytes (2 kbytes), so a single-speed data rate should achieve 150 kbytes/sec.

  The Dreamcast is supposed to possess a 12X optical drive. This would imply a maximum data rate of 150 kbytes/sec * 12 = 1800 kbytes/sec.

  Rigging up a trivial experiment using the RoQ samples burned on a few different CD-R discs, the best data rate I can see is about 500-525 kbytes/sec, or around 3.5X.

  Where’s the discrepancy ? My first theory has to do with the fact that not all optical media is created equal. This is why optical drives often advertise a slew of numbers which refer to the best theoretical speed for reading a CD vs. writing a CD-R vs. writing a CD-RW, etc. Perhaps the DC drive can’t read CD-Rs very quickly. To test this theory, I tried streaming a large file from a conventionally mastered CD-ROM. This worked well for the closest CD-ROM I had on hand : I was able to stream data at a rate that works out to about 6.5X.

  I smell a science project for another evening : Profiling read speeds from a mastered CD-ROM, burned CD-R, and also a mastered GD-ROM, on each of the 3 Dreamcast consoles I possess (I’ve heard that there’s variance between optical drives depending on manufacturing run).

  The Good News
  I added a little finer-grained code to profile just the video decoding functions. The good news is that the decoder meets my real time goals : That 8.5-second RoQ sample encoded at 640x480x30fps makes its way through the video decoding functions on the DC in a little less than 5 seconds. If the optical drive can supply the data fast enough, the video decoder can take care of the rest.

  The RoQ encoder included with FFmpeg does not honor any bitrate parameters. Instead, I encoded the same file at 320x240. It reportedly decoded in real time and can be streamed in real time as well.

  I say "reportedly" because I’m simply working from textual output at this point ; the next phase is to hook the decoder up to the display hardware.

• ffmpeg in C++ "av_register_all not declared in this scope"

  4 juillet 2013, par Tom

  I'm trying to compile this simple program in C++ (in Code Blocks) :

  #ifdef __cplusplus
  
  extern "C" {
  
      #endif
  
      #include <libavutil></libavutil>avutil.h>
  
      #include <libavcodec></libavcodec>avcodec.h>
  
  #ifdef __cplusplus
  
  };
  
  #endif
  
  
  
  int main(int argc, char *argv[]) {
  
      av_register_all();
  
      return 0;
  
  }

  However I keep getting the error message :
  |11|error : ‘av_register_all’ was not declared in this scope|

  Other people seem to have had this problem and adding the extern "C" section seems to have solved it form them but not me. Does anyone have any suggestions ?

  Thanks

• C - Transcoding to UDP using FFmpeg ?

  30 avril 2013, par golmschenk

  I'm trying to use the FFmpeg libraries to take an existing video file and stream it over a UDP connection. Specifically, I've been looking at the muxing.c and demuxing.c example files in the source code doc/example directory of FFmpeg. The demuxing file presents code which allows an input video to be converted into the video and audio streams. The muxing file presents code which creates fake data and can already be output to a UDP connection as I would like. I've begun work combining the two. Below can be found my code which is basically a copy of the muxing file with some parts replaced/appended with parts of the demuxing file. Unfortunately I'm running into plenty of complications attempting my goal through this approach. Is there an existing source code example which does the transcoding I'm looking for ? Or at least a tutorial on how one might create this ? If not, at least a few pointers might be helpful in directing my work in combing the two files to achieve my goal. Specifically, I'm getting the error :

  [NULL @ 0x23b4040] Unable to find a suitable output format for 'udp://localhost:7777'
  
  Could not deduce output format from file extension: using MPEG.
  
  Output #0, mpeg, to 'udp://localhost:7777':

  Even though the muxing file could accept UDP formats. Any suggestions ? Thank you much !

  #include 
  
  #include 
  
  #include 
  
  #include 
  
  
  
  #include <libavutil></libavutil>mathematics.h>
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libswscale></libswscale>swscale.h>
  
  
  
  /* 5 seconds stream duration */
  
  #define STREAM_DURATION   200.0
  
  #define STREAM_FRAME_RATE 25 /* 25 images/s */
  
  #define STREAM_NB_FRAMES  ((int)(STREAM_DURATION * STREAM_FRAME_RATE))
  
  #define STREAM_PIX_FMT    AV_PIX_FMT_YUV420P /* default pix_fmt */
  
  
  
  //FROM DE
  
  static AVFormatContext *fmt_ctx = NULL;
  
  static AVCodecContext *video_dec_ctx = NULL, *audio_dec_ctx;
  
  static AVStream *video_stream = NULL, *audio_stream = NULL;
  
  static const char *src_filename = NULL;
  
  static const char *video_dst_filename = NULL;
  
  static const char *audio_dst_filename = NULL;
  
  static FILE *video_dst_file = NULL;
  
  static FILE *audio_dst_file = NULL;
  
  
  
  static uint8_t *video_dst_data[4] = {NULL};
  
  static int      video_dst_linesize[4];
  
  static int video_dst_bufsize;
  
  
  
  static uint8_t **audio_dst_data = NULL;
  
  static int       audio_dst_linesize;
  
  static int audio_dst_bufsize;
  
  
  
  static int video_stream_idx = -1, audio_stream_idx = -1;
  
  static AVFrame *frame = NULL;
  
  static AVPacket pkt;
  
  static int video_frame_count = 0;
  
  static int audio_frame_count = 0;
  
  //END DE
  
  
  
  static int sws_flags = SWS_BICUBIC;
  
  
  
  /**************************************************************/
  
  /* audio output */
  
  
  
  static float t, tincr, tincr2;
  
  static int16_t *samples;
  
  static int audio_input_frame_size;
  
  
  
  /* Add an output stream. */
  
  static AVStream *add_stream(AVFormatContext *oc, AVCodec **codec,
  
                              enum AVCodecID codec_id)
  
  {
  
      AVCodecContext *c;
  
      AVStream *st;
  
  
  
      /* find the encoder */
  
      *codec = avcodec_find_encoder(codec_id);
  
      if (!(*codec)) {
  
          fprintf(stderr, "Could not find encoder for &#39;%s&#39;\n",
  
                  avcodec_get_name(codec_id));
  
          exit(1);
  
      }
  
  
  
      st = avformat_new_stream(oc, *codec);
  
      if (!st) {
  
          fprintf(stderr, "Could not allocate stream\n");
  
          exit(1);
  
      }
  
      st->id = oc->nb_streams-1;
  
      c = st->codec;
  
  
  
      switch ((*codec)->type) {
  
      case AVMEDIA_TYPE_AUDIO:
  
          st->id = 1;
  
          c->sample_fmt  = AV_SAMPLE_FMT_S16;
  
          c->bit_rate    = 64000;
  
          c->sample_rate = 44100;
  
          c->channels    = 2;
  
          break;
  
  
  
      case AVMEDIA_TYPE_VIDEO:
  
          c->codec_id = codec_id;
  
  
  
          c->bit_rate = 400000;
  
          /* Resolution must be a multiple of two. */
  
          c->width    = 352;
  
          c->height   = 288;
  
          /* timebase: This is the fundamental unit of time (in seconds) in terms
  
           * of which frame timestamps are represented. For fixed-fps content,
  
           * timebase should be 1/framerate and timestamp increments should be
  
           * identical to 1. */
  
          c->time_base.den = STREAM_FRAME_RATE;
  
          c->time_base.num = 1;
  
          c->gop_size      = 12; /* emit one intra frame every twelve frames at most */
  
          c->pix_fmt       = STREAM_PIX_FMT;
  
          if (c->codec_id == AV_CODEC_ID_MPEG2VIDEO) {
  
              /* just for testing, we also add B frames */
  
              c->max_b_frames = 2;
  
          }
  
          if (c->codec_id == AV_CODEC_ID_MPEG1VIDEO) {
  
              /* Needed to avoid using macroblocks in which some coeffs overflow.
  
               * This does not happen with normal video, it just happens here as
  
               * the motion of the chroma plane does not match the luma plane. */
  
              c->mb_decision = 2;
  
          }
  
      break;
  
  
  
      default:
  
          break;
  
      }
  
  
  
      /* Some formats want stream headers to be separate. */
  
      if (oc->oformat->flags &amp; AVFMT_GLOBALHEADER)
  
          c->flags |= CODEC_FLAG_GLOBAL_HEADER;
  
  
  
      return st;
  
  }
  
  
  
  /**************************************************************/
  
  /* audio output */
  
  
  
  static float t, tincr, tincr2;
  
  static int16_t *samples;
  
  static int audio_input_frame_size;
  
  
  
  static void open_audio(AVFormatContext *oc, AVCodec *codec, AVStream *st)
  
  {
  
      AVCodecContext *c;
  
      int ret;
  
  
  
      c = st->codec;
  
  
  
      /* open it */
  
      ret = avcodec_open2(c, codec, NULL);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not open audio codec: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      /* init signal generator */
  
      t     = 0;
  
      tincr = 2 * M_PI * 110.0 / c->sample_rate;
  
      /* increment frequency by 110 Hz per second */
  
      tincr2 = 2 * M_PI * 110.0 / c->sample_rate / c->sample_rate;
  
  
  
      if (c->codec->capabilities &amp; CODEC_CAP_VARIABLE_FRAME_SIZE)
  
          audio_input_frame_size = 10000;
  
      else
  
          audio_input_frame_size = c->frame_size;
  
      samples = av_malloc(audio_input_frame_size *
  
                          av_get_bytes_per_sample(c->sample_fmt) *
  
                          c->channels);
  
      if (!samples) {
  
          fprintf(stderr, "Could not allocate audio samples buffer\n");
  
          exit(1);
  
      }
  
  }
  
  
  
  /* Prepare a 16 bit dummy audio frame of &#39;frame_size&#39; samples and
  
   * &#39;nb_channels&#39; channels. */
  
  static void get_audio_frame(int16_t *samples, int frame_size, int nb_channels)
  
  {
  
      int j, i, v;
  
      int16_t *q;
  
  
  
      q = samples;
  
      for (j = 0; j &lt; frame_size; j++) {
  
          v = (int)(sin(t) * 10000);
  
          for (i = 0; i &lt; nb_channels; i++)
  
              *q++ = v;
  
          t     += tincr;
  
          tincr += tincr2;
  
      }
  
  }
  
  
  
  static void write_audio_frame(AVFormatContext *oc, AVStream *st)
  
  {
  
      AVCodecContext *c;
  
      AVPacket pkt = { 0 }; // data and size must be 0;
  
      AVFrame *frame = avcodec_alloc_frame();
  
      int got_packet, ret;
  
  
  
      av_init_packet(&amp;pkt);
  
      c = st->codec;
  
  
  
      get_audio_frame(samples, audio_input_frame_size, c->channels);
  
      frame->nb_samples = audio_input_frame_size;
  
      avcodec_fill_audio_frame(frame, c->channels, c->sample_fmt,
  
                               (uint8_t *)samples,
  
                               audio_input_frame_size *
  
                               av_get_bytes_per_sample(c->sample_fmt) *
  
                               c->channels, 1);
  
  
  
      ret = avcodec_encode_audio2(c, &amp;pkt, frame, &amp;got_packet);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Error encoding audio frame: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      if (!got_packet)
  
          return;
  
  
  
      pkt.stream_index = st->index;
  
  
  
      /* Write the compressed frame to the media file. */
  
      ret = av_interleaved_write_frame(oc, &amp;pkt);
  
      if (ret != 0) {
  
          fprintf(stderr, "Error while writing audio frame: %s\n",
  
                  av_err2str(ret));
  
          exit(1);
  
      }
  
      avcodec_free_frame(&amp;frame);
  
  }
  
  
  
  static void close_audio(AVFormatContext *oc, AVStream *st)
  
  {
  
      avcodec_close(st->codec);
  
  
  
      av_free(samples);
  
  }
  
  
  
  /**************************************************************/
  
  /* video output */
  
  
  
  static AVFrame *frame;
  
  static AVPicture src_picture, dst_picture;
  
  static int frame_count;
  
  
  
  static void open_video(AVFormatContext *oc, AVCodec *codec, AVStream *st)
  
  {
  
      int ret;
  
      AVCodecContext *c = st->codec;
  
  
  
      /* open the codec */
  
      ret = avcodec_open2(c, codec, NULL);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not open video codec: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      /* allocate and init a re-usable frame */
  
      frame = avcodec_alloc_frame();
  
      if (!frame) {
  
          fprintf(stderr, "Could not allocate video frame\n");
  
          exit(1);
  
      }
  
  
  
      /* Allocate the encoded raw picture. */
  
      ret = avpicture_alloc(&amp;dst_picture, c->pix_fmt, c->width, c->height);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not allocate picture: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      /* If the output format is not YUV420P, then a temporary YUV420P
  
       * picture is needed too. It is then converted to the required
  
       * output format. */
  
      if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
  
          ret = avpicture_alloc(&amp;src_picture, AV_PIX_FMT_YUV420P, c->width, c->height);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Could not allocate temporary picture: %s\n",
  
                      av_err2str(ret));
  
              exit(1);
  
          }
  
      }
  
  
  
      /* copy data and linesize picture pointers to frame */
  
      *((AVPicture *)frame) = dst_picture;
  
  }
  
  
  
  /* Prepare a dummy image. */
  
  static void fill_yuv_image(AVPicture *pict, int frame_index,
  
                             int width, int height)
  
  {
  
      int x, y, i;
  
  
  
      i = frame_index;
  
  
  
      /* Y */
  
      for (y = 0; y &lt; height; y++)
  
          for (x = 0; x &lt; width; x++)
  
              pict->data[0][y * pict->linesize[0] + x] = x + y + i * 3;
  
  
  
      /* Cb and Cr */
  
      for (y = 0; y &lt; height / 2; y++) {
  
          for (x = 0; x &lt; width / 2; x++) {
  
              pict->data[1][y * pict->linesize[1] + x] = 128 + y + i * 2;
  
              pict->data[2][y * pict->linesize[2] + x] = 64 + x + i * 5;
  
          }
  
      }
  
  }
  
  
  
  static void write_video_frame(AVFormatContext *oc, AVStream *st)
  
  {
  
      int ret;
  
      static struct SwsContext *sws_ctx;
  
      AVCodecContext *c = st->codec;
  
  
  
      if (frame_count >= STREAM_NB_FRAMES) {
  
          /* No more frames to compress. The codec has a latency of a few
  
           * frames if using B-frames, so we get the last frames by
  
           * passing the same picture again. */
  
      } else {
  
          if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
  
              /* as we only generate a YUV420P picture, we must convert it
  
               * to the codec pixel format if needed */
  
              if (!sws_ctx) {
  
                  sws_ctx = sws_getContext(c->width, c->height, AV_PIX_FMT_YUV420P,
  
                                           c->width, c->height, c->pix_fmt,
  
                                           sws_flags, NULL, NULL, NULL);
  
                  if (!sws_ctx) {
  
                      fprintf(stderr,
  
                              "Could not initialize the conversion context\n");
  
                      exit(1);
  
                  }
  
              }
  
              fill_yuv_image(&amp;src_picture, frame_count, c->width, c->height);
  
              sws_scale(sws_ctx,
  
                        (const uint8_t * const *)src_picture.data, src_picture.linesize,
  
                        0, c->height, dst_picture.data, dst_picture.linesize);
  
          } else {
  
              fill_yuv_image(&amp;dst_picture, frame_count, c->width, c->height);
  
          }
  
      }
  
  
  
      if (oc->oformat->flags &amp; AVFMT_RAWPICTURE) {
  
          /* Raw video case - directly store the picture in the packet */
  
          AVPacket pkt;
  
          av_init_packet(&amp;pkt);
  
  
  
          pkt.flags        |= AV_PKT_FLAG_KEY;
  
          pkt.stream_index  = st->index;
  
          pkt.data          = dst_picture.data[0];
  
          pkt.size          = sizeof(AVPicture);
  
  
  
          ret = av_interleaved_write_frame(oc, &amp;pkt);
  
      } else {
  
          AVPacket pkt = { 0 };
  
          int got_packet;
  
          av_init_packet(&amp;pkt);
  
  
  
          /* encode the image */
  
          ret = avcodec_encode_video2(c, &amp;pkt, frame, &amp;got_packet);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Error encoding video frame: %s\n", av_err2str(ret));
  
              exit(1);
  
          }
  
          /* If size is zero, it means the image was buffered. */
  
  
  
          if (!ret &amp;&amp; got_packet &amp;&amp; pkt.size) {
  
              pkt.stream_index = st->index;
  
  
  
              /* Write the compressed frame to the media file. */
  
              ret = av_interleaved_write_frame(oc, &amp;pkt);
  
          } else {
  
              ret = 0;
  
          }
  
      }
  
      if (ret != 0) {
  
          fprintf(stderr, "Error while writing video frame: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
      frame_count++;
  
  }
  
  
  
  static void close_video(AVFormatContext *oc, AVStream *st)
  
  {
  
      avcodec_close(st->codec);
  
      av_free(src_picture.data[0]);
  
      av_free(dst_picture.data[0]);
  
      av_free(frame);
  
  }
  
  
  
  static int open_codec_context(int *stream_idx,
  
                                AVFormatContext *fmt_ctx, enum AVMediaType type)
  
  {
  
      int ret;
  
      AVStream *st;
  
      AVCodecContext *dec_ctx = NULL;
  
      AVCodec *dec = NULL;
  
  
  
      ret = av_find_best_stream(fmt_ctx, type, -1, -1, NULL, 0);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not find %s stream in input file &#39;%s&#39;\n",
  
                  av_get_media_type_string(type), src_filename);
  
          return ret;
  
      } else {
  
          *stream_idx = ret;
  
          st = fmt_ctx->streams[*stream_idx];
  
  
  
          /* find decoder for the stream */
  
          dec_ctx = st->codec;
  
          dec = avcodec_find_decoder(dec_ctx->codec_id);
  
          if (!dec) {
  
              fprintf(stderr, "Failed to find %s codec\n",
  
                      av_get_media_type_string(type));
  
              return ret;
  
          }
  
  
  
          if ((ret = avcodec_open2(dec_ctx, dec, NULL)) &lt; 0) {
  
              fprintf(stderr, "Failed to open %s codec\n",
  
                      av_get_media_type_string(type));
  
              return ret;
  
          }
  
      }
  
  
  
      return 0;
  
  }
  
  
  
  /**************************************************************/
  
  /* media file output */
  
  
  
  int main(int argc, char **argv)
  
  {
  
      const char *filename;
  
      AVOutputFormat *fmt;
  
      AVFormatContext *oc;
  
      AVStream *audio_st, *video_st;
  
      AVCodec *audio_codec, *video_codec;
  
      double audio_pts, video_pts;
  
      int ret = 0, got_frame;;
  
  
  
      /* Initialize libavcodec, and register all codecs and formats. */
  
      av_register_all();
  
  
  
      if (argc != 3) {
  
          printf("usage: %s input_file output_file\n"
  
                 "\n", argv[0]);
  
          return 1;
  
      }
  
  
  
      src_filename = argv[1];
  
      filename = argv[2];
  
  
  
      /* allocate the output media context */
  
      avformat_alloc_output_context2(&amp;oc, NULL, NULL, filename);
  
      if (!oc) {
  
          printf("Could not deduce output format from file extension: using MPEG.\n");
  
          avformat_alloc_output_context2(&amp;oc, NULL, "mpeg", filename);
  
      }
  
      if (!oc) {
  
          return 1;
  
      }
  
      fmt = oc->oformat;
  
  
  
      /* Add the audio and video streams using the default format codecs
  
       * and initialize the codecs. */
  
      video_stream = NULL;
  
      audio_stream = NULL;
  
  
  
      //FROM DE
  
      /* open input file, and allocate format context */
  
      if (avformat_open_input(&amp;fmt_ctx, src_filename, NULL, NULL) &lt; 0) {
  
          fprintf(stderr, "Could not open source file %s\n", src_filename);
  
          exit(1);
  
      }
  
  
  
      /* retrieve stream information */
  
      if (avformat_find_stream_info(fmt_ctx, NULL) &lt; 0) {
  
          fprintf(stderr, "Could not find stream information\n");
  
          exit(1);
  
      }
  
      if (open_codec_context(&amp;video_stream_idx, fmt_ctx, AVMEDIA_TYPE_VIDEO) >= 0) {
  
          video_stream = fmt_ctx->streams[video_stream_idx];
  
          video_dec_ctx = video_stream->codec;
  
  
  
          /* allocate image where the decoded image will be put */
  
          ret = av_image_alloc(video_dst_data, video_dst_linesize,
  
                               video_dec_ctx->width, video_dec_ctx->height,
  
                               video_dec_ctx->pix_fmt, 1);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Could not allocate raw video buffer\n");
  
              goto end;
  
          }
  
          video_dst_bufsize = ret;
  
      }
  
  
  
      if (open_codec_context(&amp;audio_stream_idx, fmt_ctx, AVMEDIA_TYPE_AUDIO) >= 0) {
  
          int nb_planes;
  
  
  
          audio_stream = fmt_ctx->streams[audio_stream_idx];
  
          audio_dec_ctx = audio_stream->codec;
  
  
  
          nb_planes = av_sample_fmt_is_planar(audio_dec_ctx->sample_fmt) ?
  
              audio_dec_ctx->channels : 1;
  
          audio_dst_data = av_mallocz(sizeof(uint8_t *) * nb_planes);
  
          if (!audio_dst_data) {
  
              fprintf(stderr, "Could not allocate audio data buffers\n");
  
              ret = AVERROR(ENOMEM);
  
              goto end;
  
          }
  
      }
  
      //END DE
  
  
  
      /* Now that all the parameters are set, we can open the audio and
  
       * video codecs and allocate the necessary encode buffers. */
  
      if (video_stream)
  
          open_video(oc, video_codec, video_stream);
  
      if (audio_stream)
  
          open_audio(oc, audio_codec, audio_stream);
  
  
  
      av_dump_format(oc, 0, filename, 1);
  
  
  
      /* open the output file, if needed */
  
      if (!(fmt->flags &amp; AVFMT_NOFILE)) {
  
          ret = avio_open(&amp;oc->pb, filename, AVIO_FLAG_WRITE);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Could not open &#39;%s&#39;: %s\n", filename,
  
                      av_err2str(ret));
  
              return 1;
  
          }
  
      }
  
  
  
      /* Write the stream header, if any. */
  
      ret = avformat_write_header(oc, NULL);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Error occurred when opening output file: %s\n",
  
                  av_err2str(ret));
  
          return 1;
  
      }
  
  
  
      if (frame)
  
          frame->pts = 0;
  
      for (;;) {
  
          /* Compute current audio and video time. */
  
          if (audio_stream)
  
              audio_pts = (double)audio_stream->pts.val * audio_stream->time_base.num / audio_stream->time_base.den;
  
          else
  
              audio_pts = 0.0;
  
  
  
          if (video_stream)
  
              video_pts = (double)video_stream->pts.val * video_stream->time_base.num /
  
                          video_stream->time_base.den;
  
          else
  
              video_pts = 0.0;
  
  
  
          if ((!audio_stream || audio_pts >= STREAM_DURATION) &amp;&amp;
  
              (!video_stream || video_pts >= STREAM_DURATION))
  
              break;
  
  
  
          /* write interleaved audio and video frames */
  
          if (!video_stream || (video_stream &amp;&amp; audio_st &amp;&amp; audio_pts &lt; video_pts)) {
  
              write_audio_frame(oc, audio_stream);
  
          } else {
  
              write_video_frame(oc, video_stream);
  
              frame->pts += av_rescale_q(1, video_stream->codec->time_base, video_stream->time_base);
  
          }
  
      }
  
  
  
      /* Write the trailer, if any. The trailer must be written before you
  
       * close the CodecContexts open when you wrote the header; otherwise
  
       * av_write_trailer() may try to use memory that was freed on
  
       * av_codec_close(). */
  
      av_write_trailer(oc);
  
  
  
      /* Close each codec. */
  
      if (video_st)
  
          close_video(oc, video_st);
  
      if (audio_st)
  
          close_audio(oc, audio_st);
  
  
  
      if (!(fmt->flags &amp; AVFMT_NOFILE))
  
          /* Close the output file. */
  
          avio_close(oc->pb);
  
  
  
      /* free the stream */
  
      avformat_free_context(oc);
  
  
  
  end:
  
      if (video_dec_ctx)
  
          avcodec_close(video_dec_ctx);
  
      if (audio_dec_ctx)
  
          avcodec_close(audio_dec_ctx);
  
      avformat_close_input(&amp;fmt_ctx);
  
      if (video_dst_file)
  
          fclose(video_dst_file);
  
      if (audio_dst_file)
  
          fclose(audio_dst_file);
  
      av_free(frame);
  
      av_free(video_dst_data[0]);
  
      av_free(audio_dst_data);
  
  
  
      return 0;
  
  }