Recherche avancée

Sur d’autres sites (12734)

• Adding C64 SID Music

  1er novembre 2012, par Multimedia Mike — General

  I have been working on adding support for SID files — the music format for the Commodore 64 — to the game music website for awhile. I feel a bit out of my element since I’m not that familiar with the C64. But why should I let that slow me down ? Allow me to go through the steps I have previously outlined in order to make this happen.

  
  
  

  I need to know what picture should represent the system in the search results page. The foregoing picture should be fine, but I’m getting way ahead of myself.

  Phase 1 is finding adequate player software. The most venerable contender in this arena is libsidplay, or so I first thought. It turns out that there’s libsidplay (originally hosted at Geocities, apparently, and no longer on the net) and also libsidplay2. Both are kind of old (libsidplay2 was last updated in 2004). I tried to compile libsidplay2 and the C++ didn’t agree with current version of g++.

  However, a recent effort named libsidplayfp is carrying on the SID emulation tradition. It works rather well, notwithstanding the fact that compiling the entire library has a habit of apparently hanging the Linux VM where I develop this stuff.

  Phase 2 is to develop a testbench app around the playback library. With the help of the libsidplayfp library maintainers, I accomplished this. The testbench app consistently requires about 15% of a single core of a fairly powerful Core i7. So I look forward to recommendations that I port that playback library to pure JavaScript.

  Phase 3 is plug into the web player. I haven’t worked on this yet. I’m confident that this will work since phase 2 worked (plus, I have a plan to combine phases 2 and 3).

  One interesting issue that has arisen is that proper operation of libsidplayfp requires that 3 C64 ROM files be present (the, ahem, KERNAL, BASIC interpreter, and character generator). While these are copyrighted ROMs, they are easily obtainable on the internet. The goal of my project is to eliminate as much friction as possible for enjoying these old tunes. To that end, I will just bake the ROM files directly into the player.

  Phase 4 is collecting a SID song corpus. This is the simplest part of the whole process thanks to the remarkable curation efforts of the High Voltage SID Collection (HVSC). Anyone can download a giant archive of every known SID file. So that’s a done deal.

  Or is it ? One small issue is that I was hoping that the first iteration of my game music website would focus on, well, game music. There is a lot of music in the HVSC that are original compositions or come from demos. The way that the archive is organized makes it difficult to automatically discern whether a particular SID file comes from a game or not.

  Phase 5 is munging the metadata. The good news here is that the files have the metadata built in. The not-so-great news is that there isn’t quite as much as I might like. Each file is tagged with title, author, and publisher/copyright. If there is more than one song in a file, they all have the same metadata. Fortunately, if I can import them all into my game music database, there is an opportunity to add a lot more metadata.

  Further, there is no play length metadata for these files. This means I will need to set each to a default length like 2 minutes and do something like I did before in order to automatically determine if any songs terminate sooner.

  Oddly, the issue I’m most concerned about is character encoding. This is the first project for which I’m making certain that I understand character encoding since I can’t reasonably get away with assuming that everything is ASCII. So far, based on the random sampling of SID files I have checked, there is a good chance of encountering metadata strings with characters that are not in the lower ASCII set. From what I have observed, these characters map to Unicode code points. So I finally get to learn about manipulating strings in such a way that it preserves the character encoding. At the very least, I need Python to rip the strings out of the binary SID files and make sure the Unicode remains intact while being inserted into an SQLite3 database.

• FFMPEG RTSP Server using muxing doc example

  11 novembre 2018, par Harshil Makwana

  I am trying to develop RTSP server using FFMPEG. For that I slightly modified muxing file located at doc/example/ folder inside FFMPEG repository.

  Giving my source code of RTSP server example :

  #include 
  
  #include 
  
  #include 
  
  #include 
  
  
  
  #include <libavutil></libavutil>avassert.h>
  
  #include <libavutil></libavutil>channel_layout.h>
  
  #include <libavutil></libavutil>opt.h>
  
  #include <libavutil></libavutil>mathematics.h>
  
  #include <libavutil></libavutil>timestamp.h>
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libswscale></libswscale>swscale.h>
  
  #include <libswresample></libswresample>swresample.h>
  
  
  
  #define STREAM_DURATION   10.0
  
  #define STREAM_FRAME_RATE 25 /* 25 images/s */
  
  #define STREAM_PIX_FMT    AV_PIX_FMT_YUV420P /* default pix_fmt */
  
  
  
  #define SCALE_FLAGS SWS_BICUBIC
  
  
  
  // a wrapper around a single output AVStream
  
  typedef struct OutputStream {
  
      AVStream *st;
  
      AVCodecContext *enc;
  
  
  
      /* pts of the next frame that will be generated */
  
      int64_t next_pts;
  
      int samples_count;
  
  
  
      AVFrame *frame;
  
      AVFrame *tmp_frame;
  
  
  
      float t, tincr, tincr2;
  
  
  
      struct SwsContext *sws_ctx;
  
      struct SwrContext *swr_ctx;
  
  } OutputStream;
  
  
  
  static void log_packet(const AVFormatContext *fmt_ctx, const AVPacket *pkt)
  
  {
  
      AVRational *time_base = &amp;fmt_ctx->streams[pkt->stream_index]->time_base;
  
  
  
      printf("pts:%s pts_time:%s dts:%s dts_time:%s duration:%s duration_time:%s stream_index:%d\n",
  
             av_ts2str(pkt->pts), av_ts2timestr(pkt->pts, time_base),
  
             av_ts2str(pkt->dts), av_ts2timestr(pkt->dts, time_base),
  
             av_ts2str(pkt->duration), av_ts2timestr(pkt->duration, time_base),
  
             pkt->stream_index);
  
  }
  
  
  
  static int write_frame(AVFormatContext *fmt_ctx, const AVRational *time_base, AVStream *st, AVPacket *pkt)
  
  {
  
      /* rescale output packet timestamp values from codec to stream timebase */
  
      av_packet_rescale_ts(pkt, *time_base, st->time_base);
  
      pkt->stream_index = st->index;
  
  
  
      /* Write the compressed frame to the media file. */
  
      log_packet(fmt_ctx, pkt);
  
      return av_interleaved_write_frame(fmt_ctx, pkt);
  
  }
  
  
  
  /* Add an output stream. */
  
  static void add_stream(OutputStream *ost, AVFormatContext *oc,
  
                         AVCodec **codec,
  
                         enum AVCodecID codec_id)
  
  {
  
      AVCodecContext *c;
  
      int i;
  
  
  
      /* find the encoder */
  
      *codec = avcodec_find_encoder(codec_id);
  
      if (!(*codec)) {
  
          fprintf(stderr, "Could not find encoder for '%s'\n",
  
                  avcodec_get_name(codec_id));
  
          exit(1);
  
      }
  
  
  
      ost->st = avformat_new_stream(oc, NULL);
  
      if (!ost->st) {
  
          fprintf(stderr, "Could not allocate stream\n");
  
          exit(1);
  
      }
  
      ost->st->id = oc->nb_streams-1;
  
      c = avcodec_alloc_context3(*codec);
  
      if (!c) {
  
          fprintf(stderr, "Could not alloc an encoding context\n");
  
          exit(1);
  
      }
  
      ost->enc = c;
  
  
  
      switch ((*codec)->type) {
  
      case AVMEDIA_TYPE_AUDIO:
  
          c->sample_fmt  = (*codec)->sample_fmts ?
  
              (*codec)->sample_fmts[0] : AV_SAMPLE_FMT_FLTP;
  
          c->bit_rate    = 64000;
  
          c->sample_rate = 44100;
  
          if ((*codec)->supported_samplerates) {
  
              c->sample_rate = (*codec)->supported_samplerates[0];
  
              for (i = 0; (*codec)->supported_samplerates[i]; i++) {
  
                  if ((*codec)->supported_samplerates[i] == 44100)
  
                      c->sample_rate = 44100;
  
              }
  
          }
  
          c->channels        = av_get_channel_layout_nb_channels(c->channel_layout);
  
          c->channel_layout = AV_CH_LAYOUT_STEREO;
  
          if ((*codec)->channel_layouts) {
  
              c->channel_layout = (*codec)->channel_layouts[0];
  
              for (i = 0; (*codec)->channel_layouts[i]; i++) {
  
                  if ((*codec)->channel_layouts[i] == AV_CH_LAYOUT_STEREO)
  
                      c->channel_layout = AV_CH_LAYOUT_STEREO;
  
              }
  
          }
  
          c->channels        = av_get_channel_layout_nb_channels(c->channel_layout);
  
          ost->st->time_base = (AVRational){ 1, c->sample_rate };
  
          break;
  
  
  
      case AVMEDIA_TYPE_VIDEO:
  
          c->codec_id = codec_id;
  
  
  
          c->bit_rate = 400000;
  
          /* Resolution must be a multiple of two. */
  
          c->width    = 352;
  
          c->height   = 288;
  
          /* timebase: This is the fundamental unit of time (in seconds) in terms
  
           * of which frame timestamps are represented. For fixed-fps content,
  
           * timebase should be 1/framerate and timestamp increments should be
  
           * identical to 1. */
  
          ost->st->time_base = (AVRational){ 1, STREAM_FRAME_RATE };
  
          c->time_base       = ost->st->time_base;
  
  
  
          c->gop_size      = 12; /* emit one intra frame every twelve frames at most */
  
          c->pix_fmt       = STREAM_PIX_FMT;
  
          if (c->codec_id == AV_CODEC_ID_MPEG2VIDEO) {
  
              /* just for testing, we also add B-frames */
  
              c->max_b_frames = 2;
  
          }
  
          if (c->codec_id == AV_CODEC_ID_MPEG1VIDEO) {
  
              /* Needed to avoid using macroblocks in which some coeffs overflow.
  
               * This does not happen with normal video, it just happens here as
  
               * the motion of the chroma plane does not match the luma plane. */
  
              c->mb_decision = 2;
  
          }
  
     break;
  
  
  
      default:
  
          break;
  
      }
  
  
  
      /* Some formats want stream headers to be separate. */
  
      if (oc->oformat->flags &amp; AVFMT_GLOBALHEADER)
  
          c->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
  
  }
  
  
  
  /**************************************************************/
  
  /* audio output */
  
  
  
  static AVFrame *alloc_audio_frame(enum AVSampleFormat sample_fmt,
  
                                    uint64_t channel_layout,
  
                                    int sample_rate, int nb_samples)
  
  {
  
      AVFrame *frame = av_frame_alloc();
  
      int ret;
  
  
  
      if (!frame) {
  
          fprintf(stderr, "Error allocating an audio frame\n");
  
          exit(1);
  
      }
  
  
  
      frame->format = sample_fmt;
  
      frame->channel_layout = channel_layout;
  
      frame->sample_rate = sample_rate;
  
      frame->nb_samples = nb_samples;
  
  
  
      if (nb_samples) {
  
          ret = av_frame_get_buffer(frame, 0);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Error allocating an audio buffer\n");
  
              exit(1);
  
          }
  
      }
  
  
  
      return frame;
  
  }
  
  
  
  static void open_audio(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
  
  {
  
      AVCodecContext *c;
  
      int nb_samples;
  
      int ret;
  
     AVDictionary *opt = NULL;
  
  
  
      c = ost->enc;
  
  
  
      /* open it */
  
      av_dict_copy(&amp;opt, opt_arg, 0);
  
      ret = avcodec_open2(c, codec, &amp;opt);
  
      av_dict_free(&amp;opt);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not open audio codec: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      /* init signal generator */
  
      ost->t     = 0;
  
      ost->tincr = 2 * M_PI * 110.0 / c->sample_rate;
  
      /* increment frequency by 110 Hz per second */
  
      ost->tincr2 = 2 * M_PI * 110.0 / c->sample_rate / c->sample_rate;
  
  
  
      if (c->codec->capabilities &amp; AV_CODEC_CAP_VARIABLE_FRAME_SIZE)
  
          nb_samples = 10000;
  
      else
  
          nb_samples = c->frame_size;
  
  
  
      ost->frame     = alloc_audio_frame(c->sample_fmt, c->channel_layout,
  
                                         c->sample_rate, nb_samples);
  
      ost->tmp_frame = alloc_audio_frame(AV_SAMPLE_FMT_S16, c->channel_layout,
  
                                         c->sample_rate, nb_samples);
  
  
  
      /* copy the stream parameters to the muxer */
  
      ret = avcodec_parameters_from_context(ost->st->codecpar, c);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not copy the stream parameters\n");
  
          exit(1);
  
      }
  
  
  
      /* create resampler context */
  
          ost->swr_ctx = swr_alloc();
  
          if (!ost->swr_ctx) {
  
              fprintf(stderr, "Could not allocate resampler context\n");
  
              exit(1);
  
          }
  
  
  
          /* set options */
  
          av_opt_set_int       (ost->swr_ctx, "in_channel_count",   c->channels,       0);
  
          av_opt_set_int       (ost->swr_ctx, "in_sample_rate",     c->sample_rate,    0);
  
          av_opt_set_sample_fmt(ost->swr_ctx, "in_sample_fmt",      AV_SAMPLE_FMT_S16, 0);
  
          av_opt_set_int       (ost->swr_ctx, "out_channel_count",  c->channels,       0);
  
          av_opt_set_int       (ost->swr_ctx, "out_sample_rate",    c->sample_rate,    0);
  
          av_opt_set_sample_fmt(ost->swr_ctx, "out_sample_fmt",     c->sample_fmt,     0);
  
  
  
          /* initialize the resampling context */
  
          if ((ret = swr_init(ost->swr_ctx)) &lt; 0) {
  
              fprintf(stderr, "Failed to initialize the resampling context\n");
  
              exit(1);
  
          }
  
  }
  
  
  
  /* Prepare a 16 bit dummy audio frame of 'frame_size' samples and
  
   * 'nb_channels' channels. */
  
  static AVFrame *get_audio_frame(OutputStream *ost)
  
  {
  
      AVFrame *frame = ost->tmp_frame;
  
      int j, i, v;
  
      int16_t *q = (int16_t*)frame->data[0];
  
  
  
      /* check if we want to generate more frames */
  
      if (av_compare_ts(ost->next_pts, ost->enc->time_base,
  
                        STREAM_DURATION, (AVRational){ 1, 1 }) >= 0)
  
          return NULL;
  
  
  
      for (j = 0; j nb_samples; j++) {
  
          v = (int)(sin(ost->t) * 10000);
  
          for (i = 0; i &lt; ost->enc->channels; i++)
  
              *q++ = v;
  
          ost->t     += ost->tincr;
  
          ost->tincr += ost->tincr2;
  
      }
  
  
  
      frame->pts = ost->next_pts;
  
      ost->next_pts  += frame->nb_samples;
  
  
  
      return frame;
  
  }
  
  
  
  /*
  
   * encode one audio frame and send it to the muxer
  
   * return 1 when encoding is finished, 0 otherwise
  
   */
  
  static int write_audio_frame(AVFormatContext *oc, OutputStream *ost)
  
  {
  
      AVCodecContext *c;
  
      AVPacket pkt = { 0 }; // data and size must be 0;
  
      AVFrame *frame;
  
      int ret;
  
      int got_packet;
  
      int dst_nb_samples;
  
  
  
      av_init_packet(&amp;pkt);
  
      c = ost->enc;
  
  
  
      frame = get_audio_frame(ost);
  
  
  
      if (frame) {
  
          /* convert samples from native format to destination codec format, using the resampler */
  
              /* compute destination number of samples */
  
              dst_nb_samples = av_rescale_rnd(swr_get_delay(ost->swr_ctx, c->sample_rate) + frame->nb_samples,
  
                                              c->sample_rate, c->sample_rate, AV_ROUND_UP);
  
              av_assert0(dst_nb_samples == frame->nb_samples);
  
  
  
          /* when we pass a frame to the encoder, it may keep a reference to it
  
           * internally;
  
          * make sure we do not overwrite it here
  
           */
  
          ret = av_frame_make_writable(ost->frame);
  
          if (ret &lt; 0)
  
              exit(1);
  
  
  
          /* convert to destination format */
  
          ret = swr_convert(ost->swr_ctx,
  
                            ost->frame->data, dst_nb_samples,
  
                            (const uint8_t **)frame->data, frame->nb_samples);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Error while converting\n");
  
              exit(1);
  
          }
  
          frame = ost->frame;
  
  
  
          frame->pts = av_rescale_q(ost->samples_count, (AVRational){1, c->sample_rate}, c->time_base);
  
          ost->samples_count += dst_nb_samples;
  
      }
  
  
  
      ret = avcodec_encode_audio2(c, &amp;pkt, frame, &amp;got_packet);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Error encoding audio frame: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      if (got_packet) {
  
          ret = write_frame(oc, &amp;c->time_base, ost->st, &amp;pkt);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Error while writing audio frame: %s\n",
  
                      av_err2str(ret));
  
              exit(1);
  
          }
  
      }
  
  
  
      return (frame || got_packet) ? 0 : 1;
  
  }
  
  
  
  /**************************************************************/
  
  /* video output */
  
  
  
  static AVFrame *alloc_picture(enum AVPixelFormat pix_fmt, int width, int height)
  
  {
  
      AVFrame *picture;
  
      int ret;
  
  
  
      picture = av_frame_alloc();
  
      if (!picture)
  
          return NULL;
  
  
  
      picture->format = pix_fmt;
  
      picture->width  = width;
  
      picture->height = height;
  
  
  
      /* allocate the buffers for the frame data */
  
      ret = av_frame_get_buffer(picture, 32);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not allocate frame data.\n");
  
          exit(1);
  
      }
  
  
  
      return picture;
  
  }
  
  
  
  static void open_video(AVFormatContext *oc, AVCodec *codec, OutputStream *ost, AVDictionary *opt_arg)
  
  {
  
      int ret;
  
      AVCodecContext *c = ost->enc;
  
      AVDictionary *opt = NULL;
  
  
  
      av_dict_copy(&amp;opt, opt_arg, 0);
  
  
  
      /* open the codec */
  
      ret = avcodec_open2(c, codec, &amp;opt);
  
      av_dict_free(&amp;opt);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not open video codec: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      /* allocate and init a re-usable frame */
  
      ost->frame = alloc_picture(c->pix_fmt, c->width, c->height);
  
      if (!ost->frame) {
  
          fprintf(stderr, "Could not allocate video frame\n");
  
          exit(1);
  
      }
  
  
  
      /* If the output format is not YUV420P, then a temporary YUV420P
  
       * picture is needed too. It is then converted to the required
  
       * output format. */
  
      ost->tmp_frame = NULL;
  
      if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
  
          ost->tmp_frame = alloc_picture(AV_PIX_FMT_YUV420P, c->width, c->height);
  
          if (!ost->tmp_frame) {
  
              fprintf(stderr, "Could not allocate temporary picture\n");
  
              exit(1);
  
          }
  
      }
  
  
  
      /* copy the stream parameters to the muxer */
  
      ret = avcodec_parameters_from_context(ost->st->codecpar, c);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Could not copy the stream parameters\n");
  
          exit(1);
  
      }
  
  }
  
  
  
  /* Prepare a dummy image. */
  
  static void fill_yuv_image(AVFrame *pict, int frame_index,
  
                             int width, int height)
  
  {
  
      int x, y, i;
  
  
  
      i = frame_index;
  
  
  
      /* Y */
  
      for (y = 0; y &lt; height; y++)
  
          for (x = 0; x &lt; width; x++)
  
              pict->data[0][y * pict->linesize[0] + x] = x + y + i * 3;
  
  
  
      /* Cb and Cr */
  
      for (y = 0; y &lt; height / 2; y++) {
  
          for (x = 0; x &lt; width / 2; x++) {
  
              pict->data[1][y * pict->linesize[1] + x] = 128 + y + i * 2;
  
              pict->data[2][y * pict->linesize[2] + x] = 64 + x + i * 5;
  
          }
  
      }
  
  }
  
  
  
  static AVFrame *get_video_frame(OutputStream *ost)
  
  {
  
      AVCodecContext *c = ost->enc;
  
  
  
      /* check if we want to generate more frames */
  
      if (av_compare_ts(ost->next_pts, c->time_base,
  
                        STREAM_DURATION, (AVRational){ 1, 1 }) >= 0)
  
          return NULL;
  
  
  
      /* when we pass a frame to the encoder, it may keep a reference to it
  
       * internally; make sure we do not overwrite it here */
  
      if (av_frame_make_writable(ost->frame) &lt; 0)
  
          exit(1);
  
  
  
      if (c->pix_fmt != AV_PIX_FMT_YUV420P) {
  
          /* as we only generate a YUV420P picture, we must convert it
  
           * to the codec pixel format if needed */
  
          if (!ost->sws_ctx) {
  
              ost->sws_ctx = sws_getContext(c->width, c->height,
  
                                            AV_PIX_FMT_YUV420P,
  
                                            c->width, c->height,
  
                                            c->pix_fmt,
  
                                            SCALE_FLAGS, NULL, NULL, NULL);
  
              if (!ost->sws_ctx) {
  
                  fprintf(stderr,
  
                          "Could not initialize the conversion context\n");
  
                  exit(1);
  
              }
  
          }
  
          fill_yuv_image(ost->tmp_frame, ost->next_pts, c->width, c->height);
  
          sws_scale(ost->sws_ctx,
  
                    (const uint8_t * const *)ost->tmp_frame->data, ost->tmp_frame->linesize,
  
                    0, c->height, ost->frame->data, ost->frame->linesize);
  
      } else {
  
          fill_yuv_image(ost->frame, ost->next_pts, c->width, c->height);
  
      }
  
  
  
      ost->frame->pts = ost->next_pts++;
  
  
  
      return ost->frame;
  
  }
  
  
  
  /*
  
   * encode one video frame and send it to the muxer
  
   * return 1 when encoding is finished, 0 otherwise
  
   */
  
  static int write_video_frame(AVFormatContext *oc, OutputStream *ost)
  
  {
  
      int ret;
  
      AVCodecContext *c;
  
      AVFrame *frame;
  
      int got_packet = 0;
  
      AVPacket pkt = { 0 };
  
  
  
      c = ost->enc;
  
  
  
      frame = get_video_frame(ost);
  
  
  
      av_init_packet(&amp;pkt);
  
  
  
      /* encode the image */
  
      ret = avcodec_encode_video2(c, &amp;pkt, frame, &amp;got_packet);
  
      if (ret &lt; 0) {
  
        fprintf(stderr, "Error encoding video frame: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      if (got_packet) {
  
          ret = write_frame(oc, &amp;c->time_base, ost->st, &amp;pkt);
  
      } else {
  
          ret = 0;
  
      }
  
  
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Error while writing video frame: %s\n", av_err2str(ret));
  
          exit(1);
  
      }
  
  
  
      return (frame || got_packet) ? 0 : 1;
  
  }
  
  
  
  static void close_stream(AVFormatContext *oc, OutputStream *ost)
  
  {
  
      avcodec_free_context(&amp;ost->enc);
  
      av_frame_free(&amp;ost->frame);
  
      av_frame_free(&amp;ost->tmp_frame);
  
      sws_freeContext(ost->sws_ctx);
  
      swr_free(&amp;ost->swr_ctx);
  
  }
  
  
  
  /**************************************************************/
  
  /* media file output */
  
  
  
  int main(int argc, char **argv)
  
  {
  
      OutputStream video_st = { 0 }, audio_st = { 0 };
  
      const char *filename;
  
      AVOutputFormat *fmt;
  
      AVFormatContext *oc;
  
      AVCodec *audio_codec, *video_codec;
  
      int ret;
  
      int have_video = 0, have_audio = 0;
  
      int encode_video = 0, encode_audio = 0;
  
      AVDictionary *opt = NULL;
  
      int i;
  
  
  
      /* Initialize libavcodec, and register all codecs and formats. */
  
      av_register_all();
  
      avformat_network_init();
  
      if (argc &lt; 2) {
  
          printf("usage: %s output_file\n"
  
                 "API example program to output a media file with libavformat.\n"
  
                 "This program generates a synthetic audio and video stream, encodes and\n"
  
                 "muxes them into a file named output_file.\n"
  
                 "The output format is automatically guessed according to the file extension.\n"
  
                 "Raw images can also be output by using '%%d' in the filename.\n"
  
                 "\n", argv[0]);
  
          return 1;
  
      }
  
  
  
      filename = argv[1];
  
      for (i = 2; i+1 &lt; argc; i+=2) {
  
          if (!strcmp(argv[i], "-flags") || !strcmp(argv[i], "-fflags"))
  
              av_dict_set(&amp;opt, argv[i]+1, argv[i+1], 0);
  
      }
  
     /* allocate the output media context */
  
      avformat_alloc_output_context2(&amp;oc, NULL, "rtsp", filename);
  
      if (!oc) {
  
          printf("Could not deduce output format from file extension: using MPEG.\n");
  
          avformat_alloc_output_context2(&amp;oc, NULL, "mpeg", filename);
  
      }
  
      if (!oc)
  
          return 1;
  
  
  
      fmt = oc->oformat;
  
  
  
      /* Add the audio and video streams using the default format codecs
  
       * and initialize the codecs. */
  
      if (fmt->video_codec != AV_CODEC_ID_NONE) {
  
          add_stream(&amp;video_st, oc, &amp;video_codec, fmt->video_codec);
  
          have_video = 1;
  
          encode_video = 1;
  
      }
  
      if (fmt->audio_codec != AV_CODEC_ID_NONE) {
  
          add_stream(&amp;audio_st, oc, &amp;audio_codec, fmt->audio_codec);
  
          have_audio = 1;
  
          encode_audio = 1;
  
      }
  
  
  
      /* Now that all the parameters are set, we can open the audio and
  
       * video codecs and allocate the necessary encode buffers. */
  
      if (have_video)
  
          open_video(oc, video_codec, &amp;video_st, opt);
  
  
  
      if (have_audio)
  
          open_audio(oc, audio_codec, &amp;audio_st, opt);
  
  
  
      av_dump_format(oc, 0, filename, 1);
  
  
  
      /* open the output file, if needed */
  
      if (!(fmt->flags &amp; AVFMT_NOFILE)) {
  
          ret = avio_open(&amp;oc->pb, filename, AVIO_FLAG_WRITE);
  
          if (ret &lt; 0) {
  
              fprintf(stderr, "Could not open '%s': %s\n", filename,
  
                      av_err2str(ret));
  
              return 1;
  
          }
  
      }
  
  
  
      /* Write the stream header, if any. */
  
      ret = avformat_write_header(oc, &amp;opt);
  
      if (ret &lt; 0) {
  
          fprintf(stderr, "Error occurred when opening output file: %s\n",
  
                  av_err2str(ret));
  
          return 1;
  
      }
  
  
  
      while (encode_video || encode_audio) {
  
          /* select the stream to encode */
  
          if (encode_video &amp;&amp;
  
             (!encode_audio || av_compare_ts(video_st.next_pts, video_st.enc->time_base,
  
                                              audio_st.next_pts, audio_st.enc->time_base) &lt;= 0)) {
  
              encode_video = !write_video_frame(oc, &amp;video_st);
  
          } else {
  
              encode_audio = !write_audio_frame(oc, &amp;audio_st);
  
          }
  
      }
  
  
  
      /* Write the trailer, if any. The trailer must be written before you
  
       * close the CodecContexts open when you wrote the header; otherwise
  
       * av_write_trailer() may try to use memory that was freed on
  
       * av_codec_close(). */
  
      av_write_trailer(oc);
  
  
  
      /* Close each codec. */
  
      if (have_video)
  
          close_stream(oc, &amp;video_st);
  
      if (have_audio)
  
          close_stream(oc, &amp;audio_st);
  
  
  
      if (!(fmt->flags &amp; AVFMT_NOFILE))
  
          /* Close the output file. */
  
          avio_closep(&amp;oc->pb);
  
  
  
      /* free the stream */
  
      avformat_free_context(oc);
  
  
  
      return 0;
  
  }

  After compiling it, I am running binary :

  $ ./muxing rtsp://127.0.0.1/test
  
  Output #0, rtsp, to 'rtsp://127.0.0.1/test':
  
      Stream #0:0: Video: mpeg4, yuv420p, 352x288, q=2-31, 400 kb/s, 25 tbn
  
      Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp, 64 kb/s
  
  [tcp @ 0x2b9d220] Connection to tcp://127.0.0.1:554?timeout=0 failed: Connection refused
  
  Error occurred when opening output file: Connection refused

  But getting Connection refused error,

• OpenCV 3.4.3 - GStream not called when VideoWriting

  14 décembre 2018, par taubhi

  I’d like to succeed writing a file with OpenCV encoding with GStreamer.

  Thus I’m using the code :

  VideoWriter.open("appsrc ! autovideoconvert ! omxh264enc ! matroskamux ! filesink location=test2.mkv sync=false",
  
  //cv::CAP_GSTREAMER, //set or not, I get the same problem
  
  0, (double)25, //fourcc, fps
  
  cv::Size(1024, 1024), 
  
  true);

  And my problem :

  A video file called appsrc ! autovideoconvert ! omxh264enc ! matroskamux ! filesink location=test2.mkv sync=false is created (rather than test2.mkv), and it is empty. The same problem is corrected and seems to work in the following thread, without any effect for me : Opening a GStreamer pipeline from OpenCV with VideoWriter

  I tried, without success :

  • Add quotes to the file name test2 :
    "appsrc ! autovideoconvert ! omxh264enc ! matroskamux ! filesink location=\"test2.mkv\" sync=false" neither
  • Encode with different format : .flv rather than .mkv
  • Encode with different codec : omxh265enc rather than omxh264en

  I checked :

  • That GStreamer is well compiled with OpenCV ; the CMake of the OpenCV project outputs :

    Video I/O:Video for Windows : YES
    DC1394 : NO
    FFMPEG : YES (prebuilt binaries)
    avcodec : YES (ver 57.107.100)
    avformat : YES (ver 57.83.100)
    avutil : YES (ver 55.78.100)
    swscale : YES (ver 4.8.100)
    avresample : YES (ver 3.7.0)
    GStreamer :

    base : YES (ver 1.0)

    video : YES (ver 1.0)

    app : YES (ver 1.0)

    riff : YES (ver 1.0)

    pbutils : YES (ver 1.0)

    DirectShow : YES

    Media Foundation : YES

  • That FFMpeg is installed on my computer ; for this I used VLC :

    File -> Stream -> Add a file then stream
    New destination : RTP/MPEG Transport Stream
    Profile : Video - H.264 + MP3 (MP4)
    Then no error happens ; Note : an error occured with Video - H.265 + MP3
    (MP4), but I’m using omxh264enc in my code, not 265

  I’m struggled :/ Do you have any idea ?
  Thanks a lot :)

  Have a good day,

  Thibaut

  PS - Complete OpenCV CMake output :

  Selecting Windows SDK version 10.0.17134.0 to target Windows 10.0.17763.
  
  AVX_512F is not supported by C++ compiler
  
  
  
  AVX512_SKX is not supported by C++ compiler
  
  
  
  Dispatch optimization AVX512_SKX is not available, skipped
  
  
  
  libjpeg-turbo: VERSION = 1.5.3, BUILD = opencv-3.4.3-libjpeg-turbo
  
  
  
  Looking for Mfapi.h
  
  
  
  Looking for Mfapi.h - found
  
  
  
  found Intel IPP (ICV version): 2017.0.3 [2017.0.3]
  
  
  
  at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippicv_win
  
  
  
  found Intel IPP IW sources: 2017.0.3
  
  
  
  at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippiw_win
  
  
  
  CUDA detected: 9.2
  
  
  
  CUDA NVCC target flags: -gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-D_FORCE_INLINES
  
  
  
  Found apache ant: C:/NVPACK/apache-ant-1.8.2/bin/ant.bat (1.8.2)
  
  
  
  Could NOT find Matlab (missing: MATLAB_MEX_SCRIPT MATLAB_INCLUDE_DIRS MATLAB_ROOT_DIR MATLAB_LIBRARIES MATLAB_LIBRARY_DIRS MATLAB_MEXEXT MATLAB_ARCH MATLAB_BIN)
  
  
  
  VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
  
  
  
  Caffe: NO
  
  
  
  Protobuf: NO
  
  
  
  Glog: YES
  
  
  
  freetype2: NO
  
  
  
  harfbuzz: NO
  
  
  
  Module opencv_ovis disabled because OGRE3D was not found
  
  
  
  No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available.
  
  
  
  Found installed version of gflags: E:/Git/gflags/BUILD-VS2015
  
  
  
  Detected gflags version: 2.2.1
  
  
  
  Checking SFM deps... FALSE
  
  
  
  Module opencv_sfm disabled because the following dependencies are not found: Eigen Glog/Gflags
  
  
  
  Excluding from source files list: <build>/modules/dnn/layers/layers_common.avx512_skx.cpp
  
  
  
  Tesseract: NO
  
  
  
  
  
  General configuration for OpenCV 3.4.3 =====================================
  
  
  
  Version control: unknown
  
  
  
  
  
  Extra modules:
  
  
  
  Location (extra): E:/OpenCV/3.4.3/opencv_contrib-3.4.3/modules
  
  
  
  Version control (extra): unknown
  
  
  
  
  
  Platform:
  
  
  
  Timestamp: 2018-11-07T16:57:18Z
  
  
  
  Host: Windows 10.0.17763 AMD64
  
  
  
  CMake: 3.12.0-rc2
  
  
  
  CMake generator: Visual Studio 14 2015 Win64
  
  
  
  CMake build tool: C:/Program Files (x86)/MSBuild/14.0/bin/MSBuild.exe
  
  
  
  MSVC: 1900
  
  
  
  
  
  CPU/HW features:
  
  
  
  Baseline: SSE SSE2 SSE3
  
  
  
  requested: SSE3
  
  
  
  Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2
  
  
  
  requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
  
  
  
  SSE4_1 (5 files): + SSSE3 SSE4_1
  
  
  
  SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
  
  
  
  FP16 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
  
  
  
  AVX (6 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
  
  
  
  AVX2 (11 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
  
  
  
  
  
  C/C++:
  
  
  
  Built as dynamic libs?: YES
  
  
  
  C++11: YES
  
  
  
  C++ Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe (ver 19.0.24215.1)
  
  
  
  C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP16 /MD /O2 /Ob2 /DNDEBUG
  
  
  
  C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP16 /MDd /Zi /Ob0 /Od /RTC1
  
  
  
  C Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
  
  
  
  C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /MP16 /MD /O2 /Ob2 /DNDEBUG
  
  
  
  C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /MP16 /MDd /Zi /Ob0 /Od /RTC1
  
  
  
  Linker flags (Release): /machine:x64 /INCREMENTAL:NO
  
  
  
  Linker flags (Debug): /machine:x64 /debug /INCREMENTAL
  
  
  
  ccache: NO
  
  
  
  Precompiled headers: YES
  
  
  
  Extra dependencies: cudart.lib nppc.lib nppial.lib nppicc.lib nppicom.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/lib/x64
  
  
  
  3rdparty dependencies:
  
  
  
  
  
  OpenCV modules:
  
  
  
  To be built: aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dpm face features2d flann fuzzy hfs highgui img_hash imgcodecs imgproc java java_bindings_generator line_descriptor ml objdetect optflow phase_unwrapping photo plot python_bindings_generator reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
  
  
  
  Disabled: js world
  
  
  
  Disabled by dependency: -
  
  
  
  Unavailable: cnn_3dobj cvv freetype hdf matlab ovis python2 python3 sfm viz
  
  
  
  Applications: tests perf_tests apps
  
  
  
  Documentation: NO
  
  
  
  Non-free algorithms: NO
  
  
  
  
  
  Windows RT support: NO
  
  
  
  
  
  GUI:
  
  
  
  Win32 UI: YES
  
  
  
  VTK support: NO
  
  
  
  
  
  Media I/O:
  
  
  
  ZLib: build (ver 1.2.11)
  
  
  
  JPEG: build-libjpeg-turbo (ver 1.5.3-62)
  
  
  
  WEBP: build (ver encoder: 0x020e)
  
  
  
  PNG: build (ver 1.6.34)
  
  
  
  TIFF: build (ver 42 - 4.0.9)
  
  
  
  JPEG 2000: build (ver 1.900.1)
  
  
  
  OpenEXR: build (ver 1.7.1)
  
  
  
  HDR: YES
  
  
  
  SUNRASTER: YES
  
  
  
  PXM: YES
  
  
  
  
  
  Video I/O:
  
  
  
  Video for Windows: YES
  
  
  
  DC1394: NO
  
  
  
  FFMPEG: YES (prebuilt binaries)
  
  
  
  avcodec: YES (ver 57.107.100)
  
  
  
  avformat: YES (ver 57.83.100)
  
  
  
  avutil: YES (ver 55.78.100)
  
  
  
  swscale: YES (ver 4.8.100)
  
  
  
  avresample: YES (ver 3.7.0)
  
  
  
  GStreamer:
  
  
  
  base: YES (ver 1.0)
  
  
  
  video: YES (ver 1.0)
  
  
  
  app: YES (ver 1.0)
  
  
  
  riff: YES (ver 1.0)
  
  
  
  pbutils: YES (ver 1.0)
  
  
  
  DirectShow: YES
  
  
  
  Media Foundation: YES
  
  
  
  
  
  Parallel framework: Concurrency
  
  
  
  
  
  Trace: YES (with Intel ITT)
  
  
  
  
  
  Other third-party libraries:
  
  
  
  Intel IPP: 2017.0.3 [2017.0.3]
  
  
  
  at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippicv_win
  
  
  
  Intel IPP IW: sources (2017.0.3)
  
  
  
  at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippiw_win
  
  
  
  Custom HAL: NO
  
  
  
  Protobuf: build (3.5.1)
  
  
  
  
  
  NVIDIA CUDA: YES (ver 9.2, CUFFT CUBLAS NVCUVID)
  
  
  
  NVIDIA GPU arch: 30 35 37 50 52 60 61 70
  
  
  
  NVIDIA PTX archs:
  
  
  
  
  
  OpenCL: YES (no extra features)
  
  
  
  Include path: E:/OpenCV/3.4.3/opencv-3.4.3/3rdparty/include/opencl/1.2
  
  
  
  Link libraries: Dynamic load
  
  
  
  
  
  Python (for build): C:/ProgramData/Miniconda3/python.exe
  
  
  
  
  
  Java:
  
  
  
  ant: C:/NVPACK/apache-ant-1.8.2/bin/ant.bat (ver 1.8.2)
  
  
  
  JNI: C:/NVPACK/jdk1.8.0_77/include C:/NVPACK/jdk1.8.0_77/include/win32 C:/NVPACK/jdk1.8.0_77/include
  
  
  
  Java wrappers: YES
  
  
  
  Java tests: YES
  
  
  
  
  
  Matlab: NO
  
  
  
  
  
  Install to: E:/OpenCV/3.4.3/build/install
  
  
  
  -----------------------------------------------------------------
  
  
  
  
  
  Configuring done
  
  </build>