Recherche avancée

Sur d’autres sites (6353)

• lavfi/delogo : use weighted interpolation

  26 juin 2013, par Jean Delvare

  lavfi/delogo : use weighted interpolation
  The original delogo algorithm interpolates both horizontally and
  
  vertically and uses the average to compute the resulting sample. This
  
  works reasonably well when the logo area is almost square. However
  
  when the logo area is significantly larger than high or higher than
  
  large, the result is largely suboptimal.
  The issue can be clearly seen by testing the delogo filter with a fake
  
  logo area that is 200 pixels large and 2 pixels high. Vertical
  
  interpolation gives a very good result in that case, horizontal
  
  interpolation gives a very bad result, and the overall result is poor,
  
  because both are given the same weight.
  Even when the logo is roughly square, the current algorithm gives poor
  
  results on the borders of the logo area, because it always gives
  
  horizontal and vertical interpolations an equal weight, and this is
  
  suboptimal on borders. For example, in the middle of the left hand
  
  side border of the logo, you want to trust the left known point much
  
  more than the right known point (which the current algorithm already
  
  does) but also much more than the top and bottom known points (which
  
  the current algorithm doesn’t do.)
  By properly weighting each known point when computing the value of
  
  each interpolated pixel, the visual result is much better, especially
  
  on borders and/or for high or large logo areas.
  The algorithm I implemented guarantees that the weight of each of the
  
  4 known points directly depends on its distance to the interpolated
  
  point. It is largely inspired from the original algorithm, the key
  
  difference being that it computes the relative weights globally
  
  instead of separating the vertical and horizontal interpolations and
  
  combining them afterward.
  Signed-off-by : Jean Delvare <khali@linux-fr.org>
  
  Signed-off-by : Stefano Sabatini <stefasab@gmail.com>
  

  • [DH] libavfilter/vf_delogo.c
  • [DH] tests/ref/fate/filter-delogo

• Revision ed2dc59c1b : Integral projection based motion estimation This commit introduces a new block

  13 février 2015, par Jingning Han

  Changed Paths :
   Modify /vp9/common/vp9_rtcd_defs.pl

  
   Modify /vp9/encoder/vp9_avg.c

  
   Modify /vp9/encoder/vp9_encodeframe.c

  
   Modify /vp9/encoder/x86/vp9_avg_intrin_sse2.c

  
  
  Integral projection based motion estimation

  This commit introduces a new block match motion estimation
  using integral projection measurement. The 2-D block and the nearby
  region is projected onto the horizontal and vertical 1-D vectors,
  respectively. It then runs vector match, instead of block match,
  over the two separate 1-D vectors to locate the motion compensated
  reference block.

  This process is run per 64x64 block to align the reference before
  choosing partitioning in speed 6. The overall CPU cycle cost due
  to this additional 64x64 block match (SSE2 version) takes around 2%
  at low bit-rate rtc speed 6. When strong motion activities exist in
  the video sequence, it substantially improves the partition
  selection accuracy, thereby achieving better compression performance
  and lower CPU cycles.

  The experiments were tested in RTC speed -6 setting :
  cloud 1080p 500 kbps
  17006 b/f, 37.086 dB, 5386 ms ->
  16669 b/f, 37.970 dB, 5085 ms (>0.9dB gain and 6% faster)

  pedestrian_area 1080p 500 kbps
  53537 b/f, 36.771 dB, 18706 ms ->
  51897 b/f, 36.792 dB, 18585 ms (4% bit-rate savings)

  blue_sky 1080p 500 kbps
  70214 b/f, 33.600 dB, 13979 ms ->
  53885 b/f, 33.645 dB, 10878 ms (30% bit-rate savings, 25% faster)

  jimred 400 kbps
  13380 b/f, 36.014 dB, 5723 ms ->
  13377 b/f, 36.087 dB, 5831 ms (2% bit-rate savings, 2% slower)

  Change-Id : Iffdb6ea5b16b77016bfa3dd3904d284168ae649c

• Video rotating to left by 90 degree when converted using ffmpeg

  15 juillet 2017, par Herdesh Verma

  I developed a below code :

  extern "C"
  
  {
  
  #include <libavutil></libavutil>imgutils.h>
  
  #include <libavutil></libavutil>opt.h>
  
  #include <libavcodec></libavcodec>avcodec.h>
  
  #include <libavutil></libavutil>mathematics.h>
  
  #include <libavutil></libavutil>samplefmt.h>
  
  #include <libavutil></libavutil>timestamp.h>
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libavfilter></libavfilter>avfiltergraph.h>
  
  #include <libswscale></libswscale>swscale.h>
  
  }
  
  #include 
  
  static AVFormatContext *fmt_ctx = NULL;
  
  
  
  static int frame_index = 0;
  
  
  
  static int j = 0, nbytes=0;
  
  uint8_t *video_outbuf = NULL;
  
  static AVPacket *pAVPacket=NULL;
  
  static int value=0;
  
  static AVFrame *pAVFrame=NULL;
  
  static AVFrame *outFrame=NULL;
  
  static AVStream *video_st=NULL;
  
  static AVFormatContext *outAVFormatContext=NULL;
  
  static AVCodec *outAVCodec=NULL;
  
  static AVOutputFormat *output_format=NULL;
  
  static AVCodecContext *video_dec_ctx = NULL, *audio_dec_ctx;
  
  static AVCodecContext *outAVCodecContext=NULL;
  
  static int width, height;
  
  static enum AVPixelFormat pix_fmt;
  
  static AVStream *video_stream = NULL, *audio_stream = NULL;
  
  static const char *src_filename = NULL;
  
  static const char *video_dst_filename = NULL;
  
  static const char *audio_dst_filename = NULL;
  
  static FILE *video_dst_file = NULL;
  
  static FILE *audio_dst_file = NULL;
  
  static uint8_t *video_dst_data[4] = {NULL};
  
  static int      video_dst_linesize[4];
  
  static int video_dst_bufsize;
  
  static int video_stream_idx = -1, audio_stream_idx = -1;
  
  static AVPacket *pkt=NULL;
  
  static AVPacket *pkt1=NULL;
  
  static AVFrame *frame = NULL;
  
  //static AVPacket pkt;
  
  static int video_frame_count = 0;
  
  static int audio_frame_count = 0;
  
  static int refcount = 0;
  
  AVCodec *codec;
  
  static struct SwsContext *sws_ctx;
  
  AVCodecContext *c= NULL;
  
   int i, out_size, size, x, y, outbuf_size;
  
   AVFrame *picture;
  
   uint8_t *outbuf, *picture_buf;
  
  int video_outbuf_size;
  
  int w, h;
  
  AVPixelFormat pixFmt;
  
  uint8_t *data[4];
  
  int linesize[4];
  
  
  
  static int open_codec_context(int *stream_idx,
  
                            AVCodecContext **dec_ctx, AVFormatContext 
  
  *fmt_ctx, enum AVMediaType type)
  
  {
  
  int ret, stream_index;
  
  AVStream *st;
  
  AVCodec *dec = NULL;
  
  AVDictionary *opts = NULL;
  
  ret = av_find_best_stream(fmt_ctx, type, -1, -1, NULL, 0);
  
  if (ret &lt; 0) {
  
      printf("Could not find %s stream in input file '%s'\n",
  
              av_get_media_type_string(type), src_filename);
  
      return ret;
  
  } else {
  
      stream_index = ret;
  
      st = fmt_ctx->streams[stream_index];
  
      /* find decoder for the stream */
  
      dec = avcodec_find_decoder(st->codecpar->codec_id);
  
      if (!dec) {
  
          printf("Failed to find %s codec\n",
  
                  av_get_media_type_string(type));
  
          return AVERROR(EINVAL);
  
      }
  
      /* Allocate a codec context for the decoder */
  
      *dec_ctx = avcodec_alloc_context3(dec);
  
      if (!*dec_ctx) {
  
          printf("Failed to allocate the %s codec context\n",
  
                  av_get_media_type_string(type));
  
          return AVERROR(ENOMEM);
  
      }
  
      /* Copy codec parameters from input stream to output codec context */
  
      if ((ret = avcodec_parameters_to_context(*dec_ctx, st->codecpar)) &lt; 0) {
  
          printf("Failed to copy %s codec parameters to decoder context\n",
  
                  av_get_media_type_string(type));
  
          return ret;
  
      }
  
      /* Init the decoders, with or without reference counting */
  
      av_dict_set(&amp;opts, "refcounted_frames", refcount ? "1" : "0", 0);
  
      if ((ret = avcodec_open2(*dec_ctx, dec, &amp;opts)) &lt; 0) {
  
          printf("Failed to open %s codec\n",
  
                  av_get_media_type_string(type));
  
          return ret;
  
      }
  
      *stream_idx = stream_index;
  
  }
  
  return 0;
  
  }
  
  
  
  
  
  
  
  int main (int argc, char **argv)
  
  {
  
   int ret = 0, got_frame;
  
  src_filename = argv[1];
  
  video_dst_filename = argv[2];
  
  audio_dst_filename = argv[3];
  
  av_register_all();
  
  avcodec_register_all();
  
  printf("Registered all\n");
  
  
  
  /* open input file, and allocate format context */
  
  if (avformat_open_input(&amp;fmt_ctx, src_filename, NULL, NULL) &lt; 0) {
  
      printf("Could not open source file %s\n", src_filename);
  
      exit(1);
  
  }
  
  
  
  /* retrieve stream information */
  
  if (avformat_find_stream_info(fmt_ctx, NULL) &lt; 0) {
  
      printf("Could not find stream information\n");
  
      exit(1);
  
  }
  
  
  
  if (open_codec_context(&amp;video_stream_idx, &amp;video_dec_ctx, fmt_ctx, 
  
  AVMEDIA_TYPE_VIDEO) >= 0) {
  
      video_stream = fmt_ctx->streams[video_stream_idx];
  
      avformat_alloc_output_context2(&amp;outAVFormatContext, NULL, NULL, 
  
  video_dst_filename);
  
      if (!outAVFormatContext)
  
      {
  
              printf("\n\nError : avformat_alloc_output_context2()");
  
              return -1;
  
      }
  
  }
  
  
  
  if (open_codec_context(&amp;audio_stream_idx, &amp;audio_dec_ctx, fmt_ctx, 
  
  AVMEDIA_TYPE_AUDIO) >= 0) {
  
      audio_stream = fmt_ctx->streams[audio_stream_idx];
  
      audio_dst_file = fopen(audio_dst_filename, "wb");
  
      if (!audio_dst_file) {
  
          printf("Could not open destination file %s\n", audio_dst_filename);
  
          ret = 1;
  
          goto end;
  
      }
  
  }
  
  /* dump input information to stderr */
  
  av_dump_format(fmt_ctx, 0, src_filename, 0);
  
  
  
  if (!audio_stream &amp;&amp; !video_stream) {
  
      printf("Could not find audio or video stream in the input, aborting\n");
  
      ret = 1;
  
      goto end;
  
  }
  
  
  
      output_format = av_guess_format(NULL, video_dst_filename, NULL);
  
      if( !output_format )
  
      {
  
       printf("\n\nError : av_guess_format()");
  
       return -1;
  
      }
  
  
  
      video_st = avformat_new_stream(outAVFormatContext ,NULL);
  
      if( !video_st )
  
      {
  
              printf("\n\nError : avformat_new_stream()");
  
        return -1;
  
      }
  
  
  
      outAVCodecContext = avcodec_alloc_context3(outAVCodec);
  
      if( !outAVCodecContext )
  
      {
  
        printf("\n\nError : avcodec_alloc_context3()");
  
        return -1;
  
      }
  
  
  
  
  
      outAVCodecContext = video_st->codec;
  
      outAVCodecContext->codec_id = AV_CODEC_ID_MPEG4;// AV_CODEC_ID_MPEG4; // 
  
   AV_CODEC_ID_H264 // AV_CODEC_ID_MPEG1VIDEO
  
      outAVCodecContext->codec_type = AVMEDIA_TYPE_VIDEO;
  
      outAVCodecContext->pix_fmt  = AV_PIX_FMT_YUV420P;
  
      outAVCodecContext->bit_rate = 400000; // 2500000
  
      outAVCodecContext->width = 1920;
  
      //outAVCodecContext->width = 500;
  
      outAVCodecContext->height = 1080;
  
      //outAVCodecContext->height = 500;
  
      outAVCodecContext->gop_size = 3;
  
      outAVCodecContext->max_b_frames = 2;
  
      outAVCodecContext->time_base.num = 1;
  
      outAVCodecContext->time_base.den = 30; // 15fps
  
  
  
      if (outAVCodecContext->codec_id == AV_CODEC_ID_H264)
  
      {
  
       av_opt_set(outAVCodecContext->priv_data, "preset", "slow", 0);
  
      }
  
  
  
      outAVCodec = avcodec_find_encoder(AV_CODEC_ID_MPEG4);
  
      if( !outAVCodec )
  
      {
  
       printf("\n\nError : avcodec_find_encoder()");
  
       return -1;
  
      }
  
  
  
      /* Some container formats (like MP4) require global headers to be 
  
   present
  
         Mark the encoder so that it behaves accordingly. */
  
  
  
      if ( outAVFormatContext->oformat->flags &amp; AVFMT_GLOBALHEADER)
  
      {
  
              outAVCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
  
      }
  
  
  
      value = avcodec_open2(outAVCodecContext, outAVCodec, NULL);
  
      if( value &lt; 0)
  
      {
  
              printf("\n\nError : avcodec_open2()");
  
              return -1;
  
      }
  
  
  
  /* create empty video file */
  
      if ( !(outAVFormatContext->flags &amp; AVFMT_NOFILE) )
  
      {
  
       if( avio_open2(&amp;outAVFormatContext->pb , video_dst_filename, 
  
  AVIO_FLAG_WRITE ,NULL, NULL) &lt; 0 )
  
       {
  
        printf("\n\nError : avio_open2()");
  
       }
  
      }
  
  
  
      if(!outAVFormatContext->nb_streams)
  
      {
  
              printf("\n\nError : Output file dose not contain any stream");
  
        return -1;
  
      }
  
  
  
      /* imp: mp4 container or some advanced container file required header 
  
   information*/
  
      value = avformat_write_header(outAVFormatContext , NULL);
  
      if(value &lt; 0)
  
      {
  
              printf("\n\nError : avformat_write_header()");
  
              return -1;
  
      }
  
  
  
      printf("\n\nOutput file information :\n\n");
  
      av_dump_format(outAVFormatContext , 0 ,video_dst_filename ,1);
  
  
  
  
  
      int flag;
  
      int frameFinished;
  
  
  
  
  
      value = 0;
  
  
  
      pAVPacket = (AVPacket *)av_malloc(sizeof(AVPacket));
  
      av_init_packet(pAVPacket);
  
  
  
      pAVFrame = av_frame_alloc();
  
      if( !pAVFrame )
  
      {
  
       printf("\n\nError : av_frame_alloc()");
  
       return -1;
  
      }
  
  
  
      outFrame = av_frame_alloc();//Allocate an AVFrame and set its fields to 
  
   default values.
  
      if( !outFrame )
  
      {
  
       printf("\n\nError : av_frame_alloc()");
  
       return -1;
  
      }
  
  
  
      nbytes = av_image_get_buffer_size(outAVCodecContext-
  
   >pix_fmt,outAVCodecContext->width,outAVCodecContext->height,32);
  
      video_outbuf = (uint8_t*)av_malloc(nbytes);
  
      if( video_outbuf == NULL )
  
      {
  
      printf("\n\nError : av_malloc()");
  
      }
  
  
  
  
  
      value = av_image_fill_arrays( outFrame->data, outFrame->linesize, 
  
   video_outbuf , AV_PIX_FMT_YUV420P, outAVCodecContext-
  
   >width,outAVCodecContext->height,1 ); // returns : the size in bytes 
  
   required for src
  
      if(value &lt; 0)
  
      {
  
      printf("\n\nError : av_image_fill_arrays()");
  
      }
  
  
  
      SwsContext* swsCtx_ ;
  
  
  
      // Allocate and return swsContext.
  
      // a pointer to an allocated context, or NULL in case of error
  
      // Deprecated : Use sws_getCachedContext() instead.
  
      swsCtx_ = sws_getContext(video_dec_ctx->width,
  
                              video_dec_ctx->height,
  
                              video_dec_ctx->pix_fmt,
  
                              video_dec_ctx->width,
  
                              video_dec_ctx->height,
  
                              video_dec_ctx->pix_fmt,
  
                              SWS_BICUBIC, NULL, NULL, NULL);
  
  
  
  
  
      AVPacket outPacket;
  
  
  
      int got_picture;
  
  
  
      while( av_read_frame( fmt_ctx , pAVPacket ) >= 0 )
  
      {
  
              if(pAVPacket->stream_index == video_stream_idx)
  
              {
  
                      value = avcodec_decode_video2(video_dec_ctx , pAVFrame , 
  
    &amp;frameFinished , pAVPacket );
  
                      if( value &lt; 0)
  
                      {
  
                              printf("Error : avcodec_decode_video2()");
  
                      }
  
  
  
                      if(frameFinished)// Frame successfully decoded :)
  
                      {
  
                              sws_scale(swsCtx_, pAVFrame->data, pAVFrame-
  
   >linesize,0, video_dec_ctx->height, outFrame->data,outFrame->linesize);
  
  //                              sws_scale(swsCtx_, pAVFrame->data, pAVFrame-
  
  >linesize,0, video_dec_ctx->height, outFrame->data,outFrame->linesize);
  
                              av_init_packet(&amp;outPacket);
  
                              outPacket.data = NULL;    // packet data will be 
  
   allocated by the encoder
  
                              outPacket.size = 0;
  
  
  
                              avcodec_encode_video2(outAVCodecContext , 
  
   &amp;outPacket ,outFrame , &amp;got_picture);
  
  
  
                              if(got_picture)
  
                              {
  
                                      if(outPacket.pts != AV_NOPTS_VALUE)
  
                                              outPacket.pts = 
  
   av_rescale_q(outPacket.pts, video_st->codec->time_base, video_st-
  
   >time_base);
  
                                      if(outPacket.dts != AV_NOPTS_VALUE)
  
                                              outPacket.dts = 
  
   av_rescale_q(outPacket.dts, video_st->codec->time_base, video_st-
  
   >time_base);
  
  
  
                                      printf("Write frame %3d (size= %2d)\n", 
  
   j++, outPacket.size/1000);
  
                                      if(av_write_frame(outAVFormatContext , 
  
   &amp;outPacket) != 0)
  
                                      {
  
                                              printf("\n\nError : 
  
   av_write_frame()");
  
                                      }
  
  
  
                              av_packet_unref(&amp;outPacket);
  
                              } // got_picture
  
  
  
                      av_packet_unref(&amp;outPacket);
  
                      } // frameFinished
  
  
  
              }
  
      }// End of while-loop
  
  
  
      value = av_write_trailer(outAVFormatContext);
  
      if( value &lt; 0)
  
      {
  
              printf("\n\nError : av_write_trailer()");
  
      }
  
  
  
  
  
      //THIS WAS ADDED LATER
  
      av_free(video_outbuf);
  
  
  
  end:
  
      avcodec_free_context(&amp;video_dec_ctx);
  
      avcodec_free_context(&amp;audio_dec_ctx);
  
      avformat_close_input(&amp;fmt_ctx);
  
      if (video_dst_file)
  
          fclose(video_dst_file);
  
     if (audio_dst_file)
  
         fclose(audio_dst_file);
  
     //av_frame_free(&amp;frame);
  
     av_free(video_dst_data[0]);
  
     return ret &lt; 0;
  
  }

  Problem with above code is that it rotates a video to left by 90 degree.

  Snapshot of video given as input to above program

  Snapshot of output video. It is rotated by 90 degree to left.

  I compiled program using below command :

  g++ -D__STDC_CONSTANT_MACROS -Wall -g ScreenRecorder.cpp -I/home/harry/Documents/compressor/ffmpeg-3.3/ -I/root/android-ndk-r14b/platforms/android-21/arch-x86_64/usr/include/ -c -o ScreenRecorder.o -w

  And linked it using below command :

  g++ -Wall -g ScreenRecorder.o -I/home/harry/Documents/compressor/ffmpeg-3.3/ -I/root/android-ndk-r14b/platforms/android-21/arch-x86_64/usr/include/ -L/usr/lib64 -L/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7/ -L/home/harry/Documents/compressor/ffmpeg-3.3/ffmpeg-build -L/root/android-ndk-r14b/platforms/android-21/arch-x86_64/usr/lib64 -o ScreenRecorder.exe -lavformat -lavcodec -lavutil -lavdevice -lavfilter -lswscale -lx264 -lswresample -lm -lpthread -ldl -lstdc++ -lc -lrt

  Program is being run using below command :

  ./ScreenRecorder.exe vertical.MOV videoH.mp4 audioH.mp3

  Note :
  - Source video is taken from iphone and is of .mov format.
  - Output video is being stored in .mp4 file.

  Can anyone please tell me why it is rotating video by 90 degree ?

  One thing i noticed in dump is shown below :

    Duration: 00:00:06.04, start: 0.000000, bitrate: 17087 kb/s
  
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 17014 kb/s, 29.98 fps, 29.97 tbr, 600 tbn, 1200 tbc (default)
  
  Metadata:
  
    rotate          : 90
  
    creation_time   : 2017-07-09T10:56:42.000000Z
  
    handler_name    : Core Media Data Handler
  
    encoder         : H.264
  
  Side data:
  
    displaymatrix: rotation of -90.00 degrees

  it says displaymatrix: rotation of -90.00 degrees. Is it responsible for rotating video by 90 degree ?