Recherche avancée

Sur d’autres sites (5614)

• How to create video using avcodec from jpeg images of type OpenCV::Mat ?

  23 juillet 2015, par theateist

  I have colored jpeg images of OpenCV::Mat type and I create from them video using avcodec. The video that I get is upside-down, black & white and each row of each frame is shifted and I got diagonal line. What could be the reason for such output ?
  Follow this link to watch the video I get using avcodec.
  I’m using acpicture_fill function to create avFrame from cv::Mat frame !

  P.S.
  Each cv::Mat cvFrame has width=810, height=610, step=2432
  I noticed that avFrame (that is filled by acpicture_fill) has linesize[0]=2430
  I tried manually setting avFrame->linesizep0]=2432 and not 2430 but it still didn’t helped.

  ======== CODE =========================================================

  AVCodec *encoder = avcodec_find_encoder(AV_CODEC_ID_H264);
  
  AVStream *outStream = avformat_new_stream(outContainer, encoder);
  
  avcodec_get_context_defaults3(outStream->codec, encoder);
  
  
  
  outStream->codec->pix_fmt = AV_PIX_FMT_YUV420P;
  
  outStream->codec->width = 810;
  
  outStream->codec->height = 610;
  
  //...
  
  
  
  SwsContext *swsCtx = sws_getContext(outStream->codec->width, outStream->codec->height, PIX_FMT_RGB24,
  
                                      outStream->codec->width, outStream->codec->height,  outStream->codec->pix_fmt, SWS_BICUBIC, NULL, NULL, NULL);
  
  
  
  for (uint i=0; i < frameNums; i++)
  
  {
  
      // get frame at location I using OpenCV
  
      cv::Mat cvFrame;
  
      myReader.getFrame(cvFrame, i); 
  
      cv::Size frameSize = cvFrame.size();    
  
      //Each cv::Mat cvFrame has  width=810, height=610, step=2432
  
  
  
  
  
  1.  // create AVPicture from cv::Mat frame
  
  2.  avpicture_fill((AVPicture*)avFrame, cvFrame.data, PIX_FMT_RGB24, outStream->codec->width, outStream->codec->height);
  
  3avFrame->width = frameSize.width;
  
  4.  avFrame->height = frameSize.height;
  
  
  
      // rescale to outStream format
  
      sws_scale(swsCtx, avFrame->data, avFrame->linesize, 0, outStream->codec->height, avFrameRescaledFrame->data, avFrameRescaledFrame ->linesize);
  
  encoderRescaledFrame->pts=i;
  
  avFrameRescaledFrame->width = frameSize.width;
  
      avFrameRescaledFrame->height = frameSize.height;
  
  
  
  av_init_packet(&avEncodedPacket);
  
      avEncodedPacket.data = NULL;
  
      avEncodedPacket.size = 0;
  
  
  
      // encode rescaled frame
  
      if(avcodec_encode_video2(outStream->codec, &avEncodedPacket, avFrameRescaledFrame, &got_frame) < 0) exit(1);
  
      if(got_frame)
  
      {
  
          if (avEncodedPacket.pts != AV_NOPTS_VALUE)
  
              avEncodedPacket.pts =  av_rescale_q(avEncodedPacket.pts, outStream->codec->time_base, outStream->time_base);
  
          if (avEncodedPacket.dts != AV_NOPTS_VALUE)
  
              avEncodedPacket.dts = av_rescale_q(avEncodedPacket.dts, outStream->codec->time_base, outStream->time_base);
  
  
  
          // outContainer is "mp4"
  
          av_write_frame(outContainer, & avEncodedPacket);
  
  
  
          av_free_packet(&encodedPacket);
  
      }
  
  }

  UPDATED

  As @Alex suggested I changed the lines 1-4 with the code below

  int width = frameSize.width, height = frameSize.height; 
  
  avpicture_alloc((AVPicture*)avFrame, AV_PIX_FMT_RGB24, outStream->codec->width, outStream->codec->height);
  
  for (int h = 0; h < height; h++)
  
  {
  
       memcpy(&(avFrame->data[0][h*avFrame->linesize[0]]), &(cvFrame.data[h*cvFrame.step]), width*3);
  
  }

  The video (here) I get now is almost perfect. It’s NOT upside-down, NOT black & white, BUT it seems that one of the RGB components is missing. Every brown/red colors became blue (in original images it should be vice-verse).
  What could be the problem ? Could rescaling(sws_scale) to AV_PIX_FMT_YUV420P format causes this ?

• Fastest way to extract raw Y' plane data from Y'Cb'Cr encoded video ?

  20 février 2024, par memeko

  I have a use-case where I'm extracting I-Frames from videos and turning them into perceptual hashes for later analysis.

  


  ⠀

  


  I'm currently using ffmpeg to do this with a command akin to :

  


  ffmpeg -skip_frame nokey -i 'in%~1.mkv' -vsync vfr -frame_pts true -vf 'keyframes/_Y/out%~1/%%06d.bmp'

  


  and then reading in the data from the resulting images.

  


  ⠀

  


  This is a bit wasteful as, to my understanding, ffmpeg does implicit YUV -> RGB colour-space conversion and I'm also needlessly saving intermediate data to disk.

  


  Most modern video codecs utilise chroma subsampling and have frames encoded in a Y'C_bC_r colour-space, where Y' is the luma component, and Cb Cr are the blue-difference, red-difference chroma components.

  


  Which in something like YUV420p used in h.264/h.265 video codecs is encoded as such :

  


  

  


  Where each Y' value is 8 bits long and corresponds to a pixel.

  


  ⠀

  


  As I use gray-scale data for generating the perceptual hashes anyway, I was wondering if there is a way to simply grab just the raw Y' values from any given I-Frame into an array and skip all of the unnecessary conversions and extra steps ?

  


  (as the luma component is essentially equivalent to the grayscale data i need for generating hashes)

  


  I came across the -vf 'extractplanes=y' filter in ffmpeg that seems like it might do just that, but according to source :

  


  


  "...what is extracted by 'extractplanes' is not raw data of the (for example) Y plane. Each extracted is converted to grayscale. That is, the converted video data has YUV (or RGB) which is different from the input."

  


  


  which makes it seem like it's touching chroma components and doing some conversion anyway, in testing applying this filter didn't affect the processing time of the I-Frame extraction either.

  


  ⠀

  


  My script is currently written in Python, but I am in the process of migrating it to C++, so I would prefer any solutions pertaining to the latter.

  


  ffmpeg seems like the ideal candidate for this task, but I really am looking for whatever solution that would ingest the data fastest, preferably saving directly to RAM, as I'll be processing a large number of video files and discarding I-Frame luma pixel data once a hash has been generated.

  


  I would also like to associate each I-Frame with its corresponding frame number in the video.

  


• libavcodec : how to encode raw video (YUV420P) frame to the valid bitmap for saving to the ".bmp" file ?

  26 octobre 2020, par Joe Dudinski

  I'm trying to save random video frames into the '.bmp' files by encoding raw video (YUV420P) into AV_CODEC_ID_BMP format using libavcodec, but result is broken image, contains 3 RGB channels :

  


  Resulting bmp


  


  Blue channel :


  


  Green channel :


  


  Red channel :


  


  Original YUV image :


  


  My code to get current yuv frame and encode it is into BMP :

  


  int __create_bmp(AVPacket *pkt, AVCodecContext *video_dec_ctx, int video_stream_idx) {
    int err = 0;
    int got_frame = 0;
    static int vfnum = 0; //current v.frame num

    if (pkt->stream_index == video_stream_idx) {

        fprintf(stderr, "AVCodecContext codec: (name: %s), ID %d\n",
                video_dec_ctx->codec->name, video_dec_ctx->codec_id);

        /* decode video frame */
        AVFrame *frame = av_frame_alloc();

        // !!! 'avcodec_decode_video2' IS DEPRECATED !!!
        err = avcodec_decode_video2(video_dec_ctx, frame, got_frame, pkt);
        if (err < 0) {
            fprintf(stderr, "Error decoding video frame (%s)\n", av_err2str(err));
            return err;
        }
        if(got_frame == 0) return 1;

        // !!! this dbg func save mjpeg YUV420P into '.jpg' file FINE !!!
        __encode_image_frame(frame);

        AVCodec *bmpc = avcodec_find_encoder(AV_CODEC_ID_BMP);
        if(bmpc == NULL){
            fprintf(stderr, "'AV_CODEC_ID_BMP' encoder NOT FOUND!.\n");
            exit(-1);
        }
        AVCodecContext *__bmpcc = avcodec_alloc_context3(bmpc);
        __bmpcc->width = frame->width;
        __bmpcc->height = frame->height;
        __bmpcc->pix_fmt = AV_PIX_FMT_BGR24;
        __bmpcc->time_base = (AVRational){1,1};

        err = avcodec_open2(__bmpcc, bmpc, NULL);
        if(err == 0){
            fprintf(stderr, "avcodec_open2 success.\n");
        }else{
            fprintf(stderr, "avcodec_open2 ERROR!\n");
        }

        err = avcodec_send_frame(__bmpcc, frame);
        if(err == 0){
            fprintf(stderr, "avcodec_send_frame success.\n");
        }
        else{
            fprintf(stderr, "avcodec_send_frame ERROR! (code: %d)\n",err);
        }

        AVPacket *bmp_pkt = av_packet_alloc();
        err = avcodec_receive_packet(__bmpcc, bmp_pkt);
        if(err == 0){
            fprintf(stderr, "avcodec_receive_packet success.\n");
        }
        else{
            fprintf(stderr, "avcodec_receive_packet ERROR! (code: %d)\n",err);
        }
        av_frame_unref(frame);

        /* write to bitmap file */
        char outimg[2048] = {0};
        snprintf(outimg, 2048,"out-%06d.bmp", vfnum);
        FILE *outf = fopen(outimg, "wb");
        vfnum++;
        if (!outf) {
            fprintf(stderr, "Could not open destination file %s\n", outimg);
        }else{
            fwrite(bmp_pkt->data, 1, bmp_pkt->size , outf);
            fclose(outf);
        }
        av_packet_unref(bmp_pkt);
        return 0;
    }
    return 1;
}


  


  Does anyone know how to do this correctly ?