Recherche avancée

Sur d’autres sites (12368)

• avcodec : disallow hwaccel with frame threads

  23 octobre 2015, par Hendrik Leppkes

  avcodec : disallow hwaccel with frame threads
  HWAccels with frame threads are fundamentally flawed in avcodecs current
  
  design, and there are several known problems ranging from image corruption
  
  to driver crashes.
  These problems come down to two design problems in the interaction of
  
  threads and HWAccel decoding :
  (1)
  
  While avcodec prevents parallel decoding and as such simultaneous access
  
  to the hardware accelerator from the decoding threads, it cannot account
  
  for the user code and its access to the hardware surfaces and the hardware
  
  itself.
  This can result in image corruption or even driver crashes if the
  
  user code locks image surfaces while they are being used by the decoder
  
  threads as reference frames.
  The current HWAccel API does not offer any way to ensure exclusive access
  
  to the hardware or the surfaces if frame threading is used.
  (2)
  
  Initialization of the HWAccel with frame threads is non-trivial, and many
  
  decoders had and still have issues that cause excess calls to the
  
  get_format callback.
  This will potentially cause duplicate HWAccel initialization, which in
  
  extreme cases can even lead to driver crashes if the HWAccel is
  
  re-initialized while the user code is actively accessing the hardware
  
  surfaces associated with it, or lead to image corruption due to lost
  
  reference frames.
  While both of these issues are solvable, fixing (1) would at least require
  
  a huge API redesign which would move a lot of complexity into the user
  
  code.
  The only reason the combination of frame threads and HWAccel was
  
  considered useful is to allow a seamless fallback to multi-threaded
  
  software decoding if the HWAccel is not available, however the issues
  
  outlined above far outweigh this.
  The proper solution for a fallback is to re-open the AVCodecContext with
  
  threading enabled if the HWAccel failed, which is a practice commonly used
  
  by various user applications using avcodec today already.
  Reviewed-by : Gwenole Beauchesne <gb.devel@gmail.com>
  
  Reviewed-by : wm4 <nfxjfg@googlemail.com>
  
  Signed-off-by : Hendrik Leppkes <h.leppkes@gmail.com>
  

  • [DH] libavcodec/utils.c

• FFmpeg, access violation on av_frame_free when running though Unity

  27 octobre 2016, par Mockarutan

  I’m working on a video recording plugin for Unity using ffmpeg. I’m new to the video encoding domain and just got my code to work today. But I think a might have a few memory leaks and trying to fix them crashes Unity.

  The plugin is written i c++ (as external "C" code) and imported in a c# script in unity with a simple DllImport. Again, this is not my comfort area either, but it works.

  When a screen buffer is rendered, I put in a RGB24 buffer and send it to my c++ function, this one :

  int encode_frame(uint8_t* rgb24Data)
  
  {
  
      AVFrame *frame = av_frame_alloc();
  
      if (!frame)
  
          return COULD_NOT_ALLOCATE_FRAME;
  
  
  
      frame->format = codec_context->pix_fmt;
  
      frame->width = codec_context->width;
  
      frame->height = codec_context->height;
  
  
  
      int ret = av_image_alloc(frame->data, frame->linesize, codec_context->width, codec_context->height, codec_context->pix_fmt, 32);
  
      if (ret < 0)
  
          return COULD_NOT_ALLOCATE_PIC_BUF;
  
  
  
      SwsContext * ctx = sws_getContext(codec_context->width, codec_context->height,
  
          AV_PIX_FMT_RGB24, codec_context->width, codec_context->height,
  
          AV_PIX_FMT_YUV420P, 0, 0, 0, 0);
  
  
  
  
  
      uint8_t * inData[1] = { rgb24Data };
  
      int inLinesize[1] = { 3 * codec_context->width };
  
  
  
      sws_scale(ctx, inData, inLinesize, 0, codec_context->height, frame->data, frame->linesize); // From RGB to YUV
  
  
  
      frame->pts = frame_counter++;
  
  
  
      ret = avcodec_send_frame(codec_context, frame);
  
      if (ret < 0)
  
          return ERROR_ENCODING_FRAME_SEND;
  
  
  
      AVPacket pkt;
  
      av_init_packet(&pkt);
  
      pkt.data = NULL;
  
      pkt.size = 0;
  
  
  
      while (true)
  
      {
  
          ret = avcodec_receive_packet(codec_context, &pkt);
  
          if (!ret)
  
          {
  
              if (pkt.pts != AV_NOPTS_VALUE)
  
                  pkt.pts = av_rescale_q(pkt.pts, codec_context->time_base, video_st->time_base);
  
              if (pkt.dts != AV_NOPTS_VALUE)
  
                  pkt.dts = av_rescale_q(pkt.dts, codec_context->time_base, video_st->time_base);
  
  
  
              av_write_frame(outctx, &pkt);
  
              av_packet_unref(&pkt);
  
          }
  
          else if (ret == AVERROR(EAGAIN))
  
          {
  
              frame->pts = frame_counter++;
  
              ret = avcodec_send_frame(codec_context, frame);
  
              if (ret < 0)
  
                  return ERROR_ENCODING_FRAME_SEND;
  
          }
  
          else if (ret < 0)
  
              return ERROR_ENCODING_FRAME_RECEIVE;
  
          else
  
              break;
  
      }
  
  
  
      // This one
  
      av_frame_free(&frame);
  
  }

  Now, this code might have a lot of issues that I’m not aware of, and you are free to point them out if you like. But the line that gives me error is av_frame_free(&frame);.

  If I run this in a synthetic test app in c++ that I made, it works. I can even run it in a c# synthetic test app (exactly like the c++ one), and it works. But if I run it though Unity, it crashes on the first frame. The log says "Read from location fe7f8097 caused an access violation.".

  I have tried with av_freep() and av_free(). Not sure exactly what makes them different (different example codes use different ones), but none work.

  So, what I’m I missing ? The frame is leaking if I don’t free it right ? But why does it crash in Unity ?

  The whole thing works great in Unity if I don’t have the av_frame_free(&frame);. Resulting video looks great !

  PS. I’m aware (as far as I know) that the frame also leaks if something fails and returns an error code. But one thing at a time.

• FFmpeg, access violation on av_frame_free when running though Unity

  1er février 2021, par Mockarutan

  I'm working on a video recording plugin for Unity using ffmpeg. I'm new to the video encoding domain and just got my code to work today. But I think a might have a few memory leaks and trying to fix them crashes Unity.

  



  The plugin is written i c++ (as external "C" code) and imported in a c# script in unity with a simple DllImport. Again, this is not my comfort area either, but it works.

  



  When a screen buffer is rendered, I put in a RGB24 buffer and send it to my c++ function, this one :

  



  int encode_frame(uint8_t* rgb24Data)
{
    AVFrame *frame = av_frame_alloc();
    if (!frame)
        return COULD_NOT_ALLOCATE_FRAME;

    frame->format = codec_context->pix_fmt;
    frame->width = codec_context->width;
    frame->height = codec_context->height;

    int ret = av_image_alloc(frame->data, frame->linesize, codec_context->width, codec_context->height, codec_context->pix_fmt, 32);
    if (ret < 0)
        return COULD_NOT_ALLOCATE_PIC_BUF;

    SwsContext * ctx = sws_getContext(codec_context->width, codec_context->height,
        AV_PIX_FMT_RGB24, codec_context->width, codec_context->height,
        AV_PIX_FMT_YUV420P, 0, 0, 0, 0);


    uint8_t * inData[1] = { rgb24Data };
    int inLinesize[1] = { 3 * codec_context->width };

    sws_scale(ctx, inData, inLinesize, 0, codec_context->height, frame->data, frame->linesize); // From RGB to YUV

    frame->pts = frame_counter++;

    ret = avcodec_send_frame(codec_context, frame);
    if (ret < 0)
        return ERROR_ENCODING_FRAME_SEND;

    AVPacket pkt;
    av_init_packet(&pkt);
    pkt.data = NULL;
    pkt.size = 0;

    while (true)
    {
        ret = avcodec_receive_packet(codec_context, &pkt);
        if (!ret)
        {
            if (pkt.pts != AV_NOPTS_VALUE)
                pkt.pts = av_rescale_q(pkt.pts, codec_context->time_base, video_st->time_base);
            if (pkt.dts != AV_NOPTS_VALUE)
                pkt.dts = av_rescale_q(pkt.dts, codec_context->time_base, video_st->time_base);

            av_write_frame(outctx, &pkt);
            av_packet_unref(&pkt);
        }
        else if (ret == AVERROR(EAGAIN))
        {
            frame->pts = frame_counter++;
            ret = avcodec_send_frame(codec_context, frame);
            if (ret < 0)
                return ERROR_ENCODING_FRAME_SEND;
        }
        else if (ret < 0)
            return ERROR_ENCODING_FRAME_RECEIVE;
        else
            break;
    }

    // This one
    av_frame_free(&frame);
}


  



  Now, this code might have a lot of issues that I'm not aware of, and you are free to point them out if you like. But the line that gives me error is av_frame_free(&frame);.

  



  If I run this in a synthetic test app in c++ that I made, it works. I can even run it in a c# synthetic test app (exactly like the c++ one), and it works. But if I run it though Unity, it crashes on the first frame. The log says "Read from location fe7f8097 caused an access violation.".

  



  I have tried with av_freep() and av_free(). Not sure exactly what makes them different (different example codes use different ones), but none work.

  



  So, what I'm I missing ? The frame is leaking if I don't free it right ? But why does it crash in Unity ?

  



  The whole thing works great in Unity if I don't have the av_frame_free(&frame);. Resulting video looks great !

  



  PS. I'm aware (as far as I know) that the frame also leaks if something fails and returns an error code. But one thing at a time.