Recherche avancée

Sur d’autres sites (5197)

• How to extract frames at 30 fps using FFMPEG APIs on Android ?

  11 mars 2020, par Amber Beriwal

  We are working on a project that consumes FFMPEG library for video frame extraction on Android platform.

  On Windows, we have observed :

  • Using CLI, ffmpeg is capable of extracting frames at 30 fps using command ffmpeg -i input.flv -vf fps=1 out%d.png.
  • Using Xuggler, we are able to extract frames at 30 fps.
  • Using FFMPEG APIs directly in code, we are getting frames at 30 fps.

  But when we use FFMPEG APIs directly on Android (See Hardware Details), we are getting following results :

  • 720p video (1280 x 720) - 16 fps (approx. 60 ms/frame)
  • 1080p video (1920 x 1080) - 7 fps (approx. 140 ms/frame)

  We haven’t tested Xuggler/CLI on Android yet.

  Ideally, we should be able to get the data in constant time (approx. 30 ms/frame).

  How can we get 30 fps on Android ?

  Code being used on Android :

  if (avformat_open_input(&pFormatCtx, pcVideoFile, NULL, NULL)) {
  
      iError = -1;  //Couldn't open file
  
  }
  
  
  
  if (!iError) {
  
      //Retrieve stream information
  
      if (avformat_find_stream_info(pFormatCtx, NULL) < 0)
  
          iError = -2; //Couldn't find stream information
  
  }
  
  
  
  //Find the first video stream
  
  if (!iError) {
  
  
  
      for (i = 0; i < pFormatCtx->nb_streams; i++) {
  
          if (AVMEDIA_TYPE_VIDEO
  
                  == pFormatCtx->streams[i]->codec->codec_type) {
  
              iFramesInVideo = pFormatCtx->streams[i]->nb_index_entries;
  
              duration = pFormatCtx->streams[i]->duration;
  
              begin = pFormatCtx->streams[i]->start_time;
  
              time_base = (pFormatCtx->streams[i]->time_base.num * 1.0f)
  
                      / pFormatCtx->streams[i]->time_base.den;
  
  
  
              pCodecCtx = avcodec_alloc_context3(NULL);
  
              if (!pCodecCtx) {
  
                  iError = -6;
  
                  break;
  
              }
  
  
  
              AVCodecParameters params = { 0 };
  
              iReturn = avcodec_parameters_from_context(&params,
  
                      pFormatCtx->streams[i]->codec);
  
              if (iReturn < 0) {
  
                  iError = -7;
  
                  break;
  
              }
  
  
  
              iReturn = avcodec_parameters_to_context(pCodecCtx, &params);
  
              if (iReturn < 0) {
  
                  iError = -7;
  
                  break;
  
              }
  
  
  
              //pCodecCtx = pFormatCtx->streams[i]->codec;
  
  
  
              iVideoStreamIndex = i;
  
              break;
  
          }
  
      }
  
  }
  
  
  
  if (!iError) {
  
      if (iVideoStreamIndex == -1) {
  
          iError = -3; // Didn't find a video stream
  
      }
  
  }
  
  
  
  if (!iError) {
  
      // Find the decoder for the video stream
  
      pCodec = avcodec_find_decoder(pCodecCtx->codec_id);
  
      if (pCodec == NULL) {
  
          iError = -4;
  
      }
  
  }
  
  
  
  if (!iError) {
  
      // Open codec
  
      if (avcodec_open2(pCodecCtx, pCodec, NULL) < 0)
  
          iError = -5;
  
  }
  
  
  
  if (!iError) {
  
      iNumBytes = av_image_get_buffer_size(AV_PIX_FMT_RGB24, pCodecCtx->width,
  
              pCodecCtx->height, 1);
  
  
  
      // initialize SWS context for software scaling
  
      sws_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height,
  
              pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height,
  
              AV_PIX_FMT_RGB24,
  
              SWS_BILINEAR,
  
              NULL,
  
              NULL,
  
              NULL);
  
      if (!sws_ctx) {
  
          iError = -7;
  
      }
  
  }
  
  clock_gettime(CLOCK_MONOTONIC_RAW, &end);
  
  delta_us = (end.tv_sec - start.tv_sec) * 1000000
  
          + (end.tv_nsec - start.tv_nsec) / 1000;
  
  start = end;
  
  //LOGI("Starting_Frame_Extraction: %lld", delta_us);
  
  if (!iError) {
  
      while (av_read_frame(pFormatCtx, &packet) == 0) {
  
          // Is this a packet from the video stream?
  
          if (packet.stream_index == iVideoStreamIndex) {
  
              pFrame = av_frame_alloc();
  
              if (NULL == pFrame) {
  
                  iError = -8;
  
                  break;
  
              }
  
  
  
              // Decode video frame
  
              avcodec_decode_video2(pCodecCtx, pFrame, &iFrameFinished,
  
                      &packet);
  
              if (iFrameFinished) {
  
                  //OUR CODE
  
              }
  
              av_frame_free(&pFrame);
  
              pFrame = NULL;
  
          }
  
          av_packet_unref(&packet);
  
      }
  
  }

• tls_gnutls : fix hang on disconnection

  14 juin 2015, par wm4

  tls_gnutls : fix hang on disconnection
  GNUTLS_SHUT_RDWR means GnuTLS will keep waiting for the server’s
  
  termination reply. But since we don’t shutdown the TCP connection at
  
  this point yet, GnuTLS will just keep skipping actual data from the
  
  server, which basically is perceived as hang.
  Use GNUTLS_SHUT_WR instead, which doesn’t have this problem.
  Signed-off-by : Martin Storsjö <martin@martin.st>
  

  • [DH] libavformat/tls_gnutls.c

• avformat/mov : fix hang while seek on a kind of fragmented mp4

  3 février 2019, par Charles Liu

  avformat/mov : fix hang while seek on a kind of fragmented mp4
  Binary searching would hang if the fragment items do NOT have timestamp for the
  
  specified stream.
  For example, a fmp4 consists of separated 'moof' boxes for each track, and
  
  separated 'sidx' for each segment, but no 'mfra' box.  Then every fragment item
  
  only have the timestamp for one of its tracks.
  Example :
  
  ffmpeg -f lavfi -i testsrc -f lavfi -i sine -movflags dash+frag_keyframe+skip_trailer+separate_moof -t 1 out.mp4
  
  ffmpeg -ss 0.5 -i out.mp4 -f null none
  Also fixes the hang in ticket #7572, but not the reason for having
  
  AV_NOPTS_VALUE timestamps there.
  Signed-off-by : Charles Liu <liuchh83@gmail.com>
  
  Signed-off-by : Marton Balint <cus@passwd.hu>
  

  • [DH] libavformat/mov.c