Recherche avancée

Sur d’autres sites (7867)

• build : generalise rules and variable settings for av* programs

  4 août 2012, par Mans Rullgard

  build : generalise rules and variable settings for av* programs

• build : simplify rules for metal

  20 décembre 2021, par Zhao Zhili

  build : simplify rules for metal
  Signed-off-by : Aman Karmani <aman@tmm1.net>
  

  • [DH] ffbuild/common.mak

• FFMPEG making requests for each frame when decoding a stream, slow performance

  3 juillet 2020, par Byti

  I am having an issue playing MOV camera captured files from an iPhone. My FFMPEG implementation has no problem playing most file formats, this issue is exclusive only for camera captured MOV.

  



  When trying to open the file, I can see in the logs that many requests are made, each requests decoding only one frame, before making a new request which results the video being buffered extremely slowly.
It takes roughly a minute to buffer about a few seconds of the video.

  



  Another thing to mention is that the very same problematic file is played without an issue locally. The problem is when trying to decode while streaming.

  



  I compiled my code on Xcode 11, iOS SDK 13, with cocoapods mobile-ffmpeg-https 4.2.

  



  Here is a rough representation of my code, its pretty standard :

  



  

  1. Here is how I open AVFormatContext :

  


  



  AVFormatContext *context = avformat_alloc_context();
    context->interrupt_callback.callback = FFMPEGFormatContextIOHandler_IO_CALLBACK;
    context->interrupt_callback.opaque = (__bridge void *)(handler);

    av_log_set_level(AV_LOG_TRACE);

    int result = avformat_open_input(&context, [request.urlAsString UTF8String], NULL, NULL);

    if (result != 0) {
        if (context != NULL) {
            avformat_free_context(context);
        }

        return nil;
    }

    result = avformat_find_stream_info(context, NULL);

    if (result < 0) {
        avformat_close_input(&context);
        return nil;
    }


  



  

  2. Video decoder is opened like so, audio decoder is nearly identical

  


  



  AVCodecParameters *params = context->streams[streamIndex]->codecpar;
    AVCodec *codec = avcodec_find_decoder(params->codec_id);

    if (codec == NULL) {
        return NULL;
    }

    AVCodecContext *codecContext = avcodec_alloc_context3(codec);

    if (codecContext == NULL) {
        return NULL;
    }

    codecContext->thread_count = 6;

    int result = avcodec_parameters_to_context(codecContext, params);

    if (result < 0) {
        avcodec_free_context(&codecContext);
        return NULL;
    }

    result = avcodec_open2(codecContext, codec, NULL);

    if (result < 0) {
        avcodec_free_context(&codecContext);
        return NULL;
    }


  



  

  3. I read the data from the server like so :

  


  



  AVPacket packet;

int result = av_read_frame(formatContext, &avPacket);

if (result == 0) {
   avcodec_send_packet(codecContext, &avPacket);

   // .... decode ....
}


  



  Logs after opening the decoders :

  



  // [tls] Request is made here
// [tls] Request response headers are here
Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:2048
Probing mp3 score:1 size:2048
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] type:'ftyp' parent:'root' sz: 20 8 23077123
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] ISO: File Type Major Brand: qt  
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] type:'wide' parent:'root' sz: 8 28 23077123
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] type:'mdat' parent:'root' sz: 23066642 36 23077123
// [tls] Request is made here
// [tls] Request response headers are here
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] stream 0, sample 4, dts 133333
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] stream 1, sample 48, dts 1114558
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] stream 2, sample 1, dts 2666667
[h264 @ 0x116080200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
// [tls] Request is made here
// [tls] Request response headers are here
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] stream 0, sample 4, dts 133333
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] stream 1, sample 48, dts 1114558
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x115918e00] stream 2, sample 1, dts 2666667
[h264 @ 0x116080200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
// [tls] Request is made here
// [tls] Request response headers are here
// ...


  



  These are some warnings I found in the log

  



  [mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] interrupted
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] stream 0: start_time: 0.000 duration: 11.833
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] stream 1: start_time: 0.000 duration: 11.832
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] stream 2: start_time: 0.000 duration: 11.833
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] stream 3: start_time: 0.000 duration: 11.833
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] format: start_time: 0.000 duration: 11.833 bitrate=15601 kb/s
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] Could not find codec parameters for stream 0 (Video: h264, 1 reference frame (avc1 / 0x31637661), none(bt709, left), 1920x1080, 1/1200, 15495 kb/s): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x11c030800] After avformat_find_stream_info() pos: 23077123 bytes read:16293 seeks:1 frames:0


  



  Also when calling avformat_open_input(...), 2 GET requests are made, before the returning.
Notice the "Probing mp3 score:1", that is not shown for other MOV files or any other files.

  



  I have tried different versions of ffmpeg, I have tried messing around with the delays of the stream, I tried removing my custom interrupt callback, nothing have worked.

  



  Code works fine with any other videos I have tested (mp4, mkv, avi).

  



  Metadata of the test file :

  



  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2019-04-14T08:17:03.000000Z
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 7
    com.apple.quicktime.software: 12.2
    com.apple.quicktime.creationdate: 2019-04-14T11:17:03+0300
  Duration: 00:00:16.83, bitrate: N/A
    Stream #0:0(und), 0, 1/600: Video: h264, 1 reference frame (avc1 / 0x31637661), none(bt709), 1920x1080 (0x0), 0/1, 15301 kb/s, 30 fps, 30 tbr, 600 tbn (default)
    Metadata:
      creation_time   : 2019-04-14T08:17:03.000000Z
      handler_name    : Core Media Video
      encoder         : H.264
    Stream #0:1(und), 0, 1/44100: Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, 100 kb/s (default)
    Metadata:
      creation_time   : 2019-04-14T08:17:03.000000Z
      handler_name    : Core Media Audio
    Stream #0:2(und), 0, 1/600: Data: none (mebx / 0x7862656D), 0/1, 0 kb/s (default)
    Metadata:
      creation_time   : 2019-04-14T08:17:03.000000Z
      handler_name    : Core Media Metadata
    Stream #0:3(und), 0, 1/600: Data: none (mebx / 0x7862656D), 0/1, 0 kb/s (default)
    Metadata:
      creation_time   : 2019-04-14T08:17:03.000000Z
      handler_name    : Core Media Metadata