Recherche avancée

Sur d’autres sites (5288)

• avformat/mov : fix seeking with HEVC open GOP files

  18 février 2022, par Clément Bœsch

  avformat/mov : fix seeking with HEVC open GOP files
  This was tested with medias recorded from an iPhone XR and an iPhone 13.
  Here is how a typical stream looks like in coding order :
      ┌────────┬─────┬─────┬──────────┐
  
      │ sample | PTS | DTS | keyframe |
  
      ├────────┼─────┼─────┼──────────┤
  
      ┊        ┊     ┊     ┊          ┊
  
      │   53   │ 560 │ 510 │    No    │
  
      │   54   │ 540 │ 520 │    No    │
  
      │   55   │ 530 │ 530 │    No    │
  
      │   56   │ 550 │ 540 │    No    │
  
      │   57   │ 600 │ 550 │    Yes   │
  
      │ * 58   │ 580 │ 560 │    No    │
  
      │ * 59   │ 570 │ 570 │    No    │
  
      │ * 60   │ 590 │ 580 │    No    │
  
      │   61   │ 640 │ 590 │    No    │
  
      │   62   │ 620 │ 600 │    No    │
  
      ┊        ┊     ┊     ┊          ┊
  In composition/display order :
      ┌────────┬─────┬─────┬──────────┐
  
      │ sample | PTS | DTS | keyframe |
  
      ├────────┼─────┼─────┼──────────┤
  
      ┊        ┊     ┊     ┊          ┊
  
      │   55   │ 530 │ 530 │    No    │
  
      │   54   │ 540 │ 520 │    No    │
  
      │   56   │ 550 │ 540 │    No    │
  
      │   53   │ 560 │ 510 │    No    │
  
      │ * 59   │ 570 │ 570 │    No    │
  
      │ * 58   │ 580 │ 560 │    No    │
  
      │ * 60   │ 590 │ 580 │    No    │
  
      │   57   │ 600 │ 550 │    Yes   │
  
      │   63   │ 610 │ 610 │    No    │
  
      │   62   │ 620 │ 600 │    No    │
  
      ┊        ┊     ┊     ┊          ┊
  Sample/frame 58, 59 and 60 are B-frames which actually depends on the
  
  key frame (57). Here the key frame is not an IDR but a "CRA" (Clean
  
  Random Access).
  Initially, I thought I could rely on the sdtp box (independent and
  
  disposable samples), but unfortunately :
      sdtp[54] is_leading:0 sample_depends_on:1 sample_is_depended_on:0 sample_has_redundancy:0
  
      sdtp[55] is_leading:0 sample_depends_on:1 sample_is_depended_on:2 sample_has_redundancy:0
  
      sdtp[56] is_leading:0 sample_depends_on:1 sample_is_depended_on:2 sample_has_redundancy:0
  
      sdtp[57] is_leading:0 sample_depends_on:2 sample_is_depended_on:0 sample_has_redundancy:0
  
      sdtp[58] is_leading:0 sample_depends_on:1 sample_is_depended_on:0 sample_has_redundancy:0
  
      sdtp[59] is_leading:0 sample_depends_on:1 sample_is_depended_on:2 sample_has_redundancy:0
  
      sdtp[60] is_leading:0 sample_depends_on:1 sample_is_depended_on:2 sample_has_redundancy:0
  
      sdtp[61] is_leading:0 sample_depends_on:1 sample_is_depended_on:0 sample_has_redundancy:0
  
      sdtp[62] is_leading:0 sample_depends_on:1 sample_is_depended_on:0 sample_has_redundancy:0
  The information that might have been useful here would have been
  
  is_leading, but all the samples are set to 0 so this was unusable.
  Instead, we need to rely on sgpd/sbgp tables. In my case the video track
  
  contained 3 sgpd tables with the following grouping types : tscl, sync
  
  and tsas. In the sync table we have the following 2 entries (only) :
      sgpd.sync[1] : sync nal_unit_type:0x14
  
      sgpd.sync[2] : sync nal_unit_type:0x15
  (The count starts at 1 because 0 carries the undefined semantic, we'll
  
  see that later in the reference table).
  The NAL unit types presented here correspond to :
      libavcodec/hevc.h :    HEVC_NAL_IDR_N_LP       = 20,
  
      libavcodec/hevc.h :    HEVC_NAL_CRA_NUT        = 21,
  In parallel, the sbgp sync table contains the following :
      ┌────┬───────┬─────┐
  
      │ id │ count │ gdi │
  
      ├────┼───────┼─────┤
  
      │  0 │   1   │  1  │
  
      │  1 │   56  │  0  │
  
      │  2 │   1   │  2  │
  
      │  3 │   59  │  0  │
  
      │  4 │   1   │  2  │
  
      │  5 │   59  │  0  │
  
      │  6 │   1   │  2  │
  
      │  7 │   59  │  0  │
  
      │  8 │   1   │  2  │
  
      │  9 │   59  │  0  │
  
      │ 10 │   1   │  2  │
  
      │ 11 │   11  │  0  │
  
      └────┴───────┴─────┘
  The gdi column (group description index) directly refers to the index in
  
  the sgpd.sync table. This means the first frame is an IDR, then we have
  
  batches of undefined frames interlaced with CRA frames. No IDR ever
  
  appears again (tried on a 30+ seconds sample).
  With that information, we can build an heuristic using the presentation
  
  order.
  A few things needed to be introduced in this commit :
  1. min_sample_duration is extracted from the stts : we need the minimal
  
     step between sample in order to PTS-step backward to a valid point
  
  2. In order to avoid a loop over the ctts table systematically during a
  
     seek, we build an expanded list of sample offsets which will be used
  
     to translate from DTS to PTS
  
  3. An open_key_samples index to keep track of all the non-IDR key
  
     frames ; for now it only supports HEVC CRA frames. We should probably
  
     add BLA frames as well, but I don't have any sample so I prefered to
  
     leave that for later
  It is entirely possible I missed something obvious in my approach, but I
  
  couldn't come up with a better solution. Also, as mentioned in the diff,
  
  we could optimize is_open_key_sample(), but the linear scaling overhead
  
  should be fine for now since it only happens in seek events.
  Fixing this issue prevents sending broken packets to the decoder. With
  
  FFmpeg hevc decoder the frames are skipped, with VideoToolbox the frames
  
  are glitching.
  

  • [DH] libavformat/isom.h
  • [DH] libavformat/mov.c

• avcodec_decode_audio4 decoded data different on device and simulator

  5 décembre 2013, par Jona

  I'm having issues where the decoded audio sounds punchy/garbled simply can't tell what is being heard when decoded on an iPhone/iPad device. However, when decoding the audio on simulator the audio is perfect.

  I do realize emulator runs on a different CPU architecture i386 while actual device is ARMv7 so the issue could be something around that.

  I have printed the compressed data on device and simulator before being decoded and it matches. Then I printed the decoded data for device and simulator and they are different. That leads me to think something is happening with the decoder. I can see that it is not a byte swap issue since the data is just completely different.

  Any ideas on what I could or look into ? I don't have this issue on Android...

  First decoded audio frame.

  DEVICE :

  0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x70, 0x08, 0x00, 0x00, 0x00, 0x00, 0x03, 0x00, 0xda, 0x0a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,

  EMULATOR :

  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,

  ---

  UPDATE 1
  I was able to resolve the issue by disabling asm with --disable-asm. Something must not be working correctly with the ARM asm files.

• Add a 2nd audio stream using ffmpeg so you can switch audio streams on iOS ?

  26 décembre 2019, par Tony M

  I have an m.mp4 with English audio and want to add a second audio stream in Italian so that I can play either language using the iPhone. When I used this command :

  ffmpeg -i m.mp4 -i ita.mp3 -c copy -map 0 -map 1 out.mp4 

  The output from ffmpeg -i out.mp4 (see below) shows that a new audio stream is added, and I can switch between audio streams using the VLC player. Yet when I transfer the video to the TV app and play it on either MacOS or iOS it only plays the original English and I see no options to play other languages.

  How do I get it to be able to switch between either audio on iOS ?

  Here is the output from ffmpeg -i out.mp4:

      Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 400x300 [SAR 1:1 DAR 4:3], q=2-31, 482 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 24k tbc (default)
  
      Metadata:
  
        handler_name    : VideoHandler
  
      Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 102 kb/s (default)
  
      Metadata:
  
        handler_name    : SoundHandler
  
      Stream #0:2: Audio: mp3 (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 152 kb/s
  
      Metadata:
  
        encoder         : LAME3.100
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (copy)
  
    Stream #0:1 -> #0:1 (copy)
  
    Stream #1:0 -> #0:2 (copy)