Recherche avancée

Sur d’autres sites (11836)

• How to Skip frames while decoding H264 stream ?

  16 septembre 2018, par TTGroup

  I’m using FFMPEG to decode H264 (or H265) RTSP Stream.

  My system have 2 software : Server and Client

  Server: Read frames from RTSP stream --> Forward frames to Client    
  
  Client: Receive frames from Server --> Decode --> Render

  I have implemented and it worked ok, but there is a case make my system work not good. That is when internet from Server - Client is slow, frames can not transfer real-time to Client.

  In present, I deal with this issue by Skip some frames (not send to Client) when the Queue is reached limit of count. The following is my summary code

  //At Server Software (include 2 threads A and B)
  
  //Thread A: Read AVPacket and forward to Client
  
  while(true)
  
  {
  
      AVPacket packet;
  
      av_init_packet(&packet);
  
      packet.size = 0;
  
      packet.data = NULL;
  
      int ret = AVERROR(EAGAIN);
  
      while (AVERROR(EAGAIN) == ret)
  
          ret = av_read_frame(pFormatCtx, &packet);
  
      if(packet.size > 0)
  
      {
  
          if(mySendQueue.count < 120) //limit 120 packet in queue
  
              mySendQueue.Enqueue(packet); ////Thread B will read from this queue, to send packets to Client via TCP socket
  
          else
  
              ;//SkipThisFrame ***: No send
  
      }
  
  }
  
  //Thread B: Send To Client via TCP Socket
  
  While(true)
  
  {
  
      AVPacket packet;
  
      if(mySendQueue.Dequeue(packet))
  
      {
  
          SendPacketToClient(packet);
  
      }
  
  }
  
  
  
  //At Server Software : Receive AVPacket from Server --> Decode --> Render
  
  While(true)
  
  {
  
      AVPacket packet;
  
      AVFrame frame;
  
      ReadPacketFromServer(packet);
  
      if (av_decode_asyn(pCodecCtx, &frame, &frameFinished, &packet) == RS_OK)
  
      {
  
          if (frameFinished)
  
          {
  
              RenderFrame(frame);
  
          }
  
      }           
  
  }
  
  UINT32 __clrcall av_decode_asyn(AVCodecContext *pCodecCtx, AVFrame *frame, int *frameFinished, AVPacket *packet)
  
  {
  
      int ret = -1;
  
      *frameFinished = 0;
  
      if (packet) 
  
      {
  
          ret = avcodec_send_packet(pCodecCtx, packet);
  
          // In particular, we don't expect AVERROR(EAGAIN), because we read all
  
          // decoded frames with avcodec_receive_frame() until done.
  
          if (ret < 0 && ret != AVERROR_EOF)
  
              return RS_NOT_OK;
  
      }
  
  
  
      ret = avcodec_receive_frame(pCodecCtx, frame);
  
      if (ret < 0 && ret != AVERROR(EAGAIN))
  
      {
  
          return RS_NOT_OK;
  
      }
  
      if (ret >= 0)
  
          *frameFinished = 1;
  
  
  
      return RS_OK;
  
  }

  My question is focus in line of code SkipThisFrame ***, this algorithm skip frame continuously, so it maybe make the decoder on Client occur unexpectedly error or Crash ?

  And when skip frame like that, make Client Render frames is not normally ?

  And someone call show me the proper algorithm to skip frames in my case ?

  Thank you very much !

• How to seek mp4 aac audio using Media Source Extensions

  29 août 2018, par Chris

  Please can someone offer me a few pointers when trying to seek within streamed aac audio in mp4 containers ? I’m trying to develop a music download service that sips data via ranged requests rather than simply link to a mp4 file as an <audio></audio> src. (which will instead buffer the whole file as quickly as possible, and so be rather wasteful and expensive).

  So far I’ve managed to successfully append sequential audio range buffers to the SourceBuffer object using partial/ranged requests, attached to my suitably mime-typed MediaSource object. But as soon as I try to seek, the wheels come off and I receive a ’CHUNK_DEMUXER_ERROR_APPEND_FAILED’ error, with the specific issue : ’stream parsing failed’.

  I’ve prepared my mp4 files by encoding them with ffmpeg (via the fluent ffmpeg module), rewriting the movie header box at the start of the file (via the -movflags faststart setting) so that the duration can be parsed. I then fragment the file with mp4fragment (part of the Bento4 tools) with the default settings, and check to ensure the structure of the file matches the format specified by ISO BMFF, with pairs of movie fragments and data boxes (moof/mdat) describing the audio stream. Given the source buffer has no problem playing from the beginning, with contiguous subsequent ranges, this appears to confirm that the format of the mp4 file is acceptable.

  As an aside, I’ve tried fragmenting the file completely in ffmpeg/fluent ffmpeg (using the ’-movflags empty_moov+default_base_moof’ options), but while this works, it also removes the duration from the moov as you’d expect, so the file gets larger during playback as more fragments are fetched and appended. If I set the file duration manually, I still have the issue of not being able to seek to unbuffered audio, so I only seem to be making life more difficult trying to fragment the file solely in ffmpeg.

  So how should I go about seeking within the stream ? I gather that seeking effectively ’needle-drops’ randomly, and so the source buffer might struggle to parse the data out of context, but I imagined that it would skip to the next available fragment in the range that I fetch (which is calculated using the percentage of the seek bar width to set the player.currentTime, which is then converted to a suitable byte range using the 128kbps CBR figure to convert seconds to bytes, to send a 206 partial range request).

  I’ve seen mention of buffer offsets, but I don’t understand how these apply. Most of the dev examples I’ve seen just focus on whole files or segmented videos, rather than fragmented single audio files for seeking ? Do I need to somehow retain a portion of the data from the moov box when seeking for the source buffer to be able to parse it ? In the trun box I have a data offset that varies between two values throughout the file, 444 and 448, depending on whether the sample count is 86 or 87. I’m not sure why it’s not consistent.

  Here’s what the moov looks like from my audio file :

  [ftyp] size=8+24
  
    major_brand = isom
  
    minor_version = 200
  
    compatible_brand = isom
  
    compatible_brand = iso2
  
    compatible_brand = mp41
  
    compatible_brand = iso5
  
  [moov] size=8+620
  
    [mvhd] size=12+96
  
      timescale = 1000
  
      duration = 350047
  
      duration(ms) = 350047
  
    [trak] size=8+448
  
      [tkhd] size=12+80, flags=7
  
        enabled = 1
  
        id = 1
  
        duration = 350047
  
        width = 0.000000
  
        height = 0.000000
  
      [edts] size=8+28
  
        [elst] size=12+16
  
          entry count = 1
  
          entry/segment duration = 350000
  
          entry/media time = 2048
  
          entry/media rate = 1
  
      [mdia] size=8+312
  
        [mdhd] size=12+20
  
          timescale = 44100
  
          duration = 0
  
          duration(ms) = 0
  
          language = und
  
        [hdlr] size=12+41
  
          handler_type = soun
  
          handler_name = Bento4 Sound Handler
  
        [minf] size=8+219
  
          [smhd] size=12+4
  
            balance = 0
  
          [dinf] size=8+28
  
            [dref] size=12+16
  
              [url ] size=12+0, flags=1
  
                location = [local to file]
  
          [stbl] size=8+159
  
            [stsd] size=12+79
  
              entry-count = 1
  
              [mp4a] size=8+67
  
                data_reference_index = 1
  
                channel_count = 2
  
                sample_size = 16
  
                sample_rate = 44100
  
                [esds] size=12+27
  
                  [ESDescriptor] size=2+25
  
                    es_id = 0
  
                    stream_priority = 0
  
                    [DecoderConfig] size=2+17
  
                      stream_type = 5
  
                      object_type = 64
  
                      up_stream = 0
  
                      buffer_size = 0
  
                      max_bitrate = 128006
  
                      avg_bitrate = 128006
  
                      DecoderSpecificInfo = 12 10 
  
                    [Descriptor:06] size=2+1
  
            [stts] size=12+4
  
              entry_count = 0
  
            [stsc] size=12+4
  
              entry_count = 0
  
            [stsz] size=12+8
  
              sample_size = 0
  
              sample_count = 0
  
            [stco] size=12+4
  
              entry_count = 0
  
    [mvex] size=8+48
  
      [mehd] size=12+4
  
        duration = 350047
  
      [trex] size=12+20
  
        track id = 1
  
        default sample description index = 1
  
        default sample duration = 0
  
        default sample size = 0
  
        default sample flags = 0

  And here’s a typical fragment :

  [moof] size=8+428
  
    [mfhd] size=12+4
  
      sequence number = 1
  
    [traf] size=8+404
  
      [tfhd] size=12+8, flags=20008
  
        track ID = 1
  
        default sample duration = 1024
  
      [tfdt] size=12+8, version=1
  
        base media decode time = 0
  
      [trun] size=12+352, flags=201
  
        sample count = 86
  
        data offset = 444
  
  [mdat] size=8+32653

  Does that all look good ? Any pointers for seeking within such a file would be hugely appreciated. Thanks !

• YTDL-Core MP4 to MP3 doesn't work with FFMPEG (Code 101)

  14 septembre 2018, par doamatto

  In a simple YouTube downloader, I keep trying to get an MP4 to convert to an MP3 using FFMPEG/Fluent-FFMPEG (Fluent was used as recommened by the ytdl-core examples). However, both times resolve in one of two errors : Cannot find ffmpeg or code: 101, msg: "The input file path must be a string"

  I have updated the source to all be on GitHub (sorry for that slight inconvenience as opposed to it being here. It’s linked below. However, I have tried two things : using the normal "node-ffmpeg" library as the require for "ffmpeg", and using the "fluent-ffmpeg" library as the require for "ffmpeg." I mainly did this because via the ytdl-core uses such and I figured it may work in that case.

  Thanks in advance and thanks as a whole !

  The GitHub Repo : https://gist.github.com/Incrested/d9ef8125bd41afbb7e6720ec3a78e331

  Once again : thanks for any and all assistance !

  Edit : I’ve changed the URL to a better snippet of focus of where I think the issue is.