Recherche avancée

Sur d’autres sites (2681)

• avcodec/vc1 : correct ff_vc1_dqscale

  23 avril 2018, par Jerome Borsboom

  avcodec/vc1 : correct ff_vc1_dqscale
  According to VC-1 spec table 74, the last value in ff_vc1_dqscale should be
  
  0x1041 instead of 0x1000.
  Signed-off-by : Jerome Borsboom <jerome.borsboom@carpalis.nl>
  

  • [DH] libavcodec/vc1data.c

• AVIOContext custom stream playback glitches/corruption on UDP stream

  21 décembre 2017, par WLGfx

  I’ve written a general all round player (for Android OS) which plays file based streams as well as streams from network sources, and it all works good. File based streams I’ve got the seeking which works too.

  What it is I’m now struggling with is now I’m using AVIOContext to latch onto a UDP stream which saves the packet data partially in memory and then to transient storage. This is so I can pause live TV and also seek within it.

  However, after much faffing about, during playback (seeking is only partial at the moment), either the video frame rate will drop from 25FPS (will be deinterlaced on higher spec devices) down to between 17 and 19 frames per second, or, it will glitch and grey out.

  When I playback the record TS data from the file, it plays perfect, so the UDP buffering and writing out the overflow is sound. (This is currently not true now, only currently a minor issue)

  I’m at the point were I’ve spent a lot of time on this and I’m at a loss as to why I get either frame drops or glitches.

  The class def :

  #define PKT_SIZE (188)
  
  #define PKT_SIZE7 (PKT_SIZE * 7)
  
  
  
  #define UDP_QUEUE_SIZE (12000)
  
  #define UDP_THRESHOLD (100)
  
  
  
  #define FILE_QUEUE_PKTS (200000)
  
  
  
  #define AVIO_QUEUE_SIZE (24)
  
  #define AVIO_THRESHOLD (10)
  
  
  
  extern "C" {
  
  #include "libavformat/avio.h"
  
  };
  
  
  
  /*
  
   * PracticalSocket class as found here with examples:
  
   * http://cs.baylor.edu/~donahoo/practical/CSockets/practical/
  
   */
  
  
  
  #include "PracticalSocket.h"
  
  
  
  class FFIOBufferManager2 {
  
  
  
  public:
  
  
  
      AVIOContext *avioContext = nullptr;
  
  
  
      bool quit = false;
  
  
  
      char udp_buffer[UDP_QUEUE_SIZE][PKT_SIZE7];
  
      int udp_write_pos, udp_size; // by PKT_SIZE7
  
      char *get_udp_buffer(int index);
  
      int get_udp_buffer_size() { return udp_size; }
  
  
  
      int file_write_pos, file_size; // by PKT_SIZE7
  
      std::fstream file_out, file_in;
  
      std::mutex udp_mutex, file_mutex;
  
      std::thread udp_thread, file_thread;
  
  
  
      static void udp_thread_func(FFIOBufferManager2 *io, const char *ip, int port);
  
      static void file_thread_func(FFIOBufferManager2 *io, const char *dir);
  
      void udp_thread_run(const char *ip, int port);
  
      void file_thread_run();
  
  
  
      char avio_buffer[AVIO_QUEUE_SIZE * 7 * PKT_SIZE];
  
      int64_t avio_read_offset; // controlled by udp mutex (quickest)
  
      static int avio_read(void *ptr, uint8_t *buff, int buf_size);
  
      static int64_t avio_seek(void *ptr, int64_t pos, int whence);
  
      int avio_read_run(uint8_t *buf, int buf_size);
  
      int64_t avio_seek_run(int64_t pos, int whence);
  
      void write_udp_overflow();
  
  
  
      void start(const char *ip, int port, const char *dir);
  
  
  
      void get_size_and_pos(int64_t *size, int64_t *pos);
  
  
  
      ~FFIOBufferManager2();
  
  };

  The classes methods :

  #include 
  
  #include "FFIOBufferManager2.h"
  
  
  
  #include "LOG.h"
  
  
  
  void FFIOBufferManager2::start(const char *ip, int port, const char *dir) {
  
  
  
      file_write_pos = 0;
  
      file_size = 0;
  
  
  
      udp_write_pos = 0;
  
      udp_size = 0;
  
      avio_read_offset = 0;
  
  
  
      file_thread = std::thread(&FFIOBufferManager2::file_thread_func, this, dir);
  
      udp_thread = std::thread(&FFIOBufferManager2::udp_thread_func, this, ip, port);
  
  
  
      LOGD("Initialising avioContext");
  
  
  
      avioContext = avio_alloc_context((uint8_t*)avio_buffer,
  
                                       AVIO_QUEUE_SIZE * PKT_SIZE7,
  
                                       0,
  
                                       this,
  
                                       avio_read,
  
                                       NULL,
  
                                       avio_seek);
  
  }
  
  
  
  void FFIOBufferManager2::udp_thread_func(FFIOBufferManager2 *io, const char *ip, int port) {
  
  
  
      LOGD("AVIO UDP thread started address %s port %d - %08X", ip, port, (uint)io);
  
  
  
      io->udp_thread_run(ip, port); // run inside class
  
  
  
      LOGD("AVIO UDP thread stopped");
  
  }
  
  
  
  void FFIOBufferManager2::udp_thread_run(const char *ip, int port) {
  
  
  
      std::string addr = ip;
  
  
  
      UDPSocket socket(addr, (uint16_t)port);
  
      socket.joinGroup(addr);
  
  
  
      LOGD("UDP loop starting");
  
  
  
      while (!quit) {
  
  
  
          if (socket.recv(get_udp_buffer(udp_write_pos), PKT_SIZE7) == PKT_SIZE7) {
  
  
  
              udp_mutex.lock();
  
  
  
              udp_write_pos = (udp_write_pos + 1) % UDP_QUEUE_SIZE;
  
              udp_size++;
  
              if (udp_size >= UDP_QUEUE_SIZE) udp_size--;
  
              else avio_read_offset += PKT_SIZE7;
  
  
  
              udp_mutex.unlock();
  
          }
  
      }
  
  }
  
  
  
  void FFIOBufferManager2::file_thread_func(FFIOBufferManager2 *io, const char *dir) {
  
  
  
      LOGD("AVIO FILE thread started");
  
  
  
      std::string file = dir;
  
      const char *tsfile_name = "/tsdata.ts";
  
      file += tsfile_name;
  
  
  
      LOGD("Deleting old file %s", file.c_str());
  
  
  
      remove(file.c_str());
  
  
  
      {
  
          fstream temp; // create the ts file
  
          temp.open(file.c_str());
  
          temp.close();
  
      }
  
  
  
      LOGD("Opening %s for read and write", file.c_str());
  
  
  
      io->file_out.open(file, fstream::out | fstream::binary);
  
      io->file_in.open(file, fstream::in | fstream::binary);
  
  
  
      io->file_thread_run(); // continue inside the class to lessen pointer use
  
  
  
      LOGD("AVIO FILE thread stopped");
  
  }
  
  
  
  void FFIOBufferManager2::file_thread_run() {
  
  
  
      LOGD("FILE thread run");
  
  
  
      if (!file_out.is_open() || !file_in.is_open()) {
  
  
  
          LOGE("TS data file, error opening...");
  
          quit = true;
  
          return;
  
      }
  
  
  
      int udp_threshold = UDP_QUEUE_SIZE - (UDP_THRESHOLD * 4);
  
  
  
      while (!quit) {
  
  
  
          if (udp_size >= udp_threshold) write_udp_overflow();
  
          else usleep(1000 * 1);
  
      }
  
  }
  
  
  
  void FFIOBufferManager2::write_udp_overflow() {
  
  
  
      file_mutex.lock();
  
  
  
      udp_mutex.lock();
  
      int udp_write_pos_current = udp_write_pos;
  
      int udp_size_current = udp_size;
  
      udp_mutex.unlock();
  
  
  
      int udp_index = udp_write_pos_current - udp_size_current;
  
      if (udp_index < 0) udp_index += UDP_QUEUE_SIZE;
  
      int written = 0;
  
  
  
      //file_out.seekp((int64_t)file_write_pos * PKT_SIZE7);
  
  
  
      while (written < UDP_THRESHOLD) {
  
          file_out.write(get_udp_buffer(udp_index), PKT_SIZE7);
  
          written++;
  
          udp_index = (udp_index + 1) % UDP_QUEUE_SIZE;
  
  
  
          file_write_pos++;
  
          if (file_write_pos >= FILE_QUEUE_PKTS) {
  
              file_write_pos = 0;
  
              file_out.seekp(0);
  
          }
  
          file_size++;
  
          if (file_size > FILE_QUEUE_PKTS) file_size = FILE_QUEUE_PKTS;
  
      }
  
  
  
      udp_mutex.lock();
  
      udp_size -= UDP_THRESHOLD; // we've written this amount out
  
      udp_mutex.unlock();
  
  
  
      //file_out.flush();
  
  
  
      file_mutex.unlock();
  
  
  
      //LOGD("Written UDP overflow at %d of %d blocks file size %d",
  
      //     udp_index, written, file_size);
  
  }
  
  
  
  char *FFIOBufferManager2::get_udp_buffer(int index) {
  
      if (index < 0 || index >= UDP_QUEUE_SIZE) return nullptr;
  
      return ((char*)udp_buffer + (index * PKT_SIZE7));
  
  }
  
  
  
  /*
  
   * The avio_read and avio_seek now work on either 188 byte alignment or
  
   * byte alignment for the benefit of ffmpeg - byte positioning at the moment
  
   *
  
   * The file_mutex allows for either a read or write operation at a time
  
   */
  
  
  
  int FFIOBufferManager2::avio_read(void *ptr, uint8_t *buff, int buf_size) {
  
      FFIOBufferManager2 *io = (FFIOBufferManager2*)ptr;
  
      return io->avio_read_run(buff, buf_size);
  
  }
  
  
  
  int64_t FFIOBufferManager2::avio_seek(void *ptr, int64_t pos, int whence) {
  
      FFIOBufferManager2 *io = (FFIOBufferManager2*)ptr;
  
      return io->avio_seek_run(pos, whence);
  
  }
  
  
  
  int FFIOBufferManager2::avio_read_run(uint8_t *buf, int buf_size) {
  
  
  
      file_mutex.lock();
  
      udp_mutex.lock();
  
  
  
      int64_t cur_udp_write_pos = (int64_t) udp_write_pos * PKT_SIZE7;
  
      int64_t cur_udp_size = (int64_t) udp_size * PKT_SIZE7;
  
  
  
      int64_t cur_file_write_pos = (int64_t) file_write_pos * PKT_SIZE7;
  
      int64_t cur_file_size = (int64_t) file_size * PKT_SIZE7;
  
  
  
      int64_t cur_avio_read_offset = avio_read_offset; // already int64_t (under the udp_mutex)
  
  
  
      udp_mutex.unlock();
  
  
  
      if (cur_avio_read_offset < (AVIO_THRESHOLD * 4) * PKT_SIZE7) {
  
          file_mutex.unlock();
  
          return 0;
  
      }
  
  
  
      int64_t udp_buffer_max = (int64_t) (UDP_QUEUE_SIZE * PKT_SIZE7);
  
      int64_t file_buffer_max = (int64_t) (FILE_QUEUE_PKTS * PKT_SIZE7);
  
      uint8_t *ptr_udp_buffer = (uint8_t*)udp_buffer;
  
      int cur_written = 0;
  
  
  
      int file_reads = 0, udp_reads = 0; // for debugging
  
  
  
      int64_t cur_file_offset = cur_file_write_pos - cur_udp_size - cur_avio_read_offset;
  
      while (cur_file_offset < 0) cur_file_offset += file_buffer_max;
  
  
  
      if (cur_file_offset >= 0) {
  
          file_in.seekg(cur_file_offset);
  
  
  
          while (//cur_avio_read_offset > 0
  
                 cur_avio_read_offset > cur_udp_size
  
                 && cur_written < buf_size) { // read from file first
  
  
  
              file_in.read(&avio_buffer[cur_written], PKT_SIZE); // get 1 or 188 byte/s
  
  
  
              cur_file_offset+=PKT_SIZE;
  
              if (cur_file_offset >= file_buffer_max) { // back to file beginning
  
                  cur_file_offset = 0;
  
                  file_in.seekg(0);
  
              }
  
  
  
              cur_avio_read_offset-=PKT_SIZE;
  
  
  
              cur_written+=PKT_SIZE;
  
              file_reads+=PKT_SIZE;
  
          }
  
      }
  
  
  
      int64_t cur_udp_offset = (cur_udp_write_pos - cur_avio_read_offset);
  
      if (cur_udp_offset < 0) cur_udp_offset += udp_buffer_max;
  
  
  
      while (cur_avio_read_offset > AVIO_THRESHOLD * PKT_SIZE7
  
              && cur_avio_read_offset <= cur_udp_size
  
              && cur_written < buf_size) { // read the rest from udp buffer
  
  
  
          buf[cur_written] = ptr_udp_buffer[cur_udp_offset]; // get byte
  
  
  
          cur_udp_offset = (cur_udp_offset + 1) % udp_buffer_max;
  
          if (cur_udp_offset == 0) LOGD("AVIO UDP BUFFER to start");
  
  
  
          cur_avio_read_offset--;
  
  
  
          cur_written++;
  
          udp_reads++;
  
      }
  
  
  
      udp_mutex.lock();
  
      avio_read_offset -= cur_written;
  
      udp_mutex.unlock();
  
  
  
      file_mutex.unlock();
  
  
  
      if (cur_written) {
  
          LOGD("AVIO_READ: Written %d of %d, avio_offset %lld, file reads %d, udp reads %d, udp offset %lld, file offset %lld, file size %lld",
  
               cur_written, buf_size,
  
               cur_avio_read_offset,
  
               file_reads, udp_reads,
  
               cur_udp_write_pos, cur_file_write_pos, cur_file_size);
  
      }
  
  
  
      return cur_written;
  
  }
  
  
  
  int64_t FFIOBufferManager2::avio_seek_run(int64_t pos, int whence) {
  
      // SEEK_SET(0), SEEK_CUR(1), SEEK_END(2), AVSEEK_SIZE
  
  
  
      int64_t new_pos = -1;
  
      int64_t full_length = (udp_size + file_size) * PKT_SIZE7;
  
  
  
      switch (whence) {
  
  
  
          case AVSEEK_SIZE:
  
              LOGD("AVSEEK_SIZE pos %lld", pos);
  
              break;
  
  
  
          case SEEK_SET:
  
              LOGD("AVSEEK_SET pos %lld", pos);
  
              if (pos > full_length) new_pos = full_length;
  
              else new_pos = full_length - pos;
  
              break;
  
  
  
          case SEEK_CUR:
  
              LOGD("AVSEEK_CUR pos %lld", pos);
  
              break;
  
  
  
          case SEEK_END:
  
              LOGD("AVSEEK_END pos %lld", pos);
  
              new_pos = pos;
  
              break;
  
  
  
          default:
  
              LOGD("UNKNOWN AVIO SEEK whence %d pos %lld", whence, pos);
  
              break;
  
      }
  
  
  
      if (new_pos >= 0) {
  
  
  
          udp_mutex.lock();
  
  
  
          new_pos = (new_pos / PKT_SIZE) * PKT_SIZE; // align to packet boundary
  
          avio_read_offset = new_pos;
  
          //file_out.seekg(full_length - new_pos);
  
  
  
          udp_mutex.unlock();
  
  
  
          return full_length - new_pos;
  
      }
  
  
  
      return -1;
  
  }
  
  
  
  FFIOBufferManager2::~FFIOBufferManager2() {
  
      if (avioContext) ;// TODO whoops
  
      quit = true;
  
      if (udp_thread.joinable()) udp_thread.join();
  
      if (file_thread.joinable()) file_thread.join();
  
  }
  
  
  
  void FFIOBufferManager2::get_size_and_pos(int64_t *size, int64_t *pos) {
  
      file_mutex.lock();
  
      udp_mutex.lock();
  
  
  
      *size = (udp_size + file_size) * PKT_SIZE7;
  
      *pos = *size - avio_read_offset;
  
  
  
      udp_mutex.unlock();
  
      file_mutex.unlock();
  
  }

  It’ll play for a few seconds before any of the glitches start to appear. I have checked against the udp_buffer and the avio_buffer, but my suspicions lie with one of two things :

  1. Reading and writing to the file.
  2. the avio_read method is wrong.

  Has anybody got any input as to why this is occurring ? Any thoughts would be greatly appreciated.

  If you need any more information I’ll be glad to provide more details.

  EDIT : Seeking now actually moves to any point within the stream, but now doesn’t read from the file recording. Although that’s only a minor issue at the moment.

  The main two issues still stand, frame rate drops dramatically and the glitches after approximately 8 seconds.

• Progress with rtc.io

  12 août 2014, par silvia

  At the end of July, I gave a presentation about WebRTC and rtc.io at the WDCNZ Web Dev Conference in beautiful Wellington, NZ.

  

  Putting that talk together reminded me about how far we have come in the last year both with the progress of WebRTC, its standards and browser implementations, as well as with our own small team at NICTA and our rtc.io WebRTC toolbox.

  

  One of the most exciting opportunities is still under-exploited : the data channel. When I talked about the above slide and pointed out Bananabread, PeerCDN, Copay, PubNub and also later WebTorrent, that’s where I really started to get Web Developers excited about WebRTC. They can totally see the shift in paradigm to peer-to-peer applications away from the Server-based architecture of the current Web.

  Many were also excited to learn more about rtc.io, our own npm nodules based approach to a JavaScript API for WebRTC.

  

  We believe that the World of JavaScript has reached a critical stage where we can no longer code by copy-and-paste of JavaScript snippets from all over the Web universe. We need a more structured module reuse approach to JavaScript. Node with JavaScript on the back end really only motivated this development. However, we’ve needed it for a long time on the front end, too. One big library (jquery anyone ?) that does everything that anyone could ever need on the front-end isn’t going to work any longer with the amount of functionality that we now expect Web applications to support. Just look at the insane growth of npm compared to other module collections :

  

  For those that – like myself – found it difficult to understand how to tap into the sheer power of npm modules as a font end developer, simply use browserify. npm modules are prepared following the CommonJS module definition spec. Browserify works natively with that and “compiles” all the dependencies of a npm modules into a single bundle.js file that you can use on the front end through a script tag as you would in plain HTML. You can learn more about browserify and module definitions and how to use browserify.

  For those of you not quite ready to dive in with browserify we have prepared prepared the rtc module, which exposes the most commonly used packages of rtc.io through an “RTC” object from a browserified JavaScript file. You can also directly download the JavaScript file from GitHub.

  

  So, I hope you enjoy rtc.io and I hope you enjoy my slides and large collection of interesting links inside the deck, and of course : enjoy WebRTC ! Thanks to Damon, JEeff, Cathy, Pete and Nathan – you’re an awesome team !

  On a side note, I was really excited to meet the author of browserify, James Halliday (@substack) at WDCNZ, whose talk on “building your own tools” seemed to take me back to the times where everything was done on the command-line. I think James is using Node and the Web in a way that would appeal to a Linux Kernel developer. Fascinating !!

  The post Progress with rtc.io first appeared on ginger’s thoughts.