Recherche avancée

Sur d’autres sites (6754)

• RoQ on Dreamcast

  18 mars 2011, par Multimedia Mike — Sega Dreamcast

  I have been working on that challenge to play back video on the Sega Dreamcast. To review, I asserted that the RoQ format would be a good fit for the Sega Dreamcast hardware. The goal was to play 640x480 video at 30 frames/second. Short version : I have determined that it is possible to decode such video in real time. However, I ran into certain data rate caveats.

  First off : Have you ever wondered if the Dreamcast can read an 80mm optical disc ? It can ! I discovered this when I only had 60 MB of RoQ samples to burn on a disc and a spindle full of these 210MB-capacity 80mm CD-Rs that I never have occasion to use.

  
  
  

  New RoQ Library
  There are open source RoQ decoders out there but I decided to write a new one. A few reasons : 1) RoQ is so simple that I didn’t think it would take too long ; 2) it would be nice to have a RoQ library that is license-compatible (BSD-like) with the rest of the KallistiOS distribution ; 3) the idroq.tar.gz distribution, while license-compatible, has enough issues that I didn’t want to correct it.

  Thankfully, I was correct about the task not being too difficult : I put together a new RoQ decoder in short order. I’m a bit embarrassed to admit that the part I had the most trouble with was properly converting YUV -> RGB.
  
  About the approach I took : While the original idroq.tar.gz decoder maintains YUV 4:2:0 codebooks (which led to chroma bugs during motion compensation) and FFmpeg’s decoder maintains YUV 4:4:4 codebooks, this decoder is built to convert the YUV 4:2:0 vectors into RGB565 vectors during the vector unpacking phase. Thus, the entire frame is rendered in RGB565 — no lengthy YUV -> RGB conversion after decoding — and all pixels are shuffled around as 16-bit units (minor speedup vs. shuffling everything as bytes).

  I also entertained the idea of maintaining YUYV codebooks (since the DC supports that colorspace as a texture format). But I scrapped that idea when I remembered it would lead to the same chroma bleeding problem seen in the original idroq.tar.gz decoder.

  Onto The Dreamcast
  I developed the library on a Linux computer, allowing it to output a series of PNM files for visual verification and debugging. Dropping it into a basic DC/KOS-compatible program was trivial and the first order of business was profiling.

  At first, I profiled the entire decode operation : open file, then read and decode each chunk while tossing away the results. I was roundly disappointed to see that, e.g., an 8.5-second RoQ sample needed a little more than 20 seconds to complete. Not real time. I performed a series of optimizations on the decoding library that netted notable performance gains when profiling on Linux. When I brought these same optimizations over to the DC, decoding time didn’t improve at all. This was my first suspicion that perhaps my assumptions regarding the DC’s optical drive’s data rate were not correct.

  Dreamcast Data Rate Profiling
  Let’s start with some definitions : In terms of data rate, an ’X’, i.e., 1X is the minimum data rate needed to read CD quality audio from a disc. At that speed, a drive should be able to stream 75 sectors each second. When reading mode 1/form 1 CD-ROM data, each sector has 2048 bytes (2 kbytes), so a single-speed data rate should achieve 150 kbytes/sec.

  The Dreamcast is supposed to possess a 12X optical drive. This would imply a maximum data rate of 150 kbytes/sec * 12 = 1800 kbytes/sec.

  Rigging up a trivial experiment using the RoQ samples burned on a few different CD-R discs, the best data rate I can see is about 500-525 kbytes/sec, or around 3.5X.

  Where’s the discrepancy ? My first theory has to do with the fact that not all optical media is created equal. This is why optical drives often advertise a slew of numbers which refer to the best theoretical speed for reading a CD vs. writing a CD-R vs. writing a CD-RW, etc. Perhaps the DC drive can’t read CD-Rs very quickly. To test this theory, I tried streaming a large file from a conventionally mastered CD-ROM. This worked well for the closest CD-ROM I had on hand : I was able to stream data at a rate that works out to about 6.5X.

  I smell a science project for another evening : Profiling read speeds from a mastered CD-ROM, burned CD-R, and also a mastered GD-ROM, on each of the 3 Dreamcast consoles I possess (I’ve heard that there’s variance between optical drives depending on manufacturing run).

  The Good News
  I added a little finer-grained code to profile just the video decoding functions. The good news is that the decoder meets my real time goals : That 8.5-second RoQ sample encoded at 640x480x30fps makes its way through the video decoding functions on the DC in a little less than 5 seconds. If the optical drive can supply the data fast enough, the video decoder can take care of the rest.

  The RoQ encoder included with FFmpeg does not honor any bitrate parameters. Instead, I encoded the same file at 320x240. It reportedly decoded in real time and can be streamed in real time as well.

  I say "reportedly" because I’m simply working from textual output at this point ; the next phase is to hook the decoder up to the display hardware.

• Exit Status with ffmpeg Command in Rails

  25 novembre 2012, par DragonFire353

  I am trying to get uploaded videos to be converted in the background, running windows. I am using :

  gem 'paperclip'
  
  gem 'aasm'
  
  gem 'delayed_job_active_record'
  
  gem 'ffmpeg'

  I was using purely paperclip before and making the user wait and it worked great, now I am having problems with the return of the error status for the command, I have tried editing to possible fix the command wondering if it was failing in the first place but I keep getting :

  undefined method `exitstatus' for nil:NilClass

  no matter what. I've tried looking this up and it's supposedly valid syntax that should work... Also I've commented out the actual spawn do part because I get another error if I leave that in :

  wrong number of arguments

  Does anyone know how to properly get this working ? I've went through a few tutorials that have bits and pieces of what I need but I can't get them working together. Here's what I have so far, lemme know if you need more :

  Model :

      class Video < ActiveRecord::Base
  
    include AASM
  
  
  
    belongs_to :user
  
    has_many :comments, dependent: :destroy
  
    attr_accessible :video, :user_id, :video_file_name, :title, :public, :description, :views
  
  
  
    has_attached_file :video, url: "/users/:user_id/videos/:id/:basename_:style.:extension"
  
    #, :styles => { 
  
       # :video => { geometry: "800x480>", format: 'webm' },
  
       # :thumb => { geometry: "200x200>", format: 'png', time: 3 },
  
     # }, processors: [:ffmpeg], url: "/users/:user_id/videos/:id/:basename_:style.:extension"
  
  
  
    #process_in_background :video #causes death
  
  
  
    validates :video, presence: true
  
    validates :description, presence: true, length: { minimum: 5, maximum: 100}
  
    validates :title, presence: true, length: { minimum: 1, maximum: 15 }
  
  
  
    validates_attachment_size :video, less_than: 1.gigabytes
  
    validates_attachment :video, presence: true
  
  
  
    default_scope order: 'created_at DESC'
  
  
  
    Paperclip.interpolates :user_id do |attachment, style|attachment.instance.user_id
  
    end
  
  
  
    #acts as state machine plugin
  
    aasm state: :pending do
  
     state :pending, initial: true
  
     state :converting
  
     state :converted
  
     #, enter: :set_new_filename
  
     state :error
  
  
  
      event :convert do
  
       transitions from: :pending, to: :converting
  
     end
  
  
  
      event :converted do
  
       transitions from: :converting, to: :converted
  
     end
  
  
  
      event :failure do
  
        transitions from: :converting, to: :error
  
      end
  
    end
  
  
  
     # This method is called from the controller and takes care of the converting
  
    def convert
  
      self.convert!
  
  
  
      #spawn a new thread to handle conversion
  
      #spawn do
  
        success = delay.system(convert_command)
  
        logger.debug 'Converting File: ' + success.to_s
  
        if success && $?.exitstatus.to_i == 0
  
          self.converted!
  
        else
  
          self.failure!
  
        end
  
      #end
  
    end
  
  
  
    def self.search(search)
  
      if search
  
        find(:all, conditions: ["public = 't' AND title LIKE ?", "%#{search}%"], order: "created_at DESC")
  
      else
  
        find(:all, conditions: ["public = 't'"], order: "created_at DESC")
  
      end
  
    end
  
  
  
    def self.admin_search(search)
  
      if search
  
        find(:all, conditions: ['title LIKE ?', "%#{search}%"], order: "created_at DESC")
  
      else
  
        find(:all, order: "created_at DESC")
  
      end
  
    end
  
  
  
    private
  
      def convert_command
  
        #construct new file extension
  
        webm =  "." + id.to_s + ".webm"
  
  
  
        #build the command to execute ffmpeg
  
        command = <<-end_command
  
          ffmpeg -i #{ RAILS_ROOT + '/public/users/:user_id/videos/:id/:basename_:style.:extension' }  -ar 22050 -ab 32 -s 1280x720 -vcodec webm -r 25 -qscale 8 -f webm -y #{ RAILS_ROOT + '/public/users/:user_id/videos/:id/:basename_.webm' }
  
  
  
        end_command
  
  
  
        logger.debug "Converting video...command: " + command
  
        command
  
      end
  
  
  
      handle_asynchronously :convert_command
  
  
  
      # This updates the stored filename with the new flash video file
  
      def set_new_filename
  
        #update_attribute(:filename, "#{filename}.#{id}.webm")
  
        update_attribute(:content_type, "video/x-webm")
  
      end
  
  
  
  end

  Controller :

  class VideosController < ApplicationController
  
  before_filter :signed_in_user, only: [:upload, :update, :destroy]
  
  before_filter :admin_user, only: :admin_index
  
  
  
  def upload
  
      @video = Video.new
  
      # generate a unique id for the upload
  
      @uuid = (0..29).to_a.map {|x| rand(10)}
  
  end
  
  
  
  def create
  
      @video = Video.new(params[:video])
  
      @video.user_id = current_user.id
  
  
  
      if @video.save
  
          @video.convert
  
          flash[:success] = "Uploaded Succefully!"
  
          redirect_to @video.user
  
      else
  
          render 'upload'
  
      end
  
  end
  
  
  
  def show
  
      @video = Video.find(params[:id])
  
      @comments = @video.comments.paginate(page: params[:page], per_page: 6)
  
      if !@video.public
  
          if !signed_in? || current_user.id != @video.user_id  && !current_user.admin && !current_user.approved?(@video.user)
  
          flash[:notice] = "Video is private"
  
          redirect_to root_path
  
      end
  
  end
  
  end
  
  
  
  def update
  
      @video = Video.find(params[:id])
  
      if @video.update_attributes(params[:video])
  
    flash[:success] = "Video preferences saved"
  
  else
  
      flash[:fail] = "Failed to update video preferences"
  
  end
  
  redirect_to :back
  
  end
  
  
  
  def destroy
  
      @video = Video.find(params[:id])
  
      @video.destroy
  
      flash[:deleted] = "Deleted Succefully!"
  
      redirect_to :back
  
  end
  
  
  
  def index
  
      @videos = Video.paginate(page: params[:page], per_page: 6).search(params[:search])
  
  end
  
  
  
  def admin_index
  
      @videos = Video.paginate(page: params[:page], per_page: 6).admin_search(params[:search])
  
  end
  
  
  
  def ajax_video_comments
  
      @video = Video.find(params[:id])
  
      @comments = @video.comments.paginate(page: params[:page], per_page: 6)
  
  
  
      respond_to do |format|
  
      format.js   { render partial: 'shared/comments', content_type: 'text/html' }
  
  end
  
  end
  
  
  
  def ajax_video_watched
  
      @video = Video.find(params[:id])
  
      @video.views += 1
  
      @video.save
  
  end
  
  
  
  private
  
  
  
  def signed_in_user
  
      redirect_to root_path, notice: "Please Login." unless signed_in?
  
  end
  
  
  
  def admin_user
  
      redirect_to(root_path) unless current_user.admin?
  
  end
  
  
  
  end

• Muxing with libav

  14 février 2014, par LordDoskias

  I have a program which is supposed to demux input mpeg-ts, transcode the mpeg2 into h264 and then mux the audio alongside the transcoded video. When I open the resulting muxed file with VLC I neither get audio nor video. Here is the relevant code.

  My main worker loop is as follows :

  void
  
  *writer_thread(void *thread_ctx) {
  
  
  
      struct transcoder_ctx_t *ctx = (struct transcoder_ctx_t *) thread_ctx;
  
      AVStream *video_stream = NULL, *audio_stream = NULL;
  
      AVFormatContext *output_context = init_output_context(ctx, &video_stream, &audio_stream);
  
      struct mux_state_t mux_state = {0};
  
  
  
      //from omxtx
  
      mux_state.pts_offset = av_rescale_q(ctx->input_context->start_time, AV_TIME_BASE_Q, output_context->streams[ctx->video_stream_index]->time_base);
  
  
  
      //write stream header if any
  
      avformat_write_header(output_context, NULL);
  
  
  
      //do not start doing anything until we get an encoded packet
  
      pthread_mutex_lock(&ctx->pipeline.video_encode.is_running_mutex);
  
      while (!ctx->pipeline.video_encode.is_running) {
  
          pthread_cond_wait(&ctx->pipeline.video_encode.is_running_cv, &ctx->pipeline.video_encode.is_running_mutex);
  
      }
  
  
  
      while (!ctx->pipeline.video_encode.eos || !ctx->processed_audio_queue->queue_finished) {
  
          //FIXME a memory barrier is required here so that we don't race 
  
          //on above variables 
  
  
  
          //fill a buffer with video data 
  
          OERR(OMX_FillThisBuffer(ctx->pipeline.video_encode.h, omx_get_next_output_buffer(&ctx->pipeline.video_encode)));
  
  
  
          write_audio_frame(output_context, audio_stream, ctx); //write full audio frame 
  
          //FIXME no guarantee that we have a full frame per packet?
  
          write_video_frame(output_context, video_stream, ctx, &mux_state); //write full video frame
  
          //encoded_video_queue is being filled by the previous command
  
  
  
      }
  
  
  
      av_write_trailer(output_context);
  
  
  
      //free all the resources
  
      avcodec_close(video_stream->codec);
  
      avcodec_close(audio_stream->codec);
  
      /* Free the streams. */
  
      for (int i = 0; i < output_context->nb_streams; i++) {
  
          av_freep(&output_context->streams[i]->codec);
  
          av_freep(&output_context->streams[i]);
  
      }
  
  
  
      if (!(output_context->oformat->flags & AVFMT_NOFILE)) {
  
          /* Close the output file. */
  
          avio_close(output_context->pb);
  
      }
  
  
  
  
  
      /* free the stream */
  
      av_free(output_context);
  
      free(mux_state.pps);
  
      free(mux_state.sps);
  
  }

  The code for initialising libav output context is this :

  static 
  
  AVFormatContext *
  
  init_output_context(const struct transcoder_ctx_t *ctx, AVStream **video_stream, AVStream **audio_stream) {
  
      AVFormatContext *oc;
  
      AVOutputFormat *fmt;
  
      AVStream *input_stream, *output_stream;
  
      AVCodec *c;
  
      AVCodecContext *cc;
  
      int audio_copied = 0; //copy just 1 stream
  
  
  
      fmt = av_guess_format("mpegts", NULL, NULL);
  
      if (!fmt) {
  
          fprintf(stderr, "[DEBUG] Error guessing format, dying\n");
  
          exit(199);
  
      }
  
  
  
      oc = avformat_alloc_context();
  
      if (!oc) {
  
          fprintf(stderr, "[DEBUG] Error allocating context, dying\n");
  
          exit(200);
  
      }
  
  
  
      oc->oformat = fmt;
  
      snprintf(oc->filename, sizeof(oc->filename), "%s", ctx->output_filename);
  
      oc->debug = 1;
  
      oc->start_time_realtime = ctx->input_context->start_time;
  
      oc->start_time = ctx->input_context->start_time;
  
      oc->duration = 0;
  
      oc->bit_rate = 0;
  
  
  
      for (int i = 0; i < ctx->input_context->nb_streams; i++) {
  
          input_stream = ctx->input_context->streams[i];
  
          output_stream = NULL;
  
          if (input_stream->index == ctx->video_stream_index) {
  
              //copy stuff from input video index
  
              c = avcodec_find_encoder(CODEC_ID_H264);
  
              output_stream = avformat_new_stream(oc, c);
  
              *video_stream = output_stream;
  
              cc = output_stream->codec;
  
              cc->width = input_stream->codec->width;
  
              cc->height = input_stream->codec->height;
  
              cc->codec_id = CODEC_ID_H264;
  
              cc->codec_type = AVMEDIA_TYPE_VIDEO;
  
              cc->bit_rate = ENCODED_BITRATE;
  
              cc->time_base = input_stream->codec->time_base;
  
  
  
              output_stream->avg_frame_rate = input_stream->avg_frame_rate;
  
              output_stream->r_frame_rate = input_stream->r_frame_rate;
  
              output_stream->start_time = AV_NOPTS_VALUE;
  
  
  
          } else if ((input_stream->codec->codec_type == AVMEDIA_TYPE_AUDIO) && !audio_copied)  { 
  
              /* i care only about audio */
  
              c = avcodec_find_encoder(input_stream->codec->codec_id);
  
              output_stream = avformat_new_stream(oc, c);
  
              *audio_stream = output_stream;
  
              avcodec_copy_context(output_stream->codec, input_stream->codec);
  
              /* Apparently fixes a crash on .mkvs with attachments: */
  
              av_dict_copy(&output_stream->metadata, input_stream->metadata, 0);
  
              /* Reset the codec tag so as not to cause problems with output format */
  
              output_stream->codec->codec_tag = 0;
  
              audio_copied = 1;
  
          }
  
      }
  
  
  
      for (int i = 0; i < oc->nb_streams; i++) {
  
          if (oc->oformat->flags & AVFMT_GLOBALHEADER)
  
              oc->streams[i]->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;
  
          if (oc->streams[i]->codec->sample_rate == 0)
  
              oc->streams[i]->codec->sample_rate = 48000; /* ish */
  
      }
  
  
  
      if (!(fmt->flags & AVFMT_NOFILE)) {
  
          fprintf(stderr, "[DEBUG] AVFMT_NOFILE set, allocating output container\n");
  
          if (avio_open(&oc->pb, ctx->output_filename, AVIO_FLAG_WRITE) < 0) {
  
              fprintf(stderr, "[DEBUG] error creating the output context\n");
  
              exit(1);
  
          }
  
      }
  
  
  
      return oc;
  
  }

  Finally this is the code for writing audio :

  static
  
  void
  
  write_audio_frame(AVFormatContext *oc, AVStream *st, struct transcoder_ctx_t *ctx) {
  
      AVPacket pkt = {0}; // data and size must be 0;
  
      struct packet_t *source_audio;
  
      av_init_packet(&pkt);
  
  
  
      if (!(source_audio = packet_queue_get_next_item_asynch(ctx->processed_audio_queue))) {
  
          return;
  
      }
  
  
  
      pkt.stream_index = st->index;
  
      pkt.size = source_audio->data_length;
  
      pkt.data = source_audio->data;
  
      pkt.pts = source_audio->PTS;
  
      pkt.dts = source_audio->DTS;
  
      pkt.duration = source_audio->duration;
  
      pkt.destruct = avpacket_destruct;
  
      /* Write the compressed frame to the media file. */
  
      if (av_interleaved_write_frame(oc, &pkt) != 0) {
  
          fprintf(stderr, "[DEBUG] Error while writing audio frame\n");
  
      }
  
  
  
      packet_queue_free_packet(source_audio, 0);
  
  }

  A resulting mpeg4 file can be obtained from here : http://87.120.131.41/dl/mpeg4.h264

  I have ommited the write_video_frame code since it is a lot more complicated and I might be making something wrong there as I'm doing timebase conversation etc. For audio however I'm doing 1:1 copy. Each packet_t packet contains data from av_read_frame from the input mpegts container. In the worst case I'd expect that my audio is working and not my video. However I cannot get either of those to work. Seems the documentation is rather vague on making things like that - I've tried both libav and ffmpeg irc channels to no avail. Any information regarding how I can debug the issue will be greatly appreciated.