Recherche avancée

Sur d’autres sites (6821)

• Decoding MediaRecorder produced webm stream

  15 août 2019, par sgmg

  I am trying to decode a video stream from the browser using the ffmpeg API. The stream is produced by the webcam and recorded with MediaRecorder as webm format. What I ultimately need is a vector of opencv cv::Mat objects for further processing.

  I have written a C++ webserver using the uWebsocket library. The video stream is sent via websocket from the browser to the server once per second. On the server, I append the received data to my custom buffer and decode it with the ffmpeg API.

  If I just save the data on the disk and later I play it with a media player, it works fine. So, whatever the browser sends is a valid video.

  I do not think that I correctly understand how should the custom IO behave with network streaming as nothing seems to be working.

  The custom buffer :

   struct Buffer
  
      {
  
          std::vector data;
  
          int currentPos = 0;
  
      };

  The readAVBuffer method for custom IO

  int MediaDecoder::readAVBuffer(void* opaque, uint8_t* buf, int buf_size)
  
  {
  
      MediaDecoder::Buffer* mbuf = (MediaDecoder::Buffer*)opaque;
  
      int count = 0;
  
      for(int i=0;icurrentPos;
  
          if(index >= (int)mbuf->data.size())
  
          {
  
              break;
  
          }
  
          count++;
  
          buf[i] = mbuf->data.at(index);
  
      }
  
      if(count > 0) mbuf->currentPos+=count;
  
  
  
      std::cout << "read : "<currentPos<<", buff size:"<data.size() << std::endl;
  
      if(count <= 0) return AVERROR(EAGAIN); //is this error that should be returned? It cannot be EOF since we're not done yet, most likely
  
      return count;
  
  }

  The big decode method, that’s supposed to return whatever frames it could read

  std::vector MediaDecoder::decode(const char* data, size_t length)
  
  {
  
      std::vector frames;
  
      //add data to the buffer
  
      for(size_t i=0;i/do not invoke the decoders until we have 1MB of data
  
      if(((buf.data.size() - buf.currentPos) < 1*1024*1024) && !initializedCodecs) return frames;
  
  
  
      std::cout << "decoding data length "</initialize ffmpeg objects. Custom I/O, format, decoder, etc.
  
      {       
  
          //these are just members of the class 
  
          avioCtxPtr = std::unique_ptr(
  
                      avio_alloc_context((uint8_t*)av_malloc(4096),4096,0,&buf,&readAVBuffer,nullptr,nullptr),
  
                      avio_context_deleter());
  
          if(!avioCtxPtr)
  
          {
  
              std::cerr << "Could not create IO buffer" << std::endl;
  
              return frames;
  
          }                
  
  
  
          fmt_ctx = std::unique_ptr(avformat_alloc_context(),
  
                                                                            avformat_context_deleter());
  
          fmt_ctx->pb = avioCtxPtr.get();
  
          fmt_ctx->flags |= AVFMT_FLAG_CUSTOM_IO ;
  
          //fmt_ctx->max_analyze_duration = 2 * AV_TIME_BASE; // read 2 seconds of data
  
          {
  
              AVFormatContext *fmtCtxRaw = fmt_ctx.get();            
  
              if (avformat_open_input(&fmtCtxRaw, "", nullptr, nullptr) < 0) {
  
                  std::cerr << "Could not open movie" << std::endl;
  
                  return frames;
  
              }
  
          }
  
          if (avformat_find_stream_info(fmt_ctx.get(), nullptr) < 0) {
  
              std::cerr << "Could not find stream information" << std::endl;
  
              return frames;
  
          }
  
          if((video_stream_idx = av_find_best_stream(fmt_ctx.get(), AVMEDIA_TYPE_VIDEO, -1, -1, nullptr, 0)) < 0)
  
          {
  
              std::cerr << "Could not find video stream" << std::endl;
  
              return frames;
  
          }
  
          AVStream *video_stream = fmt_ctx->streams[video_stream_idx];
  
          AVCodec *dec = avcodec_find_decoder(video_stream->codecpar->codec_id);
  
  
  
          video_dec_ctx = std::unique_ptr (avcodec_alloc_context3(dec),
  
                                                                                avcodec_context_deleter());
  
          if (!video_dec_ctx)
  
          {
  
              std::cerr << "Failed to allocate the video codec context" << std::endl;
  
              return frames;
  
          }
  
          avcodec_parameters_to_context(video_dec_ctx.get(),video_stream->codecpar);
  
          video_dec_ctx->thread_count = 1;
  
         /* video_dec_ctx->max_b_frames = 0;
  
          video_dec_ctx->frame_skip_threshold = 10;*/
  
  
  
          AVDictionary *opts = nullptr;
  
          av_dict_set(&opts, "refcounted_frames", "1", 0);
  
          av_dict_set(&opts, "deadline", "1", 0);
  
          av_dict_set(&opts, "auto-alt-ref", "0", 0);
  
          av_dict_set(&opts, "lag-in-frames", "1", 0);
  
          av_dict_set(&opts, "rc_lookahead", "1", 0);
  
          av_dict_set(&opts, "drop_frame", "1", 0);
  
          av_dict_set(&opts, "error-resilient", "1", 0);
  
  
  
          int width = video_dec_ctx->width;
  
          videoHeight = video_dec_ctx->height;
  
  
  
          if(avcodec_open2(video_dec_ctx.get(), dec, &opts) < 0)
  
          {
  
              std::cerr << "Failed to open the video codec context" << std::endl;
  
              return frames;
  
          }
  
  
  
          AVPixelFormat  pFormat = AV_PIX_FMT_BGR24;
  
          img_convert_ctx = std::unique_ptr(sws_getContext(width, videoHeight,
  
                                           video_dec_ctx->pix_fmt,   width, videoHeight, pFormat,
  
                                           SWS_BICUBIC, nullptr, nullptr,nullptr),swscontext_deleter());
  
  
  
          frame = std::unique_ptr(av_frame_alloc(),avframe_deleter());
  
          frameRGB = std::unique_ptr(av_frame_alloc(),avframe_deleter());
  
  
  
  
  
          int numBytes = av_image_get_buffer_size(pFormat, width, videoHeight,32 /*https://stackoverflow.com/questions/35678041/what-is-linesize-alignment-meaning*/);
  
          std::unique_ptr imageBuffer((uint8_t *) av_malloc(numBytes*sizeof(uint8_t)),avbuffer_deleter());
  
          av_image_fill_arrays(frameRGB->data,frameRGB->linesize,imageBuffer.get(),pFormat,width,videoHeight,32);
  
          frameRGB->width = width;
  
          frameRGB->height = videoHeight;
  
  
  
          initializedCodecs = true;
  
      }    
  
      AVPacket pkt;
  
      av_init_packet(&pkt);
  
      pkt.data = nullptr;
  
      pkt.size = 0;
  
  
  
      int read_frame_return = 0;
  
      while ( (read_frame_return=av_read_frame(fmt_ctx.get(), &pkt)) >= 0)
  
      {
  
          readFrame(&frames,&pkt,video_dec_ctx.get(),frame.get(),img_convert_ctx.get(),
  
                    videoHeight,frameRGB.get());
  
          //if(cancelled) break;
  
      }
  
      avioCtxPtr->eof_reached = 0;
  
      avioCtxPtr->error = 0;
  
  
  
  
  
      //flush
  
     // readFrame(frames.get(),nullptr,video_dec_ctx.get(),frame.get(),
  
       //         img_convert_ctx.get(),videoHeight,frameRGB.get());
  
  
  
      avioCtxPtr->eof_reached = 0;
  
      avioCtxPtr->error = 0;
  
  
  
      if(frames->size() <= 0)
  
      {
  
          std::cout << "buffer pos: "<code>

  What I would expect to happen would be for a continuous extraction of cv::Mat frames as I feed it more and more data. What actually happens is that after the the buffer is fully read I see :

  [matroska,webm @ 0x507b450] Read error at pos. 1278266 (0x13813a)
  
  [matroska,webm @ 0x507b450] Seek to desired resync point failed. Seeking to earliest point available instead.

  And then no more bytes are read from the buffer even if later I increase the size of it.

  There is something terribly wrong I’m doing here and I don’t understand what.

• Revision 33588 : un bouton vider devant la ligne vide de creation d’une RC

  8 décembre 2009, par cedric@… — Log

  un bouton vider devant la ligne vide de creation d’une RC

• Running list of cmd.exe commands from maya in Python

  17 décembre 2015, par user3541686

  I am writing a maya python script to batch render a scene into jpgs then use ffmpeg to turn them into a mov. Currently the script saves a string of commands as a bat file which works fine but I’d prefer to just run cmd.exe from maya and execute the list of commands without creating that bat first.

  I’ve been reading about "subprocess.popen()" but I can’t figure out how to get it to iterate through each line, run that command, then move onto the next one when done ie :

  1) Run maya render.exe & save scene as jpegs

  2) ffmpeg jpgs to mov

  I’ve shortened the directories but its essentially this :

  `C:\PROGRA~1\Autodesk\Maya2015\bin\Render.exe -r hw2 -s 100 -e 200 -of jpg -fnc 7 -x 1920 -y 1080 -rd 
  
  "C:\RENDER" 
  
  "C:\SCENE.ma" ffmpeg -y -r 25 -start_number 0100 -i C:\SCENE_%%04d.jpg C:\SCENE.mov

  How would I be able to do this ?

  Thanks !