Recherche avancée

Sur d’autres sites (10217)

• Watson NarrowBand Speech to Text not accepting ogg file

  19 janvier 2017, par Bob Dill

  NodeJS app using ffmpeg to create ogg files from mp3 & mp4. If the source file is broadband, Watson Speech to Text accepts the file with no issues. If the source file is narrow band, Watson Speech to Text fails to read the ogg file. I’ve tested the output from ffmpeg and the narrowband ogg file has the same audio content (e.g. I can listen to it and hear the same people) as the mp3 file. Yes, in advance, I am changing the call to Watson to correctly specify the model and content_type. Code follows :

  exports.createTranscript = function(req, res, next)
  
  { var _name = getNameBase(req.body.movie);
  
    var _type = getType(req.body.movie);
  
    var _voice = (_type == "mp4") ? "en-US_BroadbandModel" : "en-US_NarrowbandModel" ;
  
    var _contentType = (_type == "mp4") ? "audio/ogg" : "audio/basic" ;
  
    var _audio = process.cwd()+"/HTML/movies/"+_name+'ogg';
  
    var transcriptFile = process.cwd()+"/HTML/movies/"+_name+'json';
  
  
  
    speech_to_text.createSession({model: _voice}, function(error, session) {
  
      if (error) {console.log('error:', error);}
  
      else
  
        {
  
          var params = { content_type: _contentType, continuous: true,
  
           audio: fs.createReadStream(_audio),
  
            session_id: session.session_id
  
            };
  
            speech_to_text.recognize(params, function(error, transcript) {
  
              if (error) {console.log('error:', error);}
  
              else
  
                { fs.writeFile(transcriptFile, JSON.stringify(transcript), function(err) {if (err) {console.log(err);}});
  
                  res.send(transcript);
  
                }
  
            });
  
        }
  
    });
  
  }

  _type is either mp3 (narrowband from phone recording) or mp4 (broadband)
  model: _voice has been traced to ensure correct setting
  content_type: _contentType has been traced to ensure correct setting

  Any ogg file submitted to Speech to Text with narrowband settings fails with Error: No speech detected for 30s. Tested with both real narrowband files and asking Watson to read a broadband ogg file (created from mp4) as narrowband. Same error message. What am I missing ?

• FPS drop in FFMPEG streaming processes to FB from production server

  30 janvier 2017, par Aakash Gupta

  I have made a rails app that can stream live videos to facebook rtmp server and deployed it on AWS. I have used nginx as web server. The major problem that I am encountering after viewing log files of FFMpeg processes is that sometimes the FPS of FFmpeg process starts to drop. In some cases, it remains stable at 25 FPS but in some cases, it remains at 25 only for sometime, and after that it starts to drop and sometimes it falls to even 3-4 FPS which is unacceptable during live streaming. As FFMpeg process is quite heavy, I would also like to share my CPU info as well.

  CPU information is :

  cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
stepping    : 2
microcode   : 0x25
cpu MHz     : 2400.070
cache size  : 30720 KB
physical id : 0
siblings    : 1
core id     : 0
cpu cores   : 1
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm xsaveopt fsgsbase bmi1 avx2 smep bmi2 erms invpcid
bogomips    : 4800.14
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

  FFMPEG log file with unstable fps : https://drive.google.com/open?id=0B1gtp1iXJppkUndFamk4M0lRYzA

  FFMPEG log file with stable fps : https://drive.google.com/open?id=0B1gtp1iXJppkMkVCZEJjYWJrVTA

  When FPS was stable, I also tried to run another parallel FFMpeg process from the same server which resulted in FPS dropping of both the processes to 13-14 FPS.

  I am currently using this FFMPEG command :

  ffmpeg -loop 1 -re -y -f image2 -i "image_path" -i "audio_path.aac" -acodec copy -bsf:a aac_adtstoasc -pix_fmt yuv420p -profile:v high -s 1280x720 -vb 400k -maxrate 400k -minrate 400k -bufsize 600k -deinterlace -vcodec libx264 -preset veryfast -g 30 -r 30 -t 14400 -strict -2 -f flv "rtmp_server_link"

  I never face this problem when I try to stream to FB using app on my localhost.

  So, my questions are :

  1. What can be the reason for this FPS drop ?
  2. Can upscaling production server help me fix this issue ?
  3. Can I run multiple FFMpeg processes for streaming from same server without performance drop ?

  Thanks in advance :)

• FFMPEG RTSP stream to MPEG4/H264 file using libx264

  4 décembre 2016, par Phi

  Heyo folks,

  I’m attempting to transcode/remux an RTSP stream in H264 format into a MPEG4 container, containing just the H264 video stream. Basically, webcam output into a MP4 container.

  I can get a poorly coded MP4 produced, using this code :

  // Variables here for demo
  
  AVFormatContext * video_file_output_format = nullptr;
  
  AVFormatContext * rtsp_format_context = nullptr;
  
  AVCodecContext * video_file_codec_context = nullptr;
  
  AVCodecContext * rtsp_vidstream_codec_context = nullptr;
  
  AVPacket packet = {0};
  
  AVStream * video_file_stream = nullptr;
  
  AVCodec * rtsp_decoder_codec = nullptr;
  
  int errorNum = 0, video_stream_index = 0;
  
  std::string outputMP4file = "D:\\somemp4file.mp4";
  
  
  
  // begin
  
  AVDictionary * opts = nullptr;
  
  av_dict_set(&opts, "rtsp_transport", "tcp", 0);
  
  
  
  if ((errorNum = avformat_open_input(&rtsp_format_context, uriANSI.c_str(), NULL, &opts)) < 0) {
  
      errOut << "Connection failed: avformat_open_input failed with error " << errorNum << ":\r\n" << ErrorRead(errorNum);
  
      TacticalAbort();
  
      return;
  
  }
  
  
  
  rtsp_format_context->max_analyze_duration = 50000;
  
  if ((errorNum = avformat_find_stream_info(rtsp_format_context, NULL)) < 0) {
  
      errOut << "Connection failed: avformat_find_stream_info failed with error " << errorNum << ":\r\n" << ErrorRead(errorNum);
  
      TacticalAbort();
  
      return;
  
  }
  
  
  
  video_stream_index = errorNum = av_find_best_stream(rtsp_format_context, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
  
  
  
  if (video_stream_index < 0) {
  
      errOut << "Connection in unexpected state; made a connection, but there was no video stream.\r\n"
  
          "Attempts to find a video stream resulted in error " << errorNum << ": " << ErrorRead(errorNum);
  
      TacticalAbort();
  
      return;
  
  }
  
  
  
  rtsp_vidstream_codec_context = rtsp_format_context->streams[video_stream_index]->codec;
  
  
  
  av_init_packet(&packet);
  
  
  
  if (!(video_file_output_format = av_guess_format(NULL, outputMP4file.c_str(),  NULL))) {
  
      TacticalAbort();
  
      throw std::exception("av_guess_format");
  
  }
  
  
  
  if (!(rtsp_decoder_codec = avcodec_find_decoder(rtsp_vidstream_codec_context->codec_id))) {
  
      errOut << "Connection failed: connected, but avcodec_find_decoder returned null.\r\n"
  
          "Couldn't find codec with an AV_CODEC_ID value of " << rtsp_vidstream_codec_context->codec_id << ".";
  
      TacticalAbort();
  
      return;
  
  }
  
  
  
  video_file_format_context = avformat_alloc_context();
  
  video_file_format_context->oformat = video_file_output_format;
  
  
  
  if (strcpy_s(video_file_format_context->filename, sizeof(video_file_format_context->filename), outputMP4file.c_str())) {
  
      errOut << "Couldn't open video file: strcpy_s failed with error " << errno << ".";
  
      std::string log = errOut.str();
  
      TacticalAbort();
  
      throw std::exception("strcpy_s");
  
  }
  
  
  
  if (!(video_file_encoder_codec = avcodec_find_encoder(video_file_output_format->video_codec))) {
  
      TacticalAbort();
  
      throw std::exception("avcodec_find_encoder");
  
  }
  
  
  
  // MARKER ONE
  
  
  
  if (!outputMP4file.empty() &&
  
      !(video_file_output_format->flags & AVFMT_NOFILE) &&
  
      (errorNum = avio_open2(&video_file_format_context->pb, outputMP4file.c_str(), AVIO_FLAG_WRITE, nullptr, &opts)) < 0) {
  
      errOut << "Couldn't open video file \"" << outputMP4file << "\" for writing : avio_open2 failed with error " << errorNum << ": " << ErrorRead(errorNum);
  
      TacticalAbort();
  
      return;
  
  }
  
  
  
  // Create stream in MP4 file
  
  if (!(video_file_stream = avformat_new_stream(video_file_format_context, video_file_encoder_codec))) {
  
      TacticalAbort();
  
      return;
  
  }
  
  
  
  AVCodecContext * video_file_codec_context = video_file_stream->codec;
  
  
  
  // MARKER TWO
  
  
  
  // error -22/-21 in avio_open2 if this is skipped
  
  if ((errorNum = avcodec_copy_context(video_file_codec_context, rtsp_vidstream_codec_context)) != 0) {
  
      TacticalAbort();
  
      throw std::exception("avcodec_copy_context");
  
  }
  
  
  
  //video_file_codec_context->codec_tag = 0;
  
  
  
  /*
  
  // MARKER 3 - is this not needed? Examples suggest not.
  
  if ((errorNum = avcodec_open2(video_file_codec_context, video_file_encoder_codec, &opts)) < 0)
  
  {
  
      errOut << "Couldn't open video file codec context: avcodec_open2 failed with error " << errorNum << ": " << ErrorRead(errorNum);
  
      std::string log = errOut.str();
  
      TacticalAbort();
  
      throw std::exception("avcodec_open2, video file");
  
  }*/
  
  
  
  //video_file_format_context->flags |= AVFMT_FLAG_GENPTS;
  
  if (video_file_format_context->oformat->flags & AVFMT_GLOBALHEADER)
  
  {
  
      video_file_codec_context->flags |= CODEC_FLAG_GLOBAL_HEADER;
  
  }
  
  
  
  if ((errorNum = avformat_write_header(video_file_format_context, &opts)) < 0) {
  
      errOut << "Couldn't open video file: avformat_write_header failed with error " << errorNum << ":\r\n" << ErrorRead(errorNum);
  
      std::string log = errOut.str();
  
      TacticalAbort();
  
      return;
  
  }

  However, there are several issues :

  1. I can’t pass any x264 options to the output file. The output H264 matches the input H264’s profile/level - switching cameras to a different model switches H264 level.
  2. The timing of the output file is off, noticeably.
  3. The duration of the output file is off, massively. A few seconds of footage becomes hours, although playtime doesn’t match. (FWIW, I’m using VLC to play them.)

  Passing x264 options

  If I manually increment PTS per packet, and set DTS equal to PTS, it plays too fast, 2-3 seconds’ worth of footage in one second playtime, and duration is hours long. The footage also blurs past several seconds, about 10 seconds’ footage in a second.

  If I let FFMPEG decide (with or without GENPTS flag), the file has a variable frame rate (probably as expected), but it plays the whole file in an instant and has a long duration too (over forty hours for a few seconds). The duration isn’t "real", as the file plays in an instant.

  At Marker One, I try to set the profile by passing options to avio_open2. The options are simply ignored by libx264. I’ve tried :

  av_dict_set(&opts, "vprofile", "main", 0);
  
  av_dict_set(&opts, "profile", "main", 0); // error, missing '('
  
  // FF_PROFILE_H264_MAIN equals 77, so I also tried
  
  av_dict_set(&opts, "vprofile", "77", 0); 
  
  av_dict_set(&opts, "profile", "77", 0);

  It does seem to read the profile setting, but it doesn’t use them. At Marker Two, I tried to set it after the avio_open2, before avformat_write_header .

  // I tried all 4 av_dict_set from earlier, passing it to avformat_write_header.
  
  // None had any effect, they weren't consumed.
  
  av_opt_set(video_file_codec_context, "profile", "77", 0);
  
  av_opt_set(video_file_codec_context, "profile", "main", 0);
  
  video_file_codec_context->profile = FF_PROFILE_H264_MAIN;
  
  av_opt_set(video_file_codec_context->priv_data, "profile", "77", 0);
  
  av_opt_set(video_file_codec_context->priv_data, "profile", "main", 0);

  Messing with privdata made the program unstable, but I was trying anything at that point.
  I’d like to solve issue 1 with passing settings, since I imagine it’d bottleneck any attempt to solve issues 2 or 3.

  I’ve been fiddling with this for the better part of a month now. I’ve been through dozens of documentation, Q&As, examples. It doesn’t help that quite a few are outdated.

  Any help would be appreciated.

  Cheers