Recherche avancée

Sur d’autres sites (12647)

• continuous record/recognize audio with pocketsphinx/ffmpeg

  10 novembre 2017, par Michael

  as the title already says, I want to continuous record raw audio through my microphone.
  So the idea was running a simple C program in the background as service that would create chunks of audio and send those files through the sphinx speech recognition.

  After that I can do some processing with the recognized words.

  The problem is the (continuous) recognition. I can’t just record audio chunks containing 10 seconds what i’ve said, because maybe chunk[33] -> chunk[34] belong together and then sphinx would output something like :

  recognized chunk[33] -> ["enable light"]
  
  recognized chunk[34] -> ["5 with 50 percent"]

  Another approach would be to continuous record audio but then I can’t process big audio files with sphinx.

  I’m using the basic example from pocketsphinx :

  #include 
  
  
  
  int main(int argc, char *argv[])
  
  {
  
  ps_decoder_t *ps;
  
  cmd_ln_t *config;
  
  FILE *fh;
  
  char const *hyp, *uttid;
  
  int16 buf[512];
  
  int rv;
  
  int32 score;
  
  
  
  config = cmd_ln_init(NULL, ps_args(), TRUE,
  
               "-hmm", MODELDIR "/en-us/en-us",
  
               "-lm", MODELDIR "/en-us/en-us.lm.bin",
  
               "-dict", MODELDIR "/en-us/cmudict-en-us.dict",
  
               NULL);
  
  if (config == NULL) {
  
  fprintf(stderr, "Failed to create config object, see log for details\n");
  
  return -1;
  
  }
  
  
  
  ps = ps_init(config);
  
  if (ps == NULL) {
  
  fprintf(stderr, "Failed to create recognizer, see log for details\n");
  
  return -1;
  
  }
  
  
  
  fh = fopen("audiochunk_33.raw", "rb");
  
  if (fh == NULL) {
  
  fprintf(stderr, "Unable to open input file goforward.raw\n");
  
  return -1;
  
  }
  
  
  
  rv = ps_start_utt(ps);
  
  
  
  while (!feof(fh)) {
  
  size_t nsamp;
  
  nsamp = fread(buf, 2, 512, fh);
  
  rv = ps_process_raw(ps, buf, nsamp, FALSE, FALSE);
  
  }
  
  
  
  rv = ps_end_utt(ps);
  
  hyp = ps_get_hyp(ps, &score);
  
  printf("Recognized: %s\n", hyp);
  
  
  
  fclose(fh);
  
  ps_free(ps);
  
  cmd_ln_free_r(config);
  
  
  
  return 0;

  }

  And here is a basic example using ffmpeg to create a simple audio file/chunk :

  #include 
  
  #include 
  
  #include 
  
  
  
  #define N 44100
  
  
  
  void main()
  
  {
  
  // Create audio buffer
  
  int16_t buf[N] = {0}; // buffer
  
  int n;                // buffer index
  
  double Fs = 44100.0;  // sampling frequency
  
  
  
  // Generate 1 second of audio data - it's just a 1 kHz sine wave
  
  for (n=0 ; n<n></n>Fs);
  
  
  
  // Pipe the audio data to ffmpeg, which writes it to a wav file
  
  FILE *pipeout;
  
  pipeout = popen("ffmpeg -y -f s16le -ar 44100 -ac 1 -i - beep.wav", "w");
  
  fwrite(buf, 2, N, pipeout);
  
  pclose(pipeout);
  
  }

  BR
  Michael

• Best way to record a HTML Canvas/WebGL animation server-side into a video ?

  19 novembre 2022, par Abhinav

  I have a set of animations which I can make in Canvas (fabric.js) or WebGL (three.js). I need to record them automatically, server-side, through a script and output a video file.

  



  The animations include :
  

  



  

  1. Pictures
     

  


  2. Videos (with audio)
     

  


  3. Other animations/effects

  


  



  I have researched a lot during last few months on this.

  



  Results
  
1. Use PhantomJS + FFMPEG
  
Run HTML Canvas animations on headless browser(PhantomJS) and record with FFMPEG. Here the issue is PhantomJS supports neither WebGL nor Video element. http://phantomjs.org/supported-web-standards.html

  
2. Use Websockets to send data back to server using DataURL
  
Here again, we will need to run the animations on browser (which we can't because we have to do everything on server).

  
3. Use node-canvas
  
This is a library by TJ Holowaychuk which allows rendering HTML Canvas on Node.js. But it has its own limitations plus I haven't really explored this field much. 
(If someone could shed more light on this library)

  



  If anyone has done it before or can guide me somewhere useful.
  
All we need to do is use some data to create animations and record it into a video, everything on server side.

  


• ffmpeg azure function consumption plan low CPU availability for high volume requests

  27 novembre 2017, par The Lemon

  I am running an azure queue function on a consumption plan ; my function starts an FFMpeg process and accordingly is very CPU intensive. When I run the function with less than 100 items in the queue at once it works perfectly, azure scales up and gives me plenty of servers and all of the tasks complete very quickly. My problem is once I start doing more than 300 or 400 items at once, it starts fine but after a while the CPU slowly goes from 80% utilisation to only around 10% utilisation - my functions cant finish in time with only 10% CPU. This can be seen in the image shown below.
  Does anyone know why the CPU useage is going lower the more instances my function creates ? Thanks in advance Cuan

  edit : the function is set to only run one at a time per instance, but the problem exists when set to 2 or 3 concurrent processes per instance in the host.json

  edit : the CPU drops get noticeable at 15-20 servers, and start causing failures at around 60. After that the CPU bottoms out at an average of 8-10% with individuals reaching 0-3%, and the server count seems to increase without limit (which would be more helpful if I got some CPU with the servers)

  Thanks again, Cuan.

  I’ve also added the function code to the bottom of this post in case it helps.

  

  

  using System.Net;
  
  using System;
  
  using System.Diagnostics;
  
  using System.ComponentModel;
  
  
  
  public static void Run(string myQueueItem, TraceWriter log)
  
  {
  
      log.Info($"C# Queue trigger function processed a request: {myQueueItem}");
  
      //Basic Parameters
  
          string ffmpegFile = @"D:\home\site\wwwroot\CommonResources\ffmpeg.exe";
  
          string outputpath = @"D:\home\site\wwwroot\queue-ffmpeg-test\output\";
  
          string reloutputpath = "output/";
  
          string relinputpath = "input/";
  
          string outputfile = "video2.mp4";
  
          string dir =  @"D:\home\site\wwwroot\queue-ffmpeg-test\";
  
  
  
      //Special Parameters
  
  
  
          string videoFile = "1 minute basic.mp4";
  
          string sub = "1 minute sub.ass";
  
      //guid tmp files
  
  
  
          // Guid g1=Guid.NewGuid();
  
          // Guid g2=Guid.NewGuid();
  
          // string f1 = g1 + ".mp4";
  
          // string f2 = g2 + ".ass";
  
          string f1 = videoFile;
  
          string f2 = sub;
  
      //guid output - we will now do this at the caller level
  
          string g3 = myQueueItem;
  
          string outputGuid = g3+".mp4";
  
      //get input files
  
      //argument
  
          string tmp = subArg(f1, f2, outputGuid );
  
      //String.Format("-i \"" + @"input/tmp.mp4" + "\" -vf \"ass = '" + sub + "'\" \"" + reloutputpath +outputfile + "\" -y");
  
      log.Info("ffmpeg argument is: "+tmp);
  
  
  
  
  
      //startprocess parameters
  
      Process process = new Process();
  
      process.StartInfo.FileName = ffmpegFile;
  
      process.StartInfo.Arguments =  tmp;
  
      process.StartInfo.UseShellExecute = false;
  
      process.StartInfo.RedirectStandardOutput = true;
  
      process.StartInfo.RedirectStandardError = true;
  
      process.StartInfo.WorkingDirectory = dir;
  
      //output handler
  
  
  
      process.OutputDataReceived += new DataReceivedEventHandler(
  
          (s, e) => 
  
          { 
  
              log.Info("O: "+e.Data);
  
          }
  
      );
  
      process.ErrorDataReceived += new DataReceivedEventHandler(
  
          (s, e) => 
  
          { 
  
              log.Info("E: "+e.Data);
  
          }
  
      );
  
      //start process
  
      process.Start();
  
      log.Info("process started");
  
      process.BeginOutputReadLine();
  
      process.BeginErrorReadLine();
  
      process.WaitForExit();
  
  }
  
  public static void getFile(string link, string fileName, string dir, string relInputPath){
  
      using (var client = new WebClient()){
  
          client.DownloadFile(link, dir + relInputPath+ fileName);
  
          }
  
  
  
  }
  
  public static string subArg(string input1, string input2, string output1){
  
      return String.Format("-i \"" + @"input/" +input1+ "\" -vf \"ass = '" + @"input/"+input2 + "'\" \"" + @"output/" +output1 + "\" -y");
  
  
  
  }