Recherche avancée

Sur d’autres sites (6008)

• Google Speech API + Go - Transcribing Audio Stream of Unknown Length

  14 février 2018, par Josh

  I have an rtmp stream of a video call and I want to transcribe it. I have created 2 services in Go and I’m getting results but it’s not very accurate and a lot of data seems to get lost.

  Let me explain.

  I have a transcode service, I use ffmpeg to transcode the video to Linear16 audio and place the output bytes onto a PubSub queue for a transcribe service to handle. Obviously there is a limit to the size of the PubSub message, and I want to start transcribing before the end of the video call. So, I chunk the transcoded data into 3 second clips (not fixed length, just seems about right) and put them onto the queue.

  The data is transcoded quite simply :

  var stdout Buffer
  
  
  
  cmd := exec.Command("ffmpeg", "-i", url, "-f", "s16le", "-acodec", "pcm_s16le", "-ar", "16000", "-ac", "1", "-")
  
  cmd.Stdout = &stdout
  
  
  
  if err := cmd.Start(); err != nil {
  
      log.Fatal(err)
  
  }
  
  
  
  ticker := time.NewTicker(3 * time.Second)
  
  
  
  for {
  
      select {
  
      case <-ticker.C:
  
          bytesConverted := stdout.Len()
  
          log.Infof("Converted %d bytes", bytesConverted)
  
  
  
          // Send the data we converted, even if there are no bytes.
  
          topic.Publish(ctx, &pubsub.Message{
  
              Data: stdout.Bytes(),
  
          })
  
  
  
          stdout.Reset()
  
      }
  
  }

  The transcribe service pulls messages from the queue at a rate of 1 every 3 seconds, helping to process the audio data at about the same rate as it’s being created. There are limits on the Speech API stream, it can’t be longer than 60 seconds so I stop the old stream and start a new one every 30 seconds so we never hit the limit, no matter how long the video call lasts for.

  This is how I’m transcribing it :

  stream := prepareNewStream()
  
  clipLengthTicker := time.NewTicker(30 * time.Second)
  
  chunkLengthTicker := time.NewTicker(3 * time.Second)
  
  
  
  cctx, cancel := context.WithCancel(context.TODO())
  
  err := subscription.Receive(cctx, func(ctx context.Context, msg *pubsub.Message) {
  
  
  
      select {
  
      case <-clipLengthTicker.C:
  
          log.Infof("Clip length reached.")
  
          log.Infof("Closing stream and starting over")
  
  
  
          err := stream.CloseSend()
  
          if err != nil {
  
              log.Fatalf("Could not close stream: %v", err)
  
          }
  
  
  
          go getResult(stream)
  
          stream = prepareNewStream()
  
  
  
      case <-chunkLengthTicker.C:
  
          log.Infof("Chunk length reached.")
  
  
  
          bytesConverted := len(msg.Data)
  
  
  
          log.Infof("Received %d bytes\n", bytesConverted)
  
  
  
          if bytesConverted > 0 {
  
              if err := stream.Send(&speechpb.StreamingRecognizeRequest{
  
                  StreamingRequest: &speechpb.StreamingRecognizeRequest_AudioContent{
  
                      AudioContent: transcodedChunk.Data,
  
                  },
  
              }); err != nil {
  
                  resp, _ := stream.Recv()
  
                  log.Errorf("Could not send audio: %v", resp.GetError())
  
              }
  
          }
  
  
  
          msg.Ack()
  
      }
  
  })

  I think the problem is that my 3 second chunks don’t necessarily line up with starts and end of phrases or sentences so I suspect that the Speech API is a recurrent neural network which has been trained on full sentences rather than individual words. So starting a clip in the middle of a sentence loses some data because it can’t figure out the first few words up to the natural end of a phrase. Also, I lose some data in changing from an old stream to a new stream. There’s some context lost. I guess overlapping clips might help with this.

  I have a couple of questions :

  1) Does this architecture seem appropriate for my constraints (unknown length of audio stream, etc.) ?

  2) What can I do to improve accuracy and minimise lost data ?

  (Note I’ve simplified the examples for readability. Point out if anything doesn’t make sense because I’ve been heavy handed in cutting the examples down.)

• ffmpeg cut flv but resulting video cannot be processed on youtube

  20 avril 2017, par user1017674

  I am using a simple ffmpeg command to cut some videos.

  The command i ran is

  ffmpeg -i test.flv -ss 0 -to 60 -c copy output.flv

  The output.flv turns out is not processable on youtube, but I am able to play this file locally using a random flv player that I downloaded.

  This is the log when cutting the video.

  The original file test.flv can be found here.
  https://drive.google.com/open?id=0B5eHJIAbpaIYaW95ZlFTMTUzNGc

  The output file output.flv can be found here.
  https://drive.google.com/open?id=0B5eHJIAbpaIYejk5dGJTOEo1UU0

  ./ffmpeg -i test.flv -ss 0 -to 60 -c copy diablo.flv
  
  ffmpeg version 3.3 Copyright (c) 2000-2017 the FFmpeg developers
  
    built with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
  
    configuration: --prefix=/Volumes/Ramdisk/sw --enable-gpl --enable-pthreads --enable-version3 --enable-libspeex --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-libgsm --enable-libvidstab --enable-libx265 --disable-doc --arch=x86_64 --enable-runtime-cpudetect
  
    libavutil      55. 58.100 / 55. 58.100
  
    libavcodec     57. 89.100 / 57. 89.100
  
    libavformat    57. 71.100 / 57. 71.100
  
    libavdevice    57.  6.100 / 57.  6.100
  
    libavfilter     6. 82.100 /  6. 82.100
  
    libswscale      4.  6.100 /  4.  6.100
  
    libswresample   2.  7.100 /  2.  7.100
  
    libpostproc    54.  5.100 / 54.  5.100
  
  Input #0, flv, from 'test.flv':
  
    Metadata:
  
      creator         : Coded
  
      metadatacreator : Yet Another Metadata Injector for FLV - Version 1.9
  
      hasKeyframes    : true
  
      hasVideo        : true
  
      hasAudio        : true
  
      hasMetadata     : true
  
      canSeekToEnd    : true
  
      datasize        : 69602014
  
      videosize       : 63600961
  
      audiosize       : 5917165
  
      lasttimestamp   : 292
  
      lastkeyframetimestamp: 292
  
      lastkeyframelocation: 69603218
  
    Duration: 00:04:51.68, start: 0.080000, bitrate: 1909 kb/s
  
      Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x720, 1742 kb/s, 25 fps, 25 tbr, 1k tbn, 50 tbc
  
      Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp, 158 kb/s
  
  Output #0, flv, to 'diablo.flv':
  
    Metadata:
  
      creator         : Coded
  
      metadatacreator : Yet Another Metadata Injector for FLV - Version 1.9
  
      hasKeyframes    : true
  
      hasVideo        : true
  
      hasAudio        : true
  
      hasMetadata     : true
  
      canSeekToEnd    : true
  
      datasize        : 69602014
  
      videosize       : 63600961
  
      audiosize       : 5917165
  
      lasttimestamp   : 292
  
      lastkeyframetimestamp: 292
  
      lastkeyframelocation: 69603218
  
      encoder         : Lavf57.71.100
  
      Stream #0:0: Video: h264 (High) ([7][0][0][0] / 0x0007), yuv420p(progressive), 1280x720, q=2-31, 1742 kb/s, 25 fps, 25 tbr, 1k tbn, 1k tbc
  
      Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 48000 Hz, stereo, fltp, 158 kb/s
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (copy)
  
    Stream #0:1 -> #0:1 (copy)
  
  Press [q] to stop, [?] for help
  
  frame= 1252 fps=0.0 q=-1.0 Lsize=   12074kB time=00:00:59.98 bitrate=1648.8kbits/s speed=1.21e+03x    
  
  video:10849kB audio:1153kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.598021%

• Splitting video file more then once

  18 mai 2017, par Fearhunter

  I’m experimenting with this github repo for cutting video’s.

  https://github.com/vdaubry/html5-video-splitter

  This is the code of video-splitter.js

  var childProcess = require("child_process");
  
  childProcess.spawn = require('cross-spawn');
  
  
  
      var http = require('http');
  
      var path = require("path"); 
  
      var fs = require("fs"); 
  
      var exec = require('child_process').exec;
  
  
  
      http.createServer(function (req, res) {
  
        if (req.method === 'OPTIONS') {
  
            console.log('!OPTIONS');
  
            var headers = {};
  
            headers["Access-Control-Allow-Origin"] = "*";
  
            headers["Access-Control-Allow-Methods"] = "POST, GET, PUT, DELETE, OPTIONS";
  
            headers["Access-Control-Allow-Credentials"] = false;
  
            headers["Access-Control-Max-Age"] = '86400';
  
            headers["Access-Control-Allow-Headers"] = "X-reqed-With, X-HTTP-Method-Override, Content-Type, Accept";
  
            res.writeHead(200, headers);
  
            res.end();
  
        } 
  
        else if (req.method == 'POST') {
  
          var body = '';
  
          req.on('data', function (data) {
  
              body += data;
  
          });
  
          req.on('end', function () {
  
              var data = JSON.parse(body);
  
              var localPath = __dirname;
  
              var inputFilePath = localPath+"/videos/"+data.inputFilePath;
  
              var outputFilePath = localPath+"/videos/output-"+data.inputFilePath
  
              var start = data.begin;
  
              var end = data.end;
  
  
  
  
  
              var command = "ffmpeg -y -ss "+start+" -t "+(end-start)+" -i "+inputFilePath+" -vcodec copy -acodec copy "+outputFilePath;
  
              exec(command, function(error, stdout, stderr) {
  
                var msg = ""
  
                if(error) {
  
                  console.log(error);
  
                  msg = error.toString();
  
                  res.writeHead(500, {'Content-Type': 'text/plain'});
  
                }
  
                else {
  
                  console.log(stdout);
  
                  res.writeHead(200, {'Content-Type': 'text/plain'});
  
                }
  
                res.end(msg);
  
              });
  
          });
  
        }
  
        else if (req.method == 'GET') {
  
          var filename = "index.html";
  
          if(req.url != "/") {
  
            filename = req.url
  
          }
  
  
  
          var ext = path.extname(filename);
  
          var localPath = __dirname;
  
          var validExtensions = {
  
            ".html" : "text/html",      
  
            ".js": "application/javascript", 
  
            ".css": "text/css",
  
            ".txt": "text/plain",
  
            ".jpg": "image/jpeg",
  
            ".gif": "image/gif",
  
            ".png": "image/png",
  
            ".ico": "image/x-icon"
  
          };
  
          var mimeType = validExtensions[ext];
  
  
  
          if (mimeType) {
  
            localPath += "/interface/"+filename;
  
            fs.exists(localPath, function(exists) {
  
              if(exists) {
  
                console.log("Serving file: " + localPath);
  
                getFile(localPath, res, mimeType);
  
              } else {
  
                console.log("File not found: " + localPath);
  
                res.writeHead(404);
  
                res.end();
  
              }
  
            });
  
  
  
          } else {
  
            console.log("Invalid file extension detected: " + ext)
  
          }
  
        }
  
      }).listen(1337, '127.0.0.1');
  
      console.log('Server running at http://127.0.0.1:1337/');
  
  
  
      function getFile(localPath, res, mimeType) {
  
        fs.readFile(localPath, function(err, contents) {
  
          if(!err) {
  
            res.setHeader("Content-Length", contents.length);
  
            res.setHeader("Content-Type", mimeType);
  
            res.statusCode = 200;
  
            res.end(contents);
  
          } else {
  
            res.writeHead(500);
  
            res.end();
  
          }
  
        });
  
      }

  He does exactly do the correct way for splitting the video. I am searching for a solution to cut more then only once. Because when I split again it overwrites the previous cut video. How can I change the code to split more then once ?

  I have already FFMPEG installed and works correct.

  The output must be for example :

  output-1.mp4
  
  output-2.mp4
  
  output-3.mp4
  
  output-4.mp4