Sur d’autres sites (8255)

• How to generate video as fast as possible with subtitles and audio on node.js + ffmpeg ?

  12 septembre 2018, par DSeregin

  Intro :

  We receive from the site some pieces of text
  Pieces arrive to node.js-server

  At the output we need to get a video, merged from all the pieces of text, voiced by the machine voice, with the added subtitles and audio substrate. So that user could be share this video in the social networks. MKV format doesn`t supported by VK.com

  The options that we have tried :
  1. Get all the text at once, generate the entire speech, create a file with subtitles, burn subtitles in the video .mp4 (vk.com does not support the .mkv container). It took 12 seconds of operations for a 45-second video on the local computer.
  2. Generate audio and video files for each piece of text (with added subtitles). It took one second for one piece of text. At the final request, we merge all pieces together. The last request (merging) took 2-3 seconds, which is already bearable.

  The second variant looks acceptable in terms of speed, but if you run 50 clients at the same time, then the computer (tested on a MacBook PRO 2013, 2.4 GHz i7, 8gb 1600 Mhz DDR3, SSD 256gb) processed only 1 piece from 1 client in 60 seconds (60 times slower), then the computer hung tight.

  • Generation of audio is done through Amazon Polly
  • Generate subtitles via https://github.com/gsantiago/subtitle.js#readme
  • From node.js we call ffmpeg using https://github.com/fluent-ffmpeg/node-fluent-ffmpeg
  • Resolution for video - 1280x720

  The commands we used :

  • Burn video subtitles and trim up to conditional 6 seconds (in the code send unix timestamp)

  ffmpeg -i import / back.mov -i export_0 / tmp.srt -scodec mov_text -t 6 export_0 / output.mov

  • Merging all audio

  ffmpeg -i audio1.mp3 .... -i audio15.mp3 merged.mp3

  • Overlay audio-substrate on the text

  ffmpeg -i merged.mp3 -i back.mp3 -filter_complex amerge -ac 2-c: a libmp3lame -q: a 4 -shortest audio.mp3

  • Merging all videos

  ffmpeg -i video.txt -f concat -c copy video.mp4

  • Overlay audio on video

  ffmpeg -i audio.mp3 -i video.mp4 -i test.mp4 -i export / output.mp3 -c: v copy -c: a aac -map 0: v: 0 -map 1: a: 0 -shortest output .mp4

  Questions that torment :

  1. Is it faster ?

  2. Can I use other codecs or methods of gluing without re-encoding ?

  3. Try to call ffmpeg directly without a wrapper ? (in fact, it gives 50-100 ms of speed)

  4. Try not to save to disk, and write data to Stream and have them glue together in the end ?

• Muxing in audio to gstreamer RTMP stream kills both video and Audio

  1er avril 2015, par Adam

  I need some genius help here - I’m trying to set up a live stream for my upcoming wedding... and I have it ALMOST working - audio seems to be the problem.

  This is my setup

  • Raspberry Pi Model B+
  • Logitech C920 (with onboard h264 encoding that I am utilising)
  • on-camera (C920) microphone
  • USB wifi to iPhone 4G connection
  • gstreamer1.0
  • Amazon EC2 Wowza RTMP server

  I have it all set up, but as soon as I mux in the audio, the streams wont play by any player.

  What Works :
  - my gstreamer pipeline WITHOUT the audio muxed in
  - Wowza receives a consistent stream, no failures
  - The various Flash players / iOS / Android and VLC all play back the video

  What doesnt :
  - enabling audio in the mux (using the pipeline below)
  - BUT gstreamer doesnt complain
  - BUT Wowza receives a consistent stream, no failures
  - The various flash players fail to play both Audio and Video. some just display the first video frame
  - VLC plays 1 video frame, and about 100ms of audio, then stops

  Ideally I’d like the muxed audio/video FLV stored on the SD card too in case the network goes down - but if the ’tee’ needs to be sacrificed to make it work, so be it.

  This is my current FAILING pipeline - I assume there’s something really stupid in it because I know practically nothing about gstreamer.... The first frame loads in all the players (except iOS.. which never shows anything)

  # set camera resolution to 720p, and the data format to H264 (alternatives are YUV and JPG)
  
  v4l2-ctl --device=/dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=1
  
  # set the frame rate
  
  v4l2-ctl --device=/dev/video0 --set-parm=10
  
  
  
  gst-launch-1.0 -v -e uvch264src initial-bitrate=300000 average-bitrate=300000 device=/dev/video0 name=src auto-start=true src.vidsrc \
  
                  ! queue \
  
                  ! video/x-h264,width=1280,height=720,framerate=10/1 \
  
                  ! h264parse \
  
                  ! flvmux streamable=true name=mux \
  
                  ! queue \
  
                  ! tee name=t \
  
                  ! queue \
  
                  ! filesink location=/home/pi/wedding.flv t. \
  
                  ! queue \
  
                  ! rtmpsink location='rtmp://wowzaserver/live/wedding live=1' >>/home/pi/wedding.log 2>&1

  Some of the things I can’t really afford to change at this late stage are the encapsulation (FLV) and wowza RTMP because I’ve built everything around that...

  Please Help !! Thanks !

  UPDATE

  Given that I am also saving the FLV file, I have found that if I use ffmpeg to send that FLV file (using audio copy, video copy) to the RTMP server, everything works (but obviously its not live) ! So I am now starting to believe this is a problem with the way Gstreamer encapsulates RTMP - and by putting ffmpeg in the middle it fixes it... but it’s not live of course.
  Is it possible to pipe my output to ffmpeg and using ffmpeg’s RTMP ?

• AWS Lambda function for modify video

  4 février 2017, par Gold Fish

  I want to create a Lambda function that invoked whenever someone uploads to the S3 bucket. The purpose of the function is to take the uploaded file and if its a video file (mp4) so make a new file which is a preview of the last one (using ffmpeg). The Lambda function is written in nodejs.
  I took the code here for reference, but I do something wrong for I get an error saying that no input specified for SetStartTime :

  //dependecies
  
  var async = require('async');
  
  var AWS = require('aws-sdk');
  
  var util = require('util');
  
  var ffmpeg = require('fluent-ffmpeg');
  
  
  
  // get reference to S3 client 
  
  var s3 = new AWS.S3();
  
  
  
  
  
  exports.handler = function(event, context, callback) {
  
      // Read options from the event.
  
      console.log("Reading options from event:\n", util.inspect(event, {depth: 5}));
  
      var srcBucket = event.Records[0].s3.bucket.name;
  
      // Object key may have spaces or unicode non-ASCII characters.
  
      var srcKey    =
  
      decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));  
  
      var dstBucket = srcBucket;
  
      var dstKey    = "preview_" + srcKey;
  
  
  
      // Sanity check: validate that source and destination are different buckets.
  
      if (srcBucket == dstBucket) {
  
          callback("Source and destination buckets are the same.");
  
          return;
  
      }
  
  
  
      // Infer the video type.
  
      var typeMatch = srcKey.match(/\.([^.]*)$/);
  
      if (!typeMatch) {
  
          callback("Could not determine the video type.");
  
          return;
  
      }
  
      var videoType = typeMatch[1];
  
      if (videoType != "mp4") {
  
          callback('Unsupported video type: ${videoType}');
  
          return;
  
      }
  
  
  
      // Download the video from S3, transform, and upload to a different S3 bucket.
  
      async.waterfall([
  
          function download(next) {
  
              // Download the video from S3 into a buffer.
  
              s3.getObject({
  
                      Bucket: srcBucket,
  
                      Key: srcKey
  
                  },
  
                  next);
  
              },
  
          function transform(response, next) {
  
          console.log("response.Body:\n", response.Body);
  
          ffmpeg(response.Body)
  
              .setStartTime('00:00:03')
  
              .setDuration('10')   //.output('public/videos/test/test.mp4')
  
          .toBuffer(videoType, function(err, buffer) {
  
                          if (err) {
  
                              next(err);
  
                          } else {
  
                              next(null, response.ContentType, buffer);
  
                          }
  
                   });
  
          },
  
          function upload(contentType, data, next) {
  
              // Stream the transformed image to a different S3 bucket.
  
              s3.putObject({
  
                      Bucket: dstBucket,
  
                      Key: dstKey,
  
                      Body: data,
  
                      ContentType: contentType
  
                  },
  
                  next);
  
              }
  
          ], function (err) {
  
              if (err) {
  
                  console.error(
  
                      'Unable to modify ' + srcBucket + '/' + srcKey +
  
                      ' and upload to ' + dstBucket + '/' + dstKey +
  
                      ' due to an error: ' + err
  
                  );
  
              } else {
  
                  console.log(
  
                      'Successfully modify ' + srcBucket + '/' + srcKey +
  
                      ' and uploaded to ' + dstBucket + '/' + dstKey
  
                  );
  
              }
  
  
  
              callback(null, "message");
  
          }
  
      );
  
  };

  So what am I doing wrong ?