Recherche avancée

Sur d’autres sites (9671)

• Why does my Blink based browser play hide and seek ?

  21 janvier 2016, par Caius Jard

  We have a C# tool (that I wrote) that records online broadcasts taking place a custom written (that we wrote) flash app. (There are no DRM or copyright issues here.)

  We’ve coded up a system whereby this tool is installed on a Windows Server 2012 R2 Amazon AWS instance. After we boot the instance, the tool loads, waits for the right time to start recording, launches a browser and passes the command line argument of the URL to access the broadcast. The browser will then load the flash app and the interview audio and video will start arriving at the browser instance on AWS

  By way of a virtual audio cable driver, screen / audio capture directshow filters and ffmpeg a screen recording is taken. The C# tool calls ffmpeg and ffmpeg will record the screen reliably for the entire interview, then the tool shuts the whole thing down

  The problem I’m having is that both Chrome and Electron browser sometimes simply don’t draw themselves on the screen so all ffmpeg ends up recording is a blank desktop and the audio of the broadcast (hence, the browser IS running)

  We found this out when recordings started turning up with X hours of merely recording the windows desktop and the tool’s main window with a countdown timer.

  A screenshotting facility was built into the tool and added to its web control interface, and this way we can test whether the browser is visible - a human looks at the screenshot of every broadcast, just after recording has started (the browser is supposed to be on show by this time)

  We notice that 50% of the time, the browser isn’t drawing itself on screen. By 50% I mean that every other recording that the AWS instance carries out, will be blank : AWS starts, records ok, shuts down. AWS starts again an hour later for a different broadcast, recording is blank, shuts down.. Starts/ok/shutdown. Starts/blank/shutdown. Repeat ad infinitum

  What’s even more strange is that if I run VNCviewer on my dev machine and connect up to an instance that is having a problem, the instant that the VNC connection is up and the remote desktop is showing on my screen, the browser suddenly appears as if nothing was ever wrong. A screenshot from before the VNC connect shows blank desktop, connect VNC, take another screenshot and the browser is there. All through it the audio is fine - the browser connected to the boadcast is fine, for sure

  It’s as though Chrome/Electron thinks "you know what, noone is looking at me so I’m not going to bother drawing myself". No screen saver is set, though the power plan has the setting "turn off the display after 15 minutes".

  Perhaps Chrome/Electron have a test amounts to "if the display is off, don’t draw". I can’t explain the inconsistency though - the recorder launches at least 1 hour before it’s needed, and sits there idle until it’s time to start the browser. You’d hence imagine that the "power off the monitor after 15 mins" setting would reliably have ensured the "monitor" is "off" by the time every recording start comes around

  This behaviour doesn’t happen with any of the other browsers (but unfortunately the app doesn’t and cannot work in them because it uses some weird chrome-only technology/API).

  Can anyone suggest anything to look at to help debug this, or anything I can build into the C# tool to overcome the problem ? Coding it up to connect to itself via VNC for a few seconds after it has launched the browser.. Well that just tastes nasty.

  Naturally, as soon as I connect to the machine via VNC (rather than RDP - RDP isn’t usable because the recording context is in a logged on session for a particular user) the problem goes away, which makes it frustratingly hard to debug.

• Live audio using ffmpeg, javascript and nodejs

  8 novembre 2017, par klaus

  I am new to this thing. Please don’t hang me for the poor grammar. I am trying to create a proof of concept application which I will later extend. It does the following : We have a html page which asks for permission to use the microphone. We capture the microphone input and send it via websocket to a node js app.

  JS (Client) :

  var bufferSize = 4096;
  
  var socket = new WebSocket(URL);
  
  var myPCMProcessingNode = context.createScriptProcessor(bufferSize, 1, 1);
  
  myPCMProcessingNode.onaudioprocess = function(e) {
  
    var input = e.inputBuffer.getChannelData(0);
  
    socket.send(convertFloat32ToInt16(input));
  
  }
  
  
  
  function convertFloat32ToInt16(buffer) {
  
    l = buffer.length;
  
    buf = new Int16Array(l);
  
    while (l--) {
  
      buf[l] = Math.min(1, buffer[l])*0x7FFF;
  
    }
  
    return buf.buffer;
  
  }
  
  
  
  navigator.mediaDevices.getUserMedia({audio:true, video:false})
  
                                  .then(function(stream){
  
                                    var microphone = context.createMediaStreamSource(stream);
  
                                    microphone.connect(myPCMProcessingNode);
  
                                    myPCMProcessingNode.connect(context.destination);
  
                                  })
  
                                  .catch(function(e){});

  In the server we take each incoming buffer, run it through ffmpeg, and send what comes out of the std out to another device using the node js ’http’ POST. The device has a speaker. We are basically trying to create a 1 way audio link from the browser to the device.

  Node JS (Server) :

  var WebSocketServer = require('websocket').server;
  
  var http = require('http');
  
  var children = require('child_process');
  
  
  
  wsServer.on('request', function(request) {
  
    var connection = request.accept(null, request.origin);
  
    connection.on('message', function(message) {
  
      if (message.type === 'utf8') { /*NOP*/ }
  
      else if (message.type === 'binary') {
  
        ffm.stdin.write(message.binaryData);
  
      }
  
    });
  
    connection.on('close', function(reasonCode, description) {});
  
    connection.on('error', function(error) {});
  
  });
  
  
  
  var ffm = children.spawn(
  
      './ffmpeg.exe'
  
     ,'-stdin -f s16le -ar 48k -ac 2 -i pipe:0 -acodec pcm_u8 -ar 48000 -f aiff pipe:1'.split(' ')
  
  );
  
  
  
  ffm.on('exit',function(code,signal){});
  
  
  
  ffm.stdout.on('data', (data) => {
  
    req.write(data);
  
  });
  
  
  
  var options = {
  
    host: 'xxx.xxx.xxx.xxx',
  
    port: xxxx,
  
    path: '/path/to/service/on/device',
  
    method: 'POST',
  
    headers: {
  
     'Content-Type': 'application/octet-stream',
  
     'Content-Length': 0,
  
     'Authorization' : 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
  
     'Transfer-Encoding' : 'chunked',
  
     'Connection': 'keep-alive'
  
    }
  
  };
  
  
  
  var req = http.request(options, function(res) {});

  The device supports only continuous POST and only a couple of formats (ulaw, aiff, wav)

  This solution doesn’t seem to work. In the device speaker we only hear something like white noise.

  Also, I think I may have a problem with the buffer I am sending to the ffmpeg std in -> Tried to dump whatever comes out of the websocket to a .wav file then play it with VLC -> it plays everything in the record very fast -> 10 seconds of recording played in about 1 second.

  I am new to audio processing and have searched for about 3 days now for solutions on how to improve this and found nothing.

  I would ask from the community for 2 things :

  1. Is something wrong with my approach ? What more can I do to make this work ? I will post more details if required.

  2. If what I am doing is reinventing the wheel then I would like to know what other software / 3rd party service (like amazon or whatever) can accomplish the same thing.

  Thank you.

• Reason for write EPIPE error in my implementation ?

  25 mai 2019, par Chrisl446

  I currently have a small node.js express application for uploading images using sharp & thumbnails created from .mp4 videos using simple-thumbnail package/ffmpeg.

  The app works perfectly when uploading an image, the file is uploaded sharp processes it, then passes it along and it ends up in my amazon s3 bucket as expected. No errors what so ever.

  However, when I upload an mp4 video that I use to create and upload a thumbnail from by using simple-thumbnails genThumbnail() function, which uses ffmpeg child process, the thumbnail uploads successully to my s3 bucket, HOWEVER my app returns an EPIPE write error and NOT the url of the uploaded files url on S3.

  What is causing this and how can I fix it, considering it’s pretty much working aside from the EPIPE error that is being returned ? Thanks ahead !

  The packages of concern used are as follows :

  aws-sdk

  multer

  multer-s3 <- this one with the transform option, not the standard multer-s3 package

  simple-thumbnail

  const express = require('express');
  
  const app = express();
  
  
  
  const aws = require('aws-sdk');
  
  const multer = require('multer');
  
  const multerS3 = require('multer-s3'); //github:gmenih341/multer-s3 version of multer-s3 with transform option
  
  const sharp = require('sharp');
  
  const genThumbnail = require('simple-thumbnail');
  
  
  
  app.use((req, res, next) => {
  
      res.header('Access-Control-Allow-Origin', '*');
  
      res.header('Access-Control-Allow-Headers', 'Orgin, X-Requested-With, Content-Type, Accept, Authorization');
  
      if (req.method === 'OPTIONS') {
  
          res.header('Access-Control-Allow-Methods', 'POST');
  
          return res.status(200).json({});
  
      }
  
      next();
  
  });
  
  
  
  let uniqueFileName;
  
  let s3BucketName = 'bucketname';
  
  
  
  let s3 = new aws.S3({
  
      accessKeyId: ACCESS_KEY,
  
      secretAccessKey: SECRET_KEY,
  
      Bucket: s3BucketName
  
  });
  
  
  
  let upload = multer({
  
      storage: multerS3({
  
          s3: s3,
  
          bucket: s3BucketName,
  
          acl: 'public-read',
  
          cacheControl: 'max-age=31536000',
  
          contentType: multerS3.AUTO_CONTENT_TYPE,
  
          shouldTransform: true,
  
          transforms: [{
  
              id: 'thumbnail',
  
              key: function (req, file, cb) {
  
                  uniqueFileName = Date.now().toString();
  
                  cb(null, uniqueFileName + '.jpg')
  
              },
  
              transform: function (req, file, cb) {
  
                  if (file.mimetype == 'video/mp4') {
  
                      //When using simple-thumbnails' getThumbnail() on an mp4 video it uploads succesfully to S3 but node returns EPIPE write error
  
                      cb(null, genThumbnail(null, null, '250x?'))
  
                  } else {
  
                      //When using sharp to resize an image this works perfectly and retuns the JSON below with the files S3 URL
  
                      cb(null, sharp().jpeg())
  
                  }
  
  
  
              }
  
          }]
  
      })
  
  });
  
  
  
  app.post('/upload', upload.array('theFile'), (req, res) => {
  
      res.json({
  
          fileS3Url: 'https://s3.amazonaws.com/'+ s3BucketName +'/' + uniqueFileName
  
      });
  
  });
  
  
  
  app.use((req, res, next) => {
  
      const error = new Error('Not found');
  
      error.status = 404;
  
      next(error);
  
  });
  
  
  
  app.use((error, req, res, next) => {
  
      res.status(error.status || 500);
  
      res.json({
  
          error: {
  
              message: error.message
  
          }
  
      });
  
  });
  
  
  
  module.exports = app;