Recherche avancée

Sur d’autres sites (6473)

• Live audio using ffmpeg, javascript and nodejs

  8 novembre 2017, par klaus

  I am new to this thing. Please don’t hang me for the poor grammar. I am trying to create a proof of concept application which I will later extend. It does the following : We have a html page which asks for permission to use the microphone. We capture the microphone input and send it via websocket to a node js app.

  JS (Client) :

  var bufferSize = 4096;
  
  var socket = new WebSocket(URL);
  
  var myPCMProcessingNode = context.createScriptProcessor(bufferSize, 1, 1);
  
  myPCMProcessingNode.onaudioprocess = function(e) {
  
    var input = e.inputBuffer.getChannelData(0);
  
    socket.send(convertFloat32ToInt16(input));
  
  }
  
  
  
  function convertFloat32ToInt16(buffer) {
  
    l = buffer.length;
  
    buf = new Int16Array(l);
  
    while (l--) {
  
      buf[l] = Math.min(1, buffer[l])*0x7FFF;
  
    }
  
    return buf.buffer;
  
  }
  
  
  
  navigator.mediaDevices.getUserMedia({audio:true, video:false})
  
                                  .then(function(stream){
  
                                    var microphone = context.createMediaStreamSource(stream);
  
                                    microphone.connect(myPCMProcessingNode);
  
                                    myPCMProcessingNode.connect(context.destination);
  
                                  })
  
                                  .catch(function(e){});

  In the server we take each incoming buffer, run it through ffmpeg, and send what comes out of the std out to another device using the node js ’http’ POST. The device has a speaker. We are basically trying to create a 1 way audio link from the browser to the device.

  Node JS (Server) :

  var WebSocketServer = require('websocket').server;
  
  var http = require('http');
  
  var children = require('child_process');
  
  
  
  wsServer.on('request', function(request) {
  
    var connection = request.accept(null, request.origin);
  
    connection.on('message', function(message) {
  
      if (message.type === 'utf8') { /*NOP*/ }
  
      else if (message.type === 'binary') {
  
        ffm.stdin.write(message.binaryData);
  
      }
  
    });
  
    connection.on('close', function(reasonCode, description) {});
  
    connection.on('error', function(error) {});
  
  });
  
  
  
  var ffm = children.spawn(
  
      './ffmpeg.exe'
  
     ,'-stdin -f s16le -ar 48k -ac 2 -i pipe:0 -acodec pcm_u8 -ar 48000 -f aiff pipe:1'.split(' ')
  
  );
  
  
  
  ffm.on('exit',function(code,signal){});
  
  
  
  ffm.stdout.on('data', (data) => {
  
    req.write(data);
  
  });
  
  
  
  var options = {
  
    host: 'xxx.xxx.xxx.xxx',
  
    port: xxxx,
  
    path: '/path/to/service/on/device',
  
    method: 'POST',
  
    headers: {
  
     'Content-Type': 'application/octet-stream',
  
     'Content-Length': 0,
  
     'Authorization' : 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
  
     'Transfer-Encoding' : 'chunked',
  
     'Connection': 'keep-alive'
  
    }
  
  };
  
  
  
  var req = http.request(options, function(res) {});

  The device supports only continuous POST and only a couple of formats (ulaw, aiff, wav)

  This solution doesn’t seem to work. In the device speaker we only hear something like white noise.

  Also, I think I may have a problem with the buffer I am sending to the ffmpeg std in -> Tried to dump whatever comes out of the websocket to a .wav file then play it with VLC -> it plays everything in the record very fast -> 10 seconds of recording played in about 1 second.

  I am new to audio processing and have searched for about 3 days now for solutions on how to improve this and found nothing.

  I would ask from the community for 2 things :

  1. Is something wrong with my approach ? What more can I do to make this work ? I will post more details if required.

  2. If what I am doing is reinventing the wheel then I would like to know what other software / 3rd party service (like amazon or whatever) can accomplish the same thing.

  Thank you.

• Merge commit 'e3e8eab359238486dc233f7aa89b7bb3cb19ec38'

  11 novembre 2017, par James Almer

  Merge commit 'e3e8eab359238486dc233f7aa89b7bb3cb19ec38'
  * commit 'e3e8eab359238486dc233f7aa89b7bb3cb19ec38' :
  
    vaapi_h265 : Add support for AUD NAL units
  
    vaapi_h265 : Convert to use coded bitstream infrastructure
  
    vaapi_h264 : Add support for SEI recovery points
  
    vaapi_h264 : Add support for AUD NAL units
  
    vaapi_h264 : Convert to use coded bitstream infrastructure
  
    lavc : Add hevc_metadata bitstream filter
  
    lavc : Add h264_redundant_pps bitstream filter
  
    lavc : Add h264_metadata bitstream filter
  
    lavc : Add trace_headers bitstream filter
  
    lavc : Add coded bitstream read/write support for H.265
  
    lavc : Add coded bitstream read/write support for H.264
  
    lavc : Add coded bitstream read/write API
  
    pixfmt : Support chroma-derived and ictcp color matrices
  
    h264 : Add support for alternative transfer characterics SEI
  
    vaapi_encode : Move quality option to common code
  This commit is a noop, see
  
  9c878651dbc8c795894740af74670b591551f619
  
  8c34a2024da77b50470e62789e4859b45959932e
  
  f3571048669bf876681499f49e9df492f05f73c6
  
  6734eef6b8b464139fdc140ec9bc9e8d74173869
  
  b4c915f4b3e15c3e787e319b961e4389762f6309
  
  9b0c7aa0e446eceec96ba8f4009e004fad29fba3
  
  9c7d70b49b64aa5571772a7cdb9bc426174261e0
  
  a308872b049e33f69f4b629a06f47e3681906b93
  
  8b26306294ffe78cc73357e2ddd56dd463db50ab
  
  03f982bbca4211108477e772db9a339517ecde37
  
  2e29ca2a9f19ba9a5b189f322f38497d2e2e3db0
  
  281b68b0265953ab2623a39484d927a0e921c405
  
  0bc7575ced65bf4aa4678ac12d550aaf87890d0e
  
  00179664bccd1dd6fa0d1c40db453528757bf6f7
  
  038a51258c4c5d8b77f4f9efcce6f397e5755c24
  Merged-by : James Almer <jamrial@gmail.com>
  

• ffmpeg rtp streaming errors : RTP : dropping old packet received too late

  2 février 2019, par Андрей_RnD

  I start video transfer by means of ffmpeg like this :

  ffmpeg -f video4linux2 -i /dev/video0 -vcodec libx264 -preset ultrafast -crf 20 -tune zerolatency -s 800x600 -r 25 -b:v 0.9M -sdp_file video.sdp -f rtp rtp://192.168.10.24:5010    

  I reproduce this way :

  ffplay -protocol_whitelist file,udp,rtp video.sdp    

  Everything works just fine. Then I interrupt transmission and after a couple of seconds I resume. ffplay does not immediately start to reproduce but errors occur :

  ....
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 14 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 33 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 41 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 49 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 33 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 27 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 14 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 48 times
  
  [sdp @ 0x6ebf80] RTP: dropping old packet received too lateB f=1/1
  
      Last message repeated 34 times
  
  ......    

  After a some time, playback is restored, but it is too long. Is there a way to eliminate or minimize the occurrence of errors of this nature when the incoming stream is suspended, can options or something else ? Read the manual ffmpeg nothing worthwhile about this did not naryl .... ((((