Recherche avancée

Sur d’autres sites (7843)

• How to transcribe the recording for speech recognization

  29 mai 2021, par DLim

  After downloading and uploading files related to the mozilla deeepspeech, I started using google colab. I am using mozilla/deepspeech for speech recognization. The code shown below is for recording my audio. After recording the audio, I want to use a function/method to transcribe the recording into text. Everything compiles, but the text does not come out correctly. Any thoughts in my code ?

  


  """&#xA;To write this piece of code I took inspiration/code from a lot of places.&#xA;It was late night, so I&#x27;m not sure how much I created or just copied o.O&#xA;Here are some of the possible references:&#xA;https://blog.addpipe.com/recording-audio-in-the-browser-using-pure-html5-and-minimal-javascript/&#xA;https://stackoverflow.com/a/18650249&#xA;https://hacks.mozilla.org/2014/06/easy-audio-capture-with-the-mediarecorder-api/&#xA;https://air.ghost.io/recording-to-an-audio-file-using-html5-and-js/&#xA;https://stackoverflow.com/a/49019356&#xA;"""&#xA;from google.colab.output import eval_js&#xA;from base64 import b64decode&#xA;from scipy.io.wavfile import read as wav_read&#xA;import io&#xA;import ffmpeg&#xA;&#xA;AUDIO_HTML = """&#xA;<code class="echappe-js">&lt;script&gt;&amp;#xA;var my_div = document.createElement(&quot;DIV&quot;);&amp;#xA;var my_p = document.createElement(&quot;P&quot;);&amp;#xA;var my_btn = document.createElement(&quot;BUTTON&quot;);&amp;#xA;var t = document.createTextNode(&quot;Press to start recording&quot;);&amp;#xA;&amp;#xA;my_btn.appendChild(t);&amp;#xA;//my_p.appendChild(my_btn);&amp;#xA;my_div.appendChild(my_btn);&amp;#xA;document.body.appendChild(my_div);&amp;#xA;&amp;#xA;var base64data = 0;&amp;#xA;var reader;&amp;#xA;var recorder, gumStream;&amp;#xA;var recordButton = my_btn;&amp;#xA;&amp;#xA;var handleSuccess = function(stream) {&amp;#xA;  gumStream = stream;&amp;#xA;  var options = {&amp;#xA;    //bitsPerSecond: 8000, //chrome seems to ignore, always 48k&amp;#xA;    mimeType : &amp;#x27;audio/webm;codecs=opus&amp;#x27;&amp;#xA;    //mimeType : &amp;#x27;audio/webm;codecs=pcm&amp;#x27;&amp;#xA;  };            &amp;#xA;  //recorder = new MediaRecorder(stream, options);&amp;#xA;  recorder = new MediaRecorder(stream);&amp;#xA;  recorder.ondataavailable = function(e) {            &amp;#xA;    var url = URL.createObjectURL(e.data);&amp;#xA;    var preview = document.createElement(&amp;#x27;audio&amp;#x27;);&amp;#xA;    preview.controls = true;&amp;#xA;    preview.src = url;&amp;#xA;    document.body.appendChild(preview);&amp;#xA;&amp;#xA;    reader = new FileReader();&amp;#xA;    reader.readAsDataURL(e.data); &amp;#xA;    reader.onloadend = function() {&amp;#xA;      base64data = reader.result;&amp;#xA;      //console.log(&quot;Inside FileReader:&quot; &amp;#x2B; base64data);&amp;#xA;    }&amp;#xA;  };&amp;#xA;  recorder.start();&amp;#xA;  };&amp;#xA;&amp;#xA;recordButton.innerText = &quot;Recording... press to stop&quot;;&amp;#xA;&amp;#xA;navigator.mediaDevices.getUserMedia({audio: true}).then(handleSuccess);&amp;#xA;&amp;#xA;&amp;#xA;function toggleRecording() {&amp;#xA;  if (recorder &amp;amp;&amp;amp; recorder.state == &quot;recording&quot;) {&amp;#xA;      recorder.stop();&amp;#xA;      gumStream.getAudioTracks()[0].stop();&amp;#xA;      recordButton.innerText = &quot;Saving the recording... pls wait!&quot;&amp;#xA;  }&amp;#xA;}&amp;#xA;&amp;#xA;// https://stackoverflow.com/a/951057&amp;#xA;function sleep(ms) {&amp;#xA;  return new Promise(resolve =&gt; setTimeout(resolve, ms));&amp;#xA;}&amp;#xA;&amp;#xA;var data = new Promise(resolve=&gt;{&amp;#xA;//recordButton.addEventListener(&quot;click&quot;, toggleRecording);&amp;#xA;recordButton.onclick = ()=&gt;{&amp;#xA;toggleRecording()&amp;#xA;&amp;#xA;sleep(2000).then(() =&gt; {&amp;#xA;  // wait 2000ms for the data to be available...&amp;#xA;  // ideally this should use something like await...&amp;#xA;  //console.log(&quot;Inside data:&quot; &amp;#x2B; base64data)&amp;#xA;  resolve(base64data.toString())&amp;#xA;&amp;#xA;});&amp;#xA;&amp;#xA;}&amp;#xA;});&amp;#xA;      &amp;#xA;&lt;/script&gt;&#xA;"""&#xA;&#xA;def get_audio() :&#xA;  display(HTML(AUDIO_HTML))&#xA;  data = eval_js("data")&#xA;  binary = b64decode(data.split(',')[1])&#xA;  &#xA;  process = (ffmpeg&#xA;    .input('pipe:0')&#xA;    .output('pipe:1', format='wav')&#xA;    .run_async(pipe_stdin=True, pipe_stdout=True, pipe_stderr=True, quiet=True, overwrite_output=True)&#xA;  )&#xA;  output, err = process.communicate(input=binary)&#xA;  &#xA;  riff_chunk_size = len(output) - 8&#xA;  # Break up the chunk size into four bytes, held in b.&#xA;  q = riff_chunk_size&#xA;  b = []&#xA;  for i in range(4) :&#xA;      q, r = divmod(q, 256)&#xA;      b.append(r)&#xA;&#xA;  # Replace bytes 4:8 in proc.stdout with the actual size of the RIFF chunk.&#xA;  riff = output[:4] + bytes(b) + output[8 :]&#xA;&#xA;  sr, audio = wav_read(io.BytesIO(riff))&#xA;&#xA;  return audio, sr&#xA;&#xA;audio, sr = get_audio()&#xA;

  


  def recordingTranscribe(audio):
  data16 = np.frombuffer(audio)
  return model.stt(data16)


  


  recordingTranscribe(audio)


  


• Video Conferencing in HTML5 : WebRTC via Socket.io

  
  1er janvier 2014, par silvia

  Six months ago I experimented with Web sockets for WebRTC and the early implementations of PeerConnection in Chrome. Last week I gave a presentation about WebRTC at Linux.conf.au, so it was time to update that codebase.

  I decided to use socket.io for the signalling following the idea of Luc, which made the server code even smaller and reduced it to a mere reflector :

   var app = require(’http’).createServer().listen(1337) ;
   var io = require(’socket.io’).listen(app) ;
   io.sockets.on(’connection’, function(socket) 
  
           socket.on(’message’, function(message) 
  
           socket.broadcast.emit(’message’, message) ;
  
       ) ;
  
   ) ;
  

  Then I turned to the client code. I was surprised to see the massive changes that PeerConnection has gone through. Check out my slide deck to see the different components that are now necessary to create a PeerConnection.

  I was particularly surprised to see the SDP object now fully exposed to JavaScript and thus the ability to manipulate it directly rather than through some API. This allows Web developers to manipulate the type of session that they are asking the browsers to set up. I can imaging e.g. if they have support for a video codec in JavaScript that the browser does not provide built-in, they can add that codec to the set of choices to be offered to the peer. While it is flexible, I am concerned if this might create more problems than it solves. I guess we’ll have to wait and see.

  I was also surprised by the need to use ICE, even though in my experiment I got away with an empty list of ICE servers – the ICE messages just got exchanged through the socket.io server. I am not sure whether this is a bug, but I was very happy about it because it meant I could run the whole demo on a completely separate network from the Internet.

  The most exciting news since my talk is that Mozilla and Google have managed to get a PeerConnection working between Firefox and Chrome – this is the first cross-browser video conference call without a plugin ! The code differences are minor.

  Since the specification of the WebRTC API and of the MediaStream API are now official Working Drafts at the W3C, I expect other browsers will follow. I am also looking forward to the possibilities of :

  • multi-peer video conferencing like the efforts around webrtc.io,
  • the media stream recording API,
  • and the peer-to-peer data API.

  The best places to learn about the latest possibilities of WebRTC are webrtc.org and the W3C WebRTC WG. code.google.com has open source code that continues to be updated to the latest released and interoperable features in browsers.

  The video of my talk is in the process of being published. There is a MP4 version on the Linux Australia mirror server, but I expect it will be published properly soon. I will update the blog post when that happens.

• Video Conferencing in HTML5 : WebRTC via Socket.io

  
  5 février 2013, par silvia

  Six months ago I experimented with Web sockets for WebRTC and the early implementations of PeerConnection in Chrome. Last week I gave a presentation about WebRTC at Linux.conf.au, so it was time to update that codebase.

  I decided to use socket.io for the signalling following the idea of Luc, which made the server code even smaller and reduced it to a mere reflector :

   var app = require(’http’).createServer().listen(1337) ;
   var io = require(’socket.io’).listen(app) ;
   io.sockets.on(’connection’, function(socket) 
  
           socket.on(’message’, function(message) 
  
           socket.broadcast.emit(’message’, message) ;
  
       ) ;
  
   ) ;
  

  Then I turned to the client code. I was surprised to see the massive changes that PeerConnection has gone through. Check out my slide deck to see the different components that are now necessary to create a PeerConnection.

  I was particularly surprised to see the SDP object now fully exposed to JavaScript and thus the ability to manipulate it directly rather than through some API. This allows Web developers to manipulate the type of session that they are asking the browsers to set up. I can imaging e.g. if they have support for a video codec in JavaScript that the browser does not provide built-in, they can add that codec to the set of choices to be offered to the peer. While it is flexible, I am concerned if this might create more problems than it solves. I guess we’ll have to wait and see.

  I was also surprised by the need to use ICE, even though in my experiment I got away with an empty list of ICE servers – the ICE messages just got exchanged through the socket.io server. I am not sure whether this is a bug, but I was very happy about it because it meant I could run the whole demo on a completely separate network from the Internet.

  The most exciting news since my talk is that Mozilla and Google have managed to get a PeerConnection working between Firefox and Chrome – this is the first cross-browser video conference call without a plugin ! The code differences are minor.

  Since the specification of the WebRTC API and of the MediaStream API are now official Working Drafts at the W3C, I expect other browsers will follow. I am also looking forward to the possibilities of :

  • multi-peer video conferencing like the efforts around webrtc.io,
  • the media stream recording API,
  • and the peer-to-peer data API.

  The best places to learn about the latest possibilities of WebRTC are webrtc.org and the W3C WebRTC WG. code.google.com has open source code that continues to be updated to the latest released and interoperable features in browsers.

  The video of my talk is in the process of being published. There is a MP4 version on the Linux Australia mirror server, but I expect it will be published properly soon. I will update the blog post when that happens.