Recherche avancée

Sur d’autres sites (7381)

• Audio offset get wrong after some time when streaming audios

  7 septembre 2024, par Antoine Grenard

  I use microsoft-cognitiveservices-speech-sdk (1.38.0) in order to do real time speech to text.
It seems like the offset is right when I send a full audio but it is wrong when I send it cut in a lot of audio chunks.

  


  The more there is audio chunks the more inaccurate the offset is :

  


  

  • No chunks : 1 726 300 000

  


  • 369 chunks of 0.5 seconds : 1 729 600 000

  


  • 923 chunks of 0.2 seconds : 1 744 600 000

  


  • 1443 chunks of 0.1 seconds : 1 757 900 000

  


  


  To reproduce here is some piece of code :

  


      const speechConfig = SpeechConfig.fromSubscription(<key>,  {console.log(event)}&#xA;    speechRecognizer.canceled = async (recognizer, event) => {console.log(event)}&#xA;    speechRecognizer.startContinuousRecognitionAsync();&#xA;&#xA;    for (let i = 1; i &lt;= 1443; i&#x2B;&#x2B;) {&#xA;      const formattedNumber = i.toString().padStart(4, &#x27;0&#x27;);&#xA;      const buffer = fs.readFileSync(`/var/tmp/chunks/output_${formattedNumber}.wav`);&#xA;      pushStream.write(buffer);&#xA;    }&#xA;</key>

  


  To create the audio chunks :

  


  ffmpeg -i  -f segment -segment_time 0.1 -c copy output_%04d.wav


  


  Here is the audio link : https://drive.google.com/file/d/1H_RJuqMiBaVkpo9XHrgp1bpuFdgQl64O/view?usp=sharing

  


  Thanks for your help

  


• Synchronize video subtitle with text-to-speech voice

  8 décembre 2015, par Ahmad

  I try to create a video of a text in which the text is narrated by text-to-speech.

  To create the video file, I use the VideoFileWriter of Aforge.Net as the following :

  VideoWriter = new VideoFileWriter();
  
  
  
  VideoWriter.Open(CurVideoFile, (int)(Properties.Settings.Default.VideoWidth),
  
      (int)(Properties.Settings.Default.VideoHeight), 25, VideoCodec.MPEG4, 800000);

  To read aloud the text I use SpeechSynthesizer class and write the output to a wave stream

  AudioStream = new FileStream(CurAudioFile, FileMode.Create);
  
  synth.SetOutputToWaveStream(AudioStream);

  I want to highlight the word is spoken in the video, so I synchronize them by the SpeakProgress event :

      void synth_SpeakProgress(object sender, SpeakProgressEventArgs e)
  
      {
  
  
  
          curAuidoPosition = e.AudioPosition;
  
          using (Graphics g = Graphics.FromImage(Screen))
  
          {
  
               g.DrawString(e.Text,....); 
  
          }                    
  
          VideoWriter.WriteVideoFrame(Screen, curAuidoPosition);
  
      }

  And finally, I merge the video and audio using ffmpeg

  using (Process process = new Process())
  
  {
  
            process.StartInfo.FileName = exe_path;
  
            process.StartInfo.Arguments = string.Format(@"-i ""{0}"" -i ""{1}"" -y -acodec copy -vcodec copy ""{2}""",
  
                                             avi_path, mp3_path, output_file);
  
  ......

  The problem is that for some voices like Microsoft Hazel, Zira and David, the video is not synchronized with the audio, and the audio is much faster than the shown subtitle. In windows 7, it works for Mircrosoft Sam

  How can I synchronize them so that it works for any text-to-speech voices ?

• swscale : make yuv2interleavedX more asm-friendly

  26 avril 2020, par Nelson Gomez

  swscale : make yuv2interleavedX more asm-friendly
  Extracting information from SwsContext in assembly is difficult, and
  
  rearranging SwsContext just for asm access didn't look good. These
  
  functions only need a couple of fields from it anyway, so just make
  
  them parameters in their own right.
  Signed-off-by : Nelson Gomez <nelson.gomez@microsoft.com>
  

  • [DH] libswscale/output.c
  • [DH] libswscale/swscale_internal.h
  • [DH] libswscale/vscale.c