Recherche avancée

Sur d’autres sites (6865)

• Synchronize video subtitle with text-to-speech voice

  8 décembre 2015, par Ahmad

  I try to create a video of a text in which the text is narrated by text-to-speech.

  To create the video file, I use the VideoFileWriter of Aforge.Net as the following :

  VideoWriter = new VideoFileWriter();
  
  
  
  VideoWriter.Open(CurVideoFile, (int)(Properties.Settings.Default.VideoWidth),
  
      (int)(Properties.Settings.Default.VideoHeight), 25, VideoCodec.MPEG4, 800000);

  To read aloud the text I use SpeechSynthesizer class and write the output to a wave stream

  AudioStream = new FileStream(CurAudioFile, FileMode.Create);
  
  synth.SetOutputToWaveStream(AudioStream);

  I want to highlight the word is spoken in the video, so I synchronize them by the SpeakProgress event :

      void synth_SpeakProgress(object sender, SpeakProgressEventArgs e)
  
      {
  
  
  
          curAuidoPosition = e.AudioPosition;
  
          using (Graphics g = Graphics.FromImage(Screen))
  
          {
  
               g.DrawString(e.Text,....); 
  
          }                    
  
          VideoWriter.WriteVideoFrame(Screen, curAuidoPosition);
  
      }

  And finally, I merge the video and audio using ffmpeg

  using (Process process = new Process())
  
  {
  
            process.StartInfo.FileName = exe_path;
  
            process.StartInfo.Arguments = string.Format(@"-i ""{0}"" -i ""{1}"" -y -acodec copy -vcodec copy ""{2}""",
  
                                             avi_path, mp3_path, output_file);
  
  ......

  The problem is that for some voices like Microsoft Hazel, Zira and David, the video is not synchronized with the audio, and the audio is much faster than the shown subtitle. In windows 7, it works for Mircrosoft Sam

  How can I synchronize them so that it works for any text-to-speech voices ?

• ffmpeg works at command line but I got this error when I use it in nodejs app :

  15 avril 2023, par Ibrahim Ashour

  This is the code :

  


  import fs from 'fs';
import googleTTS from 'google-tts-api';
import https from 'https';
import { exec } from 'child_process';

async function convertTextToVoice(text, fileName) {
  const outputFile = `./audios/${fileName}.mp3`;

  const words = text.split(' ');
  const chunks = [];
  let chunk = '';

  for (const word of words) {
    if (chunk.length + word.length < 200) {
      chunk += ' ' + word;
    } else {
      chunks.push(chunk.trim());
      chunk = word;
    }
  }

  chunks.push(chunk.trim());

  for (let i = 0; i < chunks.length; i++) {
    const chunkFile = `./audios/${fileName}-chunk${i}.mp3`;
    const url = await googleTTS.getAudioUrl(chunks[i], {
      lang: 'ar',
      slow: false,
      host: 'https://translate.google.com',
    });


    try {
      const file = await fs.createWriteStream(chunkFile);
      https.get(url, response => {
        response.pipe(file);
        file.on('finish', () => {
          file.close();
          console.log(`Chunk ${i} created successfully`);
        });
      }).on('error', error => {
        console.error(`Error downloading chunk ${i}: ${error}`);
      });
    } catch (error) {
      console.error(error);
    }
  }

  const chunkFiles = chunks.map((chunk, i) => `./audios/${fileName}-chunk${i}.mp3`);

  // Use ffmpeg to merge the audio files
  const concatCommand = `ffmpeg -i "concat:${chunkFiles.join('|')}" -acodec copy ${outputFile}`;
  try {
    await new Promise((resolve, reject) => {
      exec(concatCommand, (error, stdout, stderr) => {
        if (error) {
          console.error(`Error executing ffmpeg command: ${error}`);
          reject(error);
        } else {
          console.log(`Audio file ${outputFile} created successfully`);
          resolve();
        }
      });
    });

    for (const chunkFile of chunkFiles) {
      await fs.promises.unlink(chunkFile);
      console.log(`${chunkFile} deleted successfully`);
    }
  } catch (error) {
    console.error(error);
  }
}


  


  This is the Error :
./audios/تجربة-chunk0.mp3|./audios/تجربة-chunk1.mp3
ffmpeg -i "concat :./audios/تجربة-chunk0.mp3|./audios/تجربة-chunk1.mp3" -acodec copy ./audios/تجربة.mp3
Error executing ffmpeg command : Error : Command failed : ffmpeg -i "concat :./audios/تجربة-chunk0.mp3|./audios/تجربة-chunk1.mp3" -acodec copy ./audios/تجربة.mp3
ffmpeg version 2023-02-04-git-bdc76f467f-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration : —enable-gpl —enable-version3 —enable-static —disable-w32threads —disable-autodetect —enable-fontconfig —enable-iconv —enable-gnutls —enable-libxml2 —enable-gmp —enable-bzlib —enable-lzma —enable-libsnappy —enable-zlib —enable-librist —enable-libsrt —enable-libssh —enable-libzmq —enable-avisynth —enable-libbluray —enable-libcaca —enable-sdl2 —enable-libaribb24 —enable-libdav1d —enable-libdavs2 —enable-libuavs3d —enable-libzvbi —enable-librav1e —enable-libsvtav1 —enable-libwebp —enable-libx264 —enable-libx265 —enable-libxavs2 —enable-libxvid —enable-libaom —enable-libjxl —enable-libopenjpeg —enable-libvpx —enable-mediafoundation —enable-libass —enable-frei0r —enable-libfreetype —enable-libfribidi —enable-liblensfun —enable-libvidstab —enable-libvmaf —enable-libzimg —enable-amf —enable-cuda-llvm —enable-cuvid —enable-ffnvcodec —enable-nvdec —enable-nvenc —enable-d3d11va —enable-dxva2 —enable-libvpl —enable-libshaderc —enable-vulkan —enable-libplacebo —enable-opencl —enable-libcdio —enable-libgme —enable-libmodplug —enable-libopenmpt —enable-libopencore-amrwb —enable-libmp3lame —enable-libshine —enable-libtheora —enable-libtwolame —enable-libvo-amrwbenc —enable-libilbc —enable-libgsm —enable-libopencore-amrnb —enable-libopus —enable-libspeex —enable-libvorbis —enable-ladspa —enable-libbs2b —enable-libflite —enable-libmysofa —enable-librubberband —enable-libsoxr —enable-chromaprint
libavutil 57. 44.100 / 57. 44.100
libavcodec 59. 63.100 / 59. 63.100
libavformat 59. 38.100 / 59. 38.100
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 56.100 / 8. 56.100
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100
[mp3 @ 00000177df48a1c0] Format mp3 detected only with low score of 1, misdetection possible !
[mp3 @ 00000177df48a1c0] Failed to read frame size : Could not seek to 1026.
concat :./audios/تجربة-chunk0.mp3|./audios/تجربة-chunk1.mp3 : Invalid argument

  


  Error : Command failed : ffmpeg -i "concat :./audios/تجربة-chunk0.mp3|./audios/تجربة-chunk1.mp3" -acodec copy ./audios/تجربة.mp3
ffmpeg version 2023-02-04-git-bdc76f467f-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration : —enable-gpl —enable-version3 —enable-static —disable-w32threads —disable-autodetect —enable-fontconfig —enable-iconv —enable-gnutls —enable-libxml2 —enable-gmp —enable-bzlib —enable-lzma —enable-libsnappy —enable-zlib —enable-librist —enable-libsrt —enable-libssh —enable-libzmq —enable-avisynth —enable-libbluray —enable-libcaca —enable-sdl2 —enable-libaribb24 —enable-libdav1d —enable-libdavs2 —enable-libuavs3d —enable-libzvbi —enable-librav1e —enable-libsvtav1 —enable-libwebp —enable-libx264 —enable-libx265 —enable-libxavs2 —enable-libxvid —enable-libaom —enable-libjxl —enable-libopenjpeg —enable-libvpx —enable-mediafoundation —enable-libass —enable-frei0r —enable-libfreetype —enable-libfribidi —enable-liblensfun —enable-libvidstab —enable-libvmaf —enable-libzimg —enable-amf —enable-cuda-llvm —enable-cuvid —enable-ffnvcodec —enable-nvdec —enable-nvenc —enable-d3d11va —enable-dxva2 —enable-libvpl —enable-libshaderc —enable-vulkan —enable-libplacebo —enable-opencl —enable-libcdio —enable-libgme —enable-libmodplug —enable-libopenmpt —enable-libopencore-amrwb —enable-libmp3lame —enable-libshine —enable-libtheora —enable-libtwolame —enable-libvo-amrwbenc —enable-libilbc —enable-libgsm —enable-libopencore-amrnb —enable-libopus —enable-libspeex —enable-libvorbis —enable-ladspa —enable-libbs2b —enable-libflite —enable-libmysofa —enable-librubberband —enable-libsoxr —enable-chromaprint
libavutil 57. 44.100 / 57. 44.100
libavcodec 59. 63.100 / 59. 63.100
libavformat 59. 38.100 / 59. 38.100
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 56.100 / 8. 56.100
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100
[mp3 @ 00000177df48a1c0] Format mp3 detected only with low score of 1, misdetection possible !
[mp3 @ 00000177df48a1c0] Failed to read frame size : Could not seek to 1026.
concat :./audios/تجربة-chunk0.mp3|./audios/تجربة-chunk1.mp3 : Invalid argument

  


  at ChildProcess.exithandler (node:child_process:419:12)
at ChildProcess.emit (node:events:513:28)
at maybeClose (node:internal/child_process:1091:16)
at ChildProcess._handle.onexit (node:internal/child_process:302:5) {


  


  code : 1,
killed : false,
signal : null,
cmd : 'ffmpeg -i "concat :./audios/تجربة-chunk0.mp3|./audios/تجربة-chunk1.mp3" -acodec copy ./audios/تجربة.mp3'
}
Chunk 1 created successfully
Chunk 0 created successfully

  


  I expected the command to work in the nodejs app like in the command line

  


• avcodec/ac3_parser : improve false positive detection when parsing sync frames

  22 octobre 2022, par James Almer

  avcodec/ac3_parser : improve false positive detection when parsing sync frames
  A two byte sync word is not enough to ensure we got a real syncframe, nor are
  
  all the range checks we do in the first seven bytes. Do therefore an integrity
  
  check for the sync frame in order to prevent the parser from filling avctx with
  
  bogus information.
  Signed-off-by : James Almer <jamrial@gmail.com>
  

  • [DH] libavcodec/aac_ac3_parser.c
  • [DH] libavcodec/aac_ac3_parser.h
  • [DH] libavcodec/ac3_parser.c