Recherche avancée

Sur d’autres sites (11531)

• Computer crashing when using python tools in same script

  5 février 2023, par SL1997

  I am attempting to use the speech recognition toolkit VOSK and the speech diarization package Resemblyzer to transcibe audio and then identify the speakers in the audio.

  


  Tools :

  


  https://github.com/alphacep/vosk-api
  
https://github.com/resemble-ai/Resemblyzer

  


  I can do both things individually but run into issues when trying to do them when running the one python script.

  


  I used the following guide when setting up the diarization system :

  


  https://medium.com/saarthi-ai/who-spoke-when-build-your-own-speaker-diarization-module-from-scratch-e7d725ee279

  


  Computer specs are as follows :

  


  Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz, 3912 Mhz, 2 Core(s), 4 Logical Processor(s)
  
32GB RAM

  


  The following is my code, I am not to sure if using threading is appropriate or if I even implemented it correctly, how can I best optimize this code as to achieve the results I am looking for and not crash.

  


  from vosk import Model, KaldiRecognizer
from pydub import AudioSegment
import json
import sys
import os
import subprocess
import datetime
from resemblyzer import preprocess_wav, VoiceEncoder
from pathlib import Path
from resemblyzer.hparams import sampling_rate
from spectralcluster import SpectralClusterer
import threading
import queue
import gc



def recognition(queue, audio, FRAME_RATE):

    model = Model("Vosk_Models/vosk-model-small-en-us-0.15")

    rec = KaldiRecognizer(model, FRAME_RATE)
    rec.SetWords(True)

    rec.AcceptWaveform(audio.raw_data)
    result = rec.Result()

    transcript = json.loads(result)#["text"]

    #return transcript
    queue.put(transcript)



def diarization(queue, audio):

    wav = preprocess_wav(audio)
    encoder = VoiceEncoder("cpu")
    _, cont_embeds, wav_splits = encoder.embed_utterance(wav, return_partials=True, rate=16)
    print(cont_embeds.shape)

    clusterer = SpectralClusterer(
        min_clusters=2,
        max_clusters=100,
        p_percentile=0.90,
        gaussian_blur_sigma=1)

    labels = clusterer.predict(cont_embeds)

    def create_labelling(labels, wav_splits):

        times = [((s.start + s.stop) / 2) / sampling_rate for s in wav_splits]
        labelling = []
        start_time = 0

        for i, time in enumerate(times):
            if i > 0 and labels[i] != labels[i - 1]:
                temp = [str(labels[i - 1]), start_time, time]
                labelling.append(tuple(temp))
                start_time = time
            if i == len(times) - 1:
                temp = [str(labels[i]), start_time, time]
                labelling.append(tuple(temp))

        return labelling

    #return
    labelling = create_labelling(labels, wav_splits)
    queue.put(labelling)



def identify_speaker(queue1, queue2):

    transcript = queue1.get()
    labelling = queue2.get()

    for speaker in labelling:

        speakerID = speaker[0]
        speakerStart = speaker[1]
        speakerEnd = speaker[2]

        result = transcript['result']
        words = [r['word'] for r in result if speakerStart < r['start'] < speakerEnd]
        #return
        print("Speaker",speakerID,":",' '.join(words), "\n")





def main():

    queue1 = queue.Queue()
    queue2 = queue.Queue()

    FRAME_RATE = 16000
    CHANNELS = 1

    podcast = AudioSegment.from_mp3("Podcast_Audio/Film-Release-Clip.mp3")
    podcast = podcast.set_channels(CHANNELS)
    podcast = podcast.set_frame_rate(FRAME_RATE)

    first_thread = threading.Thread(target=recognition, args=(queue1, podcast, FRAME_RATE))
    second_thread = threading.Thread(target=diarization, args=(queue2, podcast))
    third_thread = threading.Thread(target=identify_speaker, args=(queue1, queue2))

    first_thread.start()
    first_thread.join()
    gc.collect()

    second_thread.start()
    second_thread.join()
    gc.collect()

    third_thread.start()
    third_thread.join()
    gc.collect()

    # transcript = recognition(podcast,FRAME_RATE)
    #
    # labelling = diarization(podcast)
    #
    # print(identify_speaker(transcript, labelling))


if __name__ == '__main__':
    main()


  


  When I say crash I mean everything freezes, I have to hold down the power button on the desktop and turn it back on again. No blue/blank screen, just frozen in my IDE looking at my code. Any help in resolving this issue would be greatly appreciated.

  


• Compile FFmpeg with x264 for MacOS and Windows on Linux

  9 mars 2023, par RobinFrcd

  I successfully managed to compile a minimal standalone FFmpeg binary to create MP4 videos from JPG images encoded with x264. The binary is 100% functional and is 5.2MB.

  


  To do that, I used :

  


  ./configure \
--disable-everything \
--enable-decoder=mjpeg \
--enable-encoder=libx264 \
--enable-protocol=concat,file \
--enable-demuxer=image2 \
--enable-muxer=mp4 \
--enable-filter=scale \
--enable-gpl \
--enable-libx264 \
--extra-ldexeflags="-static" \
--pkg-config="pkg-config --static"


  


  I now would like to build the macOS and windows binaries directly from my Linux machine. I tried this repo and replaced the config args with mine, but the output exe is 30MB+. And I don't find anything about building for MacOS.

  


  Is there a solution to make this minimal build cross-platform compatible ?

  


• ffmpeg fast seek large MP4 over HTTP

  28 juillet 2024, par Gmanicus

  I'm attempting to download snapshots from a video provided by the U.S House of Representatives :

  


  https://houseliveprod-f9h4cpb9dyb8gegg.a01.azurefd.net/east/2024-04-11T08-55-12_Download/video_3000000_1.mp4


  


  I am using fluent-ffmpeg in Node to execute this command :

  


  ffmpeg('https://houseliveprod-f9h4cpb9dyb8gegg.a01.azurefd.net/east/2024-04-11T08-55-12_Download/video_3000000_1.mp4')
  .inputOption(`-ss 03:33:33`)
  .outputOptions([
     '-vframes 1'
  ])
  .output('test.png')

// Effectively:
// ffmpeg -ss 03:33:33 -i  -y -vframes 1 test.png


  


  My intention is to fast-seek to the desired timestamp and take a snapshot over HTTP. However, when doing so, the performance is not great. A snapshot takes about 10s per 3hrs of video and seems to increase fairly linearly at that rate.

  


  However, when using ffmpeg on the same video locally, it's super fast ! Sub-500ms regardless of the desired timestamp.

  


  Is there some magic that could be done via ffmpeg options or perhaps some sort of technique with manual requests to get a snapshot at the desired segment of video more efficiently ?