Recherche avancée

Sur d’autres sites (11301)

• Computer crashing when using python tools in same script

  5 février 2023, par SL1997

  I am attempting to use the speech recognition toolkit VOSK and the speech diarization package Resemblyzer to transcibe audio and then identify the speakers in the audio.

  


  Tools :

  


  https://github.com/alphacep/vosk-api
  
https://github.com/resemble-ai/Resemblyzer

  


  I can do both things individually but run into issues when trying to do them when running the one python script.

  


  I used the following guide when setting up the diarization system :

  


  https://medium.com/saarthi-ai/who-spoke-when-build-your-own-speaker-diarization-module-from-scratch-e7d725ee279

  


  Computer specs are as follows :

  


  Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz, 3912 Mhz, 2 Core(s), 4 Logical Processor(s)
  
32GB RAM

  


  The following is my code, I am not to sure if using threading is appropriate or if I even implemented it correctly, how can I best optimize this code as to achieve the results I am looking for and not crash.

  


  from vosk import Model, KaldiRecognizer
from pydub import AudioSegment
import json
import sys
import os
import subprocess
import datetime
from resemblyzer import preprocess_wav, VoiceEncoder
from pathlib import Path
from resemblyzer.hparams import sampling_rate
from spectralcluster import SpectralClusterer
import threading
import queue
import gc



def recognition(queue, audio, FRAME_RATE):

    model = Model("Vosk_Models/vosk-model-small-en-us-0.15")

    rec = KaldiRecognizer(model, FRAME_RATE)
    rec.SetWords(True)

    rec.AcceptWaveform(audio.raw_data)
    result = rec.Result()

    transcript = json.loads(result)#["text"]

    #return transcript
    queue.put(transcript)



def diarization(queue, audio):

    wav = preprocess_wav(audio)
    encoder = VoiceEncoder("cpu")
    _, cont_embeds, wav_splits = encoder.embed_utterance(wav, return_partials=True, rate=16)
    print(cont_embeds.shape)

    clusterer = SpectralClusterer(
        min_clusters=2,
        max_clusters=100,
        p_percentile=0.90,
        gaussian_blur_sigma=1)

    labels = clusterer.predict(cont_embeds)

    def create_labelling(labels, wav_splits):

        times = [((s.start + s.stop) / 2) / sampling_rate for s in wav_splits]
        labelling = []
        start_time = 0

        for i, time in enumerate(times):
            if i > 0 and labels[i] != labels[i - 1]:
                temp = [str(labels[i - 1]), start_time, time]
                labelling.append(tuple(temp))
                start_time = time
            if i == len(times) - 1:
                temp = [str(labels[i]), start_time, time]
                labelling.append(tuple(temp))

        return labelling

    #return
    labelling = create_labelling(labels, wav_splits)
    queue.put(labelling)



def identify_speaker(queue1, queue2):

    transcript = queue1.get()
    labelling = queue2.get()

    for speaker in labelling:

        speakerID = speaker[0]
        speakerStart = speaker[1]
        speakerEnd = speaker[2]

        result = transcript['result']
        words = [r['word'] for r in result if speakerStart < r['start'] < speakerEnd]
        #return
        print("Speaker",speakerID,":",' '.join(words), "\n")





def main():

    queue1 = queue.Queue()
    queue2 = queue.Queue()

    FRAME_RATE = 16000
    CHANNELS = 1

    podcast = AudioSegment.from_mp3("Podcast_Audio/Film-Release-Clip.mp3")
    podcast = podcast.set_channels(CHANNELS)
    podcast = podcast.set_frame_rate(FRAME_RATE)

    first_thread = threading.Thread(target=recognition, args=(queue1, podcast, FRAME_RATE))
    second_thread = threading.Thread(target=diarization, args=(queue2, podcast))
    third_thread = threading.Thread(target=identify_speaker, args=(queue1, queue2))

    first_thread.start()
    first_thread.join()
    gc.collect()

    second_thread.start()
    second_thread.join()
    gc.collect()

    third_thread.start()
    third_thread.join()
    gc.collect()

    # transcript = recognition(podcast,FRAME_RATE)
    #
    # labelling = diarization(podcast)
    #
    # print(identify_speaker(transcript, labelling))


if __name__ == '__main__':
    main()


  


  When I say crash I mean everything freezes, I have to hold down the power button on the desktop and turn it back on again. No blue/blank screen, just frozen in my IDE looking at my code. Any help in resolving this issue would be greatly appreciated.

  


• FFMPEG stereo track stops capturing at random times during a capture session

  26 mai 2022, par mrwassen

  I am currently working on building a workflow to capture and archive a large stash of family and friends PAL and NTSC VHS tapes. The hardware setup is as follows :

  


  

  • JVC HR-7860S VCR

  


  • s-video / RCA audio >

  


  • ADVC-3000 converter

  


  • SDI / BNC cable >

  


  • Blackmagic Decklink Mini Recorder 4K PCIe card

  


  • installed in a fairly hi-spec windows machine : AMD Ryzen 9 5900X 3.7 Ghz base 12 core, GEFORCE RTX 3060 12 gB, 32 gB ram

  


  


  The plan is to capture to lossless AVI, then drop into an NLE (Vegas Pro v.16) to do a minimal amount of cleanup / trimming, then render to a more compressed video format (TBD) for upload to AWS S3 accessible through a family website.

  


  The issue I am having is that when I run the capture using ffmpeg/directshow e.g. for a perfectly fine 90 min. PAL tape, at some random point of time during the capture one of the 2 stereo channels just stops capturing. This has happened with all of the tapes I have tested so far, and it happens at different times during the same video. I have examined the frames surrounding points in time when this happens, and it doesn't correlate to any transitions or jitter, but often just randomly in the middle of a perfectly smooth scene. Once the one channel stops capturing it never starts back up again during that capture session.

  


  The ADVC-3000 and the VCR are both showing both stereo channels playing normally throughout the capture. The windows machine running the capture hardly breaks a sweat at any time, and the transfer easily keeps up constantly showing a speed = 1x which I assume means nothing lagging. Also there are no video/audio sync issues at any point in time even towards the end of long tapes e.g. 90 mins.

  


  I am fairly new at ffmpeg, so I have spent extensive amounts of time reading up on forum posts and experimenting and have ended up with the following syntax :

  


  ffmpeg -y -f dshow -rtbufsize 2000M -i video="Blackmagic WDM Capture":audio="Blackmagic WDM Capture" -codec:v v210 -pix_fmt yuv422p -codec:a pcm_s16le -b:a 128k -t 02:00:00 -r 25 -threads 4 -maxrate 2500k -filter:a "volume=1.5" output_v210_audio.avi


  


  The capture runs without a single dropped frame, the only error I am getting when launching (and perhaps this is a smoking gun ?) is :

  


  


  "Non-monotonous DTS in output stream 0:1 ; previous : 0, current : -30 ;
changing to 1. This may result in incorrect timestamps in the output
file."

  


  


  I have tried to troubleshoot this in the hopes that it is tied to my issue but so far without luck.

  


  Hoping somebody can help correct or modify my command line or perhaps other ideas to help resolve the issue.

  


• ffmpeg, /dev/video0, -f decklink

  20 mars 2019, par Camille Goudeseune

  I’m trying to capture video from a PCI card, the Blackmagic DeckLink Mini Recorder, via ffmpeg, on a headless host running Ubuntu 18.04.2 LTS, hopefully with a command like

  ffmpeg -f decklink -i /dev/video0 ...

  How can I make that work ? I have two obstacles.

  No /dev/video0

  ffmpeg -i /dev/video0 ... fails : /dev/video0: No such device or address.
  v4l2-ctl --list-devices fails with the same error message.

  I built /dev/video0, and it looks okay :

  mknod /dev/video0 c 81 0
  
  chown root.video /dev/video0
  
  chmod g+rw /dev/video0

  To compare this file with a working one, I ran strace cat /dev/video0 on this host, and on another host (Ubuntu 14) with a working /dev/video0. The outputs began to differ here (good, then bad) :

  fstat(1, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
  
  open("/dev/video0", O_RDONLY)           = 3  
  
  fstat(3, {st_mode=S_IFCHR|0660, st_rdev=makedev(81, 0), ...}) = 0
  
  fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
  
  ----
  
  
  
  fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
  
  openat(AT_FDCWD, "/dev/video0", O_RDONLY) = -1 ENXIO (No such device or address)

  So /dev/video0 is broken at a level lower than ffmpeg or v4l2 or even cat.

  On Ubuntu 14, man 8 MAKEDEV suggests that the error message means that "the kernel does not have the driver configured or loaded."

  This Ubuntu 18 host lacks that manpage, but it does have a few /snap/core/*/sbin/MAKEDEV, all the same, so I tried

  /snap/core/6350/sbin/MAKEDEV -n -v video

  It would have created over a hundred devices videoXX, radioXX, vtxXX, vbiXX. Those devices didn’t exist yet, so it seemed harmless to try it.

  rm /dev/video0; /snap/core/6350/sbin/MAKEDEV video

  That rebuilt /dev/video0, but "No such device" remains, from cat or ffmpeg.

  No decklink

  ffmpeg -f decklink ... fails with Unknown input format: 'decklink'.

  Neither black nor deck nor link is mentioned by ffmpeg -devices (fbdev, lavfi, oss, v4l2) and ffmpeg -formats (about 350), either for Ubuntu’s own version 3.4.4-0ubuntu0.18.04.1, or for version N-93330-g7ff89574c7 compiled from source on 2019 Mar 13 :

  git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
  
  cd ffmpeg
  
  ./configure --enable-nonfree --disable-doc --disable-w32threads --enable-pthreads

  (Although ./configure --help mentions --enable-decklink, using that yielded "ERROR : DeckLinkAPI.h not found." updatedb && locate DeckLinkAPI.h finds no file with that name, either.)

  The DeckLink PCI card is recognized by hwinfo and lspci.

  lsmod reports the loaded modules blackmagic and blackmagic_io.

  Maybe the PCI card is installed ok, but ffmpeg just can’t reach it because I can’t configure it for that.

  Edit : Rebooting didn’t fix anything.