Recherche avancée

Sur d’autres sites (7334)

• Computer crashing when using python tools in same script

  5 février 2023, par SL1997

  I am attempting to use the speech recognition toolkit VOSK and the speech diarization package Resemblyzer to transcibe audio and then identify the speakers in the audio.

  


  Tools :

  


  https://github.com/alphacep/vosk-api
  
https://github.com/resemble-ai/Resemblyzer

  


  I can do both things individually but run into issues when trying to do them when running the one python script.

  


  I used the following guide when setting up the diarization system :

  


  https://medium.com/saarthi-ai/who-spoke-when-build-your-own-speaker-diarization-module-from-scratch-e7d725ee279

  


  Computer specs are as follows :

  


  Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz, 3912 Mhz, 2 Core(s), 4 Logical Processor(s)
  
32GB RAM

  


  The following is my code, I am not to sure if using threading is appropriate or if I even implemented it correctly, how can I best optimize this code as to achieve the results I am looking for and not crash.

  


  from vosk import Model, KaldiRecognizer
from pydub import AudioSegment
import json
import sys
import os
import subprocess
import datetime
from resemblyzer import preprocess_wav, VoiceEncoder
from pathlib import Path
from resemblyzer.hparams import sampling_rate
from spectralcluster import SpectralClusterer
import threading
import queue
import gc



def recognition(queue, audio, FRAME_RATE):

    model = Model("Vosk_Models/vosk-model-small-en-us-0.15")

    rec = KaldiRecognizer(model, FRAME_RATE)
    rec.SetWords(True)

    rec.AcceptWaveform(audio.raw_data)
    result = rec.Result()

    transcript = json.loads(result)#["text"]

    #return transcript
    queue.put(transcript)



def diarization(queue, audio):

    wav = preprocess_wav(audio)
    encoder = VoiceEncoder("cpu")
    _, cont_embeds, wav_splits = encoder.embed_utterance(wav, return_partials=True, rate=16)
    print(cont_embeds.shape)

    clusterer = SpectralClusterer(
        min_clusters=2,
        max_clusters=100,
        p_percentile=0.90,
        gaussian_blur_sigma=1)

    labels = clusterer.predict(cont_embeds)

    def create_labelling(labels, wav_splits):

        times = [((s.start + s.stop) / 2) / sampling_rate for s in wav_splits]
        labelling = []
        start_time = 0

        for i, time in enumerate(times):
            if i > 0 and labels[i] != labels[i - 1]:
                temp = [str(labels[i - 1]), start_time, time]
                labelling.append(tuple(temp))
                start_time = time
            if i == len(times) - 1:
                temp = [str(labels[i]), start_time, time]
                labelling.append(tuple(temp))

        return labelling

    #return
    labelling = create_labelling(labels, wav_splits)
    queue.put(labelling)



def identify_speaker(queue1, queue2):

    transcript = queue1.get()
    labelling = queue2.get()

    for speaker in labelling:

        speakerID = speaker[0]
        speakerStart = speaker[1]
        speakerEnd = speaker[2]

        result = transcript['result']
        words = [r['word'] for r in result if speakerStart < r['start'] < speakerEnd]
        #return
        print("Speaker",speakerID,":",' '.join(words), "\n")





def main():

    queue1 = queue.Queue()
    queue2 = queue.Queue()

    FRAME_RATE = 16000
    CHANNELS = 1

    podcast = AudioSegment.from_mp3("Podcast_Audio/Film-Release-Clip.mp3")
    podcast = podcast.set_channels(CHANNELS)
    podcast = podcast.set_frame_rate(FRAME_RATE)

    first_thread = threading.Thread(target=recognition, args=(queue1, podcast, FRAME_RATE))
    second_thread = threading.Thread(target=diarization, args=(queue2, podcast))
    third_thread = threading.Thread(target=identify_speaker, args=(queue1, queue2))

    first_thread.start()
    first_thread.join()
    gc.collect()

    second_thread.start()
    second_thread.join()
    gc.collect()

    third_thread.start()
    third_thread.join()
    gc.collect()

    # transcript = recognition(podcast,FRAME_RATE)
    #
    # labelling = diarization(podcast)
    #
    # print(identify_speaker(transcript, labelling))


if __name__ == '__main__':
    main()


  


  When I say crash I mean everything freezes, I have to hold down the power button on the desktop and turn it back on again. No blue/blank screen, just frozen in my IDE looking at my code. Any help in resolving this issue would be greatly appreciated.

  


• Discord Bot can't play audio from YouTube all of a sudden

  22 mars 2023, par Aidan Tweedy

  Last night my Discord bot stopped working out of the blue. The bot had been working perfectly for months until this point - no code changes occurred and it was running in a Docker container on my home server. Then last night, my !play <url></url> command stopped working. The bot would join the voice channel, but not actually play anything. Then it would need to be manually disconnected from the voice channel in order for it to do anything again. After the first play fails, and I try again I get this error message :

  


   [youtube] crOZk88eCcg: Downloading webpage
 [youtube] crOZk88eCcg: Downloading android player API JSON
 [youtube] Extracting URL: https://www.youtube.com/watch?v=crOZk88eCcg&ab_channel=0foofighter0
 [youtube] crOZk88eCcg: Downloading webpage
 [youtube] crOZk88eCcg: Downloading android player API JSON
 [2023-03-22 00:27:59] [ERROR   ] discord.ext.commands.bot: Ignoring exception in command play
 Traceback (most recent call last):
   File "/usr/local/lib/python3.10/dist-packages/discord/ext/commands/core.py", line 190, in wrapped
     ret = await coro(*args, **kwargs)
   File "/usr/src/app/./main.py", line 180, in play
     voice.play(player, after=lambda e: print(f'Player error: {e}') if e else None)
   File "/usr/local/lib/python3.10/dist-packages/discord/voice_client.py", line 600, in play
     raise ClientException('Not connected to voice.')
 discord.errors.ClientException: Not connected to voice.
 
 The above exception was the direct cause of the following exception:
 
 Traceback (most recent call last):
   File "/usr/local/lib/python3.10/dist-packages/discord/ext/commands/bot.py", line 1347, in invoke
     await ctx.command.invoke(ctx)
   File "/usr/local/lib/python3.10/dist-packages/discord/ext/commands/core.py", line 986, in invoke
     await injected(*ctx.args, **ctx.kwargs)  # type: ignore
   File "/usr/local/lib/python3.10/dist-packages/discord/ext/commands/core.py", line 199, in wrapped
     raise CommandInvokeError(exc) from exc
 discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ClientException: Not connected to voice.
 [2023-03-22 00:27:59] [INFO    ] discord.player: ffmpeg process 12 has not terminated. Waiting to terminate...
 [2023-03-22 00:27:59] [INFO    ] discord.player: ffmpeg process 12 should have terminated with a return code of -9.


  


  My first instinct was that ffmpeg was borking somehow, since it has been difficult to get working in the past. However I'm not sure how ffmpeg could've stopped working, since the bot never went down between working and not working. That leads me to believe it could be a change on Youtube/Discord's side ?

  


  For context, here is a snippet of my !play code :

  


  @client.command(pass_context = True)
async def play(ctx):
    url = ctx.message.content.split("!play ",1)[1]
    voice = discord.utils.get(client.voice_clients)
    if (ctx.author.voice):
        if voice == None:
            channel = ctx.message.author.voice.channel
            voice = await channel.connect()

        # Youtube Magic
        ydl_opts = {
        'format': 'worstaudio/worst',
        'postprocessors': [{
            'key': 'FFmpegExtractAudio',
            'preferredcodec': 'mp3',
            'preferredquality': '192',
        }],
        }

        if "http" not in url:
            yt = YoutubeSearch(url, max_results=1).to_json()
            print(yt)
            yt_id = str(json.loads(yt)['videos'][0]['id'])
            yt_channel = str(json.loads(yt)['videos'][0]['channel']).strip().replace(" ", "")
            url = 'https://www.youtube.com/watch?v='+yt_id+"&ab_channel="+yt_channel

        with yt_dlp.YoutubeDL(ydl_opts) as ydl:
            song_info=ydl.extract_info(url, download=False)
        for file in os.listdir("./"):
            if file.endswith(".mp3"):
                os.rename(file, "song.mp3")

        voice.stop()

        embed_trk = discord.Embed(
                title="Now playing",
                color=ctx.author.color,
        )
        video_title = song_info.get('title', None)
        video_channel = song_info.get('uploader', None)
        embed_trk.add_field(name="Track title", value=video_title, inline=False)
        embed_trk.add_field(name="Channel", value=video_channel, inline=False)

        await ctx.send(embed=embed_trk)
        player = await YTDLSource.from_url(url, loop=client.loop, stream=True)
        voice.play(player, after=lambda e: print(f'Player error: {e}') if e else None)
        while voice.is_playing():
            await asyncio.sleep(1)
        
        await request_record(ctx, video_title)

    else:
        await ctx.send("You must be in a voice channel to run this command")


  


• Is there a built-in audio preprocessing within Android OS ?

  29 août 2022, par Larry lu

  I am running a dev team to build an Android application the major feature is to record a user's voice and merge it into a long movie. It's a dub film application.

  


  One struggle thing to my dev team is whether the Android OS pre-processes the recorded audio file. because we want the very original audio data from the built-in Microphone.
If there is a built-in pre-processing inside of Android OS that keeps updating some data of the recorded audio file, we want to know what is changed. We can't find any official post from Android speaking about it but would like to confirm from a professional.

  


  Many thanks