Recherche avancée

Sur d’autres sites (15666)

• Realtime video processing with javacv on Android

  7 août 2014, par moonie

  I wish to do realtime video processing on Android using opencv/javacv. My basic approach can be summarized as follow :

  onCreate(), generating a new JavaCameraView

  mCameraView = new JavaCameraView(this, mCameraIndex);
  
  mCameraView.setCvCameraViewListener(this);   
  
  setContentView(mCameraView); 

  when the user clicks a button, a ffmpegframe recorder is created

  try{
  
              final long currentTimeMillis = System.currentTimeMillis();
  
              final String appName=getString(R.string.app_name);
  
              final String galleryPath=Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).toString();
  
              final String albumPath=galleryPath+"/"+appName;
  
              final String photoPath=albumPath+"/"+currentTimeMillis+".avi";
  
              FFmpegFrameRecorder recorder=FFmpegFrameRecorder.createDefault(photoPath, 640, 360);
  
              recorder.setFrameRate(30);
  
              recorder.start();
  
              }catch(FrameRecorder.Exception e){
  
              e.printStackTrace();
  
              }

  then for each CameraFrame, use the following code to process and save the frame

      public Mat onCameraFrame(final CvCameraViewFrame inputFrame){
  
      final Mat rgba = inputFrame.rgba();
  
      //...apply some filters on rgba
  
      if(mIsTakingVideo){
  
          try{
  
            recorder.record(MatToIplImage(rgba,640,320));           
  
          }catch(FrameRecorder.Exception {
  
              e.printStackTrace();
  
              }       
  
      }
  
      return rgba;
  
  }
  
  
  
   public IplImage MatToIplImage(Mat m){
  
    Bitmap bmp=Bitmap.createBitmap(m.width(), m.height(), Config.ARGB_8888);
  
    Utils.matToBitmap(m, bmp);
  
    IplImage image=IplImage.create(m.width(),m.height(),IPL_DEPTH_8U,1);
  
    bmp.copyPixelsToBuffer(image.getByteBuffer());
  
    return image;}

  When the user clicks the button again, the recorder is stopped and released. However the video I create is an empty video. Any idea why ?

  EDIT :
  The video was empty because the format avi is not supported on my test phone...
  Now the video is not empty but it has the following errors

  1. The recorded video has two split windows instead of just one window
  2. The color is very different from the object I recorded
  3. The play rate is too fast.

• How can I rename a file from a txt file with Windows bat file ?

  12 septembre 2022, par user1264599

  I have a batch script that renames a file to input.mkv so it can be processed by a string of other commands in the bat file with a final file called ProcessedVideo.mkv. I capture the OG file name using "dir *.mkv /b>OG_FileName.txt" before being renamed.

  


  How can I rename the final processed mkv file to the name captured in the OG_FileName.txt and maybe add "_Added-Text.mkv" as the last part of my Batch Script ? (Adding text to the file name is not that important if it is too much trouble).

  


  I really thought this would be easy but I'm defeated.

  


• Computer crashing when using python tools in same script

  5 février 2023, par SL1997

  I am attempting to use the speech recognition toolkit VOSK and the speech diarization package Resemblyzer to transcibe audio and then identify the speakers in the audio.

  


  Tools :

  


  https://github.com/alphacep/vosk-api
  
https://github.com/resemble-ai/Resemblyzer

  


  I can do both things individually but run into issues when trying to do them when running the one python script.

  


  I used the following guide when setting up the diarization system :

  


  https://medium.com/saarthi-ai/who-spoke-when-build-your-own-speaker-diarization-module-from-scratch-e7d725ee279

  


  Computer specs are as follows :

  


  Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz, 3912 Mhz, 2 Core(s), 4 Logical Processor(s)
  
32GB RAM

  


  The following is my code, I am not to sure if using threading is appropriate or if I even implemented it correctly, how can I best optimize this code as to achieve the results I am looking for and not crash.

  


  from vosk import Model, KaldiRecognizer
from pydub import AudioSegment
import json
import sys
import os
import subprocess
import datetime
from resemblyzer import preprocess_wav, VoiceEncoder
from pathlib import Path
from resemblyzer.hparams import sampling_rate
from spectralcluster import SpectralClusterer
import threading
import queue
import gc



def recognition(queue, audio, FRAME_RATE):

    model = Model("Vosk_Models/vosk-model-small-en-us-0.15")

    rec = KaldiRecognizer(model, FRAME_RATE)
    rec.SetWords(True)

    rec.AcceptWaveform(audio.raw_data)
    result = rec.Result()

    transcript = json.loads(result)#["text"]

    #return transcript
    queue.put(transcript)



def diarization(queue, audio):

    wav = preprocess_wav(audio)
    encoder = VoiceEncoder("cpu")
    _, cont_embeds, wav_splits = encoder.embed_utterance(wav, return_partials=True, rate=16)
    print(cont_embeds.shape)

    clusterer = SpectralClusterer(
        min_clusters=2,
        max_clusters=100,
        p_percentile=0.90,
        gaussian_blur_sigma=1)

    labels = clusterer.predict(cont_embeds)

    def create_labelling(labels, wav_splits):

        times = [((s.start + s.stop) / 2) / sampling_rate for s in wav_splits]
        labelling = []
        start_time = 0

        for i, time in enumerate(times):
            if i > 0 and labels[i] != labels[i - 1]:
                temp = [str(labels[i - 1]), start_time, time]
                labelling.append(tuple(temp))
                start_time = time
            if i == len(times) - 1:
                temp = [str(labels[i]), start_time, time]
                labelling.append(tuple(temp))

        return labelling

    #return
    labelling = create_labelling(labels, wav_splits)
    queue.put(labelling)



def identify_speaker(queue1, queue2):

    transcript = queue1.get()
    labelling = queue2.get()

    for speaker in labelling:

        speakerID = speaker[0]
        speakerStart = speaker[1]
        speakerEnd = speaker[2]

        result = transcript['result']
        words = [r['word'] for r in result if speakerStart < r['start'] < speakerEnd]
        #return
        print("Speaker",speakerID,":",' '.join(words), "\n")





def main():

    queue1 = queue.Queue()
    queue2 = queue.Queue()

    FRAME_RATE = 16000
    CHANNELS = 1

    podcast = AudioSegment.from_mp3("Podcast_Audio/Film-Release-Clip.mp3")
    podcast = podcast.set_channels(CHANNELS)
    podcast = podcast.set_frame_rate(FRAME_RATE)

    first_thread = threading.Thread(target=recognition, args=(queue1, podcast, FRAME_RATE))
    second_thread = threading.Thread(target=diarization, args=(queue2, podcast))
    third_thread = threading.Thread(target=identify_speaker, args=(queue1, queue2))

    first_thread.start()
    first_thread.join()
    gc.collect()

    second_thread.start()
    second_thread.join()
    gc.collect()

    third_thread.start()
    third_thread.join()
    gc.collect()

    # transcript = recognition(podcast,FRAME_RATE)
    #
    # labelling = diarization(podcast)
    #
    # print(identify_speaker(transcript, labelling))


if __name__ == '__main__':
    main()


  


  When I say crash I mean everything freezes, I have to hold down the power button on the desktop and turn it back on again. No blue/blank screen, just frozen in my IDE looking at my code. Any help in resolving this issue would be greatly appreciated.