Recherche avancée

Sur d’autres sites (6660)

• Opencv is working for my project on my laptop but stop working on raspberry pi

  10 mars 2021, par Clovis Tiwange

  Good morning all ! I despair (This is my first post).
I am trying to set up a human detection system using the openvino toolkit. I am using an example project provided by openvino at the following openvino multi target tracking. I first tested them on my computer (ubuntu) and it worked. I have now tried to run the project on raspberry pi but it gets stuck at opencv level. I followed the following tutorial Install openvino on rasbian for setting up openvino.
While debugging, I realized that the problem was with cv2.videocapure (link). Apparently there is a problem with the backend ffmepg but also GStreamer.

  


  I run the following command

  


  python3 multi_camera_multi_target_tracking.py     -i http://192.168.137.160:4747/video    --m_detector model/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml     --m_reid model/intel/person-reidentification-retail-0031/FP32/person-reidentification-retail-0031.xml     --config config.py -l /opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/libcpu_extension_avx2.so


  


  And i have the following results

  


  INFO: 2021-03-07 21:07:34: Opening file http://192.168.137.160:4747/video
[DEBUG:0] global ../opencv/modules/videoio/src/videoio_registry.cpp (171) VideoBackendRegistry VIDEOIO: Builtin backends(8): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); V4L2(960); CV_IMAGES(950); CV_MJPEG(940); UEYE(930)

[DEBUG:0] global ../opencv/modules/videoio/src/videoio_registry.cpp (195) VideoBackendRegistry VIDEOIO: Available backends(8): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); V4L2(960); CV_IMAGES(950); CV_MJPEG(940); UEYE(930)

[ INFO:0] global ../opencv/modules/videoio/src/videoio_registry.cpp (197) VideoBackendRegistry VIDEOIO: Enabled backends(8, sorted by priority): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); MSMF(970); V4L2(960); CV_IMAGES(950); CV_MJPEG(940); UEYE(930)

[ WARN:0] global ../opencv/modules/videoio/src/cap.cpp (108) open VIDEOIO(FFMPEG): trying capture filename='http://192.168.137.160:4747/video' ...

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (359) getPluginCandidates VideoIO pluigin (FFMPEG): glob is 'libopencv_videoio_ffmpeg*.so', 1 location(s)

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (366) getPluginCandidates     - /opt/intel/openvino/opencv/lib: 1

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (370) getPluginCandidates Found 1 plugin(s) for FFMPEG

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (175) libraryLoad load /opt/intel/openvino/opencv/lib/libopencv_videoio_ffmpeg.so => FAILED

[ WARN:0] global ../opencv/modules/videoio/src/cap.cpp (170) open VIDEOIO(FFMPEG): backend is not available (plugin is missing, or can't be loaded due dependencies or it is not compatible)

[ WARN:0] global ../opencv/modules/videoio/src/cap.cpp (108) open VIDEOIO(GSTREAMER): trying capture filename='http://192.168.137.160:4747/video' ...

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (359) getPluginCandidates VideoIO pluigin (GSTREAMER): glob is 'libopencv_videoio_gstreamer*.so', 1 location(s)

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (366) getPluginCandidates     - /opt/intel/openvino/opencv/lib: 1

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (370) getPluginCandidates Found 1 plugin(s) for GSTREAMER

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (175) libraryLoad load /opt/intel/openvino/opencv/lib/libopencv_videoio_gstreamer.so => OK

[ INFO:0] global ../opencv/modules/videoio/src/backend_plugin.cpp (236) PluginBackend Video I/O: loaded plugin 'GStreamer OpenCV Video I/O plugin'

[ INFO:0] global ../opencv/modules/videoio/src/cap_gstreamer.cpp (711) open OpenCV | GStreamer: http://192.168.137.160:4747/video

[ INFO:0] global ../opencv/modules/videoio/src/cap_gstreamer.cpp (744) open OpenCV | GStreamer: mode - URI


  


  The program seems to be stuck there. No exception is thrown.

  


• lavfi/dnn_classify : add filter dnn_classify for classification based on detection...

  17 mars 2021, par Guo, Yejun

  lavfi/dnn_classify : add filter dnn_classify for classification based on detection bounding boxes
  classification is done on every detection bounding box in frame's side data,
  
  which are the results of object detection (filter dnn_detect).
  Please refer to commit log of dnn_detect for the material for detection,
  
  and see below for classification.
   download material for classifcation :
  
  wget https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/2021.1/emotions-recognition-retail-0003.bin
  
  wget https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/2021.1/emotions-recognition-retail-0003.xml
  
  wget https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/2021.1/emotions-recognition-retail-0003.label
   run command as :
  
  ./ffmpeg -i cici.jpg -vf dnn_detect=dnn_backend=openvino:model=face-detection-adas-0001.xml:input=data:output=detection_out:confidence=0.6:labels=face-detection-adas-0001.label,dnn_classify=dnn_backend=openvino:model=emotions-recognition-retail-0003.xml:input=data:output=prob_emotion:confidence=0.3:labels=emotions-recognition-retail-0003.label:target=face,showinfo -f null -
  We'll see the detect&classify result as below :
  
  [Parsed_showinfo_2 @ 0x55b7d25e77c0]   side data - detection bounding boxes :
  
  [Parsed_showinfo_2 @ 0x55b7d25e77c0] source : face-detection-adas-0001.xml, emotions-recognition-retail-0003.xml
  
  [Parsed_showinfo_2 @ 0x55b7d25e77c0] index : 0,  region : (1005, 813) -> (1086, 905), label : face, confidence : 10000/10000.
  
  [Parsed_showinfo_2 @ 0x55b7d25e77c0]            classify :  label : happy, confidence : 6757/10000.
  
  [Parsed_showinfo_2 @ 0x55b7d25e77c0] index : 1,  region : (888, 839) -> (967, 926), label : face, confidence : 6917/10000.
  
  [Parsed_showinfo_2 @ 0x55b7d25e77c0]            classify :  label : anger, confidence : 4320/10000.
  Signed-off-by : Guo, Yejun <yejun.guo@intel.com>
  

  • [DH] configure
  • [DH] doc/filters.texi
  • [DH] libavfilter/Makefile
  • [DH] libavfilter/allfilters.c
  • [DH] libavfilter/vf_dnn_classify.c

• How to resample from 8K to 16K with librosa or torchaudio as ffmpeg do it ?

  28 décembre 2023, par user3668129

  In my app,

  


  


  • I'm getting array of audio sample (with sample rate =8000) which was loaded with torchaudio.load

    


  


  • I need to use this audio array and run whisper (STT).

    


  


  • I want to avoid from loading the wav file again with whisper (load_audio) (for efficiency) and to resample the array to 16000.

    


  


  • whisper.load_audio use ffmpeg to load and resample the audio to 16000.
I'm trying to use librosa or torchaudio and resample the audio array but It always seems that the resample methods are not the same.

    


  


  • (I assume that if I use other resample method not as the whisper model was trained on, I can get bad results).

    


  


  


  Example :
loading test.wav file (with SR=8000) and print the 5 first cells :
whisper_audio = whisper.load_audio(file) => [-0.00082397 -0.00115967 -0.00186157 -0.00231934 -0.00222778, ...]

  


  loading with torchaudio and resample it with librosa :
librosa.resample(vad_audio, orig_sr=8000, target_sr=16000, scale=True, res_type='kaiser_best')
=> [-0.00082317 -0.0010577  -0.0013937  -0.0016688  -0.00186235

  


  seems different values.

  


  How can I resample the audio in the exact way ffmpeg do it ?