Recherche avancée

Sur d’autres sites (5031)

• opencv ffmpeg vaapi 1080p resolution not working

  18 avril 2023, par yeo

  I want to use hardware acceleration with opencv manual build.
My gpu uses an i965 intel cpu built-in graphics card, and it is a debain11 environment.

  


  [OPENCV:FFMPEG:40] Reinit context to 1920x1088, pix_fmt: vaapi_vld


  


  If you look at some of the error messages below, it seems that the original file is 1920x1080 because it is converted to 1088 while reinit.
I've read that vaapi_vld reads 16 bits at a time.
In fact, it seems to work when the original file is changed to 1920x1072.
Is there a way to fix it without changing the original file resolution ?
Please advise seniors.
Sorry for my poor English skills
Thank you

  


  manual build CMAKE option

  


  "-DCMAKE_VERBOSE_MAKEFILE=ON -DWITH_VA_INTEL=ON -DWITH_VA=ON -DOPENCV_FFMPEG_ENABLE_LIBAVDEVICE=ON -DOPENCV_ENABLE_GLX=ON -DOPENCV_FFMPEG_SKIP_BUILD_CHECK=ON -DWITH_OPENVINO=ON -DWITH_INF_ENGINE=ON"



  


  build infomation

  


    OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java python2 ts
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           GTK3
    GTK+:                        YES (ver 3.24.24)
      GThread :                  YES (ver 2.66.8)
      GtkGlExt:                  NO
    VTK support:                 NO
  Media I/O: 
    ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 62)
    WEBP:                        /usr/lib/x86_64-linux-gnu/libwebp.so (ver encoder: 0x020e)
    PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.2.0)
    JPEG 2000:                   build (ver 2.4.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES
  Video I/O:
    DC1394:                      YES (2.2.6)
    FFMPEG:                      YES
      avcodec:                   YES (58.91.100)
      avformat:                  YES (58.45.100)
      avutil:                    YES (56.51.100)
      swscale:                   YES (5.7.100)
      avresample:                YES (4.0.0)
    GStreamer:                   YES (1.18.4)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2020.0.0 Gold [2020.0.0]
    VA:                          YES
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)

  OpenCL:                        YES (INTELVA)
    Include path:                /home/xxx
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.9.2)
    Libraries:                   /usr/lib/x86_64-linux-gnu/libpython3.9.so (ver 3.9.2)
    numpy:                       /home/../include (ver 1.19.3)
    install path:                python/cv2/python-3


  


  vainfo

  


  libva info: VA-API version 1.10.0
libva info: User environment variable requested driver 'i965'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_8
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.10 (libva 2.10.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Mobile - 2.4.1
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264MultiviewHigh      : VAEntrypointVLD
      VAProfileH264MultiviewHigh      : VAEntrypointEncSlice
      VAProfileH264StereoHigh         : VAEntrypointVLD
      VAProfileH264StereoHigh         : VAEntrypointEncSlice
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileJPEGBaseline           : VAEntrypointVLD


  


  import os
import cv2

os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "hw_decoders_any;vaapi,vdpau" +

cap = cv2.VideoCapture(file_name,cv2.CAP_FFMPEG(cv2.CAP_PROP_HW_ACCELERATION,cv2.VIDEO_ACCELERATION_ANY))  



  


  error code

  


  [ INFO:0@0.187] global /home/u/opencv-python/opencv/modules/videoio/src/videoio_registry.cpp (223) VideoBackendRegistry VIDEOIO: Enabled backends(8, sorted by priority): FFMPEG(1000); GSTREAMER(990); INTEL_MFX(980); V4L2(970); CV_IMAGES(960); CV_MJPEG(950); FIREWIRE(940); UEYE(930)
[OPENCV:FFMPEG:40] Reinit context to 1920x1088, pix_fmt: yuv420p
[OPENCV:FFMPEG:40] Trying to use DRM render node for device 0.
[OPENCV:FFMPEG:40] libva: VA-API version 1.10.0
libva: User environment variable requested driver 'i965'
libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva: Found init function __vaDriverInit_1_8
libva: va_openDriver() returns 0
Initialised VAAPI connection: version 1.10
[OPENCV:FFMPEG:40] VAAPI driver: Intel i965 driver for Intel(R) Haswell Mobile - 2.4.1.
[OPENCV:FFMPEG:40] Driver not found in known nonstandard list, using standard behaviour.
[ INFO:0@0.228] global /home/u/opencv-python/opencv/modules/videoio/src/cap_ffmpeg_hw.hpp (276) hw_check_device FFMPEG: Using vaapi video acceleration on device: Intel i965 driver for Intel(R) Haswell Mobile - 2.4.1
[ INFO:0@0.228] global /home/u/opencv-python/opencv/modules/videoio/src/cap_ffmpeg_hw.hpp (566) hw_create_device FFMPEG: Created video acceleration context (av_hwdevice_ctx_create) for vaapi on device 'default'
[ INFO:0@0.228] global /home/u/opencv-python/opencv/modules/core/src/ocl.cpp (1186) haveOpenCL Initialize OpenCL runtime...
[ INFO:0@0.228] global /home/u/opencv-python/opencv/modules/core/src/ocl.cpp (1192) haveOpenCL OpenCL: found 0 platforms
File open : ./videoplayback1.mp4
[OPENCV:FFMPEG:40] Reinit context to 1920x1088, pix_fmt: vaapi_vld
[OPENCV:FFMPEG:16] Failed to read image from surface 0x4000014: 18 (invalid parameter).
[ERROR:0@0.245] global /home/u/opencv-python/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (1575) retrieveFrame Error copying data from GPU to CPU (av_hwframe_transfer_data)
Play video ... size=1920x1080, file=./videoplayback1.mp4
[OPENCV:FFMPEG:16] Failed to read image from surface 0x4000012: 18 (invalid parameter).
[ERROR:0@0.277] global /home/u/opencv-python/opencv/modules/videoio/src/cap_ffmpeg_impl.hpp (1575) retrieveFrame Error copying data from GPU to CPU (av_hwframe_transfer_data)
OpenCV(4.6.0) Error: Assertion failed (!image.empty()) in imencode, file /home/u/opencv-python/opencv/modules/imgcodecs/src/loadsave.cpp, line 976
err =  OpenCV(4.6.0) /home/u/opencv-python/opencv/modules/imgcodecs/src/loadsave.cpp:976: error: (-215:Assertion failed) !image.empty() in function 'imencode'



  


  I tried to do video capture by ffmpeg hwacceleration with opencv, but an error message occurred

  


• VLC Player shows broken HLS stream with 4k HDR10 mkv

  8 avril 2023, par goodkid38

  I am trying to convert a 4k mkv to an HLS stream but I am not having any luck. I have tried a few ffmpeg commands to try and fix the issue but none have worked. Here are the commands I have tried.

  


  

  1. Basic copy command :

  


  


  ffmpeg -i "video.mkv" -c copy -f hls "plexTemp/out.m3u8"

  


  

  2. Command used to see if it was an HDR color issue :

  


  


  ffmpeg -i "video.mkv" -c copy -pix_fmt yuv420p10le -f hls "plexTemp/out.m3u8"
3. Command used to revert to 8 bit color :

  


  ffmpeg -i "video.mkv" -c copy -pix_fmt yuv420p -f hls "plexTemp/out.m3u8"

  


  

  4. I tried removing extra streams and just focusing on audio and video :

  


  


  ffmpeg -i "out.mkv" -map 0:v:0 -map 0:a:1 -c copy -pix_fmt yuv420p10le -f hls "plexTemp/out.m3u8"

  


  I also saw these warnings when running each command.Stream HEVC is not hvc1, you should use tag:v hvc1 to set it.
And this

  


  [matroska,webm @ 000001d7921803c0] Stream #12: not enough frames to estimate rate; consider increasing probesize
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 6 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 7 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 8 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 9 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 10 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 000001d7921803c0] Could not find codec parameters for stream 11 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options


  


  So I tried increasing the analyzeduration and probesize and adding the tag like so :
ffmpeg -analyzeduration 10000000 -probesize 10000000 -i "out.mkv" -c copy -tag:v hvc1 -f hls "plexTemp/out.m3u8"

  


  To no avail. Here is what the output looks like on VLC. It's mostly black with a few lines of color that randomly change.

  


  On my TV I see this :

  


  

  


  One thing I see that stands out in ffmpegs output is this :

  


  [hls @ 00000207239a9ec0] Opening 'plexTemp/out0.ts' for writing7 bitrate=  -0.0kbits/s speed=N/A
[hls @ 00000207239a9ec0] Opening 'plexTemp/out.m3u8.tmp' for writing


  


  Here is the audio and video info on the mkv :

  


  General
Unique ID                                : 92280908398971492516286250889389584022 (0x456CA80EF29B1357B572719D6EC4AE96)
Complete name                            : I:\video.mkv
Format                                   : Matroska
Format version                           : Version 2
File size                                : 49.4 GiB
Duration                                 : 1 h 39 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 71.2 Mb/s
Frame rate                               : 23.976 FPS
Movie name                               : video
Encoded date                             : 2023-04-06 22:39:53 UTC
Writing application                      : MakeMKV v1.16.7 win(x64-release)
Writing library                          : libmakemkv v1.16.7 (1.3.10/1.5.2) win(x64-release)
Cover                                    : Yes
Attachments                              : cover.jpg

Video
ID                                       : 1
ID in the original source medium         : 4113 (0x1011)
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 h 39 min
Bit rate                                 : 63.6 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.320
Stream size                              : 44.1 GiB (89%)
Writing library                          : ATEME Titan File 3.9.6 (4.9.6.2)        
Language                                 : English
Default                                  : No
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 1000 cd/m2
Maximum Frame-Average Light Level        : 140 cd/m2
Original source medium                   : Blu-ray

Audio #1
ID                                       : 2
ID in the original source medium         : 4352 (0x1100)
Format                                   : DTS XLL X
Format/Info                              : Digital Theater Systems
Commercial name                          : DTS:X
Codec ID                                 : A_DTS
Duration                                 : 1 h 39 min
Bit rate mode                            : Variable
Bit rate                                 : 4 174 kb/s
Channel(s)                               : 8 channels
Channel layout                           : C L R LFE Lb Rb Lss Rss
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Stream size                              : 2.89 GiB (6%)
Title                                    : Surround 7.1
Language                                 : English
Default                                  : Yes
Forced                                   : No
Original source medium                   : Blu-ray
Here is information on the HLS output:
General
Complete name                            : I:\out.m3u8
Format                                   : HLS
Format profile                           : Media
File size                                : 67.4 MiB
Duration                                 : 8 s 138 ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 69.4 Mb/s
Frame rate                               : 23.976 FPS


  


  And my
Here is the output of my HLS stream :

  


  Video
ID                                       : 256 (0x100)
Menu ID                                  : 1 (0x1)
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
HDR format                               : SMPTE ST 2086, HDR10 compatible
Muxing mode                              : MPEG-TS
Codec ID                                 : 36
Duration                                 : 8 s 49 ms
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Writing library                          : ATEME Titan File 3.9.6 (4.9.6.2)        
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 1000 cd/m2
Maximum Frame-Average Light Level        : 140 cd/m2
Source                                   : out92.ts

Audio
ID                                       : 257 (0x101)
Menu ID                                  : 1 (0x1)
Format                                   : DTS XLL X
Format/Info                              : Digital Theater Systems
Commercial name                          : DTS:X
Muxing mode                              : MPEG-TS
Codec ID                                 : 130
Duration                                 : 8 s 138 ms
Bit rate mode                            : Variable
Channel(s)                               : 8 channels
Channel layout                           : C L R LFE Lb Rb Lss Rss
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Delay relative to video                  : -125 ms
Language                                 : English
Source                                   : out92.ts


  


• Unable to open audio file on Heroku using Librosa

  15 mars 2020, par Rohan Bojja

  I have a feature extraction REST API written in Python using the Librosa library (Extracting audio features), it receives an audio file through HTTP POST and responds with a list of features(such as MFCC,etc).

  Since librosa depends on SoundFile (libsndfile1 / libsndfile-dev), it doesn’t support all the formats, I’m converting the audio file using ffmpeg-python wrapper (https://kkroening.github.io/ffmpeg-python/) .

  It works just fine on my Windows 10 machine with Conda, but when I deploy it on Heroku, the librosa.load() functions returns an unknown format error, no matter what format I convert it to. I have tried FLAC, AIFF and WAV.

  My first guess is that the converted format isn’t supported by libsndfile1, but it works on my local server (plus, their documentation says AIFF and WAV are supported), so I’m a little lost.

  I have attached all the relevant snippets of code below, I can provide anything extra if necessary. Any help is highly appreciated. Thanks.

  UPDATE1 :

  I am using pipes instead of writing and reading from disk, worth a mention as the question could be misleading otherwise.

  The log :

  File "/app/app.py", line 31, in upload
  
  x , sr = librosa.load(audioFile,mono=True,duration=5)
  
  File "/app/.heroku/python/lib/python3.6/site-packages/librosa/core/audio.py", line 164, in load
  
  six.reraise(*sys.exc_info())
  
  File "/app/.heroku/python/lib/python3.6/site-packages/six.py", line 703, in reraise
  
  raise value
  
  File "/app/.heroku/python/lib/python3.6/site-packages/librosa/core/audio.py", line 129, in load
  
  with sf.SoundFile(path) as sf_desc:
  
  File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 629, in __init__
  
  self._file = self._open(file, mode_int, closefd)
  
  File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 1184, in _open
  
  "Error opening {0!r}: ".format(self.name))
  
  File "/app/.heroku/python/lib/python3.6/site-packages/soundfile.py", line 1357, in _error_check
  
  raise RuntimeError(prefix + _ffi.string(err_str).decode('utf-8', 'replace'))
  
  RuntimeError: Error opening <_io.BytesIO object at 0x7f46ad28beb8>: File contains data in an unknown format.
  
  10.69.244.94 - - [15/Mar/2020:12:37:28 +0000] "POST /receiveWav HTTP/1.1" 500 290 "-" "curl/7.55.1"

  Flask/Librosa code deployed on Heroku (app.py) :

  from flask import Flask, jsonify, request
  
  import scipy.optimize
  
  import os,pickle
  
  import numpy as np
  
  from sklearn.preprocessing import StandardScaler
  
  import librosa
  
  import logging
  
  import soundfile as sf
  
  from pydub import AudioSegment
  
  import subprocess as sp
  
  import ffmpeg
  
  from io import BytesIO
  
  
  
  logging.basicConfig(level=logging.DEBUG)
  
  
  
  app = Flask(__name__) 
  
  
  
  @app.route('/receiveWav',methods = ['POST'])
  
  def upload():
  
      if(request.method == 'POST'):
  
          f = request.files['file']
  
          app.logger.info(f'AUDIO FORMAT\n\n\n\n\n\n\n\n\n\n: {f}')
  
          proc = (
  
              ffmpeg.input('pipe:')
  
              .output('pipe:', format='aiff')
  
              .run_async(pipe_stdin=True,pipe_stdout=True, pipe_stderr=True)
  
          )
  
          audioFile,err = proc.communicate(input=f.read())
  
          audioFile =  BytesIO(audioFile)
  
          scaler = pickle.load(open("scaler.ok","rb"))
  
          x , sr = librosa.load(audioFile,mono=True,duration=5)
  
          y=x
  
          #Extract the features
  
          chroma_stft = librosa.feature.chroma_stft(y=y, sr=sr)
  
          spec_cent = librosa.feature.spectral_centroid(y=y, sr=sr)
  
          spec_bw = librosa.feature.spectral_bandwidth(y=y, sr=sr)
  
          rolloff = librosa.feature.spectral_rolloff(y=y, sr=sr)
  
          zcr = librosa.feature.zero_crossing_rate(y)
  
          rmse = librosa.feature.rms(y=y)
  
          mfcc = librosa.feature.mfcc(y=y, sr=sr)
  
          features = f'{np.mean(chroma_stft)} {np.mean(rmse)} {np.mean(spec_cent)} {np.mean(spec_bw)} {np.mean(rolloff)} {np.mean(zcr)}'    
  
          for e in mfcc:
  
              features += f' {np.mean(e)}'
  
          input_data2 = np.array([float(i) for i in features.split(" ")]).reshape(1,-1)
  
          input_data2 = scaler.transform(input_data2)
  
          return jsonify(input_data2.tolist())
  
  
  
  # driver function 
  
  if __name__ == '__main__':   
  
      app.run(debug = True) 

  Aptfile :

  libsndfile1
  
  libsndfile-dev
  
  libav-tools
  
  libavcodec-extra-53 
  
  libavcodec-extra-53
  
  ffmpeg 

  requirements.txt :

  aniso8601==8.0.0
  
  audioread==2.1.8
  
  certifi==2019.11.28
  
  cffi==1.14.0
  
  Click==7.0
  
  decorator==4.4.2
  
  ffmpeg-python==0.2.0
  
  Flask==1.1.1
  
  Flask-RESTful==0.3.8
  
  future==0.18.2
  
  gunicorn==20.0.4
  
  itsdangerous==1.1.0
  
  Jinja2==2.11.1
  
  joblib==0.14.1
  
  librosa==0.7.2
  
  llvmlite==0.31.0
  
  MarkupSafe==1.1.1
  
  marshmallow==3.2.2
  
  numba==0.48.0
  
  numpy==1.18.1
  
  pycparser==2.20
  
  pydub==0.23.1
  
  pytz==2019.3
  
  resampy==0.2.2
  
  scikit-learn==0.22.2.post1
  
  scipy==1.4.1
  
  six==1.14.0
  
  SoundFile==0.10.3.post1
  
  Werkzeug==1.0.0
  
  wincertstore==0.2