Recherche avancée

Sur d’autres sites (9430)

• Can't set seeker in GSTREAMER cv2, python

  29 avril, par Alperen Ölçer

  I want to skip n seconds forward and backward in gstreamer cv2 capture for recorded videos. But when I use cap_gstreamer.set(cv2.CAP_PROP_POS_FRAMES, fps*skip_second) it resets seeker to beginning of video. How can I solve it ? I wrote an example, used recorded clock video.

  


  import cv2

video_p = '/home/alperenlcr/Videos/clock.mp4'

cap_gstreamer = cv2.VideoCapture(video_p, cv2.CAP_GSTREAMER)
cap_ffmpeg = cv2.VideoCapture(video_p, cv2.CAP_FFMPEG)

fps = cap_gstreamer.get(cv2.CAP_PROP_FPS)
skip_second = 100

im1 = cv2.resize(cap_gstreamer.read()[1], (960, 540))
im1_ffmpeg = cv2.resize(cap_ffmpeg.read()[1], (960, 540))

cap_gstreamer.set(cv2.CAP_PROP_POS_FRAMES, fps*skip_second)
cap_ffmpeg.set(cv2.CAP_PROP_POS_FRAMES, fps*skip_second)

im2 = cv2.resize(cap_gstreamer.read()[1], (960, 540))
im2_ffmpeg = cv2.resize(cap_ffmpeg.read()[1], (960, 540))

merge_gstreamer = cv2.hconcat([im1, im2])
merge_ffmpeg = cv2.hconcat([im1_ffmpeg, im2_ffmpeg])

cv2.imshow(str(skip_second) + ' gstreamer', merge_gstreamer)
cv2.imshow(str(skip_second) + ' ffmpeg', merge_ffmpeg)
cv2.waitKey(0)
cv2.destroyAllWindows()

cap_gstreamer.release()
cap_ffmpeg.release()



  


  Result :


  


  My cv2 build is like :

  


  >>> print(cv2.getBuildInformation())

General configuration for OpenCV 4.8.1 =====================================
  Version control:               4.8.1-dirty

  Extra modules:
    Location (extra):            /home/alperenlcr/SourceInstalls/opencv_contrib/modules
    Version control (extra):     4.8.1

  Platform:
    Timestamp:                   2024-12-02T13:44:58Z
    Host:                        Linux 6.8.0-49-generic x86_64
    CMake:                       3.22.1
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/gmake
    Configuration:               RELEASE

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (18 files):         + SSSE3 SSE4_1
      SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (8 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (37 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      NO
    C++ standard:                11
    C++ Compiler:                /usr/bin/c++  (ver 10.5.0)
    C++ flags (Release):         -fsigned-char -ffast-math -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -ffast-math -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -ffast-math -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -ffast-math -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
    Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib/x86_64-linux-gnu/libpng.so /usr/lib/x86_64-linux-gnu/libtiff.so /usr/lib/x86_64-linux-gnu/libz.so /usr/lib/x86_64-linux-gnu/libfreetype.so /usr/lib/x86_64-linux-gnu/libharfbuzz.so Iconv::Iconv m pthread cudart_static dl rt nppc nppial nppicc nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/lib/x86_64-linux-gnu -L/usr/lib/cuda/lib64
    3rdparty dependencies:       libprotobuf ade ittnotify libwebp libopenjp2 IlmImf quirc ippiw ippicv

  OpenCV modules:
    To be built:                 aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
    Disabled:                    cudacodec world
    Disabled by dependency:      -
    Unavailable:                 alphamat cvv hdf java julia matlab ovis python2 sfm viz
    Applications:                tests perf_tests examples apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:                           GTK2
    QT:                          NO
    GTK+:                        YES (ver 2.24.33)
      GThread :                  YES (ver 2.72.4)
      GtkGlExt:                  NO
    OpenGL support:              NO
    VTK support:                 NO

  Media I/O: 
    ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
    JPEG 2000:                   build (ver 2.5.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      swresample:                YES (3.9.100)
    GStreamer:                   YES (1.20.3)
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            TBB (ver 2021.5 interface 12050)

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2021.8 [2021.8.0]
           at:                   /home/alperenlcr/SourceInstalls/opencv/build/3rdparty/ippicv/ippicv_lnx/icv
    Intel IPP IW:                sources (2021.8.0)
              at:                /home/alperenlcr/SourceInstalls/opencv/build/3rdparty/ippicv/ippicv_lnx/iw
    VA:                          NO
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)
    Flatbuffers:                 builtin/3rdparty (23.5.9)

  NVIDIA CUDA:                   YES (ver 11.5, CUFFT CUBLAS NVCUVID NVCUVENC FAST_MATH)
    NVIDIA GPU arch:             86
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.6.0)

  OpenCL:                        YES (no extra features)
    Include path:                /home/alperenlcr/SourceInstalls/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  ONNX:                          NO

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.10.12)
    Libraries:                   /usr/lib/x86_64-linux-gnu/libpython3.10.so (ver 3.10.12)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.21.5)
    install path:                lib/python3.10/dist-packages/cv2/python-3.10

  Python (for build):            /usr/bin/python3

  Java:                          
    ant:                         NO
    Java:                        NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /usr/local
-----------------------------------------------------------------



  


• Seek function of HTML5 video and Maximum Keyframe Interval in video encoding compression

  20 décembre 2019, par Sanxofon

  This is a very specific question about how the seek function (of an HTML5 video element) interprets a video, in this case one in WEBM format and what specific relationship it has with the Maximum Keyframe Interval in the process of coding of said video.

  I’ve made this fiddle to show what I mean :

  https://jsfiddle.net/sanxofon/hctuxo3e/

  What it does : The idea is to control the position of a paused video with the scroll wheel of the mouse ... in addition, each frame of the video is copied to a canvas element, but I think that has no relation. I mention it just in case.

  What to watch : In Chrome (v66) and slightly less in Firefox (v59) the scrolling looks pretty good when the video is encoded with a Maximum Keyframe Interval of 6 or less, but jumps are appreciated when the interval is every 24 frames or more. This is noticeable in the video and even more on the canvas.

  FFMPEG : When encoding a video with FFMPEG this is achieved with the option -g6 or -g24 of Maximum Keyframe Interval. However, the file becomes heavier as we decrease the interval. It can be seen that there is no difference between both formats when the video is in play.

  You can switch the video in the snippet with the buttons.

  • Case 1 : When we use the video encoded with -g 6 the video scrolling is acceptable but the size increases : 6.229 Mb.

  FFMPEG string used :

   ffmpeg -i INPUT.MOV -c: v libvpx -qmin 0 -deadline best -qmax 50 -crf 1 -b: v 100K -g 6 test / video_g6.webm

  • Case 2 : When we use the video encoded with -g 24 the displacement is not smooth and suffers from jumps but the size decreases : 4,477 Mb.

  FFMPEG string used :

   ffmpeg -i INPUT.MOV -c: v libvpx -qmin 0 -deadline best -qmax 50 -crf 1 -b: v 100K -g 24 test / video_g24.webm

  Why does this happen ?

  What about -keyint_min or -force_key_frames ? Do they have any positive effect ? Is it better to use something like cgop (closed gop) ?

  I would appreciate some reference of consultation on this subject or a more or less detailed explanation of this relationship for both the WEBM container and for MP4 and OGG video.

  I am not looking so much for a magical ffmpeg chain (although I would appreciate it) but rather an explanation of how this relationship between the keyframes and the seek of a javascript video works.

  Thank you very much for reading here.

  P.S. One more thing, if the seek function only stops in a keyframe, is it possible that this frame has more quality than the others so that the quality increases when the video stops ?

• ffmpeg h264 interesting bright video fail ?

  23 novembre 2019, par Max Paython

  I am trying streaming with ffmpeg using the information I found here.

  • Server

    ffmpeg -f dshow -i video="john":audio="doe" -vcodec libx264
    
    -preset ultrafast -tune zerolatency -acodec aac -f mpegts udp://localhost:1234

  • Client

    ffmpeg -i udp://localhost:1234?listen test.mp4

  The client will actually live stream the video, but for testing purposes I am writing the output to a file for now.

  Here comes the interesting part. I began the stream(video and audio) in a dark room, everything is fine. I turn on the room’s lights and the client cries with many errors. I later checked the output, it is very corrupted indeed on the light turned-on parts. What could be the reason behind this ? I am posting the errors here if it could be of assistance.

  Some of the errors :

  udp://localhost:1234?listen: corrupt decoded frame in stream 0trate= 845.6kbits/s dup=1250 drop=0 speed=1.08x
  
      Last message repeated 1 times
  
  [h264 @ 0000026c4af69400] Invalid NAL unit 1, skipping.7.61 bitrate= 864.7kbits/s dup=1441 drop=0 speed=1.07x
  
  [h264 @ 0000026c4af69400] Invalid level prefix
  
  [h264 @ 0000026c4af69400] error while decoding MB 18 8
  
  [h264 @ 0000026c4af69400] concealing 911 DC, 911 AC, 911 MV errors in P frame
  
  [h264 @ 0000026c4af4b780] Invalid NAL unit 1, skipping.
  
  [h264 @ 0000026c4af4b780] negative number of zero coeffs at 10 14
  
  [h264 @ 0000026c4af4b780] error while decoding MB 10 14
  
  [h264 @ 0000026c4af4b780] concealing 679 DC, 679 AC, 679 MV errors in P frame
  
  udp://localhost:1234?listen: corrupt decoded frame in stream 0
  
      Last message repeated 1 times
  
  [h264 @ 0000026c4ba1e1c0] Invalid NAL unit 1, skipping.8.21 bitrate= 858.0kbits/s dup=1460 drop=0 speed=1.07x
  
  [h264 @ 0000026c4ba1e1c0] out of range intra chroma pred mode
  
  [h264 @ 0000026c4ba1e1c0] error while decoding MB 34 18
  
  [h264 @ 0000026c4ba1e1c0] concealing 495 DC, 495 AC, 495 MV errors in P frame
  
  [h264 @ 0000026c4bb16840] top block unavailable for requested intra mode
  
  [h264 @ 0000026c4bb16840] error while decoding MB 10 0
  
  [h264 @ 0000026c4bb16840] concealing 160 DC, 160 AC, 160 MV errors in P frame
  
  udp://localhost:1234?listen: corrupt decoded frame in stream 0
  
  [h264 @ 0000026c4bb16cc0] Invalid NAL unit 0, skipping.8.58 bitrate= 853.9kbits/s dup=1463 drop=0 speed=1.06x
  
  [h264 @ 0000026c4bb16cc0] corrupted macroblock 16 28 (total_coeff=-1)
  
  [h264 @ 0000026c4bb16cc0] error while decoding MB 16 28
  
  [h264 @ 0000026c4bb16cc0] concealing 113 DC, 113 AC, 113 MV errors in P frame
  
  [h264 @ 0000026c4bb17140] cbp too large (84) at 12 0
  
  [h264 @ 0000026c4bb17140] error while decoding MB 12 0
  
  [h264 @ 0000026c4bb17140] concealing 160 DC, 160 AC, 160 MV errors in P frame
  
  [mpegts @ 0000026c4aeb8d80] PES packet size mismatch

  Edit : As suggestion, I added these parameters to the server.

  -b:v 1M -bufsize 2M

  Now the errors are mostly gone. Except one time where it seemed like it occured just as I was turning off the lights, although I could not reproduce it after trying 4-5 times.

  Errors :

  [h264 @ 000002646f0d2f80] cbp too large (118) at 23 1422.25 bitrate= 659.5kbits/s dup=368 drop=0 speed=1.33x
  
  [h264 @ 000002646f0d2f80] error while decoding MB 23 14
  
  [mpegts @ 000002646e578d80] PES packet size mismatch
  
  [h264 @ 000002646f0d2f80] concealing 666 DC, 666 AC, 666 MV errors in P frame
  
  [h264 @ 000002646f0d0f80] concealing 160 DC, 160 AC, 160 MV errors in P frame
  
  [aac @ 000002646e64d640] Number of bands (59) exceeds limit (43).
  
  Error while decoding stream #0:1: Invalid data found when processing input
  
  [aac @ 000002646e64d640] Multiple frames in a packet.
  
  [aac @ 000002646e64d640] Reserved bit set.
  
  [aac @ 000002646e64d640] Number of bands (31) exceeds limit (29).
  
  Error while decoding stream #0:1: Invalid data found when processing input
  
  [h264 @ 000002646e5e11c0] concealing 160 DC, 160 AC, 160 MV errors in P frame
  
  udp://localhost:1234?listen: corrupt decoded frame in stream 0trate= 633.1kbits/s dup=368 drop=0 speed=1.35x
  
      Last message repeated 2 times

  The errors decreased because I increased the bitrate ? (I don’t know the default bitrate by the way). Bright decoding fails because it carries more information, data ? (same in raw video, but maybe file is heavier encoded bright frames)