Sur d’autres sites (7742)

• Frames from Video Using OpenCV in Ubuntu with Java 17

  2 août 2023, par bstrdn

  I've been trying to extract frames from a video using OpenCV in an Ubuntu environment with Java 17, but all my attempts have been unsuccessful so far. The main issue is that ffmpeg is not detected by the OpenCV library on ubuntu.

  


  Here's my test code :

  


  Main class

  


  import org.opencv.core.Core;
import org.opencv.videoio.VideoCapture;

public class Main {
  public static String videoPath = "";
  static {
    String osName = System.getProperty("os.name").toLowerCase();
    if (osName.contains("win")) {
      System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
      videoPath = ClassLoader.getSystemResource("video.mp4").getPath();
    } else {
      videoPath = "/video.mp4";
//      System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
      System.load("/usr/lib/opencv_java470.so");
    }
  }

  public static void main(String[] args) throws InterruptedException {
    VideoCapture videoCapture = new VideoCapture(videoPath);
    if (!videoCapture.isOpened()) {
      System.out.println("Error opening video file.");
    } else {
      System.out.println("Everything is fine.");
    }
    System.out.println(Core.getBuildInformation());
    Thread.sleep(1000000000000000000L);
  }
}


  


  Maven :

  


  &lt;?xml version="1.0" encoding="UTF-8"?>&#xA;<project xmlns="http://maven.apache.org/POM/4.0.0">&#xA;  <modelversion>4.0.0</modelversion>&#xA;  <parent>&#xA;    <groupid>org.springframework.boot</groupid>&#xA;    <artifactid>spring-boot-starter-parent</artifactid>&#xA;    <version>3.0.2</version>&#xA;    <relativepath></relativepath> &#xA;  </parent>&#xA;  <groupid>org.example</groupid>&#xA;  <artifactid>untitled</artifactid>&#xA;  <version>1.0-SNAPSHOT</version>&#xA;  <properties>&#xA;    17&#xA;    17&#xA;    UTF-8&#xA;  </properties>&#xA;&#xA;<dependencies>&#xA;<dependency>&#xA;  <groupid>org.bytedeco</groupid>&#xA;  <artifactid>opencv-platform</artifactid>&#xA;  <version>4.7.0-1.5.9</version>&#xA;</dependency>&#xA;&#xA;<dependency>&#xA;  <groupid>org.springframework.boot</groupid>&#xA;  <artifactid>spring-boot-starter-web</artifactid>&#xA;</dependency>&#xA;</dependencies>&#xA;<build>&#xA;<plugins>&#xA;  <plugin>&#xA;    <groupid>org.springframework.boot</groupid>&#xA;    <artifactid>spring-boot-maven-plugin</artifactid>&#xA;  </plugin>&#xA;</plugins>&#xA;</build>&#xA;</project>&#xA;

  


  Dockerfile :

  


  FROM ubuntu:20.04
RUN apt-get update && apt-get install -y openjdk-17-jdk
RUN apt-get update && apt-get install -y pkg-config
RUN apt-get update && apt-get install -y ffmpeg libavformat-dev libavcodec-dev libswscale-dev libavresample-dev
ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-amd64
COPY target/*.jar /app.jar
COPY src/main/resources/video.mp4 /
COPY opencv_java470.so /usr/lib
ENV JAVA_TOOL_OPTIONS="-Djava.library.path=/usr/lib:/lib:/usr/local/lib:/usr/local:/usr/lib/x86_64-linux-gnu"
ENV LD_LIBRARY_PATH=/usr/lib:/lib:/usr/local/lib:/usr/local:/usr/lib/x86_64-linux-gnu
ENTRYPOINT [ "sh", "-c", "java $JAVA_TOOL_OPTIONS -jar /app.jar" ]


  


  The application displays the following text (video is not opened) :

  


  Error opening video file.

2023-08-02T13:30:58.448538297Z 
2023-08-02T13:30:58.448572597Z General configuration for OpenCV 4.7.0 =====================================
2023-08-02T13:30:58.448587327Z   Version control:               v4.7.0
2023-08-02T13:30:58.448590607Z 
2023-08-02T13:30:58.448593217Z   Platform:
2023-08-02T13:30:58.448596057Z     Timestamp:                   2023-03-27T23:13:34Z
2023-08-02T13:30:58.448598747Z     Host:                        Linux 5.4.0-1103-azure x86_64
2023-08-02T13:30:58.448601377Z     CMake:                       3.25.2
2023-08-02T13:30:58.448604277Z     CMake generator:             Unix Makefiles
2023-08-02T13:30:58.448606837Z     CMake build tool:            /usr/bin/make
2023-08-02T13:30:58.448609327Z     Configuration:               RELEASE
2023-08-02T13:30:58.448611867Z 
.......there is a lot of text......
2023-08-02T13:30:58.448772865Z 
2023-08-02T13:30:58.448775355Z   Video I/O:
2023-08-02T13:30:58.448777845Z     DC1394:                      NO
2023-08-02T13:30:58.448780535Z     FFMPEG:                      NO
2023-08-02T13:30:58.448783105Z       avcodec:                   NO
2023-08-02T13:30:58.448785725Z       avformat:                  NO
2023-08-02T13:30:58.448788295Z       avutil:                    NO
2023-08-02T13:30:58.448790755Z       swscale:                   NO
2023-08-02T13:30:58.448793295Z       avresample:                NO
2023-08-02T13:30:58.448795845Z     GStreamer:                   NO
2023-08-02T13:30:58.448798255Z     v4l/v4l2:                    YES (linux/videodev2.h)
2023-08-02T13:30:58.448800885Z 
2023-08-02T13:30:58.448803525Z   Parallel framework:            pthreads
2023-08-02T13:30:58.448806345Z 
............
2023-08-02T13:30:58.448859654Z   Java:                          export all functions
2023-08-02T13:30:58.448862024Z     ant:                         /usr/bin/ant (ver 1.10.5)
2023-08-02T13:30:58.448864504Z     JNI:                         /opt/hostedtoolcache/jdk/8.0.362/x64/include /opt/hostedtoolcache/jdk/8.0.362/x64/include/linux /opt/hostedtoolcache/jdk/8.0.362/x64/include
2023-08-02T13:30:58.448867224Z     Java wrappers:               YES
2023-08-02T13:30:58.448869804Z     Java tests:                  NO
2023-08-02T13:30:58.448872314Z 
2023-08-02T13:30:58.448874734Z   Install to:                    /usr/local


  


  Moreover, ffmpeg is installed on Ubuntu.

  


  How can i fix this ? I'm deathly tired(

  


  Thank you in advance for your assistance !

  


  First I tried installation following the official instructions.https://github.com/bytedeco/javacv#required-software

  


  Then the solution provided in https://stackoverflow.com/a/76750478/8087508,

  


  but the result is the same.

  


• How to enable LHLS in FFMPEG 4.1 ?

  27 décembre 2020, par mehdi.r

  I am trying to create a low latency CMAF video stream using FFMPEG.
To do so, I would like to enable the lhls option in FFMPEG in order to have the #EXT-X-PREFETCH tag written in the HLS manifest.

  



  From the FFMPEG doc :

  



  https://www.ffmpeg.org/ffmpeg-all.html

  



  


  Enable Low-latency HLS(LHLS). Adds #EXT-X-PREFETCH tag with current >segment’s URI. Apple doesn’t have an official spec for LHLS. Meanwhile >hls.js player folks are trying to standardize a open LHLS spec. The >draft spec is available in https://github.com/video-dev/hlsjs->rfcs/blob/lhls-spec/proposals/0001-lhls.md This option will also try >to comply with the above open spec, till Apple’s spec officially >supports it. Applicable only when streaming and hls_playlist options >are enabled. This is an experimental feature.

  


  



  I am using the following command with FFMPEG 4.1 :

  



  ffmpeg -re -i ~/Documents/videos/BigBuckBunny.mp4 \
    -map 0 -map 0 -map 0 -c:a aac -c:v libx264 -tune zerolatency \
    -b:v:0 2000k -s:v:0 1280x720 -profile:v:0 high \
    -b:v:1 1500k -s:v:1 640x340  -profile:v:1 main \
    -b:v:2 500k -s:v:2 320x170  -profile:v:2 baseline \
    -bf 1 \
    -keyint_min 24 -g 24 -sc_threshold 0 -b_strategy 0 -ar:a:1 22050 -use_timeline 1 -use_template 1 \
    -window_size 5 -adaptation_sets "id=0,streams=v id=1,streams=a" \
    -hls_playlist 1 -seg_duration 1 -streaming 1  -strict experimental -lhls 1 -remove_at_exit 1 \
    -f dash manifest.mpd



  



  The kind of HLS manifest I obtained for a specific resolution :

  



  #EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:1
#EXT-X-MEDIA-SEQUENCE:8
#EXT-X-MAP:URI="init-stream0.mp4"
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:56.966+0900
chunk-stream0-00008.mp4
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:57.964+0900
chunk-stream0-00009.mp4
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:58.963+0900
chunk-stream0-00010.mp4
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:59.961+0900
chunk-stream0-00011.mp4
#EXTINF:1.021678,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:14:00.960+0900
chunk-stream0-00012.mp4
...



  



  As you can see the #EXT-X-PREFETCH tag is missing.

  



  Any help would be highly appreciated.

  



  Edit

  



  I also compiled FFmpeg from its master branch by doing the following :

  



  nasm

  



  sudo apt-get install nasm mingw-w64


  



  Codecs

  



  sudo apt-get install libx265-dev libnuma-dev libx264-dev libvpx-dev libfdk-aac-dev libmp3lame-dev libopus-dev


  



  FFmpeg

  



  mkdir lhls
cd lhls 
git init 
git clone https://github.com/FFmpeg/FFmpeg.git
cd FFmpeg 
git checkout master


  



  AOM (inside FFmpeg dir)

  



  git -C aom pull 2> /dev/null || git clone --depth 1 https://aomedia.googlesource.com/aom && \
mkdir -p aom_build && \
cd aom_build && \
PATH="$HOME/bin:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$HOME/ffmpeg_build" -DENABLE_SHARED=off -DENABLE_NASM=on ../aom && \
PATH="$HOME/bin:$PATH" make && \
make install
cd..


  



  Compiling

  



  PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
  --prefix="$HOME/ffmpeg_build" \
  --pkg-config-flags="--static" \
  --extra-cflags="-I$HOME/ffmpeg_build/include" \
  --extra-ldflags="-L$HOME/ffmpeg_build/lib" \
  --extra-libs="-lpthread -lm" \
  --bindir="$HOME/bin" \
  --enable-gpl \
  --enable-libaom \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-nonfree && \
PATH="$HOME/bin:$PATH" make 


  



  Unfortunately, the #EXT-X-PREFETCH is still missing in the HLS Manifest.

  



  I also tried nightly builds from https://ffmpeg.zeranoe.com/builds/ , same result.

  



  Any help would be highly appreciated.

  



  EDIT 2 :resolved

  



  Thanks to @aergistal and @Gyan , the #EXT-X-PREFETCH tag is now present in my HLS manifest.

  



  Here the FFMPEG command I am using :

  



  ./ffmpeg -re -i ~/videos/BigBuckBunny.mp4 -loglevel debug \
  -map 0 -map 0 -map 0 -c:a aac -c:v libx264 -tune zerolatency \
  -b:v:0 2000k -s:v:0 1280x720 -profile:v:0 high -b:v:1 1500k -s:v:1 640x340  -profile:v:1 main -b:v:2 500k -s:v:2 320x170  -profile:v:2 baseline -bf 1 \
 -keyint_min 24 -g 24 -sc_threshold 0 -b_strategy 0 -ar:a:1 22050 -use_timeline 1 -use_template 1 -window_size 5 \
 -adaptation_sets "id=0,streams=v id=1,streams=a" -hls_playlist 1 -seg_duration 3 -streaming 1 \
 -strict experimental -lhls 1 -remove_at_exit 0 -master_m3u8_publish_rate 3 \
 -f dash -method PUT -http_persistent 1  https://example.com/manifest.mpd


  



  Apparently the mime types are not passed to the server & FFmpeg seems to ignore the -headers option.

  


• ffmpeg xfade audio out of sync

  1er décembre 2020, par Richard

  hi I am using ffmpeg xfade to concat two clips, the transition looks fine, but the second clip has no audio in the output. I don't find more information about this in https://trac.ffmpeg.org/wiki/Xfade and the official doc. Can I keep the audio of both clips in sync ?

  


  I'm puttting the complete log below, it's massive

  


  ffmpeg -i teacher1.mp4 -i teacher2.mp4 -filter_complex xfade=transition=fadewhite:duration=1:offset=14.5 output.mp4
ffmpeg version 4.3.1-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'teacher1.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.29.100
  Duration: 00:00:15.02, start: 0.000000, bitrate: 2447 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 2315 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'teacher2.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.29.100
  Duration: 00:00:15.02, start: 0.000000, bitrate: 1643 kb/s
    Stream #1:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 1509 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 (h264) -> xfade:main (graph 0)
  Stream #1:0 (h264) -> xfade:xfade (graph 0)
  xfade (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x64603c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x64603c0] profile High 4:4:4 Predictive, level 4.0, 4:4:4, 8-bit
[libx264 @ 0x64603c0] 264 - core 161 r3018 db0d417 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p, 1920x1080, q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.91.100 aac
frame=  738 fps= 25 q=-1.0 Lsize=    7317kB time=00:00:29.40 bitrate=2038.9kbits/s speed=1.01x    
video:7064kB audio:236kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.241368%
[libx264 @ 0x64603c0] frame I:8     Avg QP:16.56  size:100307
[libx264 @ 0x64603c0] frame P:332   Avg QP:19.50  size: 15211
[libx264 @ 0x64603c0] frame B:398   Avg QP:24.35  size:  3467
[libx264 @ 0x64603c0] consecutive B-frames:  6.2% 62.6%  8.9% 22.2%
[libx264 @ 0x64603c0] mb I  I16..4: 18.7% 60.5% 20.9%
[libx264 @ 0x64603c0] mb P  I16..4:  4.5%  7.6%  0.6%  P16..4: 24.1%  5.6%  1.9%  0.0%  0.0%    skip:55.7%
[libx264 @ 0x64603c0] mb B  I16..4:  0.4%  0.5%  0.1%  B16..8: 14.3%  0.9%  0.1%  direct: 0.4%  skip:83.3%  L0:42.9% L1:53.9% BI: 3.3%
[libx264 @ 0x64603c0] 8x8 transform intra:59.2% inter:84.3%
[libx264 @ 0x64603c0] coded y,u,v intra: 25.3% 11.9% 13.0% inter: 3.2% 1.7% 1.8%
[libx264 @ 0x64603c0] i16 v,h,dc,p: 49% 25% 15% 12%
[libx264 @ 0x64603c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 17% 37%  2%  2%  3%  2%  3%  2%
[libx264 @ 0x64603c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 19% 15%  6%  7%  7%  6%  6%  5%
[libx264 @ 0x64603c0] Weighted P-Frames: Y:3.3% UV:1.5%
[libx264 @ 0x64603c0] ref P L0: 66.0%  8.0% 18.5%  7.5%  0.0%
[libx264 @ 0x64603c0] ref B L0: 84.9% 13.1%  2.0%
[libx264 @ 0x64603c0] ref B L1: 98.8%  1.2%
[libx264 @ 0x64603c0] kb/s:1960.04
[aac @ 0x64d1280] Qavg: 171.394