Recherche avancée

Sur d’autres sites (10509)

• How to play video file with audio with DearPyGUI (Python) ?

  1er mars 2023, par Vi Tiet

  I'm using DearPyGUI to make a simple media player that can play video file (mp4, etc.) together with it's audio. The pre-requisite is that DearPyGUI is a must, however video feature will not exist until v2.0, which is still far in the future.

  


  Currently, I can only render the frames using OpenCV library for Python, however, the problem is how can I play the audio as well as play it in sync with the output video frames ?

  


  For context, I'm quite new to Python, and I don't know much about video and audio streaming, but I've thought of some approaches to this problem by looking through help posts online (However, I still have no idea how I can implement any of these seamlessly) :

  


  


  1. OpenCV for video frames, and audio ??? some libraries like ffmpeg-python or miniaudio to play sound... (How...?)

     


  


  2. Extract video frames and audio here and then use the raw data to play it (How...?)

     


  


  3. This example here is pretty close to what I want excluding the playing video and audio part, but I have no idea where to go from there. The video stream and the audio stream are instances of ffmpeg.nodes.FilterableStream, and they appear to hold addresses to somewhere. (No idea...)

     


  


  4. Another very close idea is using ffpyplayer I was able to get the video frame. However, the below code yields a blueish purple color tint to the video, and the frame rate is very slow compared to original (So close...)

     


  


  


  import time
import numpy as np
import cv2 as cv
from ffpyplayer.player import MediaPlayer


# https://github.com/Kazuhito00/Image-Processing-Node-Editor/blob/main/node_editor/util.py 
def cv2dpg(frame): 

    data = cv.resize(frame, (VIDEO_WIDTH, VIDEO_HEIGHT))
    data = np.flip(frame, 2)
    data = data.ravel()
    data = np.asfarray(data, dtype=np.float32)

    return np.true_divide(data, 255.0)


# https://stackoverflow.com/questions/59611075/how-would-i-go-about-playing-a-video-stream-with-ffpyplayer
# https://matham.github.io/ffpyplayer/examples.html#examples
def play_video(loaded_file_path):

    global player, is_playing
    player = MediaPlayer(loaded_file_path)

    while is_playing:

        frame, val = player.get_frame()

        if val == 'eof':
            is_playing = False
            break

        elif not frame:
            time.sleep(0.01)

        elif val != 'eof' and frame is not None:
            img, t = frame
            w = img.get_size()[0]
            h = img.get_size()[1]
            cv_mat = np.uint8(np.asarray(list(img.to_bytearray()[0])).reshape((h, w, 3)))
            texture_data = cv2dpg(cv_mat)
            dpg.set_value(VIDEO_CANVAS_TAG, texture_data)

    dpg.set_value(VIDEO_CANVAS_TAG, DEFAULT_VIDEO_TEXTURE)


  


  I still need to do more research, but any pointer to somewhere good to start off (either handling raw data or using different libraries) would be greatly appreciated !

  


  EDIT :
For more context, I'm using raw texture like this example of DearPyGUI official documentation to render the video frames that were extracted in the while loop.

  


• Undefined symbol despite libraries being linked

  17 septembre 2020, par Areopag

  currently I'm trying to write a JNI function using ffmpeg features, but when I'm trying to run my java executable after a clean compile I get the following error :

  


  /usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java: symbol lookup error: /lib/libmediaserv_ffmpeg.so: undefined symbol: _Z19avformat_open_inputPP15AVFormatContextPKcP13AVInputFormatPP12AVDictionary


  


  I can't figure out why ffmpeg is not linked correctly.

  


  mediaserv_ffmpeg.cpp :

  


  #include <iostream>&#xA;#include <libavformat></libavformat>avformat.h>&#xA;#include <libavutil></libavutil>avutil.h>&#xA;&#xA;JNIEXPORT jobject JNICALL Java_mediaserv_ffmpeg_getMetadata&#xA;        (JNIEnv* env, jclass cls, jstring path) { ... }&#xA;</iostream>

  


  What I've tried so far is inspecting libmediaserv_ffmpeg.so :

  


   nm libmediaserv_ffmpeg.so --format=sysv | grep avformat
_Z19avformat_open_inputPP15AVFormatContextPKcP13AVInputFormatPP12AVDictionary|                |   U  |            NOTYPE|                |     |*UND*
_Z20avformat_close_inputPP15AVFormatContext|                |   U  |            NOTYPE|                |     |*UND*
_Z25avformat_find_stream_infoP15AVFormatContextPP12AVDictionary|                |   U  |            NOTYPE|                |     |*UND*


  


  Which, I think, is just another way of saying that there are undefined symbols in the .so file.

  


  The commands used to build the lib (taken from make VERBOSE=1) :

  


  /usr/bin/c++  -Dmediaserv_ffmpeg_EXPORTS -I/usr/lib/jvm/java-11-openjdk-amd64/include -I/usr/lib/jvm/java-11-openjdk-amd64/include/linux -I/mnt/g/Workspace/mediaserv/third-party/ffmpeg  -fPIC   -std=gnu++1z -o CMakeFiles/mediaserv_ffmpeg.dir/mediaserv_ffmpeg.cpp.o -c /mnt/g/Workspace/mediaserv/third-party/mediaserv_ffmpeg.cpp
/usr/bin/c++ -fPIC   -shared -Wl,-soname,libmediaserv_ffmpeg.so -o /mnt/g/Workspace/mediaserv/build/libs/libmediaserv_ffmpeg.so CMakeFiles/mediaserv_ffmpeg.dir/mediaserv_ffmpeg.cpp.o  -Wl,-rpath,/usr/lib/jvm/java-11-openjdk-amd64/lib:/usr/lib/jvm/java-11-openjdk-amd64/lib/server:/mnt/g/Workspace/mediaserv/third-party/ffmpeg/libavutil:/mnt/g/Workspace/mediaserv/third-party/ffmpeg/libavformat /usr/lib/jvm/java-11-openjdk-amd64/lib/libjawt.so /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so ../ffmpeg/libavutil/libavutil.so ../ffmpeg/libavformat/libavformat.so


  


  I've already tried to seek for errors in GCC's trace option for the linker and it lists me which libraries it tries to link / is linking but the result stays the same.

  


  /usr/bin/ld: mode elf_x86_64
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
CMakeFiles/mediaserv_ffmpeg.dir/mediaserv_ffmpeg.cpp.o
/usr/lib/jvm/java-11-openjdk-amd64/lib/libjawt.so
/usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
../ffmpeg/libavutil/libavutil.so
../ffmpeg/libavformat/libavformat.so
-lstdc++ (/usr/lib/gcc/x86_64-linux-gnu/7/libstdc++.so)
/lib/x86_64-linux-gnu/libm.so.6
/lib/x86_64-linux-gnu/libmvec.so.1
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/lib/x86_64-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o


  


  CMakeLists.txt :

  


  cmake_minimum_required(VERSION 3.17)
project(mediaserv_ffmpeg)

set(CMAKE_CXX_STANDARD 17)
set(BUILD_SHARED_LIBS ON)
add_library(${PROJECT_NAME} SHARED mediaserv_ffmpeg.cpp mediaserv_ffmpeg.h)

find_package(JNI REQUIRED)

find_path(AVUTIL_INCLUDE_DIR libavutil/avutil.h HINTS ffmpeg REQUIRED)
find_library(AVUTIL_LIBRARY avutil HINTS ${PROJECT_SOURCE_DIR}/ffmpeg/libavutil PATHS REQUIRED)

find_path(AVFORMAT_INCLUDE_DIR libavformat/avformat.h HINTS ffmpeg REQUIRED)
find_library(AVFORMAT_LIBRARY avformat HINTS ${PROJECT_SOURCE_DIR}/ffmpeg/libavformat REQUIRED)

target_include_directories(${PROJECT_NAME} PRIVATE ${JNI_INCLUDE_DIRS} ${AVUTIL_INCLUDE_DIR} ${AVFORMAT_INCLUDE_DIR})

target_link_libraries(${PROJECT_NAME} ${JNI_LIBRARIES} ${AVUTIL_LIBRARY} ${AVFORMAT_LIBRARY})


  


  Neither ffmpeg taken from the official ubuntu bionic repository nor a self-compiled version with configure --enable-shared works for me.

  


  Where is my mistake or what could I still inspect ?

  


  Thanks in advance.

  


• ffmpeg : Overlay audios with different lengths using amix and apad

  23 novembre 2017, par Valdir

  I’m trying to overlay two audio files with the following command :

  ffmpeg -y -i mp3/test1.mp3 -i mp3/test2.mp3 -filter_complex "[0:0][1:0] amix=inputs=2:duration=longest" -c:a libmp3lame merged.mp3

  But it fails if the length of the audios are different :

  ffmpeg -y -i live.mp3 -i mp3/test1.mp3 -filter_complex "[0:0][1:0] amix=inputs=2:duration=longest" -c:a libmp3lame merged2.mp3
  
  ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
  
    built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  
    configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  
    libavutil      54. 31.100 / 54. 31.100
  
    libavcodec     56. 60.100 / 56. 60.100
  
    libavformat    56. 40.101 / 56. 40.101
  
    libavdevice    56.  4.100 / 56.  4.100
  
    libavfilter     5. 40.101 /  5. 40.101
  
    libavresample   2.  1.  0 /  2.  1.  0
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  2.101 /  1.  2.101
  
    libpostproc    53.  3.100 / 53.  3.100
  
  [mp3 @ 0x22cf760] Format mp3 detected only with low score of 1, misdetection possible!
  
  [mp3 @ 0x22cf760] Could not find codec parameters for stream 0 (Audio: mp3, 0 channels, s16p): unspecified frame size
  
  Consider increasing the value for the 'analyzeduration' and 'probesize' options
  
  live.mp3: could not find codec parameters
  
  Input #0, mp3, from 'live.mp3':
  
    Duration: N/A, start: 0.000000, bitrate: N/A
  
      Stream #0:0: Audio: mp3, 0 channels, s16p
  
  [mp3 @ 0x22d10a0] Skipping 0 bytes of junk at 18034.
  
  Input #1, mp3, from 'mp3/test1.mp3':
  
    Metadata:
  
      title           : Nothing Else Matters [Official Music Video]
  
      artist          : Metallica
  
    Duration: 00:06:25.80, start: 0.025057, bitrate: 192 kb/s
  
      Stream #1:0: Audio: mp3, 44100 Hz, stereo, s16p, 192 kb/s
  
      Metadata:
  
        encoder         : Lavc56.14
  
  [abuffer @ 0x22fa2c0] Value inf for parameter 'time_base' out of range [0 - 2.14748e+09]
  
      Last message repeated 3 times
  
  [abuffer @ 0x22fa2c0] Error setting option time_base to value 1/0.
  
  [graph 0 input from stream 0:0 @ 0x22fa3e0] Error applying options to the filter.
  
  Error configuring complex filters.
  
  Numerical result out of range

  How do I add the apad filter so that silence is added to the shortest audio ?