Recherche avancée

Sur d’autres sites (13817)

• How to play video file with audio with DearPyGUI (Python) ?

  1er mars 2023, par Vi Tiet

  I'm using DearPyGUI to make a simple media player that can play video file (mp4, etc.) together with it's audio. The pre-requisite is that DearPyGUI is a must, however video feature will not exist until v2.0, which is still far in the future.

  


  Currently, I can only render the frames using OpenCV library for Python, however, the problem is how can I play the audio as well as play it in sync with the output video frames ?

  


  For context, I'm quite new to Python, and I don't know much about video and audio streaming, but I've thought of some approaches to this problem by looking through help posts online (However, I still have no idea how I can implement any of these seamlessly) :

  


  


  1. OpenCV for video frames, and audio ??? some libraries like ffmpeg-python or miniaudio to play sound... (How...?)

     


  


  2. Extract video frames and audio here and then use the raw data to play it (How...?)

     


  


  3. This example here is pretty close to what I want excluding the playing video and audio part, but I have no idea where to go from there. The video stream and the audio stream are instances of ffmpeg.nodes.FilterableStream, and they appear to hold addresses to somewhere. (No idea...)

     


  


  4. Another very close idea is using ffpyplayer I was able to get the video frame. However, the below code yields a blueish purple color tint to the video, and the frame rate is very slow compared to original (So close...)

     


  


  


  import time
import numpy as np
import cv2 as cv
from ffpyplayer.player import MediaPlayer


# https://github.com/Kazuhito00/Image-Processing-Node-Editor/blob/main/node_editor/util.py 
def cv2dpg(frame): 

    data = cv.resize(frame, (VIDEO_WIDTH, VIDEO_HEIGHT))
    data = np.flip(frame, 2)
    data = data.ravel()
    data = np.asfarray(data, dtype=np.float32)

    return np.true_divide(data, 255.0)


# https://stackoverflow.com/questions/59611075/how-would-i-go-about-playing-a-video-stream-with-ffpyplayer
# https://matham.github.io/ffpyplayer/examples.html#examples
def play_video(loaded_file_path):

    global player, is_playing
    player = MediaPlayer(loaded_file_path)

    while is_playing:

        frame, val = player.get_frame()

        if val == 'eof':
            is_playing = False
            break

        elif not frame:
            time.sleep(0.01)

        elif val != 'eof' and frame is not None:
            img, t = frame
            w = img.get_size()[0]
            h = img.get_size()[1]
            cv_mat = np.uint8(np.asarray(list(img.to_bytearray()[0])).reshape((h, w, 3)))
            texture_data = cv2dpg(cv_mat)
            dpg.set_value(VIDEO_CANVAS_TAG, texture_data)

    dpg.set_value(VIDEO_CANVAS_TAG, DEFAULT_VIDEO_TEXTURE)


  


  I still need to do more research, but any pointer to somewhere good to start off (either handling raw data or using different libraries) would be greatly appreciated !

  


  EDIT :
For more context, I'm using raw texture like this example of DearPyGUI official documentation to render the video frames that were extracted in the while loop.

  


• rtmp : Support play method in listen mode

  15 septembre 2013, par Luca Barbato

  rtmp : Support play method in listen mode
  

  • [DBH] libavformat/rtmpproto.c

• How to play audio using libao and FFmpeg in C ?

  12 mars 2024, par OmegaLol21

  I am trying to create a simple prototype application that opens a video file and plays its audio. I am using the FFmpeg libraries (libavcodec, libavformat, etc) to open and decode the video, and I am attempting to use libao to play the audio. I tried looking up code examples but a lot of them either don't work or use deprecated functions.

  


  So far, I managed to come up with this :

  


  #include <libavcodec></libavcodec>avcodec.h>&#xA;#include <libavformat></libavformat>avformat.h>&#xA;#include <libswresample></libswresample>swresample.h>&#xA;#include <ao></ao>ao.h>&#xA;&#xA;int main(int argc, char** argv) {&#xA;    AVFormatContext* format_ctx = avformat_alloc_context();&#xA;    int audio_stream_index = -1;&#xA;    AVCodecContext* codec_ctx = NULL;&#xA;    AVCodec* codec = NULL;&#xA;    AVPacket packet;&#xA;    AVFrame* frame = NULL;&#xA;    ao_device* device = NULL;&#xA;    ao_sample_format sample_format;&#xA;    uint8_t* output_buffer = NULL;&#xA;    int output_linesize;&#xA;&#xA;    avformat_open_input(&amp;format_ctx, "test.mp4", NULL, NULL);&#xA;    avformat_find_stream_info(format_ctx, NULL);&#xA;&#xA;    for (int i = 0; i &lt; format_ctx->nb_streams; i&#x2B;&#x2B;) {&#xA;        if (format_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {&#xA;            audio_stream_index = i;&#xA;            break;&#xA;        }&#xA;    }&#xA;&#xA;    codec_ctx = avcodec_alloc_context3(NULL);&#xA;    avcodec_parameters_to_context(codec_ctx, format_ctx->streams[audio_stream_index]->codecpar);&#xA;    codec = avcodec_find_decoder(codec_ctx->codec_id);&#xA;    avcodec_open2(codec_ctx, codec, NULL);&#xA;&#xA;    // Initialize libao&#xA;    ao_initialize();&#xA;    int driver_id = ao_default_driver_id();&#xA;&#xA;    sample_format.bits = 16;&#xA;    sample_format.channels = 2;&#xA;    sample_format.rate = 44100;&#xA;    sample_format.byte_format = AO_FMT_NATIVE;&#xA;    sample_format.matrix = 0;&#xA;    device = ao_open_live(driver_id, &amp;sample_format, NULL);&#xA;    if (!device) {&#xA;        fprintf(stderr, "Could not open audio device\n");&#xA;        return 1;&#xA;    }&#xA;&#xA;    frame = av_frame_alloc();&#xA;&#xA;    while (av_read_frame(format_ctx, &amp;packet) >= 0) {&#xA;        if (packet.stream_index == audio_stream_index) {&#xA;            int ret = avcodec_send_packet(codec_ctx, &amp;packet);&#xA;            if (ret &lt; 0) {&#xA;                fprintf(stderr, "Error sending packet for decoding\n");&#xA;                break;&#xA;            }&#xA;&#xA;            while (ret >= 0) {&#xA;                ret = avcodec_receive_frame(codec_ctx, frame);&#xA;                if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {&#xA;                    break;&#xA;                }&#xA;                else if (ret &lt; 0) {&#xA;                    fprintf(stderr, "Error during decoding\n");&#xA;                    break;&#xA;                }&#xA;&#xA;                ao_play(device, (char*)frame->data[0], frame->nb_samples * 2 * frame->channels);&#xA;            }&#xA;        }&#xA;        av_packet_unref(&amp;packet);&#xA;    }&#xA;&#xA;    // Clean up&#xA;    av_frame_free(&amp;frame);&#xA;    avcodec_free_context(&amp;codec_ctx);&#xA;    avformat_close_input(&amp;format_ctx);&#xA;    ao_close(device);&#xA;    ao_shutdown();&#xA;    av_freep(&amp;output_buffer);&#xA;&#xA;    return 0;&#xA;}&#xA;

  


  This is the closest I have gotten to play audio. It plays audio but there is a lot of static sound in the background. I did try using using frame->linesize[0] for num_bytes in ao_play but that didn't produce a sound that sounded at all like the video.

  


  Is there something I am doing wrong or missing ?

  


  EDIT : While doing more testing, I managed to find out that the above code sample outputs pure static in the left speaker, however, in the right speaker, it does play the audio, albeit heavily distorted ?