Recherche avancée

Sur d’autres sites (7812)

• How to play video file with audio with DearPyGUI (Python) ?

  1er mars 2023, par Vi Tiet

  I'm using DearPyGUI to make a simple media player that can play video file (mp4, etc.) together with it's audio. The pre-requisite is that DearPyGUI is a must, however video feature will not exist until v2.0, which is still far in the future.

  


  Currently, I can only render the frames using OpenCV library for Python, however, the problem is how can I play the audio as well as play it in sync with the output video frames ?

  


  For context, I'm quite new to Python, and I don't know much about video and audio streaming, but I've thought of some approaches to this problem by looking through help posts online (However, I still have no idea how I can implement any of these seamlessly) :

  


  


  1. OpenCV for video frames, and audio ??? some libraries like ffmpeg-python or miniaudio to play sound... (How...?)

     


  


  2. Extract video frames and audio here and then use the raw data to play it (How...?)

     


  


  3. This example here is pretty close to what I want excluding the playing video and audio part, but I have no idea where to go from there. The video stream and the audio stream are instances of ffmpeg.nodes.FilterableStream, and they appear to hold addresses to somewhere. (No idea...)

     


  


  4. Another very close idea is using ffpyplayer I was able to get the video frame. However, the below code yields a blueish purple color tint to the video, and the frame rate is very slow compared to original (So close...)

     


  


  


  import time
import numpy as np
import cv2 as cv
from ffpyplayer.player import MediaPlayer


# https://github.com/Kazuhito00/Image-Processing-Node-Editor/blob/main/node_editor/util.py 
def cv2dpg(frame): 

    data = cv.resize(frame, (VIDEO_WIDTH, VIDEO_HEIGHT))
    data = np.flip(frame, 2)
    data = data.ravel()
    data = np.asfarray(data, dtype=np.float32)

    return np.true_divide(data, 255.0)


# https://stackoverflow.com/questions/59611075/how-would-i-go-about-playing-a-video-stream-with-ffpyplayer
# https://matham.github.io/ffpyplayer/examples.html#examples
def play_video(loaded_file_path):

    global player, is_playing
    player = MediaPlayer(loaded_file_path)

    while is_playing:

        frame, val = player.get_frame()

        if val == 'eof':
            is_playing = False
            break

        elif not frame:
            time.sleep(0.01)

        elif val != 'eof' and frame is not None:
            img, t = frame
            w = img.get_size()[0]
            h = img.get_size()[1]
            cv_mat = np.uint8(np.asarray(list(img.to_bytearray()[0])).reshape((h, w, 3)))
            texture_data = cv2dpg(cv_mat)
            dpg.set_value(VIDEO_CANVAS_TAG, texture_data)

    dpg.set_value(VIDEO_CANVAS_TAG, DEFAULT_VIDEO_TEXTURE)


  


  I still need to do more research, but any pointer to somewhere good to start off (either handling raw data or using different libraries) would be greatly appreciated !

  


  EDIT :
For more context, I'm using raw texture like this example of DearPyGUI official documentation to render the video frames that were extracted in the while loop.

  


• How to force specific AVInputFormat in code (FFMPEG) ?

  18 février 2020, par kugipark

  ++Plaease understand that maybe some words or sentences could not be correct English.++

  I’m novice programmer and developing a video player app for Android which can play 5ch .avi video from http. It based on ffplay.c in FFMPEG libarary.

  Currently I have 2 problems for this below. It has occured only in android devices.

  ---

  1) Too much time taken for detecting format.

  • opening and finding stream info takes more than a minute when I trying to open 5ch video source whereas normal .mp4 (h264) source open almost immediately.

  2) Demuxing is too slow when it comes to large size video.

  • If the resolution of video getting larger, then displaying frame rate getting slower even though there are enough memory, network and CPU resources physically.

  To resolve my problem, I tried to force the format and decoders but I couldn’t found some information for specifying the input source in code level.
  The official doc site only refers to a sentence about those parameters like "If non-NULL, this parameter forces a specific input format. Otherwise the format is autodetected.". So I don’t have any clues how to set the number of streams, decoders, decoders’ private options, and etc. (And also which parameters should I manage.) If someone knows how to set the options(like AVDictionary), and pass to the av functions, please let me know an exmple. The source contains 2 video streams, 1 audio stream, and 2 more extra streams (for custom data like gps). The stream information of video is below. I printed it manually, and these are auto-detected information.

      ---------- File format information ----------
  
  flags=2097152
  
  video_codec_id=0 (NONE)
  
  audio_codec_id=0 (NONE)
  
  ctx_flags=0
  
  data_codec_id=0 (NONE)
  
  format_whitelist=(null)
  
  iformat=0xa19d0d2c
  
  ---------- Stream information ----------
  
  stream 1 of 5:
  
  ----- common ----------
  
  bit_rate: 11383235
  
  bits_per_coded_sample: 24
  
  bits_per_raw_sample: 0
  
  codec_id: 0x1C (H264)
  
  codec_tag: 875967048
  
  extradata_size: 0
  
  level: -99
  
  profile: -99
  
  sample_rate: 0
  
  ----- Video Stream ----------
  
  chroma_location: 0
  
  color_primaries: 2
  
  color_space: 2
  
  color_trc: 2
  
  field_order: 0
  
  format: -1 (NONE)
  
  height: 1080
  
  width: 1920
  
  sample_aspect_ratio.den: 1
  
  sample_aspect_ratio.num: 0
  
  video_delay: 0
  
  ----------------------------------------
  
  stream 2 of 5:
  
  ----- common ----------
  
  bit_rate: 6185438
  
  bits_per_coded_sample: 24
  
  bits_per_raw_sample: 0
  
  codec_id: 0x1C (H264)
  
  codec_tag: 875967048
  
  extradata_size: 0
  
  level: -99
  
  profile: -99
  
  sample_rate: 0
  
  ----- Video Stream ----------
  
  chroma_location: 0
  
  color_primaries: 2
  
  color_space: 2
  
  color_trc: 2
  
  field_order: 0
  
  format: -1 (NONE)
  
  height: 720
  
  width: 1280
  
  sample_aspect_ratio.den: 1
  
  sample_aspect_ratio.num: 0
  
  video_delay: 0
  
  ----------------------------------------
  
  stream 3 of 5:
  
  ----- common ----------
  
  bit_rate: 352800
  
  bits_per_coded_sample: 16
  
  bits_per_raw_sample: 0
  
  codec_id: 0x10000 (PCM_S16LE)
  
  codec_tag: 1
  
  extradata_size: 0
  
  level: -99
  
  profile: -99
  
  sample_rate: 22050
  
  ----- Audio Stream ----------
  
  block_align: 2
  
  channels: 1
  
  channel_layout: 0
  
  color_range: 0
  
  frame_size: 0
  
  initial_padding: 0
  
  seek_preroll: 0
  
  trailing_padding: 0
  
  ----------------------------------------
  
  stream 4 of 5:
  
  ----- common ----------
  
  bit_rate: 15625
  
  bits_per_coded_sample: 0
  
  bits_per_raw_sample: 0
  
  codec_id: 0x0 (NONE)
  
  codec_tag: 0
  
  extradata_size: 0
  
  level: -99
  
  profile: -99
  
  sample_rate: 0
  
  ----- Subtitle Stream ----------
  
  ----------------------------------------
  
  stream 5 of 5:
  
  ----- common ----------
  
  bit_rate: 33862
  
  bits_per_coded_sample: 0
  
  bits_per_raw_sample: 0
  
  codec_id: 0x0 (NONE)
  
  codec_tag: 0
  
  extradata_size: 0
  
  level: -99
  
  profile: -99
  
  sample_rate: 0
  
  ----- Subtitle Stream ----------

• Huge memory leak when filtering video with libavfilter

  29 mai 2017, par Captain Jack

  I have a relatively simple FFMPEG C program, to which a video frame is fed, processed via filter graph and sent to frame renderer.

  Here are some code snippets :

  /* Filter graph here */
  
  char args[512];
  
  enum AVPixelFormat pix_fmts[] = {AV_PIX_FMT_RGB32 };    
  
  AVFilterGraph   *filter_graph;
  
  avfilter_register_all();
  
  AVFilter *buffersrc  = avfilter_get_by_name("buffer");
  
  AVFilter *buffersink = avfilter_get_by_name("ffbuffersink");
  
  AVBufferSinkParams *buffersink_params;
  
  AVFilterInOut *outputs = avfilter_inout_alloc();
  
  AVFilterInOut *inputs  = avfilter_inout_alloc();
  
  filter_graph = avfilter_graph_alloc();
  
  
  
  snprintf(args, sizeof(args),
  
          "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
  
          av->codec_ctx->width, av->codec_ctx->height, av->codec_ctx->pix_fmt,
  
          av->codec_ctx->time_base.num, av->codec_ctx->time_base.den,
  
          av->codec_ctx->sample_aspect_ratio.num, av->codec_ctx->sample_aspect_ratio.den);
  
  
  
  if(avfilter_graph_create_filter(&av->buffersrc_ctx, buffersrc, "in",args, NULL, filter_graph) < 0) 
  
  {
  
      fprintf(stderr, "Cannot create buffer source\n");
  
      return(0);
  
  }
  
  
  
   /* buffer video sink: to terminate the filter chain. */
  
  buffersink_params = av_buffersink_params_alloc();
  
  buffersink_params->pixel_fmts = pix_fmts;
  
  
  
  if(avfilter_graph_create_filter(&av->buffersink_ctx, buffersink, "out",NULL, buffersink_params, filter_graph) < 0) 
  
  {
  
      printf("Cannot create buffer sink\n");
  
      return(HACKTV_ERROR);
  
  }
  
  
  
    /* Endpoints for the filter graph. */
  
      outputs->name       = av_strdup("in");
  
      outputs->filter_ctx = av->buffersrc_ctx;
  
      outputs->pad_idx    = 0;
  
      outputs->next       = NULL;
  
  
  
      inputs->name       = av_strdup("out");
  
      inputs->filter_ctx = av->buffersink_ctx;
  
      inputs->pad_idx    = 0;
  
      inputs->next       = NULL;
  
  
  
  const char *filter_descr = "vflip";
  
  
  
      if (avfilter_graph_parse_ptr(filter_graph, filter_descr, &inputs, &outputs, NULL) < 0)
  
  {
  
      printf("Cannot parse filter graph\n");
  
      return(0);
  
  }
  
  
  
   if (avfilter_graph_config(filter_graph, NULL) < 0) 
  
  {
  
      printf("Cannot configure filter graph\n");
  
      return(0);
  
  }
  
  
  
  av_free(buffersink_params);
  
  avfilter_inout_free(&inputs);
  
  avfilter_inout_free(&outputs);

  The above code is called by these elsewhere :

  av->frame_in->pts = av_frame_get_best_effort_timestamp(av->frame_in);
  
  
  
  /* push the decoded frame into the filtergraph*/
  
  if (av_buffersrc_add_frame(av->buffersrc_ctx, av->frame_in) < 0) 
  
  {
  
      printf( "Error while feeding the filtdergraph\n");
  
      break;
  
   }
  
  
  
   /* pull filtered pictures from the filtergraph */ 
  
   if(av_buffersink_get_frame(av->buffersink_ctx, av->frame_out) < 0) 
  
   {
  
        printf( "Error while sourcing the filtergraph\n");
  
         break;
  
    }  
  
  
  
  /* do stuff with frame */

  Now, the code works absolutely fine and the video comes out the way I expect it to (vertically flipped for testing purposes).

  The biggest issue I have is that there is a massive memory leak. An high res video will consume 2Gb in a matter of seconds and crash the program. I traced the leak to this piece of code :

  /* push the decoded frame into the filtergraph*/
  
  if (av_buffersrc_add_frame(av->buffersrc_ctx, av->frame_in) < 0) 

  If I bypass the filter by doing av->frame_out=av->frame_in; without pushing the frame into it (and obviously not pulling from it), there is no leak and memory usage is stable.

  Now, I am very new to C, so be gentle, but it seems like I should be clearing out the buffersrc_ctx somehow but no idea how. I’ve looked in official documentations but couldn’t find anything.

  Can someone advise ?