Recherche avancée

Sur d’autres sites (11825)

• C# / FFMPEG - Is this the best way to programmatically combine multiple video files in different formats and encodings into one ?

  28 avril 2021, par buggybud

  I've been trying to concat multiple videos into one. These videos may have different file types and extensions. As it stands I am only working with MP4 files that seem to have different resolutions, framerates, you name it.

  


  After having first followed the Stackoverflow answers that talked about using an intermediate file (convert all the files into one format, then concate that) I also came across a solution that uses what I think is called a 'concat video filter.'

  


  This would allow me to ignore the intermediate steps and just combine all the files by specifying their individual settings within one single FFMPEG command.

  


                  -i 1.mp4 -i 2.mp4 -i 3.mp4 -filter_complex \
                "[0:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,format=yuv420p[v0];
                 [1:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,format=yuv420p[v1];
                 [2:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,format=yuv420p[v2];
                 [v0][0:a][v1][1:a][v2][2:a]concat=n=3:v=1:a=1[v][a]" \
                -map "[v]" -map "[a]" -c:v libx264 -c:a aac -movflags +faststart output.mp4


  


  Provided above is the snippet that shows you how to combine three videos into one. I've used this Snippet within my code but as opposed to manually specifying the input files, and not finding a way to use the list file, I came across the very hacky solution to programatically create the above command parameters.

  


              var command = "";

            for (var i = 0; i < video_paths.Count; i++)
            {
                command += $"-i \"{video_paths[i]}\" ";
            }

            command += "-filter_complex ";
            command += "\"";

            for (var i = 0; i < video_paths.Count; i++)
            {
                command += $"[{i}:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1,setsar=1,fps=30,format=yuv420p[v{i}];";
            }

            for (var i = 0; i < video_paths.Count; i++)
            {
                command += $"[v{i}][{i}:a]";
            }

            command += $"concat=n={video_paths.Count}:v=1:a=1[v][a]\" ";
            command += $"-map \"[v]\" -map \"[a]\" -c:v libx264 -c:a aac -movflags +faststart \"{path}\"";

            ffmpeg(command);


  


  The above code is the solution to my problem. It works.

  


  The reason I made this Stackoverflow question is the following : Is this the best way to programatically make the arguments ? What is the maximum string limit of these arguments as my video paths are all absolute paths ? How can I make this look nicer and less chaotic in code ?

  


  Bud

  


• Recording RTSP steam with Python

  6 mai 2022, par ロジャー

  Currently I am using MediaPipe with Python to monitor RTSP steam from my camera, working as a security camera. Whenever the MediaPipe holistic model detects humans, the script writes the frame to a file.

  


  i.e.

  


  # cv2.VideoCapture(RTSP)
# read frame
# while mediapipe detect
#   cv2.VideoWriter write frame
# store file


  


  Recently I want to add audio recording support. I have done some research that it is not possible to record audio with OpenCV. It has to be done with FFMPEG or PyAudio.

  


  I am facing these difficulities.

  


  


  1. When a person walk through in front of the camera, it takes maybe less than 2 seconds. For the RTSP stream being read by OpenCV, human is detected with MediaPipe, and start FFMPEG for recording, that human should have walked far far away already. So FFMPEG method seems not working for me.

     


  


  2. For PyAudio method I am currently studying, I need to create 2 threads establishing individual RTSP connections. One thread is for video to be read by OpenCV and MediaPipe. The other thread is for audio to be recorded when the OpenCV thread notice human is detected. I have tried using several devices to read the RTSP streams. The devices are showing timestamps (watermark on the video) with several seconds in difference. So I doubt if I can get video from OpenCV and audio from PyAudio in sync when merging them into one single video.

     


  


  


  Is there any suggestion how to solve this problem ?

  


  Thanks.

  


• Programmatically accessing PTS times in MP4 container

  9 novembre 2022, par mcandril

  Background

  


  For a research project, we are recording video data from two cameras and feed a synchronization pulse directly into the microphone ADC every second.

  


  Problem

  


  We want to derive a frame time stamp in the clock of the pulse source for each camera frame to relate the camera images temporally. With our current methods (see below), we get a frame offset of around 2 frames between the cameras. Unfortunately, inspection of the video shows that we are clearly 6 frames off (at least at one point) between the cameras.
I assume that this is because we are relating audio and video signal wrong (see below).

  


  Approach I think I need help with

  


  I read that in the MP4 container, there should be PTS times for video and audio. How do we access those programmatically. Python would be perfect, but if we have to call ffmpeg via system calls, we may do that too ...

  


  What we currently fail with

  


  The original idea was to find video and audio times as

  


  audio_sample_times = range(N_audiosamples)/audio_sampling_rate
video_frame_times = range(N_videoframes)/video_frame_rate


  


  then identify audio_pulse_times in audio_sample_times base, calculate the relative position of each video_time to the audio_pulse_times around it, and select the same relative value to the corresponding source_pulse_times.

  


  However, a first indication that this approach is problematic is already that for some videos, N_audiosamples/audio_sampling_rate differs from N_videoframes/video_frame_rate by multiple frames.

  


  What I have found by now

  


  OpenCV's cv2.CAP_PROP_POS_MSEC seems to do exactly what we do, and not access any PTS ...

  


  Edit : What I took from the winning answer

  


  container = av.open(video_path)
signal = []
audio_sample_times = []
video_sample_times = []

for frame in tqdm(container.decode(video=0, audio=0)):
    if isinstance(frame, av.audio.frame.AudioFrame):
        sample_times = (frame.pts + np.arange(frame.samples)) / frame.sample_rate
        audio_sample_times += list(sample_times)
        signal_f_ch0 = frame.to_ndarray().reshape((-1, len(frame.layout.channels))).T[0]
        signal += list(signal_f_ch0)
    elif isinstance(frame, av.video.frame.VideoFrame):
        video_sample_times.append(float(frame.pts*frame.time_base))

signal = np.abs(np.array(signal))
audio_sample_times = np.array(audio_sample_times)
video_sample_times = np.array(video_sample_times)


  


  Unfortunately, in my particular case, all pts are consecutive and gapless, so the result is the same as with the naive solution ...
By picture clues, we identified a section of 10s in the videos, somewhere in which they desync, but can't find any traces of that in the data.