Recherche avancée

Sur d’autres sites (8560)

• Merge image, audio, video with no audio, video with audio, with ffmpeg

  17 février 2021, par Basj

  Similarly to Merge videos and images using ffmpeg (which is not a duplicate for the reasons explained below), I'd like to merge multiple inputs which can be either :

  


  

  • image only,

  


  • audio only,

  


  • video with audio,

  


  • video without audio

  


  


  into one output video, with stereo audio.

  


  Note : If multiple audio channels are playing at the same time, they should be mixed ; idem for video : the images from multiple sources should overlap.

  


  I tried this (comments added here) :

  


  ffmpeg 
  -i tmp/%04d.png       # [0]
  -f lavfi -t 0.1 -i anullsrc   # [1], if needed for inputs without sound?
  -i a.mp3              # [2], we keep 1 sec. from it; should start at 0'05" in output video
  -i b.mp3              # [3], we keep 2 sec. from it; should start at 0'06" in output video
  -i with_sound.mp4     # [4], we keep 3 sec. from it; should start at 0'07" in output video
  -i without_sound.mp4  # [5], we keep 4 sec. from it; should start at 0'08" in output video
  -filter_complex 
    [2]atrim=start=0:duration=1.0,asetpts=PTS-STARTPTS[s2];[s2]adelay=5000|5000[t2];
    [3]atrim=start=0:duration=2.0,asetpts=PTS-STARTPTS[s3];[s3]adelay=6000|6000[t3];
    [4]atrim=start=0:duration=3.0,asetpts=PTS-STARTPTS[s4];[s4]adelay=7000|7000[t4];
    [5]atrim=start=0:duration=4.0,asetpts=PTS-STARTPTS[s5];[s5]adelay=8000|8000[t5];
    [0][1][t2][t3][t4][t5]concat=n=6:a=1:v=1:unsafe=1[outv][outa]
  -map [outv] -map [outa] out.mp4


  


  I tried with various values concat=n=5, n=6, etc. and added unsafe=1, but I always get similar errors :

  


  


  [Parsed_adelay_2 @ 00000000006e8140] Media type mismatch between the 'Parsed_adelay_2' filter output pad 0 (audio) and the 'Parsed_concat_6' filter input pad 2 (video)
  
[AVFilterGraph @ 00000000006923c0] Cannot create the link adelay:0 -> concat:2

  


  


  or for the times I got it nearly working, the videos were added one after another and not merged / mixed.

  


  Also, I'm looking for a syntax that would work even if I don't know in advance if the input videos have or don't have audio (I'm doing a script and I don't know in advance if the videos have audio channels).

  


  ---

  


  TL ;DR :

  


  Question : How to mix/merge multiple inputs (image, audio, video with-or-without-sound) with ffmpeg, with a precise starting timestamp for each, into a single video output ?

  


• How to receive upd stream with OpenCV ?

  17 février 2021, par Legion

  I need to receive my stream from Jetson Nano to my OpenCV program on my PC (Windows 10).

  


  Ok, I stream camera from my device (Jetson Nano) using :

  


  cv::VideoWriter gst_udpsink("appsrc ! video/x-raw, format=BGR ! queue ! videoconvert ! video/x-raw, format=BGRx ! nvvidconv ! nvv4l2h264enc insert-vui=1 ! video/x-h264, stream-format=byte-stream ! h264parse ! rtph264pay pt=96 config-interval=1 ! udpsink host=224.1.1.1 port=5000 auto-multicast=true", cv::CAP_GSTREAMER, 0, fps, cv::Size (width, height));


  


  I installed OpenCV with Gstreamer(following that ) and tried that command

  


  c:\gstreamer\1.0\msvc_x86_64\bin\gst-launch-1.0.exe  udpsrc uri=udp://224.1.1.1:5000 auto-multicast=true ! application/x-rtp, media=video, encoding-name=H264 ! rtpjitterbuffer latency=300 ! rtph264depay ! decodebin ! d3dvideosink


  


  it is working, unfortunately, no matter what latency I set I still got quite a big lag.
When I try to use OpenCV

  


  cv::VideoCapture cap("udpsrc uri=udp://224.1.1.1:5000 auto-multicast=true ! application/x-rtp, media=video, encoding-name=H264 ! rtpjitterbuffer latency=300 ! rtph264depay ! decodebin ! videoconvert ! video/x-raw, format=BGR ! appsink", cv::CAP_GSTREAMER);


  


  I get

  


  [ WARN:0] global F:\Code\opencv_4.5.1\opencv-4.5.1\modules\videoio\src\cap_gstreamer.cpp (734) cv::GStreamerCapture::open OpenCV | GStreamer warning: Error opening bin: no element "udpsrc"
[ WARN:0] global F:\Code\opencv_4.5.1\opencv-4.5.1\modules\videoio\src\cap_gstreamer.cpp (501) cv::GStreamerCapture::isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created


  


  And .isOpened() give me false.
I’m don’t know why did I install something wrong ?

  


  I added everything to my PATH as instructed

  


  

  


  I also tried to use FFmpeg :

  


  setenv ("OPENCV_FFMPEG_CAPTURE_OPTIONS", "protocol_whitelist;file,rtp,udp", 1);
cap = cv::VideoCapture("test.sdp", cv::CAP_FFMPEG);


  


  I get :

  


  [rtp @ 0000014dc1f83bc0] Protocol 'rtp' not on whitelist 'file,crypto,data'!


  


  I have no setenv() so I tried this and it seems that’s a problem, any idea ?

  


  Shell equivalent

  


  ffplay myFile.sdp -protocol_whitelist file,udp,rtp -fflags nobuffer


  


  Is working successfully (with delay but successfully).

  


  I'm willing to change anything to make it work ! If it's possible with FFmpeg/GStreamer/vlclib, I can change the Jetson side as well, thanks for any help !

  


• Mix audio from various sources, regardless if an input video has sound or not

  17 février 2021, par Basj

  The following code :

  


  ffmpeg -i test.png            
       -t 2 -i a.mp3          
       -t 4 -i video.mp4      
       -t 1 -i b.mp3          
       -filter_complex [1]adelay=2000|2000[s1];[3]adelay=5000|5000[s3];[s1][2][s3]amix=inputs=3[outa];[0][2]overlay[outv]^
       -map [outa] -map [outv]^
       out.mp4 -y


  


  works, and mixes the audio from the MP3s (time-shifted, as desired) and from the MP4 video.

  


  But it fails if the MP4 has no audio channel (= a no-sound video) :

  


  


  Stream specifier '' in filtergraph description ... matches no stream

  


  


  I'd like my script to work in both cases, if the video has audio or not.

  


  How to include [2] in the amix if and only if this video has sound ?

  


  ---

  


  Note : A good way would be to be able to load a MP4 with always a sound stream : the original sound stream if the video has sound, and a silence audio track if the MP4 has no sound in it. Is this possible with a single command in ffmpeg ?