Recherche avancée

Sur d’autres sites (6371)

• How to use audio frame after decode mp3 file using pyav, ffmpeg, python

  2 janvier 2021, par Long Tran Dai

  I am using using python with pyav, ffmpeg to decode mp3 in the memory. I know there are some other way to do it, like pipe ffmpeg command. However, I would like to explore pyav and ffmpeg API. So I have the following code. It works but the sound is very noisy, although hearable :

  


  import numpy as np&#xA;import av # to convert mp3 to wav using ffmpeg&#xA;import pyaudio # to play music&#xA;&#xA;mp3_path = &#x27;D:/MyProg/python/SauTimThiepHong.mp3&#x27;&#xA;&#xA;def decodeStream(mp3_path):&#xA;  # Run NOT OK&#xA;  &#xA;  container = av.open(mp3_path)&#xA;  stream = next(s for s in container.streams if s.type == &#x27;audio&#x27;)&#xA;  frame_count = 0&#xA;  data = bytearray()&#xA;  for packet in container.demux(stream):&#xA;    # <class>&#xA;    # We need to skip the "flushing" packets that `demux` generates.&#xA;    #if frame_count == 5000 : break         &#xA;    if packet.dts is None:&#xA;        continue&#xA;    for frame in packet.decode():   &#xA;        #&#xA;        # type(frame) : <class>&#xA;        #frame.samples = 1152 : 1152 diem du lieu : Number of audio samples (per channel)&#xA;        # moi frame co size = 1152 (diem) * 2 (channels) * 4 (bytes / diem) = 9216 bytes&#xA;        # 11021 frames&#xA;        #arr = frame.to_ndarray() # arr.nbytes = 9216&#xA;&#xA;        #channels = []  &#xA;        channels = frame.to_ndarray().astype("float16")&#xA;        #for plane in frame.planes:&#xA;            #channels.append(plane.to_bytes()) #plane has 4 bytes / sample, but audio has only 2 bytes&#xA;        #    channels.append(np.frombuffer(plane, dtype=np.single).astype("float16"))&#xA;            #channels.append(np.frombuffer(plane, dtype=np.single)) # kieu np.single co 4 bytes&#xA;        if not frame.is_corrupt:&#xA;            #data.extend(np.frombuffer(frame.planes[0], dtype=np.single).astype("float16")) # 1 channel: noisy&#xA;            # type(planes) : <class>&#xA;            frame_count &#x2B;= 1&#xA;            #print( &#x27;>>>> %04d&#x27; % frame_count, frame)   &#xA;            #if frame_count == 5000 : break     &#xA;            # mix channels:&#xA;            for i in range(frame.samples):                &#xA;                for ch in channels: # dec_ctx->channels&#xA;                    data.extend(ch[i]) #noisy&#xA;                    #fwrite(frame->data[ch] &#x2B; data_size*i, 1, data_size, outfile)&#xA;  return bytes(data)&#xA;</class></class></class>

  


  I use pipe ffmpeg to get decoded data to compare and find they are different :

  


  def RunFFMPEG(mp3_path, target_fs = "44100"):
    # Run OK
    import subprocess
    # init command
    ffmpeg_command = ["ffmpeg", "-i", mp3_path,
                   "-ab", "128k", "-acodec", "pcm_s16le", "-ac", "0", "-ar", target_fs, "-map",
                   "0:a", "-map_metadata", "-1", "-sn", "-vn", "-y",
                   "-f", "wav", "pipe:1"]
    # excute ffmpeg command
    pipe = subprocess.run(ffmpeg_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize= 10**8)
    # debug
    #print(pipe.stdout, pipe.stderr)
    # read signal as numpy array and assign sampling rate
    #audio_np = np.frombuffer(buffer=pipe.stdout, dtype=np.uint16, offset=44)
    #audio_np = np.frombuffer(buffer=pipe.stdout, dtype=np.uint16)
    #sig, fs  = audio_np, target_fs
    #return audio_np
    return pipe.stdout[78:]     


  


  Then I use pyaudio to play data and find it very noisy

  


  p = pyaudio.PyAudio()
streamOut = p.open(format=pyaudio.paInt16, channels=2, rate= 44100, output=True)
#streamOut = p.open(format=pyaudio.paInt16, channels=1, rate= 44100, output=True)

mydata = decodeStream(mp3_path)
print("bytes of mydata = ", len(mydata))
#print("bytes of mydata = ", mydata.nbytes)

ffMpegdata = RunFFMPEG(mp3_path)
print("bytes of ffMpegdata = ", len(ffMpegdata)) 
#print("bytes of ffMpegdata = ", ffMpegdata.nbytes)

minlen = min(len(mydata), len(ffMpegdata))
print("mydata == ffMpegdata", mydata[:minlen] == ffMpegdata[:minlen]) # ffMpegdata.tobytes()[:minlen] )

#bytes of mydata =  50784768
#bytes of ffMpegdata =  50784768
#mydata == ffMpegdata False

streamOut.write(mydata)
streamOut.write(ffMpegdata)
streamOut.stop_stream()
streamOut.close()
p.terminate()


  


  Please help me to understand decoded frame of pyav api (after for frame in packet.decode() :). Should it be processed more ? or I have some error ?

  


  It makes me crazy for 3 days. I could not guess where to go.

  


  Thank you very much.

  


• ffmpeg lags when streaming video+audio from RPi Zero W with Logitech C920

  7 janvier 2021, par Ema

  I've been trying to setup a baby monitor with a Raspberry Pi Zero and a Logitech C920 webcam. I does work with VLC (cvlc) but it lags too much and gets worse over time.

  


  So I am playing around with ffmpeg and I am getting some better results. This is what I've done so far.

  


  First I set the webcam to output h264 1080p natively (the Pi Zero W can't afford to do any transcoding).

  


  v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=1


  


  Now, if I stream audio only with

  


  ffmpeg \
-f alsa \
-i hw:1,0 \
-vn \
-flags +global_header \
-acodec aac \
-ac 1 \
-ar 16000 \
-ab 16k \
-f rtp rtp://192.168.0.10:5002 > audio.sdp


  


  it works great and the lag is about 1 second (definitely acceptable).

  


  If I stream video only with

  


  ffmpeg \
-f v4l2 \
-vcodec h264 \
-i /dev/video0 \
-an \
-vcodec copy \
-pix_fmt yuv420p \
-r 30 \
-b:v 512k \
-flags +global_header \
-f rtp rtp://192.168.0.10:5000 > video.sdp


  


  same result, very little lag (for some reason the first -vcodec is necessary to force the webcam to output h264).

  


  However, when I stream both with

  


  ffmpeg \
-f v4l2 \
-vcodec h264 \
-i /dev/video0 \
-f alsa \
-i hw:1,0 \
-an \
-preset ultrafast \
-tune zerolatency \
-vcodec copy \
-pix_fmt yuv420p \
-r 30 \
-b:v 512k \
-flags +global_header \
-f rtp rtp://192.168.0.10:5000 \
-vn \
-flags +global_header \
-acodec aac \
-ac 1 \
-ar 16000 \
-ab 16k \
-f rtp rtp://192.168.0.10:5002 > both.sdp


  


  the lag ramps up to 10 seconds and audio and video are out of sync. Does anybody know why ?

  


  I've tried UDP and TCP instead of RTP but then the lag is always high, even with audio/video only.

  


  Any suggestion is much appreciated.

  


  P.S. On the client side (MacOS) I'm receiving with

  


  ffplay -protocol_whitelist file,rtp,udp -i file.sdp


  


• scripting massive number of files with ffmpeg [closed]

  2 décembre 2020, par 8Liter

  Alright, I've got over 5000 MP4 files in a single directory that I would ultimately like to process using ffmpeg. I've got a few different solutions that all work by themselves, but put together do not make my job any easier.
The current file list looks like this, in one single directory :

  


  

  • 10-1.mp4

  


  • 10-2.mp4

  


  • 10123-1.mp4

  


  • 10123-2.mp4

  


  • 10123-3.mp4

  


  • 10123-4.mp4

  


  • 10123-5.mp4

  


  • 10123-6.mp4

  


  • 102-1.mp4

  


  • 103-1.mp4

  


  • 103-2.mp4

  


  • 103-3.mp4

  


  • 107-1.mp4

  


  • 107-2.mp4

  


  • 107-3.mp4

  


  • 107-4.mp4

  


  • 107-5.mp4

  


  • 107-6.mp4

  


  • 11-1.mp4

  


  • 11-2.mp4

  


  


  The ideal process I would like is the following :

  


  A. Take however many files in the directory have a particular prefix, for example the two "11" files at the bottom, and concatenate them into a single MP4 file. The end result is a single "11.MP4"

  


  B. Delete the original two "11-1.mp4" and "11-2.mp4", keeping only the new "11.mp4" complete file.

  


  C. Repeat steps A-B for all other files in this directory

  


  This is not apparently possible right now from what I can glean from other threads, but I've tested a more manual approach which is not clean OR fast, and this is what my workflow looks like in real life...

  


  

  1. move files with same prefix into new folder (I have a working bat file that will do this for me)

  


  2. run a ffmpeg bat file to process an "output.mp4" file (I have a working bat file that will do this for me)

  


  3. delete the original files

  


  4. rename the output.mp4 file to the prefix name (i.e. 11.mp4)

  


  5. copy that file back into the new directory

  


  6. repeat steps 1-5 a thousand times.

  


  


  I've also looked into creating all new directories BASED on the filename (I have a working bat file that will do this for me) and then copy my ffmpeg bat file into each directory, and run each bat file manually... but again it's a ton of work.

  


  (FROM STEP 1 ABOVE)

  


  @echo off
setlocal

set "basename=."
for /F "tokens=1* delims=.*" %%a in ('dir /B /A-D ^| sort /R') do (
   set "filename=%%a"
   setlocal EnableDelayedExpansion
   for /F "delims=" %%c in ("!basename!") do if "!filename:%%c=!" equ "!filename!" (
      set "basename=!filename!"
      md "!basename!"
   )
   move "!filename!.%%b" "!basename!"
   for /F "delims=" %%c in ("!basename!") do (
      endlocal
      set "basename=%%c

   )
)


  


  (FROM STEP 2 ABOVE)

  


  :: Create File List
del "F:\videos\*.txt" /s /f /q
for %%i in (*.mp4) do echo file '%%i'>> mylist.txt

:: Concatenate Files
ffmpeg.exe -f concat -safe 0 -i mylist.txt -c copy output.mp4


  


  Any ideas how I can approach this ? I'm open to powershell, batch, even python if I need to.