Recherche avancée

Sur d’autres sites (4595)

• Joining realtime raw PCM streams with ffmpeg and streaming them back out

  15 avril 2024, par Nathan Ladwig

  I am trying to use ffmpeg to join two PCM streams. I have it sorta kinda working but it's not working great.

  


  I am using Python to receive two streams from two computers running Scream Audio Driver ( ttps ://github.com/duncanthrax/scream )

  


  I am taking them in over UDP and writing them to pipes. The pipes are being received by ffmpeg and mixed, it's writing the mixed stream to another pipe. I'm reading that back in Python and sending it to the target receiver.

  


  My ffmpeg command is

  


  ['ffmpeg', 
'-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', '/tmp/ffmpeg-fifo-1',
'-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', '/tmp/ffmpeg-fifo-2',
'-filter_complex', '[0]aresample=async=1[a0],[1]aresample=async=1[a1],[a0][a1]amix', '-y',
'-f', 's24le', '-ac', '2', '-ar', '48000', '/tmp/ffmpeg-fifo-in']


  


  My main issue is that it should be reading ffmpeg-fifo-1 and ffmpeg-fifo-2 asynchronously, but it appears to be not. When the buffers get more than 50 frames out of sync with each other ffmpeg hangs and doesn't recover. I would like to fix this.

  


  In this hacky test code the number of frames sent over each stream are counted and empty frames are sent if the count hits 12. This keeps ffmpeg happy.

  


  The code below takes in two 48KHz 24-bit stereo PCM streams with Scream's header, mixes them, applies the same header, and sends them back out.

  


  It works most of the time. Sometimes I'm getting blasted with static, I think this is when only one or two bytes of a frame are making it to ffmpeg, and it loses track.

  


  The header is always 1152 bytes of pcm data with a 5 byte header. It's described in the Scream repo readme

  


  This is my header :

  


  01 18 02 03 00

  


  01 - 48KHz
18 - Sampling Rate (18h=24d, 24bit)
02 - 2 channels
03 00 - WAVEFORMATEXTENSIBLE

  


  import socket
import struct
import threading
import os
import sys
import time
import subprocess
import tempfile
import select

class Sender(threading.Thread):
    def __init__(self):
        super().__init__()
        TEMPDIR = tempfile.gettempdir() + "/"
        self.fifoin = TEMPDIR + "ffmpeg-fifo-in"
        self.start()

    def run(self):
        self.fd = open(self.fifoin, "rb")
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        while True:
            try:
                header = bytes([0x01, 0x18, 0x02, 0x03, 0x00])  # 48khz, 24-bit, stereo
                data = self.fd.read(1152)
                sendbuf = header + data
                self.sock.sendto(sendbuf, ("192.168.3.199", 4010))  # Audio sink
            except Exception as e:
                print("Except")
                print(e)

class Receiver(threading.Thread):
    def __init__(self):
        super().__init__()
        TEMPDIR = tempfile.gettempdir() + "/"
        self.fifo1 = TEMPDIR + "ffmpeg-fifo-1"
        self.fifo2 = TEMPDIR + "ffmpeg-fifo-2"
        self.fifoin = TEMPDIR + "ffmpeg-fifo-in"
        self.fifos = [self.fifo1, self.fifo2]
        try:
            try:
                os.remove(self.fifoin)
            except:
                pass
            os.mkfifo(self.fifoin)
        except:
            pass
        self.start()
        sender=Sender()

    def run(self):
        ffmpeg_command=['ffmpeg', '-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', self.fifo1,
                                  '-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', self.fifo2,
                                  '-filter_complex', '[0]aresample=async=1[a0],[1]aresample=async=1[a1],[a0][a1]amix', "-y", '-f', 's24le', '-ac', '2', '-ar', '48000', self.fifoin]
        print(ffmpeg_command)

        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        sock.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF,4096)
        sock.bind(("", 16401))

        recvbuf = bytearray(1157)
        framecount = [0,0]
        closed = 1
        while True:
            ready = select.select([sock], [], [], .2)
            if ready[0]:
                recvbuf, addr = sock.recvfrom(1157)
                if closed == 1:
                    for fifo in self.fifos:
                        try:
                            try:
                                os.remove(fifo)
                            except:
                                pass
                            os.mkfifo(fifo)
                        except:
                            pass
                    framecount = [0,0]
                    print("data, starting ffmpeg")
                    ffmpeg = subprocess.Popen (ffmpeg_command, shell=False, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
                    fifo1_fd = os.open(self.fifo1, os.O_RDWR)
                    fifo1_file = os.fdopen(fifo1_fd, 'wb', 0)
                    fifo2_fd = os.open(self.fifo2, os.O_RDWR)
                    fifo2_file = os.fdopen(fifo2_fd, 'wb', 0)
                    closed = 0
                    for i in range(0,6):
                        fifo1_file.write(bytes([0]*1157))
                        fifo2_file.write(bytes([0]*1157))

                if addr[0] == "192.168.3.199":
                    fifo1_file.write(recvbuf[5:])
                    framecount[0] = framecount[0] + 1

                if addr[0] == "192.168.3.119":
                    fifo2_file.write(recvbuf[5:])
                    framecount[1] = framecount[1] + 1

                # Keep buffers roughly in sync while playing
                targetframes=max(framecount)
                if targetframes - framecount[0] > 11:
                    while (targetframes - framecount[0]) > 0:
                        fifo1_file.write(bytes([0]*1157))
                        framecount[0] = framecount[0] + 1

                if targetframes - framecount[1] > 11:
                    while (targetframes - framecount[1]) > 0:
                        fifo2_file.write(bytes([0]*1157))
                        framecount[1] = framecount[1] + 1
            else:
                if closed == 0:
                    ffmpeg.kill()
                    print("No data, killing ffmpeg")
                    fifo1_file.close()
                    fifo2_file.close()
                    closed = 1
receiver=Receiver()

while True:
    time.sleep(50000)


  


  Does anybody have any pointers on how I can make this better ?

  


• Files created with a direct stream copy using FFmpeg's libavformat API play back too fast at 3600 fps

  2 octobre 2013, par Chris Ballinger

  I am working on a libavformat API wrapper that converts MP4 files with H.264 and AAC to MPEG-TS segments suitable for streaming. I am just doing a simple stream copy without re-encoding, but the files I produce play the video back at 3600 fps instead of 24 fps.

  Here are some outputs from ffprobe https://gist.github.com/chrisballinger/6733678, the broken file is below :

  r_frame_rate=1/1
  
  avg_frame_rate=0/0
  
  time_base=1/90000
  
  start_pts=0
  
  start_time=0.000000
  
  duration_ts=2999
  
  duration=0.033322

  The same input file manually sent through ffmpeg has proper timestamp information :

  r_frame_rate=24/1
  
  avg_frame_rate=0/0
  
  time_base=1/90000
  
  start_pts=126000
  
  start_time=1.400000
  
  duration_ts=449850
  
  duration=4.998333

  I believe the problem lies somewhere in my setup of libavformat here : https://github.com/OpenWatch/FFmpegWrapper/blob/master/FFmpegWrapper/FFmpegWrapper.m#L349 where I repurposed a bunch of code from ffmpeg.c that was required for the direct stream copy.

  Since 3600 seems like a "magic number" (60*60), it could be as simple as me not setting the time scale properly, but I can't figure out where my code diverges from ffmpeg/avconv itself.

  Similar question here, but I don't think they got as far as I did : Muxing a H.264 Annex B & AAC stream using libavformat with vcopy/acopy

• Revision 6c2035744a : Merge "Nextgen branch cleanup : add back some SVC functions" into nextgen

  9 juillet 2015, par Yunqing Wang

  Merge "Nextgen branch cleanup : add back some SVC functions" into nextgen