Recherche avancée

Sur d’autres sites (7886)

• Joining realtime raw PCM streams with ffmpeg and streaming them back out

  15 avril 2024, par Nathan Ladwig

  I am trying to use ffmpeg to join two PCM streams. I have it sorta kinda working but it's not working great.

  


  I am using Python to receive two streams from two computers running Scream Audio Driver ( ttps ://github.com/duncanthrax/scream )

  


  I am taking them in over UDP and writing them to pipes. The pipes are being received by ffmpeg and mixed, it's writing the mixed stream to another pipe. I'm reading that back in Python and sending it to the target receiver.

  


  My ffmpeg command is

  


  ['ffmpeg', 
'-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', '/tmp/ffmpeg-fifo-1',
'-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', '/tmp/ffmpeg-fifo-2',
'-filter_complex', '[0]aresample=async=1[a0],[1]aresample=async=1[a1],[a0][a1]amix', '-y',
'-f', 's24le', '-ac', '2', '-ar', '48000', '/tmp/ffmpeg-fifo-in']


  


  My main issue is that it should be reading ffmpeg-fifo-1 and ffmpeg-fifo-2 asynchronously, but it appears to be not. When the buffers get more than 50 frames out of sync with each other ffmpeg hangs and doesn't recover. I would like to fix this.

  


  In this hacky test code the number of frames sent over each stream are counted and empty frames are sent if the count hits 12. This keeps ffmpeg happy.

  


  The code below takes in two 48KHz 24-bit stereo PCM streams with Scream's header, mixes them, applies the same header, and sends them back out.

  


  It works most of the time. Sometimes I'm getting blasted with static, I think this is when only one or two bytes of a frame are making it to ffmpeg, and it loses track.

  


  The header is always 1152 bytes of pcm data with a 5 byte header. It's described in the Scream repo readme

  


  This is my header :

  


  01 18 02 03 00

  


  01 - 48KHz
18 - Sampling Rate (18h=24d, 24bit)
02 - 2 channels
03 00 - WAVEFORMATEXTENSIBLE

  


  import socket
import struct
import threading
import os
import sys
import time
import subprocess
import tempfile
import select

class Sender(threading.Thread):
    def __init__(self):
        super().__init__()
        TEMPDIR = tempfile.gettempdir() + "/"
        self.fifoin = TEMPDIR + "ffmpeg-fifo-in"
        self.start()

    def run(self):
        self.fd = open(self.fifoin, "rb")
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        while True:
            try:
                header = bytes([0x01, 0x18, 0x02, 0x03, 0x00])  # 48khz, 24-bit, stereo
                data = self.fd.read(1152)
                sendbuf = header + data
                self.sock.sendto(sendbuf, ("192.168.3.199", 4010))  # Audio sink
            except Exception as e:
                print("Except")
                print(e)

class Receiver(threading.Thread):
    def __init__(self):
        super().__init__()
        TEMPDIR = tempfile.gettempdir() + "/"
        self.fifo1 = TEMPDIR + "ffmpeg-fifo-1"
        self.fifo2 = TEMPDIR + "ffmpeg-fifo-2"
        self.fifoin = TEMPDIR + "ffmpeg-fifo-in"
        self.fifos = [self.fifo1, self.fifo2]
        try:
            try:
                os.remove(self.fifoin)
            except:
                pass
            os.mkfifo(self.fifoin)
        except:
            pass
        self.start()
        sender=Sender()

    def run(self):
        ffmpeg_command=['ffmpeg', '-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', self.fifo1,
                                  '-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', self.fifo2,
                                  '-filter_complex', '[0]aresample=async=1[a0],[1]aresample=async=1[a1],[a0][a1]amix', "-y", '-f', 's24le', '-ac', '2', '-ar', '48000', self.fifoin]
        print(ffmpeg_command)

        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        sock.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF,4096)
        sock.bind(("", 16401))

        recvbuf = bytearray(1157)
        framecount = [0,0]
        closed = 1
        while True:
            ready = select.select([sock], [], [], .2)
            if ready[0]:
                recvbuf, addr = sock.recvfrom(1157)
                if closed == 1:
                    for fifo in self.fifos:
                        try:
                            try:
                                os.remove(fifo)
                            except:
                                pass
                            os.mkfifo(fifo)
                        except:
                            pass
                    framecount = [0,0]
                    print("data, starting ffmpeg")
                    ffmpeg = subprocess.Popen (ffmpeg_command, shell=False, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
                    fifo1_fd = os.open(self.fifo1, os.O_RDWR)
                    fifo1_file = os.fdopen(fifo1_fd, 'wb', 0)
                    fifo2_fd = os.open(self.fifo2, os.O_RDWR)
                    fifo2_file = os.fdopen(fifo2_fd, 'wb', 0)
                    closed = 0
                    for i in range(0,6):
                        fifo1_file.write(bytes([0]*1157))
                        fifo2_file.write(bytes([0]*1157))

                if addr[0] == "192.168.3.199":
                    fifo1_file.write(recvbuf[5:])
                    framecount[0] = framecount[0] + 1

                if addr[0] == "192.168.3.119":
                    fifo2_file.write(recvbuf[5:])
                    framecount[1] = framecount[1] + 1

                # Keep buffers roughly in sync while playing
                targetframes=max(framecount)
                if targetframes - framecount[0] > 11:
                    while (targetframes - framecount[0]) > 0:
                        fifo1_file.write(bytes([0]*1157))
                        framecount[0] = framecount[0] + 1

                if targetframes - framecount[1] > 11:
                    while (targetframes - framecount[1]) > 0:
                        fifo2_file.write(bytes([0]*1157))
                        framecount[1] = framecount[1] + 1
            else:
                if closed == 0:
                    ffmpeg.kill()
                    print("No data, killing ffmpeg")
                    fifo1_file.close()
                    fifo2_file.close()
                    closed = 1
receiver=Receiver()

while True:
    time.sleep(50000)


  


  Does anybody have any pointers on how I can make this better ?

  


• threads : change the default for threads back to 1

  18 janvier 2012, par Janne Grunau

  threads : change the default for threads back to 1

• threads : change the default for threads back to 1

  18 janvier 2012, par Janne Grunau

  threads : change the default for threads back to 1