Recherche avancée

Sur d’autres sites (10188)

• Video playing, how to play a video back a a rapid rate at random timestamp locations

  10 avril 2020, par Zarc Rowden

  Note : this is a mildly general question that is looking more for pointers in the right direction and not exactly requiring a concise coded answer. I appreciate any and all input, thank you for lending your brain power to me for this moment :)

  



  I have a script that receives midi messages in real time and triggers playback of a single video on various timestamps that are changed/selected regularly and randomly by a user. Currently this is working in the browser, however, I've realized that there is some noticeable latency between (i'm guessing the cause here so please correct me) the moment a request to play a video at a specific time is made(note : the video is not being requested over the wire, this action does not take place until a JS Blob Url is loaded into the player) and the moment where that request is fulfilled and delivered from storage to pixels on the screen.

  



  My question is : Is it reasonable to assume that there is a tool out there, that given the correct video format and optimizations both in the code and in the file that could load an entire, say : 1 gb video into memory and play it back at random timestamps every 60 milliseconds at completely random, constantly changing timestamps.

  



  If you're now all the way down here... Thanks for reading this far, or scanning ! Please let me know if this question makes any sense / could be improved, I'm happy to clarify further.

  


• Joining realtime raw PCM streams with ffmpeg and streaming them back out

  15 avril 2024, par Nathan Ladwig

  I am trying to use ffmpeg to join two PCM streams. I have it sorta kinda working but it's not working great.

  


  I am using Python to receive two streams from two computers running Scream Audio Driver ( ttps ://github.com/duncanthrax/scream )

  


  I am taking them in over UDP and writing them to pipes. The pipes are being received by ffmpeg and mixed, it's writing the mixed stream to another pipe. I'm reading that back in Python and sending it to the target receiver.

  


  My ffmpeg command is

  


  ['ffmpeg', 
'-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', '/tmp/ffmpeg-fifo-1',
'-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', '/tmp/ffmpeg-fifo-2',
'-filter_complex', '[0]aresample=async=1[a0],[1]aresample=async=1[a1],[a0][a1]amix', '-y',
'-f', 's24le', '-ac', '2', '-ar', '48000', '/tmp/ffmpeg-fifo-in']


  


  My main issue is that it should be reading ffmpeg-fifo-1 and ffmpeg-fifo-2 asynchronously, but it appears to be not. When the buffers get more than 50 frames out of sync with each other ffmpeg hangs and doesn't recover. I would like to fix this.

  


  In this hacky test code the number of frames sent over each stream are counted and empty frames are sent if the count hits 12. This keeps ffmpeg happy.

  


  The code below takes in two 48KHz 24-bit stereo PCM streams with Scream's header, mixes them, applies the same header, and sends them back out.

  


  It works most of the time. Sometimes I'm getting blasted with static, I think this is when only one or two bytes of a frame are making it to ffmpeg, and it loses track.

  


  The header is always 1152 bytes of pcm data with a 5 byte header. It's described in the Scream repo readme

  


  This is my header :

  


  01 18 02 03 00

  


  01 - 48KHz
18 - Sampling Rate (18h=24d, 24bit)
02 - 2 channels
03 00 - WAVEFORMATEXTENSIBLE

  


  import socket
import struct
import threading
import os
import sys
import time
import subprocess
import tempfile
import select

class Sender(threading.Thread):
    def __init__(self):
        super().__init__()
        TEMPDIR = tempfile.gettempdir() + "/"
        self.fifoin = TEMPDIR + "ffmpeg-fifo-in"
        self.start()

    def run(self):
        self.fd = open(self.fifoin, "rb")
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        while True:
            try:
                header = bytes([0x01, 0x18, 0x02, 0x03, 0x00])  # 48khz, 24-bit, stereo
                data = self.fd.read(1152)
                sendbuf = header + data
                self.sock.sendto(sendbuf, ("192.168.3.199", 4010))  # Audio sink
            except Exception as e:
                print("Except")
                print(e)

class Receiver(threading.Thread):
    def __init__(self):
        super().__init__()
        TEMPDIR = tempfile.gettempdir() + "/"
        self.fifo1 = TEMPDIR + "ffmpeg-fifo-1"
        self.fifo2 = TEMPDIR + "ffmpeg-fifo-2"
        self.fifoin = TEMPDIR + "ffmpeg-fifo-in"
        self.fifos = [self.fifo1, self.fifo2]
        try:
            try:
                os.remove(self.fifoin)
            except:
                pass
            os.mkfifo(self.fifoin)
        except:
            pass
        self.start()
        sender=Sender()

    def run(self):
        ffmpeg_command=['ffmpeg', '-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', self.fifo1,
                                  '-use_wallclock_as_timestamps', 'true', '-f', 's24le', '-ac', '2', '-ar', '48000', '-i', self.fifo2,
                                  '-filter_complex', '[0]aresample=async=1[a0],[1]aresample=async=1[a1],[a0][a1]amix', "-y", '-f', 's24le', '-ac', '2', '-ar', '48000', self.fifoin]
        print(ffmpeg_command)

        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        sock.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF,4096)
        sock.bind(("", 16401))

        recvbuf = bytearray(1157)
        framecount = [0,0]
        closed = 1
        while True:
            ready = select.select([sock], [], [], .2)
            if ready[0]:
                recvbuf, addr = sock.recvfrom(1157)
                if closed == 1:
                    for fifo in self.fifos:
                        try:
                            try:
                                os.remove(fifo)
                            except:
                                pass
                            os.mkfifo(fifo)
                        except:
                            pass
                    framecount = [0,0]
                    print("data, starting ffmpeg")
                    ffmpeg = subprocess.Popen (ffmpeg_command, shell=False, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
                    fifo1_fd = os.open(self.fifo1, os.O_RDWR)
                    fifo1_file = os.fdopen(fifo1_fd, 'wb', 0)
                    fifo2_fd = os.open(self.fifo2, os.O_RDWR)
                    fifo2_file = os.fdopen(fifo2_fd, 'wb', 0)
                    closed = 0
                    for i in range(0,6):
                        fifo1_file.write(bytes([0]*1157))
                        fifo2_file.write(bytes([0]*1157))

                if addr[0] == "192.168.3.199":
                    fifo1_file.write(recvbuf[5:])
                    framecount[0] = framecount[0] + 1

                if addr[0] == "192.168.3.119":
                    fifo2_file.write(recvbuf[5:])
                    framecount[1] = framecount[1] + 1

                # Keep buffers roughly in sync while playing
                targetframes=max(framecount)
                if targetframes - framecount[0] > 11:
                    while (targetframes - framecount[0]) > 0:
                        fifo1_file.write(bytes([0]*1157))
                        framecount[0] = framecount[0] + 1

                if targetframes - framecount[1] > 11:
                    while (targetframes - framecount[1]) > 0:
                        fifo2_file.write(bytes([0]*1157))
                        framecount[1] = framecount[1] + 1
            else:
                if closed == 0:
                    ffmpeg.kill()
                    print("No data, killing ffmpeg")
                    fifo1_file.close()
                    fifo2_file.close()
                    closed = 1
receiver=Receiver()

while True:
    time.sleep(50000)


  


  Does anybody have any pointers on how I can make this better ?

  


• threads : change the default for threads back to 1

  18 janvier 2012, par Janne Grunau

  threads : change the default for threads back to 1