Recherche avancée

Sur d’autres sites (11027)

• ffmpeg lags when streaming video+audio from RPi Zero W with Logitech C920

  7 janvier 2021, par Ema

  I've been trying to setup a baby monitor with a Raspberry Pi Zero and a Logitech C920 webcam. I does work with VLC (cvlc) but it lags too much and gets worse over time.

  


  So I am playing around with ffmpeg and I am getting some better results. This is what I've done so far.

  


  First I set the webcam to output h264 1080p natively (the Pi Zero W can't afford to do any transcoding).

  


  v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=1


  


  Now, if I stream audio only with

  


  ffmpeg \
-f alsa \
-i hw:1,0 \
-vn \
-flags +global_header \
-acodec aac \
-ac 1 \
-ar 16000 \
-ab 16k \
-f rtp rtp://192.168.0.10:5002 > audio.sdp


  


  it works great and the lag is about 1 second (definitely acceptable).

  


  If I stream video only with

  


  ffmpeg \
-f v4l2 \
-vcodec h264 \
-i /dev/video0 \
-an \
-vcodec copy \
-pix_fmt yuv420p \
-r 30 \
-b:v 512k \
-flags +global_header \
-f rtp rtp://192.168.0.10:5000 > video.sdp


  


  same result, very little lag (for some reason the first -vcodec is necessary to force the webcam to output h264).

  


  However, when I stream both with

  


  ffmpeg \
-f v4l2 \
-vcodec h264 \
-i /dev/video0 \
-f alsa \
-i hw:1,0 \
-an \
-preset ultrafast \
-tune zerolatency \
-vcodec copy \
-pix_fmt yuv420p \
-r 30 \
-b:v 512k \
-flags +global_header \
-f rtp rtp://192.168.0.10:5000 \
-vn \
-flags +global_header \
-acodec aac \
-ac 1 \
-ar 16000 \
-ab 16k \
-f rtp rtp://192.168.0.10:5002 > both.sdp


  


  the lag ramps up to 10 seconds and audio and video are out of sync. Does anybody know why ?

  


  I've tried UDP and TCP instead of RTP but then the lag is always high, even with audio/video only.

  


  Any suggestion is much appreciated.

  


  P.S. On the client side (MacOS) I'm receiving with

  


  ffplay -protocol_whitelist file,rtp,udp -i file.sdp


  


• Normalizing audio in ffmpeg - how ?

  11 novembre 2020, par Betty Crokker

  I'm creating one of those "Brady Bunch" videos for a choir using a C# application I'm writing that uses ffmpeg for all the heavy lifting, and for the most part it's working great but I'm having trouble getting the audio levels just right.

  


  What I'm doing right now, is first "normalizing" the audio from the individual singers like this :

  


  

  • Extract audio into a WAV file using ffmpeg

  


  • Load the WAV file into my application using NAudio

  


  • Find the maximum 16-bit value

  


  • When I create the merged video, specify a volume for this stream that boosts the maximum value to 32767

  


  


  So, for example, if I have 3 streams : stream A's maximum audio is 32767 already, stream B's maximum audio is 32000, and stream C's maximum audio is 16000, then when I merge these videos I will specify

  


  [0:a]volume=1.0,aresample=async=1:first_pts=0[aud0]
[1:a]volume=1.02,aresample=async=1:first_pts=0[aud1]
[2:a]volume=2.05,aresample=async=1:first_pts=0[aud2]
[aud0][aud1][aud2]amix=inputs=3[a]


  


  (I have an additional "volume tweak" that lets me adjust the volume level of individual singers as necessary, but we can ignore that for this question)

  


  I am reading the ffmpeg wiki on Audio Volume Manipulation, and I will implement that next, but I don't know what to do with the output it generates. It looks like I'm going to get mean and max volume levels in dB and while I understand decibels in a "yeah, I learned about those in college 30 years ago" kind of way, I don't know how to use those values to normalize the audio of my input videos.

  


  The problem is, in the ffmpeg output video, the audio level is quite low. If I do the same process of extracting the audio and looking at the WAV file in the merged video that ffmpeg generated, the maximum value is only 4904.

  


  How do I implement an algorithm that automatically sets the output volume to a "reasonable" level ? I realize I can simply add a manual volume filter and have the human set the level, but that's going to be a lot of back & forth of generating the merged video, listening to it, adjusting the level, merging again, etc. I want a way where my application figures out an appropriate output volume (possibly with human adjustment allowed).

  


  EDIT

  


  Asking ffmpeg to determine the mean and max volume of each clip does provide mean and max volume in dB, and I can then use those values to scale each input clip :

  


  [0:a]volume=3.40dB,aresample=async=1:first_pts=0[aud0]
[1:a]volume=3.90dB,aresample=async=1:first_pts=0[aud1]
[2:a]volume=4.40dB,aresample=async=1:first_pts=0[aud2]
[3:a]volume=-0.00dB,aresample=async=1:first_pts=0[aud3]


  


  But my final video is still strangely quiet. For now, I've added a manually-entered volume factor that gets applied at the very end :

  


  [aud0][aud1][aud2]amix=inputs=3[a]
[a]volume=volume=3.00[b]


  


  So my question is, in effect, how do I determine algorithmically what this final volume factor needs to be ?

  


  MORE EDIT

  


  There's something deeper going on here, I just set the volume filter to 100 and the output is only slightly louder. Here are my filters, and the relevant portions of the command line :

  


  color=size=1920x1080:c=0x0000FF [base];
[0:v] scale=576x324 [clip0];
[0:a]volume=1.48,aresample=async=1:first_pts=0[aud0];
[1:v] crop=808:1022:202:276,scale=384x486 [clip1];
[1:a]volume=1.57,aresample=async=1:first_pts=0[aud1];
[2:v] crop=1160:1010:428:70,scale=558x486 [clip2];
[2:a]volume=1.66,aresample=async=1:first_pts=0[aud2];
[3:v] crop=1326:1080:180:0,scale=576x469 [clip3];
[3:a]volume=1.70,aresample=async=1:first_pts=0[aud3];
[4:a]volume=0.20,aresample=async=1:first_pts=0[aud4];
[5:a]volume=0.73,aresample=async=1:first_pts=0[aud5];
[6:v] crop=1326:1080:276:0,scale=576x469 [clip4];
[6:a]volume=1.51,aresample=async=1:first_pts=0[aud6];
[base][clip0] overlay=shortest=1:x=32:y=158 [tmp0];
[tmp0][clip1] overlay=shortest=1:x=768:y=27 [tmp1];
[tmp1][clip2] overlay=shortest=1:x=1321:y=27 [tmp2];
[tmp2][clip3] overlay=shortest=1:x=32:y=625 [tmp3];
[tmp3][clip4] overlay=shortest=1:x=672:y=625 [tmp4];
[aud0][aud1][aud2][aud3][aud4][aud5][aud6]amix=inputs=7[a];
[a]adelay=delays=200:all=1[b];
[b]volume=volume=100.00[c];
[c]asplit[a1][a2];

ffmpeg -y ....
   -map "[tmp4]" -map "[a1]" -c:v libx264 "D:\voutput.mp4" 
   -map "[a2]" "D:\aoutput.mp3""


  


  When I do this, the audio I want is louder (loud enough to clip and get distorted), but definitely not 100x louder.

  


• c# Process data piping : 'The pipe has ended'

  9 août 2024, par user2224583

  There are two process I am handling. Ffmpeg and fpcalc.

  


  I have completed my appplication by encoding a wav file with ffmpeg, and reading it for purposes of chromaprinting with fpcalc.

  


  However, I think the next step is to skip the saving of a wav file, and pipe the data directly into the fpcalc process.

  


  I've been trying to understand how to pipe the stream data from ffpmeg stdout into the fpcalc process stdin.

  


  Some of the code I've been testing and reading (below) has come from other answers, with regards to piping data out of a Process object in c#, specifically with ffmpeg, found here on Stack

  


  private static void ExtractPCMAudio(string input, int duration)
    {
        Process proc = new Process();

        proc.StartInfo.FileName = @"ffmpeg.exe";
        proc.StartInfo.Arguments = $"-t 00:3:00 -i \"{input}\" -f wav -ac 1 -acodec pcm_s16le -ar 16000 -"; //"-" Dash pipes the stream
        proc.StartInfo.UseShellExecute = false;
        proc.StartInfo.RedirectStandardInput = true;
        proc.StartInfo.RedirectStandardOutput = true;

        proc.Start();

        FileStream baseStream = proc.StandardOutput.BaseStream as FileStream;

        byte[] audioBytes = null;
        int lastRead = 0;

        using (MemoryStream ms = new MemoryStream())
        {            
            byte[] buffer = new byte[4096];
            do
            {
                lastRead = baseStream.Read(buffer, 0, buffer.Length);
                ms.Write(buffer, 0, lastRead);
            } while (lastRead > 0);

            audioBytes = ms.ToArray();
        }
        
        proc.StandardInput.BaseStream.Close();
        
        baseStream.Close();
        FingerPrintAudio(audioBytes);
        Console.WriteLine("Done!");
        
    }


private static void FingerPrintAudio(byte[] audioBytes)
    {
        
        var fpcalc = "fpcalc.exe";
        var procStartInfo =
            new ProcessStartInfo(fpcalc, $" -raw -length 600 -json") 
            {
                RedirectStandardOutput = true,
                RedirectStandardInput = true,
                RedirectStandardError  = true,
                UseShellExecute        = false,
                CreateNoWindow         = true,
            };

        var process = new Process {StartInfo = procStartInfo};
        process.Start();            
        
        process.StandardInput.BaseStream.Write(audioBytes, 0, audioBytes.Length); //<--This is where the piping error happens.

        string processOutput = null;
        using (var sr = new StreamWriter("Test.json"))
        {
            while ((processOutput = process.StandardOutput.ReadLine()) != null)
            {
                sr.WriteLine(processOutput);
                Console.WriteLine(processOutput);
            }
            sr.Close();
        }
        
    }


  


  Throughout many variations of the code, posted above, I seem to encounter an error : 'the pipe has ended'.
I think it is a buffering error, where the process handling fpcalc is expecting more byte data, but is cut short ?

  


  Perhaps there is a better way to send stream data from one process and use it in a second process ?