Newest 'ffmpeg' Questions - Stack Overflow

Les articles publiés sur le site

• How to create a circular video and overlay it on an image background while preserving transparency for MP4 output ?

  12 mars, par Mykyta Manuilenko

  I have the following FFmpeg command that overlays a transparent webm video on a background image and preserves the transparency:

  ffmpeg -t 5 -framerate 25 -loop 1 -i "bg_image.png" -c:v libvpx-vp9 -i "transparent_video.webm" -y -filter_complex "[0:v]format=yuv420p,pad=ceil(iw/2)*2:ceil(ih/2)*2[bg_image];[1:v]setpts=PTS-STARTPTS,setsar=1,pad=ceil(iw/2)*2:ceil(ih/2)*2[el_out_1];[bg_image][el_out_1]overlay=82:250,format=yuv420p,pad=ceil(iw/2)*2:ceil(ih/2)*2[raw_video];[raw_video]format=yuv420p,pad=ceil(iw/2)*2:ceil(ih/2)*2" -vcodec libx264 -f mp4 -t 5 -an -crf 23 -preset medium -copyts "output.mp4"
  

  It works fine - the transparency is preserved. But now I want to make it a circle but keep the transparency, so I tweaked the FFmpeg command a bit:

  ffmpeg -t 5 -framerate 25 -loop 1 -i "bg_image.png" -c:v libvpx-vp9 -i "transparent_video.webm" -y -filter_complex "[0:v]format=yuv420p,pad=ceil(iw/2)*2:ceil(ih/2)*2[bg_image];[1:v]setpts=PTS-STARTPTS,crop=ceil(365.091854095459/2)*2:ceil(365.091854095459/2)*2:180.34759712219238:0,format=rgba,geq=lum='p(X,Y)':a='if(lt(sqrt(pow(X-182.5459270477295,2)+pow(Y-182.5459270477295,2)),182.5459270477295-0.5),255,if(gt(sqrt(pow(X-182.5459270477295,2)+pow(Y-182.5459270477295,2)),182.5459270477295+0.5),0,255*(182.5459270477295+0.5 - sqrt(pow(X-182.5459270477295,2)+pow(Y-182.5459270477295,2)))))',scale=ceil(549.2381964584453/2)*2:ceil(549.2381964584451/2)*2,setsar=1,pad=ceil(iw/2)*2:ceil(ih/2)*2[el_out_1];[bg_image][el_out_1]overlay=84:252,format=yuv420p,pad=ceil(iw/2)*2:ceil(ih/2)*2[raw_video];[raw_video]format=yuv420p,pad=ceil(iw/2)*2:ceil(ih/2)*2" -vcodec libx264 -f mp4 -t 5 -an -crf 23 -preset medium -copyts "output.mp4"
  

  Input a transparent webm video became a circle, but the transparency is lost, instead of transparency I see a black background inside the circle.

  Is there any way to create a circular video and overlay it on an image background while preserving transparency?

  Note that I can't output it to webm or mov with ProRes 4444 which natively support alpha channel, the output should only be mp4.

• How to split audio file into equal-length segments with ffmpeg ?

  11 mars, par GPWR

  I want to split an audio file into several equal-length segments using FFmpeg. I want to specify the general segment duration (no overlap), and I want FFmpeg to render as many segments as it takes to go over the whole audio file (in other words, the number of segments to be rendered is unspecified). Also, since I am not very experienced with FFmpeg (I only use it to make simple file conversions with few arguments), I would like a description of the code you should use to do this, rather than just a piece of code that I won't necessarily understand, if possible. Thank you in advance.

  P.S. Here's the context for why I'm trying to do this: I would like to sample a song into single-bar loops automatically, instead of having to chop them manually using a DAW. All I want to do is align the first beat of the song to the beat grid in my DAW, and then export that audio file and use it to generate one-bar loops in FFmpeg.

  In the future, I will try to do something like a batch command in which one can specify the tempo and key signature, and it will generate the loops using FFmpeg automatically (as long as the loop is aligned to the beat grid, as I've mentioned earlier). 😀

• How to improve web camera streaming latency to v4l2loopback device with ffmpeg ?

  11 mars, par Made by Moses

  I'm trying to stream my iPhone camera to my PC on LAN.

  What I've done:

  1. HTTP server with html page and streaming script:

     I use WebSockets here and maybe WebRTC is better choice but it seems like network latency is good enough

  async function beginCameraStream() {
    const mediaStream = await navigator.mediaDevices.getUserMedia({
      video: { facingMode: "user" },
    });
  
    websocket = new WebSocket(SERVER_URL);
  
    websocket.onopen = () => {
      console.log("WS connected");
  
      const options = { mimeType: "video/mp4", videoBitsPerSecond: 1_000_000 };
      mediaRecorder = new MediaRecorder(mediaStream, options);
  
      mediaRecorder.ondataavailable = async (event) => {
        // to measure latency I prepend timestamp to the actual video bytes chunk
        const timestamp = Date.now();
        const timestampBuffer = new ArrayBuffer(8);
        const dataView = new DataView(timestampBuffer);
        dataView.setBigUint64(0, BigInt(timestamp), true);
        const data = await event.data.bytes();
  
        const result = new Uint8Array(data.byteLength + 8);
        result.set(new Uint8Array(timestampBuffer), 0);
        result.set(data, 8);
  
        websocket.send(result);
      };
  
      mediaRecorder.start(100); // Collect 100ms chunks
    };
  }
  

  2. Server to process video chunks

  import { serve } from "bun";
  import { Readable } from "stream";
  
  const V4L2LOOPBACK_DEVICE = "/dev/video10";
  
  export const setupFFmpeg = (v4l2device) => {
    // prettier-ignore
    return spawn("ffmpeg", [
      '-i', 'pipe:0',           // Read from stdin
      '-pix_fmt', 'yuv420p',    // Pixel format
      '-r', '30',               // Target 30 fps
      '-f', 'v4l2',             // Output format
      v4l2device, // Output to v4l2loopback device
    ]);
  };
  
  export class FfmpegStream extends Readable {
    _read() {
      // This is called when the stream wants more data
      // We push data when we get chunks
    }
  }
  
  function main() {
    const ffmpeg = setupFFmpeg(V4L2LOOPBACK_DEVICE);
    serve({
      port: 8000,
      fetch(req, server) {
        if (server.upgrade(req)) {
          return; // Upgraded to WebSocket
        }
      },
      websocket: {
        open(ws) {
          console.log("Client connected");
          const stream = new FfmpegStream();
          stream.pipe(ffmpeg?.stdin);
  
          ws.data = {
            stream,
            received: 0,
          };
        },
        async message(ws, message) {
          const view = new DataView(message.buffer, 0, 8);
          const ts = Number(view.getBigUint64(0, true));
          ws.data.received += message.byteLength;
          const chunk = new Uint8Array(message.buffer, 8, message.byteLength - 8);
  
          ws.data.stream.push(chunk);
  
          console.log(
            [
              `latency: ${Date.now() - ts} ms`,
              `chunk: ${message.byteLength}`,
              `total: ${ws.data.received}`,
            ].join(" | "),
          );
        },
      },
    });
  }
  
  main();
  

  After I try to open the v4l2loopback device

  cvlc v4l2:///dev/video10
  

  picture is delayed for at least 1.5 sec which is unacceptable for my project.

  Thoughts:

  • Problem doesn't seems to be with network latency

  latency: 140 ms | chunk: 661 Bytes | total: 661 Bytes
  latency: 206 ms | chunk: 16.76 KB | total: 17.41 KB
  latency: 141 ms | chunk: 11.28 KB | total: 28.68 KB
  latency: 141 ms | chunk: 13.05 KB | total: 41.74 KB
  latency: 199 ms | chunk: 11.39 KB | total: 53.13 KB
  latency: 141 ms | chunk: 16.94 KB | total: 70.07 KB
  latency: 139 ms | chunk: 12.67 KB | total: 82.74 KB
  latency: 142 ms | chunk: 13.14 KB | total: 95.88 KB
  

  ~150ms is actually too much for 15KB on LAN but there can some issue with my router

  • As far as I can tell it neither ties to ffmpeg throughput:

  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pipe:0':
    Metadata:
      major_brand     : iso5
      minor_version   : 1
      compatible_brands: isomiso5hlsf
      creation_time   : 2025-03-09T17:16:49.000000Z
    Duration: 00:00:01.38, start:
  0.000000, bitrate: N/A
      Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc), 1280x720, 4012 kb/s, 57.14 fps, 29.83 tbr, 600 tbn, 1200 tbc (default)
      Metadata:
        rotate          : 90
        creation_time   : 2025-03-09T17:16:49.000000Z
        handler_name    : Core Media Video
      Side data:
        displaymatrix: rotation of -90.00 degrees
  
  Stream mapping:
    Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
  
  [swscaler @ 0x55d8d0b83100] deprecated pixel format used, make sure you did set range correctly
  
  Output #0, video4linux2,v4l2, to '/dev/video10':
    Metadata:
      major_brand     : iso5
      minor_version   : 1
      compatible_brands: isomiso5hlsf
      encoder         : Lavf58.45.100
  
  Stream #0:0(und): Video: rawvideo (I420 / 0x30323449), yuv420p, 720x1280, q=2-31, 663552 kb/s, 60 fps, 60 tbn, 60 tbc (default)
      Metadata:
        encoder         : Lavc58.91.100 rawvideo
        creation_time   : 2025-03-09T17:16:49.000000Z
        handler_name    : Core Media Video
      Side data:
        displaymatrix: rotation of -0.00 degrees
  
  frame=   99 fps=0.0 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A dup=50 drop=0 speed=2.77x
  frame=  137 fps=114 q=-0.0 size=N/A time=00:00:02.28 bitrate=N/A dup=69 drop=0 speed=1.89x
  frame=  173 fps= 98 q=-0.0 size=N/A time=00:00:02.88 bitrate=N/A dup=87 drop=0 speed=1.63x
  frame=  210 fps= 86 q=-0.0 size=N/A time=00:00:03.50 bitrate=N/A dup=105 drop=0 speed=1.44x
  frame=  249 fps= 81 q=-0.0 size=N/A time=00:00:04.15 bitrate=N/A dup=125 drop=0 speed=1.36
  frame=  279 fps= 78 q=-0.0 size=N/A time=00:00:04.65 bitrate=N/A dup=139 drop=0 speed=1.31x
  

  • I also tried to write the video stream directly to video.mp4 file and immediately open it with vlc but it only can be successfully opened after ~1.5 sec.

  • I've tried to use OBS v4l2 input source instead of vlc but the latency is the same

  Update №1

  When i try to stream actual .mp4 file to ffmpeg it works almost immediately with 0.2sec delay to spin up the ffmpeg itself:

  cat video.mp4 | ffmpeg -re -i pipe:0 -pix_fmt yuv420p -f v4l2 /dev/video10 & ; sleep 0.2 && cvlc v4l2:///dev/video10
  

  So the problem is apparently with streaming process

• Is it possible to redirect the messages libffmpeg prints to stderr ?

  11 mars, par wallefan

  I'm writing a program that uses libffmpeg to do some transcoding internally, but I find it has a tendency to print a lot of things to the process's stderr. I want to redirect those messages somewhere more intelligent.

  To be clear, I am NOT talking about spawning the ffmpeg executable as a subprocess and redirecting its stdio! That is trivial in any programming language. I'm using ffmpeg as a C library (well, Rust bindings around a C library in my case) and I'm wondering if libffmpeg provides a hook for writing to stderr that I can override.

  If no such hook exists, would my best bet be to fork a child process with stdio redirection and only use libffmpeg from that subprocess? That feels really hacky to me, especially since it would be really nice to not have to send the results libffmpeg spits out across a process boundary.

• av_log() function in ffmpeg to output to a file

  11 mars, par TilakVarisetty

  I am logging time-stamps, and frames using av_log() to the console. Unfortunately, the console does not pipe to a file.

  Is there a way to add function or other options to log to the file directly, instead of the console?