Recherche avancée

Sur d’autres sites (3121)

• How to improve web camera streaming latency to v4l2loopback device with ffmpeg ?

  11 mars, par Made by Moses

  I'm trying to stream my iPhone camera to my PC on LAN.

  


  What I've done :

  


  


  1. HTTP server with html page and streaming script :

     


     I use WebSockets here and maybe WebRTC is better choice but it seems like network latency is good enough

     


  


  


  async function beginCameraStream() {
  const mediaStream = await navigator.mediaDevices.getUserMedia({
    video: { facingMode: "user" },
  });

  websocket = new WebSocket(SERVER_URL);

  websocket.onopen = () => {
    console.log("WS connected");

    const options = { mimeType: "video/mp4", videoBitsPerSecond: 1_000_000 };
    mediaRecorder = new MediaRecorder(mediaStream, options);

    mediaRecorder.ondataavailable = async (event) => {
      // to measure latency I prepend timestamp to the actual video bytes chunk
      const timestamp = Date.now();
      const timestampBuffer = new ArrayBuffer(8);
      const dataView = new DataView(timestampBuffer);
      dataView.setBigUint64(0, BigInt(timestamp), true);
      const data = await event.data.bytes();

      const result = new Uint8Array(data.byteLength + 8);
      result.set(new Uint8Array(timestampBuffer), 0);
      result.set(data, 8);

      websocket.send(result);
    };

    mediaRecorder.start(100); // Collect 100ms chunks
  };
}


  


  


  2. Server to process video chunks

     


  


  


  import { serve } from "bun";
import { Readable } from "stream";

const V4L2LOOPBACK_DEVICE = "/dev/video10";

export const setupFFmpeg = (v4l2device) => {
  // prettier-ignore
  return spawn("ffmpeg", [
    '-i', 'pipe:0',           // Read from stdin
    '-pix_fmt', 'yuv420p',    // Pixel format
    '-r', '30',               // Target 30 fps
    '-f', 'v4l2',             // Output format
    v4l2device, // Output to v4l2loopback device
  ]);
};

export class FfmpegStream extends Readable {
  _read() {
    // This is called when the stream wants more data
    // We push data when we get chunks
  }
}

function main() {
  const ffmpeg = setupFFmpeg(V4L2LOOPBACK_DEVICE);
  serve({
    port: 8000,
    fetch(req, server) {
      if (server.upgrade(req)) {
        return; // Upgraded to WebSocket
      }
    },
    websocket: {
      open(ws) {
        console.log("Client connected");
        const stream = new FfmpegStream();
        stream.pipe(ffmpeg?.stdin);

        ws.data = {
          stream,
          received: 0,
        };
      },
      async message(ws, message) {
        const view = new DataView(message.buffer, 0, 8);
        const ts = Number(view.getBigUint64(0, true));
        ws.data.received += message.byteLength;
        const chunk = new Uint8Array(message.buffer, 8, message.byteLength - 8);

        ws.data.stream.push(chunk);

        console.log(
          [
            `latency: ${Date.now() - ts} ms`,
            `chunk: ${message.byteLength}`,
            `total: ${ws.data.received}`,
          ].join(" | "),
        );
      },
    },
  });
}

main();


  


  After I try to open the v4l2loopback device

  


  cvlc v4l2:///dev/video10


  


  picture is delayed for at least 1.5 sec which is unacceptable for my project.

  


  Thoughts :

  


  

  • Problem doesn't seems to be with network latency

  


  


  latency: 140 ms | chunk: 661 Bytes | total: 661 Bytes
latency: 206 ms | chunk: 16.76 KB | total: 17.41 KB
latency: 141 ms | chunk: 11.28 KB | total: 28.68 KB
latency: 141 ms | chunk: 13.05 KB | total: 41.74 KB
latency: 199 ms | chunk: 11.39 KB | total: 53.13 KB
latency: 141 ms | chunk: 16.94 KB | total: 70.07 KB
latency: 139 ms | chunk: 12.67 KB | total: 82.74 KB
latency: 142 ms | chunk: 13.14 KB | total: 95.88 KB


  


   150ms is actually too much for 15KB on LAN but there can some issue with my router

  


  

  • As far as I can tell it neither ties to ffmpeg throughput :

  


  


  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pipe:0':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    creation_time   : 2025-03-09T17:16:49.000000Z
  Duration: 00:00:01.38, start:
0.000000, bitrate: N/A
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc), 1280x720, 4012 kb/s, 57.14 fps, 29.83 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2025-03-09T17:16:49.000000Z
      handler_name    : Core Media Video
    Side data:
      displaymatrix: rotation of -90.00 degrees

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))

[swscaler @ 0x55d8d0b83100] deprecated pixel format used, make sure you did set range correctly

Output #0, video4linux2,v4l2, to '/dev/video10':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    encoder         : Lavf58.45.100

Stream #0:0(und): Video: rawvideo (I420 / 0x30323449), yuv420p, 720x1280, q=2-31, 663552 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 rawvideo
      creation_time   : 2025-03-09T17:16:49.000000Z
      handler_name    : Core Media Video
    Side data:
      displaymatrix: rotation of -0.00 degrees

frame=   99 fps=0.0 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A dup=50 drop=0 speed=2.77x
frame=  137 fps=114 q=-0.0 size=N/A time=00:00:02.28 bitrate=N/A dup=69 drop=0 speed=1.89x
frame=  173 fps= 98 q=-0.0 size=N/A time=00:00:02.88 bitrate=N/A dup=87 drop=0 speed=1.63x
frame=  210 fps= 86 q=-0.0 size=N/A time=00:00:03.50 bitrate=N/A dup=105 drop=0 speed=1.44x
frame=  249 fps= 81 q=-0.0 size=N/A time=00:00:04.15 bitrate=N/A dup=125 drop=0 speed=1.36
frame=  279 fps= 78 q=-0.0 size=N/A time=00:00:04.65 bitrate=N/A dup=139 drop=0 speed=1.31x


  


  


  • I also tried to write the video stream directly to video.mp4 file and immediately open it with vlc but it only can be successfully opened after 1.5 sec.

    


  


  • I've tried to use OBS v4l2 input source instead of vlc but the latency is the same

    


  


  


  Update №1

  


  When i try to stream actual .mp4 file to ffmpeg it works almost immediately with 0.2sec delay to spin up the ffmpeg itself :

  


  cat video.mp4 | ffmpeg -re -i pipe:0 -pix_fmt yuv420p -f v4l2 /dev/video10 & ; sleep 0.2 && cvlc v4l2:///dev/video10


  


  So the problem is apparently with streaming process

  


• How can I crop and encode a video using flutter natively, now that flutter_ffmpeg is discontinued with no alternatives ? [closed]

  13 mars, par Rageh Azzazy

  As of January 6, 2025, FFmpegKit is officially retired (Taner's article).

  


  This also affects the flutter_ffmpeg and ffmpeg_kit_flutter packages.

  


  Most packages for executing video editing commands were built depending on them and so won't work after April 1, 2025 as mentioned in the package documentation and Taner's article. Some packages that depend on flutter_ffmpeg or ffmpeg_kit_flutter are :

  


  

  • video_trimmer

  


  • zero_video_trimmer

  


  • flutter_video_trimmer

  


  • video_trim

  


  • bemeli_compress

  


  • video_trimmer_pro

  


  • ... others

  


  


  Editing video using video_editor or video_editor_2 or video_editor_pits has become a problem.

  


  I believe downloading the binaries and doing the whole thing locally is not just tedious but illegal as well.

  


  I broke down exactly what I need to edit videos in my flutter app and I found those package but have not yet tested them.

  


  


  • ✅ Trimming : flutter_native_video_trimmer

    


  


  • ✅ Compression : video_compress or video_compress_plus

    


  


  • ✅ Muting : video_compress handles this

    


  


  • ❌ Encoding : Not sure, I just need a simple MP4 video files

    


  


  • ❌ Cropping : I can't find a solution for video cropping

    


  


  


  I don't need to rotate, reverse or do any fancy stuff to the videos, just those five functions.

  


  Now the only way forward is to find a simple video cropping function and an encoder that work on flutter IOS & Android

  


  My question is not looking for external library recommendations but :

  


  Do you have any ideas how to crop a video in flutter natively ?

  


• How to dump ALL metadata from a media file, including cover image title ? [closed]

  9 avril, par Unideal

  I have an MP3 song :

  


  # ffprobe -hide_banner -i filename.mp3
Input #0, mp3, from 'filename.mp3':
  Metadata:
    composer        : Music Author
    title           : Song Name
    artist          : Singer
    encoder         : Lavf61.7.100
    genre           : Rock
    date            : 2025
  Duration: 00:03:14.04, start: 0.023021, bitrate: 208 kb/s
  Stream #0:0: Audio: mp3 (mp3float), 48000 Hz, stereo, fltp, 192 kb/s
      Metadata:
        encoder         : Lavc61.19
  Stream #0:1: Video: png, rgb24(pc, gbr/unknown/unknown), 600x600 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn (attached pic)
      Metadata:
        title           : Cover
        comment         : Cover (front)


  


  The task is to save its metadata to a text file and restore from that file later. Both goals should be accomplished with ffmpeg.

  


  The simpliest method is to run :

  


  # ffmpeg -i filename.mp3 -f ffmetadata metadata.txt


  


  After that, metadata.txt contains :

  


  ;FFMETADATA1
composer=Music Author
title=Song Name
artist=Singer
date=2025
genre=Rock
encoder=Lavf61.7.100


  


  I got global metadata only, but stream-specific info (cover image title and comment in my case) are missing.

  


  Google suggested a more complex form of the command above to extract all metadata fields without any exclusions :

  


  # ffmpeg -y -i filename.mp3 -c copy -map_metadata 0 -map_metadata:s:v 0:s:v -map_metadata:s:a 0:s:a -f ffmetadata metadata.txt


  


  But the output is exactly the same :

  


  ;FFMETADATA1
composer=Music Author
title=Song Name
artist=Singer
date=2025
genre=Rock
encoder=Lavf61.7.100


  


  Again, no info about the attached image.

  


  Please explain what am I doing wrong.