Recherche avancée

Médias (91)

Autres articles (55)

  • Participer à sa traduction

    10 avril 2011

    Vous pouvez nous aider à améliorer les locutions utilisées dans le logiciel ou à traduire celui-ci dans n’importe qu’elle nouvelle langue permettant sa diffusion à de nouvelles communautés linguistiques.
    Pour ce faire, on utilise l’interface de traduction de SPIP où l’ensemble des modules de langue de MediaSPIP sont à disposition. ll vous suffit de vous inscrire sur la liste de discussion des traducteurs pour demander plus d’informations.
    Actuellement MediaSPIP n’est disponible qu’en français et (...)

  • L’utiliser, en parler, le critiquer

    10 avril 2011

    La première attitude à adopter est d’en parler, soit directement avec les personnes impliquées dans son développement, soit autour de vous pour convaincre de nouvelles personnes à l’utiliser.
    Plus la communauté sera nombreuse et plus les évolutions seront rapides ...
    Une liste de discussion est disponible pour tout échange entre utilisateurs.

  • HTML5 audio and video support

    13 avril 2011, par

    MediaSPIP uses HTML5 video and audio tags to play multimedia files, taking advantage of the latest W3C innovations supported by modern browsers.
    The MediaSPIP player used has been created specifically for MediaSPIP and can be easily adapted to fit in with a specific theme.
    For older browsers the Flowplayer flash fallback is used.
    MediaSPIP allows for media playback on major mobile platforms with the above (...)

Sur d’autres sites (10088)

  • How to improve web camera streaming latency to v4l2loopback device with ffmpeg ?

    11 mars, par Made by Moses

    I'm trying to stream my iPhone camera to my PC on LAN.

    


    What I've done :

    


      


    1. HTTP server with html page and streaming script :

      


      I use WebSockets here and maybe WebRTC is better choice but it seems like network latency is good enough

      


      2. 


    


    async function beginCameraStream() {
  const mediaStream = await navigator.mediaDevices.getUserMedia({
    video: { facingMode: "user" },
  });

  websocket = new WebSocket(SERVER_URL);

  websocket.onopen = () => {
    console.log("WS connected");

    const options = { mimeType: "video/mp4", videoBitsPerSecond: 1_000_000 };
    mediaRecorder = new MediaRecorder(mediaStream, options);

    mediaRecorder.ondataavailable = async (event) => {
      // to measure latency I prepend timestamp to the actual video bytes chunk
      const timestamp = Date.now();
      const timestampBuffer = new ArrayBuffer(8);
      const dataView = new DataView(timestampBuffer);
      dataView.setBigUint64(0, BigInt(timestamp), true);
      const data = await event.data.bytes();

      const result = new Uint8Array(data.byteLength + 8);
      result.set(new Uint8Array(timestampBuffer), 0);
      result.set(data, 8);

      websocket.send(result);
    };

    mediaRecorder.start(100); // Collect 100ms chunks
  };
}


    


      


    1. Server to process video chunks

      


      2. 


    


    import { serve } from "bun";
import { Readable } from "stream";

const V4L2LOOPBACK_DEVICE = "/dev/video10";

export const setupFFmpeg = (v4l2device) => {
  // prettier-ignore
  return spawn("ffmpeg", [
    '-i', 'pipe:0',           // Read from stdin
    '-pix_fmt', 'yuv420p',    // Pixel format
    '-r', '30',               // Target 30 fps
    '-f', 'v4l2',             // Output format
    v4l2device, // Output to v4l2loopback device
  ]);
};

export class FfmpegStream extends Readable {
  _read() {
    // This is called when the stream wants more data
    // We push data when we get chunks
  }
}

function main() {
  const ffmpeg = setupFFmpeg(V4L2LOOPBACK_DEVICE);
  serve({
    port: 8000,
    fetch(req, server) {
      if (server.upgrade(req)) {
        return; // Upgraded to WebSocket
      }
    },
    websocket: {
      open(ws) {
        console.log("Client connected");
        const stream = new FfmpegStream();
        stream.pipe(ffmpeg?.stdin);

        ws.data = {
          stream,
          received: 0,
        };
      },
      async message(ws, message) {
        const view = new DataView(message.buffer, 0, 8);
        const ts = Number(view.getBigUint64(0, true));
        ws.data.received += message.byteLength;
        const chunk = new Uint8Array(message.buffer, 8, message.byteLength - 8);

        ws.data.stream.push(chunk);

        console.log(
          [
            `latency: ${Date.now() - ts} ms`,
            `chunk: ${message.byteLength}`,
            `total: ${ws.data.received}`,
          ].join(" | "),
        );
      },
    },
  });
}

main();


    


    After I try to open the v4l2loopback device

    


    cvlc v4l2:///dev/video10


    


    picture is delayed for at least 1.5 sec which is unacceptable for my project.

    


    Thoughts :

    


      

    • Problem doesn't seems to be with network latency

      • 


    


    latency: 140 ms | chunk: 661 Bytes | total: 661 Bytes
latency: 206 ms | chunk: 16.76 KB | total: 17.41 KB
latency: 141 ms | chunk: 11.28 KB | total: 28.68 KB
latency: 141 ms | chunk: 13.05 KB | total: 41.74 KB
latency: 199 ms | chunk: 11.39 KB | total: 53.13 KB
latency: 141 ms | chunk: 16.94 KB | total: 70.07 KB
latency: 139 ms | chunk: 12.67 KB | total: 82.74 KB
latency: 142 ms | chunk: 13.14 KB | total: 95.88 KB


    


     150ms is actually too much for 15KB on LAN but there can some issue with my router

    


      

    • As far as I can tell it neither ties to ffmpeg throughput :

      • 


    


    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pipe:0':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    creation_time   : 2025-03-09T17:16:49.000000Z
  Duration: 00:00:01.38, start:
0.000000, bitrate: N/A
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc), 1280x720, 4012 kb/s, 57.14 fps, 29.83 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2025-03-09T17:16:49.000000Z
      handler_name    : Core Media Video
    Side data:
      displaymatrix: rotation of -90.00 degrees

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))

[swscaler @ 0x55d8d0b83100] deprecated pixel format used, make sure you did set range correctly

Output #0, video4linux2,v4l2, to '/dev/video10':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    encoder         : Lavf58.45.100

Stream #0:0(und): Video: rawvideo (I420 / 0x30323449), yuv420p, 720x1280, q=2-31, 663552 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 rawvideo
      creation_time   : 2025-03-09T17:16:49.000000Z
      handler_name    : Core Media Video
    Side data:
      displaymatrix: rotation of -0.00 degrees

frame=   99 fps=0.0 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A dup=50 drop=0 speed=2.77x
frame=  137 fps=114 q=-0.0 size=N/A time=00:00:02.28 bitrate=N/A dup=69 drop=0 speed=1.89x
frame=  173 fps= 98 q=-0.0 size=N/A time=00:00:02.88 bitrate=N/A dup=87 drop=0 speed=1.63x
frame=  210 fps= 86 q=-0.0 size=N/A time=00:00:03.50 bitrate=N/A dup=105 drop=0 speed=1.44x
frame=  249 fps= 81 q=-0.0 size=N/A time=00:00:04.15 bitrate=N/A dup=125 drop=0 speed=1.36
frame=  279 fps= 78 q=-0.0 size=N/A time=00:00:04.65 bitrate=N/A dup=139 drop=0 speed=1.31x


    


      


    • I also tried to write the video stream directly to video.mp4 file and immediately open it with vlc but it only can be successfully opened after 1.5 sec.

      


      • 


    • I've tried to use OBS v4l2 input source instead of vlc but the latency is the same

      


      • 


    


    Update №1

    


    When i try to stream actual .mp4 file to ffmpeg it works almost immediately with 0.2sec delay to spin up the ffmpeg itself :

    


    cat video.mp4 | ffmpeg -re -i pipe:0 -pix_fmt yuv420p -f v4l2 /dev/video10 & ; sleep 0.2 && cvlc v4l2:///dev/video10


    


    So the problem is apparently with streaming process

    


  • How does FFmpeg determine the “attached pic” and “timed thumbnails” dispositions of an MP4 track ?

    6 mars, par obskyr

    The Issue

    


    FFmpeg has a concept of “dispositions” – a property that describes the purpose of a stream in a media file. For example, here are the streams in a file I have lying around, with the dispositions emphasized :

    


      Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,&#xA;fltp, 251 kb/s <strong><em>(default)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Audio&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:1[0x2](und): Video: mjpeg (Baseline) (jpeg / 0x6765706A),&#xA;yuvj420p(pc, bt470bg/unknown/unknown), 1024x1024, 0 kb/s, 0.0006 fps, 3.08 tbr,&#xA;600 tbn <strong><em>(default) (attached pic) (timed thumbnails)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Video&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:2[0x3](und): Data: bin_data (text / 0x74786574)&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Text&#xA;&#xA;  Stream #0:3[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/&#xA;unknown), 1024x1024 [SAR 144:144 DAR 1:1], 90k tbr, 90k tbn <strong><em>(attached pic)</em></strong>

    &#xA;

    However, if I make any modification to this file’s chapter markers using the C++ library MP4v2 (even just re-saving the existing ones : auto f = MP4Modify("test.m4a"); MP4Chapter_t* chapterList; uint32_t chapterCount; MP4GetChapters(f, &amp;chapterList, &amp;chapterCount); MP4SetChapters(f, chapterList, chapterCount); MP4Close(f);), some of these dispositions are removed :

    &#xA;

      Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,&#xA;fltp, 251 kb/s <strong><em>(default)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Audio&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:1[0x2](und): Video: mjpeg (Baseline) (jpeg / 0x6765706A),&#xA;yuvj420p(pc, bt470bg/unknown/unknown), 1024x1024, 0 kb/s, 0.0006 fps, 3.08 tbr,&#xA;600 tbn <strong><em>(default)</em></strong> <kbd>← “attached pic” and “timed thumbnails” removed!</kbd>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Video&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:2[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/&#xA;unknown), 1024x1024 [SAR 144:144 DAR 1:1], 90k tbr, 90k tbn <strong><em>(attached pic)</em></strong>&#xA;&#xA;  Stream #0:3[0x4](und): Data: bin_data (text / 0x74786574)&#xA;  <kbd>This stream was moved to the end, but that’s intended behavior. It contains chapter titles, and we just edited the chapters.</kbd>&#xA;      Metadata:&#xA;        creation_time   : 2025-03-05T09:56:31.000000Z

    &#xA;

    It also renders the file unplayable in MPC-HC (but not in VLC !), which is apparently a bug in MP4v2. I’m currently investigating that bug to report and potentially fix it, but that’s a separate issue – in my journey there, I’m wracking my brain trying to understand what it is that MP4v2 changes to make FFmpeg stop reporting the “attached pic” and “timed thumbnails” dispositions. I’ve explored the before-and-afters in MP4 Box, and I can’t for the life of me find which atom it is that differs in a relevant way.

    &#xA;

    (I’d love to share the files, but unfortunately the contents are under copyright – if anyone knows of a way to remove the audio from an MP4 file without changing anything else, let me know and I’ll upload dummied-out versions. Without them, I can’t really ask about the issue directly. I can at least show you the files’ respective atom trees, but I’m not sure how relevant that is.)

    &#xA;

    The Question

    &#xA;

    I thought I’d read FFmpeg’s source code to find out how it determines dispositions for MP4 streams, but of course, FFmpeg is very complex. Could someone who’s more familiar with C and/or FFmpeg’s codebase help me sleuth out how FFmpeg determines dispositions for MP4 files (in particular, “attached pic” and “timed thumbnails”) ?

    &#xA;

    Some Thoughts…

    &#xA;

      &#xA;
    • I figure searching for “attached_pic” might be a good start ?

      • &#xA;

    • Could the MP4 muxer movenc.c be helpful ?

      • &#xA;

    • I’d imagine what we’d really like to look at is the MP4 demuxing process, as it’s during demuxing that FFmpeg determines dispositions from the data in the file. After poring over the code for hours, however, I’ve been utterly unable to find where that happens.

      • &#xA;

    &#xA;

  • How does FFmpeg determine the dispositions of an MP4 track ?

    5 mars, par obskyr

    The Issue

    &#xA;

    FFmpeg has a concept of “dispositions” – a property that describes the purpose of a stream in a media file. For example, here are the streams in a file I have lying around, with the dispositions emphasized :

    &#xA;

      Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,&#xA;fltp, 251 kb/s <strong><em>(default)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Audio&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:1[0x2](und): Video: mjpeg (Baseline) (jpeg / 0x6765706A),&#xA;yuvj420p(pc, bt470bg/unknown/unknown), 1024x1024, 0 kb/s, 0.0006 fps, 3.08 tbr,&#xA;600 tbn <strong><em>(default) (attached pic) (timed thumbnails)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Video&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:2[0x3](und): Data: bin_data (text / 0x74786574)&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Text&#xA;&#xA;  Stream #0:3[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/&#xA;unknown), 1024x1024 [SAR 144:144 DAR 1:1], 90k tbr, 90k tbn <strong><em>(attached pic)</em></strong>

    &#xA;

    However, if I make any modification to this file’s chapter markers using the C++ library MP4v2 (even just re-saving the existing ones : auto f = MP4Modify("test.m4a"); MP4Chapter_t* chapterList; uint32_t chapterCount; MP4GetChapters(f, &amp;chapterList, &amp;chapterCount); MP4SetChapters(f, chapterList, chapterCount); MP4Close(f);), some of these dispositions are removed :

    &#xA;

      Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo,&#xA;fltp, 251 kb/s <strong><em>(default)</em></strong>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Audio&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:1[0x2](und): Video: mjpeg (Baseline) (jpeg / 0x6765706A),&#xA;yuvj420p(pc, bt470bg/unknown/unknown), 1024x1024, 0 kb/s, 0.0006 fps, 3.08 tbr,&#xA;600 tbn <strong><em>(default)</em></strong> <kbd>← “attached pic” and “timed thumbnails” removed!</kbd>&#xA;      Metadata:&#xA;        creation_time   : 2021-11-10T20:14:06.000000Z&#xA;        handler_name    : Core Media Video&#xA;        vendor_id       : [0][0][0][0]&#xA;&#xA;  Stream #0:2[0x0]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/&#xA;unknown), 1024x1024 [SAR 144:144 DAR 1:1], 90k tbr, 90k tbn <strong><em>(attached pic)</em></strong>&#xA;&#xA;  Stream #0:3[0x4](und): Data: bin_data (text / 0x74786574)&#xA;  <kbd>This stream was moved to the end, but that’s intended behavior. It contains chapter titles, and we just edited the chapters.</kbd>&#xA;      Metadata:&#xA;        creation_time   : 2025-03-05T09:56:31.000000Z

    &#xA;

    It also renders the file unplayable in MPC-HC (but not in VLC !), which is apparently a bug in MP4v2. I’m currently investigating that bug to report and potentially fix it, but that’s a separate issue – in my journey there, I’m wracking my brain trying to understand what it is that MP4v2 changes to make FFmpeg stop reporting the “attached pic” and “timed thumbnails” dispositions. I’ve explored the before-and-afters in MP4 Box, and I can’t for the life of me find which atom it is that differs in a relevant way.

    &#xA;

    (I’d love to share the files, but unfortunately the contents are under copyright – if anyone knows of a way to remove the audio from an MP4 file without changing anything else, let me know and I’ll upload dummied-out versions. Without them, I can’t really ask about the issue directly. I can at least show you the files’ respective atom trees, but I’m not sure how relevant that is.)

    &#xA;

    The Question

    &#xA;

    I thought I’d read FFmpeg’s source code to find out how it determines dispositions for MP4 streams, but of course, FFmpeg is very complex. Could someone who’s more familiar with C and/or FFmpeg’s codebase help me sleuth out how FFmpeg determines dispositions for MP4 files (in particular, “attached pic” and “timed thumbnails”) ?

    &#xA;

    Some Thoughts…

    &#xA;

      &#xA;
    • I figure searching for “attached_pic” might be a good start ?

      • &#xA;

    • Could the MP4 muxer movenc.c be helpful ?

      • &#xA;

    • I’d imagine what we’d really like to look at is the MP4 demuxing process, as it’s during demuxing that FFmpeg determines dispositions from the data in the file. After poring over the code for hours, however, I’ve been utterly unable to find where that happens.

      • &#xA;

    &#xA;

Navigation