Recherche avancée

Sur d’autres sites (13266)

• Transcode webcam blob to RTMP using ffmpeg.wasm

  29 novembre 2023, par hassan moradnezhad

  I'm trying transcode webcam blob data to a rtmp server from browser by using ffmpeg.wasm .
  
first, i create a MediaStream.

  


          const stream = await navigator.mediaDevices.getUserMedia({
            video: true,
        });


  


  then, i create a MediaRecorder.

  


          const recorder = new MediaRecorder(stream, {mimeType: "video/webm; codecs:vp9"});
        recorder.ondataavailable = handleDataAvailable;
        recorder.start(0)


  


  when data is available, i call a function called handleDataAvailable.
  
here is the function.

  


      const handleDataAvailable = (event: BlobEvent) => {
        console.log("data-available");
        if (event.data.size > 0) {
            recordedChunksRef.current.push(event.data);
            transcode(event.data)
        }
    };


  


  in above code, i use another function which called transcode it's goal is going to send data to rtmp server using use ffmpeg.wasm.
  
here it is.

  


  const transcode = async (inputVideo: Blob | undefined) => {
        const ffmpeg = ffmpegRef.current;
        const fetchFileOutput = await fetchFile(inputVideo)
        ffmpeg?.writeFile('input.webm', fetchFileOutput)

        const data = await ffmpeg?.readFile('input.webm');
        if (videoRef.current) {
            videoRef.current.src =
                URL.createObjectURL(new Blob([(data as any)?.buffer], {type: 'video/webm'}));
        }

        // execute by node-media-server config 1
        await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c', 'copy', '-f', 'flv', "rtmp://localhost:1935/live/ttt"])

        // execute by node-media-server config 2
        // await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c:v', 'libx264', '-preset', 'veryfast', '-tune', 'zerolatency', '-c:a', 'aac', '-ar', '44100', '-f', 'flv', 'rtmp://localhost:1935/live/ttt']);

        // execute by stack-over-flow config 1
        // await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c:v', 'h264', '-c:a', 'aac', '-f', 'flv', "rtmp://localhost:1935/live/ttt"]);

        // execute by stack-over-flow config 2
        // await ffmpeg?.exec(['-i', 'input.webm', '-c:v', 'libx264', '-flags:v', '+global_header', '-c:a', 'aac', '-ac', '2', '-f', 'flv', "rtmp://localhost:1935/live/ttt"]);

        // execute by stack-over-flow config 3
        // await ffmpeg?.exec(['-i', 'input.webm', '-acodec', 'aac', '-ac', '2', '-strict', 'experimental', '-ab', '160k', '-vcodec', 'libx264', '-preset', 'slow', '-profile:v', 'baseline', '-level', '30', '-maxrate', '10000000', '-bufsize', '10000000', '-b', '1000k', '-f', 'flv', 'rtmp://localhost:1935/live/ttt']);

    }


  


  after running app and start streaming, console logs are as below.

  


  ffmpeg >>>  ffmpeg version 5.1.3 Copyright (c) 2000-2022 the FFmpeg developers index.tsx:81:20
ffmpeg >>>    built with emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.40 (5c27e79dd0a9c4e27ef2326841698cdd4f6b5784) index.tsx:81:20
ffmpeg >>>    configuration: --target-os=none --arch=x86_32 --enable-cross-compile --disable-asm --disable-stripping --disable-programs --disable-doc --disable-debug --disable-runtime-cpudetect --disable-autodetect --nm=emnm --ar=emar --ranlib=emranlib --cc=emcc --cxx=em++ --objcc=emcc --dep-cc=emcc --extra-cflags='-I/opt/include -O3 -msimd128' --extra-cxxflags='-I/opt/include -O3 -msimd128' --disable-pthreads --disable-w32threads --disable-os2threads --enable-gpl --enable-libx264 --enable-libx265 --enable-libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libopus --enable-zlib --enable-libwebp --enable-libfreetype --enable-libfribidi --enable-libass --enable-libzimg index.tsx:81:20
ffmpeg >>>    libavutil      57. 28.100 / 57. 28.100 index.tsx:81:20
ffmpeg >>>    libavcodec     59. 37.100 / 59. 37.100 index.tsx:81:20
ffmpeg >>>    libavformat    59. 27.100 / 59. 27.100 index.tsx:81:20
ffmpeg >>>    libavdevice    59.  7.100 / 59.  7.100 index.tsx:81:20
ffmpeg >>>    libavfilter     8. 44.100 /  8. 44.100 index.tsx:81:20
ffmpeg >>>    libswscale      6.  7.100 /  6.  7.100 index.tsx:81:20
ffmpeg >>>    libswresample   4.  7.100 /  4.  7.100 index.tsx:81:20
ffmpeg >>>    libpostproc    56.  6.100 / 56.  6.100 index.tsx:81:20
ffmpeg >>>  Input #0, matroska,webm, from 'input.webm': index.tsx:81:20
ffmpeg >>>    Metadata: index.tsx:81:20
ffmpeg >>>      encoder         : QTmuxingAppLibWebM-0.0.1 index.tsx:81:20
ffmpeg >>>    Duration: N/A, start: 0.000000, bitrate: N/A index.tsx:81:20
ffmpeg >>>    Stream #0:0(eng): Video: vp8, yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 15.50 tbr, 1k tbn (default)


  


  the problem is when ffmpeg.wasm try to execute the last command.
  
await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c', 'copy', '-f', 'flv', "rtmp://localhost:1935/live/ttt"]).
  
it just calls a GET Request, I will send further details about this request.
  
as u can see, i try to use lots of arg sample with ffmpeg?.exec, but non of them works.

  


  the network tab in browser, after ffmpeg.wasm execute the command is as below.

  


  

  


  it send a GET request to ws://localhost:1935/
and nothing happened after that.

  


  for backend, i use node-media-server and here is my output logs when ffmpeg.wasm trying to execute the args

  


  11/28/2023 19:33:18 55301 [INFO] [rtmp disconnect] id=JL569YOF
[NodeEvent on doneConnect] id=JL569YOF args=undefined


  


  at last here are my ques

  


  


  

  • how can i achive this option ?

  


  • is it possible to share webcam to rtmp server ?

  


  


  


• How to write a video stream to a server ?

  14 août, par The Mask

  Recently been playing with FFmpeg and it's powerful abilities. Working on a cool project where I'm trying to create a live video stream using FFmpeg. The client (reactJs) and server (nodeJS) are connected via web-socket. The client sends the byte packets to server and the server then spawns an FFmpeg process and serve it to an nginx server.

  


  Client(live-stream.js) :

  


  const stream = await navigator.mediaDevices.getUserMedia({
    video: true,
    audio: true,
  });
  videoRef.current.srcObject = stream;

  const ingestUrl = `ws://localhost:8081/ws`
  const socket = new WebSocket(ingestUrl);
  socket.binaryType = "arraybuffer";
  socket.onopen = () => {
    console.log("✅ WebSocket connection established");
    socket.send(JSON.stringify({ type: "start", stream_key: streamKey }));
    mediaRecorderRef.current.start(500);
  };
  socketRef.current = socket;

  socket.onerror = (error) => {
    console.error("❌ WebSocket error:", error);
  };

  mediaRecorderRef.current = new MediaRecorder(stream, {
    mimeType: "video/webm;codecs=vp8,opus",
    videoBitsPerSecond: 1000000,
    audioBitsPerSecond: 128000
  });
  mediaRecorderRef.current.ondataavailable = (event) => {
    if (event.data.size > 0 && socket.readyState === WebSocket.OPEN) {
      event.data.arrayBuffer().then((buffer) => socket.send(buffer));
    }
  };


  


  Server(index.js) :

  


  const http = require('http');
const WebSocket = require('ws');
const { spawn } = require('child_process');
const fs = require('fs');


const server = new WebSocket.Server({ server:wss, path:'/ws'});

const startFFmpeg = (stream_key) => {
  return ffmpeg = spawn("ffmpeg", [
    "-re",
    "-f", "matroska",
    "-i", "pipe:0",
    "-map", "0:v:0",
    "-map", "0:a:0",
    "-c:v", "libx264",
    "-c:a", "aac ",
    "-b:v", "6000k",
    "-maxrate", "6000k ",
    "-bufsize", "6000k ",
    "-pix_fmt", "yuv420p ",
    "-f", "flv",
    `rtmp://localhost/live/${stream_key}`,
  ]);
}
server.on('connection', (ws) => {
  console.log('📡 New WebSocket connection');

  let ffmpeg = null;
  let buffer = Buffer.alloc(0);
  let streamStarted = false;

  ws.on('message', (msg, isBinary) => {
    if (!isBinary) {
      const parsed = JSON.parse(msg);
      if (parsed.type === "start") {
        const { stream_key } = parsed;
        console.log(`🔑 Stream key: ${stream_key}`);
        console.log(`🎥 Starting ingest for stream key: ${stream_key}`);

        ffmpeg = startFFmpeg(stream_key)
        ffmpeg.stdin.on("error", (e) => {
          console.error("FFmpeg stdin error:", e.message);
        });

        ffmpeg.stderr.on("data", (data) => {
          console.log(`FFmpeg Data: ${data}`);
        });

        ffmpeg.on("close", (code) => {
          console.log(`FFmpeg exited with code ${code}`);
        });

        ffmpeg.on("exit", (code, signal) => {
          console.log(`FFmpeg exited with code: ${code}, signal: ${signal}`);
          if (signal === 'SIGSEGV') {
            console.log('🔄 FFmpeg segfaulted, attempting restart...');
            setTimeout(() => {
              if (ws.readyState === WebSocket.OPEN) {
                startFFmpeg(stream_key);
              }
            }, 1000);
          }
        });

        streamStarted = true;
      }
    } else if (isBinary && ffmpeg && ffmpeg.stdin.writable) {
        try {
        // Convert to Buffer if it's an ArrayBuffer
        let data;
        if (msg instanceof ArrayBuffer) {
          data = Buffer.from(msg);
        } else {
          data = Buffer.from(msg);
        }

        // Buffer the data
        buffer = Buffer.concat([buffer, data]);
        
        // Write in larger chunks to reduce overhead
        if (buffer.length >= 8192) { // 8KB threshold
          console.log(`📥 Writing ${buffer.length} bytes to FFmpeg`);
          
          if (ffmpeg.stdin.write(buffer)) {
            buffer = Buffer.alloc(0);
          } else {
            // Handle backpressure
            ffmpeg.stdin.once('drain', () => {
              buffer = Buffer.alloc(0);
              ffmpeg.stdin.setMaxListeners(20); // or a safe upper bound
            });
          }
        }
      } catch (e) {
        console.error("FFmpeg write error:", e);
      }
    }
  });
  
  ws.on('close', () => {
    console.log('❌ WebSocket closed');
    streamStarted = false;

    if (ffmpeg){     // Write any remaining buffer
      if (buffer.length > 0 && ffmpeg.stdin.writable) {
        console.log(`📥 Writing final ${buffer.length} bytes to FFmpeg`);
        ffmpeg.stdin.write(buffer);
      }
            
      // Gracefully close FFmpeg
      if (ffmpeg.stdin.writable) {
        ffmpeg.stdin.end();
      }
      
      setTimeout(() => {
        if (ffmpeg && !ffmpeg.killed) {
          ffmpeg.kill('SIGTERM');
          setTimeout(() => {
            if (ffmpeg && !ffmpeg.killed) {
              ffmpeg.kill('SIGKILL');
            }
          }, 5000);
        }
      }, 1000);
    }
  });
});

wss.listen(8081, "localhost", () => {
  console.log("🛰️ Server listening on http://localhost:8081/ws");
});


  


  The problem statment :
Been facing error like pixels drops in the video, bad quality. FFmpeg is crashing with error :

  


  FFmpeg Data: Input #0, matroska,webm, from 'pipe:0':

FFmpeg Data:   Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)

FFmpeg Data:   Stream #0:1(eng): Video: vp8, yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 
FFmpeg Data: 1k tbr, 1k tbn (default)
    Metadata:
      alpha_mode      : 1

FFmpeg Data: Unknown pixel format requested: yuv420p .

FFmpeg stdin error: write EPIPE
FFmpeg exited with code: 1, signal: null
FFmpeg exited with code 1


  


• How do i play an HLS stream when playlist.m3u8 file is constantly being updated ?

  3 janvier 2021, par Adnan Ahmed

  I am using MediaRecorder to record chunks of my live video in webm format from MediaStream and converting these chunks to .ts files on the server using ffmpeg and then updating my playlist.m3u8 file with this code :

  


  function generateM3u8Playlist(fileDataArr, playlistFp, isLive, cb) {
    var durations = fileDataArr.map(function(fd) {
        return fd.duration;
    });
    var maxT = maxOfArr(durations);

    var meta = [
        '#EXTM3U',
        '#EXT-X-VERSION:3',
        '#EXT-X-MEDIA-SEQUENCE:0',
        '#EXT-X-ALLOW-CACHE:YES',
        '#EXT-X-TARGETDURATION:' + Math.ceil(maxT),
    ];

    fileDataArr.forEach(function(fd) {
        meta.push('#EXTINF:' + fd.duration.toFixed(2) + ',');
        meta.push(fd.fileName2);
    });

    if (!isLive) {
        meta.push('#EXT-X-ENDLIST');
    }

    meta.push('');
    meta = meta.join('\n');

    fs.writeFile(playlistFp, meta, cb);
}


  


  Here fileDataArr holds information for all the chunks that have been created.

  


  After that i use this code to create a hls server :

  


  var runStreamServer = (function(streamFolder) {
    var executed = false;
    return function(streamFolder) {
        if (!executed) {
            executed = true;
            var HLSServer = require('hls-server')
            var http = require('http')

            var server = http.createServer()
            var hls = new HLSServer(server, {
                path: '/stream', // Base URI to output HLS streams
                dir: 'C:\\Users\\Work\\Desktop\\live-stream\\webcam2hls\\videos\\' + streamFolder // Directory that input files are stored
            })
            console.log("We are going to stream from folder:" + streamFolder);
            server.listen(8000);
            console.log('Server Listening on Port 8000');
        }
    };
})();


  


  The problem is that if i stop creating new chunks and then use the hls server link :
http://localhost:8000/stream/playlist.m3u8 then the video plays in VLC but if i try to play during the recording it keeps loading the file but does not play. I want it to play while its creating new chunks and updating playlist.m3u8. The quirk in generateM3u8Playlist function is that it adds '#EXT-X-ENDLIST' to the playlist file after i have stopped recording.
The software is still in production so its a bit messy code. Thank you for any answers.

  


  The client side that generates blobs is as follows :

  


  var mediaConstraints = {
            video: true,
            audio:true
        };
navigator.getUserMedia(mediaConstraints, onMediaSuccess, onMediaError);
function onMediaSuccess(stream) {
            console.log('will start capturing and sending ' + (DT / 1000) + 's videos when you press start');
            var mediaRecorder = new MediaStreamRecorder(stream);

            mediaRecorder.mimeType = 'video/webm';

            mediaRecorder.ondataavailable = function(blob) {
                var count2 = zeroPad(count, 5);
                // here count2 just creates a blob number 
                console.log('sending chunk ' + name + ' #' + count2 + '...');
                send('/chunk/' + name + '/' + count2 + (stopped ? '/finish' : ''), blob);
                ++count;
            };
        }
// Here we have the send function which sends our blob to server:
        function send(url, blob) {
            var xhr = new XMLHttpRequest();
            xhr.open('POST', url, true);

            xhr.responseType = 'text/plain';
            xhr.setRequestHeader('Content-Type', 'video/webm');
            //xhr.setRequestHeader("Content-Length", blob.length);

            xhr.onload = function(e) {
                if (this.status === 200) {
                    console.log(this.response);
                }
            };
            xhr.send(blob);
        }


  


  The code that receives the XHR request is as follows :

  


  var parts = u.split('/');
        var prefix = parts[2];
        var num = parts[3];
        var isFirst = false;
        var isLast = !!parts[4];

        if ((/^0+$/).test(num)) {
            var path = require('path');
            shell.mkdir(path.join(__dirname, 'videos', prefix));
            isFirst = true;
        }

        var fp = 'videos/' + prefix + '/' + num + '.webm';
        var msg = 'got ' + fp;
        console.log(msg);
        console.log('isFirst:%s, isLast:%s', isFirst, isLast);

        var stream = fs.createWriteStream(fp, { encoding: 'binary' });
        /*stream.on('end', function() {
            respond(res, ['text/plain', msg]);
        });*/

        //req.setEncoding('binary');

        req.pipe(stream);
        req.on('end', function() {
            respond(res, ['text/plain', msg]);

            if (!LIVE) { return; }

            var duration = 20;
            var fd = {
                fileName: num + '.webm',
                filePath: fp,
                duration: duration
            };
            var fileDataArr;
            if (isFirst) {
                fileDataArr = [];
                fileDataArrs[prefix] = fileDataArr;
            } else {
                var fileDataArr = fileDataArrs[prefix];
            }
            try {
                fileDataArr.push(fd);
            } catch (err) {
                fileDataArr = [];
                console.log(err.message);
            }
            videoUtils.computeStartTimes(fileDataArr);

            videoUtils.webm2Mpegts(fd, function(err, mpegtsFp) {
                if (err) { return console.error(err); }
                console.log('created %s', mpegtsFp);

                var playlistFp = 'videos/' + prefix + '/playlist.m3u8';

                var fileDataArr2 = (isLast ? fileDataArr : lastN(fileDataArr, PREV_ITEMS_IN_LIVE));

                var action = (isFirst ? 'created' : (isLast ? 'finished' : 'updated'));

                videoUtils.generateM3u8Playlist(fileDataArr2, playlistFp, !isLast, function(err) {
                    console.log('playlist %s %s', playlistFp, (err ? err.toString() : action));
                });
            });


            runStreamServer(prefix);
        }