Recherche avancée

Sur d’autres sites (8893)

• Problem with FFmpeg breaking when streaming the entire screen and switching tabs

  2 octobre 2024, par Ibad Ahmad

  I'm working on a screen recording and streaming setup where the user records their entire screen and streams it to Twitch. The setup works fine initially, but when I switch tabs during recording, the stream breaks on the backend, and I get the following FFmpeg errors :

  


  FFmpeg STDERR: [matroska,webm @ 0x7f9dcb904580] EBML header parsing failed
[in#0 @ 0x7f9dcb904380] Error opening input: Invalid data found when processing input
Error opening input file -.
Error opening input files: Invalid data found when processing input


  


  My frontend code captures the screen and microphone and streams it via a WebSocket to the backend, where FFmpeg processes the stream. Below is my relevant frontend code :

  


  const startRecording = async () => {
    try {
        const screenStream = await navigator.mediaDevices.getDisplayMedia({
            preferCurrentTab: true,
            systemAudio: 'include',
            surfaceSwitching: 'include',
            monitorTypeSurfaces: 'include',
            video: {
                displaySurface: 'browser',
                height: 720,
                width: 1280,
                frameRate: { ideal: 24, max: 30 },
            },
        });

        screenStream.getVideoTracks()[0].onended = () => {
            console.log('Screen sharing ended. Stopping the recorder.');
            stopRecording();
        };

        const micStream = await navigator.mediaDevices.getUserMedia({
            audio: true,
        });

        const combinedStream = new MediaStream([
            ...screenStream.getVideoTracks(),
            ...micStream.getAudioTracks(),
        ]);

        const recorder = new MediaRecorder(combinedStream, {
            mimeType: 'video/webm; codecs=vp8,opus',
            videoBitsPerSecond: 3 * 1024 * 1024,
        });

        const timeslice = 1000;

        recorder.ondataavailable = async (event) => {
            if (socket?.current?.connected && event.data.size > 0) {
                console.log('Sending chunk data:', socket.current.id);
                socket?.current.send(event.data);
                recordedChunks.current.push(event.data);
            } else if (!socket?.current?.connected) {
                handleSocketDisconnection();
            }
        };

        mediaRecorder.current = recorder;
        recorder.start(timeslice);
        setIsRecording(true);
    } catch (error) {
        console.log('Error starting screen recording:', error);
        toast.error('Failed to start screen recording: ' + error);
    }
};

const stopRecording = () => {
    if (socket?.current && mediaRecorder) {
        mediaRecorder?.current?.stop();
        socket.current.close();
        setIsRecording(false);
        downloadRecordedVideo();
    }
};



  


  And here’s my backend code with FFmpeg settings for Twitch streaming :

  


  const inputSettings = [
    '-f', 'webm', '-i', '-', '-v', 'error', '-analyzeduration', '1000000', '-probesize', '5000000',
];

const twitchSettings = (twitch) => {
    return [
        '-c:v', 'libx264', '-preset', 'veryfast', '-tune', 'zerolatency',
        '-g', '60', '-b:v', '2500k', '-maxrate', '3000k', '-bufsize', '8000k',
        '-r', '30', '-vf', 'tpad=stop_mode=clone:stop_duration=2',
        '-c:a', 'aac', '-ar', '44100', '-b:a', '96k',
        '-use_wallclock_as_timestamps', '1', '-async', '1',
        '-err_detect', 'ignore_err', '-reconnect', '1',
        '-reconnect_streamed', '1', '-reconnect_delay_max', '5',
        '-y', '-f', 'flv', twitch,
    ];
};



  


  Problem : When switching tabs during screen sharing, it seems like the frame rate drops or the stream gets interrupted, leading to FFmpeg errors like EBML header parsing failed and Invalid data found when processing input. I suspect this happens because the browser deprioritizes resources when the tab is not active, which might lead to corrupt chunks being sent to FFmpeg.

  


  Questions :

  


  

  1. Could switching tabs during screen capture be causing the issue by disrupting the frame rate or dropping frames ?

  


  2. Is there a way to ensure FFmpeg doesn’t break due to these interruptions ?

  


  3. Any suggestions on handling the stream more reliably when switching tabs or optimizing the FFmpeg setup for this scenario ?

  


  


  I tried adjusting the bitrate, frame rate, and buffer size but still experienced the same issue. I'm trying to figure out if the issue is related to how browsers handle screen capture when tab switching or something specific with FFmpeg handling the video stream.

  


  Any insights would be greatly appreciated.
Thanks in advance !

  


• Try to concatenate videos using FFmpeg Package. My videos concatenate correctly but those that i record from fornt camera rotate 90' in concatenate

  24 avril 2024, par Ahmad Akram

  Here is my code where I pass a list of image paths that concatenate. I am facing an issue with the front camera video. When concatenated completely some videos rotate 90 degrees.

  


  Future<void> mergeVideos(List<string> videoPaths) async {&#xA;    VideoHelper.showInSnackBar(&#x27;Videos merged Start&#x27;, context);&#xA;    String outputPath = await VideoHelper.generateOutputPath();&#xA;    FlutterFFmpeg flutterFFmpeg = FlutterFFmpeg();&#xA;&#xA;    // Create a text file containing the paths of the videos to concatenate&#xA;    String fileListPath =&#xA;        &#x27;${(await getTemporaryDirectory()).path}/fileList.txt&#x27;;&#xA;    File fileList = File(fileListPath);&#xA;    await fileList&#xA;        .writeAsString(videoPaths.map((path) => &#x27;file \&#x27;$path\&#x27;&#x27;).join(&#x27;\n&#x27;));&#xA;&#xA;    // Run FFmpeg command to concatenate videos&#xA;    // String command = &#x27;-f concat -safe 0 -i $fileListPath -c copy $outputPath&#x27;;&#xA;&#xA;    String command =&#xA;        &#x27;-f concat -safe 0 -i $fileListPath -vf "transpose=1" -c:a copy $outputPath&#x27;;&#xA;&#xA;    VideoHelper.showInSnackBar(&#x27;command Start&#x27;, context);&#xA;    await flutterFFmpeg.execute(command).then((value) {&#xA;      if (value == 0) {&#xA;        print("Output Path : $outputPath");&#xA;        VideoHelper.showInSnackBar(&#x27;Videos merged successfully&#x27;, context);&#xA;        Navigator.push(&#xA;            context,&#xA;            MaterialPageRoute(&#xA;                builder: (context) => VideoPlayerScreen(&#xA;                      videoFile: XFile(outputPath),&#xA;                    )));&#xA;      } else {&#xA;        VideoHelper.showInSnackBar(&#xA;            &#x27;Error merging videos  ::::: returnCode=== $value &#x27;, context);&#xA;      }&#xA;    });&#xA;  }&#xA;</string></void>

  


• FFmpeg to RTMP - no audio on output [closed]

  25 mars 2022, par John Mergene Arellano

  From my client side, I am sending a stream using the Socket.IO library. I captured the video and audio using getUserMedia API.

  


  navigator.mediaDevices.getUserMedia(constraints).then((stream) => {
    window.videoStream = video.srcObject = stream;
    let mediaRecorder = new MediaRecorder(stream, {
        videoBitsPerSecond : 3 * 1024 * 1024
    });
    mediaRecorder.addEventListener('dataavailable', (e) => {
        let data = e.data;
        socket.emit('live', data);
    });
    mediaRecorder.start(1000);
});


  


  Then my server will receive the stream and write it to FFmpeg.

  


  client.on('live', (stream)=>{
   if(ffmpeg)
       ffmpeg.stdin.write(stream);
});


  


  I tried watching the live video in VLC media player. There is a 5 seconds delay and no audio output.

  


  Please see below for FFmpeg options I used :

  


  ffmpeg = this.CHILD_PROCESS.spawn("ffmpeg", [
   '-f',
   'lavfi',
   '-i', 'anullsrc',
   '-i','-',
   '-c:v', 'libx264', '-preset', 'veryfast', '-tune', 'zerolatency',
   '-c:a', 'aac', '-ar', '44100', '-b:a', '64k',
   '-y', //force to overwrite
   '-use_wallclock_as_timestamps', '1', // used for audio sync
   '-async', '1', // used for audio sync
   '-bufsize', '1000',
   '-f',
   'flv',
   `rtmp://127.0.0.1:1935/live/stream` ]);


  


  What is wrong with my setup ?

  


  I tried removing some of the options but failed. I am expecting to have an output of video and audio from the getUserMedia API.