Sur d’autres sites (8437)

• No audio in the final video when converting webm blobs to mp4 using ffmpeg

  28 septembre 2024, par alpecca

  I trying to record user camera and microphone and using MediaRecorder to convert the stream to blobs and sending the blobs every 2 second to the backend using websocket. Everything is working fine, but when I checked the final mp4 video in the backend, it doesn't have any audio to it, I try specifying the audio codec, but still no help.

  


  My frontend code :-

  


  const micStream = await navigator.mediaDevices.getUserMedia({ audio: true });

const recorder = new MediaRecorder(stream, {
   mimeType: 'video/webm;codecs=H264',
   videoBitsPerSecond: 8000000,
   audioBitsPerSecond : 8000000
});

recorder.ondataavailable = (e: BlobEvent) => {
    websocket.send(e.data)         
}  
recorder.start(2000);


  


  And here is the backend code :-

  


  @router.websocket("/streamaudio")
async def websocket_endpoint(websocket: WebSocket):
    await manager.connect(websocket)

    recordingFile = os.path.join(os.getcwd(), f"recording_.mp4")

    command = [
        'ffmpeg', 
        '-y',
        '-i', 
        '-', 
        '-codec:v', 
        'copy', 
        '-c:a', 'aac', 
        '-y',
        '-f', 'mp4',
        recordingFile,
        # "-"
        # f'output{queueNumber}.mp4',
    ]  

    
    try:
        while True:
            try:
           
                data = await websocket.receive_bytes()
                
                process.stdin.send(data)
               
            except RuntimeError:
                break      
    except WebSocketDisconnect:
        print(f"Client disconnected: {websocket.client.host}")
    finally:
        manager.disconnect(websocket)
        await process.stdin.aclose()
        await process.wait()  


  


• Unable to read video streams on FFMPEG and send it to youTube RTMP server

  29 août 2024, par Rahul Bundele

  I'm trying to send two video stream from browser as array buffer (webcam and screen share video) to server via Web RTC data channels and want ffmpeg to add webcam as overlay on screen share video and send it to youtube RTMP server, the RTC connections are established and server does receives buffer , Im getting error in Ffmpeg..error is at bottom , any tips on to add overlay and send it to youtube RTMP server would be appreciated.

  


  Client.js

  


  `
const webCamStream = await navigator.mediaDevices.getUserMedia( video : true ,audio:true ) ;
const screenStream = await navigator.mediaDevices.getDisplayMedia( video : true ) ;

  


  const webcamRecorder = new MediaRecorder(webCamStream, { mimeType: 'video/webm' });
webcamRecorder.ondataavailable = (event) => {
    if (event.data.size > 0 && webcamDataChannel.readyState === 'open') {
        const reader = new FileReader();
        reader.onload = function () {
            const arrayBuffer = this.result;
            webcamDataChannel.send(arrayBuffer);
        };
        reader.readAsArrayBuffer(event.data);
    }
};
webcamRecorder.start(100);  // Adjust the interval as needed

// Send screen share stream data
const screenRecorder = new MediaRecorder(screenStream, { mimeType: 'video/webm' });
screenRecorder.ondataavailable = (event) => {
    if (event.data.size > 0 && screenDataChannel.readyState === 'open') {
        const reader = new FileReader();
        reader.onload = function () {
            const arrayBuffer = this.result;
            screenDataChannel.send(arrayBuffer);
        };
        reader.readAsArrayBuffer(event.data);
    }
};
screenRecorder.start(100); 


  


  `

  


  Server.js

  


  const youtubeRTMP = 'rtmp://a.rtmp.youtube.com/live2/youtube key';

// Create PassThrough streams for webcam and screen
const webcamStream = new PassThrough();
const screenStream = new PassThrough();

// FFmpeg arguments for processing live streams
const ffmpegArgs = [
  '-re',
  '-i', 'pipe:3',                  // Webcam input via pipe:3
  '-i', 'pipe:4',                  // Screen share input via pipe:4
  '-filter_complex',               // Complex filter for overlay
  '[0:v]scale=320:240[overlay];[1:v][overlay]overlay=10:10[out]',
  '-map', '[out]',                 // Map the output video stream
  '-c:v', 'libx264',               // Use H.264 codec for video
  '-preset', 'ultrafast',          // Use ultrafast preset for low latency
  '-crf', '25',                    // Set CRF for quality/size balance
  '-pix_fmt', 'yuv420p',           // Pixel format for compatibility
  '-c:a', 'aac',                   // Use AAC codec for audio
  '-b:a', '128k',                  // Set audio bitrate
  '-f', 'flv',                     // Output format (FLV for RTMP)
  youtubeRTMP                      // Output to YouTube RTMP server
];

// Spawn the FFmpeg process
const ffmpegProcess = spawn('ffmpeg', ffmpegArgs, {
  stdio: ['pipe', 'pipe', 'pipe', 'pipe', 'pipe']
});

// Pipe the PassThrough streams into FFmpeg
webcamStream.pipe(ffmpegProcess.stdio[3]);
screenStream.pipe(ffmpegProcess.stdio[4]);

ffmpegProcess.on('close', code => {
  console.log(`FFmpeg process exited with code ${code}`);
});

ffmpegProcess.on('error', error => {
  console.error(`FFmpeg error: ${error.message}`);
});

const handleIncomingData = (data, stream) => {
  const buffer = Buffer.from(data);
  stream.write(buffer);
};


  


  the server gets the video buffer via webrtc data channels

  


  pc.ondatachannel = event => {
        const dataChannel = event.channel;
        pc.dc = event.channel;
        pc.dc.onmessage = event => {
            // Spawn the FFmpeg process
            // console.log('Message from client:', event.data);
            const data = event.data;

            if (dataChannel.label === 'webcam') {
            handleIncomingData(data, webcamStream);
            } else if (dataChannel.label === 'screen') {
            handleIncomingData(data, screenStream);
            }
          
        };
        pc.dc.onopen = e=>{
            // recHead.innerText = "Waiting for user to send files"
            console.log("channel opened!")
        }
    };


  


  Im getting this error in ffmpeg

  


  [in#0 @ 0000020e585a1b40] Error opening input: Bad file descriptor
Error opening input file pipe:3.
Error opening input files: Bad file descriptor


  


• Issue with creating Video files from Binary files

  22 septembre 2022, par user20057686

  We have a bunch of binary files that represent Video data.
This is how the binary files were created :

  


  

  1. Used MediaRecorder from a React application to capture the browser window.
To capture the screen stream we used (Navigator.)MediaDevices.getDisplayMedia() API

  


  2. Each video is recorded for 1-second duration

  


  3. This data is then encoded with base64 and sent through a websocket. The server decodes the base64 string and stores the binary data in a file (without any extension)

  


  


  So we now have a bunch of binary files each containing 1 second worth of video data.

  


  The issue is, we are not able to convert all the binary files back to a single video.

  


  


  1. We tried using ffmpeg

     


     copy /b * merged.

     


     ffmpeg -i merged merged.mp4

     


  


  


  Basically first merging all the binary files and converting to mp4. It didn't work. The resulting video duration is not equal to the (number_of_files) in seconds.

  


  


  2. We also tried converting individual chunks with ffmpeg but we get the below error :

     


     [h264 @ 000001522dc74b80] [error] non-existing PPS 0 referenced
[h264 @ 000001522dc74b80] [error] non-existing PPS 0 referenced
[h264 @ 000001522dc74b80] [error] decode_slice_header error
[h264 @ 000001522dc74b80] [error] no frame !
I can provide the complete logs if needed.

     


  


  3. Next thing we tried was to use MoviePy library in Python. We programmatically concatenated the files and saved them as WebM and imported it into MoviePy as a Video.

     


  


  


  In all the above approaches, we couldn't get the full video.