Recherche avancée

Sur d’autres sites (5381)

• avfilter/avf_showcqt : cqt_calc optimization on x86

  4 juin 2016, par Muhammad Faiz

  avfilter/avf_showcqt : cqt_calc optimization on x86
  on x86_64 :
  
          time    PSNR
  
  plain   3.303   inf
  
  SSE     1.649   107.087535
  
  SSE3    1.632   107.087535
  
  AVX     1.409   106.986771
  
  FMA3    1.265   107.108437
  on x86_32 (PSNR compared to x86_64 plain) :
  
          time    PSNR
  
  plain   7.225   103.951979
  
  SSE     1.827   105.859282
  
  SSE3    1.819   105.859282
  
  AVX     1.533   105.997661
  
  FMA3    1.384   105.885377
  FMA4 test is not available
  Reviewed-by : James Almer <jamrial@gmail.com>
  
  Signed-off-by : Muhammad Faiz <mfcc64@gmail.com>
  

  • [DH] libavfilter/avf_showcqt.c
  • [DH] libavfilter/avf_showcqt.h
  • [DH] libavfilter/x86/Makefile
  • [DH] libavfilter/x86/avf_showcqt.asm
  • [DH] libavfilter/x86/avf_showcqt_init.c

• webrtc to rtmp send video from camera to rtmp link

  14 avril 2024, par Leo-Mahendra

  i cant send the video from webrtc which is converted to bufferd data for every 10seconds and send to server.js where it takes it via websockets and convert it to flv format using ffmpeg.

  


  i am trying to send it to rtmp server named restreamer for start, here i tried to convert the buffer data and send it to rtmp link using ffmpeg commands, where i initially started to suceesfully save the file from webrtc to mp4 format for a duration of 2-3 minute.

  


  after i tried to use webrtc to send video data for every 10 seconds and in server i tried to send it to rtmp but i cant send it, but i can see the connection of rtmp url and server is been taken place but i cant see the video i can see the logs in rtmp server as

  


  2024-04-14 12:35:45 ts=2024-04-14T07:05:45Z level=INFO component="RTMP" msg="no streams available" action="INVALID" address=":1935" client="172.17.0.1:37700" path="/3d30c5a9-2059-4843-8957-da963c7bc19b.stream" who="PUBLISH"
2024-04-14 12:35:45 ts=2024-04-14T07:05:45Z level=INFO component="RTMP" msg="no streams available" action="INVALID" address=":1935" client="172.17.0.1:37716" path="/3d30c5a9-2059-4843-8957-da963c7bc19b.stream" who="PUBLISH"
2024-04-14 12:35:45 ts=2024-04-14T07:05:45Z level=INFO component="RTMP" msg="no streams available" action="INVALID" address=":1935" client="172.17.0.1:37728" path="/3d30c5a9-2059-4843-8957-da963c7bc19b.stream" who="PUBLISH"   


  


  my frontend code

  


       const handleSendVideo = async () => {
        console.log("start");
    
        if (!ws) {
            console.error('WebSocket connection not established.');
            return;
        }
    
        try {
            const videoStream = await navigator.mediaDevices.getUserMedia({ video: true });
            const mediaRecorder = new MediaRecorder(videoStream);
    
            const requiredFrameSize = 460800;
            const frameDuration = 10 * 1000; // 10 seconds in milliseconds
    
            mediaRecorder.ondataavailable = async (event) => {
                if (ws.readyState !== WebSocket.OPEN) {
                    console.error('WebSocket connection is not open.');
                    return;
                }
    
                if (event.data.size > 0) {
                    const arrayBuffer = await event.data.arrayBuffer();
                    const uint8Array = new Uint8Array(arrayBuffer);
    
                    const width = videoStream.getVideoTracks()[0].getSettings().width;
                    const height = videoStream.getVideoTracks()[0].getSettings().height;
    
                    const numFrames = Math.ceil(uint8Array.length / requiredFrameSize);
    
                    for (let i = 0; i < numFrames; i++) {
                        const start = i * requiredFrameSize;
                        const end = Math.min((i + 1) * requiredFrameSize, uint8Array.length);
                        let frameData = uint8Array.subarray(start, end);
    
                        // Pad or trim the frameData to match the required size
                        if (frameData.length < requiredFrameSize) {
                            // Pad with zeros to reach the required size
                            const paddedData = new Uint8Array(requiredFrameSize);
                            paddedData.set(frameData, 0);
                            frameData = paddedData;
                        } else if (frameData.length > requiredFrameSize) {
                            // Trim to match the required size
                            frameData = frameData.subarray(0, requiredFrameSize);
                        }
    
                        const dataToSend = {
                            buffer: Array.from(frameData), // Convert Uint8Array to array of numbers
                            width: width,
                            height: height,
                            pixelFormat: 'yuv420p',
                            mode: 'SendRtmp'
                        };
    
                        console.log("Sending frame:", i);
                        ws.send(JSON.stringify(dataToSend));
                    }
                }
            };
    
            // Start recording and send data every 10 seconds
            mediaRecorder.start(frameDuration);
    
            console.log("MediaRecorder started.");
        } catch (error) {
            console.error('Error accessing media devices or starting recorder:', error);
        }
      };


  


  and my backend

  


      wss.on('connection', (ws) => {
    console.log('WebSocket connection established.');

    ws.on('message', async (data) => {
        try {
            const parsedData = JSON.parse(data);

            if (parsedData.mode === 'SendRtmp' && Array.isArray(parsedData.buffer)) {
                const { buffer, pixelFormat, width, height } = parsedData;
                const bufferArray = Buffer.from(buffer);

                await sendRtmpVideo(bufferArray, pixelFormat, width, height);
            } else {
                console.log('Received unknown or invalid mode or buffer data');
            }
        } catch (error) {
            console.error('Error parsing WebSocket message:', error);
        }
    });

    ws.on('close', () => {
        console.log('WebSocket connection closed.');
    });
    });
    const sendRtmpVideo = async (frameBuffer, pixelFormat, width, height) => {
    console.log("ffmpeg data",frameBuffer)
    try {
        const ratio = `${width}x${height}`;
        const ffmpegCommand = [
            '-re',
            '-f', 'rawvideo',
            '-pix_fmt', pixelFormat,
            '-s', ratio,
            '-i', 'pipe:0',
            '-c:v', 'libx264',
            '-preset', 'fast', // Specify the preset for libx264
            '-b:v', '3000k',    // Specify the video bitrate
            '-loglevel', 'debug',
            '-f', 'flv',
            // '-flvflags', 'no_duration_filesize', 
            RTMPLINK
        ];


        const ffmpeg = spawn('ffmpeg', ffmpegCommand);

        ffmpeg.on('exit', (code, signal) => {
            if (code === 0) {
                console.log('FFmpeg process exited successfully.');
            } else {
                console.error(`FFmpeg process exited with code ${code} and signal ${signal}`);
            }
        });

        ffmpeg.on('error', (error) => {
            console.error('FFmpeg spawn error:', error);
        });

        ffmpeg.stderr.on('data', (data) => {
            console.error(`FFmpeg stderr: ${data}`);
        });

        ffmpeg.stdin.write(frameBuffer, (err) => {
            if (err) {
                console.error('Error writing to FFmpeg stdin:', err);
            } else {
                console.log('Data written to FFmpeg stdin successfully.');
            }
            ffmpeg.stdin.end(); // Close stdin after writing the buffer
        });
        } catch (error) {
        console.error('Error in sendRtmpVideo:', error);
        }
    };



  


• Transcode webcam blob to RTMP using ffmpeg.wasm

  29 novembre 2023, par hassan moradnezhad

  I'm trying transcode webcam blob data to a rtmp server from browser by using ffmpeg.wasm .
  
first, i create a MediaStream.

  


          const stream = await navigator.mediaDevices.getUserMedia({
            video: true,
        });


  


  then, i create a MediaRecorder.

  


          const recorder = new MediaRecorder(stream, {mimeType: "video/webm; codecs:vp9"});
        recorder.ondataavailable = handleDataAvailable;
        recorder.start(0)


  


  when data is available, i call a function called handleDataAvailable.
  
here is the function.

  


      const handleDataAvailable = (event: BlobEvent) => {
        console.log("data-available");
        if (event.data.size > 0) {
            recordedChunksRef.current.push(event.data);
            transcode(event.data)
        }
    };


  


  in above code, i use another function which called transcode it's goal is going to send data to rtmp server using use ffmpeg.wasm.
  
here it is.

  


  const transcode = async (inputVideo: Blob | undefined) => {
        const ffmpeg = ffmpegRef.current;
        const fetchFileOutput = await fetchFile(inputVideo)
        ffmpeg?.writeFile('input.webm', fetchFileOutput)

        const data = await ffmpeg?.readFile('input.webm');
        if (videoRef.current) {
            videoRef.current.src =
                URL.createObjectURL(new Blob([(data as any)?.buffer], {type: 'video/webm'}));
        }

        // execute by node-media-server config 1
        await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c', 'copy', '-f', 'flv', "rtmp://localhost:1935/live/ttt"])

        // execute by node-media-server config 2
        // await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c:v', 'libx264', '-preset', 'veryfast', '-tune', 'zerolatency', '-c:a', 'aac', '-ar', '44100', '-f', 'flv', 'rtmp://localhost:1935/live/ttt']);

        // execute by stack-over-flow config 1
        // await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c:v', 'h264', '-c:a', 'aac', '-f', 'flv', "rtmp://localhost:1935/live/ttt"]);

        // execute by stack-over-flow config 2
        // await ffmpeg?.exec(['-i', 'input.webm', '-c:v', 'libx264', '-flags:v', '+global_header', '-c:a', 'aac', '-ac', '2', '-f', 'flv', "rtmp://localhost:1935/live/ttt"]);

        // execute by stack-over-flow config 3
        // await ffmpeg?.exec(['-i', 'input.webm', '-acodec', 'aac', '-ac', '2', '-strict', 'experimental', '-ab', '160k', '-vcodec', 'libx264', '-preset', 'slow', '-profile:v', 'baseline', '-level', '30', '-maxrate', '10000000', '-bufsize', '10000000', '-b', '1000k', '-f', 'flv', 'rtmp://localhost:1935/live/ttt']);

    }


  


  after running app and start streaming, console logs are as below.

  


  ffmpeg >>>  ffmpeg version 5.1.3 Copyright (c) 2000-2022 the FFmpeg developers index.tsx:81:20
ffmpeg >>>    built with emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.40 (5c27e79dd0a9c4e27ef2326841698cdd4f6b5784) index.tsx:81:20
ffmpeg >>>    configuration: --target-os=none --arch=x86_32 --enable-cross-compile --disable-asm --disable-stripping --disable-programs --disable-doc --disable-debug --disable-runtime-cpudetect --disable-autodetect --nm=emnm --ar=emar --ranlib=emranlib --cc=emcc --cxx=em++ --objcc=emcc --dep-cc=emcc --extra-cflags='-I/opt/include -O3 -msimd128' --extra-cxxflags='-I/opt/include -O3 -msimd128' --disable-pthreads --disable-w32threads --disable-os2threads --enable-gpl --enable-libx264 --enable-libx265 --enable-libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libopus --enable-zlib --enable-libwebp --enable-libfreetype --enable-libfribidi --enable-libass --enable-libzimg index.tsx:81:20
ffmpeg >>>    libavutil      57. 28.100 / 57. 28.100 index.tsx:81:20
ffmpeg >>>    libavcodec     59. 37.100 / 59. 37.100 index.tsx:81:20
ffmpeg >>>    libavformat    59. 27.100 / 59. 27.100 index.tsx:81:20
ffmpeg >>>    libavdevice    59.  7.100 / 59.  7.100 index.tsx:81:20
ffmpeg >>>    libavfilter     8. 44.100 /  8. 44.100 index.tsx:81:20
ffmpeg >>>    libswscale      6.  7.100 /  6.  7.100 index.tsx:81:20
ffmpeg >>>    libswresample   4.  7.100 /  4.  7.100 index.tsx:81:20
ffmpeg >>>    libpostproc    56.  6.100 / 56.  6.100 index.tsx:81:20
ffmpeg >>>  Input #0, matroska,webm, from 'input.webm': index.tsx:81:20
ffmpeg >>>    Metadata: index.tsx:81:20
ffmpeg >>>      encoder         : QTmuxingAppLibWebM-0.0.1 index.tsx:81:20
ffmpeg >>>    Duration: N/A, start: 0.000000, bitrate: N/A index.tsx:81:20
ffmpeg >>>    Stream #0:0(eng): Video: vp8, yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 15.50 tbr, 1k tbn (default)


  


  the problem is when ffmpeg.wasm try to execute the last command.
  
await ffmpeg?.exec(['-re', '-i', 'input.webm', '-c', 'copy', '-f', 'flv', "rtmp://localhost:1935/live/ttt"]).
  
it just calls a GET Request, I will send further details about this request.
  
as u can see, i try to use lots of arg sample with ffmpeg?.exec, but non of them works.

  


  the network tab in browser, after ffmpeg.wasm execute the command is as below.

  


  

  


  it send a GET request to ws://localhost:1935/
and nothing happened after that.

  


  for backend, i use node-media-server and here is my output logs when ffmpeg.wasm trying to execute the args

  


  11/28/2023 19:33:18 55301 [INFO] [rtmp disconnect] id=JL569YOF
[NodeEvent on doneConnect] id=JL569YOF args=undefined


  


  at last here are my ques

  


  


  

  • how can i achive this option ?

  


  • is it possible to share webcam to rtmp server ?