Recherche avancée

Sur d’autres sites (8775)

• Ffmpeg configuration to stream the frames of my webcam

  6 juillet 2023, par Ridweng

  I'm trying to build a server in NodeJs to stream in RTSP my webcam using Angular to retrieve the frames and connecting using websockets. The server side is using "express-ws" module to create the Websocket.

  


  I was successfull in sending the frames from the webcam to the server in base64, the server is receiving these messages from a function on interval of (1000 / 30)ms.

  


  The issue relies on the implementation of my Ffmpeg child process. I retrieve the message and convert it into a buffer to then write this in the function of Ffmpeg.

  


  My current implementation is this one :

  


  const { spawn } = require('child_process');
exports.stream =  (ws ,req) => {
    try{
        const mess = `connection from: ${req._remoteAddress} at ${req._startTime}.`
        const initialMess =`Started ${mess}`
        ws.uuid = uuidv4()
        console.log(initialMess)

        ws.on('message', function incoming(message) {
            message = JSON.parse(message)
            console.log(`Res: ${message.width} x ${message.height}`);
            const ffmpeg = spawn('ffmpeg', [
                '-f', 'rawvideo',
                '-pixel_format', 'rgb24',
                '-video_size', `${message.width}x${message.height}`,
                '-framerate', `${message.framerate}`,
                '-i', '-',
                '-codec:v', 'libx264',
                '-preset', 'ultrafast',
                '-tune', 'zerolatency',
                '-f', 'rtsp',
                'rtsp://127.0.0.1:554/rtsp/stream',
            ]);
            const base64Data = message.video;
            const videoData = Buffer.from(base64Data, 'base64');

            ffmpeg.stdin.write(videoData);
            ffmpeg.stdin.end();
            
            ffmpeg.stderr.on('data', (data) => {
                console.error(`FFmpeg : ${data}`);
                });

            ffmpeg.on('exit', (code, signal) => {
                if (dev) console.log(`FFmpeg process exited with code ${code} and signal ${signal}`);
        
                // Close the WebSocket connection
                ws.close();
                });
        });
        ws.on('close', () => {
            const finalMess = `Stopped ${mess}`
            rem(ws.uuid)
            console.log(finalMess)
        })
    }catch(err){
        console.log(err)
    }
}


  


  In terms of the message received, this is the Angular side sending the message :

  


  const imageData = this.canvas.toDataURL('image/jpeg');
socket.send(JSON.stringify({video: imageData, width: this.canvas.width, height: this.canvas.height, framerate: interval }));


  


  The interval variable is the divisor of the interval that is triggering the function (in this case 30).

  


  I'm currently receiving the error message from Ffmpeg :

  


  ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)

FFmpeg :   configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100

FFmpeg : [rawvideo @ 0x150005ff0] Packet corrupt (stream = 0, dts = 0).

FFmpeg : Input #0, rawvideo, from 'fd:':
  Duration: N/A, start: 0.000000, bitrate: 221184 kb/s
  Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 640x480, 221184 kb/s, 30 tbr, 30 tbn

FFmpeg : Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))

FFmpeg : fd:: corrupt input packet in stream 0
[rawvideo @ 0x14ef24290] Invalid buffer size, packet size 76343 < expected frame_size 921600

FFmpeg : Error while decoding stream #0:0: Invalid argument

FFmpeg : [libx264 @ 0x14ef25a20] using cpu capabilities: ARMv8 NEON

FFmpeg : [libx264 @ 0x14ef25a20] profile High 4:4:4 Predictive, level 3.0, 4:4:4, 8-bit

FFmpeg : [libx264 @ 0x14ef25a20] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=6 threads=7 lookahead_threads=7 sliced_threads=1 slices=7 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0

FFmpeg : [tcp @ 0x1500078a0] Connection to tcp://127.0.0.1:554?timeout=0 failed: Connection refused
[out#0/rtsp @ 0x14ef24b00] Could not write header (incorrect codec parameters ?): Connection refused

FFmpeg : [vost#0:0/libx264 @ 0x14ef256d0] Error initializing output stream: 

FFmpeg : Conversion failed!

FFmpeg process exited with code 1 and signal null


  


  This is the reason I believe the issue relies on the configuration of Ffmpeg and how to set it for this case. It would be great to being able to play this RTSP link in my VLC to conclude it as successfull.

  


  I would apreciate any suggestions or guidence.
Thanks in advance.

  


• FFmpeg transcoding video H.264 -> AV1 leads to distorted image

  21 juin 2023, par user1584149

  I'm reading video from stdin via pipe similar to this one Python gets stuck at pipe.stdin.write(image.tostring())

  


  FFmpeg is using libsvtav1 codec with options according to documentation

  


  ffmpeg -y -f rawvideo -i pipe:0 -s {frame_width}x{frame_height} -r {frame_rate} -c:a copy -c:v libsvtav1 -preset 6 -crf 30 -svtav1-params "color-format=1:film-grain-denoise=0:mbr=2M" {output_file} 


  


  Here's couple of details about used FFmpeg version as well as input video

  


  ffmpeg version n5.1.3-14-ge5b5dd6653-20230621 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 13.1.0 (crosstool-NG 1.25.0.196_227d99d)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --disable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --disable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230621
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, rawvideo, from 'pipe:0':
  Duration: N/A, start: 0.000000, bitrate: 331776 kb/s
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 331776 kb/s, 30 tbr, 30 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> av1 (libsvtav1))
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:   SVT-AV1 Encoder Lib v1.5.0-12-g0f8b3a81
Svt[info]: SVT [build]  :   GCC 13.1.0   64 bit
Svt[info]: LIB Build date: Jun 20 2023 23:48:14
Svt[info]: -------------------------------------------


  


  The output video transcoded by SVT-AV1 codec is blemished as you can see on the images below. Normally I would say it could be caused by wrong pixel format, but input stream has yuv420 format what is the same as encoder's pixel format parameter —color-format.

  


  Please share your experience or ideas how to get it transcoding correctly.

  


  Thanks in advance.

  


  


  


• Pipe opencv frames into ffmpeg

  25 juin 2023, par Dmytro Soltusyuk

  I am trying to pipe opencv frames into ffmpeg, but it does not work.

  


  After the research, I found this answer (https://stackoverflow.com/a/62807083/10676682) to work the best for me, so I have the following :

  


  def start_streaming_process(rtmp_url, width, height, fps):
    # fmt: off
    cmd = ['ffmpeg',
           '-y',
           '-f', 'rawvideo',
           '-vcodec', 'rawvideo',
           '-pix_fmt', 'bgr24',
           '-s', "{}x{}".format(width, height),
           '-r', str(fps),
           '-i', '-',
           '-c:v', 'libx264',
           '-pix_fmt', 'yuv420p',
           '-preset', 'ultrafast',
           '-f', 'flv',
           '-flvflags', 'no_duration_filesize',
           rtmp_url]
    # fmt: on

    return subprocess.Popen(cmd, stdin=subprocess.PIPE)


  


  def main():
    width, height, fps = get_video_size(SOURCE_VIDEO_PATH)
    streaming_process = start_streaming_process(
        TARGET_VIDEO_PATH,
        width,
        height,
        fps,
    )

    model = load_yolo(WEIGHTS_PATH)
    frame_iterator = read_frames(video_source=SOURCE_VIDEO_PATH)
    processed_frames_iterator = process_frames(
        model, frame_iterator, ball_target_area=400
    )

    for processed_frame in processed_frames_iterator:
        streaming_process.communicate(processed_frame.tobytes())

    streaming_process.kill()


  


  processed_frame here is an annotated OpenCV frame.

  


  However, after I do my first streaming_process.communicate call, the ffmpeg process exits with code 0 (meaning everything was ok), but it is not. I can not feed the rest of the frames into ffmpeg, because the process exited.

  


  Here are the logs :

  


  Input #0, rawvideo, from 'fd:':
  Duration: N/A, start: 0.000000, bitrate: 663552 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1280x720, 663552 kb/s, 30 tbr, 30 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
[libx264 @ 0x132e05570] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x132e05570] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x132e05570] 264 - core 164 r3095 baee400 - H.264/MPEG-4 AVC codec - Copyleft 2003-2022 - h
ttp://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme
=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 
fast_pskip=1 chroma_qp_offset=-2 threads=15 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 inter
laced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=
1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbt
ree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, flv, to 'rtmp://global-live.mux.com:5222/app/9428e064-e5d3-0bee-dc67-974ba53ce164':
  Metadata:
    encoder         : Lavf60.3.100
  Stream #0:0: Video: h264 ([7][0][0][0] / 0x0007), yuv420p(tv, progressive), 1280x720, q=2-31, 30 fps
, 1k tbn
    Metadata:
      encoder         : Lavc60.3.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=    1 fps=0.0 q=29.0 Lsize=      41kB time=00:00:00.00 bitrate=N/A speed=   0x    eed=N/A    
video:40kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.678311%
[libx264 @ 0x132e05570] frame I:1     Avg QP:25.22  size: 40589
[libx264 @ 0x132e05570] mb I  I16..4: 37.7% 33.4% 28.9%
[libx264 @ 0x132e05570] 8x8 transform intra:33.4%
[libx264 @ 0x132e05570] coded y,uvDC,uvAC intra: 51.1% 53.2% 14.4%
[libx264 @ 0x132e05570] i16 v,h,dc,p: 32% 38% 20% 10%
[libx264 @ 0x132e05570] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 36% 28%  3%  2%  2%  3%  3%  6%
[libx264 @ 0x132e05570] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 37% 17%  4%  4%  4%  5%  4%  7%
[libx264 @ 0x132e05570] i8c dc,h,v,p: 46% 37% 12%  4%
[libx264 @ 0x132e05570] kb/s:9741.36


  


  That's all. Exit code 0.