Recherche avancée

Sur d’autres sites (10988)

• What is the least CPU-intensive format to pass high resolution frames from ffmpeg to openCV ? [closed]

  3 octobre 2024, par Doctico

  I'm developing an application to process a high-resolution (2560x1440) RTSP stream from an IP camera using OpenCV.

  


  What I've Tried

  


  


  1. OpenCV's VideoCapture :

     


     

     • Performance was poor, even with CAP_PROP_FFMPEG.

     


     


  


  2. FFmpeg with MJPEG :

     


     

     • Decoded the stream as MJPEG and created OpenCV Mats from the image2pipe JPEG buffer.

     


     • Resulted in lower CPU usage for OpenCV but higher for FFmpeg.

     


     


  


  3. Current Approach :

     


     

     • Output raw video in YUV420p format from FFmpeg.

     


     • Construct OpenCV Mats from each frame buffer.

     


     • Achieves low FFmpeg CPU usage and moderately high OpenCV CPU usage.

     


     


  


  


  Current Implementation

  


  import subprocess
import cv2
import numpy as np

def stream_rtsp(rtsp_url):
    # FFmpeg command to stream RTSP and output to pipe
    ffmpeg_command = [
        'ffmpeg',
        '-hwaccel', 'auto',
        '-i', rtsp_url,
        '-pix_fmt', 'yuv420p',  # Use YUV420p format
        '-vcodec', 'rawvideo',
        '-an',  # Disable audio
        '-sn',  # Disable subtitles
        '-f', 'rawvideo',
        '-'  # Output to pipe
    ]

    # Start FFmpeg process
    process = subprocess.Popen(ffmpeg_command, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)

    # Frame dimensions
    width, height = 2560, 1440
    frame_size = width * height * 3 // 2  # YUV420p uses 1.5 bytes per pixel

    while True:
        # Read raw video frame from FFmpeg output
        raw_frame = process.stdout.read(frame_size)
        if not raw_frame:
            break

        yuv = np.frombuffer(raw_frame, np.uint8).reshape((height * 3 // 2, width))
        frame = cv2.cvtColor(yuv, cv2.COLOR_YUV2BGR_I420)
        
        processFrame(frame)

    # Clean up
    process.terminate()
    cv2.destroyAllWindows()


  


  Question

  


  Are there any other ways to improve performance when processing high-resolution frames from an RTSP stream ?

  


• Overlay a video and an image over a background video and shift that background video's position to the right

  4 août 2023, par sybr

  I'm currently working on a way to improve my production process for the videos that I'm making. I usually edit videos using two folders full of clips. Being able to automate putting these together in Premiere would save me a lot of time.

  


  I'm using the FFMpegCORE C# library, as well as Xabe.FFMpeg to achieve what I'm trying to do, and I've currently reached the point of creating two separate videos using the clips I mentioned earlier.

  


  The final step that I need to solve is to overlay the overlay video (overlay.mp4) over the background video (background.mp4), add an overlay image to add a clean divider (overlay.png) between the edges of the videos, and to somehow shift the position of the background video 33% to the right (background.mp4).

  


  This is the script I've come up with so far :

  


  ffmpeg -i overlay.mp4 -i background.mp4 -i overlay.png -filter_complex \ 
"[1:v]scale=608:1080[a]; [0:v]pad=1920:1080[b]; [b][a]overlay[out]; \
[out][2:v]overlay[out]; [0][1]amix[a];" \
-map [out] -map [a] -c:v libx264 -crf 30 -pix_fmt yuv420p -y output.mp4


  


  Is there a way to shift the background video to the right ? Or should I manually edit the video files ?

  


  Would love some help here, also eager to learn more about FFmpeg, so some explanation would be highly appreciated !

  


  Edit :

  


  Just found the solution, padding the background video for 33% and then cropping the final out back to 1920x1080 seemed to do the trick !

  


  ffmpeg -i overlay.mp4 -i background.mp4 -i overlay.png -filter_complex "[1:v]scale=608:1080[a]; [0:v]pad=2560:0:x=1920:y=1080[b]; [b][a]overlay[out]; [out][2:v]overlay[out]; [out]crop=1920:1080:0:0[out]; [0][1]amix[a];" -map [out] -map [a] -c:v libx264 -crf 30 -pix_fmt yuv420p -y output.mp4


  


• FFmpeg c api create encoder for AV_CODEC_ID_H264 crash on Windows

  30 avril 2023, par Guanyuming He

  I'm using ffmpeg (version 5.1.2) to clip a mp4 video per frame so I need to decode and encode it. However, when I'm creating the encoder for its video stream, the program always crashes at the call to avio_open2(), after H264 gives this error message :

  


  [h264_mf @ 0000025C1EBC1900] could not set output type (80004005)


  


  

  


  The configuration (time_base, pix_fmt, width, height) of the codec context of the encoder is copied from the decoder, and I checked if the pixel format is supported by finding if it's in codec->pix_fmts.

  


  I find that the problem does not involve all my other pieces of code, because this minimal program can duplicate the same problem :

  


  extern "C"&#xA;{&#xA;#include <libavcodec></libavcodec>avcodec.h>&#xA;}&#xA;&#xA;int main()&#xA;{&#xA;    auto codec = avcodec_find_encoder(AV_CODEC_ID_H264);&#xA;    auto codec_ctx = avcodec_alloc_context3(codec);&#xA;&#xA;    codec_ctx->pix_fmt = AV_PIX_FMT_YUV420P;&#xA;    codec_ctx->width = 2560;&#xA;    codec_ctx->height = 1440;&#xA;    codec_ctx->time_base.num = 1; codec_ctx->time_base.den = 180000;&#xA;&#xA;    avcodec_open2(codec_ctx, codec, NULL);&#xA;&#xA;    return 0;&#xA;}&#xA;

  


  Then I suspect if it's a bug of ffmpeg. My environment is Windows 11 64-bit, Visual Studio 2022. The ffmpeg library is obtained from vcpkg, as shown in the following image :