Recherche avancée

Sur d’autres sites (7329)

• Splitting my code into OpenCV and UDP and the differences between using OpenCV and UDP

  18 avril 2024, par Sagiv Shaniv

  1) I wrote a Python code to receive video in real-time, compress it, duplicate the video, and then send it to OpenCV and UDP using ffmpeg. I would like to know how I can duplicate the code to send it to both UDP and OpenCV (without sending it to another device) without affecting the frame rate.

  


  This is the code I used so far :

  


  import subprocess
import cv2

# Start ffmpeg process to capture video from USB, encode it in H.264, and send it over UDP and to virtual video device
ffmpeg_cmd = [
    'ffmpeg',
    '-f', 'v4l2',            # Input format for USB camera
    '-video_size', '1920x1080', # Video size
    '-i', '/dev/video2',     # USB device
    '-c:v', 'libx264',       # H.264 codec
    '-preset', 'ultrafast',  # Preset for speed
    '-tune', 'zerolatency',  # Tune for zero latency
    '-b:v', '2M',            # Bitrate
    '-bufsize', '5M',        # Buffer size
    '-pix_fmt', 'yuv420p',   # Specify pixel format
    '-filter_complex', '[0:v]split=2[out1][out2]',  # Split the video stream
    '-map', '[out1]',        # Map the first output to UDP
    '-f', 'mpegts',          # Output format for UDP
    'udp://192.168.1.100:8000',  # UDP destination
    '-map', '[out2]',        # Map the second output to virtual video device
    '-f', 'v4l2',            # Output format for virtual video device
    '-video_size', '1920x1080', # Video size for virtual video device
    '-pix_fmt', 'yuv420p',   # Specify pixel format for virtual video device
    '/dev/video1'            # Virtual video device
]

ffmpeg_process = subprocess.Popen(ffmpeg_cmd)

v4l2_cap = cv2.VideoCapture(1)

while True:
    ret, frame = v4l2_cap.read()  # Read frame from virtual video device
    if not ret:
        break
    cv2.imshow('Frame', frame)  # Display frame
    if cv2.waitKey(1) & 0xFF == ord('q'):  # Exit on 'q' key press
        break

# Clean up
cv2.destroyAllWindows()
ffmpeg_process.terminate()


  


  2) When I get the video straight from the device and send it over UDP I get 25 FPS with this code :

  


  import cv2
import subprocess
import time
import os

width = 1920
height  = 1080
fps = 40  # Increase FPS to 60
proc = None
os.environ['LD_LIBRARY_PATH'] = '/opt/vc/lib'  # Set the library path

def stream_video():
    command = [
    'ffmpeg',
    '-f', 'v4l2',
    '-input_format', 'mjpeg',
    '-video_size', '1920x1080',
    '-framerate', '30',
    '-thread_queue_size', '512',
    '-i', '/dev/video2',
    '-f', 'lavfi',
    '-i', 'sine=frequency=440:sample_rate=48000',
    '-pix_fmt', 'yuvj420p',
    '-c:v', 'libx264',
    '-c:a', 'aac',
    '-b:v', '5000k',
    '-b:a', '128k',
    '-profile:v', 'baseline',
    '-preset', 'ultrafast',
     '-x264-params', 'tune=zerolatency',
    '-g', '60',
    '-f', 'mpegts',
    'udp://192.168.1.100:8000'
]


    try:
        proc = subprocess.Popen(command, stdin=subprocess.PIPE)
    finally:
        proc.stdin.close()
        proc.wait()

if _name_ == '_main_':
    stream_video()


  


  However when I get the video from openCV I get 9 FPS with this code :

  


  import subprocess
import cv2
import numpy as np

width = 1920
height = 1080
fps = 30

cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
cap.set(cv2.CAP_PROP_FPS, fps)

command = [
    'ffmpeg',
   
    '-f', 'rawvideo',
    '-s', f'{width}x{height}',
    '-r', str(fps),
    '-i', '-',  # Read from stdin
    '-f', 'lavfi',
    '-i', 'sine=frequency=440:sample_rate=48000',
    '-pix_fmt', 'yuv420p',
    '-c:v', 'libx264',
    '-c:a', 'aac',
    '-b:v', '5M',
    '-b:a', '128k',
    '-profile:v', 'baseline',
    '-preset', 'ultrafast',
    '-x264-params', 'tune=zerolatency',
    '-g', '60',
    '-f', 'mpegts',
    'udp://192.168.1.100:8000'
]

try:
    proc = subprocess.Popen(command, stdin=subprocess.PIPE)
    while True:
        ret, frame = cap.read()
        if not ret:
            break
        proc.stdin.write(frame.tobytes())
finally:
    cap.release()
    proc.stdin.close()
    proc.wait()


  


  How can I receive video from OpenCV without affecting the frame rate (as I need it for later image processing) and without compromising on quality or resolution ?

  


  I have tried capturing video frames using OpenCV and sending them over UDP using ffmpeg. I expected to maintain the original frame rate of the video without compromising on quality or resolution. However, I noticed a significant drop in frame rate when using OpenCV compared to directly capturing from the device. Specifically, I achieved 25 FPS when capturing and sending directly from the device using ffmpeg, but only 9 FPS when capturing frames using OpenCV and then sending them over UDP using ffmpeg.

  


  Thank you

  


• Revert "swscale : disable ARM code until its build failure with clang/iphone is fixed"

  12 janvier 2014, par Michael Niedermayer

  Revert "swscale : disable ARM code until its build failure with clang/iphone is fixed"
  This reverts commit c8c7736c1025bcf5bb27e104a0d0eae749408739.
  

  • [DH] libswscale/arm/Makefile
  • [DH] libswscale/swscale_unscaled.c

• Iphone Streaming and playing Audio Problem

  20 août 2011, par KayKay

  I am trying to make an app that plays audio stream using ffmpeg, libmms.
  I can open mms server, get stream, and decode audio frame to raw frame using suitable codec.
  However I don't know how to do next.
  I think I must use AudioToolbox/AudioToolbox.h and make audioqueue.
  but however when I give audioqueuebuffer decode buffer's memory and play, Only plays the white noise.
  Here is my code.

  What am i missing ?
  Any comment and hint is very appreciated.
  Thanks very much.

  while(av_read_frame(pFormatCtx, &pkt)>=0)
  
  {
  
      int pkt_decoded_len = 0;
  
      int frame_decoded_len;
  
      int decode_buff_remain=AVCODEC_MAX_AUDIO_FRAME_SIZE * 5;
  
      if(pkt.stream_index==audiostream)
  
      {
  
          frame_decoded_len=decode_buff_remain;
  
          int16_t *decode_buff_ptr = decode_buffer;
  
          int decoded_tot_len=0;
  
          pkt_decoded_len = avcodec_decode_audio2(pCodecCtx, decode_buff_ptr, &frame_decoded_len,
  
                                                  pkt.data, pkt.size);
  
          if (pkt_decoded_len <0) break;
  
          AudioQueueAllocateBuffer(audioQueue, kBufferSize, &buffers[i]);
  
          AQOutputCallback(self, audioQueue, buffers[i], pkt_decoded_len);
  
  
  
          if(i == 1){
  
              AudioQueueSetParameter(audioQueue, kAudioQueueParam_Volume, 1.0);
  
              AudioQueueStart(audioQueue, NULL);
  
          }
  
          i++;
  
      }
  
  }
  
  
  
  
  
  void AQOutputCallback(void *inData, AudioQueueRef inAQ, AudioQueueBufferRef inBuffer, int copySize)
  
  {
  
      mmsDemoViewController *staticApp = (mmsDemoViewController *)inData;
  
      [staticApp handleBufferCompleteForQueue:inAQ buffer:inBuffer size:copySize];
  
  }
  
  
  
  - (void)handleBufferCompleteForQueue:(AudioQueueRef)inAQ
  
                            buffer:(AudioQueueBufferRef)inBuffer
  
                              size:(int)copySize
  
  {
  
      inBuffer->mAudioDataByteSize = inBuffer->mAudioDataBytesCapacity;
  
      memcpy((char*)inBuffer->mAudioData, (const char*)decode_buffer, copySize);
  
  
  
      AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);
  
  }