Recherche avancée

Sur d’autres sites (1713)

• Efficiently Fetching the Latest Frame from a Live Stream using OpenCV in Python

  10 juillet 2023, par Nicolantonio De Bari

  Problem :

  


  I have a FastAPI server that connects to a live video feed using cv2.VideoCapture. The server then uses OpenCV to process the frames for object detection and sends the results to a WebSocket. Here's the relevant part of my code :

  


  class VideoProcessor:
    # ...

    def update_frame(self, url):
        logger.info("update_frame STARTED")
        cap = cv2.VideoCapture(url)
        while self.capture_flag:
            ret, frame = cap.read()
            if ret:
                frame = cv2.resize(frame, (1280, 720))
                self.current_frame = frame
            else:
                logger.warning("Failed to read frame, retrying connection...")
                cap.release()
                time.sleep(1)
                cap = cv2.VideoCapture(url)

    async def start_model(self, url, websocket: WebSocket):
        # ...
        threading.Thread(target=self.update_frame, args=(url,), daemon=True).start()
        while self.capture_flag:
            if self.current_frame is not None:
                frame = cv2.resize(self.current_frame, (1280, 720))
                bbx = await self.process_image(frame)
                await websocket.send_text(json.dumps(bbx))
                await asyncio.sleep(0.1)


  


  Currently, I'm using a separate thread (update_frame) to continuously fetch frames from the live feed and keep the most recent one in self.current_frame.

  


  The issue with this method is that it uses multi-threading and continuously reads frames in the background, which is quite CPU-intensive. The cv2.VideoCapture.read() function fetches the oldest frame in the buffer, and OpenCV does not provide a direct way to fetch the latest frame.

  


  Goal

  


  I want to optimize this process by eliminating the need for a separate thread and fetching the latest frame directly when I need it. I want to ensure that each time I process a frame in start_model, I'm processing the most recent frame from the live feed.

  


  I have considered a method of continuously calling cap.read() in a tight loop to "clear" the buffer, but I understand that this is inefficient and can lead to high CPU usage.

  


  Attempt :

  


  What I then tried to do is use ffmpeg & subprocess to get the latest frame, but I dont seem to understand how to get the latest frame then.

  


  async def start_model(self, url, websocket: WebSocket):
    try:
        logger.info("Model Started")
        self.capture_flag = True

        command = ["ffmpeg", "-i", url, "-f", "image2pipe", "-pix_fmt", "bgr24", "-vcodec", "rawvideo", "-"]
        pipe = subprocess.Popen(command, stdout=subprocess.PIPE, bufsize=10**8)

        while self.capture_flag:
            raw_image = pipe.stdout.read(1280*720*3)  # read 720p frame (BGR)
            if raw_image == b'':
                logger.warning("Failed to read frame, retrying connection...")
                await asyncio.sleep(1) # delay for 1 second before retrying
                continue
            
            frame = np.fromstring(raw_image, dtype='uint8')
            frame = frame.reshape((720, 1280, 3))
            if frame is not None:
                self.current_frame = frame
                frame = cv2.resize(self.current_frame, (1280, 720))
                bbx = await self.process_image(frame)
                await websocket.send_text(json.dumps(bbx))
                await asyncio.sleep(0.1)
                
        pipe.terminate()

    except WebSocketDisconnect:
        logger.info("WebSocket disconnected during model operation")
        self.capture_flag = False  # Ensure to stop the model operation when the WebSocket disconnects


  


  Question

  


  Is there a more efficient way to fetch the latest frame from a live stream using OpenCV in Python ? Can I modify my current setup to get the newest frame without having to read all the frames in a separate thread ?
Or is there another library that I could use ?

  


  I know a similar question has been asked, but not related to video streaming.

  


• Cannot stream video from VLC docker

  4 avril 2023, par Snake Eyes

  I have Dockerfile :

  


  FROM fedora:34

ARG VLC_UID="1000"
ARG VLC_GID="1000"

ENV HOME="/data"


RUN groupadd -g "${VLC_GID}" vlc && \
    useradd -m -d /data -s /bin/sh -u "${VLC_UID}" -g "${VLC_GID}" vlc && \
    dnf upgrade -y && \
    rpm -ivh "https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-34.noarch.rpm" && \
    dnf upgrade -y && \
    dnf install -y vlc && \
    dnf install -y libaacs libbdplus && \
    dnf install -y libbluray-bdj && \
    dnf clean all

USER "vlc"

WORKDIR "/data"

VOLUME ["/data"]

ENTRYPOINT ["/usr/bin/cvlc"]


  


  And then run :

  


  docker run -d -v "d:\path":/data -p 8787:8787 myrepo/myvlc:v1 file:///data/Sample.mkv --sout '#transcode {vcodec=h264,acodec=mp3,samplerate=44100}:std{access=http,mux=ffmpeg{mux=flv},dst=0.0.0.0:8787/stream.flv}'


  


  I get error :

  


  2023-04-04 12:19:11 [000055933c090060] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
2023-04-04 12:19:11 [000055933c09d680] dbus interface error: Failed to connect to the D-Bus session daemon: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
2023-04-04 12:19:11 [000055933c09d680] main interface error: no suitable interface module
2023-04-04 12:19:11 [000055933bf26ad0] main libvlc error: interface "dbus,none" initialization failed
2023-04-04 12:19:11 [000055933c08d7a0] main interface error: no suitable interface module
2023-04-04 12:19:11 [000055933bf26ad0] main libvlc error: interface "globalhotkeys,none" initialization failed
2023-04-04 12:19:11 [000055933c08d7a0] dummy interface: using the dummy interface module...
2023-04-04 12:19:11 [00007f3b84001250] stream_out_standard stream out error: no mux specified or found by extension
2023-04-04 12:19:11 [00007f3b84000f30] main stream output error: stream chain failed for `standard{mux="",access="",dst="'#transcode"}'
2023-04-04 12:19:11 [00007f3b90000c80] main input error: cannot start stream output instance, aborting
2023-04-04 12:19:11 [00007f3b7c001990] stream_out_standard stream out error: no mux specified or found by extension
2023-04-04 12:19:11 [00007f3b7c001690] main stream output error: stream chain failed for `standard{mux="",access="",dst="'#transcode"}'
2023-04-04 12:19:11 [00007f3b90000c80] main input error: cannot start stream output instance, aborting


  


  I mention that I'm using cvlc and I can't stream that mkv file.

  


  I tried as well --sout '#transcode{scodec=none}:http{mux=ffmpeg{mux=flv},dst=:8787/}' but same errors.

  


  How can I solve it ?

  


• OpenCV is able to read the stream but VLC not

  25 avril 2023, par Ahmet Çavdar

  I'm trying to stream my webcam frames to an UDP address. Here is my sender code.

  


  cmd = ['ffmpeg', '-y', '-f', 'rawvideo', '-pixel_format', 'bgr24', '-video_size', f'{width}x{height}', 
       '-i', '-', '-c:v', 'mpeg4','-preset', 'ultrafast', '-tune', 'zerolatency','-b:v', '1.5M',
       '-f', 'mpegts', f'udp://@{ip_address}:{port}']
p = subprocess.Popen(cmd, stdin=subprocess.PIPE)
camera = cv2.VideoCapture(0)
while True:
    ret, frame = camera.read()
    cv2.imshow("Sender",frame)
    if not ret:
        break
    p.stdin.write(frame.tobytes())
    p.stdin.flush()
    if cv2.waitKey(1) & 0xFF == ord('q'):
            break


  


  This Python code can make stream successfully. I can read the stream with this receiver code.

  


  q = queue.Queue()
def receive():
    cap = cv2.VideoCapture('udp://@xxx.x.xxx.xxx:5000')
    ret, frame = cap.read()
    q.put(frame)
    while ret:
        ret, frame = cap.read()
        q.put(frame)
def display():
    while True:
        if q.empty() != True:
            frame = q.get()
            cv2.imshow('Receiver', frame)
        k = cv2.waitKey(1) & 0xff
        if k == 27:  # press 'ESC' to quit
            break
tr = threading.Thread(target=receive, daemon=True)
td = threading.Thread(target=display)
tr.start()
td.start()
td.join()


  


  But I can not watch the stream from VLC. I'm going to Media->Open Network Stream->
udp ://@xxx.x.xxx.xxx:5000 to watch stream. After some seconds, the timer that located bottom left of VLC starts to increase but there are no frames in screen, just VLC icon.

  


  I checked firewall rules, opened all ports to UDP connections. I am using my IP address to send frames and watch them.
Also, I tried other video codecs like h264, hvec, mpeg4, rawvideo.
Additionally, I tried to watch stream by using Windows Media Player but it didn't work.

  


  What should I do to fix this issue ?