Recherche avancée

Sur d’autres sites (7488)

• Detect bad frames in OpenCV 2.4.9

  14 mai 2014, par user3630380

  I know the title is a bit vague but I’m not sure how else to describe it.

  CentOS with ffmpeg + OpenCV 2.4.9. I’m working on a simple motion detection system which uses a stream from an IP camera (h264).

  Once in a while the stream hiccups and throws in a "bad frame" (see pic-bad.png link below). The problem is, these frames vary largely from the previous frames and causes a "motion" event to get triggered even though no actual motion occured.

  The pictures below will explain the problem.

  Good frame (motion captured) :

  

  Bad frame (no motion, just a broken frame) :

  

  The bad frame gets caught randomly. I guess I can make a bad frame detector by analyzing (looping) through the pixels going down from a certain position to see if they are all the same, but I’m wondering if there is any other, more efficient, "by the book" approach to detecting these types of bad frames and just skipping over them.

  Thank You !

  EDIT UPDATE :

  The frame is grabbed using a C++ motion detection program via cvQueryFrame(camera); so I do not directly interface with ffmpeg, OpenCV does it on the backend. I’m using the latest version of ffmpeg compiled from git source. All of the libraries are also up to date (h264, etc, all downloaded and compiled yesterday). The data is coming from an RTSP stream (ffserver). I’ve tested over multiple cameras (dahua 1 - 3 MP models) and the frame glitch is pretty persistent across all of them, although it doesn’t happen continuously, just once on a while (ex : once every 10 minutes).

• ffmpeg complex filtering : how to get around

  19 janvier 2020, par Melvin Roest

  Alright, I have my own compiled ffmpeg with --enable-lv2. This allows for 3rd-party plugins to work. The plugin I use is : https://github.com/lucianodato/speech-denoiser it’s a plugin that wraps around this RNN noise reduction library : https://github.com/GregorR/rnnoise-models

  The following commands work :

  (1) ffmpeg -i input.mov -filter_complex '[0:a]lv2=plugin=https\\://github.com/lucianodato/speech-denoiser[audio]' -map "[audio]" output.wav

  (2) ffmpeg -i input.mov -filter_complex '[0:v]copy[video]' -map "[video]" output.mov

  But when I do the combination, that doesn’t work.

  ffmpeg -i input.mov -filter_complex '[0:a]lv2=plugin=https\\://github.com/lucianodato/speech-denoiser[audio];[0:v]copy[video]' -map "[audio]" -map "[video]" output.mov

  I think the error is essentially this :

  Channel layout change is not supported
  
  Error while filtering: Not yet implemented in FFmpeg, patches welcome
  
  Failed to inject frame into filter network: Not yet implemented in FFmpeg, patches welcome
  
  Error while processing the decoded data for stream #0:0

  My guess : this 3rd-party filter is not configure to work with any other output stream other than audio.

  My question : can I somehow trick this 3rd-party plugin that it is outputting to an audio file, while still outputting everything to a video file ?

  Note : I know, I can simply split this up in 2 commands and be done with it, so I’m wondering if I can accomplish this via one ffmpeg command. How I would split it up in 2 commands is as follows :

  ffmpeg -i out_cropped.mov -af 'lv2=plugin=https\\://github.com/lucianodato/speech-denoiser' -vcodec copy out_cropped_denoised.wav 
  
  
  
  &&
  
  
  
  ffmpeg -i out_cropped.mov -i out_cropped_denoised.wav -c:v copy -map 0:v:0 -map 1:a:0 out_cropped_denoised.mov

  But I want to be able to put it all in one complex filter (ideally) or at least in one ffmpeg command.

  Appendix : here is the full interaction

  ffmpeg -i input.mov -filter_complex '[0:a]lv2=plugin=https\\://github.com/lucianodato/speech-denoiser[audio];[0:v]copy[video]' -map "[audio]" -map "[video]" output.mov
  
  ffmpeg version N-95577-g68f623d644 Copyright (c) 2000-2019 the FFmpeg developers
  
    built with Apple clang version 11.0.0 (clang-1100.0.33.8)
  
    configuration: --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libopus --enable-libxvid --enable-lv2 --samples=fate-suite/
  
    libavutil      56. 35.101 / 56. 35.101
  
    libavcodec     58. 60.100 / 58. 60.100
  
    libavformat    58. 33.100 / 58. 33.100
  
    libavdevice    58.  9.100 / 58.  9.100
  
    libavfilter     7. 65.100 /  7. 65.100
  
    libswscale      5.  6.100 /  5.  6.100
  
    libswresample   3.  6.100 /  3.  6.100
  
    libpostproc    55.  6.100 / 55.  6.100
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mov':
  
    Metadata:
  
      major_brand     : qt
  
      minor_version   : 512
  
      compatible_brands: qt
  
      encoder         : Lavf58.29.100
  
    Duration: 00:16:19.11, start: 0.000000, bitrate: 1341 kb/s
  
      Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1080x960, 1262 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
  
      Metadata:
  
        handler_name    : Core Media Video
  
        encoder         : Lavc58.54.100 libx264
  
      Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
  
      Metadata:
  
        handler_name    : Core Media Audio
  
  File 'output.mov' already exists. Overwrite? [y/N] y

  #ote : I typed yes and then this came.

  Stream mapping:
  
    Stream #0:0 (h264) -> copy
  
    Stream #0:1 (aac) -> lv2
  
    lv2 -> Stream #0:0 (aac)
  
    copy -> Stream #0:1 (libx264)
  
  Press [q] to stop, [?] for help
  
  [out_0_0 @ 0x7fa6811066c0] Channel layout change is not supported
  
  Error while filtering: Not yet implemented in FFmpeg, patches welcome
  
  Failed to inject frame into filter network: Not yet implemented in FFmpeg, patches welcome
  
  Error while processing the decoded data for stream #0:0

• Fastest way to extract moving dynamic crop from video using ffmpeg

  5 avril 2024, par NateW

  I'm working on an AI project that involves object detection and action recognition on roughly 30 minute videos.

  


  My pipeline is the following :

  


  

  • determine crops using object detection model

  


  • extract crops using Python and write them to disk as individual images for each frame.

  


  • use action recognition model by inputting a series of the crops.

  


  


  The models are fast but actual writing of the crops to disk is slow. Sure, using an SSD would speed it up but I'm sure ffmpeg would greatly speed it up.

  


  Some of the challenges with the crops :

  


  

  • the output size is always 128x128

  


  • the input size is variable

  


  • the crop moves on every frame

  


  


  My process for extracting crops is simple using cv2.imwrite(output_crop_path, crop) in a for loop.

  


  I've done experiments trying to use sndcommand and filter_complex. I tried this https://stackoverflow.com/a/67508233/4447761 but it outputs an image with black below the crop and the image gets wrapped around on the x axis.