Recherche avancée

Sur d’autres sites (5193)

• FFmpeg dnn_processing with tensorflow backend : difficulties applying a filter on an image

  9 avril 2023, par ArnoBen

  I am trying to perform a video segmentation for background blurring similar to Google Meet or Zoom using FFmpeg, and I'm not very familiar with it.

  


  Google's MediaPipe model is available as a tensorflow .pb file here (using download_hhxwww.sh).

  


  I can load it in python and it works as expected, though I do need to format the input frames : scaling to the model input dimension, adding a batch dimension, dividing the pixel values by 255 to have a range 0-1.

  


  FFmpeg has a filter that can use tensorflow models thanks to dnn_processing, but I'm wondering about these preprocessing steps. I tried to read the dnn_backend_tf.c file in ffmpeg's github repo, but C is not my forte. I'm guessing it adds a batch dimension somewhere otherwise the model wouldn't run, but I'm not sure about the rest.

  


  Here is my current command :

  


  ffmpeg \
    -i $FILE -filter_complex \
    "[0:v]scale=160:96,format=rgb24,dnn_processing=dnn_backend=tensorflow:model=$MODEL:input=input_1:output=segment[masks];[masks]extractplanes=2[mask]" \
    -map "[mask]" output.png


  


  

  • I'm already applying a scaling to match the input dimension.

  


  • I wrote this [masks]extractplanes=2[mask] because the model outputs a HxWx2 tensor (background mask and foreground mask) and I want to keep the foreground mask.

  


  


  The result I get with this command is the following (input-output) :

  


  

  


  I'm not sure how to interpret the problems in this output. In python I can easily get a nice grayscale output :

  


  

  


  I'm trying to obtain something similar with FFmpeg.

  


  Any suggestion or insights to obtain a correct output with FFmpeg would be greatly appreciated.

  


  PS : If I try to apply this on a video file, it hits a Segmentation Fault somewhere before getting any output so I stick with testing on an image for now.

  


• how to use ffmplay in android

  8 mai 2016, par Thanh Dat Mai

  I want to show a video with filter. Use this command on pc

  ffplay input.mp4 -vf scale=320:240,curves=psfile=cur.acv

  I was able to show videos. But I do not do this on android, Who can tell me any direction yet or library support for my work. thank you

• OpenCV returns an Empty Frame on video.read

  20 mars 2019, par Ikechukwu Anude

  Below is the relevant code

  import cv2 as cv
  
  import numpy as np
  
  
  
  video = cv.VideoCapture(0) #tells obj to use built in camera\
  
  
  
  #create a face cascade object 
  
  face_cascade  = 
  
  cv.CascadeClassifier(r"C:\Users\xxxxxxx\AppData\Roaming\Python\Python36\site- 
  
  packages\cv2\data\haarcascade_frontalcatface.xml")
  
  
  
  a = 1
  
  #create loop to display a video
  
  while True:
  
      a = a + 1
  
      check, frame = video.read()
  
      print(frame)
  
  
  
      #converts to a gray scale img
  
      gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
  
  
  
      #create the faces
  
      faces = face_cascade.detectMultiScale(gray, scaleFactor=1.5, minNeighbors=5)
  
  
  
      for(x, y, w, h) in faces:
  
          print(x, y, w, h)
  
  
  
      #show the image
  
      cv.imshow('capturing', gray)
  
  
  
      key = cv.waitKey(1) #gen a new frame every 1ms
  
  
  
      if key == ord('q'): #once you enter 'q' the loop will be exited
  
          break
  
  
  
  print(a) #this will print the number of frames
  
  
  
  #captures the first frame
  
  video.release() #end the web cam
  
  
  
  #destroys the windows when you are not defined
  
  cv.destroyAllWindows()

  The code displays a video captured from my webcam camera. Despite that, OpevCV doesn’t seem to be processing any frames as all the frames look like this

   [[0 0 0]
  
    [0 0 0]
  
    [0 0 0]
  
    ...
  
    [0 0 0]
  
    [0 0 0]
  
    [0 0 0]]]

  which I assume means that they are empty.

  This I believe is preventing the algorithm from being able to detect my face in the frame. I have a feeling that the issue lies in the ffmpeg codec, but I’m not entirely sure how to proceed even if that is the case.

  OS : Windows 10
  Language : Python

  Why is the frame empty and how can I get OpenCV to detect my face in the frame ?