Recherche avancée

Sur d’autres sites (7905)

• FFmpeg dnn_processing with tensorflow backend : difficulties applying a filter on an image

  9 avril 2023, par ArnoBen

  I am trying to perform a video segmentation for background blurring similar to Google Meet or Zoom using FFmpeg, and I'm not very familiar with it.

  


  Google's MediaPipe model is available as a tensorflow .pb file here (using download_hhxwww.sh).

  


  I can load it in python and it works as expected, though I do need to format the input frames : scaling to the model input dimension, adding a batch dimension, dividing the pixel values by 255 to have a range 0-1.

  


  FFmpeg has a filter that can use tensorflow models thanks to dnn_processing, but I'm wondering about these preprocessing steps. I tried to read the dnn_backend_tf.c file in ffmpeg's github repo, but C is not my forte. I'm guessing it adds a batch dimension somewhere otherwise the model wouldn't run, but I'm not sure about the rest.

  


  Here is my current command :

  


  ffmpeg \
    -i $FILE -filter_complex \
    "[0:v]scale=160:96,format=rgb24,dnn_processing=dnn_backend=tensorflow:model=$MODEL:input=input_1:output=segment[masks];[masks]extractplanes=2[mask]" \
    -map "[mask]" output.png


  


  

  • I'm already applying a scaling to match the input dimension.

  


  • I wrote this [masks]extractplanes=2[mask] because the model outputs a HxWx2 tensor (background mask and foreground mask) and I want to keep the foreground mask.

  


  


  The result I get with this command is the following (input-output) :

  


  

  


  I'm not sure how to interpret the problems in this output. In python I can easily get a nice grayscale output :

  


  

  


  I'm trying to obtain something similar with FFmpeg.

  


  Any suggestion or insights to obtain a correct output with FFmpeg would be greatly appreciated.

  


  PS : If I try to apply this on a video file, it hits a Segmentation Fault somewhere before getting any output so I stick with testing on an image for now.

  


• How do I access the ipod-library for decoding ffmpeg ?

  16 janvier 2014, par M_On

  Thank you to look at my question.

  There is a problem with too slow to access the music file using ffmpeg.

  Get the URL of the music that is in the ipod-library by using the 'valueForProperty:MPMediaItemPropertyAssetURL' method.

  Use the (AVAssetExportSession Class) TSLibraryImport.m, extract the music files from ipod-library.

  However, I took the extraction time is too long. (4 seconds of the 3GS iPhone, iPhone 5 is 1 second)

  It was not possible to be decoded using FFmpeg If you do not the Export.

  In order to use FFmpeg, it's AVPlayer can not be used.

  I will never forget the grace if you can answer if you folded to know how other maybe.

• How do I access the ipod-library for decoding ffmpeg in the xcode ?

  16 janvier 2014, par M_On

  Thank you to look at my question.

  There is a problem with too slow to access the music file using ffmpeg.

  Get the URL of the music that is in the ipod-library by using the 'valueForProperty:MPMediaItemPropertyAssetURL' method.

  Use the (AVAssetExportSession Class) TSLibraryImport.m, extract the music files from ipod-library.

  However, I took the extraction time is too long. (4 seconds of the 3GS iPhone, iPhone 5 is 1 second)

  It was not possible to be decoded using FFmpeg If you do not the Export.

  In order to use FFmpeg, it's AVPlayer can not be used.

  I will never forget the grace if you can answer if you folded to know how other maybe.