Recherche avancée

Sur d’autres sites (12016)

• avfilter/vf_dnn_processing : add a generic filter for image proccessing with dnn networks

  31 octobre 2019, par Guo, Yejun

  avfilter/vf_dnn_processing : add a generic filter for image proccessing with dnn networks
  This filter accepts all the dnn networks which do image processing.
  
  Currently, frame with formats rgb24 and bgr24 are supported. Other
  
  formats such as gray and YUV will be supported next. The dnn network
  
  can accept data in float32 or uint8 format. And the dnn network can
  
  change frame size.
  The following is a python script to halve the value of the first
  
  channel of the pixel. It demos how to setup and execute dnn model
  
  with python+tensorflow. It also generates .pb file which will be
  
  used by ffmpeg.
  import tensorflow as tf
  
  import numpy as np
  
  import imageio
  
  in_img = imageio.imread('in.bmp')
  
  in_img = in_img.astype(np.float32)/255.0
  
  in_data = in_img[np.newaxis, :]
  
  filter_data = np.array([0.5, 0, 0, 0, 1., 0, 0, 0, 1.]).reshape(1,1,3,3).astype(np.float32)
  
  filter = tf.Variable(filter_data)
  
  x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
  
  y = tf.nn.conv2d(x, filter, strides=[1, 1, 1, 1], padding='VALID', name='dnn_out')
  
  sess=tf.Session()
  
  sess.run(tf.global_variables_initializer())
  
  output = sess.run(y, feed_dict=x : in_data)
  
  graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
  
  tf.train.write_graph(graph_def, '.', 'halve_first_channel.pb', as_text=False)
  
  output = output * 255.0
  
  output = output.astype(np.uint8)
  
  imageio.imsave("out.bmp", np.squeeze(output))
  To do the same thing with ffmpeg :
  
   generate halve_first_channel.pb with the above script
  
   generate halve_first_channel.model with tools/python/convert.py
  
   try with following commands
  
    ./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.model:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=native -y out.native.png
  
    ./ffmpeg -i input.jpg -vf dnn_processing=model=halve_first_channel.pb:input=dnn_in:output=dnn_out:fmt=rgb24:dnn_backend=tensorflow -y out.tf.png
  Signed-off-by : Guo, Yejun <yejun.guo@intel.com>
  
  Signed-off-by : Pedro Arthur <bygrandao@gmail.com>
  

  • [DH] configure
  • [DH] doc/filters.texi
  • [DH] libavfilter/Makefile
  • [DH] libavfilter/allfilters.c
  • [DH] libavfilter/vf_dnn_processing.c

• avcodec/v210dec : add support for frame and slice threading

  25 novembre 2019, par Limin Wang

  avcodec/v210dec : add support for frame and slice threading
  1, Test server configure :
  
  [root@localhost ]# cat /proc/cpuinfo  |grep "model name"
  
   model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
  
   model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
  
   ...
  [root@localhost ]# free -h
  
                total        used        free      shared  buff/cache   available
  
                Mem :           102G        1.1G        100G         16M        657M        100G
  
                Swap :          4.0G          0B        4.0G
  2, Test result :
  
  encode the v210 input data for testing :
  
  ./ffmpeg -y -i 4k_422.ts  -c:v v210 -vframes 10 test.avi
  master :
  
  ./ffmpeg -y -threads 1 -stream_loop 1000 -i ./test.avi -benchmark -f null -
  
  frame=10010 fps= 60 q=-0.0 Lsize=N/A time=00:38:26.30 bitrate=N/A speed=13.7x
  
  video:5240kB audio:432432kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead : unknown
  
  bench : utime=101.869s stime=66.181s rtime=167.996s
  
  bench : maxrss=186552kB
  patch applied :
  
  ./ffmpeg -y -threads 2 -thread_type slice -stream_loop 1000 -i ./test.avi -benchmark -f null -
  
  frame=10010 fps= 72 q=-0.0 Lsize=N/A time=00:38:26.30 bitrate=N/A speed=16.5x
  
  video:5240kB audio:432432kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead : unknown
  
  bench : utime=103.562s stime=74.858s rtime=139.599s
  
  bench : maxrss=188616kB
  ./ffmpeg -y -threads 2 -thread_type frame -stream_loop 1000 -i ./test.avi -benchmark -f null -
  
  frame=10010 fps= 85 q=-0.0 Lsize=N/A time=00:38:26.30 bitrate=N/A speed=19.6x
  
  video:5240kB audio:432432kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead : unknown
  
  bench : utime=114.310s stime=92.685s rtime=117.693s
  
  bench : maxrss=231896kB
  Signed-off-by : Limin Wang <lance.lmwang@gmail.com>
  
  Signed-off-by : Michael Niedermayer <michael@niedermayer.cc>
  

  • [DH] libavcodec/v210dec.c
  • [DH] libavcodec/v210dec.h

• AJAX upload file with progres + FFMPEG progress

  24 octobre 2019, par Сергей Барахтенко

  The task is to implement the following :
  1. On the page there is a form for downloading a file, the file should be loaded asynchronously, without rebooting with the help of AJAX and displaying the upload progress
  2. If the file is successfully downloaded, send a command to the server, which should check the queue, and if the queue is empty, inform the client that the processing of the file has begun, then start the processing itself
  3. If processing has begun, then show the progress of this processing

  What did you manage to implement
  File upload was successfully implemented, after a successful upload, the server issues a line with the status successfully, and the full path of the downloaded file

  Upload JS snippet

  $.ajax({
  
      url: 'api/upload/load',
  
      type: 'post',
  
      data: fd,
  
      contentType: false,
  
      processData: false,
  
      xhr: function() {
  
          var xhr = new window.XMLHttpRequest();
  
          xhr.upload.addEventListener("progress", function(evt) {
  
              if (evt.lengthComputable) {
  
                  var percent = Math.ceil((evt.loaded / evt.total) * 100);
  
                  console.log(percent) // Upload progress
  
              }
  
          }, false);
  
          return xhr;
  
      },
  
      success: function(data){
  
          var response = JSON.parse(data);
  
          if(response.status == 'success'){
  
              //if upload success, run command
  
              startConversion(response.path)
  
          }
  
      }
  
  });

  Reponse from api/upload/load

  {"status":"success","path":"<path>"}
  
  </path>

  In the AJAX script, I do a check : if the response.status == ’success’, then we call the startConversion function, which takes the full path of the newly loaded file as an argument. This function sends another AJAX request to the server, where the queue check is in progress.

  startConversion JS Snippet

  function startConversion(path){
  
      $.ajax({
  
          url: '/api/upload/conversion',
  
          type: 'POST',
  
          data: { path: path },
  
          success: function(data){
  
              var response = JSON.parse(data);
  
              // if response.status == 'start', run checkConversion function
  
          }
  
      })

  Code /api/upload/conversion

  $queue = $this->model->getQueue();
  
  if($queue < 5){
  
      // If queue is empty, notify browser
  
      $output = array('progress' => 'start');
  
      echo json_encode($output);
  
      // Then start shell_ecxec
  
      $cmd = 'some_command';
  
      shell_exec('ffmpeg.exe -i ' . $path . ' ' . $cmd . ' 1>log.txt 2>&1 ');
  
  } else {
  
      $output = array('progress' => 'waiting');
  
      echo json_encode($output);
  
  }

  Here, for some reason, a message is already not being displayed stating that processing has begun (echo before $cmd)
  
  Full link

  And here the most interesting begins. It would seem that it would be possible in the method to success call the check on the status of the processing (start, queue or an error) and if the processing began to call the third function, which, again AJAX, polls the server returns the progress of the processing

  But in reality, the following happens : as soon as the second function sends the request, nothing is returned in response, but processing is in progress, and in the console it shows the status of the request - PENDING. As soon as the server completes processing, a response is issued from the server that the queue is empty and you can start processing the progress, although the processing has already been completed

  There are suggestions that further execution of the script is blocked by the command shell_exec() until the end of her work
  Also, it is not clear to me why the response from the server is not issued when the echo is clearly registered in it, and why the browser is waiting for the complete completion of the work shel_exec() because after it, the code does not have one. What should be done, that is, as if according to the logic of things and by the code, the server should give an answer in the form of a JSON string, the browser should show that the request has completed with the status of 200, and the server, in the meantime, should start the conversion, but for some reason this does not happen...