Recherche avancée

Sur d’autres sites (7575)

• dnn_backend_native_layer_mathunary : add atanh support

  29 juin 2020, par Ting Fu

  dnn_backend_native_layer_mathunary : add atanh support
  It can be tested with the model generated with below python script :
  import tensorflow as tf
  
  import numpy as np
  
  import imageio
  in_img = imageio.imread('input.jpeg')
  
  in_img = in_img.astype(np.float32)/255.0
  
  in_data = in_img[np.newaxis, :]
  x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
  please uncomment the part you want to test
  x_sinh_1 = tf.sinh(x)
  
  x_out = tf.divide(x_sinh_1, 1.176) # sinh(1.0)
  x_cosh_1 = tf.cosh(x)
  
  x_out = tf.divide(x_cosh_1, 1.55) # cosh(1.0)
  x_tanh_1 = tf.tanh(x)
  
  x__out = tf.divide(x_tanh_1, 0.77) # tanh(1.0)
  x_asinh_1 = tf.asinh(x)
  
  x_out = tf.divide(x_asinh_1, 0.89) # asinh(1.0/1.1)
  x_acosh_1 = tf.add(x, 1.1)
  
  x_acosh_2 = tf.acosh(x_acosh_1) # accept (1, inf)
  
  x_out = tf.divide(x_acosh_2, 1.4) # acosh(2.1)
  x_atanh_1 = tf.divide(x, 1.1)
  
  x_atanh_2 = tf.atanh(x_atanh_1) # accept (-1, 1)
  
  x_out = tf.divide(x_atanh_2, 1.55) # atanhh(1.0/1.1)
  y = tf.identity(x_out, name='dnn_out') #please only preserve the x_out you want to test
  sess=tf.Session()
  
  sess.run(tf.global_variables_initializer())
  graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
  
  tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)
  print("image_process.pb generated, please use \
  
  path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")
  output = sess.run(y, feed_dict=x : in_data)
  
  imageio.imsave("out.jpg", np.squeeze(output))
  Signed-off-by : Ting Fu <ting.fu@intel.com>
  

  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathunary.c
  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathunary.h
  • [DH] tools/python/convert_from_tensorflow.py
  • [DH] tools/python/convert_header.py

• dnn_backend_native_layer_mathunary : add floor support

  6 août 2020, par Mingyu Yin

  dnn_backend_native_layer_mathunary : add floor support
  It can be tested with the model generated with below python script :
  import tensorflow as tf
  
  import os
  
  import numpy as np
  
  import imageio
  
  from tensorflow.python.framework import graph_util
  
  name = 'floor'
  pb_file_path = os.getcwd()
  
  if not os.path.exists(pb_file_path+'/{}_savemodel/'.format(name)) :
  
      os.mkdir(pb_file_path+'/{}_savemodel/'.format(name))
  with tf.Session(graph=tf.Graph()) as sess :
  
      in_img = imageio.imread('detection.jpg')
  
      in_img = in_img.astype(np.float32)
  
      in_data = in_img[np.newaxis, :]
  
      input_x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
  
      y_ = tf.math.floor(input_x*255)/255
  
      y = tf.identity(y_, name='dnn_out')
  
      sess.run(tf.global_variables_initializer())
  
      constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
      with tf.gfile.FastGFile(pb_file_path+'/{}_savemodel/model.pb'.format(name), mode='wb') as f :
  
          f.write(constant_graph.SerializeToString())
      print("model.pb generated, please in ffmpeg path use\n \n \
  
      python tools/python/convert.py {}_savemodel/model.pb —outdir={}_savemodel/ \n \nto generate model.model\n".format(name,name))
      output = sess.run(y, feed_dict= input_x : in_data)
  
      imageio.imsave("out.jpg", np.squeeze(output))
      print("To verify, please ffmpeg path use\n \n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow -f framemd5 {}_savemodel/tensorflow_out.md5\n  \
  
      or\n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow {}_savemodel/out_tensorflow.jpg\n \nto generate output result of tensorflow model\n".format(name, name, name, name))
      print("To verify, please ffmpeg path use\n \n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native -f framemd5 {}_savemodel/native_out.md5\n  \
  
      or \n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native {}_savemodel/out_native.jpg\n \nto generate output result of native model\n".format(name, name, name, name))
  Signed-off-by : Mingyu Yin <mingyu.yin@intel.com>
  

  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathunary.c
  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathunary.h
  • [DH] tests/dnn/dnn-layer-mathunary-test.c
  • [DH] tools/python/convert_from_tensorflow.py
  • [DH] tools/python/convert_header.py

• Desktop recording with ffmpeg won't play in anything but VLC

  12 mai 2022, par Alex

  I'm creating a desktop application for fun/education. Its primary function is to record my PCs screen and have a pretty UI to play this back. I am recording with ffmpeg like so :

  


  ffmpeg -thread_queue_size 1024 -f gdigrab -framerate 50 -video_size 1920x1080 -i desktop -f dshow -i audio="virtual-audio-capturer" -r 50 -preset fast -c:v h264_nvenc -qp 23 "D:/vid.mp4"


  


  At first glance this seems fine, a file is created and playing it back in VLC media player also works fine.

  


  The problem is that I can't play this back in anything else. Windows apps (videos + media player) just freeze up/provide no clue as to the problem. My end goal is to embed this into a HTML5 player to fit with the electron/react framework I'm writing my application in. This also doesn't play in a similar fashion.

  


  I ran mediainfo on the file in WSL and got the following :

  


  General
Complete name                            : vid.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 544 MiB
Duration                                 : 1 min 15 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 60.4 Mb/s
Writing application                      : Lavf59.10.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:4:4 Predictive@L4.2
Format settings                          : 1 Ref Frames
Format settings, CABAC                   : No
Format settings, Reference frames        : 1 frame
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 15 s
Bit rate mode                            : Variable
Bit rate                                 : 60.3 Mb/s
Maximum bit rate                         : 2 000 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 50.000 FPS
Color space                              : RGB
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.581
Stream size                              : 543 MiB (100%)
Color range                              : Full
Matrix coefficients                      : Identity
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 min 15 s
Source duration                          : 1 min 15 s
Bit rate mode                            : Constant
Bit rate                                 : 132 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz


  


  I also saw this thread (ffmpeg created timelapse won’t play on other players than VLC) suggesting the bit rate may be the problem but having tried the suggestions there had no luck.

  


  At a glance I suspected the "Codec ID" field as with comparison to other MP4 files that windows does play it is slightly different, i.e. :

  


  isom (isom/iso2/avc1/mp41) vs mp42 (mp42/mp41/isom/avc1)

  


  However fiddling with the ffmpeg encoder settings I managed to vary this and still could not play the video back.

  


  Also including the ffprobe output :

  


  ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'vid.mp4:
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.10.100
  Duration: 00:01:15.50, start: 0.000000, bitrate: 60414 kb/s
    Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), gbrp(pc, gbr/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 60277 kb/s, 50 fps, 50 tbr, 12800 tbn, 100 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 132 kb/s (default)
    Metadata:
      handler_name    : SoundHandler