Recherche avancée

Sur d’autres sites (8456)

• dnn/native : add native support for 'add'

  10 avril 2020, par Guo, Yejun

  dnn/native : add native support for 'add'
  It can be tested with the model file generated with below python script :
  import tensorflow as tf
  
  import numpy as np
  
  import imageio
  in_img = imageio.imread('input.jpg')
  
  in_img = in_img.astype(np.float32)/255.0
  
  in_data = in_img[np.newaxis, :]
  x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
  
  z1 = 0.039 + x
  
  z2 = x + 0.042
  
  z3 = z1 + z2
  
  z4 = z3 - 0.381
  
  z5 = z4 - x
  
  y = tf.math.maximum(z5, 0.0, name='dnn_out')
  sess=tf.Session()
  
  sess.run(tf.global_variables_initializer())
  graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
  
  tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)
  print("image_process.pb generated, please use \
  
  path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")
  output = sess.run(y, feed_dict=x : in_data)
  
  imageio.imsave("out.jpg", np.squeeze(output))
  Signed-off-by : Guo, Yejun <yejun.guo@intel.com>
  

  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathbinary.c
  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathbinary.h
  • [DH] tools/python/convert_from_tensorflow.py
  • [DH] tools/python/convert_header.py

• Improving Google Cloud Speech-to-Text accuracy

  6 juillet 2020, par lr_optim

  I'm working on a project where I need to perform these steps :

  


  

  1. Record a voice call (.webm -file)

  


  2. Split the webm -file into chunks with ffmpeg and convert the file into wav

  


  3. Transcribe the chunks using SpeechRecognition -library and Google Cloud API

  


  


  I've faced problems with the transcription accuracy and wondering if there is something I could do to improve it. At the time I'm splitting the original file into 30s chunks. I thought there might be one problem, that I might be missing words because of splitting so I've tried also with longer chunks under 60s but didn't notice any improve in accuracy.
Reading trough the speechRecognition docs I decided to set r.energy_threshold = 4000, I also tried to set the energy_treshold dynamically like this :

  


  with sr.AudioFile(name) as source:
    r.dynamic_energy_threshold = True
    r.adjust_for_ambient_noise(source, duration = 1)
    audio = r.record(source)


  


  I've also tested en-US and en-GB to see if there's some difference but there isn't as much as I'd want. The program is supposed to work with english language spoken by nordic people. If someone has experience about choosing a right language model for people speaking with accent, please let me know.

  


  This is the ffmpeg command is use to split the webm file into chunks : command = ['ffmpeg', '-i', filename, '-f', 'segment', '-segment_time', '30', parts_dir + outputname + '%09d.wav']

  


  Is there somethig I could do better ? I'm wondering if the quality is not good enough an Google is having hard time because of that ?

  


  The main problem is I'm getting bad results (lots of wrong words) from Google and wondering if there is something I could do about it.

  


• How to make ffmpeg remux an iPhone MOV (HEVC) that works on Apple products ?

  21 janvier 2024, par Mikael Finstad

  I want to remux a HEVC file (without encoding). I have tried most options, even removing audio, but I cannot get an output file that plays smoothly with Apple products.

  



  First I AirDrop any MOV from an iPhone recorded with the camera in HEVC.

  



  Then run :

  



  ffmpeg -i IMG_4605.MOV -c copy out.mov


  



  Then if I send out.mov back to the iPhone and try to play it back, it plays back with glitches while seeking, like this : (original to the right)

  



  


  



  When I try to drag the file into a new iMovie project and try to play it back, it freezes the whole iMovie app and needs a force quit.

  



  Doing the same with the original causes no issues, so obviously there is something wrong with the way ffmpeg remuxes it, or something wrong with Apple's software.

  



  ffmpeg -i IMG_4605.MOV 
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'IMG_4605.MOV':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2019-12-29T10:20:56.000000Z
    com.apple.quicktime.location.ISO6709: +01.3602+103.9897+024.438/
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPhone 11
    com.apple.quicktime.software: 13.3
    com.apple.quicktime.creationdate: 2019-12-29T18:20:56+0800
  Duration: 00:00:09.00, start: 0.000000, bitrate: 24882 kb/s
    Stream #0:0(und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 3840x2160, 24659 kb/s, 29.99 fps, 29.97 tbr, 600 tbn, 600 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2019-12-29T10:20:56.000000Z
      handler_name    : Core Media Video
      encoder         : HEVC
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2019-12-29T10:20:56.000000Z
      handler_name    : Core Media Audio
    Stream #0:2(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
    Metadata:
      creation_time   : 2019-12-29T10:20:56.000000Z
      handler_name    : Core Media Metadata
    Stream #0:3(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
    Metadata:
      creation_time   : 2019-12-29T10:20:56.000000Z
      handler_name    : Core Media Metadata
    Stream #0:4(und): Data: none (mebx / 0x7862656D), 17 kb/s (default)
    Metadata:
      creation_time   : 2019-12-29T10:20:56.000000Z
      handler_name    : Core Media Metadata


  



  ffmpeg -i out.mov
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf58.29.100
  Duration: 00:00:09.00, start: 0.000000, bitrate: 24860 kb/s
    Stream #0:0: Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 3840x2160, 24659 kb/s, 29.99 fps, 29.97 tbr, 19200 tbn, 19200 tbc (default)
    Metadata:
      rotate          : 90
      handler_name    : Core Media Video
      encoder         : HEVC
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      handler_name    : Core Media Audio


  



  If I open out.mov in QuickTime and do a trim and export it, then it it still "broken", so I'm thinking there's nothing wrong with the actual muxing into MOV, because QuickTime is passing the error on.

  



  I noticed that out.mov has different parameters 19200 tbn, 19200 tbc (vs original 600 tbn, 600 tbc). However, if I force set these to 600 with -video_track_timescale 600 the output file is still broken.

  



  Anyone got some insights ?