Recherche avancée

Sur d’autres sites (4972)

• dnn_backend_native_layer_mathunary : add floor support

  6 août 2020, par Mingyu Yin

  dnn_backend_native_layer_mathunary : add floor support
  It can be tested with the model generated with below python script :
  import tensorflow as tf
  
  import os
  
  import numpy as np
  
  import imageio
  
  from tensorflow.python.framework import graph_util
  
  name = 'floor'
  pb_file_path = os.getcwd()
  
  if not os.path.exists(pb_file_path+'/{}_savemodel/'.format(name)) :
  
      os.mkdir(pb_file_path+'/{}_savemodel/'.format(name))
  with tf.Session(graph=tf.Graph()) as sess :
  
      in_img = imageio.imread('detection.jpg')
  
      in_img = in_img.astype(np.float32)
  
      in_data = in_img[np.newaxis, :]
  
      input_x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
  
      y_ = tf.math.floor(input_x*255)/255
  
      y = tf.identity(y_, name='dnn_out')
  
      sess.run(tf.global_variables_initializer())
  
      constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
      with tf.gfile.FastGFile(pb_file_path+'/{}_savemodel/model.pb'.format(name), mode='wb') as f :
  
          f.write(constant_graph.SerializeToString())
      print("model.pb generated, please in ffmpeg path use\n \n \
  
      python tools/python/convert.py {}_savemodel/model.pb —outdir={}_savemodel/ \n \nto generate model.model\n".format(name,name))
      output = sess.run(y, feed_dict= input_x : in_data)
  
      imageio.imsave("out.jpg", np.squeeze(output))
      print("To verify, please ffmpeg path use\n \n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow -f framemd5 {}_savemodel/tensorflow_out.md5\n  \
  
      or\n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow {}_savemodel/out_tensorflow.jpg\n \nto generate output result of tensorflow model\n".format(name, name, name, name))
      print("To verify, please ffmpeg path use\n \n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native -f framemd5 {}_savemodel/native_out.md5\n  \
  
      or \n \
  
      ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native {}_savemodel/out_native.jpg\n \nto generate output result of native model\n".format(name, name, name, name))
  Signed-off-by : Mingyu Yin <mingyu.yin@intel.com>
  

  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathunary.c
  • [DH] libavfilter/dnn/dnn_backend_native_layer_mathunary.h
  • [DH] tests/dnn/dnn-layer-mathunary-test.c
  • [DH] tools/python/convert_from_tensorflow.py
  • [DH] tools/python/convert_header.py

• Desktop recording with ffmpeg won't play in anything but VLC

  12 mai 2022, par Alex

  I'm creating a desktop application for fun/education. Its primary function is to record my PCs screen and have a pretty UI to play this back. I am recording with ffmpeg like so :

  


  ffmpeg -thread_queue_size 1024 -f gdigrab -framerate 50 -video_size 1920x1080 -i desktop -f dshow -i audio="virtual-audio-capturer" -r 50 -preset fast -c:v h264_nvenc -qp 23 "D:/vid.mp4"


  


  At first glance this seems fine, a file is created and playing it back in VLC media player also works fine.

  


  The problem is that I can't play this back in anything else. Windows apps (videos + media player) just freeze up/provide no clue as to the problem. My end goal is to embed this into a HTML5 player to fit with the electron/react framework I'm writing my application in. This also doesn't play in a similar fashion.

  


  I ran mediainfo on the file in WSL and got the following :

  


  General
Complete name                            : vid.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 544 MiB
Duration                                 : 1 min 15 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 60.4 Mb/s
Writing application                      : Lavf59.10.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:4:4 Predictive@L4.2
Format settings                          : 1 Ref Frames
Format settings, CABAC                   : No
Format settings, Reference frames        : 1 frame
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 15 s
Bit rate mode                            : Variable
Bit rate                                 : 60.3 Mb/s
Maximum bit rate                         : 2 000 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 50.000 FPS
Color space                              : RGB
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.581
Stream size                              : 543 MiB (100%)
Color range                              : Full
Matrix coefficients                      : Identity
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 min 15 s
Source duration                          : 1 min 15 s
Bit rate mode                            : Constant
Bit rate                                 : 132 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz


  


  I also saw this thread (ffmpeg created timelapse won’t play on other players than VLC) suggesting the bit rate may be the problem but having tried the suggestions there had no luck.

  


  At a glance I suspected the "Codec ID" field as with comparison to other MP4 files that windows does play it is slightly different, i.e. :

  


  isom (isom/iso2/avc1/mp41) vs mp42 (mp42/mp41/isom/avc1)

  


  However fiddling with the ffmpeg encoder settings I managed to vary this and still could not play the video back.

  


  Also including the ffprobe output :

  


  ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'vid.mp4:
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.10.100
  Duration: 00:01:15.50, start: 0.000000, bitrate: 60414 kb/s
    Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), gbrp(pc, gbr/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 60277 kb/s, 50 fps, 50 tbr, 12800 tbn, 100 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 132 kb/s (default)
    Metadata:
      handler_name    : SoundHandler


  


• How to correctly encrypt the I-frames in video files ?

  25 mai 2023, par Bob

  I'm trying to encrypt only the I-frames of a video file so that the energy overhead is lessened compared to if I tried to encrypt the whole file. However, after encrypting what I thought were the I-frames, the video seems to play normally (except for the encrypted frames). I thought that the point of encrypting the I-frames was so that the other frames would be distorted, but that doesn't seem to be the case. I was wondering what seems to be wrong ? The file format is avi with codec MPEG-4 video (FMP4).

  


  I used ffmpeg to identity the I frames of a certain video (named "video1") : "ffprobe -select_streams v -show_frames -of csv video1.avi > video_frame_info.csv". I got this table which seems to indicate that every 12th frame is an I-frame. I then used this python program to encrypt every 12th frame with chacha20 from the pycryptodome library :

  


  from Crypto.Cipher import ChaCha20
from Crypto.Random import get_random_bytes
import cv2
import numpy as np

def read_frames(video_path):
    # Load the video file
    video = cv2.VideoCapture(video_path)

    # Initialize a list to hold frames
    frames = []

    # Loop until there are frames left in the video file
    while video.isOpened():
        ret, frame = video.read()
        if not ret:
            break

        frames.append(frame)

    video.release()

    return frames

def encrypt_iframes(frames, key):
    # Initialize the cipher
    cipher = ChaCha20.new(key=key)

    # Encrypt every 12th frame (considering the first frame as an I-frame)
    for i in range(0, len(frames), 12):
        frame = frames[i]
        # Flatten and bytes-encode the frame
        flat_frame = frame.flatten()
        bytes_frame = flat_frame.tobytes()

        # Encrypt the frame
        encrypted_frame = cipher.encrypt(bytes_frame)

        # Replace the original frame with the encrypted one
        frames[i] = np.frombuffer(encrypted_frame, dtype=flat_frame.dtype).reshape(frame.shape)

    return frames

def write_frames(frames, output_path):
    # Assume all frames have the same shape
    height, width, _ = frames[0].shape

    # Create a VideoWriter object
    out = cv2.VideoWriter(output_path, cv2.VideoWriter_fourcc(*'mpv4'), 30, (width, height))

    for frame in frames:
        out.write(frame)

    out.release()

def main():
    # Generate a random key
    key = get_random_bytes(32)

    # Read frames from video
    frames = read_frames('video1.avi')

    # Encrypt I-frames
    encrypted_frames = encrypt_iframes(frames, key)

    # Write encrypted frames to a new video file
    write_frames(encrypted_frames, 'reprise.avi')

if __name__ == "__main__":
    main()



  


  When I play the video, every 12th frame there is just noise on the screen, which makes sense since it was encrypted. However, all the other frames seem to be completely normal, even thought they are P-frames and should depend on the I-frames. What am I doing wrong, and how can I make it so that it works as intended ?
Thanks in advance